fix: xorriso append_partition for real USB boot + grub-mkstandalone
Root cause of USB boot failure: our xorriso used -e boot/grub/efi.img to embed the EFI image inside the ISO. This works for CD-ROM and QEMU but NOT for USB on real UEFI hardware. Fix: use the Will Haley / Debian live-build approach: - -append_partition 2 (GPT type EFI) appends efi.img AFTER ISO data - -e --interval:appended_partition_2:all:: references the appended partition - --mbr-force-bootable forces MBR active flag - grub-mkstandalone with embedded bootstrap config (searches for grub.cfg) - grub.cfg placed in both /boot/grub/ AND /EFI/BOOT/ on ISO - grub.cfg uses search --label ARCHIPELAGO to find the ISO root This is the exact approach used by StartOS, TAILS, and every production custom Debian live ISO that boots from USB. Also: iso-debug, iso-branding skills + reference docs, dev-start.sh option 0 for branding dev, improved dev-branding.sh and test-iso-qemu.sh. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dorian
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
013b724e02
commit
34a476d0a1
@@ -0,0 +1,367 @@
|
||||
# Custom Debian ISO Boot Chain — Technical Reference
|
||||
|
||||
Expert reference for building and debugging custom bootable Debian-based ISOs.
|
||||
Covers hybrid MBR/GPT, live-boot, debootstrap, GRUB, ISOLINUX, Plymouth, and xorriso.
|
||||
|
||||
---
|
||||
|
||||
## 1. Hybrid MBR/GPT for USB Boot
|
||||
|
||||
### What is isohdpfx.bin?
|
||||
The first 432 bytes of a hybrid-bootable ISO. Contains the Master Boot Record code that BIOS firmware executes when booting from USB. Different sources produce different MBR code:
|
||||
|
||||
| Source | First bytes | Compatibility |
|
||||
|--------|-------------|---------------|
|
||||
| Debian Live ISO (`dd if=debian-live.iso bs=1 count=432`) | `45 52` | Best — works on all tested hardware |
|
||||
| `/usr/lib/ISOLINUX/isohdpfx.bin` | `33 ed` | Generic — fails on some UEFI hardware |
|
||||
| Manually built with `isohybrid` | Varies | Unpredictable |
|
||||
|
||||
**Rule**: Always extract MBR from a known-working ISO. Never rely on the generic ISOLINUX one.
|
||||
|
||||
### xorriso flags for hybrid boot
|
||||
```bash
|
||||
xorriso -as mkisofs -o output.iso \
|
||||
-isohybrid-mbr isohdpfx.bin \ # Embeds MBR for BIOS USB boot
|
||||
-c isolinux/boot.cat \ # El Torito boot catalog
|
||||
-b isolinux/isolinux.bin \ # BIOS bootloader
|
||||
-no-emul-boot -boot-load-size 4 -boot-info-table \
|
||||
-eltorito-alt-boot \ # Second boot entry (EFI)
|
||||
-e boot/grub/efi.img \ # EFI boot image
|
||||
-no-emul-boot \
|
||||
-isohybrid-gpt-basdat \ # Adds GPT partition for EFI
|
||||
-partition_offset 16 \ # Space for GPT table — REQUIRED for UEFI
|
||||
/path/to/iso/contents
|
||||
```
|
||||
|
||||
**Critical flags**:
|
||||
- `-isohybrid-gpt-basdat`: Without this, UEFI firmware won't see the EFI partition
|
||||
- `-partition_offset 16`: Reserves 16 sectors for GPT. Without it, some UEFI firmware ignores the USB entirely
|
||||
- `-isohybrid-mbr`: Without this, the ISO won't boot from USB at all (only CD-ROM)
|
||||
|
||||
### Balena Etcher
|
||||
Writes the ISO byte-for-byte to USB — no reformatting, no special partition creation. If the ISO works with `dd`, it works with Etcher. If BIOS doesn't see the USB, the MBR code is wrong, not Etcher.
|
||||
|
||||
### Verifying hybrid structure
|
||||
```bash
|
||||
xxd -l 4 image.iso # MBR code (should be 45 52 for Debian Live)
|
||||
xxd -s 510 -l 2 image.iso # Boot signature (must be 55 aa)
|
||||
xxd -s 512 -l 8 image.iso # GPT signature at LBA 1 (should be "EFI PART")
|
||||
file image.iso # Should say "DOS/MBR boot sector" and "bootable"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. live-boot Package
|
||||
|
||||
### What it does
|
||||
Provides initramfs hooks that mount a squashfs file as the root filesystem using overlayfs. This is how every Debian/Ubuntu live ISO works.
|
||||
|
||||
Boot flow: kernel → initramfs → live-boot scripts → find squashfs → mount overlayfs → pivot_root → systemd
|
||||
|
||||
### Package structure
|
||||
- `live-boot` (~29KB): Main package, boot scripts
|
||||
- `live-boot-initramfs-tools` (~6KB): Initramfs hooks that get baked into initrd.img
|
||||
|
||||
**Critical**: `scripts/live` is a **FILE**, not a directory. Verification must use `[ -e ]` not `[ -d ]`.
|
||||
|
||||
### Kernel parameters
|
||||
| Parameter | Required | Effect |
|
||||
|-----------|----------|--------|
|
||||
| `boot=live` | YES | Activates live-boot's initramfs hooks |
|
||||
| `components` | YES | Scans live/ for additional squashfs modules |
|
||||
| `toram` | No | Copies squashfs to RAM (faster, allows USB removal) |
|
||||
| `persistence` | No | Enables writable overlay on a partition labeled "persistence" |
|
||||
| `quiet` | No | Suppresses boot messages |
|
||||
| `splash` | No | Enables Plymouth splash screen |
|
||||
| `console=ttyS0,115200` | No | Serial console for QEMU debugging |
|
||||
|
||||
### Where live-boot mounts things
|
||||
- `/run/live/medium` — The boot media (USB/CDROM) mount point
|
||||
- `/run/live/rootfs/filesystem.squashfs` — The mounted squashfs
|
||||
- `/run/live/overlay` — The tmpfs overlay for writes
|
||||
|
||||
### Verifying live-boot in initramfs
|
||||
```bash
|
||||
TMPDIR=$(mktemp -d)
|
||||
unmkinitramfs /path/to/initrd.img $TMPDIR
|
||||
# Check for live-boot scripts
|
||||
file $TMPDIR/scripts/live # Should be "ASCII text"
|
||||
# OR (some initramfs have main/ prefix)
|
||||
file $TMPDIR/main/scripts/live
|
||||
```
|
||||
|
||||
### Common failures
|
||||
1. **live-boot not in initrd**: Installed in rootfs but initramfs not regenerated after
|
||||
2. **Missing kernel params**: `boot=live` not in GRUB/ISOLINUX config
|
||||
3. **Broken initramfs**: Built without /proc /sys /dev mounted in chroot
|
||||
4. **Wrong verification**: `[ -d scripts/live ]` fails because it's a file
|
||||
|
||||
---
|
||||
|
||||
## 3. debootstrap for Installer Environments
|
||||
|
||||
### Variants
|
||||
- `--variant=minbase`: Absolute minimum (~150MB). Only essential + apt. Good for installer squashfs.
|
||||
- Default (no variant): Full base system (~300MB). More packages, fewer missing deps.
|
||||
|
||||
### --include limitations
|
||||
debootstrap's minbase resolver is simplified and **cannot resolve complex dependency chains**. Packages like `live-boot` that depend on `initramfs-tools` which depends on many other packages will silently fail or be skipped.
|
||||
|
||||
**Fix**: Install complex packages via `chroot apt-get` after debootstrap completes:
|
||||
```bash
|
||||
debootstrap --variant=minbase --include=basic,packages bookworm /installer http://deb.debian.org/debian
|
||||
# Then:
|
||||
mount --bind /proc /installer/proc
|
||||
mount --bind /sys /installer/sys
|
||||
mount --bind /dev /installer/dev
|
||||
chroot /installer apt-get update
|
||||
chroot /installer apt-get install -y live-boot live-boot-initramfs-tools
|
||||
umount /installer/dev /installer/sys /installer/proc
|
||||
```
|
||||
|
||||
### Initramfs generation inside containers
|
||||
`update-initramfs` REQUIRES `/proc`, `/sys`, `/dev` to be mounted in the chroot. Without them:
|
||||
- Module detection fails (can't read /proc/modules)
|
||||
- Device nodes missing (can't detect hardware)
|
||||
- The resulting initramfs boots but can't load kernel modules
|
||||
|
||||
### Container-in-container considerations
|
||||
When running debootstrap inside a Podman/Docker container on a CI runner:
|
||||
- `--privileged` flag needed for chroot to work
|
||||
- The container runtime may kill the container after debootstrap exits if using `set -e`
|
||||
- proc/sys/dev mounts inside the debootstrapped chroot work fine with `--privileged`
|
||||
|
||||
---
|
||||
|
||||
## 4. GRUB Theming
|
||||
|
||||
### theme.txt format
|
||||
```
|
||||
desktop-color: "#0a0a0a" # Fallback background color
|
||||
desktop-image: "background.png" # Background image (any PNG, GRUB scales)
|
||||
title-text: "" # Empty = hide title
|
||||
|
||||
+ boot_menu {
|
||||
left = 25%
|
||||
top = 40%
|
||||
width = 50%
|
||||
height = 30%
|
||||
item_color = "#aaaaaa" # Normal menu item color
|
||||
selected_item_color = "#fb923c" # Selected item color
|
||||
item_height = 36
|
||||
item_spacing = 8
|
||||
scrollbar = false
|
||||
}
|
||||
|
||||
+ label {
|
||||
left = 25%
|
||||
top = 20%
|
||||
width = 50%
|
||||
text = "Some Text"
|
||||
color = "#f7931a"
|
||||
align = "center"
|
||||
}
|
||||
```
|
||||
|
||||
**IMPORTANT**: Do NOT specify `font = "Name Size"` in theme elements unless you know the exact internal font name. If GRUB can't find the font, the ENTIRE theme fails to load and you get the ugly default.
|
||||
|
||||
### Font handling
|
||||
```bash
|
||||
# Generate .pf2 font file
|
||||
grub-mkfont -s 16 -o dejavu_16.pf2 /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf
|
||||
|
||||
# In grub.cfg, load fonts BEFORE setting theme:
|
||||
loadfont /boot/grub/font.pf2
|
||||
loadfont /boot/grub/themes/archipelago/dejavu_16.pf2
|
||||
set theme=/boot/grub/themes/archipelago/theme.txt
|
||||
```
|
||||
|
||||
### Background images
|
||||
- Any PNG works, GRUB scales to screen resolution
|
||||
- Smaller images (1024x768) load faster
|
||||
- Large images (3000x2000+) add seconds to boot and may fail on limited GRUB heap
|
||||
|
||||
### grub-mkimage — essential modules for ISO boot
|
||||
```bash
|
||||
grub-mkimage -O x86_64-efi -o BOOTX64.EFI -p /boot/grub \
|
||||
part_gpt part_msdos fat iso9660 udf \ # Filesystem access
|
||||
normal boot linux search search_fs_uuid search_fs_file search_label \
|
||||
configfile echo cat ls test true \ # Basic commands
|
||||
loopback \ # Loop device support
|
||||
gfxterm gfxmenu font png \ # Graphical display
|
||||
all_video video video_bochs video_cirrus \ # Video drivers
|
||||
efi_gop efi_uga # EFI display protocols
|
||||
```
|
||||
|
||||
Missing `all_video`/`efi_gop` = black screen on real hardware (works in QEMU).
|
||||
|
||||
### EFI boot image creation
|
||||
```bash
|
||||
dd if=/dev/zero of=efi.img bs=1M count=4
|
||||
mkfs.vfat efi.img
|
||||
mmd -i efi.img ::/EFI ::/EFI/BOOT
|
||||
mcopy -i efi.img BOOTX64.EFI ::/EFI/BOOT/BOOTX64.EFI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Plymouth Boot Splash
|
||||
|
||||
### Theme types
|
||||
- **script**: Most flexible. Lua-like scripting with sprites, animations, callbacks.
|
||||
- **two-step**: Simple logo + spinner. Less customizable but easier.
|
||||
- **fade-in**: Logo fades in. Minimal.
|
||||
|
||||
### Script theme structure
|
||||
```
|
||||
/usr/share/plymouth/themes/mytheme/
|
||||
mytheme.plymouth # Theme metadata
|
||||
mytheme.script # Animation script
|
||||
logo.png # Logo image (PNG with alpha)
|
||||
```
|
||||
|
||||
### mytheme.plymouth
|
||||
```ini
|
||||
[Plymouth Theme]
|
||||
Name=MyTheme
|
||||
Description=Custom boot splash
|
||||
ModuleName=script
|
||||
|
||||
[script]
|
||||
ImageDir=/usr/share/plymouth/themes/mytheme
|
||||
ScriptFile=/usr/share/plymouth/themes/mytheme/mytheme.script
|
||||
```
|
||||
|
||||
### Script language key functions
|
||||
```javascript
|
||||
Window.SetBackgroundTopColor(r, g, b); // 0.0-1.0 floats
|
||||
Window.SetBackgroundBottomColor(r, g, b);
|
||||
image = Image("logo.png");
|
||||
sprite = Sprite(image);
|
||||
sprite.SetX(x); sprite.SetY(y); sprite.SetOpacity(0.0-1.0);
|
||||
Plymouth.SetRefreshFunction(fn); // Called every frame
|
||||
Plymouth.SetBootProgressFunction(fn); // fn(duration, progress)
|
||||
Plymouth.SetDisplayPasswordFunction(fn); // fn(prompt, bullets)
|
||||
Plymouth.SetQuitFunction(fn);
|
||||
screen_w = Window.GetWidth();
|
||||
screen_h = Window.GetHeight();
|
||||
```
|
||||
|
||||
### Setting default theme
|
||||
```bash
|
||||
plymouth-set-default-theme mytheme
|
||||
# OR manually:
|
||||
ln -sf /usr/share/plymouth/themes/mytheme/mytheme.plymouth /etc/alternatives/default.plymouth
|
||||
```
|
||||
|
||||
### Kernel params
|
||||
- `splash` in GRUB_CMDLINE_LINUX_DEFAULT enables Plymouth
|
||||
- `quiet` suppresses text that would overlay Plymouth
|
||||
|
||||
---
|
||||
|
||||
## 6. ISOLINUX/SYSLINUX
|
||||
|
||||
### Required files
|
||||
| File | Source | Purpose |
|
||||
|------|--------|---------|
|
||||
| `isolinux.bin` | `/usr/lib/ISOLINUX/isolinux.bin` | BIOS bootloader |
|
||||
| `ldlinux.c32` | `/usr/lib/syslinux/modules/bios/ldlinux.c32` | Core library (REQUIRED) |
|
||||
| `menu.c32` | `/usr/lib/syslinux/modules/bios/menu.c32` | Text menu UI |
|
||||
| `libutil.c32` | `/usr/lib/syslinux/modules/bios/libutil.c32` | Utility library |
|
||||
| `boot.cat` | Auto-generated by xorriso | El Torito boot catalog |
|
||||
| `isohdpfx.bin` | Extracted from working ISO | Hybrid MBR code |
|
||||
|
||||
### Configuration (isolinux.cfg)
|
||||
```
|
||||
UI menu.c32
|
||||
PROMPT 0
|
||||
TIMEOUT 50 # 5 seconds (units of 1/10 second)
|
||||
DEFAULT install
|
||||
|
||||
MENU TITLE MY INSTALLER
|
||||
MENU COLOR border 30;44 #40ffffff #00000000 std
|
||||
MENU COLOR title 1;36;44 #ff00b7ff #00000000 std
|
||||
MENU COLOR sel 7;37;40 #ffffffff #ff333333 std
|
||||
MENU COLOR unsel 37;44 #ffaaaaaa #00000000 std
|
||||
|
||||
LABEL install
|
||||
MENU LABEL Install System
|
||||
KERNEL /live/vmlinuz
|
||||
APPEND initrd=/live/initrd.img boot=live components quiet
|
||||
MENU DEFAULT
|
||||
```
|
||||
|
||||
### menu.c32 vs vesamenu.c32
|
||||
- `menu.c32`: Text-mode menu. More compatible, no background image.
|
||||
- `vesamenu.c32`: VESA graphical menu. Supports background PNG, but some hardware/VMs don't support VESA.
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing Without Real Hardware
|
||||
|
||||
### QEMU UEFI boot
|
||||
```bash
|
||||
qemu-system-x86_64 \
|
||||
-machine q35 \
|
||||
-drive if=pflash,format=raw,readonly=on,file=/path/to/OVMF_CODE.fd \
|
||||
-m 4G -smp 2 \
|
||||
-boot d -cdrom image.iso \
|
||||
-drive if=virtio,format=qcow2,file=test-disk.qcow2 \
|
||||
-vga virtio -display default
|
||||
```
|
||||
|
||||
### QEMU BIOS boot (sees ISOLINUX)
|
||||
```bash
|
||||
qemu-system-x86_64 \
|
||||
-machine pc \
|
||||
-m 4G -smp 2 \
|
||||
-boot d -cdrom image.iso \
|
||||
-drive if=virtio,format=qcow2,file=test-disk.qcow2 \
|
||||
-vga virtio -display default
|
||||
```
|
||||
|
||||
### Serial console capture
|
||||
Add to QEMU: `-serial file:/tmp/serial.log`
|
||||
Add to kernel params: `console=ttyS0,115200 console=tty0`
|
||||
|
||||
### ISO structure verification (no boot required)
|
||||
```bash
|
||||
MNT=$(mktemp -d)
|
||||
sudo mount -o loop,ro image.iso $MNT
|
||||
|
||||
# Check all critical files
|
||||
for f in live/vmlinuz live/initrd.img live/filesystem.squashfs \
|
||||
isolinux/isolinux.bin EFI/BOOT/BOOTX64.EFI boot/grub/grub.cfg; do
|
||||
[ -e $MNT/$f ] && echo "OK: $f" || echo "MISSING: $f"
|
||||
done
|
||||
|
||||
# Check initramfs for live-boot
|
||||
INITRD=$(mktemp -d)
|
||||
unmkinitramfs $MNT/live/initrd.img $INITRD
|
||||
[ -e $INITRD/scripts/live ] && echo "live-boot: OK" || echo "live-boot: MISSING"
|
||||
|
||||
# Check kernel params
|
||||
grep "boot=live" $MNT/boot/grub/grub.cfg && echo "params: OK"
|
||||
|
||||
sudo umount $MNT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Security Considerations for Custom ISOs
|
||||
|
||||
### Supply chain
|
||||
- Pin the Debian mirror URL (don't use redirectors in production)
|
||||
- Verify package signatures (debootstrap does this by default)
|
||||
- Pin kernel and GRUB package versions for reproducibility
|
||||
|
||||
### Installer security
|
||||
- Auto-install.sh runs as root — validate all inputs before path construction
|
||||
- LUKS key generation must use CSPRNG (`/dev/urandom`, never `/dev/random` which blocks)
|
||||
- Drop the LUKS key file after writing to crypttab (or store in root-only location with 0400)
|
||||
|
||||
### Boot security
|
||||
- Secure Boot requires signed GRUB EFI binary (shim-signed package)
|
||||
- Without Secure Boot, the unsigned BOOTX64.EFI works but users must disable Secure Boot in BIOS
|
||||
- The MBR code (isohdpfx.bin) is not signed — Secure Boot only validates EFI path
|
||||
Reference in New Issue
Block a user