The first-boot per-device secret regeneration was fail-open: both branches
logged a warning and continued, and `touch "$MARKER"` ran unconditionally
outside both `if` blocks. Combined with the unit's ConditionPathExists=! and
the script's own marker fast-path, one transient failure left that node on the
image-wide shared SSH host key and TLS private key permanently and silently —
and the ISO is a published artefact, so every downloader holds those keys.
- Retry each generator 3 times with backoff (D-05), so a transient first-boot
condition recovers inside the same boot instead of being terminal.
- Write the completion marker ONLY when both TLS and SSH succeeded, so a
failed boot leaves the unit eligible to run again on the next boot.
- On terminal failure: durable record at
/var/lib/archipelago/first-boot-secrets.failed naming which generator
failed, plus console + logger + stderr, and exit 1 so the unit lands in
`failed` rather than `active`. The record is cleared on a later success.
- Add FIRST_BOOT_SECRETS_ROOT / FIRST_BOOT_SECRETS_BACKOFF seams. Unset in
production the behaviour is byte-identical; set, they let the fail-closed
property be asserted rather than claimed.
- Order the unit After=systemd-random-seed.service (no-op today, correct if a
seed file is ever baked).
- State the operational trade in the script header: after the rootfs strip, a
terminal failure means no SSH and no TLS and needs the physical console.
That was chosen deliberately over running on fleet-shared keys.
tests/first-boot-secrets/run-tests.sh extracts the shipped heredoc body from
the builder and drives it against a temp root with stubbed generators: both
succeed, openssl fails every attempt, ssh-keygen fails twice then succeeds.
Moving the marker touch back outside the success branch makes case 2 fail with
MARKER-SET-ON-FAILURE, which is the regression this pins.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
