Ran Task 2's blocking checkpoint on a real node. The rotation works and was
proven end to end: it fired ~15s after restart, wrote a fresh unique
credential (0600, service-owned), logged exactly one line naming the .pw path
with no value in it, left every other secret and the gateway's data untouched,
and kept the container's name and ports.
But the assumption the plan rests on is WRONG, and the checkpoint is what
caught it. 25 minutes after rotating, /proc/<pid>/environ showed the running
gatewayd still using the PRE-ROTATION credential while the file and podman
secret held the new one. The orchestrator explains itself in its own logs:
Quadlet unit drift-synced — file rewritten, .service NOT restarted
(operator restart picks up new config)
container drift detected during boot reconcile;
leaving running restart-sensitive app untouched
Two deliberate guards: the Quadlet path never restarts a unit it rewrites, and
fedimint-gateway is classified restart-sensitive so drift is detected on every
tick and then ignored — logged at 15:51, 15:53, 15:54, 15:56 and counting.
So on a real affected node the credential file becomes unique while the
gateway keeps answering to the compromised one until an unrelated reboot, and
the operator reading the .pw gets a password the gateway rejects — T-01-77
inverted. FED-07 is NOT closed and this plan alone cannot close it.
Not hand-rolled around, per the plan's own instruction. The fix needs a design
decision: whether a compromised credential is the case that should override
restart-sensitivity, or whether rotation must raise an operator-facing
"restart required" alert instead of logging into the void.
Incidentally disproved: restarting archipelago does NOT kill containers here
(29/29 then 31/31 survived; "Adopted 31 existing container(s)"). The service
is system.slice/KillMode=control-group while containers live in
user-1000.slice/…/libpod-*. The CLAUDE.md SIGKILL rule predates Quadlet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
