From 7c214e6497eb7eee8b99d4dab92f2e9b85636bac Mon Sep 17 00:00:00 2001 From: archipelago Date: Sat, 8 Aug 2026 04:03:45 -0400 Subject: [PATCH] docs(bulletproof-containers): say which parts of the plan actually shipped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Header claimed the whole 2026-04-22 plan "has been implemented". The architecture was adopted, but checking each item against the tree: - The `core/archipelago/src/reconcile/` module the doc lays out in detail — desired.rs / current.rs / diff.rs / apply.rs / derived.rs / backoff.rs — was never created. The reconciler shipped as container/boot_reconciler.rs + container/prod_orchestrator.rs instead. - FM2's named fix `reconcile::derived::render_bitcoin_conf` does not exist. The drift was eliminated a different way: bitcoind runs with an explicit `-conf` derived from secrets each start, and stale datadir configs are removed. - FM1/FM3 are partial — companion UIs are Quadlet units, main app containers are not, since `use_quadlet_backends` still defaults false. The "v1.7.48+ full reconcile module / main containers become Quadlet units" step has not happened. - **FM6 was never implemented.** There is no podman corrupt-state probe and no `system renumber` recovery anywhere in the tree. The 2026-04 failure that made a registry node unreachable would still require manual SSH today — which is precisely the "zero-manual-intervention" target this doc opens with. FM4 and FM5 did ship as described. Replaced the blanket claim with a per-item table so the doc stays useful as incident history without reading as a description of the code, and noted that the unit path throughout says /etc/containers/systemd/ while units are actually written to ~/.config/containers/systemd/ (the path is rootless). Co-Authored-By: Claude Opus 5 (1M context) --- docs/bulletproof-containers.md | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/docs/bulletproof-containers.md b/docs/bulletproof-containers.md index 737aca90..f75e845e 100644 --- a/docs/bulletproof-containers.md +++ b/docs/bulletproof-containers.md @@ -1,9 +1,25 @@ # Bulletproof Containers -**Status**: historical design record — the plan below (agreed 2026-04-22) has -been implemented. For the current description of how the lifecycle works, read -[Container lifecycle](container-lifecycle.md); this document is kept for the -incident history and the reasoning that produced the design. +**Status**: historical design record (agreed 2026-04-22). The *architecture* — +level-triggered, desired-state reconciliation — was adopted and is live. Several +specifics below were not built as written, so read this for the incident history +and the reasoning, not as a description of the code. For how the lifecycle +actually works today, read [Container lifecycle](container-lifecycle.md). + +What became of the plan, verified against the tree: + +| Item | Outcome | +|---|---| +| Level-triggered reconciler | ✅ Shipped, but as `container/boot_reconciler.rs` + `container/prod_orchestrator.rs`. The `core/archipelago/src/reconcile/` module laid out below (`desired.rs`/`current.rs`/`diff.rs`/`apply.rs`/…) **was never created** — no file in it exists | +| FM5 post-OTA probe + auto-rollback | ✅ Shipped — `update-pending-verify.json` (`update.rs:100`) | +| FM4 `host.archipelago` alias | ✅ Shipped — `AddHost=host.archipelago:10.89.0.1` in generated units | +| FM1/FM3 Quadlet ownership | ◐ Partial. Companion UIs run as Quadlet units; **main app containers do not** — `use_quadlet_backends` still defaults false, so the "v1.7.48+ full migration" below has not happened | +| FM2 bitcoin.conf drift | ◐ Solved differently. There is no `reconcile::derived::render_bitcoin_conf`; instead bitcoind is run with an explicit `-conf` derived from secrets at each start and stale datadir configs are removed (`remove_stale_bitcoin_conf`) | +| FM6 podman corrupt-state self-heal | ❌ **Not implemented.** No `podman system renumber` recovery, no startup probe for "invalid internal status". The failure that made a node unreachable in 2026-04 would still need manual SSH | + +Note also that the unit paths below say `/etc/containers/systemd/`; units are +actually written per-user to `~/.config/containers/systemd/` +(`quadlet.rs:DEFAULT_REL_UNIT_DIR`), since the whole path is rootless. **Target**: zero-manual-intervention container lifecycle. A user installs, uninstalls, reboots, updates, or loses power — every combination must leave the