lfg2025/archy
1
0
Fork 0
Code Issues 24 Pull Requests 4 Actions Packages Projects Releases 71 Wiki Activity

fix(container): unwire the companion reaper — absence is not uninstallation

Browse Source 
Caught on archi-dev-box within minutes of deploying 3ac59a73: the reaper
removed archy-bitcoin-ui and archy-lnd-ui, whose backends ARE installed.
archy-bitcoin-ui was gone for 36 minutes, until the operator reinstalled
bitcoin-knots and `reconcile` put the companion back.

Not a logic error — the arithmetic did what it was told. The inputs were
false. Both backends' containers were missing because of the clean-exit
vanishing bug (8908fb4f), and both had already aged out of
running-containers.json, which only ever records what is CURRENTLY RUNNING.
So the two signals `installed_app_ids` combines are not independent: one root
cause falsifies both simultaneously. ORPHAN_GRACE could not help either — the
condition was persistent, not transient, which is exactly the case the grace
period cannot distinguish.

The asymmetry decides it. An un-reaped orphan costs a stale UI tile. A
wrongly-reaped companion costs a working screen and turns one lost app into
two — the reaper amplifies the very failure it was meant to tidy up after.

`reap_orphans` and its tests stay, documented as NOT TO BE WIRED until a
durable record of "this app is installed" exists to drive it. Inferring
installation from runtime state cannot answer that question, however many
runtime signals are combined.

The provisioning half is untouched and is the actual fix for "fedimint
installs but does not work": driving `reconcile` from installed_app_ids means
a companion is never stood up for an app nobody installed, so no NEW orphans
appear. The one genuine orphan on this node (archy-fedimint-ui, for an app
never installed) was correctly removed before this change landed.

Unit tests passed the reaper because they verify the set arithmetic, not
whether the "installed" signal is truthful. Only the device could show that.

Container suite 221/221.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-08 12:08:48 -04:00
co-authored by Claude Opus 5
parent 0ce88de87f
commit ec0cfd1b7d
2 changed files with 46 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/archipelago/src/container/boot_reconciler.rs
+28 -15
View File
@@ -128,21 +128,34 @@ impl BootReconciler {
continue;
};
let failures = crate::container::companion::reconcile(&installed).await;
// Reap failures are logged but deliberately kept OUT of
// `failures`, which drives the backoff below. Repair keeps
// a companion available; reaping only tidies one away. A
// reap that fails persistently (a wedged systemctl, say)
// must not back the repair path off to its 1h ceiling and
// starve the thing that actually matters.
for (companion, err) in
crate::container::companion::reap_orphans(&installed).await
{
tracing::warn!(
companion = %companion,
error = %err,
"companion reap failed"
);
}
// `reap_orphans` is deliberately NOT called here. It is
// implemented and tested, and it must stay unwired until a
// DURABLE record of "this app is installed" exists.
//
// Proven harmful on archi-dev-box 2026-08-08: it removed
// archy-bitcoin-ui (36 minutes of no Bitcoin UI, until the
// operator reinstalled the backend) and archy-lnd-ui, both
// for apps that ARE installed. It was not a logic error —
// it did exactly what it was told. The inputs lied: the
// backends' containers were missing because of the
// clean-exit vanishing bug, and both had already aged out
// of running-containers.json, which only ever records what
// is CURRENTLY RUNNING. So container-presence and
// installation-evidence, the two independent signals the
// reaper trusts, were false at the same time and for the
// same underlying reason.
//
// Reaping turns one lost app into two, which is strictly
// worse than the orphan it cleans up. Leaving an orphan
// costs a stale UI tile; reaping a live app's companion
// costs the operator a working screen. Until "installed"
// can be answered without inferring it from runtime state,
// absence is not evidence of uninstallation.
//
// The provisioning half above is the actual fix for
// "fedimint installs but does not work" and stands on its
// own: a companion is never stood up for an app nobody
// installed, so no NEW orphans are created.
for (companion, err) in &failures {
tracing::warn!(
companion = %companion,
core/archipelago/src/container/companion.rs
+18
View File
@@ -682,6 +682,24 @@ fn due_after_grace(
/// Stop and remove any companion whose backend app is not installed.
///
/// ⚠️ NOT WIRED, ON PURPOSE. Do not call this from the reconciler until a
/// DURABLE record of "this app is installed" exists to drive it.
///
/// It ran on archi-dev-box on 2026-08-08 and removed two companions whose
/// backends were installed — archy-bitcoin-ui (36 minutes of no Bitcoin UI)
/// and archy-lnd-ui. Not a bug in the arithmetic below: the inputs were false.
/// Both backends' containers were missing because of the clean-exit vanishing
/// bug, and both had aged out of `running-containers.json`, which only records
/// what is CURRENTLY RUNNING. The two signals `installed_app_ids` combines are
/// therefore not independent — one root cause falsifies both at once, and
/// `ORPHAN_GRACE` cannot help because the condition is persistent, not
/// transient.
///
/// The asymmetry that settles it: an un-reaped orphan costs a stale UI tile,
/// while a wrongly-reaped companion costs a working screen and turns one lost
/// app into two. Absence of evidence of installation is not evidence of
/// uninstallation.
///
/// The counterpart to `reconcile`, which can only ever *add*. Without this,
/// a companion outlives its backend permanently: `remove_for` fires only on
/// the explicit uninstall RPC path, so an install that fails after the