diff --git a/CHANGELOG.md b/CHANGELOG.md index 2cdd976e..ecdef990 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## v1.8.22-alpha (2026-09-30) +- Network diagnostic failures no longer stop all apps or rebuild shared container networking. +- Prevented orphaned companion dashboards from repeatedly reinstalling themselves after their backend app was removed. +- Fixed companion dashboard builds still referencing a retired image registry. + - Fixed Angor Indexer health checks choosing IPv6 localhost for an IPv4 listener and unnecessarily restarting the working service. - Prevented false app restarts by probing each published port at its actual bind address; Nginx Proxy Manager now checks its internal admin API. diff --git a/core/archipelago/src/container/companion.rs b/core/archipelago/src/container/companion.rs index 42f1b8da..651b8e0b 100644 --- a/core/archipelago/src/container/companion.rs +++ b/core/archipelago/src/container/companion.rs @@ -103,6 +103,15 @@ pub fn companions_for(package_id: &str) -> &'static [CompanionSpec] { } } +/// Missing companion UIs are provisioned here, never by snapshot recovery. +/// A stale running-container snapshot must not resurrect an orphaned UI. +pub fn is_companion_app(app_id: &str) -> bool { + ALL_COMPANIONS + .iter() + .flat_map(|specs| specs.iter()) + .any(|spec| spec.image_base == app_id) +} + /// Every companion this build knows how to provision. Kept beside /// `companions_for` — a new companion must be added to both, or the reaper /// will not recognise it as one of ours and will leave it running forever. diff --git a/core/archipelago/src/container/prod_orchestrator.rs b/core/archipelago/src/container/prod_orchestrator.rs index afdede91..65424948 100644 --- a/core/archipelago/src/container/prod_orchestrator.rs +++ b/core/archipelago/src/container/prod_orchestrator.rs @@ -2071,6 +2071,10 @@ impl ProdContainerOrchestrator { Ok(ReconcileAction::Left(reason)) if mode == ReconcileMode::ExistingOnly && reason == "absent" + // companion.rs owns missing UI provisioning/removal. + // Never resurrect an orphan from a stale snapshot. + // Existing UIs still pass through security config repair. + && !super::companion::is_companion_app(&app_id) && (was_running.contains(&compute_container_name(&lm.manifest)) // The durable answer, and the one that does not // erode. `was_running` only records what was @@ -7225,6 +7229,52 @@ app: assert!(!calls.iter().any(|c| c.starts_with("start_container:"))); } + #[tokio::test] + async fn reconcile_existing_does_not_resurrect_orphaned_companions() { + let rt = Arc::new(MockRuntime::default()); + let mut orch = orch_with(rt.clone()).await; + orch.set_disk_gb_for_test(500); + let companions = [ + "bitcoin-ui", + "electrs-ui", + "lnd-ui", + "fedimint-ui", + "cuprate-ui", + ]; + let mut names = Vec::new(); + for id in companions { + let manifest = pull_manifest(id, "localhost/companion:local"); + names.push(compute_container_name(&manifest)); + orch.insert_manifest_for_test(manifest, PathBuf::from("/tmp/companion")) + .await; + } + let refs: Vec<&str> = names.iter().map(String::as_str).collect(); + crate::crash_recovery::save_container_snapshot_for_test(&orch.data_dir, &refs).await; + // Repeated passes must leave lifecycle ownership with companion.rs. + for _ in 0..3 { + let report = orch.reconcile_existing().await; + assert_eq!(report.actions.len(), companions.len()); + assert!(report + .actions + .iter() + .all(|(_, action)| *action == ReconcileAction::Left("absent".into()))); + assert!(report.failures.is_empty()); + } + let calls = rt.calls(); + for operation in [ + "pull_image:", + "create_container:", + "start_container:", + "stop_container:", + "remove_container:", + ] { + assert!( + !calls.iter().any(|call| call.starts_with(operation)), + "{calls:?}" + ); + } + } + #[tokio::test] async fn reconcile_existing_self_heals_missing_optional_installed_app() { // A non-baseline app (gitea) self-heals ONLY with installation diff --git a/docs/next-release-20260930.md b/docs/next-release-20260930.md index 16dd80d7..832f1cef 100644 --- a/docs/next-release-20260930.md +++ b/docs/next-release-20260930.md @@ -334,3 +334,54 @@ repository branch and Compose content from its own network namespace. Version preparation is 1.8.22-alpha. No new release tag or fleet-visible update manifest is published by the version commit. Optimized candidate deployment, artifact inspection, ISO smoke/boot checks and offline signatures follow. + +### Release blocker discovered during candidate observation: scheduled doctor + +The initial `02b840f2` 1.8.22 candidate is rejected for release. On the X250, +2026-09-30 21:10–21:11 UTC, the scheduled `archipelago-doctor.service` explicitly +ran `podman stop --all --time 30`, killed rootless network helpers and ran +`podman system migrate` after a two-attempt external network probe failed. +The journal attributes the stop to that unit, not the app health monitor or a +host reboot. All apps restarted, including Bitcoin, LND and the production site. +The earlier unchanged-container acceptance applies only to immediate deployment; +the later observation failed and must not be represented as a stability pass. +No persistent-data loss has been established. Keep this distinct from the closed +Framework incident; do not wipe or recreate any wallet as a recovery action. + +Containment: stopped doctor timers on both test boxes, installed a safe diagnostic +script into both the executable and runtime payload, and rejected/stopped the +old ISO build. Network failure now produces a warning without stopping apps, +killing network processes, migrating Podman or deleting network state. Repeated +failures remain warnings, never a successful repair/check. Regression cases cover +healthy, absent network, non-root invocation, host failure, transient recovery, +repeated endpoint failure and namespace access failure, with mutation tripwires. +Live scheduled-cycle observation and final rebuilt-artifact acceptance are pending. + +Recovery also exposed retired `git.tx1138.com` nginx base references in six +companion UI Dockerfiles. They now use the existing primary registry at the same +pinned version. All six images built successfully against that registry; payload +validation rejects the retired host before OTA/ISO packaging. + +The post-recovery X250 check passes: Bitcoin authenticated RPC responds and IBD +advances; NPM/Gitea/Portainer APIs respond; Portainer's real namespace fetches +`demo-portainer` at `3ae171d6b0c728665a860520fe393c0abb772798` and its Compose +file; Portainer's original persistent mounts match the earlier backup evidence; +LND wallet/channel databases remain present on their persistent mount. No new +pre-incident cryptographic wallet-identity baseline was available, so these checks +must not be described as an exact identity/balance comparison. + +The dev all-container observation also caught a separate Cuprate UI orphan loop: +`companion.rs` removed it because Cuprate was not installed, while generic desired- +state recovery resurrected it from an old running snapshot, using a unit without +nginx's required capabilities. Generic desired-state recovery now excludes missing companions +owned by `companion.rs`; existing companion provisioning/reaping remains the +single owner. Running UIs still receive the existing security configuration repairs. Regression runs repeated reconciliation against stale companion +snapshots and checks that no image/container lifecycle operations occur. + +Safe-doctor live acceptance: the X250 completed a 12-minute observation with all +running container IDs, start times and data mounts unchanged. Its journal records +successful doctor runs at 21:22:06, 21:27:51 and 21:33:10 UTC. Both doctor timers +are restored with the safe script. Dev's native Bitcoin/LND stayed running; +all-container dev acceptance remains pending the companion-loop backend fix. +Final-source UI suite: 1,133 passed. Heavy backend compilation is serialized with +remaining build steps to reduce memory/IO pressure on the syncing dev node. diff --git a/neode-ui/src/views/settings/AccountInfoSection.vue b/neode-ui/src/views/settings/AccountInfoSection.vue index 8f393d46..41cc77fe 100644 --- a/neode-ui/src/views/settings/AccountInfoSection.vue +++ b/neode-ui/src/views/settings/AccountInfoSection.vue @@ -369,6 +369,9 @@ init() September 30, 2026
+

Network diagnostic failures no longer stop all apps or rebuild shared container networking.

+

Prevented orphaned companion dashboards from repeatedly reinstalling themselves after their backend app was removed.

+

Fixed companion dashboard builds still referencing a retired image registry.

Fixed Angor Indexer health checks choosing IPv6 localhost for an IPv4 listener and unnecessarily restarting the working service.

Prevented false app restarts by probing each published port at its actual bind address; Nginx Proxy Manager now checks its internal admin API.

Added a backed-up migration for the recognized legacy Nginx Proxy Manager tunnel/LND port conflict in both OTA and ISO startup paths.