lfg2025/archy
1
0
Fork 0
Code Issues 25 Pull Requests 4 Actions Packages Projects Releases 74 Wiki Activity

fix: fresh-ISO feedback bug-bash — onboarding, status truthfulness, recovery, kiosk, logs

Browse Source 
Fixes from real fresh-install feedback (Framework node .81) + its log bundle:

Backend:
- websocket: subscribe before initial snapshot — broadcasts in the gap were
  silently lost, stranding clients on stale state until a hard refresh
  (the "everything needs ctrl-r" bug: My Apps stuck Loading, App Store
  stuck Checking, containers-scanned never arriving)
- crash recovery: check the crash marker BEFORE writing our own PID —
  recovery had never run on any node (always saw its own PID and skipped);
  PID-reuse guard via /proc cmdline
- boot status: pending-boot-starts registry (recovery, stack recovery,
  reconciler, adoption) — scanner overlays queued-but-down apps as
  Restarting instead of Stopped after a reboot; scanner-authored
  Restarting resolves immediately on a settled scan (no transitional wedge)
- install deps: bounded wait (36x5s) when a dependency is installed but
  still starting ("Waiting for Bitcoin to start…") instead of instant
  rejection; dependency-gate rejections remove the optimistic entry (no
  phantom Stopped tile) and surface as a notification
- seed backup: auth.setup persists the onboarding mnemonic as the
  encrypted seed backup (reveal previously failed on EVERY node — nothing
  ever wrote master_seed.enc); seed.restore stashes too; error sanitizer
  lets seed/2FA errors through instead of "Check server logs"
- lnd: bitcoind.rpchost resolved from the running Bitcoin variant
  (hardcoded bitcoin-knots broke Core nodes); manifest uses derived_env
- bitcoin status: clean human message for connection-reset/startup; raw
  URLs + os-error chains no longer reach the app card
- fedimint-clientd: chown /var/lib/archipelago/fmcd to 1000:1000 (root-
  created dir crash-looped the rootless container, EACCES) — first-boot
  script + pre-start self-heal
- log volume (>1GB/day on a day-old node): journald caps drop-in (ISO +
  bootstrap self-heal), bitcoind -printtoconsole=0 everywhere (90% of the
  journal was IBD UpdateTip spam), tracing default debug→info

Frontend:
- Login: Enter advances to confirm field then submits; submit always
  clickable with inline errors (was silently disabled on mismatch);
  Restart Onboarding needs a confirming second click (the mismatch →
  "onboarding restarted" trap)
- sync store: 30s state reconciliation + refetch on re-entrant connect;
  20s containers-scanned escape hatch so Checking can never show forever;
  fresh empty node reaches the real "no apps yet" state
- intro video: CRF20 re-encode (SSIM 0.988) + faststart — moov was at EOF
  so playback needed the full 15MB first (the intro lag)
- backgrounds: 10 heaviest JPEGs → WebP q90 (9.4MB→6.6MB); 7 stayed JPEG
  (WebP larger on noisy sources)
- Web5ConnectedNodes: drop unused template ref that failed vue-tsc -b

ISO/kiosk:
- nginx: /assets/ 404s no longer cached immutable for a year; HTTPS block
  gained the missing /assets/ location (served index.html as images)
- kiosk: launcher/service spliced from configs/ at ISO build (stale
  heredoc force-disabled GPU); MemoryHigh/Max 1200/1500→2200/2800M (kiosk
  rode the reclaim throttle = the lag); firmware-intel-graphics +
  firmware-amd-graphics (trixie split DMC blobs out of misc-nonfree)

Verified: cargo test 898/898 green, npm run build green with dist
contents confirmed (webp refs, lnd.png, faststart video, new strings).
Handover for ISO build + deploy: docs/HANDOVER-2026-07-02-iso-feedback.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-07-02 08:00:39 -04:00
co-authored by Claude Fable 5
parent 8256fde1a6
commit c375ecc441
65 changed files with 1514 additions and 210 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/archipelago/src/container/lnd.rs
+45 -12
View File
@@ -43,7 +43,11 @@ pub enum EnsureOutcome {
Unchanged,
}
pub async fn ensure_config(paths: &EnsurePaths, rpc_pass: &str) -> Result<EnsureOutcome> {
pub async fn ensure_config(
paths: &EnsurePaths,
rpc_pass: &str,
bitcoin_host: &str,
) -> Result<EnsureOutcome> {
fs::create_dir_all(&paths.data_dir)
.await
.with_context(|| format!("creating {}", paths.data_dir.display()))?;
@@ -52,7 +56,7 @@ pub async fn ensure_config(paths: &EnsurePaths, rpc_pass: &str) -> Result<Ensure
let existing = fs::read_to_string(&paths.conf_path)
.await
.with_context(|| format!("reading {}", paths.conf_path.display()))?;
if has_required_lnd_flags(&existing, rpc_pass) {
if has_required_lnd_flags(&existing, rpc_pass, bitcoin_host) {
return Ok(EnsureOutcome::Unchanged);
}
}
@@ -68,12 +72,11 @@ restlisten=0.0.0.0:8080\n\
bitcoin.active=true\n\
bitcoin.mainnet=true\n\
bitcoin.node=bitcoind\n\
bitcoind.rpchost=bitcoin-knots:8332\n\
bitcoind.rpchost={bitcoin_host}:8332\n\
bitcoind.rpcuser=archipelago\n\
bitcoind.rpcpass={}\n\
bitcoind.rpcpass={rpc_pass}\n\
bitcoind.rpcpolling=true\n\
bitcoind.estimatemode=ECONOMICAL\n",
rpc_pass
bitcoind.estimatemode=ECONOMICAL\n"
);
write_config_atomically(paths, &conf).await?;
@@ -653,13 +656,14 @@ fn shell_quote(s: &str) -> String {
s.replace('\'', "'\\''")
}
fn has_required_lnd_flags(conf: &str, rpc_pass: &str) -> bool {
fn has_required_lnd_flags(conf: &str, rpc_pass: &str, bitcoin_host: &str) -> bool {
let rpc_pass_line = format!("bitcoind.rpcpass={rpc_pass}");
let rpc_host_line = format!("bitcoind.rpchost={bitcoin_host}:8332");
[
"bitcoin.active=true",
"bitcoin.mainnet=true",
"bitcoin.node=bitcoind",
"bitcoind.rpchost=bitcoin-knots:8332",
rpc_host_line.as_str(),
rpc_pass_line.as_str(),
]
.iter()
@@ -678,7 +682,7 @@ mod tests {
conf_path: tmp.path().join("lnd/lnd.conf"),
};
let out = ensure_config(&paths, "secret").await.unwrap();
let out = ensure_config(&paths, "secret", "bitcoin-knots").await.unwrap();
assert_eq!(out, EnsureOutcome::Written);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoin.active=true"));
@@ -697,17 +701,46 @@ mod tests {
};
assert_eq!(
ensure_config(&paths, "first").await.unwrap(),
ensure_config(&paths, "first", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
assert_eq!(
ensure_config(&paths, "second").await.unwrap(),
ensure_config(&paths, "second", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoind.rpcpass=second"));
}
#[tokio::test]
async fn ensure_config_repairs_bitcoin_host_drift() {
// A conf written against bitcoin-knots must be rewritten when the
// node's Bitcoin variant is bitcoin-core, or LND dials a hostname
// that doesn't exist on archy-net and dies on startup.
let tmp = tempfile::TempDir::new().unwrap();
let paths = EnsurePaths {
data_dir: tmp.path().join("lnd"),
conf_path: tmp.path().join("lnd/lnd.conf"),
};
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-core").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoind.rpchost=bitcoin-core:8332"));
assert!(!conf.contains("bitcoind.rpchost=bitcoin-knots:8332"));
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-core").await.unwrap(),
EnsureOutcome::Unchanged
);
}
#[tokio::test]
async fn ensure_config_repairs_incomplete_existing_config() {
let tmp = tempfile::TempDir::new().unwrap();
@@ -721,7 +754,7 @@ mod tests {
.unwrap();
assert_eq!(
ensure_config(&paths, "repaired").await.unwrap(),
ensure_config(&paths, "repaired", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
core/archipelago/src/container/prod_orchestrator.rs
+55 -1
View File
@@ -1385,6 +1385,7 @@ impl ProdContainerOrchestrator {
.list_containers()
.await
.context("list_containers during adoption")?;
let user_stopped = crate::crash_recovery::load_user_stopped(&self.data_dir).await;
let state = self.state.read().await;
let mut report = AdoptionReport::default();
for (app_id, lm) in state.manifests.iter() {
@@ -1394,6 +1395,21 @@ impl ProdContainerOrchestrator {
.any(|c| c.name == expected || c.name == format!("/{expected}"))
{
report.adopted.push(app_id.clone());
// Adopted apps will be (re)started by boot recovery, the first
// reconcile pass, or the doctor — whichever reaches them first
// can be minutes away. Register them as pending boot-starts now
// so the scanner shows "Restarting" (not "Stopped") from the
// very first post-boot scan. Cleared per-app by the first
// reconcile pass, so a genuinely failed start surfaces.
if !state.disabled.contains(app_id)
&& !user_stopped.contains(app_id)
&& !user_stopped.contains(&expected)
{
crate::crash_recovery::pending_boot_starts_add([
app_id.clone(),
expected.clone(),
]);
}
}
}
Ok(report)
@@ -1442,8 +1458,19 @@ impl ProdContainerOrchestrator {
};
let mut report = ReconcileReport::default();
let disk_gb = self.disk_gb();
// Register every candidate before the (sequential, possibly slow)
// pass so the scanner overlays queued-but-down apps as Restarting
// instead of Stopped. Each app is deregistered as its turn finishes,
// so a start that genuinely failed shows its real state again.
crate::crash_recovery::pending_boot_starts_add(manifests.iter().flat_map(|lm| {
[
lm.manifest.app.id.clone(),
compute_container_name(&lm.manifest),
]
}));
for lm in manifests {
let app_id = lm.manifest.app.id.clone();
let container_name = compute_container_name(&lm.manifest);
if mode == ReconcileMode::ExistingOnly
&& requires_archival_bitcoin(&app_id)
&& disk_gb < ARCHIVAL_BITCOIN_DISK_GB
@@ -1452,6 +1479,8 @@ impl ProdContainerOrchestrator {
&app_id,
ReconcileAction::Left("requires-archival-bitcoin".into()),
);
crate::crash_recovery::pending_boot_start_done(&app_id);
crate::crash_recovery::pending_boot_start_done(&container_name);
continue;
}
match self.ensure_running_with_mode(&lm, mode).await {
@@ -2576,7 +2605,8 @@ impl ProdContainerOrchestrator {
}
.read("bitcoin-rpc-password")
.context("lnd pre-start: read bitcoin RPC password")?;
let outcome = lnd::ensure_config(&self.lnd_paths, &rpc_pass)
let bitcoin_host = self.bitcoin_host();
let outcome = lnd::ensure_config(&self.lnd_paths, &rpc_pass, &bitcoin_host)
.await
.context("lnd pre-start: ensure lnd.conf")?;
Ok(Some(match outcome {
@@ -2588,6 +2618,30 @@ impl ProdContainerOrchestrator {
self.ensure_btcpay_stack_dirs().await?;
Ok(Some(HookOutcome::Unchanged))
}
"fedimint-clientd" => {
// First-boot (root context) created /var/lib/archipelago/fmcd
// as root:root, but the rootless container's uid 0 maps to
// host 1000 — fmcd then crash-loops with "Permission denied
// (os error 13)". Repair ownership on every start so nodes
// installed before the first-boot fix self-heal too. (The
// generic running-container ownership sweep can't catch this:
// fmcd exits within seconds, so it's never Running when the
// sweep probes.)
let dir = "/var/lib/archipelago/fmcd";
let mkdir = host_sudo(&["mkdir", "-p", dir])
.await
.with_context(|| format!("mkdir {dir}"))?;
if !mkdir.success() {
return Err(anyhow::anyhow!("mkdir -p {dir} failed with status {mkdir}"));
}
let chown = host_sudo(&["chown", "-R", "1000:1000", dir])
.await
.with_context(|| format!("chown {dir}"))?;
if !chown.success() {
return Err(anyhow::anyhow!("chown {dir} failed with status {chown}"));
}
Ok(Some(HookOutcome::Unchanged))
}
"grafana" => {
self.cleanup_stale_grafana_port().await;
Ok(Some(HookOutcome::Unchanged))