Root cause of 'server starting up' forever / crash-on-install
(framework-pt, v1.7.114->115, 2026-07-26): on a node with many stacks,
the synchronous boot recovery (recover + start_stopped_containers) runs
BEFORE sd_notify(Ready), so the unit sits in 'activating' for minutes.
Anything touching the service in that window — a superseding
start/restart, an install-time reconcile churn — killed a half-started
instance; it exits 0 on SIGTERM and Restart=on-failure then never
restarts it. Node dead behind 'server starting up'.
Fixes:
- signal READY (+ start the watchdog keepalive) BEFORE boot recovery, so
the unit reaches 'active' in seconds; recovery/reconcile/listener
continue after. No more minutes-long activating window.
- Restart=always (was on-failure): a clean-exit SIGTERM must still bring
the daemon back. Manual is still honored.
- OTA restart via a PID1-owned transient timer (systemd-run --on-active=2)
instead of a tokio-sleep child of the process being stopped, whose
start-half was being lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phones on the FIPS mesh reach the node at http://[<fips0 ULA>], but every
shipped nginx config listened on IPv4 only — nothing answered [::]:80/443,
so mesh HTTP failed with the tunnel fully healthy (found live on
framework-pt during the vc24 5G session, 2026-07-23). Canonical conf gains
listen [::] lines and patch_nginx_conf self-heals deployed nodes (covers
the sites-enabled -> archipelago-http dev variant via the existing
canonicalize pass); validated by the existing nginx -t + rollback flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- archipelago-gamepad-keys: evdev->uinput daemon (pure python3 stdlib) that
mirrors any attached controller as a virtual keyboard — D-pad/stick to
arrows, A/B to Enter/Escape, X/Y to Space/f, shoulders to Tab/Shift+Tab.
The browser sees real keys, so controllers work inside EVERY app iframe
(IndeeHub, Jellyfin…) with zero per-app code. Ships via the ISO splice and
bootstrap self-heal (kiosk nodes only), hotplug via 5s rescans.
Implements docs/tv-input-iframe-apps.md.
- useControllerNav: an open [role=dialog][aria-modal] owns navigation —
focus is pulled inside, arrows move spatially between its controls, Enter
activates, Escape stays with the modal's own close handling. One standard
mapping for every modal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--enable-features=OverlayScrollbar gives the thin auto-hiding scrollbars a
remote browser shows (Peers view had a permanent classic scrollbar on the
TV). Rides the existing kiosk self-heal + ISO splice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of silent HDMI on kiosk TVs (framework-pt, LG TV):
1) fresh ISOs installed NO audio stack even though the kiosk launcher
expects PipeWire-Pulse — add pipewire/pipewire-pulse/pipewire-alsa/
wireplumber/alsa-utils to the rootfs and put the archipelago user in
'audio' (linger user manager gets no logind seat ACLs on /dev/snd);
2) the kiosk's boot-time Xorg modeset races the i915→HDA audio-component
bind, the ELD notify is lost, and every HDMI profile stays
'available: no' forever. Verified live: one off/on modeset re-delivers
the ELD and WirePlumber immediately switches to HDMI.
New archipelago-audio-router daemon (same splice-from-configs pattern as
the kiosk, ISO + include_str! self-heal): routes the card to the best
available HDMI profile, keeps the default sink on it, migrates live
streams on hot-plug, unmutes (HDMI forced 100% — the TV owns volume), and
re-modesets a connected external output once when no ELD reports a
monitor. bootstrap::ensure_audio_stack() installs packages + router on
already-deployed kiosk nodes via OTA (apt through systemd-run — the
service sandbox keeps /usr and dpkg read-only). main.rs also spawns the
pine satellite keeper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- GET /api/pine/status: public tier (version/uptime/bitcoin/mesh facts) for
the Pine page; bearer-token tier (Lightning balances, latest mesh message)
for the seeded Home Assistant sensors. Token minted 0600 at
secrets/pine-status-token; nginx location ships via the bootstrap
self-heal + canonical ISO conf. Replaces the per-node socat forwarder and
bitcoind RPC credentials living in HA's configuration.yaml.
- pine_ha seeder: REST sensors + four ask-Archy intents (block height,
peers, sync %, Lightning balance) with LLM tool descriptions; Claude
conversation agent (anthropic entry from the node's claude-api-key) set as
pipeline default with prefer_local_intents so exact phrases stay local and
fuzzy ones tool-route through Claude; mesh-message announce automation on
every Assist satellite; bounded config markers with legacy-block migration.
- pine-openwakeword joins the pine stack (all lifecycle sites) — on-node
wake-word engine groundwork for the custom "Yo Archy" model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The kiosk window was sized in physical pixels, but Chromium interprets
--window-size in DIPs: with any device-scale > 1 the window painted
scale-times larger than the panel, and with no window manager in the
kiosk X session --kiosk/--start-fullscreen could not snap it back. On a
4K TV at scale 3 that meant an 11520x6480 window — 9x the panel area —
which blew the compositor tile-memory budget ("tile memory limits
exceeded, some content may not draw") and froze the screen on the last
painted frame; smaller scales cropped content off the right/bottom.
- Size the window in DIPs (panel / scale) so DIPs x scale = the panel.
- Panels >=2560 wide now target a 1920-wide layout (4K -> scale 2.0,
validated on the Framework's 72" 4K TV: spacious desktop layout at 2x
sharpness); smaller panels keep the 1280 target (1080p TV -> 1.5).
- Optional ARCHIPELAGO_KIOSK_MAX_WIDTH cap: drop to the best <=N-wide
xrandr mode for weak hardware (TV upscales instead of us rasterizing).
- Source /etc/archipelago/kiosk-display.conf so the upcoming Settings
display-resolution picker can manage these knobs per node.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three fresh-install kiosk issues found on a Framework node:
- MOTD said 'Ctrl+Alt+F7 for kiosk' / 'F1 for terminal', but the kiosk's
Xorg runs on vt1 — so F7 was an empty black VT (the reported black screen
and the failed 'return to kiosk'). Corrected: kiosk=F1, terminal=F2.
- The UTF-8 block-glyph logo corrupted intermittently because returning from
the kiosk can leave the console VT out of UTF-8 mode; force it with ESC%G.
- A short power-button press shut the node down instantly. Ignore the short
press, keep long-press poweroff, so accidental brushes don't kill the node.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AIUI themes via @media (prefers-color-scheme) and falls back to its light
variant (white panels) when the browser reports no preference — which the
minimal kiosk X session does, so AIUI came up light instead of its designed
dark. The main UI hardcodes dark and is unaffected. Push two independent
dark signals on the kiosk Chromium: GTK_THEME=Adwaita:dark (version-
independent, can never force light) and --blink-settings=preferredColorScheme=0
(0 = kDark in modern Chromium). Needs on-device kiosk verification.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v1.7.104 shipped ISOs whose units crash-looped (nostr-relay 3s loop,
archipelago-diag 203/EXEC) because build-time extraction failures were mere
warnings and the cached rootfs never tracked its recipe:
- hash the rootfs-defining region of the build script and rebuild when it
changes (stale cache shipped ISOs without wpasupplicant/iw/rfkill)
- refuse to build without nvpn / nostr-rs-relay unless
ALLOW_MISSING_VPN_BINARIES=1
- ConditionPathExists on nostr-relay/nostr-vpn/diag units so a stripped
image skips them instead of crash-looping
- post-install test: every enabled archipelago*/nostr* unit must have an
existing ExecStart payload
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
.198 crash-looped 236x on a truncated OTA binary with a good backup
sitting in update-backup/ — verify_pending_update() can't help when
the new binary never runs. New scripts/ota-crash-guard.sh runs as
root from ExecStartPre: while the post-OTA pending-verify marker
exists it counts start attempts, and after 5 failures restores the
backup binary (atomic rename), replaces the marker with an
update-rolled-back.json tombstone, and lets the service come back
on the previous version.
Wiring: fresh ISOs get the ExecStartPre line in the unit file;
existing nodes get a drop-in installed by apply_update's runtime
component step on their next OTA. '+-' prefix so a missing or
failing guard can never block the service.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Chromium's SingletonLock is a symlink encoding <hostname>-<pid>. After a
node rename changes the OS hostname, the lock left from the previous
boot reads as "another computer" holding the profile; Chromium refuses
to start, --noerrdialogs hides the only dialog, and the kiosk
black-screens forever.
Two layers:
- kiosk launcher removes Singleton{Lock,Cookie,Socket} before every
Chromium start (any prior owner is dead at that point — pkill at
session start / loop respawn). Ships via ISO splice AND the binary's
include_str! self-heal, so existing nodes get it with the next OTA.
- the rename handler clears the lock immediately as a hostname
side-effect, covering nodes still running a pre-fix launcher.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 454c4bb2 kiosk flattening only covered the Dashboard's background
stack, but the kiosk boots into the login/onboarding screen —
OnboardingWrapper.vue — whose scoped 3D stack (perspective +
preserve-3d bg layers + mix-blend-mode glitch overlays) has the exact
compositing pattern that fails to repaint under the kiosk's software
compositor, leaving the black body fill visible. Extend the same
html.kiosk-mode flattening to this screen via :global() rules: 2D
opacity crossfades stay, 3D transforms / blur filters / blend-mode
glitch layers go.
Also raise the kiosk chromium JS heap cap 128→256MB: the intro
cinematic (video + audio + app) inside a 128MB V8 old-space risks
renderer OOM, and a crashed kiosk tab auto-reloads — indistinguishable
from the reported "flickers and refreshes".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The unit's ExecStartPre baked the first hostname -I token into
host-ip.env, which config.rs prefers over its own detection — once a
VPN/bridge interface existed (netbird's wg tunnel), every host_ip
consumer (install configs, VPN, mint URLs) got the tunnel IP. Read the
main-table default route's src address instead, falling back to
hostname -I on routeless hosts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
484.kitchen, Arch Presentation, Call the Operator, Next Web News
Network, Syntropy Institute, and T-0 were hardcoded web-only bookmark
apps (no manifests/containers). Removed across the frontend app grid /
curated / marketplace / session / launcher data, the backend
AppMetadata table, their icons, and the nginx /ext/ proxy blocks
(external-app-proxies.conf deleted — nothing installed it anymore).
botfights/nostrudel/gitea keep the shared external-app code paths.
Frontend: vue-tsc clean, 672 tests pass, production build verified
clean of the removed slugs. Backend: cargo check clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ThinkPad X250 (Broadwell HD 5500) internal panels show horizontal corruption
bands at the kiosk; known i915 eDP power-feature issue. Boot installed and
live systems with i915.enable_ips=0 enable_psr=0 enable_fbc=0 (no-ops off
Intel, nothing lost on a kiosk). Debug boot entries stay stock for A/B.
Also size the Chromium kiosk window from the detected mode instead of a
hardcoded 1920x1080 that clips on 1366x768 panels.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kiosk (backgrounds + FIPS mismatch, one root cause each):
- kiosk service now also waits for the web-ui asset swap (probes a large
bg asset, bounded 60s) — it launched mid-rsync and CSS background 404s
are never refetched, hence blank backgrounds on first boot
- FipsNetworkCard polls every 15s instead of fetching once on mount
- PWA updates auto-apply on the kiosk (localStorage kiosk flag) — an
unattended kiosk could never click 'Update Now' and ran stale bundles
LND App Store:
- catalog icon pointed at lnd.svg which doesn't exist (lnd.png does);
store card also retries the sibling extension before the placeholder
- image pulls retry once after fast transient failures (<30s) — fresh
first-boot networks reset connections; slow failures aren't retried
so the UI never waits longer than before
Mesh detection:
- mesh.status now always reports device_present + detected_devices
(previously only when the service wasn't running, so the UI couldn't
tell 'no radio' from 'radio present, session connecting')
- Mesh page shows 'Mesh device detected' badge + 'Device detected —
connecting' status for present-but-not-connected radios
- the 99-mesh-radio.rules udev rule was never staged into the ISO tree
(installer searched for it and found nothing) — ship it at the media
root and install it from install-to-disk.sh too
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict resolution: keep the branch's CPUQuota=200% (75% was the proven
cause of choppy HDMI audio — the file's own comment says so; the 75% on main
was an accidental regression bundled into an unrelated UI commit) and keep
main's newer MemoryMax=2800M/MemoryHigh=2200M tuning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes from real fresh-install feedback (Framework node .81) + its log bundle:
Backend:
- websocket: subscribe before initial snapshot — broadcasts in the gap were
silently lost, stranding clients on stale state until a hard refresh
(the "everything needs ctrl-r" bug: My Apps stuck Loading, App Store
stuck Checking, containers-scanned never arriving)
- crash recovery: check the crash marker BEFORE writing our own PID —
recovery had never run on any node (always saw its own PID and skipped);
PID-reuse guard via /proc cmdline
- boot status: pending-boot-starts registry (recovery, stack recovery,
reconciler, adoption) — scanner overlays queued-but-down apps as
Restarting instead of Stopped after a reboot; scanner-authored
Restarting resolves immediately on a settled scan (no transitional wedge)
- install deps: bounded wait (36x5s) when a dependency is installed but
still starting ("Waiting for Bitcoin to start…") instead of instant
rejection; dependency-gate rejections remove the optimistic entry (no
phantom Stopped tile) and surface as a notification
- seed backup: auth.setup persists the onboarding mnemonic as the
encrypted seed backup (reveal previously failed on EVERY node — nothing
ever wrote master_seed.enc); seed.restore stashes too; error sanitizer
lets seed/2FA errors through instead of "Check server logs"
- lnd: bitcoind.rpchost resolved from the running Bitcoin variant
(hardcoded bitcoin-knots broke Core nodes); manifest uses derived_env
- bitcoin status: clean human message for connection-reset/startup; raw
URLs + os-error chains no longer reach the app card
- fedimint-clientd: chown /var/lib/archipelago/fmcd to 1000:1000 (root-
created dir crash-looped the rootless container, EACCES) — first-boot
script + pre-start self-heal
- log volume (>1GB/day on a day-old node): journald caps drop-in (ISO +
bootstrap self-heal), bitcoind -printtoconsole=0 everywhere (90% of the
journal was IBD UpdateTip spam), tracing default debug→info
Frontend:
- Login: Enter advances to confirm field then submits; submit always
clickable with inline errors (was silently disabled on mismatch);
Restart Onboarding needs a confirming second click (the mismatch →
"onboarding restarted" trap)
- sync store: 30s state reconciliation + refetch on re-entrant connect;
20s containers-scanned escape hatch so Checking can never show forever;
fresh empty node reaches the real "no apps yet" state
- intro video: CRF20 re-encode (SSIM 0.988) + faststart — moov was at EOF
so playback needed the full 15MB first (the intro lag)
- backgrounds: 10 heaviest JPEGs → WebP q90 (9.4MB→6.6MB); 7 stayed JPEG
(WebP larger on noisy sources)
- Web5ConnectedNodes: drop unused template ref that failed vue-tsc -b
ISO/kiosk:
- nginx: /assets/ 404s no longer cached immutable for a year; HTTPS block
gained the missing /assets/ location (served index.html as images)
- kiosk: launcher/service spliced from configs/ at ISO build (stale
heredoc force-disabled GPU); MemoryHigh/Max 1200/1500→2200/2800M (kiosk
rode the reclaim throttle = the lag); firmware-intel-graphics +
firmware-amd-graphics (trixie split DMC blobs out of misc-nonfree)
Verified: cargo test 898/898 green, npm run build green with dist
contents confirmed (webp refs, lnd.png, faststart video, new strings).
Handover for ISO build + deploy: docs/HANDOVER-2026-07-02-iso-feedback.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Routes ALSA's "default" device through PulseAudio/PipeWire. Was a manual,
untracked fix living only on archy-x250-exp — a reprovision or fresh image
would silently lose it, same failure mode that already bit the GPU-flags
and CPUQuota fixes. Wired into deploy-to-target.sh (both the primary --live
path and the .198/.253 secondary-copy path) and deploy-tailscale.sh, mirroring
the existing 99-mesh-radio.rules udev sync pattern (diff-check, copy if changed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both had regressed back to the pre-2026-06-28-incident state: GPU_FLAGS used
--enable-gpu-rasterization on any node with a GPU, and CPUQuota was 75%.
On archy-x250-exp (Intel HD 5500 under X11) this spins a dedicated GPU
process at 55-92% CPU (falls back to software compositing anyway), and the
75% cgroup quota throttled the kiosk unit ~81% of the time (nr_throttled
4856/6005) — the exact CPU-starvation signature documented as the choppy-
audio root cause. Restores the validated fix: --in-process-gpu,
--num-raster-threads=1, GpuRasterization disabled via --disable-features,
CPUQuota=200%. Verified on archy-x250-exp: no separate gpu-process, throttling
dropped to ~3% (nr_throttled 14/503).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The launcher's sudo -u archipelago invocation set DISPLAY/HOME but not
XDG_RUNTIME_DIR, so Chromium's audio backend couldn't find the PipeWire-Pulse
socket at /run/user/<uid>/pulse/native. It silently fell back to raw ALSA
"default", which also failed ("Connection refused"), producing no HDMI audio
at all with no visible error since --noerrdialogs suppresses it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
configure_display picked whichever mode was already "active" on the HDMI
output, so if X ever booted cloned to the laptop panel's resolution it
would keep re-confirming that wrong mode forever instead of self-healing
to the display's native mode.
The kiosk chromium pinned ~92% of a core (software-compositing spin from
--enable-gpu-rasterization on a GPU-less/headless node), saturating the machine
and starving the backend + container builds — it caused the .198 receive timeout
and the deploy storms.
- archipelago-kiosk.service: CPUQuota=75% + MemoryMax/High + Delegate, so a
runaway kiosk can never take the whole node down.
- archipelago-kiosk-launcher.sh: detect /dev/dri — use GPU rasterization only
when a GPU exists, else --disable-gpu (avoids the headless spin).
- bootstrap::ensure_kiosk_hardened: OTA self-heal that installs the updated
unit+launcher on already-deployed nodes, daemon-reloads, and only try-restarts
a *running* kiosk (never re-enables an operator-disabled one).
cargo check clean; launcher bash -n clean; unit syntax valid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On production nodes /var/lib/archipelago (the app data dir AND podman's
graphroot=/var/lib/archipelago/containers/storage) is a separate
device-mapper volume. archipelago.service ordered only After=network-online
.target, so on cold boots it (and its ExecStartPre) could start BEFORE
var-lib-archipelago.mount, write to the bare mountpoint on rootfs, fail every
podman call, exit, and be restarted every 5s until the volume mounted — the
"~20x [FAILED] Failed to start over ~5min" boot flap. Proven live on .198:
"var-lib-archipelago.mount: Directory /var/lib/archipelago to mount over is
not empty, mounting anyway" — the service had written there pre-mount.
Fix: RequiresMountsFor=/var/lib/archipelago (adds Requires= + After= on the
mount unit).
- image-recipe/configs/archipelago.service: ships the directive on fresh ISOs.
- bootstrap::ensure_archipelago_mount_ordering(): self-heals already-deployed
nodes' installed unit + daemon-reload (boot-ordering only, effective next
reboot; never restarts the running service). Idempotent; harmless on rootfs
installs (maps to the always-mounted root).
Verified on .198: after applying, systemctl shows After=var-lib-archipelago
.mount and systemd-analyze verify is clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fedimint UI HTML/CSS reference absolute /assets/* paths; under /app/fedimint/
those hit the main SPA, not the fedimint container, so the UI renders
unstyled. Add the proven sub_filter asset-rewrite pattern (as indeedhub/
botfights use) to the /app/fedimint/ block in the nginx template + https
snippet (also rewrites url(...) for the CSS background image). Bootstrap
self-heal for already-deployed nodes is the documented resume point.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hotfix: archipelago.service ExecStartPre now mkdirs /run/containers and
/var/lib/containers before the unit's mount-namespace setup tries to bind
them. Without this, fresh nodes that don't have /run/containers (e.g.
nodes provisioned without a prior podman session) fail at the namespace
step with:
Failed to set up mount namespacing: /run/containers: No such file or directory
Failed at step NAMESPACE spawning /bin/bash: No such file or directory
Existing nodes don't pick up systemd unit changes via OTA — they need a
one-time `systemctl edit archipelago` adding the same mkdir. ISO installs
from this version forward have the fix baked in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The backend runs as `archipelago` and calls `install_log()` to append
audit lines to the install log on every install / update / remove /
start / stop / restart. Target path was /var/log/archipelago-container-installs.log,
which does not exist and cannot be created by the service because
/var/log/ is root-owned. OpenOptions errors were silently swallowed,
so the log was never written on any node.
Ship a tmpfiles.d rule that pre-creates /var/log/archipelago/ and
container-installs.log with archipelago:archipelago ownership. Move
the const path to match, keeping logs inside the directory logrotate
already rotates (image-recipe/configs/logrotate.conf). Install the
rule from both the ISO build and self-update, and apply it
immediately on self-update so existing nodes get a working log
without needing a reboot.
Verified on .228: file created, backend user can write, backend
binary rebuilt with new const.
Adds Delegate=memory pids cpu io to the archipelago.service unit.
Context: the service runs as User=archipelago under system.slice with
rootless podman. When podman creates transient libpod-*.scope units for
containers under user.slice, systemd needs the caller to hold
CAP_SYS_ADMIN on the target cgroup subtree \u2014 which happens iff
Delegate= lists the controllers we want to set. Without Delegate, any
future code path that goes through the podman CLI (runtime.rs) instead
of the libpod HTTP API (podman_client.rs) would hit MemoryMax
rejections that have exactly the same symptom as the bug I just fixed
in parse_memory_limit but with a completely different root cause.
Belt-and-braces: current production path uses PodmanClient and was
fixed in the preceding commit. But the DockerRuntime CLI path in
runtime.rs:262-268 (cmd.arg("--memory")) is still reachable via
AutoRuntime fallback on hosts without podman, and future rust
orchestrator code may legitimately need cgroup delegation. This
directive is no-op harmful on hosts that already delegate upstream
(systemd gracefully handles duplicate/nested delegation).
BootReconciler (in-process, 30s interval, spawned from main.rs as of
Step 6 commit 48f08aa3) fully replaces the timer-driven bash
reconciliation path. Delete the systemd unit + timer and their
ISO-builder touchpoints.
Removed:
- image-recipe/configs/archipelago-reconcile.service
- image-recipe/configs/archipelago-reconcile.timer
- image-recipe/build-auto-installer-iso.sh L412-413 (COPY unit+timer)
- image-recipe/build-auto-installer-iso.sh L449 (systemctl enable)
- image-recipe/build-auto-installer-iso.sh L542-543 (cp to WORK_DIR)
Kept (intentionally):
- scripts/reconcile-containers.sh
- scripts/container-specs.sh
Reason: core/archipelago/src/api/rpc/package/update.rs still invokes
reconcile-containers.sh at two sites (OTA update + rollback paths).
Porting those call sites to ContainerOrchestrator::upgrade() requires
manifests for every container update.rs might touch — that scope
belongs in Step 8b. Until then the script stays on disk, just no
longer runs on a periodic timer.
No Rust code changes. cargo check -p archipelago clean, 6 pre-existing
warnings. Skipped full ISO rebuild validation per user decision —
edits are 5 textual deletions with zero behavioral ambiguity; Step 9
live hot-swap on .228 will catch any regression.
Install flow
- api/rpc/package/install.rs: always append the literal image URL as a
last-resort pull candidate in do_pull_image, so images not carried by
any configured mirror (docker.io/bitcoin/bitcoin:28.4) still install
instead of masquerading as a generic pull failure across every mirror.
- api/rpc/package/install.rs: write_bitcoin_conf now skips on any stat
error, not just "file exists". Once bitcoin-knots' first-boot chowns
/var/lib/archipelago/bitcoin into the container's user namespace (700
perms, UID 100100/100101), the archipelago daemon can't even traverse
in — try_exists returns Err which unwrap_or(false) treated as "not
present" and drove a doomed write. Now errors out of the directory
traversal are treated as "conf already owned by container user" and
the write is skipped. Mirrors the lnd.conf pattern.
- api/rpc/package/install.rs: drop the hardcoded `prune=550` from the
conf default. Operators with multi-TB drives shouldn't be silently
pruned; users who want a pruned node can set it in bitcoin.conf
themselves. Full archive is the only honest default.
- api/rpc/package/config.rs: bitcoin-core now passes explicit
-server/-rpcbind/-rpcallowip/-rpcport/-printtoconsole/-datadir CLI
args. Vanilla bitcoin/bitcoin:28.4 has no entrypoint wrapper and
reads conf + argv only; without these the RPC listens on 127.0.0.1
inside the container and rootlessport can't reach it, so the
bitcoin-ui companion gets 502 on every /bitcoin-rpc/ call.
Bitcoin Knots keeps its own entrypoint-driven defaults.
- container/docker_packages.rs: split bitcoin-core out of the shared
AppMetadata arm. bitcoin-core now surfaces as "Bitcoin Core" with
bitcoin-core.svg and a Reference-implementation description; the
bitcoin + bitcoin-knots ids keep the Knots branding. Fixes the home
card showing "Bitcoin Knots" for a Core install.
Bitcoin node UI (docker/bitcoin-ui)
- index.html: impl name/tagline/logo now dynamic. applyImplBranding()
reads subversion from getnetworkinfo — /Satoshi:X/Knots:Y/ resolves
to Bitcoin Knots, plain /Satoshi:X/ resolves to Bitcoin Core. Both
get their own icon and subtitle. Settings modal replaced its
hardcoded Regtest/txindex=1/port-18443 placeholders with live values
from getblockchaininfo + getindexinfo + getzmqnotifications.
- index.html: new Storage info card (Full Archive · X GB /
Pruned · X GB from blockchainInfo.pruned + size_on_disk) visible on
the main dashboard, same level as Network. Settings modal mirrors it
with the prune height when applicable.
- Dockerfile + assets/: bitcoin-core.svg, bitcoin-knots.webp, and the
bg-network.jpg used by the dashboard are now COPY'd into the image
under /usr/share/nginx/html/assets. Previously the <img src> pointed
at paths that 404'd into the SPA fallback and the onerror handler
hid the broken logo silently.
Frontend
- appSession/appSessionConfig.ts: add bitcoin-core to APP_PORTS (8334),
HTTPS_PROXY_PATHS (/app/bitcoin-ui/), and APP_TITLES (Bitcoin Core).
Without these the AppSessionFrame showed "No URL found for
bitcoin-core" and the home/app-list title fell through to the raw id.
- settings/AccountInfoSection.vue: backfill What's New entries for
v1.7.31 through v1.7.37 that had been missed in earlier cuts.
Release plumbing
- releases/v1.7.37-alpha/: binary + frontend tarball.
- releases/manifest.json: v1.7.37-alpha, sha256/size refreshed.
- Cargo.toml / package.json: version bumps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. FIPS auto-activate at server startup only fires if fips_key already
exists on disk, which on a fresh install is never true until AFTER
onboarding. By the time the user completes seed-generate/restore,
archipelago has been running for minutes and the startup task has
long since exited. User still had to hit Activate.
Fix: call spawn_post_onboarding_fips_activate() from the tail of
handle_seed_generate and handle_seed_restore — the moment the
fips_key materialises, a detached task runs `fips::config::install`
+ `archipelago-fips.service activate`. Logged only, never blocks
the onboarding RPC.
2. Kiosk health-poll window was 30 × 2s (configs/ copy was 60 × 2s
but unused — the heredoc in build-auto-installer-iso.sh is what
actually lands on disk). On .198's slower hardware archipelago
/health wasn't ready within 60s, so Chromium launched against a
not-yet-running backend → blank window until manual reboot. Bumped
to 150 × 2s (5 min) + TimeoutStartSec=360. .253 was already well
within the window; this protects the slower box too. Standalone
configs/archipelago-kiosk.service updated in lockstep so the two
copies don't drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. nostr-vpn still failing despite last mask attempt — confirmed in
the 6th ISO's rootfs.tar: the .service file was present but
not in multi-user.target.wants. Previous `systemctl mask` silently
no-oped because the real file was already there. Fixed properly
with explicit `rm -f` + `ln -sf /dev/null` for nostr-vpn,
archipelago-wg, and archipelago-wg-address — same /dev/null
symlink state that `mask` would produce on a clean install.
2. Kiosk didn't come up on first boot, only on reboot. Extended the
ExecStartPre health-poll from 30s → 120s (unbundled ISO takes
longer to settle on first boot: archipelago initializes state,
pulls FileBrowser, frontend settles), raised TimeoutStartSec to
180s, and added After=systemd-user-sessions.service +
After=network-online.target so X / Chromium aren't racing.
3. /init: line 29: can't create /root/etc/network/interfaces error
on installer boot — debootstrap --variant=minbase omits ifupdown
so the target has no /etc/network/ directory, and live-boot's
init tries to seed it. Non-fatal but noisy. Added ifupdown +
isc-dhcp-client to the debootstrap --include list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Was seeing "upload failed: 413" on mesh attachment sends between
federated nodes — a ~7MB image becomes ~10MB base64 in the
typed_envelope wire and hit the 10m client_max_body_size on
/archipelago/, /content/, and /dwn/. Bumped those six locations
(two per server block, regular + HTTPS) to 256m so modern
attachments/blobs don't trip the proxy. /rpc/ stays at 1m —
internal JSON-RPC calls are small and don't need the headroom.
Applied to all 4 fleet nodes live; ISO source config updated so
fresh installs get the same limits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bakes the FIPS (Free Internetworking Peering System) mesh daemon into
the node stack, supervised by archipelago alongside Tor. Runs as a
system service, identity derives from the same BIP-39 master seed, and
user-triggered updates track upstream main.
Identity
seed.rs: new HKDF label archipelago/fips/secp256k1/v1 → dedicated
secp256k1 key, distinct from the Nostr-node key for crypto isolation
but still seed-recoverable
identity.rs: writes fips_key[.pub] to /data/identity on onboarding,
chmod 0600; fips_key_exists / load_fips_keys / fips_npub accessors
Transport
TransportKind::Fips=3 inserted between LAN and Tor (Tor bumps to 4)
→ router prefers FIPS over Tor for all peer traffic
PeerRecord gains fips_npub + last_fips fields (serde(default) for
backward-compat with older nodes)
transport/fips.rs: NodeTransport stub, reports unavailable until the
daemon is live so router falls through to Tor cleanly
Federation invites
FederatedNode and FederationInvite carry optional fips_npub
create_invite / accept_invite / peer-joined callback thread it end
to end; signature domain deliberately unchanged — FIPS Noise does
its own session auth, so the unsigned hint only affects path
selection
crate::fips
config.rs: renders /etc/fips/fips.yaml and sudo-installs key material
service.rs: systemctl status/activate/restart/mask wrappers
update.rs: GitHub API check against upstream main; apply stubbed
until per-commit .deb artefact source is decided
RPC + dashboard
fips.status / fips.check-update / fips.apply-update / fips.install /
fips.restart registered in dispatcher
HomeNetworkCard.vue shipped standalone (unmounted — place in Home.vue
when ready); shows state pill, version, FIPS npub, update button,
activate button when key is present but service is down
ISO + systemd
archipelago-fips.service: conditional on key presence, masked by
default — backend unmasks after onboarding writes the key
build-auto-installer-iso.sh: multi-stage Dockerfile builds the FIPS
.deb from jmcorgan/fips main (fail-loud), COPYs it into rootfs, apt
installs it so trixie resolves deps; unit copied + masked
Version bump: 1.3.5 → 1.4.0
Tests: 33 new/updated passing (seed, identity, transport, federation,
fips module, transport::fips).
Known gaps: fips.apply-update returns a clear stub error until
upstream publishes per-commit .deb artefacts; HomeNetworkCard is not
mounted in Home.vue by default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
