Archy set no radio params on Meshcore devices — SF/CR/BW/freq lived only in
device firmware, so a radio on the wrong preset could not be fixed from the
node (it hears RF energy but demodulates nothing; the fleet hit exactly this
on an RNode: docs/RETICULUM-TRANSPORT-PROGRESS.md item 4).
- protocol.rs: build_set_radio_params — wire format verified against the
MeshCore companion firmware handler (CMD_SET_RADIO_PARAMS=11:
[11][freq:u32 LE MHz*1000][bw:u32 LE kHz*1000][sf][cr]); byte-layout test
pinned to the Portugal preset 869.618 MHz / 62.5 kHz / SF8 / CR8.
- serial.rs: MeshcoreDevice::set_radio_params (device reboots on OK).
- MeshConfig.lora_radio_params: Option — None (default) leaves the radio
untouched, so only explicitly-configured deployments are affected.
- session.rs: provision on connect, gated on a persisted last-applied marker
(SELF_INFO offsets shift across firmware versions) + the same attempt cap
as region/channel so a refusing radio never reboot-loops.
- mesh.configure RPC: lora_radio_params arm with firmware-range validation.
mesh tests: 114/114 pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Register "pine" as a manifest-driven stack (like netbird) across every
lifecycle site so install/stop/start/restart/uninstall/crash-recovery treat
the launcher + two Wyoming engines as one app:
- stacks.rs: pine_stack_app_ids() + install_pine_stack() (orchestrator-first,
adopt existing, else bail — no hardcoded fallback)
- install.rs: dispatch package_id == "pine" to the stack installer
- app_ops.rs: canonical stack-member table + owning_package STACKS
- dependencies.rs: startup_order + needs_archy_net
- config.rs: all_container_names
- crash_recovery.rs: auto-start members + a StackRecoverySpec (anchor
pine-whisper) so a genuinely-installed stack self-heals but orphan debris
does not crash-loop
- docker_packages.rs: exclude the engines from the apps list
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Supersedes the unpublished v1.7.107. Covers the multi-anchor FIPS fix, WiFi
self-heal, mesh fast-rejoin, kiosk TV-scaling, kiosk dark theme / VT hints /
UTF-8 logo / power-button hardening, and the ISO build fixes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nodes depended on a single public anchor (185.18.221.160) that is
unreachable from most real networks — confirmed on the thinkpad WiFi and a
fresh Framework node, both of which islanded (is_root=true, depth=0). Its
DNS even resolves IPv6-first while the daemon is IPv4-only.
Stand up an Archipelago-operated FIPS anchor on vps2 (146.59.87.168, the
OTA host every node already reaches) and ship it as an additional default
anchor. default_public_anchor() -> default_public_anchors() -> Vec; load()
returns the set; the apply loop already dials each, so whichever the node's
network can reach wins. Verified end-to-end: a fresh node joined the tree
via vps2 in 3s (depth 1). anchors tests: 7/7 pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two fleet fixes that OTA can deliver (unlike the ISO-only polkit rule):
- WiFi (issue #99): nodes drive NetworkManager from a system-level service
with no logind seat, so the stock polkit rule denies them and Wi-Fi setup
fails with 'Insufficient privileges'. Fresh ISOs since 2026-05 ship the
scoped rule, but OTA'd nodes never got it (OTA replaces binary + web UI,
not host system config). Add a startup self-heal that installs the rule if
missing and best-effort ensures polkitd — both wrapped so an offline/locked
apt can never fail startup; the rule is written regardless so it activates
once polkitd lands. Idempotent on the rule's unique subject.user marker.
- FIPS: regenerating fips.yaml (this build does, once, on first boot after
the OTA) restarts the fips daemon; for a few seconds /run/fips/control.sock
is gone and every seed-anchor dial fails, islanding the node until the next
5-min tick. Detect that exact failure (all dials fail on control.sock) and
retry in 15s instead — bounded to 8 fast retries/episode so a node with no
fips daemon falls back to the steady cadence rather than busy-looping.
FIPS unit suite: 33/33 pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The daemon config was built by format!-ing a YAML string literal. Upstream's
config structs are #[serde(deny_unknown_fields)], so a key we get wrong does
not degrade — the daemon refuses to start and the node drops off the mesh.
String-built config made that a runtime discovery on a live node.
Replace it with a typed struct tree serialised via serde_yaml, verified
field-by-field against jmcorgan/fips v0.4.1, plus an exact-output snapshot
test so schema drift fails in CI rather than at boot. Also adds tests for
determinism (server.rs compares the render against disk to detect drift, so
instability would cause a reinstall+restart loop) and for the mDNS key path.
Enables node.discovery.lan.enabled (mDNS/DNS-SD, added upstream in v0.4.0)
so co-located nodes peer directly instead of depending on the public anchor
being reachable — an anchor blackhole on one segment currently islands a node
completely. Emitted unconditionally rather than version-gated: v0.3.0's
DiscoveryConfig has no `lan` field and no deny_unknown_fields, so a v0.3.0
daemon ignores it harmlessly and it activates on upgrade with no second
config migration.
Note: the first startup after this lands renders a config that differs from
disk, so the existing drift check reinstalls it and restarts the daemon once.
That is the intended self-healing path and settles immediately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RPC failures logged only the top-level anyhow context, so a peer-files
failure produced exactly "RPC error on content.browse-peer: Failed to
connect to peer" with the real cause discarded. That made it impossible
to tell a dead peer from a slow Tor circuit from a FIPS resolve failure
without reproducing by hand.
Switch the log line to `{:#}` so the whole context chain is rendered.
The client-facing message still uses `{}` through sanitize_error_message,
so no internal detail is leaked to callers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vpn.peer-config returned the config exactly as stored at creation time, so
after a node moves networks the reused companion peer's QR encodes the old
location's address (seen on .116: Endpoint = 10.125.9.0 from the previous
LAN) and the tunnel can never connect. Rewrite the Endpoint line with the
node's current address before rendering the QR, and persist it back so the
downloadable .conf matches. Endpoint detection is shared with
vpn.create-peer via current_wg_endpoint_host().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After an unclean shutdown, crash recovery walked the running-containers
snapshot and retried 'no such container' failures (2 attempts, 10s
backoff) for containers that no longer exist. Recovery runs before the
server binds :5678 and notifies systemd ready, so a stale snapshot
pushed startup past TimeoutStartSec=5min — systemd killed the daemon
mid-recovery, the next boot saw a crash again, and the node looped
forever (151 restarts on .116 after a hard poweroff).
- pre-filter the snapshot against 'podman ps -a' and skip vanished
containers outright (fail-open if the query fails)
- never retry a 'no such container' failure
- extend the systemd start timeout ahead of each container start so a
heavy node recovering dozens of real containers isn't killed while
making progress
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backend: tor.list-services now hides services whose name maps to a
known-but-uninstalled catalog app (aliases covered: bitcoin/knots/core,
electrumx/electrs, btcpay, mempool). The node's own service, the relay,
and custom user services always show.
ISO: first-boot tor setup no longer pre-creates hidden services for six
apps that may never be installed — only the node's own onion is baked.
It also only seeds services.json/torrc on FIRST boot instead of
clobbering backend-managed services on every boot.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The sweep podman-execs into every running container; doing that on each 30s
reconcile tick was a permanent conmon 'Failed to create container' storm on
hosts where exec from the backend's cgroup context fails (Debian 13 first
boot). Ownership drift is an install/OTA-time event — sweep on first pass
after a container appears, then at most hourly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A stray `archipelago --version` used to start a full second instance next
to the systemd one (issue #74). Handle -V/--version, -h/--help, and reject
unknown dashed options before any tracing/state init, mirroring the
ceremony subcommand's clean-stdout precedent. Version output matches the
health RPC format: <pkg-version>-<git-hash|dev>.
Fixes#74
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both tests take the global single-flight UPDATE_OP_LOCK; run concurrently
by the test harness, one saw the other's lock and failed with 'another
update operation is already running' instead of its expected refusal.
A shared test mutex makes them mutually exclusive deterministically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
.198 crash-looped 236x on a truncated OTA binary with a good backup
sitting in update-backup/ — verify_pending_update() can't help when
the new binary never runs. New scripts/ota-crash-guard.sh runs as
root from ExecStartPre: while the post-OTA pending-verify marker
exists it counts start attempts, and after 5 failures restores the
backup binary (atomic rename), replaces the marker with an
update-rolled-back.json tombstone, and lets the service come back
on the previous version.
Wiring: fresh ISOs get the ExecStartPre line in the unit file;
existing nodes get a drop-in installed by apply_update's runtime
component step on their next OTA. '+-' prefix so a missing or
failing guard can never block the service.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of .198 bricking on the 1.7.103 OTA: two concurrent
update.download RPCs shared one staging file, cancel_download wiped
staging mid-flight, a third download began re-filling it, and
apply_update mv'd the 3-second-old 17MB partial of the 49MB binary
into /usr/local/bin -> SEGV boot loop (236 restarts, no rollback).
- Single-flight UPDATE_OP_LOCK across download/apply; concurrent
callers get an explicit 'already running' error.
- apply_update now requires the .download-complete marker AND
re-verifies every staged component (size + SHA-256 + BLAKE3)
against the manifest before touching the system.
- cancel_download only wipes staging when no operation holds the
lock; otherwise it just flags the in-flight loop to bail.
- Fixed the 'file already complete' path in
download_component_resumable, which skipped verification and fell
through to the bogus 'download failed without a captured error'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Chromium's SingletonLock is a symlink encoding <hostname>-<pid>. After a
node rename changes the OS hostname, the lock left from the previous
boot reads as "another computer" holding the profile; Chromium refuses
to start, --noerrdialogs hides the only dialog, and the kiosk
black-screens forever.
Two layers:
- kiosk launcher removes Singleton{Lock,Cookie,Socket} before every
Chromium start (any prior owner is dead at that point — pkill at
session start / loop respawn). Ships via ISO splice AND the binary's
include_str! self-heal, so existing nodes get it with the next OTA.
- the rename handler clears the lock immediately as a hostname
side-effect, covering nodes still running a pre-fix launcher.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
auth.setup only wrote the web password hash to user.json, so the
archipelago system user kept the image default password ("archipelago")
on console/SSH even after the user picked a real password during
onboarding. Reuse the existing usermod-based sync (already used by
auth.changePassword's alsoChangeSsh path) at setup time, best-effort so
a failure can never break onboarding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Lightning setup (Open a Shop, Accept Payments, Run a Lightning Node)
gains a guided path to a working channel:
- New "Fund Your Bitcoin Wallet" step, gated on the blockchain finishing
its initial sync: while syncing it shows a live progress bar with a
counting-down time-remaining estimate; once synced it shows the on-chain
balance and a "Fund Wallet" button that opens the receive modal with a
fresh address, QR, and the Zeus channel limits (min 150,000 / max
1,500,000 sats) noted.
- When the sync finishes while a Lightning setup is mid-flight, a toast
pops with a "Finish setup" link straight back to the wizard (toasts now
support action links). If several setups are in flight, one is chosen —
the shared steps complete the lightning part of any of them.
- Channel steps are Zeus-branded (logo + copy) and land on the Lightning
Channels screen, which now carries an "Open a channel with Zeus" card
that prefills the open-channel modal with the Olympus peer URI, 150k
sats, and private-channel checked. "Get Zeus" links to zeusln.com.
- Setup completion now actually requires walking the manual steps (fund,
open channel, configure) — previously any step whose app was installed
was silently auto-ticked and running apps marked the whole goal done.
- Completion CTAs now go to the app you just set up ("Go to my shop
(BTCPay)" etc.) instead of the generic services list; iframe apps open
in the on-top app overlay.
- On-chain send modal gains a "Send all funds" sweep toggle, backed by
LND's send_all on the backend (amount no longer required when sweeping).
- Demo/mock: bitcoin.getinfo now returns the block_height/sync_progress
contract the UI reads and simulates a ~90s IBD ramp per visitor so the
timer, toast, and fund flow can all be demoed live; channel list data
fixed (status/channel_point/liquidity totals — the panel previously
crashed on the missing status field); sendcoins supports send_all.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PodmanClient::pull_image had a single attempt with a hard 600s wall
clock — on a fresh node, any image that legitimately takes >10 min to
download got killed mid-pull and failed the install. Podman keeps the
completed layers, so the user's retry started half-done and succeeded:
the exact "first install fails, second works" pattern reported for
btcpay and indeedhub (and earlier for LND, whose fix in 72d7fa07 only
covered the legacy install path — this is the same cure ported to the
orchestrator path both fresh installs and upgrades use).
A pull is now only killed when nothing is observably happening — no
stderr output AND no staged-byte growth in TMPDIR for 180s — or at a
1800s absolute ceiling, with 3 attempts and backoff on top. Dead
registries still fail fast (no bytes ever land). Failures now surface
podman's actual stderr tail instead of a bare exit status, and the
pull stages via the user's containers tmp like every other pull path
(rootless /var/tmp is read-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Orchestrator-installed stacks (indeedhub, btcpay, netbird, immich,
mempool) sat at "Preparing… 5%" for the entire install — the
install_stack_via_orchestrator path never called set_install_phase /
set_install_progress after the initial Preparing, so a 7-image,
multi-gigabyte indeedhub pull gave the user zero feedback ("just says
preparing for ages"). The legacy installers already report phases and
the frontend already interpolates X-of-N across the PullingImage band;
the new path simply never fed it.
Now: PullingImage phase up front, an X-of-N counter plus a per-member
"Installing postgres (2 of 7)…" message as each member installs, and
the legacy installers' truthful PostInstall→Done settle at the end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full backup carried identity/lnd_aezeed.enc but NOT the per-node
wallet secret that encrypts it (secrets/lnd-wallet-password), so a
restored node could never decrypt its Lightning seed. The secrets dir
now rides the backup (archive v3; the .bak itself is
passphrase-encrypted). Restore gains a staging-presence guard so
restoring an older archive without a secrets dir can no longer displace
and delete the node's live secrets — covered by two new tests.
Backups can now also be downloaded through the browser: a session-gated
GET /api/blob/backup/<id> endpoint (under the existing nginx /api/blob
prefix, so no fleet nginx changes) plus a Download button next to
Verify / USB / Restore / Delete in Settings → Backup & Restore.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The health monitor opened a bare TCP connection to every published host
port each cycle. Against TLS listeners that connect+close logs noise on
the app side — LND printed 'http: TLS handshake error … EOF' endlessly.
Containers that define a podman healthcheck (LND uses lncli getinfo)
now rely on it alone; bare containers keep the port probe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two confirmed fresh-node failure modes made 'install LND' fail on the
first try and succeed on the second:
- The dependency gate failed fast when Bitcoin was still mid-install
(image pulling, container not created yet): DepProbe.existing only
covered 'podman ps -a', so an in-flight dependency looked identical to
'not installed at all'. The probe now also reports packages in
Installing/Updating state, and the gate waits up to 30 minutes for an
in-flight dependency install to finish (surfacing 'Waiting for X to
finish installing…' on the card) before its normal 3-minute
started-but-not-running window.
- Image pulls had a hard 300s per-URL wall clock, which killed
slow-but-progressing pulls of large images (LND on fresh-node WiFi);
the retry then succeeded off the warm layer cache. Pulls are now
stall-aware: killed only after 3 minutes with no podman output AND no
byte movement in the staging dir, with a 30-minute absolute ceiling.
Dead registries still fail in ~3 minutes.
Adds dep-gate unit tests for the in-flight-install wait and the
neither-installed-nor-installing fail-fast.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
server.set-name ran hostnamectl but left Debian's 127.0.1.1 line on the
old name (breaking sudo's 'unable to resolve host' and hostname -f) and
waited on avahi to notice the change by itself. Sync /etc/hosts and
avahi-set-host-name (fallback: try-restart avahi-daemon) right after a
successful rename, so http(s)://<hostname>.local works immediately —
best-effort, never blocks the rename.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Add Service' only worked for the 19 apps in the hardcoded
known_service_port map — everything else failed with 'Unknown app'.
tor.create-service and tor.toggle-app now fall back to the app's live
launch address from the scanner state, so any manifest-driven app gets
a .onion without a per-app entry (the static map still wins for
protocol services like bitcoin, which must expose 8333, not a UI port).
toggle-app also no longer writes a broken port-0 HiddenServicePort for
unknown apps.
Every successful package.install now auto-creates the app's hidden
service (detached, best-effort, retries while the scanner derives the
port; skips protocol services and existing entries).
The demo mock gains stateful tor.create-service/tor.delete-service so
the demo round-trips instead of erroring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a fresh ISO node, NetBird's own WireGuard tunnel (10.44.0.1) sorted
ahead of the real NIC in hostname -I, so the NetBird launch URL (and the
{{HOST_IP}} baked into its cert/config) pointed at the tunnel instead of
the LAN address. The main routing table's default route names the
physical uplink even while a VPN is active (NetBird/Tailscale steer
traffic via policy-routing rules, not the main table), so read src/dev
from 'ip -4 route show default' first, then fall back to a connected UDP
socket's source address, then to the old hostname -I scan.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
barkd (Ark wallet daemon, pinned 0.3.0, checksum-verified release binary)
packaged as an installable app; thin HTTP bridge in wallet/ark_client.rs
mirrors the fedimint_client pattern — the bark SDK stays out of the node
binary. wallet.ark-* RPCs cover status/balance/address/send/invoice/
board/offboard/history/configure; Ark movements merge into the unified
ecash history (kind="ark") and spendable Ark sats into total_sats.
Signet defaults (Second's public Ark server) until Ark matures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The optimistic package_data entry painted at install-spawn survived
failure as a 'Stopped' tile for an app with zero footprint (netbird on
the fresh ISO). On INSTALL FAIL with no container/user-stop marker for
the app, the entry and the pre-saved dynamic app config are removed and
an error notification is pushed; failures that left a real container
keep today's behavior (visible exited state for retry/diagnosis).
Install suite 39/39.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
write_bitcoin_conf silently produced an unpruned (~810GB and growing)
config regardless of disk: the framework node got a full-archive chain
on a 205GB volume. Volumes under 1.2TB now get prune = 25% of the
volume, clamped to [550MB, 100GB]; bigger volumes keep full archive;
unknown df results stay unpruned (don't surprise big iron). Framework
node switched live to prune=50000.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
936b4cca made ANY loaded manifest with an absent container self-heal via
install_fresh — but the signed catalog and the ISO ship manifests for
EVERY app, installed or not, so 'manifest loaded' is not installation
evidence. On a fresh node this mass-installed the catalog: the framework
node grew portainer, vaultwarden, searxng, strfry, mempool, immich and
fedimint uninvited within an hour (fleet nodes showed the same loop as
reconcile-failed noise for unpullable images).
ExistingOnly (boot) reconcile now creates containers from nothing only
for required-baseline apps (filebrowser, fedimint-clientd); every other
absent app is Left('absent') and recreated ONLY via desired-state
recovery when it was running at the last snapshot — which still covers
the vanished-container regressions (indeedhub/immich) that motivated
936b4cca. Reconcile suite 16/16.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
