Fixes 4 test failures introduced by 8e814ca0, which I pushed after running
only the container-crate tests while the full suite was still compiling. Both
failures were real defects, not stale assertions.
1. Catalog-driven installs would have failed fleet-wide.
8e814ca0 dropped the old registry address from TRUSTED_REGISTRIES, but the
signed catalog still advertises image refs on it — deliberately, since
rewriting a signed artifact invalidates its signature. Nodes resolve apps
through the catalog, so every install would have been refused with "not
from a trusted registry". Reinstated as LEGACY_REGISTRY_HOST, documented
as transitional and removable only once the catalog is re-signed.
2. The update fallback lost the property it exists for.
update.rs keeps two mirrors on purpose: the domain as primary, and the
old IP over plain HTTP as a fallback, because a node whose DNS or clock is
wrong (both break TLS) must still be able to update itself — the signature,
not the transport, is what makes either source safe. The bulk rewrite
pointed both constants at the domain, leaving the escape hatch dependent on
exactly what it exists to survive. Restored to its original value.
Separately, validate-app-manifest.sh is ported from ruby to python3+PyYAML.
It shelled out to ruby with stderr discarded, so on any machine without ruby
a missing interpreter was reported as "Valid YAML with top-level app block:
FAIL" and every manifest came back REJECTED. This is the first tool an app
developer runs, and it sent them to fix YAML that was never broken. Ruby was
also the odd dependency out — the repo already ships three python scripts.
It now checks for python3 and PyYAML up front and names what is missing, then
parses with PyYAML. Missing keys resolve to an absent-value object that
indexes to itself and prints empty, so call sites lost their per-hop guards:
(((app["container"] || {})["build"] || {})["context"])
becomes app["container"]["build"]["context"]. Booleans still print as
true/false rather than Python's True/False — call sites compare == "true",
so Python's capitalisation would have silently inverted the readonly_root
and no_new_privileges security checks.
Verified: full rust suite 1148/1148, 0 failed. All 56 app manifests validate
(0 rejected, 0 errored) where previously every one was rejected. No signed
artifact modified.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the registry host across 86 files: 309 references, covering all 40
app manifests, the orchestrator and container crates, the release and catalog
scripts, both demo-images workflows, the ISO builder, demo-deploy, and the
frontend marketplace data.
Verified the domain actually serves the registry before rewriting anything,
rather than assuming the web host implies the registry:
- TLS verifies clean, HTTP/2 on the web root
- an anonymous token grants a manifest fetch (HTTP 200) with no credentials
- skopeo inspect --no-creds resolves an image and lists its tags
That last check is the one that matters: an outside developer with no account
can now pull, which was the functional blocker for publishing at all.
Plain-HTTP references become HTTPS in the same pass, so OTA downloads stop
crossing the network in the clear.
Deliberately NOT rewritten:
- The public FIPS anchor on port 8444. It is a functional network endpoint
every node dials to bootstrap the mesh — closer to Bitcoin Core's hardcoded
seeds than to leaked infrastructure. The domain does resolve to the same
host, so it could become a hostname, but that adds a DNS dependency to the
path used precisely when things are broken. Worth a deliberate decision,
not a side effect of this change.
- The companion APK on port 2100. The domain returns 404 for that path, so
rewriting it would swap a working URL for a broken one. The Releases page
does serve (200), which is where the plan already wants those binaries.
- releases/app-catalog.json, releases/manifest.json and release-manifest.json.
These carry `signature` and `signed_by`; editing their contents invalidates
the signature and the fleet refuses artifacts that fail verification. They
were rewritten in a first pass and reverted — they must be regenerated and
re-signed through the signing ceremony instead, which needs the mnemonic.
So the catalog still advertises the old host until that ceremony runs. Nodes
resolve images through the signed catalog, not the on-disk manifests, so this
commit alone does not change what a node pulls.
Verified: archipelago-container 75/75; every manifest still parses with a
top-level app block; no signed artifact modified.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keeps the dev and test tooling an outside contributor would want, and takes
our node addresses out of it.
Scripts that silently defaulted to one of our nodes now require an explicit
host and exit 2 without one: smoke-test.sh, trust-archipelago-cert.sh,
dev-container-test.sh (which also derives its RPC and health URLs from the
SSH target instead of a second hardcoded copy), and image-recipe/dev-branding.sh.
A default that points at a machine the user does not own is worse than no
default: it fails confusingly, or reaches a stranger's device.
Usage examples, mock data and test fixtures move to the RFC 5737
documentation range (192.0.2.0/24). CGNAT test values stay inside
100.64.0.0/10 so the range-check semantics they exercise still hold, and
192.168.1.0/.1/.254 are left alone — those are gateway logic and UI
placeholders, not our addresses.
Playwright and the perf spec defaulted their baseURL to one of our nodes;
they now default to localhost:8100, the local dev server.
Removed neode-ui APP_URLS entirely. It is dead code — exported, never
imported — and it pinned fedimint's *prod* launch URL to 192.168.1.228:8175.
Had anything consumed it, every user's node would have tried to reach an
address that on their LAN is either nothing or someone else's machine.
Deleting beats sanitizing dead config.
Verified: frontend 868/868 vitest across 108 files; archipelago-container
75/75; mesh tests 9/9; audit-secrets 5/5. Zero node addresses and zero node
names remain in tracked files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo is source code and guidelines only. Nothing about how Archipelago's
own fleet is run, or how the team works, stays in it.
Untracked (kept on disk, gitignored) — 250 files:
- .planning/ (199) and loop/ — internal development process
- fleet operations tooling that targets specific nodes: deploy-to-target,
deploy-tailscale, deploy-config-defaults, setup-target-dev, setup-aiui-server,
setup-https-dev, debug-frontend, node-profile, fleet-fips-pair/unpair,
image-recipe/sync-from-live.sh
- image-recipe/INTEGRATION-GUIDE.md and docs/multinode-testing-plan.md, both of
which are live-server workflow and fleet node inventories
- the Phase 10 on-node verification and evidence records, which cite .planning/
as their evidence base
KEY-05-ENTROPY-ENFORCEMENT.md was initially moved out with the other Phase 10
docs and then put back: it is cited as normative rationale from ten places in
the codebase, including core/clippy.toml, which bans rand::thread_rng and
points at it for the reason. That makes it a guideline, not an internal record.
Node names removed from source (48 occurrences across comments, manifests and
test fixtures): archi-dev-box, archy-x250*, shorty-s, framework-pt,
zaza-optiplex, archi-thinkpad. Comments keep the engineering context and the
date, which is what carried the meaning; the machine name did not.
Three of those were live test values rather than comments and were replaced
with valid stand-ins, not prose: two mDNS hostnames and a mesh peer name.
An earlier pass substituted "a test node" into a hostname assertion, producing
an invalid hostname; caught and fixed as test-node.local.
Wipe mechanism: .local-only/manifest.txt inventories every local-only path and
.local-only/wipe.sh deletes them on one confirmation, refusing to touch
anything git still tracks. Both are themselves untracked, so the public repo
does not carry a map of internal filenames.
Verified: cargo check -p archipelago --all-features clean; archipelago-container
75/75 tests pass; appOrigin vitest 7/7; audit-secrets 5/5; every relative link
in tracked markdown resolves (0 broken).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An app port must answer whatever the browser asks for: an HTTP dashboard
embeds http://host:PORT, an HTTPS one embeds https://host:PORT, and an HTTPS
page cannot embed an HTTP frame at all. So the choice is per-node, not
per-fleet, and a second port number would mean every manifest changes and
torrc doubles.
Instead the gate peeks the first byte. A TLS ClientHello is 0x16; no HTTP
method starts with it. peek() leaves the bytes in the socket buffer, so the
acceptor still sees a complete, untouched ClientHello. TLS and plain share one
generic serve_http(), so authentication, proxying and upgrade handling cannot
drift apart by scheme.
EXISTING NODES ARE UNAFFECTED BY CONSTRUCTION. Anything that is not a TLS
handshake takes the identical path as before, and a node with no certificate
serves plain HTTP exactly as today — TLS is strictly additive.
rustls does NOT verify that a private key matches its certificate. Established
by test, not assumed: with_single_cert accepted a pair from two different keys
and would only have failed mid-handshake in a user's browser — a security
control that reports success and does nothing, the exact shape this module's
own docs warn about. So the pairing is now proven explicitly (sign a fixed
message with the key, verify against the certificate's public key) and a
mismatch refuses to serve.
Also: cert and key mtimes are stamped as a PAIR, because reissuing writes them
separately and keying on one would serve a certificate that no longer matches
its key; a 15s first-byte timeout closes the slowloris window one step earlier
than the existing header-read timeout; PKCS#8 and PKCS#1 keys are both
accepted so a hand-made key does not silently downgrade a working node.
Deps pinned to the rustls 0.21 line reqwest already resolves — no new vendor,
no second rustls major. Test fixtures are throwaway (localhost SANs only), not
any node's identity.
38/38 appgate tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applying RF settings deliberately restarts the radio daemon (~15-20s).
Two things treated that healthy, expected gap as a fault (operator,
2026-08-06):
- radio_state was single-shot: a query landing inside the restart
window reported "The radio daemon did not answer the state query"
for a restart that was working correctly. It now retries for ~30s
and says the radio is restarting while it waits. A real device-level
refusal (not an RNode) still returns immediately.
- The device-setup modal auto-opens for any detected-but-unconnected
port, so the restart looked like a newly plugged stick and
interrupted the apply. Apply and Reboot now suppress auto-detect for
90s via mesh.suppressDeviceDetect().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying the .126 LoRa panel ahead of its daemon made every button
report "Operation failed. Check server logs for details." — the panel
was calling RPCs the older binary doesn't have, and the sanitizer
masked "Unknown method: mesh.rnode-config" into that generic string.
Read as "the feature is broken" rather than "this node needs its
update" (operator, 2026-08-06).
Allowlisted: "Unknown method" (a frontend newer than its daemon should
say so), every RNode RF validation message (each names the field and
its legal range — the entire point of validating before touching the
radio), and the actionable mesh preconditions (no device connected,
mesh service not running, MeshCore has no remote reboot, radio daemon
did not answer, RNode interface disabled).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The port map deferred to DISK manifests for any app with a build source
— which is exactly the four companion UIs (lnd-ui, bitcoin-ui,
electrs-ui, fips-ui). Their disk manifests reach nodes only via the
frontend runtime payload or a per-node repo checkout, and in the
v1.7.125 rollout both proved stale or entirely absent: one node had no
checkout at all, others restored an older payload over apps/ at every
boot. Result: session_passthrough never reached the gate, so the node's
own screens 401'd on every data call, and on nodes whose UI rebuilt
from a stale context the app held its port UNGATED.
Classification now uses a ports-only overlay that accepts build-source
manifests (install/orchestration still defers to disk — unchanged). The
signed catalog is the freshest, operator-signed source, and the gate's
address binds fail safely against a container publishing differently
(logged CANNOT PROTECT), so this can only tighten policy, never expose.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- mesh.rnode-config: persisted RF settings + best-effort live radio
state (radio-confirmed r_* values) for the LoRa panel.
- mesh.rnode-config-apply: validate → persist → restart the radio
daemon → poll the read-back until the radio reports online, returning
{applied, confirmed, live, message}. Failure modes report what
actually happened instead of pretending success.
- RebootRadio carries a reply channel: Meshtastic reboots firmware,
Reticulum restarts the sidecar (re-detect + reapply RF config),
MeshCore honestly reports it has no remote reboot — previously the
Reticulum/MeshCore arms returned Ok(()) doing NOTHING: the operator's
"button gives no feedback" bug.
- MeshCommand::QueryRadioState plumbs the sidecar's radio_state to the
service layer with a timeout instead of fire-and-forget.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The .126 LoRa panel's Rust half:
- mesh::rnode_settings: RNodeRfSettings persisted at
<data_dir>/rnode-rf-settings.json — every RNodeInterface parameter
(enabled, port override, frequency, bandwidth, sf, cr, txpower,
airtime_limit_short/long), validated against the bounds RNS itself
enforces. Defaults are byte-identical to the sidecar's historical
argparse defaults.
- FIRST-RUN ADOPTION (operator requirement: the update must change NO
device's applied settings): with no settings file yet, the node's
existing RNS config (~/.archy-reticulum, else ~/.reticulum) is parsed
and its RNodeInterface values adopted verbatim as the initial
settings — proven by a test carrying the operator's literal
"RNode LoRa Portugal" config.
- Serial spawns pass the settings as explicit sidecar args (frequency/
bandwidth/txpower/sf/cr + airtime locks); the operator port override
wins over auto-detect but still passes the KISS probe gate; a
disabled interface refuses to open with a readable error.
- ReticulumLink::query_radio_state(): asks the sidecar for the live
RNodeInterface state (radio-confirmed r_* values) — the panel's
apply-confirmation read-back source.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
chown_for_rootless_container prefers `podman unshare chown` (which maps
container uid N through the userns), but when that failed once it fell
back to `sudo chown -R <literal>` — writing e.g. host uid 999 for
container uid 999 and reporting success. Host-999 maps to nobody inside
the userns, so the app could not open its own data while everything
claimed the chown worked: botfights on framework-pt crash-looped every
10s on SQLITE_CANTOPEN over a data dir the daemon itself had just
"fixed".
The sudo fallback now translates container ids (1..99999) to
subuid_base + id - 1 (fleet base 100000; container root maps to the
service user, 1000). Already-mapped ids and uid 0 pass through.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A sweep of 92 unconfirmed/dust sats failed with LND's debug-flavored
"insufficient input to create sweep tx: input_sum=0 BTC, output_sum=
0.00000092 BTC" — and the RPC sanitizer then masked even that behind
"Operation failed. Check server logs." (framework-pt, 2026-08-06).
The sweep mechanics are untouched (balance minus fee, as always) —
this only makes the refusal say WHY in plain language.
- lnd.sendcoins translates the sweep refusal: balance below Bitcoin's
dust minimum or not yet confirmed, so no transaction can be built
(LND's original message kept in parens).
- "Failed to send" joins the sanitizer's user-facing allowlist — the
same lesson as "Insufficient balance"/"Payment failed" before it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The periodic reconcile runs ExistingOnly — merely listing a catalog
manifest must never install an app — and its only absent-container
recovery keyed on the last running-names snapshot, which ages out after
a few daemon restarts. An absent member of an installed stack then stays
absent forever: .38 ran indeedhub with no minio/postgres for days, nginx
down on 'host not found in upstream "minio"', and nothing ever put the
members back.
A live sibling container is proof the stack is installed on this node,
so an absent member is now treated as a hole to repair, not a choice to
respect: the recovery guard also fires when another member of the same
stack (app_ops::stack_member_app_ids) has a container in any state.
A stack with no containers at all is left untouched, and sibling app ids
resolve through the loaded-manifest container names (immich-postgres
runs as immich_postgres).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both predate this session's changes and were masked by the release
gate's cargo-test-weekly compile timeout:
- login_page_sources_its_art_from_the_gate still asserted the retired
wordmark (logo-archipelago.svg); the login page ships the sidebar A
mark (favico-black-v2.svg) since the 2026-08-05 rework.
- unauthenticated_ports_are_all_accounted_for lagged at 17; the
v1.7.123 port-policy round grew the rationale-carrying exempt set
to 25 (reviewed and enumerated in the test comment).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two daemon bugs, one debugging arc (2026-08-05, operator-reported):
1. The app gate removed the ENTIRE Cookie header before proxying. That
broke the data plane of every first-party companion UI behind the gate
(lnd-ui/bitcoin-ui/electrs-ui/fips-ui render their shell, then every
/proxy/* and /lnd-connect-info call 401s — observed as "LND UI
unreachable"), and silently logged users out of every gated app with
its own cookie login (vaultwarden, nextcloud, gitea) on each request.
The gate now strips only its own cookie pairs (session, csrf_token);
a new per-port manifest opt-in `session_passthrough: true` forwards
the node session to first-party UIs whose nginx proxies the daemon's
authenticated endpoints. Undeclared ports never get passthrough.
2. podman_client::create_container sent named volumes to the libpod API
as bind mounts with the bare volume name as source, so creating any
manifest app with a `type: volume` mount failed. On .38 the reconciler
removed indeedhub-postgres/-minio for env drift and then could never
create their replacements, leaving the stack half-missing forever.
Named volumes now ride the spec's `volumes` field ({Name, Dest,
Options}). Also: the reconcile-failure log now prints the full anyhow
chain — `%e` showed only "create_container X" and hid the real error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unbreaks Bitcoin on every node running the 1.7.124 catalog: the embedded
start script had a shell syntax error, so bitcoind never launched and the
app vanished. Delivered by catalog rather than a release because manifests
reach nodes through the signed catalog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
generate-app-catalog.py writes APP_LAUNCH_PORTS one entry per line; rustfmt
packs it. The release gate checks formatting, so the generated file has to
be formatted after regeneration or every catalog sync fails the gate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Portainer's image reaches the public catalog (the release gate caught the
manifest and catalog disagreeing), and fips-ui 8336 joins the mesh relay's
port list now that it declares a port — it is auth: gated, so the relay
withholds it rather than bridging it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pre-bumped so the release gate compiles the test profile at the final
version — create-release bumps after the gate, so the gate would otherwise
run on the old version and the bump would invalidate the cache, timing out
cargo-test-weekly on the compile rather than the tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
auth.session-policy.get/set plus a card under Account. Presented as two
plain questions rather than the token mechanism underneath, because the
distinction that matters to an operator is which control actually ends a
session: the dashboard polls constantly, so an idle timeout alone never
fires on an open tab — the absolute cap is what guarantees it.
Values are clamped server-side and the stored result is echoed back, so
the bounds are discoverable instead of an error. Presets rather than a free
number field: a box accepting '5' invites locking yourself out. A short
idle choice warns that it is the payments-industry posture.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
austin-sapien (100.70.96.88) sat dead for over two hours after taking
v1.7.122 — 'server starting' in the UI, service inactive, exit status
0/SUCCESS. It did not crash: the in-process updater replaces the binary and
exits cleanly for systemd to restart it, and that node's unit still carried
Restart=on-failure from an older install. systemd read the clean exit as
success and left it stopped. Every node with the old unit has this waiting
for it on the next update.
self-update.sh does refresh units, but the in-process update path never
runs it, so nothing was repairing them. The daemon now checks its own unit
at boot and rewrites only the Restart= line, so a node that starts even
once ends up with a policy that survives the next update.
Also carries the session-policy wiring: validate() now honours the
configured idle and absolute limits and the per-device class, instead of
the single hard-coded 24h constant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mesh right panel: a >=2560px screen hid the tab bar and stacked all five
tool panels in fixed grid rows. On a real display that clipped the Bitcoin,
Dead Man and AI headings to a few pixels each, letterboxed the map, and
pushed Radio Settings into a scroll — more screen producing a worse view.
Very wide now uses the same tabbed column as every other desktop width,
with the selected panel filling the column and the map running edge to edge
(it is the one panel with nothing to scroll).
Session policy: idle timeout, absolute cap and a re-prompt-for-funds flag,
persisted and clamped. Two tokens already existed — a session token and a
30-day login token — so the knob changes how long a quiet tab stays usable
without putting a long-lived credential on every request. Kiosk screens are
exempt from the idle timeout (nobody is there to log a TV back in) but keep
the absolute cap so a stolen box does not stay authenticated forever. The
cap is not optional theatre: idle alone never fires on a polling dashboard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Portainer: nodes have been running :latest — which is 2.39.1 — while the
manifest pinned 2.19.4 from two years ago. The port migration recreated the
container onto that old pin and Portainer refused to start: it migrates a
database forward, never backward, so an existing install died with 'schema
version does not align' and My Apps showed 'app is not responding'
(100.82.34.38). 2.39.1 published as an immutable tag and pinned forward, so
existing databases keep working and older ones migrate up.
Bitcoin: complements PR #131. That removes the code which kept writing a
datadir bitcoin.conf; -allowignoredconf=1 additionally makes an existing
one non-fatal, so a node already carrying the file recovers on restart
instead of crash-looping until something reinstalls it.
App gate login: rebuilt against the dashboard's own design — rotating
intro backgrounds served from the gate, the glass panel, the Archipelago
mark in its gradient ring, the app's icon as a My Apps tile, and the glass
button. Crucially it no longer sends X-Frame-Options: DENY, which made
every gated app render as unreachable inside My Apps' embedded frame;
frame-ancestors expresses 'only this node may frame me', which
X-Frame-Options cannot.
OTA origin: primary mirror is now source.archipelago-foundation.org over
TLS instead of a bare IP on plaintext. The IP stays as an automatic
fallback for nodes whose DNS or clock is broken — both break TLS, and the
signature, not the transport, is what establishes trust.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the Bitcoin crash-loop on 100.82.34.38: since a597c1d9
bitcoind launches with -conf=/tmp/rpc.conf and never reads the datadir
bitcoin.conf, but write_bitcoin_conf / ensure_bitcoin_rpc_config /
run_bitcoin_rpc_repair kept writing one on every install and restart.
Bitcoin Core's own datadir-conflict check then refuses to start at all.
Conflict resolved in favour of the PR: HEAD still carried
write_bitcoin_conf, whose deletion is the fix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mint failures surfaced raw JSON ({"detail":"proofs already spent"}) to
the user; now the top-level message is actionable while the raw body stays
in logs via {:#}.
A rebuilt image never reached a running companion. ensure_image_present
rebuilds in place under the same tag, so the quadlet body is identical,
write_if_changed reports no change, and enable_now is a no-op on a running
service — the container keeps the old layers indefinitely.
That is precisely how archi-dev-box kept serving the LND, FIPS, Electrs and
Guardian screens on 0.0.0.0 after v1.7.123 rebuilt every one of those images
to bind loopback: correct images on disk, three-day-old containers still
running. Closing those ports needed a manual 'podman rm -f' per container,
which no other node would ever get. Compare the running container's image ID
against the built one and restart when they diverge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scanning archi-dev-box from OUTSIDE found five ports serving their screens
with no login — lnd-ui 18083, bitcoin-ui 8334, fips-ui 8336, electrs-ui
50002 and the Fedimint Guardian 8175 — none of which appeared in the gate's
unprotected list. They are host-networked, so Podman publishes nothing to
pin and their manifests declared 'ports: []'; the gate builds its map from
declared ports, so it neither protected them nor reported them. An audit
that reports success while five screens are open is worse than no audit.
Their nginx now listens on 127.0.0.1 instead of 0.0.0.0, and each port is
declared 'auth: gated' so the daemon owns the outside. 'bind:' on a
host-networked app is a statement of where the container listens, not a
publish instruction — quadlet already skips PublishPort in host mode.
Guardian 8175 is declared on the fedimint app because its companion has no
manifest, and the gate keys on port, not container.
Credential paths were NOT exposed and are verified so: /lnd-connect-info,
the /proxy/lnd/ passthrough, container logs and every RPC method through
these screens all return 401 unauthenticated. What leaked was the page
shell.
Also fixes the delivery gap that would have made this unshippable: only
bitcoin-ui, lnd-ui and electrs-ui were ever rsynced to
/opt/archipelago/docker, so edits to fips-ui and fedimint-ui reached nodes
through no path at all. All five now sync; the two whose rebuilds the
daemon owns are synced without being handed to container-specs.
Every remaining undeclared port is now declared with a stated reason —
gated: botfights 9100, router 8084, pine 10380; exempt with rationale:
fedimint consensus 8173/8174, gateway 8176/9737, netbird 8086/8087 (TLS +
own auth, and enrolled devices cannot hold a session), pine TLS 10381,
lightning-stack REST 8091 (macaroon, mirrors lnd). Zero undeclared ports
remain across all 56 manifests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mint HTTP failures (swap/melt/mint-quote) were surfacing raw JSON bodies
like {"detail":"proofs already spent","code":11001} straight to the
user. Add a translator for the NUT-02/03/04/05 transaction-validation
error codes (10001-11017, 12001-12003; see
https://github.com/cashubtc/nuts/blob/main/error_codes.md) and layer it
onto the mint_client bail sites via anyhow context, so the top-level
message is actionable while the raw status/body stays available via
{:#} for logs. receive_token now surfaces the real reason (e.g. "This
ecash has already been redeemed") instead of a generic "Failed to
receive any proofs from token" when every mint in a token fails.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Leads with what changes for the operator: app screens now require the node
password across LAN, Tailscale, mesh and Tor; the wallet/protocol ports that
must stay open stayed open; the mesh leak found during on-node verification;
nodes repairing their own legacy containers; and the signing-key rotation.
Known gaps disclosed, including the eleven still-undeclared ports and that
non-browser clients will now meet the login page.
The new block uses <strong> rather than the literal ** markers in earlier
entries, which render as asterisks in the modal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DO NOT MERGE INTO A RELEASE SIGNED WITH THE NEW KEY. See below.
The previous release root (z6Mkkid…q7ur, pinned 2026-07-02) was exposed
in a chat transcript and is treated as compromised. It signs both OTA
manifests and the app catalog, so anyone holding it could sign updates
the fleet would install.
Pins the new key in trust::anchor and moves EXPECTED_DID in all three
signing/publishing scripts.
ORDERING IS CRITICAL — nodes pin the OLD key:
* The release CARRYING this commit must be signed with the OLD key.
That is the only signature a node running the previous binary will
accept, and it is what installs the binary pinning the new key.
* Only the release AFTER that may be signed with the new key.
* Signing this release with the new key makes every node reject it,
ending OTA fleet-wide and requiring hands-on recovery per node.
sign-catalog.sh moves in the same commit, so the app catalog must also be
re-signed with the new key once this ships, or nodes accept the binary
and reject the catalog.
Key verified before pinning: the hex and the did:key are the same
keypair, checked with a base58 decoder round-tripped against the previous
known-good pair. An earlier candidate hex (cb830e13…) was rejected
because it decoded to a different DID than the one supplied — pinning it
would have made every node reject every future update.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Caught verifying the gate fixes on archi-dev-box: [fips0-ULA]:32838
answered HTTP 200 straight from nbxplorer with no credential. The
catalog declares that port auth: local — host-local by intent, pinned to
loopback, the gate deliberately keeps its hands off — but the mesh relay
bridges a STATIC port list to 127.0.0.1, so it republished it to the
whole mesh. Same bug class as the Tor onion gap: a transport that
converges on the app loopback without consulting the declaration.
PortMap now records declared-local ports and the relay withholds them
(tearing down an existing bridge if a catalog refresh newly declares
one), alongside the declared-gated withhold. Undeclared ports keep
todays behaviour — silence is not an instruction in either direction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dev-box verification of the Tor/FIPS fixes caught a pre-existing split
brain: the orchestrator publishes containers from the signed catalog's
embedded manifests (origin-wins), but the gate classified ports from the
stale disk manifests — so it externally bound nbxplorer 32838, a port
the catalog declares auth: local and pins to loopback. Reachable behind
a login, but reachable where it deliberately was not.
- build_port_map now consults the catalog overlay first, via the same
parse/validate/image-only filter the orchestrator uses (moved to
app_catalog::catalog_manifest_overlay so the two cannot diverge again).
- GatedPort carries . The gated set still includes undeclared
Session-default ports for challenge/audit, but every action that
REDIRECTS traffic — the torrc 127.0.0.2 repoint, the FIPS relay
stand-down, the Tor-upstream bind — now keys on the declaration.
- The sweep releases held claims whose port left the gated set, so a
catalog refresh that withdraws a port (gated → local/none) takes
effect without a daemon restart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Legacy pre-quadlet containers kept publishing 0.0.0.0 after the catalog
pinned their app to loopback, because host_port_bindings_drifted only
compared host PORT numbers — closing them needed a manual package.update
per app per node. The drift check now also compares the bind ADDRESS,
but only when the manifest declares one: an empty bind never fires,
since recreating a loopback-published container to wildcard on silence
is exactly the v1.7.121 Bitcoin-RPC incident. With this, every node
recreates its legacy containers to the declared state on its own after
the OTA.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mesh relay is a raw unauthenticated forward to the app's loopback,
and whether it or the gate owned a fips0 ULA port was decided by a bind
race — the dev box happened to be safe because the gate bound first.
The relay now skips ports declared auth: gated and tears down any
existing bridge for a port that became gated since it was bridged
(catalog refresh), releasing the bind for the gate's next sweep.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tor carries no session cookie, so HiddenServicePort → 127.0.0.1:<port>
reached the app around the gate — the last transport the gate did not
cover. The gate now binds 127.0.0.2 (its own loopback, distinct from the
app's 127.0.0.1, so no app needs a second port), and regenerate_torrc
forwards declared-gated ports there. Undeclared ports keep today's
target: absence of the field is not an instruction.
The 127.0.0.2 claim deliberately does not count toward the unprotected
audit — a port whose only claim is the Tor loopback is still wide open
on the LAN and must keep warning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two live incidents on archi-dev-box today, one bug. Both times a safety
decision read an ABSENT manifest field as if it were a value, and a
node's installed manifests always lag the binary — so "absent" is the
state of essentially every port on every node.
1. Gating any `session` port regardless of `bind` published Bitcoin's
loopback-only RPC 8332 on the LAN, Tailscale and IPv6 within seconds
of deploy.
2. The `bind`-keyed replacement looked safe because it protected
`bind: 127.0.0.1` ports — but LND's gRPC 10009 and REST 18080 carry
an EMPTY bind, so they fell through. One container recreate from
pinning them to loopback and breaking Zeus and every remote wallet.
`auth` is now `Option<PortAuth>`, separating two questions that were
conflated:
* `auth_policy()` — what to CLASSIFY the port as. Undeclared reports as
Session, i.e. shows in the audit as something that should be behind
the gate. Reporting is always safe.
* `auth_is_declared()` — whether the daemon may ACT. Only an explicit
declaration authorises changing how a port is published.
Also reverts the daemon-side publish rewriting entirely. The node proved
it wrong twice over: the recreate path that actually ran was in
package::install, not podman_client, so the pin never fired; and even
`bind: 127.0.0.1` written directly into the node's manifest was
overridden by the signed catalog. Publishes are built in several places
and all of them already honour `bind`, so the migration belongs in the
catalog as data — not in daemon-side inference that can only ever cover
one path and guess wrong on the rest.
Tests: 75/75 container, incl. the LND wallet-port shape (`host: 10009`,
empty bind, no auth) asserted to be non-actionable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Peering a node was not enough to message it — you had to be in radio
range once before chat worked, which defeats the point of federating.
`send_message` chose its transport from the attached radio:
let use_typed_envelope =
archy && matches!(device_type, Meshcore | Reticulum);
Only the typed path knows about FIPS/Tor. Everything else fell through to
`peer_dest_prefix`, which resolves an over-the-air ROUTING key — so on a
node running Meshtastic, or with no radio at all, sending to a federated
peer failed. It only worked once a LoRa advert had created a radio twin
for the same archipelago identity, which is precisely the "connect on
LoRa first" the operator hit.
Federation contacts are reachable off-radio by definition — that is what
`upsert_federation_peer` records with `reachable: true` — so the
transport choice must not depend on which radio is plugged in. A
federation-synthetic contact id now always takes the typed path.
This loses no radio-first behaviour: `send_typed_wire` already prefers a
REACHABLE radio twin when the payload fits the frame, and only then falls
back to FIPS and Tor. The fix routes federation contacts INTO that logic
rather than around it.
Test pins the predicate across every device type, including the two that
failed (Meshtastic, Unknown), and asserts ordinary radio contacts and
stock clients still route exactly as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Observed live on archi-dev-box: archy-fedimint-ui rebuilt 45 times in 10
minutes, every ~35s, indefinitely. bitcoin-ui, lnd-ui and electrs-ui were
all one reconcile away from the same loop.
`context_is_newer_than_image` decides to rebuild when the build context's
newest mtime is later than `podman image inspect .Created`. The rebuild
that follows is a full layer-cache hit, so podman reuses the identical
image and leaves .Created untouched — the condition that triggered the
rebuild is still true afterwards. The check cannot converge: it rebuilds
on every reconcile tick forever, burning CPU and churning the container.
It bites after any deploy that refreshes /opt/archipelago/docker/*, which
makes the contexts newer than the shipped images — so this is fleet-wide
on every OTA, not local to one node.
Fix: stamp the context mtime that was built into an image label and
compare against that instead. A label is part of the image config, so a
cache-hit build with a new value still produces a new image — the thing
being tested does change, and the comparison settles after exactly one
rebuild. Verified against real podman before writing it: two cache-hit
builds with different label values produced distinct image IDs
(6cfdbc9bcd3e vs a9b10eb9a558), each carrying its stamp; the indexed
inspect format was checked against an image with real labels, and a
missing label prints empty (handled, along with "<no value>").
Images built before this carry no label and fall back to .Created, so
behaviour is unchanged for them and each self-heals on its first
reconcile after upgrade — nodes fix themselves rather than needing the
manual `podman build --no-cache` pass this needed by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both found while setting up the on-node test, and both fail silently in
the same direction — the gate reports success while protecting nothing,
which is the exact failure the module was written to prevent.
1. Loopback-pinned ports were skipped entirely.
`identity.rs` dropped any port whose manifest sets `bind: 127.0.0.1`,
reasoning that a loopback publish is not externally reachable. But
`listener.rs` requires loopback-pinning as the PRECONDITION for gating —
while an app holds 0.0.0.0:<port> the kernel will not let the gate bind
that port at all. So the two contradicted each other: pinning an app, the
one action that lets the gate take over, was also what removed it from
the gated set. Completing the entire migration would have gated nothing,
and GateStatus would have reported zero unprotected ports while doing it.
`bind` cannot carry this decision, because two unrelated intentions
produce an identical loopback publish: Bitcoin's RPC 8332 is pinned so
the LAN CANNOT reach it (fronting it would newly expose it on every host
address, behind a login but exposed where it deliberately was not),
whereas a migrated app is pinned precisely so the gate CAN. Inferring
from `bind` breaks one or the other, so the intent is now declared:
`PortAuth::Local` means the first case. The three ports that are
host-local by intent (bitcoin-core/knots 8332, aiui 5180 — all already
`bind: 127.0.0.1`) say so, and a loopback publish with `auth: session`
stays gated. A test pins that property.
2. The port map was never refreshed.
`AppGate::refresh()` existed, was documented as making catalog changes
apply without a restart, and was called by nothing. The map was built
once in `new()`, so an app installed while the daemon runs would never be
gated — and would never appear in `unprotected` either, so the node would
report itself fully enforced while serving a brand-new app to anyone who
asked. The sweep now refreshes before classifying.
Tests: 22/22 appgate, 73/73 archipelago-container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reproduced again on this node today: with no session cookie, six app
ports answered HTTP 200 with their real UIs (18083 LND, 8334, 8175
Fedimint Guardian, 8336 FIPS Mesh, 8090, 7777), all bound 0.0.0.0 and so
served on every host address. Same bug class as the /lnd-connect-info
and /bitcoin-rpc/ leaks closed in v1.7.120, but across every app.
LAN, Tailscale, Tor and the FIPS mesh all converge on 127.0.0.1:<port>,
so this is one gate rather than four. It lives in the daemon rather than
a per-app sidecar (umbrel's app_proxy model): rootless, no extra
container per app, and it can reuse machinery that already exists.
It invents no authentication policy. verify_password, TOTP secret
decryption, verify_code with used-step replay protection, the session
store, and — importantly — the SAME LoginRateLimiter instance as the
JSON-RPC path, so an attacker cannot get a fresh budget of password
guesses by moving to an app port. Only the transport differs, an HTML
form instead of JSON-RPC, because a browser being sent to an app cannot
speak JSON-RPC.
2FA comes for free: a session still pending its TOTP step fails
validate(), so the gate rejects it without knowing what a second factor
is.
Details worth keeping:
- 401, not a redirect. A redirect to a login page is indistinguishable
from the app itself redirecting, and machine clients would follow it
and parse HTML as their API response.
- Cookie and Authorization are stripped before proxying. The app has no
use for the node session and must never be able to log or forward it.
- The challenge page names and pictures the app being opened, so the
visitor can confirm what they are authenticating to.
- device_tokens grew `apps: Option<Vec<String>>` and verify_for_app for
machine clients. None = node-wide, which every existing companion
token is; migrating them by guessing a scope would silently revoke
access nobody asked to revoke. An empty list is rejected rather than
minted, since it reads as unrestricted while authorising nothing.
The rollout is necessarily per-app and the gate is built to say so. A
container publishing 0.0.0.0:<port> claims every host address, so the
gate cannot bind that port until the app is pinned to bind: 127.0.0.1
and recreated — gate-first is impossible, and all-at-once would recreate
every container on a node simultaneously. Every port it cannot claim is
logged at warn each sweep and recorded in GateStatus::unprotected,
surfaced by security.app-gate-status. The failure mode being designed
against is a gate that binds nothing, logs at debug, and reports success
while every app stays exactly as open as before — worse than no gate,
because it stops anyone looking. Same reasoning that ruled out an
nft drop-in, whose absence is a silent no-op.
Not yet done: pinning the 39 gated ports to loopback, repointing
HiddenServicePort at the gate, and on-node verification.
Tests: 21/21 appgate, workspace builds clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Groundwork for the app gate (item 1): before anything can enforce
authentication on app ports, the node has to know which ports are
*supposed* to be reachable without it.
`PortMapping` grows `auth` (PortAuth::Session | None, defaulting to
Session) and `auth_rationale`. The default is deliberately the protected
one. Every app port on this node answered with no credential at all over
LAN, Tailscale, Tor and the FIPS mesh alike — reproduced 2026-08-03 —
precisely because exposure was what you got by saying nothing. Inverting
the default means a new app is protected unless its manifest argues for
an exemption.
Validation makes the argument mandatory: `auth: none` without a
rationale is rejected, and so is a rationale without `auth: none` (that
combination means the author wrote an exemption and did not get one —
shipping it silently would leave them believing otherwise).
17 ports across 12 apps are declared exempt, each with its reason. They
are the ports that cannot sit behind an HTTP login page at all: Lightning
p2p (BOLT-8 noise), LND gRPC/REST and CLN gRPC (macaroon / mutual TLS —
Zeus and remote wallets dial these directly), Bitcoin p2p gossip,
electrum wire protocol, Wyoming voice streams, git-over-SSH, and the UDP
discovery protocols (mDNS, SSDP, STUN). Everything else — 39 published
ports — now defaults to gated.
Bitcoin's RPC 8332 is deliberately NOT exempted: it is already
`bind: 127.0.0.1`, so the gate never sees it, and claiming an exemption
it does not need would put a line in the audit list that means nothing.
If the loopback bind is ever dropped, it fails closed.
Two corpus tests keep this honest: every shipped manifest must parse
under the new rules, and the exempt set is pinned at 17 so any change to
the node's unauthenticated surface has to be a deliberate edit.
Tests: 73/73 archipelago-container, workspace builds clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
