Fresh installs pull *-ui images from the registry, so the source fix alone
left a newly flashed node serving the Bitcoin, LND, Electrs, FIPS and
Guardian screens with no login. All five rebuilt and pushed to
146.59.87.168:3000/lfg2025 as 1.7.123-alpha AND :latest — both tags,
because first-boot resolves the pinned tag from image-versions.sh while the
daemon's companion installer hardcodes :latest, and a stale :latest would
have quietly undone the fix on exactly the path that rebuilds companions.
Verified by pulling each image back from the registry anonymously and
reading /etc/nginx/conf.d/default.conf inside it — a private package would
make fresh nodes fall back to a stale local image without saying so.
Also fixes the FOURTH copy of bitcoin-ui's listen directive
(scripts/reconcile-containers.sh wrote 'listen 8334' into the rendered
nginx.conf on every reconcile, which would have re-opened the port after
the image and template were both corrected).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scanning archi-dev-box from OUTSIDE found five ports serving their screens
with no login — lnd-ui 18083, bitcoin-ui 8334, fips-ui 8336, electrs-ui
50002 and the Fedimint Guardian 8175 — none of which appeared in the gate's
unprotected list. They are host-networked, so Podman publishes nothing to
pin and their manifests declared 'ports: []'; the gate builds its map from
declared ports, so it neither protected them nor reported them. An audit
that reports success while five screens are open is worse than no audit.
Their nginx now listens on 127.0.0.1 instead of 0.0.0.0, and each port is
declared 'auth: gated' so the daemon owns the outside. 'bind:' on a
host-networked app is a statement of where the container listens, not a
publish instruction — quadlet already skips PublishPort in host mode.
Guardian 8175 is declared on the fedimint app because its companion has no
manifest, and the gate keys on port, not container.
Credential paths were NOT exposed and are verified so: /lnd-connect-info,
the /proxy/lnd/ passthrough, container logs and every RPC method through
these screens all return 401 unauthenticated. What leaked was the page
shell.
Also fixes the delivery gap that would have made this unshippable: only
bitcoin-ui, lnd-ui and electrs-ui were ever rsynced to
/opt/archipelago/docker, so edits to fips-ui and fedimint-ui reached nodes
through no path at all. All five now sync; the two whose rebuilds the
daemon owns are synced without being handed to container-specs.
Every remaining undeclared port is now declared with a stated reason —
gated: botfights 9100, router 8084, pine 10380; exempt with rationale:
fedimint consensus 8173/8174, gateway 8176/9737, netbird 8086/8087 (TLS +
own auth, and enrolled devices cannot hold a session), pine TLS 10381,
lightning-stack REST 8091 (macaroon, mirrors lnd). Zero undeclared ports
remain across all 56 manifests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rotation commit pointed create-release.sh and publish-release-assets.sh
at the NEW root in the same commit that pins it in the binary. But the
release CARRYING the rotation must be signed with the OLD root: every node
is still running the previous binary, which pins the old key. So the
tooling would have rejected the only signature the fleet can accept, and
the signature it demanded would have ended OTA fleet-wide.
Both checks now expect the old DID for this cycle, with the flip to the new
one called out for v1.7.123+. sign-manifest.sh documents the
ARCHY_RELEASE_ROOT_PUBKEY override needed because the signer built from
this tree already pins the new anchor and would fail to verify its own
correct output.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DO NOT MERGE INTO A RELEASE SIGNED WITH THE NEW KEY. See below.
The previous release root (z6Mkkid…q7ur, pinned 2026-07-02) was exposed
in a chat transcript and is treated as compromised. It signs both OTA
manifests and the app catalog, so anyone holding it could sign updates
the fleet would install.
Pins the new key in trust::anchor and moves EXPECTED_DID in all three
signing/publishing scripts.
ORDERING IS CRITICAL — nodes pin the OLD key:
* The release CARRYING this commit must be signed with the OLD key.
That is the only signature a node running the previous binary will
accept, and it is what installs the binary pinning the new key.
* Only the release AFTER that may be signed with the new key.
* Signing this release with the new key makes every node reject it,
ending OTA fleet-wide and requiring hands-on recovery per node.
sign-catalog.sh moves in the same commit, so the app catalog must also be
re-signed with the new key once this ships, or nodes accept the binary
and reject the catalog.
Key verified before pinning: the hex and the did:key are the same
keypair, checked with a base58 decoder round-tripped against the previous
known-good pair. An earlier candidate hex (cb830e13…) was rejected
because it decoded to a different DID than the one supplied — pinning it
would have made every node reject every future update.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Publishing v1.7.121-alpha failed on auth after the manifest had already
passed every check. The script required an `http://user:token@` remote,
which left only `gitea-vps2` — whose token is dead — and rejected
`gitea-ai`, the https remote whose credential actually works for git
push. Same Gitea instance (146.59.87.168, v1.27.1) either way, so the
restriction bought nothing and blocked the one usable path.
Accepts http and https, and carries the scheme through to the API URL
instead of hardcoding it.
Note for diagnosis next time: `/api/v1/repos/.../releases` is publicly
readable, so a 200 there does NOT prove the credential works. Use
`/api/v1/user`, which requires real auth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every cycle has needed a manual check that releases/manifest.json got
signed, because the script would happily commit and tag one that hadn't.
The signing step is conditional: with no TTY and no
RELEASE_MASTER_MNEMONIC it prints a warning and falls through. The commit
at step 7 then ran regardless, so the release commit — and its tag —
carried an unsigned manifest.
publish-release-assets.sh already refuses to ship one, but that backstop
arrives a step too late. Nodes fetch releases/manifest.json straight from
branch `main` (the same URLs this script prints for verification), so the
COMMIT is what exposes it to the fleet, not the publish. By the time
publishing is refused, the unsigned manifest is already on main and nodes
are already declining to auto-apply.
So the same gate now runs before the commit: presence of a signature,
signed_by matching the release root, and `ceremony verify` for the crypto.
A release commit carrying a manifest no node will accept has no valid use,
so this refuses to create one rather than leave a tag that has to be
re-cut. The earlier warning is corrected too — it promised the run would
continue, which is no longer true.
Verified the predicate against three manifests: signed -> allow, signature
stripped -> refuse, signed_by swapped to another DID -> refuse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reported: Framework PT took the OTA and got the new bitcoin-ui but not
lnd-ui. Two causes, both in the update path rather than the app.
1. LND_UI_IMAGE was "lnd-ui:latest" while BITCOIN_UI_IMAGE was pinned to
1.7.119-alpha. Podman does not re-pull a tag it already holds locally,
so a node that ever pulled lnd-ui:latest keeps that copy forever and
every subsequent release silently no-ops. Pinned to 1.7.119-alpha, so
a version change is what triggers the pull — the same mechanism that
made bitcoin-ui update correctly.
2. first-boot-containers.sh declared lnd-ui as bridge with -p 18083:80.
docker/lnd-ui/nginx.conf listens on 18083 DIRECTLY (it must, to proxy
the backend on 127.0.0.1:5678 same-origin), so that maps a host port
onto a container port nothing serves — reproduced on-node as HTTP 000.
This is the THIRD copy of the same declaration: container-specs.sh and
apps/lnd-ui/manifest.yml were both already corrected, this one was
missed, and it is the copy fresh installs use. Now host-networked with
no published ports, matching its siblings and the other two copies.
The underlying hazard is that one container spec lives in three files
that can disagree; recorded as a follow-up rather than refactored here.
Also opens .planning/RELEASE-1.7.121-TASKS.md — every outstanding item
for the next release with its evidence, so nothing in a fast-moving queue
gets lost between sessions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things needed for the new UI to actually reach users.
The rendered nginx.conf served index.html with only ETag/Last-Modified and
no Cache-Control, so browsers applied heuristic caching to it. Confirmed on
archi-dev-box: after rebuilding and recreating the container, :8334 and
/app/bitcoin-ui/ both served the new markup immediately, but the app iframe
in the main UI kept showing the previous UI until a hard refresh.
docker/lnd-ui/nginx.conf has always carried this header, which is why only
bitcoin-ui showed the stale copy. Using "no-cache" (revalidate) rather than
"no-store" keeps the ETag doing its job when nothing has changed.
Validated by mounting the rendered config into a throwaway container from
the built image and running nginx -t. (An earlier attempt to test it inside
the running container was meaningless — conf.d/default.conf is a read-only
bind mount, so the copy failed and nginx -t just re-checked the original.)
The 8 container::bitcoin_ui tests still pass; their assertions cover the
placeholder, the 8332 proxy_pass and the listen directive, none of which
this touches.
BITCOIN_UI_IMAGE was still pinned to 1.7.84-alpha, so a fresh install would
pull a bitcoin-ui from many releases ago regardless of what the OTA ships —
first-boot-containers.sh tries the registry image before building from
source. Bumped to 1.7.119-alpha, matching the current release, and the
image is pushed under that tag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All four found by verifying on archi-dev-box rather than assuming.
container-specs.sh: archy-lnd-ui was specified as a BRIDGE container with
SPEC_PORTS="18083:80", but docker/lnd-ui/nginx.conf listens on 18083
directly (it must, to proxy the backend on 127.0.0.1:5678 same-origin).
Recreating from that spec publishes host 18083 to container port 80, where
nothing listens. Reproduced on the node: the app came back with :18083
refusing connections, HTTP 000. This never fired before because the running
containers are created by first-boot-containers.sh, which is host-networked
and never reads this file; the spec is only consulted when self-update.sh
rebuilds a UI image, and that only happens when a file under docker/lnd-ui/
changes — which is exactly what the previous two commits did. So the next
OTA would have taken lnd-ui down on every node. Now SPEC_NETWORK="host"
with no port mapping, matching what actually runs. NET_BIND_SERVICE dropped
with it: 18083 is unprivileged.
lnd-ui channels link: pointed at /apps/lnd/channels, but that route is a
CHILD of the /dashboard record in neode-ui's router, so the real path is
/dashboard/apps/lnd/channels. nginx's SPA fallback returns 200 for the
wrong path, so it failed as vue-router's NotFound view rather than an HTTP
404 — both the Payment Channels card and the Manage Channels button.
Both apps, copy buttons: navigator.clipboard only exists in a secure
context, and nodes serve these apps over plain http; the main UI also
embeds them in an iframe, where the async Clipboard API is separately gated
by the clipboard-write permission policy. Every copy button silently did
nothing there. Added an execCommand('copy') fallback behind a copyText()
helper and routed all six call sites through it.
lnd-ui Node ID: showed the bare pubkey whenever getinfo.uris was empty,
which is the common case — LND only populates uris once it is advertising
an external address. The bare pubkey is not what a peer pastes to open a
channel. The full pubkey@host:9735 URI is now built from the Tor onion
where available, falling back to this node's address, with a hint saying
which and what its reachability is. The QR encodes the URI too.
Verified on archi-dev-box: both images rebuilt and containers recreated
from the specs; lnd-ui and bitcoin-ui both serve 200 with the new assets;
and the RPCs the new tabs depend on all answer on the live node —
getblockstats returns every field the charts read, getpeerinfo returns 11
peers carrying relaytxes and network values the classifier handles.
Note for whoever tests bitcoin-ui's Insights/Peers tabs: /bitcoin-rpc/ now
sits behind auth_request /_session_check (a05956c4 et al), so it answers
401 to an unauthenticated curl by design. A logged-in browser sends the
session cookie same-origin, which is how those tabs get their data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Operator tool for the fix in a05956c4. GET /lnd-connect-info answered
unauthenticated callers with the LND ADMIN MACAROON, the TLS cert, the
gRPC/REST ports and the node's onion address. Every macaroon on an
affected node must be treated as known to an attacker.
Rotation removes the macaroon root key from macaroons.db plus the issued
macaroon files; LND mints a fresh root key on unlock, so every
previously issued macaroon — including any the attacker holds — stops
verifying. Coins live in wallet.db and channel state in channel.db, and
macaroons are bearer tokens rather than keys, so neither database is
touched, opened or moved.
Safety properties, in the order they matter:
- Detect-only by DEFAULT. --apply additionally requires --yes.
- An ORDERING GUARD refuses to rotate on a node whose binary lacks the
fix, because the new macaroon would leak through the same door within
seconds and the operator would believe they were safe. Overridable
only via an explicit --force-unpatched.
- It backs up the old material to a 0700 dir OUTSIDE the dir LND
rescans, verifies the backup file count matches, and refuses to delete
anything if it does not.
- It never reads, prints or copies a macaroon's CONTENT. Everything it
reports is a SHA-256 digest, so the material is proven changed without
disclosing it to the terminal or scrollback. lncli runs INSIDE the
container and reads the macaroon off its own disk, so the secret never
crosses into this script's output.
- It records the node identity pubkey and channel census BEFORE, and
aborts loudly if the identity or the OPEN channel total changed.
Two deliberate non-assertions, both of which would otherwise produce
frightening false alarms on a completely healthy rotation:
- wallet.db is NOT asserted byte-identical. btcwallet records chain sync
progress inside it, so it legitimately changes on every start.
- num_active_channels alone is NOT asserted. It counts channels whose
peer is currently online and so legitimately dips after any restart
while peers reconnect. The safety invariant is the ACTIVE+INACTIVE
total, which is what gets asserted; a changed active count is reported
as normal post-restart behaviour.
Enumeration runs under sudo rather than as a shell glob: the LND data
dir is 0700 owned by the container's mapped uid, so "$LND_DIR"/*.macaroon
does not expand in an unprivileged shell — it stays literal, which would
have made both the backup and the removal loop silent no-ops. Verified
on a live node: sudo find returns 9 files where the glob returns 0.
Exercised in detect mode against a real LND on a live node: correctly
reported the binary as unpatched, live-probed :18083 as LEAKING (200),
read both digests and the node identity, and exited 2 without changing
anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
[Rule 3 — blocking] bootstrap.rs installs systemd units from the runtime
payload at image-recipe/configs/, but create-release-manifest.sh copies only
archipelago-doctor.service and .timer into that directory. The new
archipelago-host-secrets-audit.service would therefore never exist on any
node: bootstrap looks for it, `src.exists()` is false, and it silently
installs nothing. No error, no log line — the whole deployed-node half of
10-04 would have been inert on arrival.
Two enumerations of the same list in two languages in two files is the drift
that caused it, so the loop now carries a KEEP IN SYNC pointer naming the
array in bootstrap.rs, and the redundant `if [ -f doctor.service ] || [ -f
doctor.timer ]` wrapper is gone — the per-unit `-f` test inside the loop
already does that job, and the wrapper would have skipped the whole block on
a tree that had the new unit but not the doctor ones.
Outside 10-04's declared files_modified. Taken because the alternative was to
ship a deliverable that cannot reach its target and file the gap as a
follow-up. Staged by path; no other agent had uncommitted work in this file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
10-03 closed the build half of F-03: the ISO no longer bakes SSH host keys or
a TLS keypair into the shared rootfs, and first-boot regeneration fails closed.
Nodes already in the field receive none of that — the first-boot script is
installed by the installer, not shipped by OTA — so a node that hit the old
fail-open path is still running key material that every downloader of its ISO
also holds, and its completion marker guarantees it will never try again.
scripts/security/host-secrets-audit.sh decides, from the node's own disk alone,
which of those it is. Four signals in a fixed precedence: missing material can
never be shared material; the fail-open fingerprint (marker present plus the
literal `WARNING: TLS regeneration failed` / `WARNING: ssh-keygen -A failed`
lines the old script emitted) is direct evidence and outranks timestamps and
also names WHICH class survived; then key mtime against a first-boot anchor
(.secrets-regenerated, falling back to the installer's LUKS key then
machine-id). Verdicts are per-node / shared / fail-closed-missing / unknown,
and every one of them carries the evidence strings that produced it, each
naming the file it was read from.
per-node is never claimed from an absent signal. No anchor means `unknown`, and
a standing first-boot-secrets.failed record also means `unknown` — a clean
mtime is not evidence that generation succeeded. That is T-10-37: a false
per-node verdict leaves an exposed node looking clean, which is worse than no
verdict at all.
Rotation (D-06: detect-report-then-apply, recorded in
docs/security/KEY-02-FLEET-ROTATION.md):
- --detect is the default and is read-only; it always exits 0, because
detection is informational and must never fail a boot.
- --apply without --yes writes nothing at all, not even its own verdict file.
"Touches nothing" is worth being able to say without a footnote.
- --apply --yes refuses unless the verdict is `shared`, so the wrong node
cannot be rotated even deliberately.
- It stages the full replacement TLS pair AND host-key set before touching
anything live and aborts if either fails; records the OLD fingerprints
before the swap; does TLS first (a dead web UI is recoverable over SSH, the
converse is not); replaces host keys by mv-onto-the-existing-path rather
than rm-then-mv, so the directory is never momentarily empty; and RELOADS
sshd, never restarts it, so the operator's own session survives its own
rotation.
bootstrap.rs ships the boot unit through the existing run_runtime_assets
promotion and enables it --now, so the verdict lands with the OTA rather than
at the next reboot. handle_system_stats gains a host_secrets object read from
the on-disk verdict — cheap, never an error however malformed the file, and
deliberately carrying no fingerprints, because a payload polled every few
seconds does not need digests an operator on the node can already read.
tests/first-boot-secrets/rotation-tests.sh: 8 cases against temp roots through
the HOST_SECRETS_ROOT seam. Negative controls run and reverted, each reddening
exactly one case: dry run writing its verdict file (STATE-DIR-CHANGED); the
old fingerprints recorded after the swap instead of before (caught by an
ordering observation, not a content comparison — the systemctl stub records
whether the file existed at the moment of the first reload); a tolerated
generation failure leaving a half-rotated node; and `per-node` claimed with no
anchor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Measures EXPOSURE (auth.isOnboardingComplete) and SESSION ENFORCEMENT
(seed.status) separately; the audit's C-6 probed with seed.status alone,
which is not allowlisted and returns 401 by design, so its "Fail: 200"
criterion could never fire.
- Read-only by construction: methods come from a fixed READONLY_METHODS
array, never from an argument; the one mutating request is behind
--destructive with a red disposable-nodes-only banner.
- The refusal check uses the published BIP-39 all-abandon/art test vector,
so no real key material is ever generated, handled or printed.
- No node address, onion address or credential embedded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six code paths configured the Lightning gateway with a bcrypt hash committed
to this repository — and one deploy path with a plaintext password literal —
whenever the per-install secret was missing. Anyone holding a copy of the repo
held the admin credential for every gateway that ever took a fallback.
container::secrets now owns the credential end to end: ensure_gateway_credential
(idempotent, delegates to ensure_one's bcrypt arm) and gateway_bcrypt_hash,
which returns Err when the secret is missing/empty and when the stored value is
on the KNOWN_DEFAULT_GATEWAY_HASHES denylist — so this codebase cannot hand
back the compromised value even to a node already carrying it.
get_app_config was widened to Result so a credential-less install cannot reach
podman run at all; configure_fedimint_lnd takes the resolved hash instead of
re-reading with its own fallback. The four shell paths stop generating
credentials entirely (dropping the htpasswd host dependency) and skip container
creation with a printed reason rather than substituting anything.
Naming converges on the manifest's fedimint-gateway-hash/.pw, with legacy
fedimint-gateway-password values copied forward rather than regenerated so no
node loses a working unique credential. Plan 01-16 owns rotation of installs
already carrying the default.
Verified: cargo build clean; cargo test -p archipelago 999 passed (2
boot_reconciler timing tests failed under concurrent load, green in isolation,
untouched by this diff); bash -n clean on all five scripts; the compromised
literal now appears exactly once in the tree, as the denylist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ships botfight repo commits f5f57e6, c162d5e, 41f1b93, 10d4209:
- Fixed 6 instances of the same trust-a-client-supplied-pubkey pattern
across auth/payments/queue routes, two of them critical: GET
/winnings/:botId had no auth at all and leaked live spendable Cashu
bearer tokens; POST /connect-wallet let anyone redirect a victim
bot's future payouts to an attacker's wallet by pubkey (public by
design in nostr).
- Added owner-reachable AI-answer settings (existing bots, not just at
creation) and a "claim your winnings" UI (Cashu payouts were minted
server-side but had no frontend consumer at all until now).
Co-Authored-By: Claude <noreply@anthropic.com>
Bumps to botfights:1.2.9, which carries: Cashu token payment wired as the
primary entry-fee UX in WalletConnect.vue (was built server-side already
but never called from any UI), Lightning/NWC demoted to secondary, and a
fix so anonymous poll-mode bots (not just nostr-authenticated humans) can
use ranked/staked fights — join-ranked previously required a pubkey
unconditionally, silently locking out the entire AI-agent audience.
No archy-side manifest changes beyond the version/image bump.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix_tor_permissions() exact-matched stat's output against the literal
string "700", but stat -c '%a' omits leading zeros so Tor's own
setgid HiddenServiceDir mode (2700) never matched. Every ~5-minute
doctor run "fixed" it back to 700, restarted Tor, and Tor immediately
set 2700 again — so Tor never survived long enough to build a usable
consensus/HSDir cache, breaking the mesh's Tor fallback entirely.
- Compare only the last 3 mode digits (owner/group/other), which is
the property that actually matters, so 700 and 2700 both pass while
750/707/2755 etc. are still corrected.
- Add a 30-minute restart backoff (timestamp file under
/var/lib/archipelago/) so no future condition can reproduce a
restart storm even if the fix fires repeatedly.
- Log clearly in both directions: a debug-level no-op line when a
directory is already correct, and an explicit "was NOT fully
denied" line when a real fix is applied, plus a line when a restart
is skipped by the backoff.
Verified statically against temp directories (2700 accepted with 0
restarts; 750/707/2755 corrected with exactly 1 restart; a second
real fix inside the backoff window logs a skip instead of
restarting). No live Tor/doctor/systemd unit was touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps to botfights:1.2.8, which carries: the CSP img-src fix (nostr profile
pictures come from user-supplied kind:0 metadata URLs on arbitrary domains
— img-src was locked to 'self'/data:/blob: with no https:, so every
external profile picture rendered as a broken image), and discoverability
fixes for the new AI-answer feature (poll mode is now the default
connection mode, matching BOTFIGHTS.md's own documented default, and the
AI-answer section is expanded by default instead of collapsed behind an
extra click).
Canonical arena on VPS2 already rolled to 1.2.8 directly (docker compose
pull/up) ahead of this catalog publish, since it's a separate deployment
from the archy-catalog-driven per-node install path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bumps to botfights:1.2.7, which carries the botfight repo's new
"let BotFights answer for me" feature (ca5b634) — an operator can paste an
Anthropic or OpenAI API key so this node's BotFights server answers fight
challenges automatically for a poll-mode bot, no external script needed.
Storage/security follows the same pattern this node already uses for its
own AIUI Anthropic key (system.settings.set "claude_api_key" in
core/archipelago/src/api/rpc/system/handlers.rs): 0600 file, never echoed
back. Also carries the nostr-provider.js 404 fix, DocsPage/round-jump
fixes, and the HomePage "Latest Bouts" short-viewport visibility fix from
the prior 1.2.4-1.2.6 iterations that were built and tested locally but
not yet pushed through the signed-catalog path.
No archy-side manifest changes beyond the version/image bump — 1.2.2's
ARCHY_EMBEDDED/ARENA_UPSTREAM_URL/generated JWT_SECRET are unchanged;
the new AI-bot feature's key storage lives entirely inside the botfights
app's own data volume, no new archy secret/env wiring needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
deploy-to-target.sh rsyncs with --delete to TARGET_DIR=/home/archipelago/archy,
which is a symlink to /home/archipelago/Projects/archy. archi-dev-box is this
same machine over loopback SSH, so deploying from the main checkout is a no-op
(source and destination resolve identically) — but deploying from a git worktree
nested under it mirrored that worktree onto the main checkout and deleted
everything else: ~1810 tracked files, the deploying worktree itself mid-run, a
running dev server, and two concurrent sessions' uncommitted work.
Guard compares /etc/machine-id across the SSH hop and, when source and
destination are on the same host, refuses if either path contains the other.
Identical paths still pass, so normal local deploys are unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps to botfights:1.2.3, which carries the JoinBoutPage.vue fix from the
botfight repo (commit 603e09b): the poll/webhook mode picker on the bot
setup step now visibly reacts when clicked — a colored banner in the guide
viewer and a prepended line in the copied prompt point at "Option A:
Polling Bot" or "Option B: Webhook Bot" within the single unified doc
(BOT-02), instead of silently refetching the same file with no visible
change. No manifest/env changes beyond the version/image bump — 1.2.2's
ARCHY_EMBEDDED/ARENA_UPSTREAM_URL/generated JWT_SECRET are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1.2.x's auth-hardening work added Hono secureHeaders() with a default
X-Frame-Options: SAMEORIGIN, which unconditionally blocked the Archipelago
node dashboard's iframe (different origin by port) — a real regression
versus 1.1.0, which never sent this header. Fixed upstream in the botfight
repo (commit 8eb27ed): X-Frame-Options is now conditional on ARCHY_EMBEDDED,
disabled only for the first-party node-embedded instance.
apps/botfights/manifest.yml: image/version -> 1.2.2, adds
ARCHY_EMBEDDED=1 to environment, drops the interim
metadata.launch.open_in_new_tab workaround (no longer needed — the app can
now be framed). app-catalog/catalog.json, scripts/image-versions.sh,
neode-ui/public/catalog.json bumped in lockstep via
scripts/generate-app-catalog.py. core/archipelago/src/fips/app_ports.rs
regenerated (formatting only, same port set).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
apps/botfights/manifest.yml: image tag 1.1.0 -> 1.2.1, adds container.generated_secrets
(botfights-jwt-secret, kind hex32) + secret_env (JWT_SECRET) so a fresh install no longer
crash-loops on the 1.2.x image's jwt.ts import-time throw when JWT_SECRET is unset under
NODE_ENV=production (D-04/BOT-04). Adds ARENA_UPSTREAM_URL=https://botfights.archipelago-foundation.org
to turn on shared public arena federation by default (BOT-03/D-03), with an in-manifest comment
documenting the per-node opt-out (remove the line to run standalone).
app-catalog/catalog.json + scripts/image-versions.sh bumped in lockstep to keep the legacy
catalog and the image-version drift checker consistent with the manifest.
releases/app-catalog.json intentionally NOT committed in this change — it has been
regenerated locally (scripts/generate-app-catalog.sh) with the new manifest embedded, but is
unsigned. It is committed separately (09-06 Task 3) only after the release-root signing
ceremony, so no unsigned catalog state ever lands even transiently on a branch this repo's
mirrors could serve to nodes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v1.7.117 broke Reticulum mesh on OTA-only fleet nodes two ways: the
update swaps only the backend binary and frontend tarball, so nodes
kept a stale archy-reticulum-daemon whose argparse exits on the new
--enable-transport flag (mesh session died on every spawn — confirmed
on framework-pt), and they never had archy-rnodeconf at all, so the
in-app Flash LoRa flow failed with a bare "No such file or directory".
Four-part fix:
- The Rust supervisor probes `daemon --help` and only passes
--enable-transport when the daemon advertises it; unsupported daemons
run edge-only exactly as pre-1.7.117 (tested against stub daemons
both ways + missing-binary fail-safe).
- Both PyInstaller tools ride the frontend tarball's runtime payload
(radio-tools/) and bootstrap.rs promotes them to /usr/local/bin on
startup when bytes differ — the first OTA path that ever updates
them. create-release.sh now rebuilds them every release and the
manifest script hard-fails if they're missing.
- The ISO bundles archy-rnodeconf alongside the daemon (it never did).
- Flash LoRa reports "tool not installed — update the node" instead of
the bare spawn error when rnodeconf is absent everywhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both sides independently fixed the serial-alias dedup and the ESP32
boot-reset races; kept the branch's defer-to-auto-detect for unpinned
preferred paths (single probe pass per cycle) on top of main's
advert-name threading, Reticulum name propagation and radio-first
routing. Modal keeps main's 'Set Recommended' naming + probe progress
bar alongside the branch's in-app firmware flasher step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed OTA manifest (release root verified). Also: create-release-manifest
tarball perms check made SIGPIPE-proof so releases cut on macOS too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The companion opens catalog apps by direct port over the mesh; the
fips0 default-deny baseline blocked every one of them (apps 'stuck'
from the phone, 2026-07-26). Core now writes a second fips.d drop-in
from the generated catalog launch-port list on every install/upgrade.
Service/RPC ports (bitcoind 8332, LND 10009, Tor) stay closed.
generate-app-catalog.py emits the Rust port list alongside the TS one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- scripts/build-iso-release.sh: single gated command from signed release
to tested ISO (preflight version/signature parity, release gate harness,
strict catalog drift, full Rust suite, artifact version checks, build,
mount smoke, best-effort QEMU boot)
- scripts/iso-smoke-test.sh: standalone mount-level ISO verification incl.
stale-binary version assertion inside the payload
- .gitea/workflows/build-iso.yml: resurrected ISO CI as dispatch-only job
calling the gated orchestrator (old one was stranded in _archived/)
- tests/release/run.sh: catalog drift now runs --strict (was silently
always-pass)
- test-iso-qemu.sh: headless mode used -append without -kernel, which
QEMU rejects; use -display none so -serial file: capture works
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--erase-all hit the exact same "ROM does not support function erase_flash"
error as a standalone erase_flash — confirmed live: esptool's --erase-all
is implemented as the same full-chip-erase command, not a per-sector loop,
so it needs the stub just as much. The real fix isn't finding another way
around the ROM bootloader's limitations — it's restoring the stub Debian's
package is missing.
Fetched the exact missing file (stub_flasher_32s3.json) from the matching
upstream esptool release tag and installed it directly on archy-x250-dev;
verified live with a read-only `flash_id` that stub mode now loads
("Uploading stub... Running stub..."). Removed --no-stub from flash.rs now
that normal stub-loader mode works correctly (faster and fully-featured vs.
the ROM-only fallback). scripts/self-update.sh now fetches and installs
this same file automatically whenever it's missing, so this isn't a
one-off manual fix tied to a single node.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pine-openwakeword manifest (wyoming-openwakeword 2.1.0, :10400, /custom model
dir for the future Yo Archy model). Pine 1.3.0: launcher page gains a live
node-status card fed by /api/pine/status via a same-origin nginx proxy, and
copy for the new intents / Claude fallback / mesh announcements. Catalogs,
app-session config, drift ids regenerated; release catalog embeds 56
manifests (unsigned until the ceremony). Framework PT test plan in docs/.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HA 2024.1's Wyoming integration predates ping/pong keepalives. The PineVoice
satellite firmware (wyoming 1.7.2) drops connections idle for ~8s, so HA
looped on 'Satellite disconnected' forever and the speaker never worked.
HA 2026.7.3 pings satellites and holds the connection (verified on
framework-pt: link stays ESTABLISHED, wake word reaches Assist).
Catalog regenerated with the bump — needs the signing ceremony before the
origin publish.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Package the PineVoice/Wyoming local voice assistant as ONE store app "Pine"
that runs three rootless containers:
- pine-whisper — Wyoming faster-whisper STT (docker.io/rhasspy/wyoming-whisper
3.4.1, model base-int8/en), publishes :10300
- pine-piper — Wyoming Piper TTS (2.2.2, voice en_GB-alba-medium),
publishes :10200
- pine — nginx launcher serving a self-contained setup/status page
(WiFi provisioning + Home Assistant wiring instructions), the Open target
The two engines publish their Wyoming ports so Home Assistant reaches them via
host.containers.internal. All rootless (cap-drop=ALL, no_new_privileges),
manifest-driven — no hardcoded podman/installer. The engines are internal stack
members (added to the drift-check INTERNAL_MANIFEST_IDS allowlist); only "pine"
carries a catalog entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v1.7.104 shipped ISOs whose units crash-looped (nostr-relay 3s loop,
archipelago-diag 203/EXEC) because build-time extraction failures were mere
warnings and the cached rootfs never tracked its recipe:
- hash the rootfs-defining region of the build script and rebuild when it
changes (stale cache shipped ISOs without wpasupplicant/iw/rfkill)
- refuse to build without nvpn / nostr-rs-relay unless
ALLOW_MISSING_VPN_BINARIES=1
- ConditionPathExists on nostr-relay/nostr-vpn/diag units so a stripped
image skips them instead of crash-looping
- post-install test: every enabled archipelago*/nostr* unit must have an
existing ExecStart payload
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
.198 crash-looped 236x on a truncated OTA binary with a good backup
sitting in update-backup/ — verify_pending_update() can't help when
the new binary never runs. New scripts/ota-crash-guard.sh runs as
root from ExecStartPre: while the post-OTA pending-verify marker
exists it counts start attempts, and after 5 failures restores the
backup binary (atomic rename), replaces the marker with an
update-rolled-back.json tombstone, and lets the service come back
on the previous version.
Wiring: fresh ISOs get the ExecStartPre line in the unit file;
existing nodes get a drop-in installed by apply_update's runtime
component step on their next OTA. '+-' prefix so a missing or
failing guard can never block the service.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fresh offline installs came up without fedimint-clientd and filebrowser
(they only appeared after connecting internet). Three root causes:
- archipelago-load-images.service ran 'podman load' as root, but every
container runs rootless as archipelago — bundled images landed in
root's storage where the rootless runtime can't see them, so all
container creation silently depended on registry pulls. The loader now
loads into the archipelago user's storage (with linger + runtime-dir
wait + system migrate).
- The unbundled ISO bundled only filebrowser.tar; fmcd (fedimint-clientd)
is a baseline first-boot app too and is now part of the unbundled core
bundle.
- first-boot's pull_with_fallback always hit the network; it now uses an
already-loaded local image first and skips the pull entirely.
Also: fedimint-clientd added to the UI's hardcoded curated-app fallback
list (it was missing when all catalog fetches fail offline), plus its
INSTALLED_ALIASES entry, and the stale fmcd bundling comment in
image-versions.sh corrected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
barkd (Ark wallet daemon, pinned 0.3.0, checksum-verified release binary)
packaged as an installable app; thin HTTP bridge in wallet/ark_client.rs
mirrors the fedimint_client pattern — the bark SDK stays out of the node
binary. wallet.ark-* RPCs cover status/balance/address/send/invoice/
board/offboard/history/configure; Ark movements merge into the unified
ecash history (kind="ark") and spendable Ark sats into total_sats.
Signet defaults (Second's public Ark server) until Ark matures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- image: create archipelago user with dialout so the backend can open
/dev/ttyUSB*/ttyACM* LoRa radios; doctor Fix 14 heals existing nodes
(verified live on the .65 RC install — device detected after the fix)
- Web5Federation.vue called nonexistent federation.list (Unknown method);
now uses federation.list-nodes + federation.list-pending-requests with
a last-seen-based online count
- sanitizer: let app-install dependency errors through — 'LND requires a
running Bitcoin node' was masked as 'Check server logs', so install
failures on fresh nodes looked random
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- archipelago-first-boot-secrets.service regenerates the self-signed TLS
keypair (device hostname in SAN) and all SSH host keys on the installed
system's first boot, before ssh/nginx start — the squashfs bakes one
key set at build time, so every flashed device shared them (§F 🔴)
- staging-first swaps: a failed regeneration keeps the baked keys rather
than leaving the device keyless
- the builder now emits <iso>.sha256 next to the ISO, and
scripts/sign-iso-checksums.sh signs {artifact,sha256,size} as a JSON
doc with the release-root ceremony (verify: ceremony verify) (§F 🟠)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict resolution: keep the branch's CPUQuota=200% (75% was the proven
cause of choppy HDMI audio — the file's own comment says so; the 75% on main
was an accidental regression bundled into an unrelated UI commit) and keep
main's newer MemoryMax=2800M/MemoryHigh=2200M tuning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tx1138 Gitea was retired as a release server 2026-06-13 and its registry
frontend is fully dead (500 on every /v2 manifest read, observed 2026-07-10).
Nothing may reference it anymore:
- registry.rs: no longer a default registry, no longer force-enabled on
load; saved configs are stripped of it on load (same one-time migration
treatment as the decommissioned Hetzner mirror), with a regression test.
- image_policy.rs: removed from TRUSTED_REGISTRIES — refs through the dead
host are now refused at the pull site (rejection test added).
- api/handler: dropped the legacy catalog-proxy fallback URL.
- .gitmodules: indeedhub submodule repointed to the OVH Gitea.
- scripts, image-recipe, app-catalog data, neode-ui strings, docs, and all
test fixtures repointed to 146.59.87.168:3000 (or neutral example hosts).
- image-versions.sh: ARCHY_REGISTRY_FALLBACK emptied (guarded consumers
skip it); reconcile-containers.sh candidate guard hardened.
The only remaining occurrences of the host string are the strip/reject
enforcement paths and their regression tests — the code that guarantees it
is never used again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Debian's podman only Recommends catatonit; nodes installed before
install-podman.sh gained the dependency fail any init-enabled container
deploy (observed on shorty-s deploying sites via Portainer). Doctor
Fix 13 installs it via apt when absent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A store on a node whose LND has only unannounced channels mints BOLT11
invoices with no route hints when lightningPrivateRouteHints is off, and
external wallets fail with "no way to pay this invoice" (hit on shorty-s
2026-07-10 with a Blink payer). Hints cost nothing on public channels, so
the doctor now flips the flag on for every store on each run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The netns-egress repair only stopped/started containers, which reuses the
existing netns — its holders (aardvark-dns, podman pause) survive, so a
missing pasta tap was never rebuilt. On shorty-s (2026-07-10) this cycled all
35 containers every timer run for ~an hour, bouncing bitcoind/LND/BTCPay the
whole time, with 'egress still broken after cycle' after every pass.
Now the cycle kills the netns holders, runs podman system migrate, and clears
/run/user/<uid>/containers/networks so the first container start recreates
pasta + aardvark-dns from scratch (the sequence that actually fixed the node).
A failure counter (/var/lib/archipelago/doctor-netns-cycle-failures) stops the
fleet-wide cycling after 3 consecutive failed rebuilds until egress is
observed healthy again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- create-release.sh: assert the freshly built frontend dist actually embeds
the bumped version before packaging — the ui-dist-version guard in
tests/release/run.sh is behind --with-build, which the release path never
passes, so a silently no-opped npm build could ship a stale dist with a
valid sha256. Verified against the real dist (passes on current version,
trips on a missing one).
- publish-release-assets.sh: verify published assets by downloading and
comparing sha256 against the manifest, not just Content-Length — a
size-correct/content-wrong mirror asset now fails the publish gate.
Verified live against the v1.7.99-alpha assets on the vps2 mirror.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RNS's own rnodeconf utility (frequency/bandwidth/spreading-factor/coding-rate
read+write, firmware signature verification, board bootstrap) has been the
one tool that reliably diagnoses real RNode hardware — it's what finally
proved two live nodes were silently running at different spreading factors
(SF5 vs SF10, invisible from any of our own probe/logging code, and the
actual reason two correctly-flashed radios could detect each other's RF but
never decode a packet). Every node should have it, not just whichever one an
agent happened to build a throwaway venv on to debug a specific incident.
- reticulum-daemon/build.sh: also PyInstaller-package archy-rnodeconf
alongside the existing archy-reticulum-daemon, same --collect-submodules/
-d noarchive flags (same RNS.Interfaces __all__-glob requirement applies).
- scripts/deploy-to-target.sh: actually wire both packaged binaries into the
live deploy path (neither was wired in before — a pre-existing gap noted
in docs/RETICULUM-TRANSPORT-PROGRESS.md; this is why reticulum-daemon
previously only worked via manual dev-venv setup on rsync-deployed nodes,
not the ISO-imaged ones). Non-fatal on build/deploy failure — archipelago
already falls back to its dev-venv path if the packaged binary is absent.
Also installs the missing `python3.<minor>-venv` package when needed
(same "ensurepip not available" gap hit manually twice this session).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
check_for_updates now fetches the manifest as raw JSON and runs
trust::verify_detached before parsing: a tampered or wrong-signer
signature rejects the mirror outright, and unsigned manifests are
offered for MANUAL apply only — the 3 AM auto-apply scheduler refuses
them, closing the unattended remote-root hole (§A of the 1.8.0
hardening plan). UpdateState gains manifest_signed so the UI can
surface authenticity.
Publisher side: create-release.sh signs the manifest during the
release (ceremony, mnemonic via TTY/env only), publish-release-assets
hard-refuses to ship an unsigned manifest (grep + new 'ceremony
verify' cryptographic gate), and scripts/sign-manifest.sh covers
re-signing outside a release run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pin RELEASE_ROOT_PUBKEY_HEX from the 2026-07-02 release-root signing ceremony
(signer did:key:z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur) so nodes verify
the publisher identity of the app-catalog. Sign releases/app-catalog.json in place.
Fix two floats that made the catalog unsignable: archy-btcpay-db manifest version
-> string, fedimint-clientd cpu_limit 0.25 -> 1 (u32). Add scripts/sign-catalog.sh
helper, the 1.8.0 release-hardening plan/tracker, and the commit-and-push project
rule in CLAUDE.md.
Backward-compatible: old binaries still accept the signed catalog; the pinned-anchor
binary ships in the next build/OTA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ssmithxandClaude Sonnet 5
