The repo is source code and guidelines only. Nothing about how Archipelago's
own fleet is run, or how the team works, stays in it.
Untracked (kept on disk, gitignored) — 250 files:
- .planning/ (199) and loop/ — internal development process
- fleet operations tooling that targets specific nodes: deploy-to-target,
deploy-tailscale, deploy-config-defaults, setup-target-dev, setup-aiui-server,
setup-https-dev, debug-frontend, node-profile, fleet-fips-pair/unpair,
image-recipe/sync-from-live.sh
- image-recipe/INTEGRATION-GUIDE.md and docs/multinode-testing-plan.md, both of
which are live-server workflow and fleet node inventories
- the Phase 10 on-node verification and evidence records, which cite .planning/
as their evidence base
KEY-05-ENTROPY-ENFORCEMENT.md was initially moved out with the other Phase 10
docs and then put back: it is cited as normative rationale from ten places in
the codebase, including core/clippy.toml, which bans rand::thread_rng and
points at it for the reason. That makes it a guideline, not an internal record.
Node names removed from source (48 occurrences across comments, manifests and
test fixtures): archi-dev-box, archy-x250*, shorty-s, framework-pt,
zaza-optiplex, archi-thinkpad. Comments keep the engineering context and the
date, which is what carried the meaning; the machine name did not.
Three of those were live test values rather than comments and were replaced
with valid stand-ins, not prose: two mDNS hostnames and a mesh peer name.
An earlier pass substituted "a test node" into a hostname assertion, producing
an invalid hostname; caught and fixed as test-node.local.
Wipe mechanism: .local-only/manifest.txt inventories every local-only path and
.local-only/wipe.sh deletes them on one confirmation, refusing to touch
anything git still tracks. Both are themselves untracked, so the public repo
does not carry a map of internal filenames.
Verified: cargo check -p archipelago --all-features clean; archipelago-container
75/75 tests pass; appOrigin vitest 7/7; audit-secrets 5/5; every relative link
in tracked markdown resolves (0 broken).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All four found by verifying on archi-dev-box rather than assuming.
container-specs.sh: archy-lnd-ui was specified as a BRIDGE container with
SPEC_PORTS="18083:80", but docker/lnd-ui/nginx.conf listens on 18083
directly (it must, to proxy the backend on 127.0.0.1:5678 same-origin).
Recreating from that spec publishes host 18083 to container port 80, where
nothing listens. Reproduced on the node: the app came back with :18083
refusing connections, HTTP 000. This never fired before because the running
containers are created by first-boot-containers.sh, which is host-networked
and never reads this file; the spec is only consulted when self-update.sh
rebuilds a UI image, and that only happens when a file under docker/lnd-ui/
changes — which is exactly what the previous two commits did. So the next
OTA would have taken lnd-ui down on every node. Now SPEC_NETWORK="host"
with no port mapping, matching what actually runs. NET_BIND_SERVICE dropped
with it: 18083 is unprivileged.
lnd-ui channels link: pointed at /apps/lnd/channels, but that route is a
CHILD of the /dashboard record in neode-ui's router, so the real path is
/dashboard/apps/lnd/channels. nginx's SPA fallback returns 200 for the
wrong path, so it failed as vue-router's NotFound view rather than an HTTP
404 — both the Payment Channels card and the Manage Channels button.
Both apps, copy buttons: navigator.clipboard only exists in a secure
context, and nodes serve these apps over plain http; the main UI also
embeds them in an iframe, where the async Clipboard API is separately gated
by the clipboard-write permission policy. Every copy button silently did
nothing there. Added an execCommand('copy') fallback behind a copyText()
helper and routed all six call sites through it.
lnd-ui Node ID: showed the bare pubkey whenever getinfo.uris was empty,
which is the common case — LND only populates uris once it is advertising
an external address. The bare pubkey is not what a peer pastes to open a
channel. The full pubkey@host:9735 URI is now built from the Tor onion
where available, falling back to this node's address, with a hint saying
which and what its reachability is. The QR encodes the URI too.
Verified on archi-dev-box: both images rebuilt and containers recreated
from the specs; lnd-ui and bitcoin-ui both serve 200 with the new assets;
and the RPCs the new tabs depend on all answer on the live node —
getblockstats returns every field the charts read, getpeerinfo returns 11
peers carrying relaytxes and network values the classifier handles.
Note for whoever tests bitcoin-ui's Insights/Peers tabs: /bitcoin-rpc/ now
sits behind auth_request /_session_check (a05956c4 et al), so it answers
401 to an unauthenticated curl by design. A logged-in browser sends the
session cookie same-origin, which is how those tabs get their data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six code paths configured the Lightning gateway with a bcrypt hash committed
to this repository — and one deploy path with a plaintext password literal —
whenever the per-install secret was missing. Anyone holding a copy of the repo
held the admin credential for every gateway that ever took a fallback.
container::secrets now owns the credential end to end: ensure_gateway_credential
(idempotent, delegates to ensure_one's bcrypt arm) and gateway_bcrypt_hash,
which returns Err when the secret is missing/empty and when the stored value is
on the KNOWN_DEFAULT_GATEWAY_HASHES denylist — so this codebase cannot hand
back the compromised value even to a node already carrying it.
get_app_config was widened to Result so a credential-less install cannot reach
podman run at all; configure_fedimint_lnd takes the resolved hash instead of
re-reading with its own fallback. The four shell paths stop generating
credentials entirely (dropping the htpasswd host dependency) and skip container
creation with a printed reason rather than substituting anything.
Naming converges on the manifest's fedimint-gateway-hash/.pw, with legacy
fedimint-gateway-password values copied forward rather than regenerated so no
node loses a working unique credential. Plan 01-16 owns rotation of installs
already carrying the default.
Verified: cargo build clean; cargo test -p archipelago 999 passed (2
boot_reconciler timing tests failed under concurrent load, green in isolation,
untouched by this diff); bash -n clean on all five scripts; the compromised
literal now appears exactly once in the tree, as the denylist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sizes bitcoind -dbcache to host RAM (~1/16, floor 300MB, cap 4096) instead of a
fixed 2048/4096. A multi-GB UTXO cache on an 8GB node running the full app stack
pushed memory past physical RAM and triggered system-wide swap thrash: the disk
saturated, bitcoind could not answer its own RPC, and the dashboard backend's
sqlite reads stalled — surfacing as fleet-wide /rpc/v1 502s and a blank Bitcoin
UI. Applied in scripts/container-specs.sh (reconciler path) and the config.rs
bitcoin-core path.
Bitcoin status cache now polls every 5s (was 10/15) with an 8s timeout (was 20s)
and fetches the four RPCs concurrently, so the cached snapshot tracks bitcoind's
responsive windows during IBD and the UI stops dwelling on "reconnecting...".
Unifies the divergent discover AppGrid/FeaturedApps image-error handlers onto the
canonical placeholder fallback so missing app icons render the placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The reconcile spec for bitcoin-knots auto-enables prune=550 when
DISK_GB < 1000. DISK_GB was measured via `df /`, which on every
archy install reports the ~30 GB OS partition because user data
lives on a separate encrypted /var/lib/archipelago volume.
Result: every archy node with a 2 TB data drive was silently being
configured as a pruned node, and any bitcoin-knots container
recreated by reconcile would delete its historical blocks down to
the 550 MB prune window on next start.
Observed on .228 (2 TB box): blocks dir went from 384 GB to 926 MB
after a reconcile-triggered restart. Historical archive unrecoverable
without full re-IBD from genesis.
Fix: check /var/lib/archipelago first (where bitcoin data actually
lives). Fall back to / only on first-boot before the data partition
is mounted.
Follow-up to 8b7cb002 (no version bump — same v1.7.0-alpha manifest):
* WireGuard peer persistence. Kernel peer state is ephemeral; the add-peer
RPC wrote each peer to data_dir/nostr-vpn/peers/*.json but nothing
re-pushed them on reboot. Result on .198: wg0 came up listening with zero
peers after last night's reboot. Added vpn::restore_wg_peers() — reads
the peers dir, waits up to 30s for wg0 to exist, then replays each via
`archipelago-wg add-peer`. Spawned from main.rs alongside the other
startup tasks.
* Reconcile + filebrowser drift. scripts/container-specs.sh load_spec_
filebrowser now declares SPEC_NETWORK="archy-net" (to match what
first-boot-containers.sh creates) and pins the filebrowser-data volume
+ wget-style healthcheck so the reconciler stops reporting network
drift. Without this, reconcile would kill the healthy first-boot
filebrowser container and recreate it on bridge, breaking the archy-net
DNS name the backend proxies to.
Manifest binary sha/size refreshed:
6c178a76…3582cc, 40361912 bytes.
Rebuilt ISO at image-recipe/results/archipelago-installer-unbundled-x86_64.iso
(Apr 20 07:10) carries both fixes baked in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes bundled into the OTA:
1. update.download hard-fail on git-path nodes. handle_update_check's git
branch reported update_available=true + update_method="git" but never
populated state.available_update, so update.download returned "No update
available to download" even though the UI showed one. SystemUpdate.vue
now routes update_method=="git" through update.git-apply (pull+rebuild+
restart via self-update.sh); manifest-path nodes keep the download→apply
flow. i18n strings + confirm modal added for the git path.
2. Reconciler creating containers behind the user's back. On fresh
unbundled installs (.198, .253) archy-mempool-db and archy-btcpay-db
materialised ~10 min after first boot because reconcile-containers.sh
walked container-specs.sh's canonical tier list and created any
"missing" container. reset_spec() now defaults SPEC_OPTIONAL="true",
so reconcile is strictly a repair tool — baseline comes from
first-boot-containers.sh (filebrowser on unbundled), everything else
from the install RPC.
Also forces OTA trigger for nodes on 1.6.0-alpha that otherwise saw
"I'm at manifest.version, nothing to do" and skipped the refreshed 1.6
artifacts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
App catalog served from Gitea repos (app-catalog) with 35 apps.
Nodes fetch catalog dynamically — new apps appear without frontend
rebuild. Test app added and removed to verify pipeline.
Gitea manifest updated with internal_port/nginx_proxy for iframe.
Updated catalog.json, nginx configs, app session configs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LND crashes with "netlinkrib: address family not supported by protocol"
in rootless podman because it needs NET_RAW to enumerate network
interfaces during TLS certificate generation. Added to capabilities
in config.rs, first-boot-containers.sh, and container-specs.sh.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the Bitcoin RPC 60-second gate that blocked 13+ dependent containers
(mempool, electrumx, btcpay, lnd, fedimint) from being created on first boot.
Containers now always get created and auto-restart via health monitor once
Bitcoin becomes responsive — the designed recovery path.
Additional hardening:
- Validate archy-net creation with retry (silent failure broke DNS)
- Verify critical images are loaded, re-load from tarballs if missing
- Create SearXNG settings.yml before container start (was missing)
- Run reconciler automatically after first-boot failures
- Add load-images as explicit systemd dependency with 900s timeout
- Propagate config write errors in install.rs (bitcoin.conf, lnd.conf)
- FileBrowser password change: retry loop (6 attempts) + 0o600 perms
- Post-start verification: detect containers that exit immediately
- Add 2s dependency waits between container starts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Nginx needs CHOWN, SETUID, SETGID to chown cache directories and drop
privileges on startup. LND UI additionally needs NET_BIND_SERVICE to
bind port 80 inside the container. Without these, cap-drop ALL causes
nginx to crash with "Operation not permitted" on chown or bind.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The health check command goes through multiple shell layers
(assignment → variable expansion → eval → podman → sh -c). Inner
double quotes need \\\" escaping to survive as literal " in Python.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The electrumx container image doesn't include curl. Replace the HTTP
health check with a Python socket connection test to the RPC port.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- electrumx: add DAC_OVERRIDE to SPEC_CAPS — rootless podman maps container
UID 0 to host UID 1000, but volumes are owned by host UID 100000; without
DAC_OVERRIDE the container can't write to its own data directory
- lnd: replace curl-based health check with lncli using readonly macaroon —
the REST API requires macaroon auth, so unauthenticated curl always fails
- grafana: add DAC_OVERRIDE to SPEC_CAPS for the same rootless volume issue
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All container image references now pull from 80.71.235.15:3000/archipelago/
instead of Docker Hub and ghcr.io. image-versions.sh is the single source
of truth; all scripts use $*_IMAGE variables instead of hardcoded refs.
Files updated:
- scripts/image-versions.sh: central ARCHY_REGISTRY variable
- core/*/config.rs: registry whitelist includes app registry
- core/*/stacks.rs: Immich + Penpot stack images
- scripts/{first-boot,deploy-to-target,container-specs}.sh: use variables
- docker/*/Dockerfile: nginx base image from registry
- image-recipe/: ISO build, podman config, menu script
- scripts/{container-doctor,deploy-bitcoin-knots,fix-indeedhub,validate-app-manifest}.sh
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Dashboard.vue: move DashboardMobileNav outside <main> so position:fixed
isn't broken by will-change:transform on the perspective container
- Add container-specs.sh and reconcile-containers.sh utility scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
