Compare commits

...
Author SHA1 Message Date
archipelago ab27fb97f8 chore: release v1.7.68-alpha 2026-05-19 09:37:47 -04:00
archipelago d736364ad7 fix(apps): stabilize btcpay and public proxy launch flows 2026-05-19 09:26:43 -04:00
archipelago e9898ead76 chore: update release lockfile 2026-05-18 11:55:20 -04:00
archipelago b25d41c5c6 chore: release v1.7.67-alpha 2026-05-18 11:54:57 -04:00
archipelago 32902d3891 fix(ui): stabilize system status metrics 2026-05-18 11:47:12 -04:00
archipelago 92c578d3d9 chore: update release lockfile 2026-05-18 10:17:20 -04:00
archipelago 6240064acf chore: release v1.7.66-alpha 2026-05-18 10:15:56 -04:00
archipelago 19dbf60f03 fix(apps): detect stale npm created containers 2026-05-18 10:04:22 -04:00
archipelago b49d8f1f8a chore: update release lockfile 2026-05-18 09:31:57 -04:00
archipelago ec36ac7e2c chore: release v1.7.65-alpha 2026-05-18 09:31:41 -04:00
archipelago 7104ba0cbf fix(apps): repair orchestrator starts before launch 2026-05-18 09:20:12 -04:00
archipelago d0b08d2790 chore: update release lockfile 2026-05-17 23:25:16 -04:00
archipelago 76288f541e chore: release v1.7.64-alpha 2026-05-17 23:24:39 -04:00
archipelago b701e125b4 fix(update): relax apply rate limit 2026-05-17 23:15:07 -04:00
archipelago 837ba63466 chore: update release lockfile 2026-05-17 23:03:44 -04:00
archipelago 8191d92bed chore: release v1.7.63-alpha 2026-05-17 23:03:06 -04:00
archipelago ae8359da4b fix(release): rebuild backend artifacts 2026-05-17 22:54:37 -04:00
archipelago d91b858d9b chore: release v1.7.62-alpha 2026-05-17 22:40:36 -04:00
archipelago 19f2125a4d fix(apps): repair stale nginx proxy manager ports 2026-05-17 22:38:04 -04:00
archipelago a992abcd06 chore: release v1.7.61-alpha 2026-05-17 22:13:21 -04:00
archipelago 4d6b4f76af chore: release v1.7.60-alpha 2026-05-17 20:45:56 -04:00
archipelago 0a94c0097f chore: release v1.7.59-alpha 2026-05-17 19:44:54 -04:00
archipelago 413d50116e fix(apps): restore mobile and website launching 2026-05-17 19:22:18 -04:00
archipelago daad50325b chore(release): require curated release notes 2026-05-17 18:59:12 -04:00
archipelago e05e356d64 chore: release v1.7.58-alpha 2026-05-17 18:40:50 -04:00
archipelago cfb304a001 feat(mesh): add meshtastic serial radio support 2026-05-17 18:07:40 -04:00
archipelago 7804223152 chore: release v1.7.57-alpha 2026-05-17 17:30:04 -04:00
archipelago a322b04021 fix(iso): avoid polkit in live debootstrap seed 2026-05-15 18:32:14 -04:00
archipelago 645cf69ed7 chore(release): refresh v1.7.56-alpha manifest after wifi fix 2026-05-15 18:26:17 -04:00
archipelago 01ec0565a6 fix: restore wifi setup and ssh password updates 2026-05-15 18:15:06 -04:00
archipelago 30505f41ff chore(release): refresh v1.7.56-alpha notes and artifacts 2026-05-15 17:54:32 -04:00
Dorian 5818541721 chore: release v1.7.56-alpha 2026-05-14 09:13:58 -04:00
Dorian b8053c00ca fix: clear stale health notifications 2026-05-14 08:57:54 -04:00
Dorian f95e9a1cd0 fix: quote quadlet environment values 2026-05-14 01:15:22 -04:00
Dorian be50dc3235 fix: avoid bootstrap bitcoin restarts 2026-05-14 00:03:16 -04:00
Dorian 2ff47f88a7 fix: harden container reconcile and launch behavior 2026-05-13 22:59:55 -04:00
Dorian 835c525218 chore(release): stage v1.7.55-alpha 2026-05-13 15:09:22 -04:00
Dorian 3202b79e41 chore(release): move artifacts to gitea releases 2026-05-13 14:11:42 -04:00
archipelago c0751e2551 chore(release): stage v1.7.54-alpha 2026-05-06 09:23:57 -04:00
archipelago 1a0d8a432c chore(release): stage v1.7.53-alpha 2026-05-05 13:59:50 -04:00
archipelago 745cb1c626 chore(release): stage v1.7.52-alpha 2026-05-05 11:29:18 -04:00
archipelagoandClaude Opus 4.7 10fbb8f87c docs(testing): track Phase 3.4 race fix + drift-sync hook
* L0 unit count: 630 → 631 (translate_health_check_http_does_not_double_prefix_scheme)
* Phase 3 row: add TimeoutStartSec=600 race fix (44f275ed) + drift-sync hook (0889367d)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 11:53:18 -04:00
archipelagoandClaude Opus 4.7 aad0ba5234 feat(orchestrator): drift-sync existing Quadlet units on each reconcile
When a Quadlet unit file already exists for an orchestrator-managed
backend, sync its on-disk bytes against what the current renderer
produces. write_if_changed makes this idempotent — when bytes match,
no IO; when they differ (post-deploy of a renderer change), the file
is rewritten and systemctl --user daemon-reload runs once.

We deliberately do NOT restart the .service when the file changes:
running containers keep their current config until the operator
restarts them. That's the right tradeoff — file updates are cheap and
non-destructive; service restarts are the SIGKILL cascade we're
trying to eliminate.

Why this matters: pre-this-commit, every renderer change required a
fresh package.install RPC per app to take effect. Observed live on
.228 2026-05-02 — the TimeoutStartSec=600 fix shipped in code but
existing units stayed on the old format because nothing triggered a
re-render. Combined with state.json being empty (so the reconciler's
auto-install path didn't fire either), the fix was invisible until
manual unit deletion.

Companions (UI_APP_IDS) are skipped — companion.rs renders those units
with a different shape; syncing here would clobber them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 11:43:18 -04:00
archipelagoandClaude Opus 4.7 281e65e697 fix(quadlet): TimeoutStartSec=600 when Notify=healthy is set
Bug surfaced live on .228 2026-05-02 — every backend Quadlet unit
(lnd, electrumx, fedimint, btcpay-server, mempool-api, bitcoin-knots)
hit systemd's default 90s start timeout because Notify=healthy makes
systemctl wait for the first green health probe, but
HealthInterval=30s × HealthRetries=3 = 90s minimum even on a healthy
service. Race: timeout fires the moment the third probe MIGHT succeed.

Result was three different post-states (inactive+running, failed+missing,
inactive+stopped) depending on whether systemd's ExecStopPost ran
podman rm before the orchestrator's adoption logic re-grabbed the
container.

Fix: when health is set, render TimeoutStartSec=600 (10 minutes) into
[Service]. Long enough for slow-starting backends (electrumx index
replay, lnd wallet unlock) without being so long that a truly stuck
unit hangs forever. Companions stay unchanged (no health → no override,
default 90s applies).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 07:14:48 -04:00
archipelagoandClaude Opus 4.7 384f12de7a fix(quadlet): http:// double-prefix + companion migration race
Two bugs surfaced by the first real-node validation of Phase 3.2-3.4
on .228 (2026-05-02), both caught before flipping the default.

Bug 1 — translate_health_check double-prefixed http://. Manifests in
the wild carry the scheme inside the endpoint string ("http://localhost:8175"),
and we were prepending another http:// unconditionally. Result on .228:
every backend HealthCmd read `curl -fsS -m 5 http://http://localhost...`,
every probe failed, fedimint hit a 14-restart loop. Now we accept either
form and skip appending hc.path when the endpoint already carries one.
Regression test asserts no double-prefix and that an in-endpoint path
is honoured.

Bug 2 — Phase 3.3 migration ran for UI companions (bitcoin-ui /
electrs-ui / lnd-ui) that have shipped via Quadlet since v1.7.41.
Migration tore down the running companion + raced companion.rs render,
producing "Phase 3.3: re-install archy-bitcoin-ui via Quadlet" reconcile
errors and leaving archy-bitcoin-ui down. Companions now short-circuit
out of migrate_to_quadlet_if_needed before any IO. Also: when try_exists
returns Err for an unrelated reason (permissions, EIO), we now skip
migration instead of treating "I can't tell" as "go ahead and migrate" —
migrating on top of a possibly-existing unit is destructive.

What this does not fix yet:
  * the orchestrator's reconciler iterating every manifest in
    /opt/archipelago/apps/, not just installed apps. Pre-existing
    behavior (also affects the legacy path) — separate scope.
  * fedimint /data UID mismatch surfaced when Quadlet started fedimint
    fresh. Likely orthogonal — defer.
  * no rollback when install_via_quadlet fails after a remove_container.
    Tracked as Phase 3.3.1 — defer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 06:37:37 -04:00
archipelagoandClaude Opus 4.7 bd96c0475d feat(config): ARCHIPELAGO_USE_QUADLET_BACKENDS env override
Adds an env-var lever for Phase 3.2's use_quadlet_backends flag so the
20× harness can flip the path on per-node without a config.json edit
(which would require an archipelago.service restart — and that triggers
FM3 cgroup cascade until Phase 3.5 ships, so we can't ask anyone to
reconfigure live nodes that way today).

Truthy parsing centralised in `parse_truthy_env` (1, true, yes, on —
case-insensitive, whitespace-trimmed). Anything else is false. The
helper is unit-tested so future env-var flags can reuse the same shape.

Also adds a default-off regression test for use_quadlet_backends so
flipping the default ahead of the 20× verification fires immediately.

TESTING.md documents the Environment= snippet for the systemd drop-in
so the next operator can flip the flag on a debug node without
re-deriving the recipe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 05:44:09 -04:00
archipelagoandClaude Opus 4.7 9a89a000d4 test(lifecycle): post-condition gate for use_quadlet_backends path
A six-test bats suite that validates what install_via_quadlet (Phase 3.2)
is supposed to leave behind:

  * `.container` unit on disk in $XDG_CONFIG_HOME/containers/systemd/
    with [Container] / [Service] / [Install] sections, Image= present,
    and Restart=on-failure (the backend invariant — companions use Always)
  * Phase 3.4 cross-check: any unit with HealthCmd= must also emit
    Notify=healthy, otherwise systemctl start won't gate on health
  * `systemctl --user is-active` returns 0 for the .service
  * podman shows the container running
  * the container's cgroup is under user.slice/, NOT under
    archipelago.service — the kernel-level proof that FM3 cgroup
    cascade SIGKILL is structurally fixed for this container

Auto-skips on every test when no backend Quadlet units exist (today's
default state, use_quadlet_backends=false) — so the suite is a no-op
on current fleet boxes and turns into a hard regression gate the
moment anyone flips the flag and reinstalls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 05:34:47 -04:00
archipelagoandClaude Opus 4.7 97ce23d773 feat(quadlet): Phase 3.4 — health-gated startup via Notify=healthy
QuadletUnit gains an optional HealthSpec; from_manifest translates the
manifest's health_check (tcp/http/cmd) into a HealthCmd= directive and
emits Notify=healthy alongside it. systemctl start <unit>.service then
blocks until the container's first green probe — eliminating the
"container up but RPC not ready" race the orchestrator currently papers
over with post-start polling.

Translation policy:
* tcp,  endpoint "host:port"        -> nc -z host port
* http, endpoint "host:port", path  -> curl -fsS -m 5 http://endpoint<path>
* cmd,  endpoint "<shell command>"  -> verbatim
* unknown type / malformed endpoint -> None (skip Notify=healthy rather
  than emit a HealthCmd that hangs the unit start forever)

Companion units leave health: None and remain byte-identical to before
this PR — the renderer only emits the Health* / Notify= block when set.

+4 quadlet unit tests (19 total). Dropped a never-used test setter that
was generating a dead_code warning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 05:21:57 -04:00
archipelagoandClaude Opus 4.7 65576bd755 feat(orchestrator): Phase 3.3 — in-place migration to Quadlet
When use_quadlet_backends flips from off → on, existing fleet boxes
have backend containers parented under archipelago.service's cgroup
(the bad shape that triggers FM3 cascade SIGKILL on every archipelago
restart). ensure_running now notices and corrects this:

* If there's already a `<name>.container` unit on disk → no-op
  (subsequent reconcile ticks take this fast path).
* Else if a podman container with that name exists → it's a pre-3.3
  artifact. Stop+remove it (volumes survive — bind mounts are not
  touched by `podman rm`), then write the Quadlet unit, daemon-reload,
  and start the new managed service.
* Else → fall through to install_fresh, which already routes through
  install_via_quadlet when the flag is on.

The migration is idempotent and self-healing: if a fleet box is
half-migrated (unit on disk but no service active, or service active
but stale unit), the next reconcile tick converges. Bitcoin chain
data, lnd wallet state, and electrumx index all live on host bind
mounts and are unaffected by the container-record swap.

Volume safety audited per backend in `uses_orchestrator_install_flow`
allowlist — every entry mounts its data dir as a host bind mount.

Default still off. To migrate a node:
  /etc/archipelago/config.toml: use_quadlet_backends = true
followed by `systemctl restart archipelago` — the next reconcile tick
walks every managed app and migrates each in turn.

Tests: 624 passing, 0 cargo warnings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 17:27:59 -04:00
archipelagoandClaude Opus 4.7 5b2e02bd43 feat(orchestrator): Phase 3.2 — wire Quadlet path behind feature flag
prod_orchestrator::install_fresh now branches on the new
Config::use_quadlet_backends flag (default false):

* off (today's production behavior) — unchanged: runtime.create_container
  + start_container, container parented under archipelago.service's
  cgroup, FM3 cascade SIGKILL on every archipelago restart.
* on  — install_via_quadlet renders the manifest as a Quadlet unit via
  QuadletUnit::from_manifest, writes it atomically into
  ~/.config/containers/systemd/, calls daemon-reload, and starts the
  generated <name>.service. Container ends up under user.slice — no
  more cgroup parented under archipelago, so archipelago restarts
  don't touch the container's lifetime.

Default off so this commit is structurally safe to ship: nothing
changes at runtime until an operator opts in. Flip the default once
tests/lifecycle/run-20x.sh has gone green against the new path on
.228 + .198 (the v1.7.52 release gate).

Plumbing:
* config.rs — `use_quadlet_backends: bool` w/ Default false
* prod_orchestrator.rs — flag stored on the struct, threaded through
  new(), with set_use_quadlet_backends(bool) test setter
* prod_orchestrator.rs — install_via_quadlet helper
* dropped the Phase-3.1 #[allow(dead_code)] markers on from_manifest /
  parse_memory_mib / RestartPolicy::OnFailure now that the call path
  exists; if a future revert removes the wiring, the warnings come back.

Tests: 624 passing, cargo check clean (0 warnings). Existing companion
behavior unaffected — render_skips_backend_directives_when_default
still passes byte-equal to before quadlet.rs grew the new fields.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 17:22:10 -04:00
archipelagoandClaude Opus 4.7 9becafafd3 feat(quadlet): backend-manifest renderer (Phase 3.1 of v1.7.52)
The QuadletUnit struct now covers everything a backend manifest needs
(ports, environment, devices, add_hosts, entrypoint+command, read-only
root, no_new_privileges, cpu_quota, restart policy choice). Adds
QuadletUnit::from_manifest(&AppManifest, name) that translates a parsed
manifest into a unit, plus parse_memory_mib for "1g"/"512m"/raw-MiB
forms. The renderer skips empty/false directives so existing companion
units render byte-identically — no behavior change for shipping
companions; the backend renderer is dead code until Phase 3.2 wires it
into the orchestrator.

Eight new unit tests cover:
* parse_memory_mib forms (1024, 512m, 2g, garbage)
* shell_join quoting (whitespace, embedded quotes)
* RestartPolicy → systemd string mapping
* render emits backend directives when set
* render skips them when defaulted (companion regression gate)
* from_manifest happy path on a bitcoin-knots-shaped manifest
* from_manifest read-only volume detection
* from_manifest tmpfs filtering
* end-to-end manifest → render bytes assertion

Tests: 615 → 624 (+9 net; one pre-existing parse_memory_mib path was
implicitly covered before but is now explicit). Cargo warnings: 0.

`from_manifest`, `parse_memory_mib`, and `RestartPolicy::OnFailure` are
marked allow(dead_code) with explicit references to Phase 3.2 — if
3.2 doesn't wire them, the dead-code warning resurfaces.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 17:09:50 -04:00
archipelagoandClaude Opus 4.7 5074572373 test(lifecycle): add btcpay + fedimint + mempool suites
Brings L1 (RPC API) + L3 (lifecycle survival) parity coverage to the
three multi-app stacks that were previously only touched by
required-stack.bats. Combined with bitcoin-knots / lnd / electrumx
already shipping, the six core apps now have dedicated bats files.

Each suite is shaped like the existing single-container suites
(bitcoin-knots / lnd / electrumx) and gates every assertion on the
backing container actually being present, so a node without the stack
installed gets clean skip messages instead of false fails.

* btcpay.bats — 9 tests, including stack-wide presence and a
  "supporting containers don't cascade-restart" guard
* fedimint.bats — 8 tests, single container
* mempool.bats — 9 tests, mixed legacy + orchestrator-managed stack;
  reuses the :8999 mempool-api probe from required-stack for parity

Total bats now: 88 (was 53 → +35).
TESTING.md matrix advances 23 → 50 of 110 cells.
UI URL coverage for these three apps already lives in
ui-coverage.bats, so this PR doesn't duplicate proxy-path probes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:55:31 -04:00
archipelagoandClaude Opus 4.7 ec1dce93a9 docs(testing): canonical scorecard for container subsystem testing
Single source of truth for "where are we, where are we going" on the
v1.7.52 container excellence work. Replaces ad-hoc tracking in chat.

Sections:
* Test layers L0..L6 with toolchain + per-iteration latency
* Per-app × per-state coverage matrix (23 of 110 cells today; goal 110)
* Layer-by-layer status (L0+L1+L2 ●; L3 ◐; L4..L6 ○)
* Run commands (single suite / full suite / 20×)
* LoC budget — -270 committed, ~1,616 more possible if Phase 3 ships
* Performance KPIs (TBD — measure first, target second)
* Release gates — 8 boxes that must tick before v1.7.52 ships

The file lives in-repo so PR diffs to it answer "what did this commit
improve?". If you can't tick the box, the change isn't ready.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:52:42 -04:00
archipelagoandClaude Opus 4.7 b9eb6eb18a test(lifecycle): add UI surface coverage — HTTPS proxy + iframe URLs
Closes the coverage gap where existing bats suites would report green
on a node whose dashboard tiles 502 because the proxy upstream is dead.
First pass against .198 caught real prod issues immediately:
  /app/lnd/       → 502 (lnd container exited)
  /app/mempool/   → 502 (mempool container exited)
  /app/fedimint/  → 502 (fedimint container exited)
while existing tests reported only "container is up: false" with no
404/502 distinction.

* lib/ui-probes.bash — sourced helper. probe_https_200,
  probe_app_url (skip-if-container-down else assert-200),
  probe_dashboard_shell (asserts the Vue SPA HTML, not nginx default —
  catches the layout regression from feedback_release_tarball_layout.md),
  probe_dashboard_catalog (asserts /catalog.json non-empty).
* bats/ui-coverage.bats — 9 @test cases covering the dashboard +
  bitcoin-ui :8334 + the seven HTTPS_PROXY_PATHS most users hit
  (lnd, electrumx, mempool, fedimint, btcpay, filebrowser).

URL list mirrors HTTPS_PROXY_PATHS in
neode-ui/src/views/appSession/appSessionConfig.ts. Divergence between
the two is the exact bug class we're guarding against.

Loops clean under run-20x.sh. Container-state oracle is via local
podman inspect, so the suite must run on the archy host (same as
companion-survives-archipelago-restart.bats).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:49:30 -04:00
archipelagoandClaude Opus 4.7 c55a4f4e86 test(bootstrap): regression gate for the heal_podman_state socket bug
Extracted the heal_podman_state cleanup list as a module-level
HEAL_RUNTIME_SUBDIRS const so a unit test can structurally enforce
the invariant: the list must contain "containers" + "libpod" but
must NOT contain "podman" (which holds systemd's podman.sock
listener and was the bug fixed in commit bb421803).

If anyone re-adds "podman" — accidentally, by reverting, or by
copy-paste from old plan memory — this test fires before we ship,
not on the next deploy when it nukes the orchestrator's HTTP path.

Total tests: 614 → 615.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:32:59 -04:00
archipelagoandClaude Opus 4.7 01f416ae5d test(lifecycle): regression gate for FM3 cgroup-cascade SIGKILL
Sister suite to companion-survives-archipelago-restart.bats. That one
tests the same property for UI companions, which already ship via
Quadlet (commit 6e716f68) and so already pass.

This new suite tests the property for backend containers (bitcoin-knots
/ bitcoin-core / lnd / electrumx). Until v1.7.52 Phase 3 ships these
under Quadlet too, the suite is EXPECTED TO FAIL on fleet boxes — it's
the executable definition of "FM3 fixed".

Observed live on .198 on 2026-05-01: `sudo systemctl stop archipelago`
killed every container in archipelago.service's cgroup. The dedicated
"backends survive archipelago restart" test catches exactly that, and
also verifies the SAME container instance survives (compares pre/post
.Id), so an orchestrator that recreates a fresh container after the
SIGKILL doesn't read as pass.

Three @test cases:
* destructive gate (skip-marker for the suite)
* baseline: at least one backend installed + running
* backends survive: same .Id pre + post archipelago restart

Don't gate releases on this passing until Phase 3 lands; before then
treat it as a "expected to fail / shows progress" indicator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:17:27 -04:00
archipelagoandClaude Opus 4.7 f80daff8ba test(lifecycle): add dedicated electrumx.bats suite
Same shape as bitcoin-knots.bats and lnd.bats so the 20× release-gate
exercises electrumx through the same state matrix it uses for the other
two core apps. electrumx previously had a single TCP-port check inside
required-stack.bats; this adds destructive + cascade-destructive tiers.

10 @test cases:
* read-only: presence, valid state, TCP port (50001) reachable, no
  orphan containers beyond {electrumx, archy-electrs-ui}
* destructive: stop, start, restart, TCP port recovers within 120s of
  cold restart (longer than bitcoind because electrumx replays its
  index against bitcoind on start)
* cascade: uninstall, reinstall (240s timeout for index rebuild)

With this suite, the three single-container core apps (bitcoin-knots,
lnd, electrumx) now have parity coverage. Multi-container stacks
(btcpay, mempool, fedimint) come next.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:11:02 -04:00
archipelagoandClaude Opus 4.7 1103c2c710 test(lifecycle): add dedicated lnd.bats suite
Mirrors bitcoin-knots.bats so the 20× release-gate run exercises lnd
through the same state matrix. lnd previously had only a single
read-only check inside required-stack.bats; this adds the destructive
and cascade-destructive tiers that match what we already test for
bitcoin-knots.

10 @test cases:
* read-only: presence, valid state, lncli getinfo, no orphan containers
* destructive (ARCHY_ALLOW_DESTRUCTIVE=1): stop, start, restart,
  RPC recovers within 90s of cold restart (longer than bitcoind
  because the wallet has to unlock first)
* cascade (ARCHY_ALLOW_CASCADE_DESTRUCTIVE=1): uninstall, reinstall

Reuses the same lncli invocation as required-stack.bats so divergence
shows up clearly if either test breaks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:09:43 -04:00
archipelagoandClaude Opus 4.7 1b6c500657 test(lifecycle): add setup-teardown + run-20x harness scaffolding
Phase 4 of the v1.7.52 container excellence plan: a release-gate harness
that loops the bats suite N times in a row, with teardown between
iterations, and reports a pass/fail tally.

* setup-teardown.sh — clears /tmp/archy-rpc-session-* between runs so
  iteration N+1 doesn't reuse a logged-out cookie from iteration N.
  Idempotent; safe to run anytime. Designed to grow as we add suites
  that leave other transient state.
* run-20x.sh — wraps run.sh in a loop of ARCHY_ITERATIONS (default 20).
  Tracks per-iteration pass/fail with wall-clock timing, prints a
  results block, exits non-zero on any failure. Honors ARCHY_FAIL_FAST
  for short-circuit during dev.

Suggested release-gate command:
  ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 \
    tests/lifecycle/run-20x.sh

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:06:09 -04:00
archipelagoandClaude Opus 4.7 5be2febe13 fix(bootstrap): don't nuke podman socket dir during runtime self-heal
Observed live on .198: heal_podman_state was removing
$XDG_RUNTIME_DIR/podman/ alongside containers/ and libpod/. That dir
holds the systemd-bound podman.sock — the listener systemd creates for
socket-activated podman.service. Removing it broke every libpod HTTP
call from the orchestrator until `systemctl --user restart
podman.socket` ran. Far worse than any wedge it was trying to repair.

Drop podman/ from the cleanup list. The runtime state we actually want
to clean for FM6 (bolt_state.db drift) lives in containers/ and
libpod/ only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:57:15 -04:00
archipelagoandClaude Opus 4.7 6bbe1b96cf refactor: drop dead code surfaced by cargo
cargo check was showing five real warnings, all genuinely dead:

* container/mod.rs   — re-exports compute_container_name, AdoptionReport,
                       ReconcileAction, ReconcileReport were unused outside
                       prod_orchestrator. Drop from the pub use line.
* prod_orchestrator  — with_runtime + insert_manifest_for_test only exist
                       for the test module in the same file. Mark them
                       #[cfg(test)] so they don't appear in release builds.
* async_lifecycle    — remove_package_entry has no callers; doc claims
                       "used for install-failure cleanup" but nothing
                       cleans up. Delete (10 lines).
* registry.rs        — `use tracing::{debug, info};` had no consumers.
* fips.rs            — unused-assignment chain on last_status. The poll
                       loop always sets it on every break path, so the
                       initial `None` and the unwrap_or_else fallback
                       were both dead. Refactored to `let after = loop
                       { ...; break s; };`.

cargo check is now clean. cargo test --workspace --bins: 614 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:34:02 -04:00
archipelagoandClaude Opus 4.7 8f13298805 fix(bootstrap): self-heal wedged podman runtime state at startup
Closes FM6 (podman bolt_state.db / runtime drift) — observed live on
.198 today: bitcoind was running for several minutes, but podman's
state DB reported the container as Exited. The reconciler then tried
to "restart" it, racing the still-bound port 8332 and failing in a
loop.

heal_podman_state() runs as the last bootstrap stage, BEFORE the
orchestrator's reconcile loop ticks. It probes `podman info` with a
5s timeout; on failure it removes the runtime-state dirs under
$XDG_RUNTIME_DIR and re-probes. Persistent storage under
~/.local/share/containers/storage/ is never touched, so containers
re-discover from manifests on next call.

Cleanup never includes `podman system reset` or `system renumber` —
those are destructive and must stay operator-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:23:36 -04:00
archipelagoandClaude Opus 4.7 ba2eece9aa refactor(container): drop unused dependency_resolver module
DependencyResolver had zero call sites in prod or tests outside the
module itself. The actual install-time dependency check lives in
install.rs::detect_running_deps + check_install_deps; this DAG-walk
solver was never wired up. -268 LoC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:22:07 -04:00
archipelagoandClaude Opus 4.7 6603227874 fix(install): auto-clean stuck OTHER-variant bitcoin container
If bitcoin-core was installed but never started (e.g. port 8332 already
bound by bitcoin-knots), the container sticks in `created` state forever.
The old conflict check refused EVERY future bitcoin install — including
re-install of the running variant — leaving no UI path to recovery.

Now the check distinguishes states:
  - missing                       → no conflict, continue
  - running                       → real conflict, refuse install
  - created/exited/configured/... → stuck; auto-remove and continue

Volumes are untouched; only the dead container record goes away.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:59:11 -04:00
archipelagoandClaude Opus 4.7 27ff1d5b52 fix(install): generate bitcoin RPC password before orchestrator install
Bitcoin containers were exiting in ms after start because the orchestrator
install path skipped the credential-materialisation step the legacy path
did. resolve_secret_env then failed to read
/var/lib/archipelago/secrets/bitcoin-rpc-password, the container started
with no password, and bitcoind crashed before logs were useful.

Two changes:

1. install.rs — call bitcoin_rpc_credentials() for bitcoin/bitcoin-core/
   bitcoin-knots before any install branch runs. The function generates +
   persists on first call (OnceCell-cached), so this is idempotent.

2. manifest.rs::resolve_secret_env — return ManifestError::Invalid when a
   resolved secret trims to empty, instead of silently producing
   `KEY=` env vars that crash auth.

Adds a unit test for the empty-secret rejection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:39:56 -04:00
archipelagoandClaude Opus 4.7 f9e34fd0c6 refactor(install): route orchestrator-managed apps through orchestrator first
Phase 3a of the install path consolidation. Two coupled changes:

1. install.rs handle_package_install: gate the legacy "container exists →
   adopt + return" probe on !orchestrator_managed. Apps the orchestrator
   knows about (bitcoin-knots, bitcoin-core, lnd, electrumx, fedimint,
   filebrowser, btcpay-server stack apps, mempool stack apps, plus the
   companion UIs that just moved to Quadlet) skip the legacy probe and
   fall straight into the orchestrator branch.

   The legacy adopt block was returning success on a bare `podman start`
   exit-0 — even when the process inside the container crashed seconds
   later. That's the .228 "running but unreachable" failure mode. The
   orchestrator's ensure_running honors the manifest's health check and
   pre-start hooks (e.g. re-renders bitcoin-ui's nginx.conf if the RPC
   password rotated), so this is a behavioral upgrade, not just a
   refactor.

2. ProdContainerOrchestrator::install: make idempotent. Previously it
   blindly called install_fresh which would fail on `podman create` if
   the container name already existed. Now it delegates to ensure_running:
     - Container Running + healthy → no-op (refresh hooks, restart if
       config rewritten)
     - Container Stopped/Exited → start (with hook refresh)
     - Container missing → install_fresh
     - Container in wedged state (Created/Paused/Unknown) → force-recreate

   Without this, change #1 would regress every "container already exists"
   case for the 18 orchestrator-managed app IDs. With it, install becomes
   the single source of truth for "make app X be in the desired state."

Tests: 654 passed across the workspace (614 unit + 37 orchestration + 3
rpc), 0 failures. The 20 prod_orchestrator tests cover the install /
ensure_running / reconcile paths the new install delegates through.

Net delta: install.rs grows by ~30 lines (gating wrapper + comments),
prod_orchestrator.rs grows by ~30 lines (idempotent install body). Both
are temporary — the larger deletions (~1700 lines) come once every app
has been verified through the orchestrator path in subsequent phases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 11:12:52 -04:00
archipelagoandClaude Opus 4.7 23c4e7441f refactor(container): move companion UIs to systemd via Quadlet
Companion UI containers (archy-bitcoin-ui, archy-lnd-ui,
archy-electrs-ui) used to be launched as fire-and-forget tokio::spawn
blocks from install.rs. If archipelago crashed mid-spawn or the
container's cgroup was reaped, companions vanished from podman ps -a
and only a manual rm/run could bring them back (the .228 incident).

Now each companion is rendered as a Quadlet .container unit under
~/.config/containers/systemd/, daemon-reloaded, and started via
systemctl --user. systemd owns supervision from that point on:

- archipelago can crash, restart, or be uninstalled without touching
  any companion.
- Quadlet's Restart=always + RestartSec=10 handles container exits.
- A 30s reconcile tick in boot_reconciler enumerates expected
  companion units and re-installs any whose unit file or service
  vanished — defense-in-depth against external tampering.

New module layout:
- container/quadlet.rs: pure unit renderer + atomic write_if_changed
  + systemctl helpers (daemon_reload_user / enable_now / disable_remove
  / is_active). 6 unit tests, no I/O in the renderer.
- container/companion.rs: per-app companion specs, install/remove/
  reconcile, image presence (build local first, fall back to insecure
  registry only via image_uses_insecure_registry whitelist). 2 tests.

install.rs handle_package_install now ends with a single call to
companion::install_for(package_id), replacing 287 lines of spawn-and-
hope shellouts plus a ~120-line nginx auth-injector helper that worked
around per-node RPC password baking. The helper is gone too — the
pre-start hook renders the per-node nginx.conf to /var/lib/archipelago/
bitcoin-ui/nginx.conf and the Quadlet unit bind-mounts it read-only.

runtime.rs handle_package_uninstall now disables companions before
the container rm loop. Otherwise systemd's Restart=always would
respawn each companion within ~10s of removal.

Tests: 53 container tests pass, including 6 quadlet renderer tests
(host network, bridge network, capability set, atomic write idempotence)
and 2 companion specs (per-app companion lookup, build_unit shape).
boot_reconciler tests gain a #[cfg(test)] without_companion_stage()
flag so the paused-clock fixtures don't race the real systemctl I/O.

A bats regression test (companion-survives-archipelago-restart.bats,
gated on ARCHY_ALLOW_DESTRUCTIVE=1) asserts the .228 failure mode
cannot recur: every installed companion has a unit file, services
stay active across systemctl --user restart archipelago, and a
deleted unit file is recreated within one reconcile tick.

Net delta: +941 / -363, but the +941 is mostly tests (~440 lines)
and the new declarative layer; the imperative tokio::spawn block and
its nginx-auth helper are gone, removing two failure classes
(orphan companions on archipelago crash, and post-start exec races
under tightly-confined cgroups) that previously needed manual SSH
recovery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 10:45:07 -04:00
archipelagoandClaude Opus 4.7 2bf8181110 refactor(security): tighten capability + TLS-bypass surface
Three small, focused tightenings:

- core/container/src/podman_client.rs: drop the legacy Hetzner
  23.182.128.160:3000 mirror from image_uses_insecure_registry().
  It was decommissioned in v1.7.x and is stripped from active
  registry config at load time; leaving it in the bypass list let
  a stale config still skip TLS. Replace the inline match with a
  named INSECURE_REGISTRY_HOSTS slice so future entries are one
  line. Test now also pins the spoofing-immune semantics
  ("evil.example/146.59.87.168:3000/x" must NOT match).

- core/archipelago/src/api/rpc/package/config.rs: split bitcoin
  from lnd in get_app_capabilities(). bitcoind never opens raw
  sockets — drop CAP_NET_RAW from bitcoin/bitcoin-core/bitcoin-knots.
  lnd/fedimint/fedimint-gateway keep it because they enumerate
  network interfaces during cert generation.

- core/archipelago/src/bootstrap.rs: tighten_secrets_dir()
  enforces 0700 on /var/lib/archipelago/secrets and 0600 on every
  file inside on each startup. The dir-mode is the load-bearing
  isolation boundary against rootless container escapes (their UID
  maps to >=100000, can't traverse uid=1000/0700). The per-file
  sweep is defense-in-depth against any installer that wrote 0644.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 08:59:11 -04:00
archipelagoandClaude Opus 4.7 0684491072 chore: baseline codex hardening before lifecycle refactor
Snapshots the in-flight hardening work so subsequent reconcile/Quadlet
phases land on a clean before/after diff.

Changes:
- core/container/src/podman_client.rs: image_uses_insecure_registry()
  whitelist for the OVH (146.59.87.168:3000) and legacy Hetzner
  (23.182.128.160:3000) HTTP mirrors; podman_network_settings() lifts
  custom networks into the Networks map so containers can join them.
- core/archipelago/src/container/prod_orchestrator.rs:
  ensure_container_network() creates per-manifest networks on demand;
  apply_data_uid() now goes through host_sudo for mkdir -p + chown so
  bind-mount roots get created and chowned without password prompts.
- core/archipelago/src/api/rpc/package/{install,update,stacks}.rs:
  podman pull adds --tls-verify=false only for whitelisted registries.
- core/archipelago/src/bootstrap.rs: removes stale dev-mode systemd
  override on startup (live nodes carried it from old installers).
- core/archipelago/src/config.rs: ignore ARCHIPELAGO_DEV_MODE in prod
  binaries — it had been silently rerouting volumes to /tmp.
- apps/bitcoin-{core,knots}/manifest.yml: locate bitcoind at runtime
  so image-layout differences don't break entrypoint.
- scripts/app-catalog-image-smoke-test.py: production catalog/image
  smoke test that probes a target node before users click Install.
- .gitignore: cover .codex, .pnpm-store, __pycache__, *.bak.

Removes filebrowser.rs.bak and two stale catalog.json.bak files
(verified identical to live counterparts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 08:52:29 -04:00
archipelago 05e6c2e738 fix: release v1.7.51-alpha install hardening 2026-05-01 05:02:39 -04:00
archipelago be9f9528c3 fix: release v1.7.50-alpha OTA runtime repair 2026-05-01 03:14:07 -04:00
archipelago 7ab788d178 chore: release v1.7.49-alpha 2026-04-30 16:37:54 -04:00
191 changed files with 17386 additions and 2498 deletions
+8
View File
@@ -7,6 +7,14 @@
# Allow demo assets (AIUI pre-built dist)
!demo/
# Allow backend source for ISO source builds
!core/
!scripts/
!image-recipe/
image-recipe/build/
image-recipe/results/
image-recipe/output/
# Exclude nested node_modules (will npm install in container)
neode-ui/node_modules
neode-ui/dist
+12
View File
@@ -57,6 +57,11 @@ coverage/
*.dmg
*.app
# Release artifacts live in Gitea Release attachments, not Git history.
releases/**
!releases/
!releases/manifest.json
# macOS build output
build/macos/
@@ -77,3 +82,10 @@ web/
# Resilience harness reports (generated, contains session cookies)
scripts/resilience/reports/
# Codex / pnpm / python caches / editor backups
.codex
.pnpm-store/
**/__pycache__/
*.bak
.claude/scheduled_tasks.lock
+156 -1
View File
@@ -1,5 +1,160 @@
# Changelog
## v1.7.68-alpha (2026-05-19)
- BTCPay Server now ships on the official `docker.io/btcpayserver/btcpayserver:2.3.9` image, fixing the plugin catalog crash caused by newer plugin dependency version metadata while preserving existing datadirs and Postgres databases.
- BTCPay release and first-boot health checks no longer depend on `curl` inside the container; they use a bash TCP probe that works with the official image out of the box.
- Host nginx now serves Nginx Proxy Manager HTTP-01 challenge files before the Archipelago SPA fallback and is marked as the default HTTP/HTTPS virtual host, so public proxy hosts can issue certificates without hijacking local API traffic.
- Nginx Proxy Manager first-boot, runtime repair, and container-doctor paths now pre-create the ACME webroot, keep bind mounts owned by the rootless Archipelago user, and sync issued public proxy hosts into host nginx vhosts.
- The Nginx Proxy Manager host-nginx sync now skips proxy hosts with missing certificate files and rolls back the generated nginx include if validation fails, preventing a bad certificate path from poisoning later nginx reloads.
- App session close buttons now return to the previous dashboard screen when possible and otherwise fall back to My Apps, avoiding the 404 page after closing an app launched from an invalid or stale history entry.
- System Update confirmation and mirror modals now teleport to the document body with a full-screen overlay, so they cover the whole app instead of only the right-hand dashboard panel.
- Mobile app launches stay inside Archipelago's app-session webview and hide desktop-only new-tab launch affordances, including apps such as Home Assistant that previously looked like they would leave the mobile shell.
- Live recovery on `100.70.96.88` upgraded only the `btcpay-server` container to `docker.io/btcpayserver/btcpayserver:2.3.9`, preserved the existing datadir and Postgres database, and confirmed the container is healthy after a pre-upgrade backup.
- Public validation confirmed `spay.tx1138.com`/`www` redirect to BTCPay login over HTTPS and `sapien.tx1138.com`/`www` serve the L484 page over HTTPS using the issued Let's Encrypt certificates.
## v1.7.67-alpha (2026-05-18)
- Home dashboard status cards now keep the last known good system, VPN, Bitcoin, and FIPS values while route changes or transient RPC failures are in flight, avoiding false "not configured" or "not running" flashes.
- Home, Web5 Monitoring, and the Monitoring page headline cards now share the same live system-stat snapshot for CPU, memory, disk, uptime, and load so the visible numbers agree across the UI.
- Settings What's New is filled through `v1.7.67-alpha`, including the missing historical `v1.7.44-alpha` through `v1.7.66-alpha` entries.
- Bitcoin/Knots/Core shell lifecycle specs now match the Rust app config memory policy: 8 GiB on normal hosts, 4 GiB on low-memory hosts, and pruned Knots uses a larger dbcache on hosts with enough RAM to improve IBD throughput.
- ElectrumX/electrs shell lifecycle specs now match the 4 GiB memory policy used by the Rust app config, reducing drift between first boot, reconcile, and app lifecycle paths.
- Live assessment of `100.70.96.88` identified the current IBD bottlenecks as CPU/thermal/I/O pressure rather than RAM exhaustion, with follow-up work planned for existing-node swap repair, kiosk Chromium CPU reduction, and reconcile failure cleanup.
## v1.7.66-alpha (2026-05-18)
- Nginx Proxy Manager stale-port repair now detects stopped or `Created` Podman records by inspecting `podman ps -a` port metadata, covering records where `podman port nginx-proxy-manager` returns no mapping until start.
- Live recovery on `100.70.96.88` removed only the stale Nginx Proxy Manager container record and recreated it with `8081:81`, `8084:80`, and `8444:443`, preserving `/var/lib/archipelago/nginx-proxy-manager` data.
- Validation confirmed Nginx Proxy Manager recovered as healthy and responds through direct admin port `8081`, host compatibility port `81`, and `/app/nginx-proxy-manager/`.
## v1.7.65-alpha (2026-05-18)
- Orchestrator-backed app starts now run the same pre-start repairs as the legacy Podman path, so Nginx Proxy Manager stale `81:81` container metadata is removed and recreated before the orchestrator tries to start it.
- Live diagnostics on `100.70.96.88` confirmed host nginx is healthy while Nginx Proxy Manager has no listeners on `8081`, `8084`, or `8444`, causing host nginx `502` responses for NPM proxy paths.
## v1.7.64-alpha (2026-05-18)
- Update apply rate limiting is relaxed for authenticated admins from 2 attempts per 10 minutes to 10 attempts per minute, preventing the System Update page from getting stuck behind `429 Too Many Requests` during legitimate OTA retry/troubleshooting flows.
- The corrected backend artifact rebuild protection from `v1.7.63-alpha` remains in place, so this release is built from a fresh Rust backend binary before publishing.
## v1.7.63-alpha (2026-05-18)
- Release automation now rebuilds the Rust backend after bumping the version and before hashing release artifacts, preventing OTA manifests from pointing at a stale backend binary.
- This corrected release carries the Nginx Proxy Manager stale-port repair in an updated backend binary, so nodes running `1.7.61-alpha` can actually receive and execute the fix.
- Validation confirmed the previously published `v1.7.62-alpha` backend artifact still contained `1.7.61-alpha`, explaining why nodes did not advance after applying that update.
## v1.7.62-alpha (2026-05-18)
- Nginx Proxy Manager start and restart now repair stale Podman containers that still publish the admin UI on host port `81`, which conflicts with host nginx on updated nodes.
- The repair recreates only the stale Nginx Proxy Manager container metadata while preserving `/var/lib/archipelago/nginx-proxy-manager` data and using the current `8081:81`, `8084:80`, and `8444:443` mappings.
- Runtime stale-listener cleanup for Nginx Proxy Manager is shared across start and restart paths so rootless port helper leftovers are still cleared before lifecycle retries.
- Validation passed with `cargo fmt --all --check --manifest-path core/Cargo.toml` and `cargo check -p archipelago --manifest-path core/Cargo.toml`.
## v1.7.61-alpha (2026-05-18)
- Multi-container stack installs now keep their app card in the `Installing` state for up to 20 minutes while dependency containers are being pulled and prepared.
- BTCPay Server installs no longer appear to vanish or fail after two minutes while Postgres and NBXplorer are still being created before the primary `btcpay-server` container exists.
- The stale-transition escape hatch remains short for start, stop, restart, update, and removal operations, so genuinely wedged lifecycle actions still recover quickly.
- Live validation on `100.70.96.88` confirmed BTCPay Server completed installation and responds on port `23000` with the expected HTTP redirect.
## v1.7.60-alpha (2026-05-18)
- Meshtastic serial detection now rejects malformed or incomplete handshakes instead of accepting unrelated serial devices as a fallback Meshtastic radio.
- Mesh radio auto-detection now skips known non-mesh serial devices such as Sierra Wireless LTE modems and Zooz/Z-Wave sticks, avoiding interference with production peripherals.
- Meshtastic config sync now sends `want_config_id` with the correct protobuf wire type, fixing radio-side `ignore malformed toradio` errors and allowing node-info/contact ingestion.
- The stable `/dev/mesh-radio` udev rule no longer claims every `ttyACM*` device; it only matches known mesh USB serial adapters and known USB CDC ACM radio vendors.
- Live validation on `100.70.96.88` confirmed Archipelago selects `/dev/ttyUSB0`, identifies the Meshtastic node, and refreshes 103 mesh contacts.
## v1.7.59-alpha (2026-05-17)
- Mobile app launching now keeps known container apps inside Archipelago's app-session flow instead of forcing desktop-only new-tab behavior on phones.
- App sessions on mobile now respect the status-bar safe area so foreground iframe content starts below the device chrome while the fullscreen backdrop remains edge-to-edge.
- Prepackaged website launch buttons now resolve their curated website URLs before website-container fallback logic, restoring launches for the L484 sites and adding the Arch Presentation bookmark.
- Meshtastic contact discovery now drains the radio config stream through completion and retries config sync when the contact cache is empty, so nearby nodes already known by the radio are more likely to appear in Archipelago.
- The Apps page now includes a compact sideload button and modal for installing trusted Docker images with optional title, description, and port mapping metadata.
- Sideloaded app title and description metadata now persist through the backend app-config file so refreshed package scans do not collapse custom apps back to generic IDs.
- Validation passed with `npm test -- appLauncher`, `npm run build`, `cargo check -p archipelago`, and `cargo fmt --all --check`.
## v1.7.58-alpha (2026-05-17)
- Mesh networking now supports Meshtastic radios over the Meshtastic serial API in addition to existing MeshCore Companion USB radios.
- The mesh listener now probes preferred and auto-detected serial paths for both MeshCore and Meshtastic firmware, preserving the existing reconnect loop so unplug/replug and firmware hot-swap behavior stays consistent.
- Meshtastic text packets are translated into the existing Archipelago mesh frame pipeline, so current RPC handlers, transport routing, message storage, typed-message decoding, and UI state continue to work without a separate frontend path.
- Meshtastic node information is surfaced as normal mesh contacts using stable synthetic public keys derived from Meshtastic node numbers, allowing peer refresh and message attribution to reuse existing MeshCore contact handling.
- Outbound Archipelago mesh messages can now be sent through Meshtastic as channel text packets using the same command path used by MeshCore channel broadcasts.
- Device status now reports the detected firmware family as `meshcore` or `meshtastic` from the shared listener abstraction.
- Radio udev rules now include USB CDC ACM serial devices (`ttyACM*`) alongside CP2102, CH340, and FTDI adapters so Meshtastic boards are more likely to appear through the stable `/dev/mesh-radio` symlink.
- Host nginx now serves `/assets/*` hashed frontend chunks as immutable static files with a hard 404 on misses instead of falling back to `index.html`, preventing strict MIME errors when a browser has a stale pre-update HTML shell.
- The SPA HTML shell and service-worker files now revalidate on every load, reducing stale frontend references after OTA updates.
- OTA runtime promotion now installs the bundled `nginx-archipelago.conf` into `/etc/nginx/sites-available/archipelago` and reloads nginx after a successful config test, so frontend cache/fallback fixes reach existing nodes without a manual deploy.
- Local validation passed with `cargo check -p archipelago`; live SSH testing against `100.70.96.88` was not completed because temporary public-key authentication was rejected on the target.
## v1.7.57-alpha (2026-05-17)
- Nginx Proxy Manager now avoids privileged rootless Podman host port `81`, preferring `8081:81` while host nginx keeps a compatibility proxy on `:81` for stale cached launch buttons.
- App installs now allocate ports by checking live host bind availability, falling back to a free high port when preferred ports are already occupied.
- Portainer-created launchable containers are separated into a `Websites` tab and launch through their discovered published host port instead of hard-coded app URLs.
- Internal BuildKit helper containers such as `buildx_buildkit_default` are hidden from the Apps UI.
- Portainer works out of the box on Debian 13/Podman installs by including `catatonit` and by preserving the Podman socket mount as a socket rather than creating it as a directory.
## v1.7.56-alpha (2026-05-15)
- Health notifications now clear when an app is no longer unhealthy, including stale alerts for removed containers such as Portainer.
- Fresh installs now include the full Wi-Fi userspace stack (`wpasupplicant`, `wireless-regdb`, `iw`, `rfkill`, `polkitd`, `pciutils`, and `usbutils`) so NetworkManager can scan and connect with Intel Wi-Fi cards out of the box.
- The installed system now grants the `archipelago` service user explicit NetworkManager PolicyKit access for web-triggered Wi-Fi scans and connection changes.
- Wi-Fi connect now replaces stale/partial NetworkManager profiles and creates an explicit WPA-PSK profile with the supplied password, avoiding no-secret retry failures after a failed attempt.
- Settings password changes now update the Linux/SSH password through non-interactive sudo, so the web password and SSH password stay in sync when the checkbox is enabled.
- Quadlet environment values with spaces or shell metacharacters are quoted consistently, preventing env drift recreate loops for apps like nostr-rs-relay and Grafana.
- Boot/bootstrap reconcile avoids restarting running Bitcoin containers while repairing RPC config, preserving IBD progress on active nodes.
- Exit code 137 is labeled as SIGKILL instead of assuming OOM, avoiding false OOM alerts for orchestrator-managed recreates.
- Container reconcile force-recreates Podman records stuck in `Stopping`, preserving bind-mounted app data while recovering wedged containers automatically.
- Container health reporting is honest for running containers: Archipelago surfaces Podman's actual health state instead of marking every running container healthy.
- Quadlet reconciliation restarts services when stale health gates, port bindings, network aliases, exec commands, or healthchecks drift from the current manifest.
- Bitcoin Knots sync performance improves on fresh installs and updates with 8Gi container memory, a 4Gi dbcache, and full CPU parallelism.
- ElectrumX initial indexing gets more headroom: CPU caps are removed, memory is raised to 4Gi, cache is raised to 3Gi, and oversized sends are allowed for heavier wallet/indexing workloads.
- Mempool/ElectrumX lifecycle qualification respects pruned/non-archival Bitcoin nodes instead of installing a half-running stack with unhealthy dependencies.
- LND wallet/RPC helpers are more tolerant of container-owned files and updated REST port metadata, improving LND lifecycle and wallet-connect flows.
- Marketplace/catalog metadata carries richer container config so remote lifecycle tests install apps using the same settings users get from the UI.
- The app screensaver no longer activates during media-heavy app sessions such as IndeeHub, Jellyfin, Immich, PhotoPrism, and File Browser; apps can also pause/resume it with media playback messages.
- A fresh `1.7.56-alpha` unbundled installer ISO is built from the same primary VPS2 release line for easy download and USB flashing.
## v1.7.55-alpha (2026-05-13)
- Container reconcile now force-recreates Podman records stuck in `Stopping`, preserving bind-mounted app data while recovering wedged containers automatically.
- `.198` is green after the container-layer hardening pass: focused and broad non-destructive lifecycle audits pass, raw Podman health/state sweep is clean, and direct app probes return healthy responses.
- Release-candidate artifacts are staged separately from live update publishing while Gitea artifact hosting is repaired.
## v1.7.54-alpha (2026-05-06)
- Existing installs now self-repair nginx backend proxy locations for `/bitcoin-status` and `/api/app-catalog`, including hosts where `sites-enabled/archipelago` is a copied active file instead of a symlink.
- LND UI is consistently served on `18083` across first boot, Tor config, companion Quadlet reconciliation, OTA runtime payloads, and ISO scripts; stale companion units/images are rewritten instead of only checking service active state.
- OTA frontend tarballs now carry a clean runtime payload with updated scripts, docker UI sources, and canonical nginx config, preventing startup promotion from reintroducing stale host assets.
- Release ISO builds now support the primary HTTP app registry when bundling core images, so unbundled media includes File Browser/Cloud support instead of requiring a post-install Marketplace download.
- `.116` was live-updated with the new backend and runtime scripts; focused non-destructive lifecycle audit passes for Bitcoin Knots, LND, BTCPay, Mempool, and Grafana.
## v1.7.53-alpha (2026-05-05)
- Bitcoin Knots/Core config generation no longer duplicates RPC bind and port settings between `bitcoin.conf` and container command args, fixing `Unable to bind all endpoints for RPC server` startup failures.
- Legacy Bitcoin container healthchecks no longer depend on `bitcoin-cli`, which is absent from current Knots images and can wedge Podman healthcheck runners.
- Update checks now prefer manifest OTA releases over stale git remotes unless `ARCHIPELAGO_GIT_UPDATES` is explicitly enabled, so installed nodes can see published releases from the VPS mirror.
## v1.7.52-alpha (2026-05-05)
- Tailscale now launches the local installed web UI on port `8240` and starts `tailscaled` before `tailscale web`, fixing unreachable installs after container creation.
- Grafana install/start/restart now repairs missing rootless host listeners on port `3000`, matching the existing SearXNG, Uptime Kuma, and Gitea recovery path.
- Debian 13/Trixie ISO and disk-install paths now force security updates from `trixie-security` during image/install creation so rebuilt release media includes patched base packages.
- Broad `.198` lifecycle audit passes with the current qualified app set; known absent blockers remain `electrumx`, `photoprism`, `dwn`, and `ollama`.
## v1.7.49-alpha (2026-04-30)
- Bitcoin Knots/Core UI now reports connection, reconnecting, syncing, and error states from a backend status bridge instead of showing a stale "Unable to connect" message while the node is warming up.
- ElectrumX UI now exposes indexed height, local Bitcoin height, known headers, status, and progress source so indexing/waiting states are readable during long initial sync.
- Added container doctor timer and smoke/lifecycle test coverage for Bitcoin Knots/Core, ElectrumX, Mempool, BTCPay/NBXplorer, and UI surface availability.
- Bitcoin Core and Bitcoin Knots are mutually exclusive variants, with a real Bitcoin Core manifest and corrected install conflict handling.
- IndeeHub now launches only on direct web UI port `7778`; the broken `/app/indeedhub/` path proxy was removed, and port `7777` remains the Nostr relay.
- BTCPay/NBXplorer Postgres environment formatting fixed so installs do not carry malformed connection strings.
## v1.7.48-alpha (2026-04-29)
- archipelago.service no longer fails to start with "Failed to set up mount namespacing: /run/containers: No such file or directory" on nodes where /run/containers wasn't pre-created. ExecStartPre now creates it. Existing nodes need a one-time `systemctl edit archipelago` to add the mkdir; ISO installs from this version forward have the fix baked in.
@@ -7,7 +162,7 @@
## v1.7.47-alpha (2026-04-29)
- Bitcoin Knots/Core sync is now significantly faster. The container now uses every available core for script verification (was capped at 2) and has 8GB of memory instead of 4GB so its 4GB UTXO cache has headroom for the mempool and peer connections. Existing nodes pick up the new limits on next install/update; freshly-installed nodes start at full speed.
- ElectrumX initial indexing is faster too. Its container memory bumped from 1GB to 2GB and its internal cache is now 2GB (default was 1.2GB).
- ElectrumX initial indexing is faster too. Its CPU cap is removed, container memory is 4GB, and its internal cache is now 3GB (default was 1.2GB).
## v1.7.46-alpha (2026-04-29)
+74 -16
View File
@@ -26,7 +26,7 @@
"id": "bitcoin-core",
"title": "Bitcoin Core",
"version": "28.4",
"description": "Reference implementation of the Bitcoin protocol. Run a full node validating and relaying blocks.",
"description": "Reference Bitcoin node implementation. Alternative to Bitcoin Knots; uninstall Knots before switching.",
"icon": "/assets/img/app-icons/bitcoin-core.svg",
"author": "Bitcoin Core contributors",
"category": "money",
@@ -52,13 +52,13 @@
{
"id": "btcpay-server",
"title": "BTCPay Server",
"version": "1.13.7",
"version": "2.3.9",
"description": "Self-hosted Bitcoin payment processor.",
"icon": "/assets/img/app-icons/btcpay-server.png",
"author": "BTCPay Server Foundation",
"category": "commerce",
"tier": "core",
"dockerImage": "146.59.87.168:3000/lfg2025/btcpayserver:1.13.7",
"dockerImage": "docker.io/btcpayserver/btcpayserver:2.3.9",
"repoUrl": "https://github.com/btcpayserver/btcpayserver",
"requires": [
"bitcoin-knots"
@@ -85,7 +85,7 @@
"title": "ElectrumX",
"version": "1.18.0",
"description": "Electrum protocol server. Index the blockchain for fast wallet lookups.",
"icon": "/assets/img/app-icons/electrumx.webp",
"icon": "/assets/img/app-icons/electrumx.png",
"author": "Luke Childs",
"category": "money",
"tier": "core",
@@ -115,7 +115,12 @@
"author": "BotFights",
"category": "community",
"dockerImage": "146.59.87.168:3000/lfg2025/botfights:1.1.0",
"repoUrl": "https://botfights.net"
"repoUrl": "https://botfights.net",
"containerConfig": {
"ports": ["9100:9100"],
"volumes": ["/var/lib/archipelago/botfights:/app/server/data"],
"env": ["NODE_ENV=production", "PORT=9100", "FIGHT_LOOP_ENABLED=true", "ARCHY_EMBEDDED=1"]
}
},
{
"id": "gitea",
@@ -126,7 +131,12 @@
"author": "Gitea",
"category": "development",
"dockerImage": "146.59.87.168:3000/lfg2025/gitea:1.23",
"repoUrl": "https://gitea.com"
"repoUrl": "https://gitea.com",
"containerConfig": {
"ports": ["3001:3000", "2222:22"],
"volumes": ["/var/lib/archipelago/gitea/data:/data", "/var/lib/archipelago/gitea/config:/etc/gitea"],
"env": ["GITEA__database__DB_TYPE=sqlite3", "GITEA__server__SSH_PORT=2222", "GITEA__server__SSH_LISTEN_PORT=22", "GITEA__server__LFS_START_SERVER=true", "GITEA__packages__ENABLED=true", "GITEA__repository__ENABLE_PUSH_CREATE_USER=true", "GITEA__repository__ENABLE_PUSH_CREATE_ORG=true", "GITEA__security__X_FRAME_OPTIONS="]
}
},
{
"id": "filebrowser",
@@ -138,7 +148,12 @@
"category": "data",
"tier": "core",
"dockerImage": "146.59.87.168:3000/lfg2025/filebrowser:v2.27.0",
"repoUrl": "https://github.com/filebrowser/filebrowser"
"repoUrl": "https://github.com/filebrowser/filebrowser",
"containerConfig": {
"ports": ["8083:80"],
"volumes": ["/var/lib/archipelago/filebrowser:/srv", "/var/lib/archipelago/filebrowser-data:/data"],
"args": ["--database=/data/database.db", "--root=/srv", "--address=0.0.0.0", "--port=80"]
}
},
{
"id": "vaultwarden",
@@ -150,7 +165,11 @@
"category": "data",
"tier": "recommended",
"dockerImage": "146.59.87.168:3000/lfg2025/vaultwarden:1.30.0-alpine",
"repoUrl": "https://github.com/dani-garcia/vaultwarden"
"repoUrl": "https://github.com/dani-garcia/vaultwarden",
"containerConfig": {
"ports": ["8082:80"],
"volumes": ["/var/lib/archipelago/vaultwarden:/data"]
}
},
{
"id": "searxng",
@@ -162,7 +181,11 @@
"category": "data",
"tier": "recommended",
"dockerImage": "146.59.87.168:3000/lfg2025/searxng:latest",
"repoUrl": "https://github.com/searxng/searxng"
"repoUrl": "https://github.com/searxng/searxng",
"containerConfig": {
"ports": ["8888:8080"],
"volumes": ["/var/lib/archipelago/searxng:/etc/searxng"]
}
},
{
"id": "fedimint",
@@ -184,7 +207,11 @@
"author": "Jellyfin",
"category": "data",
"dockerImage": "146.59.87.168:3000/lfg2025/jellyfin:10.8.13",
"repoUrl": "https://github.com/jellyfin/jellyfin"
"repoUrl": "https://github.com/jellyfin/jellyfin",
"containerConfig": {
"ports": ["8096:8096"],
"volumes": ["/var/lib/archipelago/jellyfin/config:/config", "/var/lib/archipelago/jellyfin/cache:/cache"]
}
},
{
"id": "immich",
@@ -206,7 +233,12 @@
"author": "Home Assistant",
"category": "home",
"dockerImage": "146.59.87.168:3000/lfg2025/home-assistant:2024.1",
"repoUrl": "https://github.com/home-assistant/core"
"repoUrl": "https://github.com/home-assistant/core",
"containerConfig": {
"ports": ["8123:8123"],
"volumes": ["/var/lib/archipelago/home-assistant:/config"],
"env": ["TZ=UTC"]
}
},
{
"id": "grafana",
@@ -218,7 +250,12 @@
"category": "data",
"tier": "recommended",
"dockerImage": "146.59.87.168:3000/lfg2025/grafana:10.2.0",
"repoUrl": "https://github.com/grafana/grafana"
"repoUrl": "https://github.com/grafana/grafana",
"containerConfig": {
"ports": ["3000:3000"],
"volumes": ["/var/lib/archipelago/grafana:/var/lib/grafana"],
"env": ["GF_PATHS_DATA=/var/lib/grafana", "GF_USERS_ALLOW_SIGN_UP=false"]
}
},
{
"id": "tailscale",
@@ -230,7 +267,13 @@
"category": "networking",
"tier": "recommended",
"dockerImage": "146.59.87.168:3000/lfg2025/tailscale:stable",
"repoUrl": "https://github.com/tailscale/tailscale"
"repoUrl": "https://github.com/tailscale/tailscale",
"containerConfig": {
"ports": ["8240:8240"],
"volumes": ["/var/lib/archipelago/tailscale:/var/lib/tailscale"],
"env": ["TS_STATE_DIR=/var/lib/tailscale"],
"args": ["sh", "-c", "tailscaled --tun=userspace-networking & sleep 2; tailscale web --listen 0.0.0.0:8240 & wait"]
}
},
{
"id": "uptime-kuma",
@@ -242,7 +285,13 @@
"category": "data",
"tier": "recommended",
"dockerImage": "146.59.87.168:3000/lfg2025/uptime-kuma:1",
"repoUrl": "https://github.com/louislam/uptime-kuma"
"repoUrl": "https://github.com/louislam/uptime-kuma",
"containerConfig": {
"ports": ["3002:3001"],
"volumes": ["/var/lib/archipelago/uptime-kuma:/app/data"],
"env": ["TZ=UTC"],
"args": ["--", "node", "server/server.js"]
}
},
{
"id": "photoprism",
@@ -253,7 +302,12 @@
"author": "PhotoPrism",
"category": "data",
"dockerImage": "146.59.87.168:3000/lfg2025/photoprism:240915",
"repoUrl": "https://github.com/photoprism/photoprism"
"repoUrl": "https://github.com/photoprism/photoprism",
"containerConfig": {
"ports": ["2342:2342"],
"volumes": ["/var/lib/archipelago/photoprism:/photoprism/storage"],
"env": ["PHOTOPRISM_ADMIN_PASSWORD=archipelago", "PHOTOPRISM_DEFAULT_LOCALE=en"]
}
},
{
"id": "nextcloud",
@@ -264,7 +318,11 @@
"author": "Nextcloud",
"category": "data",
"dockerImage": "146.59.87.168:3000/lfg2025/nextcloud:28",
"repoUrl": "https://github.com/nextcloud/server"
"repoUrl": "https://github.com/nextcloud/server",
"containerConfig": {
"ports": ["8085:80"],
"volumes": ["/var/lib/archipelago/nextcloud:/var/www/html"]
}
}
]
}
+1 -1
View File
@@ -19,7 +19,7 @@ This document lists all port assignments for Archipelago apps.
| searxng | 8888 | TCP | Web UI | 18888 |
| onlyoffice | 8088 | TCP | Web UI | 18088 |
| penpot | 8089 | TCP | Web UI | 18089 |
| lnd | 9735, 10009, 8080 | TCP | P2P, gRPC, REST | 19735, 20009, 18080 |
| lnd | 9735, 10009, 18080 | TCP | P2P, gRPC, REST | 19735, 20009, 28080 |
| core-lightning | 9736, 9835 | TCP | P2P, gRPC | 19736, 19835 |
| nostr-rs-relay | 8081 | TCP | HTTP/WebSocket | 18081 |
| strfry | 8082 | TCP | HTTP/WebSocket | 18082 |
-6
View File
@@ -27,12 +27,6 @@ app:
container: 8080
protocol: tcp
volumes:
- type: bind
source: /var/lib/archipelago/mempool/nginx.conf
target: /etc/nginx/conf.d/default.conf
options: [ro]
environment:
- FRONTEND_HTTP_PORT=8080
- BACKEND_MAINNET_HTTP_HOST=mempool-api
+3 -1
View File
@@ -47,7 +47,9 @@ app:
- NBXPLORER_BIND=0.0.0.0:32838
- NBXPLORER_BTCRPCURL=http://bitcoin-knots:8332
- NBXPLORER_BTCRPCUSER=archipelago
- NBXPLORER_POSTGRES=User ID=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=nbxplorer;Include Error Detail=true
- NBXPLORER_BTCNODEENDPOINT=bitcoin-knots:8333
- NBXPLORER_NOAUTH=1
- NBXPLORER_POSTGRES=Username=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=nbxplorer
health_check:
type: http
+15 -7
View File
@@ -1,27 +1,35 @@
app:
id: bitcoin-core
name: Bitcoin Knots
name: Bitcoin Core
version: 28.4.0
description: Full Bitcoin Knots node with dynamic prune/full-mode startup based on host disk.
description: Reference Bitcoin Core node with dynamic prune/full-mode startup based on host disk.
container_name: bitcoin-knots
container_name: bitcoin-core
container:
image: 146.59.87.168:3000/lfg2025/bitcoin-knots:latest
image: 146.59.87.168:3000/lfg2025/bitcoin:28.4
pull_policy: if-not-present
network: archy-net
entrypoint: ["sh", "-lc"]
custom_args:
# Sync-speed flags: -par=0 uses every core (was capped at 2 by
# --cpus=2, now removed for bitcoin/electrumx). -dbcache sized to
# the IBD sweet spot 4GB on full nodes, 1GB on pruned. Container
# the IBD sweet spot - 4GB on full nodes, 1GB on pruned. Container
# --memory=8g (config.rs::get_memory_limit) leaves headroom for
# mempool + connections.
- >-
BITCOIND="$(command -v bitcoind || true)";
if [ -z "$BITCOIND" ]; then
BITCOIND="$(find /opt -path '*/bin/bitcoind' -type f 2>/dev/null | sort | tail -n 1)";
fi;
if [ -z "$BITCOIND" ]; then
echo "bitcoind not found in image" >&2;
exit 127;
fi;
if [ "${DISK_GB:-0}" -lt 1000 ]; then
exec bitcoind -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=1024 -par=0 -maxconnections=125 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=1024 -par=0 -maxconnections=125 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
else
exec bitcoind -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
fi
derived_env:
- key: DISK_GB
+86
View File
@@ -0,0 +1,86 @@
app:
id: bitcoin-knots
name: Bitcoin Knots
version: 28.1.0
description: Full Bitcoin Knots node with dynamic prune/full-mode startup based on host disk.
container_name: bitcoin-knots
container:
image: 146.59.87.168:3000/lfg2025/bitcoin-knots:latest
pull_policy: if-not-present
network: archy-net
entrypoint: ["sh", "-lc"]
custom_args:
# Sync-speed flags: -par=0 uses every core (was capped at 2 by
# --cpus=2, now removed for bitcoin/electrumx). -dbcache sized to
# the IBD sweet spot - 4GB on full nodes, 1GB on pruned. Container
# --memory=8g (config.rs::get_memory_limit) leaves headroom for
# mempool + connections.
- >-
BITCOIND="$(command -v bitcoind || true)";
if [ -z "$BITCOIND" ]; then
BITCOIND="$(find /opt -path '*/bin/bitcoind' -type f 2>/dev/null | sort | tail -n 1)";
fi;
if [ -z "$BITCOIND" ]; then
echo "bitcoind not found in image" >&2;
exit 127;
fi;
RPC_USER="$(printenv BITCOIN_RPC_USER)";
RPC_PASS="$(printenv BITCOIN_RPC_PASS)";
DISK_GB_VALUE="$(printenv DISK_GB || true)";
if [ "${DISK_GB_VALUE:-0}" -lt 1000 ]; then
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=2048 -par=0 -maxconnections=125 -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
else
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
fi
derived_env:
- key: DISK_GB
template: "{{DISK_GB}}"
secret_env:
- key: BITCOIN_RPC_PASS
secret_file: bitcoin-rpc-password
data_uid: "100101:100101"
dependencies:
- storage: 500Gi
resources:
cpu_limit: 0
memory_limit: 8Gi
disk_limit: 500Gi
security:
capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE]
readonly_root: false
network_policy: isolated
ports:
- host: 8332
container: 8332
protocol: tcp
- host: 8333
container: 8333
protocol: tcp
volumes:
- type: bind
source: /var/lib/archipelago/bitcoin
target: /home/bitcoin/.bitcoin
options: [rw]
environment:
- BITCOIN_RPC_USER=archipelago
health_check:
type: tcp
endpoint: localhost:8332
interval: 30s
timeout: 5s
retries: 3
bitcoin_integration:
rpc_access: admin
sync_required: true
testnet_support: false
pruning_support: true
+15 -4
View File
@@ -1,11 +1,11 @@
app:
id: btcpay-server
name: BTCPay Server
version: 1.13.7
version: 2.3.9
description: Self-hosted Bitcoin payment processor. Accept Bitcoin payments without intermediaries.
container:
image: git.tx1138.com/lfg2025/btcpayserver:1.13.7
image: docker.io/btcpayserver/btcpayserver:2.3.9
pull_policy: if-not-present
network: archy-net
secret_env:
@@ -13,6 +13,9 @@ app:
secret_file: bitcoin-rpc-password
- key: BTCPAY_DB_PASS
secret_file: btcpay-db-password
derived_env:
- key: BTCPAY_HOST
template: "{{HOST_IP}}:23000"
dependencies:
- app_id: bitcoin-core
@@ -46,12 +49,11 @@ app:
environment:
- ASPNETCORE_URLS=http://0.0.0.0:49392
- BTCPAY_PROTOCOL=http
- BTCPAY_HOST=127.0.0.1:23000
- BTCPAY_CHAINS=btc
- BTCPAY_BTCEXPLORERURL=http://archy-nbxplorer:32838
- BTCPAY_BTCRPCURL=http://bitcoin-knots:8332
- BTCPAY_BTCRPCUSER=archipelago
- BTCPAY_POSTGRES=User ID=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=btcpay;Include Error Detail=true
- BTCPAY_POSTGRES=Username=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=btcpay
health_check:
type: http
@@ -68,3 +70,12 @@ app:
lightning_integration:
payment_processing: false
invoice_management: true
interfaces:
main:
name: Web UI
description: BTCPay Server dashboard
type: ui
port: 23000
protocol: http
path: /
+8 -4
View File
@@ -8,23 +8,24 @@ app:
image: git.tx1138.com/lfg2025/electrumx:v1.18.0
pull_policy: if-not-present
network: archy-net
data_uid: "1000:1000"
entrypoint: ["sh", "-lc"]
custom_args:
- >-
export DAEMON_URL="http://archipelago:${BITCOIN_RPC_PASS}@bitcoin-knots:8332/";
export DAEMON_URL="http://archipelago:$(printenv BITCOIN_RPC_PASS)@bitcoin-knots:8332/";
exec electrumx_server
secret_env:
- key: BITCOIN_RPC_PASS
secret_file: bitcoin-rpc-password
dependencies:
- app_id: bitcoin-core
- app_id: bitcoin-knots
version: ">=26.0"
- storage: 50Gi
resources:
cpu_limit: 2
memory_limit: 2Gi
cpu_limit: 0
memory_limit: 4Gi
disk_limit: 50Gi
security:
@@ -47,6 +48,8 @@ app:
- COIN=Bitcoin
- DB_DIRECTORY=/data
- SERVICES=tcp://:50001,rpc://0.0.0.0:8000
- CACHE_MB=3072
- MAX_SEND=10000000
health_check:
type: tcp
@@ -58,3 +61,4 @@ app:
bitcoin_integration:
rpc_access: read-only
sync_required: true
pruning_support: false
+3 -3
View File
@@ -12,16 +12,16 @@ app:
custom_args:
- >-
if [ -f /lnd/tls.cert ] && [ -f /lnd/data/chain/bitcoin/mainnet/admin.macaroon ]; then
exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://bitcoin-knots:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" lnd --lnd-rpc-host lnd:10009 --lnd-tls-cert /lnd/tls.cert --lnd-macaroon /lnd/data/chain/bitcoin/mainnet/admin.macaroon;
exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://host.archipelago:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" lnd --lnd-rpc-host lnd:10009 --lnd-tls-cert /lnd/tls.cert --lnd-macaroon /lnd/data/chain/bitcoin/mainnet/admin.macaroon;
else
exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://bitcoin-knots:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" ldk --ldk-lightning-port 9737 --ldk-alias archipelago-gateway;
exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://host.archipelago:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" ldk --ldk-lightning-port 9737 --ldk-alias archipelago-gateway;
fi
secret_env:
- key: FM_BITCOIND_PASSWORD
secret_file: bitcoin-rpc-password
- key: FEDI_HASH
secret_file: fedimint-gateway-hash
data_uid: "100000:100000"
data_uid: "1000:1000"
dependencies:
- app_id: bitcoin-core
+2 -2
View File
@@ -16,7 +16,7 @@ app:
secret_env:
- key: FM_BITCOIND_PASSWORD
secret_file: bitcoin-rpc-password
data_uid: "100000:100000"
data_uid: "1000:1000"
dependencies:
- app_id: bitcoin-core
@@ -52,7 +52,7 @@ app:
environment:
- FM_DATA_DIR=/data
- FM_BITCOIND_URL=http://bitcoin-knots:8332
- FM_BITCOIND_URL=http://host.archipelago:8332
- FM_BITCOIND_USERNAME=archipelago
- FM_BITCOIN_NETWORK=bitcoin
- FM_BIND_P2P=0.0.0.0:8173
+2 -3
View File
@@ -29,9 +29,8 @@ environment:
GITEA__repository__ENABLE_PUSH_CREATE_USER: "true"
GITEA__repository__ENABLE_PUSH_CREATE_ORG: "true"
# Gitea hardcodes X-Frame-Options: SAMEORIGIN which blocks iframe embedding.
# Container binds to internal_port (3001), nginx proxies public port (3000)
# stripping the X-Frame-Options header so the app works in Archipelago's iframe.
# Gitea hardcodes X-Frame-Options: SAMEORIGIN, so Archipelago opens it in a
# new tab on host port 3001 instead of embedding it in an iframe.
nginx_proxy:
listen: 3000
proxy_pass: "http://127.0.0.1:3001"
+4 -3
View File
@@ -8,6 +8,7 @@ app:
image: grafana/grafana:10.2.0
image_signature: cosign://...
pull_policy: if-not-present
data_uid: "472:472"
dependencies:
- storage: 5Gi
@@ -27,7 +28,7 @@ app:
apparmor_profile: grafana
ports:
- host: 3001
- host: 3000
container: 3000
protocol: tcp # Web UI
@@ -40,12 +41,12 @@ app:
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}
- GF_SERVER_ROOT_URL=http://localhost:3001
- GF_SERVER_ROOT_URL=http://localhost:3000
- GF_INSTALL_PLUGINS=
health_check:
type: http
endpoint: http://localhost:3001
endpoint: http://localhost:3000
path: /api/health
interval: 30s
timeout: 5s
+1 -1
View File
@@ -53,7 +53,7 @@ app:
health_check:
type: http
endpoint: http://localhost:8123
path: /api/
path: /
interval: 30s
timeout: 5s
retries: 3
+11 -4
View File
@@ -8,6 +8,7 @@ app:
container:
image: 146.59.87.168:3000/lfg2025/indeedhub:latest
pull_policy: always # Pull from registry; falls back to local build
network: indeedhub-net
dependencies:
- storage: 1Gi
@@ -27,9 +28,9 @@ app:
apparmor_profile: default
ports:
- host: 7777
container: 3000
protocol: tcp # Web UI (Next.js)
- host: 7778
container: 7777
protocol: tcp # Web UI. Port 7777 on the host is reserved for Nostr relay.
volumes:
- type: tmpfs
@@ -38,6 +39,12 @@ app:
- type: tmpfs
target: /app/.next/cache
options: [rw,noexec,nosuid,size=128m]
- type: tmpfs
target: /run
options: [rw,nosuid,nodev,size=16m]
- type: tmpfs
target: /var/cache/nginx
options: [rw,nosuid,nodev,size=32m]
environment:
- NODE_ENV=production
@@ -57,7 +64,7 @@ app:
name: Web UI
description: Stream Bitcoin documentaries with Nostr identity
type: ui
port: 7777
port: 7778
protocol: http
path: /
+3 -3
View File
@@ -25,9 +25,9 @@ app:
network_policy: bridge
# Bridge networking via archy-net. Container nginx listens on 80;
# host nginx proxies /app/lnd/ -> 127.0.0.1:8081 -> container:80.
# host nginx proxies /app/lnd/ -> 127.0.0.1:18083 -> container:80.
ports:
- host: 8081
- host: 18083
container: 80
protocol: tcp
@@ -37,7 +37,7 @@ app:
health_check:
type: http
endpoint: http://127.0.0.1:8081
endpoint: http://127.0.0.1:18083
path: /
interval: 30s
timeout: 5s
+1 -1
View File
@@ -34,7 +34,7 @@ app:
- host: 10009
container: 10009
protocol: tcp
- host: 8080
- host: 18080
container: 8080
protocol: tcp
+3 -2
View File
@@ -15,7 +15,7 @@ app:
secret_file: mempool-db-password
dependencies:
- app_id: bitcoin-core
- app_id: bitcoin-knots
version: ">=26.0"
- app_id: electrumx
version: ">=1.18.0"
@@ -58,7 +58,7 @@ app:
health_check:
type: http
endpoint: http://localhost:8999
path: /
path: /api/v1/backend-info
interval: 30s
timeout: 5s
retries: 3
@@ -66,3 +66,4 @@ app:
bitcoin_integration:
rpc_access: read-only
sync_required: true
pruning_support: false
+4 -3
View File
@@ -8,6 +8,7 @@ app:
image: scsibug/nostr-rs-relay:0.8.9
image_signature: cosign://...
pull_policy: verify-signature
data_uid: "1000:1000"
dependencies:
- storage: 10Gi # For event storage
@@ -34,7 +35,7 @@ app:
volumes:
- type: bind
source: /var/lib/archipelago/nostr-relay
target: /app/db
target: /usr/src/app/db
options: [rw]
environment:
@@ -45,8 +46,8 @@ app:
health_check:
type: http
endpoint: http://localhost:8081
path: /health
endpoint: http://localhost:8080
path: /
interval: 30s
timeout: 5s
retries: 3
+3 -4
View File
@@ -1,12 +1,11 @@
app:
id: searxng
name: SearXNG
version: 2024.1.0
version: 1.0.0
description: Privacy-respecting metasearch engine. Search the web without tracking.
container:
image: searxng/searxng:2024.1.0
image_signature: cosign://...
image: 146.59.87.168:3000/lfg2025/searxng:latest
pull_policy: if-not-present
dependencies:
@@ -43,7 +42,7 @@ app:
health_check:
type: http
endpoint: http://localhost:8888
endpoint: http://localhost:8080
path: /
interval: 30s
timeout: 5s
+1 -1
View File
@@ -80,7 +80,7 @@ checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61"
[[package]]
name = "archipelago"
version = "1.7.48-alpha"
version = "1.7.67-alpha"
dependencies = [
"anyhow",
"archipelago-container",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "archipelago"
version = "1.7.48-alpha"
version = "1.7.68-alpha"
edition = "2021"
description = "Archipelago Bitcoin Node OS - Native backend"
authors = ["Archipelago Team"]
+1
View File
@@ -429,6 +429,7 @@ impl ApiHandler {
// Electrs status — unauthenticated (read-only sync status)
(Method::GET, "/electrs-status") => Self::handle_electrs_status().await,
(Method::GET, "/bitcoin-status") => Self::handle_bitcoin_status().await,
// App-catalog proxy — fetches catalog.json from the configured
// upstream URLs server-side so the browser doesn't hit CORS
+20 -6
View File
@@ -1,5 +1,7 @@
use super::build_response;
use crate::api::rpc::lnd::LND_REST_BASE_URL;
use crate::api::rpc::RpcHandler;
use crate::bitcoin_status;
use crate::electrs_status;
use anyhow::Result;
use hyper::{Response, StatusCode};
@@ -76,11 +78,23 @@ impl ApiHandler {
pub(super) async fn handle_electrs_status() -> Result<Response<hyper::Body>> {
let status = electrs_status::get_electrs_sync_status().await;
let body = serde_json::to_vec(&status).unwrap_or_default();
Ok(build_response(
StatusCode::OK,
"application/json",
hyper::Body::from(body),
))
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/json")
.header("Cache-Control", "no-store")
.body(hyper::Body::from(body))
.unwrap_or_else(|_| Response::new(hyper::Body::from("{}"))))
}
pub(super) async fn handle_bitcoin_status() -> Result<Response<hyper::Body>> {
let status = bitcoin_status::get_bitcoin_status().await;
let body = serde_json::to_vec(&status).unwrap_or_default();
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/json")
.header("Cache-Control", "no-store")
.body(hyper::Body::from(body))
.unwrap_or_else(|_| Response::new(hyper::Body::from("{}"))))
}
pub(super) async fn handle_lnd_connect_info(
@@ -110,7 +124,7 @@ impl ApiHandler {
cors_origin: &str,
) -> Result<Response<hyper::Body>> {
let suffix = path.strip_prefix("/proxy/lnd").unwrap_or("/");
let url = format!("http://127.0.0.1:8080{}", suffix);
let url = format!("{LND_REST_BASE_URL}{suffix}");
match reqwest::get(&url).await {
Ok(resp) => {
let status = resp.status().as_u16();
+201 -13
View File
@@ -2,6 +2,10 @@ use super::package::validate_app_id;
use super::transitional::Op;
use super::RpcHandler;
use anyhow::{Context, Result};
use std::time::Duration;
const PODMAN_INSPECT_TIMEOUT: Duration = Duration::from_secs(10);
const PODMAN_PS_TIMEOUT: Duration = Duration::from_secs(10);
impl RpcHandler {
pub(super) async fn handle_container_install(
@@ -379,11 +383,26 @@ impl RpcHandler {
// If app_id is provided, get health for that app.
if let Some(params) = params {
if let Some(app_id) = params.get("app_id").and_then(|v| v.as_str()) {
let health = orchestrator
.health(app_id)
.await
.context("Failed to get container health")?;
return Ok(serde_json::json!({ app_id: health }));
if let Some(health) = self.stack_health(app_id).await? {
return Ok(serde_json::json!({ app_id: health }));
}
let mut last_err: Option<anyhow::Error> = None;
for candidate in status_app_id_candidates(app_id) {
match orchestrator.health(&candidate).await {
Ok(health) => return Ok(serde_json::json!({ app_id: health })),
Err(e) => last_err = Some(e),
}
}
for name in status_container_name_candidates(app_id) {
if let Some(health) = inspect_container_health_value(&name).await {
return Ok(serde_json::json!({ app_id: health }));
}
}
if let Some(e) = last_err {
return Err(e.context("Failed to get container health"));
}
return Err(anyhow::anyhow!("Failed to get container health"));
}
}
@@ -423,6 +442,78 @@ impl RpcHandler {
Ok(serde_json::Value::Object(health_map))
}
async fn stack_health(&self, app_id: &str) -> Result<Option<String>> {
let Some(members) = stack_health_members(app_id) else {
return Ok(None);
};
let orchestrator = self
.orchestrator
.as_ref()
.ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
let mut saw_starting = false;
let mut saw_unknown = false;
for member in members {
match member_health(orchestrator.as_ref(), member)
.await
.as_deref()
{
Ok(health) if health == "healthy" => {}
Ok(health) if health == "starting" => saw_starting = true,
Ok(health) if health == "unknown" => saw_unknown = true,
Ok(_) => return Ok(Some("unhealthy".to_string())),
Err(_) => saw_unknown = true,
}
}
if saw_unknown {
Ok(Some("unknown".to_string()))
} else if saw_starting {
Ok(Some("starting".to_string()))
} else {
Ok(Some("healthy".to_string()))
}
}
}
async fn member_health(
orchestrator: &dyn crate::container::traits::ContainerOrchestrator,
app_id: &str,
) -> Result<String> {
if let Ok(health) = orchestrator.health(app_id).await {
return Ok(health);
}
for name in status_container_name_candidates(app_id) {
if let Some(health) = inspect_container_health_value(&name).await {
return Ok(health);
}
}
Ok("unknown".to_string())
}
fn stack_health_members(app_id: &str) -> Option<&'static [&'static str]> {
match app_id {
"mempool" | "mempool-web" => {
Some(&["archy-mempool-db", "mempool-api", "archy-mempool-web"])
}
"btcpay-server" | "btcpayserver" | "btcpay" => {
Some(&["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"])
}
"immich" => Some(&["immich_postgres", "immich_redis", "immich_server"]),
"indeedhub" => Some(&[
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
]),
"fedimint" => Some(&["fedimint"]),
_ => None,
}
}
fn status_app_id_candidates(app_id: &str) -> Vec<String> {
@@ -449,6 +540,14 @@ fn status_app_id_candidates(app_id: &str) -> Vec<String> {
push("mempool-electrs");
push("electrumx");
}
"mempool" | "mempool-web" => {
push("mempool");
push("archy-mempool-web");
}
"immich" => {
push("immich");
push("immich_server");
}
_ => push(app_id),
}
@@ -469,6 +568,8 @@ fn status_container_name_candidates(app_id: &str) -> Vec<String> {
"lnd-ui" => push("archy-lnd-ui"),
"electrs-ui" => push("archy-electrs-ui"),
"electrs" | "mempool-electrs" => push("electrumx"),
"mempool" | "mempool-web" | "archy-mempool-web" => push("mempool"),
"immich" => push("immich_server"),
_ => {}
}
@@ -483,15 +584,21 @@ fn status_container_name_candidates(app_id: &str) -> Vec<String> {
}
async fn inspect_container_state_value(name: &str) -> Option<serde_json::Value> {
let out = tokio::process::Command::new("podman")
.args([
"inspect",
name,
"--format",
"{{.State.Status}} {{.State.Running}}",
])
.output()
if let Some(v) = ps_container_state_value(name).await {
return Some(v);
}
let mut cmd = tokio::process::Command::new("podman");
cmd.args([
"inspect",
name,
"--format",
"{{.State.Status}} {{.State.Running}} {{if .State.Healthcheck}}{{.State.Healthcheck.Status}}{{else}}none{{end}}",
]);
cmd.kill_on_drop(true);
let out = tokio::time::timeout(PODMAN_INSPECT_TIMEOUT, cmd.output())
.await
.ok()?
.ok()?;
if !out.status.success() {
return None;
@@ -504,10 +611,91 @@ async fn inspect_container_state_value(name: &str) -> Option<serde_json::Value>
let mut parts = line.split_whitespace();
let status = parts.next().unwrap_or("unknown");
let running = parts.next().unwrap_or("false") == "true";
let health = parts.next().unwrap_or("none");
Some(serde_json::json!({
"name": name,
"status": status,
"state": status,
"running": running,
"health": health,
}))
}
async fn ps_container_state_value(name: &str) -> Option<serde_json::Value> {
let mut cmd = tokio::process::Command::new("podman");
cmd.args([
"ps",
"-a",
"--filter",
&format!("name={name}"),
"--format",
"{{.Names}}|{{.Status}}",
]);
cmd.kill_on_drop(true);
let out = tokio::time::timeout(PODMAN_PS_TIMEOUT, cmd.output())
.await
.ok()?
.ok()?;
if !out.status.success() {
return None;
}
let stdout = String::from_utf8_lossy(&out.stdout);
for line in stdout.lines() {
let mut parts = line.splitn(2, '|');
let container_name = parts.next().unwrap_or_default();
if container_name != name {
continue;
}
let status = parts.next().unwrap_or_default();
let state = state_from_podman_status(status);
let health = parse_health_from_status(status).unwrap_or("none");
return Some(serde_json::json!({
"name": name,
"status": state,
"state": state,
"running": state.eq_ignore_ascii_case("running"),
"health": health,
}));
}
None
}
fn state_from_podman_status(status: &str) -> &str {
if status.starts_with("Up ") {
"running"
} else if status.starts_with("Exited ") {
"exited"
} else if status.starts_with("Created") {
"created"
} else if status.starts_with("Stopping") {
"stopping"
} else if status.starts_with("Removing") {
"removing"
} else {
"unknown"
}
}
fn parse_health_from_status(status: &str) -> Option<&str> {
let start = status.rfind('(')?;
let end = status.rfind(')')?;
(start < end).then(|| &status[start + 1..end])
}
async fn inspect_container_health_value(name: &str) -> Option<String> {
let v = inspect_container_state_value(name).await?;
if let Some(health) = v.get("health").and_then(|s| s.as_str()) {
if health != "none" {
return Some(health.to_string());
}
}
match v.get("state").and_then(|s| s.as_str()).unwrap_or("unknown") {
"running" => Some("healthy".to_string()),
"created" => Some("starting".to_string()),
"paused" => Some("paused".to_string()),
"stopping" => Some("unhealthy".to_string()),
"exited" | "stopped" => Some("unhealthy".to_string()),
other => Some(format!("unknown:{other}")),
}
}
@@ -13,6 +13,7 @@ impl RpcHandler {
match method {
"echo" => self.handle_echo(params).await,
"server.echo" => self.handle_echo(params).await,
"server.get-state" => self.handle_server_get_state().await,
"health" => self.handle_health().await,
"auth.login" => self.handle_auth_login(params).await,
"auth.logout" => self.handle_auth_logout().await,
@@ -530,6 +531,11 @@ impl RpcHandler {
Ok(serde_json::json!({ "message": "Hello from Archipelago!" }))
}
async fn handle_server_get_state(&self) -> Result<serde_json::Value> {
let (data, rev) = self.state_manager.get_snapshot().await;
Ok(serde_json::json!({ "data": data, "rev": rev }))
}
pub(super) async fn handle_health(&self) -> Result<serde_json::Value> {
let recovery_complete = crate::crash_recovery::is_recovery_complete();
let uptime = crate::crash_recovery::uptime_seconds();
+4 -11
View File
@@ -98,21 +98,14 @@ impl RpcHandler {
// Anchor bootstrap window: poll the status every ~3s for up to
// 20s. Bail as soon as the anchor is connected.
let mut last_status: Option<fips::FipsStatus> = None;
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(20);
loop {
let after = loop {
tokio::time::sleep(std::time::Duration::from_secs(3)).await;
let s = fips::FipsStatus::query(&self.config.data_dir).await;
if s.anchor_connected {
last_status = Some(s);
break;
if s.anchor_connected || std::time::Instant::now() >= deadline {
break s;
}
last_status = Some(s);
if std::time::Instant::now() >= deadline {
break;
}
}
let after = last_status.unwrap_or_else(|| before.clone());
};
let recovered = after.anchor_connected && !before.anchor_connected;
let likely_cause = if after.anchor_connected {
+51 -3
View File
@@ -307,14 +307,62 @@ async fn scan_wifi() -> Result<Vec<serde_json::Value>> {
/// Connect to a WiFi network using nmcli.
async fn connect_wifi(ssid: &str, password: &str) -> Result<()> {
let conn_name = format!("archipelago-wifi-{ssid}");
// Delete prior profiles for this SSID/name first. Failed attempts can leave
// a profile with key-mgmt but no saved PSK, causing future retries to fail
// with "no secrets" before the supplied password is used.
let _ = tokio::process::Command::new("nmcli")
.args(["connection", "delete", &conn_name])
.output()
.await;
let _ = tokio::process::Command::new("nmcli")
.args(["connection", "delete", ssid])
.output()
.await;
let output = tokio::process::Command::new("nmcli")
.args(["device", "wifi", "connect", ssid, "password", password])
.args([
"connection",
"add",
"type",
"wifi",
"con-name",
&conn_name,
"ifname",
"*",
"ssid",
ssid,
"wifi-sec.key-mgmt",
"wpa-psk",
"wifi-sec.psk",
password,
"ipv4.method",
"auto",
"ipv6.method",
"auto",
])
.output()
.await
.context("Failed to run nmcli wifi profile create")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("WiFi profile create failed: {}", stderr);
}
let activate = tokio::process::Command::new("nmcli")
.args(["connection", "up", &conn_name])
.output()
.await
.context("Failed to run nmcli wifi connect")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
if !activate.status.success() {
let stderr = String::from_utf8_lossy(&activate.stderr);
let _ = tokio::process::Command::new("nmcli")
.args(["connection", "delete", &conn_name])
.output()
.await;
anyhow::bail!("WiFi connection failed: {}", stderr);
}
+7 -5
View File
@@ -3,6 +3,8 @@ use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use tracing::info;
use super::LND_REST_BASE_URL;
#[derive(Debug, Serialize)]
struct ChannelInfo {
chan_id: String,
@@ -62,7 +64,7 @@ impl RpcHandler {
let (client, macaroon_hex) = self.lnd_client().await?;
let channels_resp: LndListChannelsResponse = client
.get("https://127.0.0.1:8080/v1/channels")
.get(format!("{LND_REST_BASE_URL}/v1/channels"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -72,7 +74,7 @@ impl RpcHandler {
.context("Failed to parse LND channels response")?;
let pending_resp: LndPendingChannelsResponse = match client
.get("https://127.0.0.1:8080/v1/channels/pending")
.get(format!("{LND_REST_BASE_URL}/v1/channels/pending"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -211,7 +213,7 @@ impl RpcHandler {
"perm": true
});
let _ = client
.post("https://127.0.0.1:8080/v1/peers")
.post(format!("{LND_REST_BASE_URL}/v1/peers"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&connect_body)
.send()
@@ -224,7 +226,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v1/channels")
.post(format!("{LND_REST_BASE_URL}/v1/channels"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&open_body)
.send()
@@ -291,7 +293,7 @@ impl RpcHandler {
let (client, macaroon_hex) = self.lnd_client().await?;
let url = format!(
"https://127.0.0.1:8080/v1/channels/{}/{}?force={}",
"{LND_REST_BASE_URL}/v1/channels/{}/{}?force={}",
parts[0], parts[1], force
);
+6 -6
View File
@@ -3,7 +3,7 @@ use anyhow::{Context, Result};
use base64::Engine;
use serde::{Deserialize, Serialize};
use super::{read_lnd_admin_macaroon, LndAmount, LndBalanceResponse};
use super::{read_lnd_admin_macaroon, LndAmount, LndBalanceResponse, LND_REST_BASE_URL};
#[derive(Debug, Serialize)]
struct LndInfo {
@@ -44,7 +44,7 @@ impl RpcHandler {
.context("Failed to create HTTP client")?;
let get_info: LndGetInfoResponse = client
.get("https://127.0.0.1:8080/v1/getinfo")
.get(format!("{LND_REST_BASE_URL}/v1/getinfo"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -54,7 +54,7 @@ impl RpcHandler {
.context("Failed to parse LND getinfo response")?;
let channel_balance: LndChannelBalanceResponse = match client
.get("https://127.0.0.1:8080/v1/balance/channels")
.get(format!("{LND_REST_BASE_URL}/v1/balance/channels"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -70,7 +70,7 @@ impl RpcHandler {
};
let wallet_balance: LndBalanceResponse = match client
.get("https://127.0.0.1:8080/v1/balance/blockchain")
.get(format!("{LND_REST_BASE_URL}/v1/balance/blockchain"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -166,7 +166,7 @@ impl RpcHandler {
"cert_base64url": cert_b64url,
"macaroon_base64url": macaroon_b64url,
"tor_onion": tor_onion,
"rest_port": 8080,
"rest_port": 18080,
"grpc_port": 10009,
}))
}
@@ -186,7 +186,7 @@ impl RpcHandler {
.context("Failed to build HTTP client")?;
let resp = client
.get("https://127.0.0.1:8080/v1/channels/backup")
.get(format!("{LND_REST_BASE_URL}/v1/channels/backup"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
+1
View File
@@ -9,6 +9,7 @@ use anyhow::{anyhow, Context, Result};
/// Canonical on-host path for LND's admin macaroon.
pub(crate) const LND_ADMIN_MACAROON_PATH: &str =
"/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
pub(in crate::api) const LND_REST_BASE_URL: &str = "https://127.0.0.1:18080";
// Shared LND response types used by multiple submodules
#[derive(Debug, serde::Deserialize)]
+4 -2
View File
@@ -2,6 +2,8 @@ use crate::api::rpc::RpcHandler;
use anyhow::{Context, Result};
use tracing::info;
use super::LND_REST_BASE_URL;
impl RpcHandler {
/// Pay a Lightning invoice.
pub(in crate::api::rpc) async fn handle_lnd_payinvoice(
@@ -35,7 +37,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v1/channels/transactions")
.post(format!("{LND_REST_BASE_URL}/v1/channels/transactions"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&pay_body)
.send()
@@ -91,7 +93,7 @@ impl RpcHandler {
let (client, macaroon_hex) = self.lnd_client().await?;
let resp = client
.get("https://127.0.0.1:8080/v1/transactions")
.get(format!("{LND_REST_BASE_URL}/v1/transactions"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
+11 -9
View File
@@ -4,13 +4,15 @@ use base64::Engine;
use tracing::info;
use zeroize::Zeroize;
use super::LND_REST_BASE_URL;
impl RpcHandler {
/// Generate a new on-chain Bitcoin address.
pub(in crate::api::rpc) async fn handle_lnd_newaddress(&self) -> Result<serde_json::Value> {
let (client, macaroon_hex) = self.lnd_client().await?;
let resp = client
.get("https://127.0.0.1:8080/v1/newaddress")
.get(format!("{LND_REST_BASE_URL}/v1/newaddress"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.send()
.await
@@ -69,7 +71,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v1/transactions")
.post(format!("{LND_REST_BASE_URL}/v1/transactions"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&send_body)
.send()
@@ -129,7 +131,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v1/invoices")
.post(format!("{LND_REST_BASE_URL}/v1/invoices"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&invoice_body)
.send()
@@ -231,7 +233,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v2/wallet/psbt/fund")
.post(format!("{LND_REST_BASE_URL}/v2/wallet/psbt/fund"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&fund_body)
.send()
@@ -289,7 +291,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v2/wallet/psbt/finalize")
.post(format!("{LND_REST_BASE_URL}/v2/wallet/psbt/finalize"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&finalize_body)
.send()
@@ -322,7 +324,7 @@ impl RpcHandler {
});
let pub_resp = client
.post("https://127.0.0.1:8080/v2/wallet/tx")
.post(format!("{LND_REST_BASE_URL}/v2/wallet/tx"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&publish_body)
.send()
@@ -387,7 +389,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v2/wallet/psbt/fund")
.post(format!("{LND_REST_BASE_URL}/v2/wallet/psbt/fund"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&fund_body)
.send()
@@ -416,7 +418,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v2/wallet/psbt/finalize")
.post(format!("{LND_REST_BASE_URL}/v2/wallet/psbt/finalize"))
.header("Grpc-Metadata-macaroon", &macaroon_hex)
.json(&finalize_body)
.send()
@@ -514,7 +516,7 @@ impl RpcHandler {
});
let resp = client
.post("https://127.0.0.1:8080/v1/initwallet")
.post(format!("{LND_REST_BASE_URL}/v1/initwallet"))
.json(&init_body)
.send()
.await
+1 -1
View File
@@ -188,7 +188,7 @@ impl RpcHandler {
let (client, macaroon_hex) = self.lnd_client().await?;
let url = format!("https://127.0.0.1:8080/v1/invoice/{}", r_hash);
let url = format!("{}/v1/invoice/{r_hash}", super::lnd::LND_REST_BASE_URL);
let paid = match client
.get(&url)
.header("Grpc-Metadata-macaroon", &macaroon_hex)
@@ -53,6 +53,7 @@ pub(super) fn sanitize_error_message(msg: &str) -> String {
"Unauthorized",
"Forbidden",
"Not supported",
"Requires",
"requires",
"must be",
"cannot",
+1 -1
View File
@@ -12,7 +12,7 @@ mod fips;
mod handshake;
mod identity;
mod interfaces;
mod lnd;
pub(in crate::api) mod lnd;
mod marketplace;
mod mesh;
mod middleware;
@@ -54,6 +54,7 @@ impl RpcHandler {
.ok_or_else(|| anyhow::anyhow!("Missing package id"))?
.to_string();
super::validation::validate_app_id(&package_id)?;
super::dependencies::check_bitcoin_pruning_compatibility(&package_id).await?;
// Reject if already in a transitional lifecycle (prevents double-click
// queuing two installs on the same package).
@@ -410,16 +411,6 @@ async fn set_package_state_and_clear_uninstall_stage(
}
}
/// Remove a package entry from state. Used for install-failure cleanup
/// (since there's no pre-state to revert to — the entry was created
/// speculatively when we flipped to Installing).
async fn remove_package_entry(state_manager: &StateManager, package_id: &str) {
let (mut data, _) = state_manager.get_snapshot().await;
if data.package_data.remove(package_id).is_some() {
state_manager.update_data(data).await;
}
}
/// Kick the container scanner to run immediately and wait for it to finish
/// (with a 2s timeout). Used by install/update success paths so the fresh
/// manifest — with `interfaces.main.ui` populated from the now-running
+252 -87
View File
@@ -1,6 +1,96 @@
use super::validation::validate_app_id;
use crate::port_allocator::PortAllocator;
use anyhow::{Context, Result};
use std::time::Duration;
const PODMAN_LIST_TIMEOUT: Duration = Duration::from_secs(15);
fn is_platform_managed_app(app_id: &str) -> bool {
matches!(
app_id,
"bitcoin"
| "bitcoin-core"
| "bitcoin-knots"
| "bitcoin-ui"
| "lnd"
| "lnd-ui"
| "electrumx"
| "electrs"
| "mempool-electrs"
| "electrs-ui"
| "mempool"
| "mempool-web"
| "mempool-api"
| "archy-mempool-db"
| "archy-mempool-web"
| "btcpay"
| "btcpay-server"
| "btcpayserver"
| "archy-btcpay-db"
| "archy-nbxplorer"
| "fedimint"
| "fedimint-gateway"
| "indeedhub"
| "immich"
)
}
fn safe_dynamic_arg(value: &str) -> bool {
!value.is_empty()
&& value.len() <= 512
&& !value.chars().any(|c| matches!(c, '\0' | '\n' | '\r'))
}
async fn dynamic_app_config(
app_id: &str,
) -> Option<(
Vec<String>,
Vec<String>,
Vec<String>,
Option<String>,
Option<Vec<String>>,
)> {
if is_platform_managed_app(app_id) {
return None;
}
let config_path = format!("/var/lib/archipelago/app-configs/{}.json", app_id);
let data = tokio::fs::read_to_string(&config_path).await.ok()?;
let cfg = serde_json::from_str::<serde_json::Value>(&data).ok()?;
let string_array = |key: &str| -> Vec<String> {
cfg.get(key)
.and_then(|v| v.as_array())
.map(|a| {
a.iter()
.filter_map(|v| v.as_str())
.filter(|s| safe_dynamic_arg(s))
.map(String::from)
.collect()
})
.unwrap_or_default()
};
let command = cfg
.get("command")
.and_then(|v| v.as_str())
.filter(|s| safe_dynamic_arg(s))
.map(String::from);
let args = cfg.get("args").and_then(|v| v.as_array()).map(|a| {
a.iter()
.filter_map(|v| v.as_str())
.filter(|s| safe_dynamic_arg(s))
.map(String::from)
.collect::<Vec<_>>()
});
tracing::info!(app_id = %app_id, "loaded catalog runtime config for generic app");
Some((
string_array("ports"),
string_array("volumes"),
string_array("env"),
command,
args.filter(|a| !a.is_empty()),
))
}
/// Trusted Docker registries. Only images from these sources are allowed.
#[allow(dead_code)]
@@ -63,26 +153,39 @@ pub(super) fn get_app_capabilities(app_id: &str) -> Vec<String> {
"--cap-add=SETGID".to_string(),
"--cap-add=DAC_OVERRIDE".to_string(),
],
// Nginx Proxy Manager needs to bind low ports
// Nginx Proxy Manager initializes/chowns mounted state on first boot.
"nginx-proxy-manager" => vec![
"--cap-add=CHOWN".to_string(),
"--cap-add=FOWNER".to_string(),
"--cap-add=SETUID".to_string(),
"--cap-add=SETGID".to_string(),
"--cap-add=DAC_OVERRIDE".to_string(),
"--cap-add=NET_BIND_SERVICE".to_string(),
],
// Bitcoin and Lightning need file ownership ops + NET_BIND_SERVICE for port binding
// LND additionally needs NET_RAW for TLS certificate generation (netlinkrib interface enumeration)
"bitcoin" | "bitcoin-core" | "bitcoin-knots" | "lnd" | "fedimint" | "fedimint-gateway" => {
vec![
"--cap-add=CHOWN".to_string(),
"--cap-add=FOWNER".to_string(),
"--cap-add=SETUID".to_string(),
"--cap-add=SETGID".to_string(),
"--cap-add=DAC_OVERRIDE".to_string(),
"--cap-add=NET_BIND_SERVICE".to_string(),
"--cap-add=NET_RAW".to_string(),
]
}
// Bitcoin needs only file-ownership ops + NET_BIND_SERVICE for the
// RPC port. NO NET_RAW — bitcoind never opens raw sockets and
// dropping it removes a class of intra-pod spoofing capability.
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => vec![
"--cap-add=CHOWN".to_string(),
"--cap-add=FOWNER".to_string(),
"--cap-add=SETUID".to_string(),
"--cap-add=SETGID".to_string(),
"--cap-add=DAC_OVERRIDE".to_string(),
"--cap-add=NET_BIND_SERVICE".to_string(),
],
// LND additionally needs NET_RAW for TLS certificate generation
// (netlink interface enumeration during `lnd --tlscertpath` first run).
// Fedimint inherits the same set because the gateway also enumerates
// network interfaces on startup.
"lnd" | "fedimint" | "fedimint-gateway" => vec![
"--cap-add=CHOWN".to_string(),
"--cap-add=FOWNER".to_string(),
"--cap-add=SETUID".to_string(),
"--cap-add=SETGID".to_string(),
"--cap-add=DAC_OVERRIDE".to_string(),
"--cap-add=NET_BIND_SERVICE".to_string(),
"--cap-add=NET_RAW".to_string(),
],
// Vaultwarden needs file ownership + NET_BIND_SERVICE (binds port 80 internally)
"vaultwarden" => vec![
"--cap-add=CHOWN".to_string(),
@@ -153,22 +256,21 @@ pub(super) fn is_readonly_compatible(app_id: &str) -> bool {
/// Get container health check arguments for podman run.
/// Returns (health-cmd, interval, retries) args to append to run_args.
pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String> {
// bitcoin-cli reads the .cookie file from -datadir automatically (no plaintext creds needed)
let btc_health =
"bitcoin-cli -datadir=/home/bitcoin/.bitcoin getblockchaininfo || exit 1".to_string();
let (cmd, interval, retries) = match app_id {
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => (btc_health.as_str(), "30s", "3"),
// Bitcoin images do not consistently ship bitcoin-cli/curl/nc. Rely on
// process state here; manifests still describe the desired TCP check.
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => return vec![],
"lnd" => ("lncli getinfo || exit 1", "30s", "3"),
"btcpay-server" | "btcpayserver" => {
("curl -sf http://localhost:49392/ || exit 1", "30s", "3")
("bash -ec '</dev/tcp/127.0.0.1/49392'", "30s", "3")
}
"mempool-api" => (
"curl -sf http://localhost:8999/api/v1/backend-info || exit 1",
http_probe_cmd("http://localhost:8999/api/v1/backend-info"),
"30s",
"3",
),
"mempool" | "mempool-web" | "archy-mempool-web" => {
("curl -sf http://localhost:8080/ || exit 1", "30s", "3")
(http_probe_cmd("http://localhost:8080/"), "30s", "3")
}
"electrumx" | "mempool-electrs" | "electrs" => {
("curl -sf http://localhost:8000/ || exit 1", "60s", "3")
@@ -179,10 +281,10 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
"3",
),
"homeassistant" | "home-assistant" => {
("curl -sf http://localhost:8123/api/ || exit 1", "30s", "3")
("curl -sf http://localhost:8123/ || exit 1", "30s", "3")
}
"grafana" => (
"curl -sf http://localhost:3000/api/health || exit 1",
"test -w /var/lib/grafana && test -w /var/lib/grafana/grafana.db && curl -sf http://localhost:3000/api/health || exit 1",
"30s",
"3",
),
@@ -199,7 +301,7 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
"30s",
"3",
),
"searxng" => ("wget -q -O /dev/null http://localhost:8080/ || exit 1", "30s", "3"),
"searxng" => (http_probe_cmd("http://localhost:8080/"), "30s", "3"),
"photoprism" => (
"curl -sf http://localhost:2342/api/v1/status || exit 1",
"60s",
@@ -219,10 +321,8 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
"ollama" => ("curl -sf http://localhost:11434/ || exit 1", "30s", "3"),
"fedimint" => ("curl -sf http://localhost:8175/ || exit 1", "60s", "3"),
"fedimint-gateway" => ("curl -sf http://localhost:8176/ || exit 1", "60s", "3"),
"nostr-rs-relay" | "nostr-relay" => {
("curl -sf http://localhost:8080/ || exit 1", "30s", "3")
}
"nginx-proxy-manager" => ("curl -sf http://localhost:81/api/ || exit 1", "30s", "3"),
"nostr-rs-relay" | "nostr-relay" => (http_probe_cmd("http://localhost:8080/"), "30s", "3"),
"nginx-proxy-manager" => (http_probe_cmd("http://localhost:81/"), "30s", "3"),
"routstr" => (
"curl -sf http://localhost:8000/v1/models || exit 1",
"30s",
@@ -237,10 +337,21 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
format!("--health-cmd={}", cmd),
format!("--health-interval={}", interval),
format!("--health-retries={}", retries),
"--health-timeout=10s".to_string(),
"--health-start-period=60s".to_string(),
]
}
fn http_probe_cmd(url: &'static str) -> &'static str {
match url {
"http://localhost:8999/api/v1/backend-info" => "if command -v wget >/dev/null 2>&1; then wget -q -T 5 -O /dev/null http://localhost:8999/api/v1/backend-info; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 5 http://localhost:8999/api/v1/backend-info; else exit 0; fi",
"http://localhost:8080/" => "if command -v wget >/dev/null 2>&1; then wget -q -T 5 -O /dev/null http://localhost:8080/; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 5 http://localhost:8080/; else exit 0; fi",
"http://localhost:81/api/" => "if command -v wget >/dev/null 2>&1; then wget -q -T 5 -O /dev/null http://localhost:81/api/; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 5 http://localhost:81/api/; else exit 0; fi",
"http://localhost:81/" => "if command -v wget >/dev/null 2>&1; then wget -q -T 5 -O /dev/null http://localhost:81/; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 5 http://localhost:81/; else exit 0; fi",
_ => "exit 0",
}
}
/// Get per-app memory limit.
pub(super) fn get_memory_limit(app_id: &str) -> &'static str {
match app_id {
@@ -249,10 +360,10 @@ pub(super) fn get_memory_limit(app_id: &str) -> &'static str {
// memory + I/O. 4g caused OOM-cascades during IBD. 8g is the
// floor; ideally this would be host-RAM aware (next pass).
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => "8g",
// ElectrumX: bumped from 1g to 2g so its CACHE_MB has somewhere
// to live during initial blockchain indexing. CACHE_MB=2048 in
// env vars below requires this much.
"electrumx" | "mempool-electrs" | "electrs" => "2g",
// ElectrumX: large cache materially speeds initial history indexing.
// CACHE_MB=3072 below needs container headroom for Python, rocksdb,
// socket buffers, and reorg/indexing spikes.
"electrumx" | "mempool-electrs" | "electrs" => "4g",
"cryptpad" => "512m",
"ollama" => "4g",
// Medium apps
@@ -282,7 +393,8 @@ pub(super) fn get_memory_limit(app_id: &str) -> &'static str {
"nginx-proxy-manager" => "256m",
// Databases
"archy-btcpay-db" | "archy-mempool-db" | "mysql-mempool" => "512m",
"immich_postgres" | "penpot-postgres" => "256m",
"immich_postgres" => "2g",
"penpot-postgres" => "256m",
"immich_redis" | "penpot-valkey" => "128m",
// Default
_ => "512m",
@@ -298,16 +410,23 @@ pub(super) fn all_container_names(package_id: &str) -> Vec<String> {
let archy = format!("archy-{}", package_id);
match package_id {
// Bitcoin: multiple historical names
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => vec![
// Bitcoin variants share the UI but not the backend process. Keep
// backend names precise so stopping one implementation does not clear
// stop markers or issue podman operations for the other.
"bitcoin" | "bitcoin-knots" => vec![
"bitcoin-knots".into(),
"bitcoin".into(),
"bitcoin-core".into(),
"archy-bitcoin-knots".into(),
"archy-bitcoin".into(),
"bitcoin-ui".into(),
"archy-bitcoin-ui".into(),
],
"bitcoin-core" => vec![
"bitcoin-core".into(),
"archy-bitcoin-core".into(),
"bitcoin-ui".into(),
"archy-bitcoin-ui".into(),
],
// LND + UI
"lnd" => vec!["lnd".into(), "archy-lnd".into(), "archy-lnd-ui".into()],
// Electrumx: multiple aliases
@@ -366,6 +485,15 @@ pub(super) fn all_container_names(package_id: &str) -> Vec<String> {
"penpot-exporter".into(),
"penpot-frontend".into(),
],
"indeedhub" => vec![
"indeedhub-postgres".into(),
"indeedhub-redis".into(),
"indeedhub-minio".into(),
"indeedhub-relay".into(),
"indeedhub-api".into(),
"indeedhub-ffmpeg".into(),
"indeedhub".into(),
],
"nostr-vpn" => vec![
"nostr-vpn".into(),
"archy-nostr-vpn".into(),
@@ -380,12 +508,14 @@ pub(super) fn all_container_names(package_id: &str) -> Vec<String> {
/// Find all running/stopped containers that belong to a given app.
/// Uses the canonical name list from all_container_names().
pub(super) async fn get_containers_for_app(package_id: &str) -> Result<Vec<String>> {
pub(in crate::api::rpc) async fn get_containers_for_app(package_id: &str) -> Result<Vec<String>> {
validate_app_id(package_id)?;
let output = tokio::process::Command::new("podman")
.args(["ps", "-a", "--format", "{{.Names}}"])
.output()
let mut cmd = tokio::process::Command::new("podman");
cmd.args(["ps", "-a", "--format", "{{.Names}}"]);
cmd.kill_on_drop(true);
let output = tokio::time::timeout(PODMAN_LIST_TIMEOUT, cmd.output())
.await
.context("podman ps timed out while listing containers")?
.context("Failed to list containers")?;
let stdout = String::from_utf8_lossy(&output.stdout);
let all: Vec<&str> = stdout.lines().filter(|s| !s.is_empty()).collect();
@@ -400,11 +530,43 @@ pub(super) async fn get_containers_for_app(package_id: &str) -> Result<Vec<Strin
Ok(result)
}
#[cfg(test)]
mod tests {
use super::{all_container_names, get_health_check_args};
#[test]
fn bitcoin_variant_container_names_are_precise() {
let core = all_container_names("bitcoin-core");
assert!(core.contains(&"bitcoin-core".to_string()));
assert!(!core.contains(&"bitcoin-knots".to_string()));
let knots = all_container_names("bitcoin-knots");
assert!(knots.contains(&"bitcoin-knots".to_string()));
assert!(!knots.contains(&"bitcoin-core".to_string()));
}
#[test]
fn grafana_health_requires_writable_data_and_http_health() {
let args = get_health_check_args("grafana", "unused");
let health_cmd = args
.iter()
.find_map(|arg| arg.strip_prefix("--health-cmd="))
.expect("grafana should have a health command");
assert!(health_cmd.contains("test -w /var/lib/grafana"));
assert!(health_cmd.contains("test -w /var/lib/grafana/grafana.db"));
assert!(health_cmd.contains("http://localhost:3000/api/health"));
}
}
/// Get data directories to clean for an app.
/// Caller must validate package_id before calling.
pub(super) fn get_data_dirs_for_app(package_id: &str) -> Vec<String> {
let base = "/var/lib/archipelago";
match package_id {
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => {
vec![format!("{}/bitcoin", base), format!("{}/bitcoin-ui", base)]
}
"mempool" | "mempool-web" => vec![
format!("{}/mempool", base),
format!("{}/mysql-mempool", base),
@@ -481,6 +643,10 @@ pub(super) async fn get_app_config(
Option<String>,
Option<Vec<String>>,
) {
if let Some(config) = dynamic_app_config(app_id).await {
return config;
}
match app_id {
"homeassistant" | "home-assistant" => (
vec!["8123:8123".to_string()],
@@ -541,7 +707,7 @@ pub(super) async fn get_app_config(
vec![
"9735:9735".to_string(),
"10009:10009".to_string(),
"8080:8080".to_string(),
"18080:8080".to_string(),
],
vec!["/var/lib/archipelago/lnd:/root/.lnd".to_string()],
vec![],
@@ -617,9 +783,10 @@ pub(super) async fn get_app_config(
"DB_DIRECTORY=/data".to_string(),
"SERVICES=tcp://:50001,rpc://0.0.0.0:8000".to_string(),
// Sync-speed: bigger LRU/write cache during initial
// history index. Default is 1200MB, container now
// gets 2g (config.rs::get_memory_limit) so 2048 fits.
"CACHE_MB=2048".to_string(),
// history index. Default is 1200MB; the container gets
// 4g (config.rs::get_memory_limit) so 3072 fits with
// headroom.
"CACHE_MB=3072".to_string(),
// Block-fetcher concurrency — defaults are conservative
// for shared hosts; 4 is plenty for one bitcoind backend.
"MAX_SEND=10000000".to_string(),
@@ -759,20 +926,34 @@ pub(super) async fn get_app_config(
]),
)
}
"nginx-proxy-manager" => (
vec![
"81:81".to_string(),
"8084:80".to_string(),
"8443:443".to_string(),
],
vec![
"/var/lib/archipelago/nginx-proxy-manager/data:/data".to_string(),
"/var/lib/archipelago/nginx-proxy-manager/letsencrypt:/etc/letsencrypt".to_string(),
],
vec![],
None,
None,
),
"nginx-proxy-manager" => {
let admin_port = allocator
.allocate_or_get(app_id, 8081, 81)
.await
.unwrap_or(8081);
let http_port = allocator
.allocate_or_get("nginx-proxy-manager-http", 8084, 80)
.await
.unwrap_or(8084);
let https_port = allocator
.allocate_or_get("nginx-proxy-manager-https", 8444, 443)
.await
.unwrap_or(8444);
(
vec![
format!("{}:81", admin_port),
format!("{}:80", http_port),
format!("{}:443", https_port),
],
vec![
"/var/lib/archipelago/nginx-proxy-manager/data:/data".to_string(),
"/var/lib/archipelago/nginx-proxy-manager/letsencrypt:/etc/letsencrypt".to_string(),
],
vec![],
None,
None,
)
}
"portainer" => (
vec!["9000:9000".to_string()],
vec![
@@ -788,7 +969,11 @@ pub(super) async fn get_app_config(
vec!["/var/lib/archipelago/uptime-kuma:/app/data".to_string()],
vec!["TZ=UTC".to_string()],
None,
None,
Some(vec![
"--".to_string(),
"node".to_string(),
"server/server.js".to_string(),
]),
),
"tailscale" => (
vec!["8240:8240".to_string()],
@@ -803,7 +988,7 @@ pub(super) async fn get_app_config(
Some(vec![
"sh".to_string(),
"-c".to_string(),
"tailscale web --listen 0.0.0.0:8240 & exec tailscaled".to_string(),
"tailscaled --tun=userspace-networking & sleep 2; tailscale web --listen 0.0.0.0:8240 & wait".to_string(),
]),
),
"fedimint" => (
@@ -964,8 +1149,8 @@ pub(super) async fn get_app_config(
None,
)
}
// Gitea binds to 3001 internally. Nginx on port 3000 strips X-Frame-Options
// so Gitea works in Archipelago's iframe. See nginx-gitea-iframe.conf.
// Gitea listens on container port 3000 and is launched directly on
// host port 3001 because it blocks iframe embedding.
"gitea" => (
vec!["3001:3000".to_string(), "2222:22".to_string()],
vec![
@@ -986,31 +1171,11 @@ pub(super) async fn get_app_config(
None,
),
_ => {
// Unknown app: try to load config from /var/lib/archipelago/app-configs/{id}.json
// This allows dynamic apps from the remote catalog to be installed
// without hardcoding their config here.
let config_path = format!("/var/lib/archipelago/app-configs/{}.json", app_id);
if let Ok(data) = tokio::fs::read_to_string(&config_path).await {
if let Ok(cfg) = serde_json::from_str::<serde_json::Value>(&data) {
let ports = cfg.get("ports")
.and_then(|v| v.as_array())
.map(|a| a.iter().filter_map(|v| v.as_str().map(String::from)).collect())
.unwrap_or_default();
let volumes = cfg.get("volumes")
.and_then(|v| v.as_array())
.map(|a| a.iter().filter_map(|v| v.as_str().map(String::from)).collect())
.unwrap_or_default();
let env_vars = cfg.get("env")
.and_then(|v| v.as_array())
.map(|a| a.iter().filter_map(|v| v.as_str().map(String::from)).collect())
.unwrap_or_default();
tracing::info!("Loaded dynamic config for app: {}", app_id);
return (ports, volumes, env_vars, None, None);
}
}
// No config found — use minimal defaults (container's own EXPOSE/VOLUME)
tracing::warn!("No config found for app: {} — using minimal defaults", app_id);
// No catalog runtime metadata found; use minimal defaults
// (container's own EXPOSE/VOLUME). New generic apps should declare
// containerConfig in the registry catalog instead of adding Rust cases.
tracing::warn!("No catalog runtime config found for app: {} — using minimal defaults", app_id);
(vec![], vec![], vec![], None, None)
},
}
}
}
@@ -1,5 +1,7 @@
use super::config::get_containers_for_app;
use anyhow::Result;
use crate::data_model::{PackageDataEntry, PackageState};
use anyhow::{Context, Result};
use std::collections::HashMap;
use tracing::info;
/// Names of container variants that represent a running Bitcoin node
@@ -7,6 +9,20 @@ const BITCOIN_NAMES: &[&str] = &["bitcoin-knots", "bitcoin-core", "bitcoin"];
/// Names of container variants that represent a running Electrum indexer
const ELECTRUM_NAMES: &[&str] = &["electrumx", "mempool-electrs", "electrs"];
const ARCHIVAL_BITCOIN_DISK_GB: u64 = 1000;
fn requires_unpruned_bitcoin(package_id: &str) -> bool {
matches!(
package_id,
"electrumx" | "mempool-electrs" | "electrs" | "mempool" | "mempool-web"
)
}
fn archival_bitcoin_required_message(package_id: &str) -> String {
format!(
"Requires an archival Bitcoin node while indexing: {package_id}. This node is running pruned Bitcoin because it does not have enough disk for full block history. Add enough storage for an archival node (about 1 TB or more), resync Bitcoin without pruning/with txindex, then install {package_id}."
)
}
/// Snapshot of which dependency services are currently running.
pub(super) struct RunningDeps {
@@ -15,13 +31,43 @@ pub(super) struct RunningDeps {
pub has_lnd: bool,
}
pub(super) fn detect_running_deps_from_package_data(
packages: &HashMap<String, PackageDataEntry>,
) -> RunningDeps {
let is_running = |names: &[&str]| {
names.iter().any(|name| {
packages
.get(*name)
.map(|pkg| pkg.state == PackageState::Running)
.unwrap_or(false)
})
};
RunningDeps {
has_bitcoin: is_running(BITCOIN_NAMES),
has_electrumx: is_running(ELECTRUM_NAMES),
has_lnd: is_running(&["lnd"]),
}
}
/// Query podman for currently running containers and return dependency status.
pub(super) async fn detect_running_deps() -> Result<RunningDeps> {
let dep_check = tokio::process::Command::new("podman")
.args(["ps", "--format", "{{.Names}}"])
.output()
.await
.map_err(|e| anyhow::anyhow!("Failed to check running containers: {}", e))?;
let dep_check = tokio::time::timeout(
std::time::Duration::from_secs(30),
tokio::process::Command::new("podman")
.args(["ps", "--format", "{{.Names}}"])
.output(),
)
.await
.map_err(|_| anyhow::anyhow!("Timed out checking running containers"))?
.map_err(|e| anyhow::anyhow!("Failed to check running containers: {}", e))?;
if !dep_check.status.success() {
anyhow::bail!(
"Failed to check running containers: {}",
String::from_utf8_lossy(&dep_check.stderr).trim()
);
}
let running = String::from_utf8_lossy(&dep_check.stdout);
let is_running = |names: &[&str]| {
@@ -76,6 +122,107 @@ pub(super) fn check_install_deps(package_id: &str, deps: &RunningDeps) -> Result
}
}
/// ElectrumX and Mempool's Electrum backend need historical blocks from an
/// unpruned node while building their indexes. A pruned Bitcoin node can be
/// running and RPC-reachable but still leave them stuck with closed ports.
pub(super) async fn check_bitcoin_pruning_compatibility(package_id: &str) -> Result<()> {
if !requires_unpruned_bitcoin(package_id) {
return Ok(());
}
let (rpc_user, rpc_pass) = crate::bitcoin_rpc::bitcoin_rpc_credentials().await;
let body = serde_json::json!({
"jsonrpc": "1.0",
"id": "package-install-prune-check",
"method": "getblockchaininfo",
"params": [],
});
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.context("building Bitcoin RPC client")?;
let mut last_error = None;
for _ in 0..3 {
match client
.post(crate::constants::BITCOIN_RPC_URL)
.basic_auth(&rpc_user, Some(&rpc_pass))
.header("Content-Type", "application/json")
.json(&body)
.send()
.await
{
Ok(resp) => {
let status = resp.status();
match resp.json::<serde_json::Value>().await {
Ok(json) if status.is_success() => {
if let Some(error) = json.get("error").filter(|e| !e.is_null()) {
last_error = Some(format!(
"Bitcoin RPC error while checking pruning status: {error}"
));
} else {
return check_blockchain_info_for_pruning(package_id, &json);
}
}
Ok(json) => {
last_error = Some(format!(
"Bitcoin RPC returned {status} while checking pruning status: {json}"
));
}
Err(e) => {
last_error = Some(format!("decode Bitcoin RPC response: {e}"));
}
}
}
Err(e) => {
last_error = Some(format!("checking Bitcoin pruning status: {e}"));
}
}
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
if detect_disk_gb() < ARCHIVAL_BITCOIN_DISK_GB {
anyhow::bail!(archival_bitcoin_required_message(package_id));
}
anyhow::bail!(
"Bitcoin RPC unavailable while checking pruning status: {}",
last_error.unwrap_or_else(|| "unknown error".to_string())
);
}
fn check_blockchain_info_for_pruning(package_id: &str, json: &serde_json::Value) -> Result<()> {
let Some(result) = json.get("result") else {
anyhow::bail!("Bitcoin RPC response missing result while checking pruning status");
};
if result
.get("pruned")
.and_then(|v| v.as_bool())
.unwrap_or(false)
{
anyhow::bail!(archival_bitcoin_required_message(package_id));
}
Ok(())
}
fn detect_disk_gb() -> u64 {
let output = std::process::Command::new("df")
.args(["-BG", "/var/lib/archipelago"])
.output();
let Ok(output) = output else {
return u64::MAX;
};
let stdout = String::from_utf8_lossy(&output.stdout);
stdout
.lines()
.nth(1)
.and_then(|line| line.split_whitespace().nth(1))
.and_then(|size| size.trim_end_matches('G').parse::<u64>().ok())
.unwrap_or(u64::MAX)
}
/// Log informational messages about optional dependencies.
pub(super) fn log_optional_dep_info(package_id: &str, deps: &RunningDeps) {
if matches!(package_id, "btcpay-server" | "btcpayserver") && !deps.has_lnd {
@@ -129,6 +276,18 @@ pub(super) fn startup_order(package_id: &str) -> &'static [&'static str] {
"mempool",
],
"immich" => &["immich_postgres", "immich_redis", "immich_server"],
"indeedhub" => &[
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
],
"btcpay-server" | "btcpayserver" | "btcpay" => {
&["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"]
}
"penpot" | "penpot-frontend" => &[
"penpot-postgres",
"penpot-valkey",
@@ -144,17 +303,23 @@ pub(super) fn startup_order(package_id: &str) -> &'static [&'static str] {
/// order for the given app. Unknown containers sort to the end.
pub(super) async fn ordered_containers_for_start(package_id: &str) -> Result<Vec<String>> {
let containers = get_containers_for_app(package_id).await?;
if containers.is_empty() {
return Ok(vec![format!("archy-{}", package_id)]);
}
let order = startup_order(package_id);
// If no special order defined, fall back to mempool order (legacy behavior)
if order.is_empty() && containers.is_empty() {
return Ok(vec![package_id.to_string()]);
}
let mut sorted = containers;
for required in order {
if !sorted.iter().any(|name| name == required) {
sorted.push((*required).to_string());
}
}
// If no special order is defined, fall back to mempool order for legacy
// multi-container names that may still be returned by config lookups.
let effective_order: &[&str] = if order.is_empty() {
startup_order("mempool")
} else {
order
};
let mut sorted = containers;
sorted.sort_by_key(|c| effective_order.iter().position(|o| *o == c).unwrap_or(99));
Ok(sorted)
}
@@ -211,3 +376,32 @@ pub(super) fn configure_fedimint_lnd(
]);
}
}
#[cfg(test)]
mod tests {
use super::{requires_unpruned_bitcoin, startup_order};
#[test]
fn btcpay_start_order_includes_required_stack_members() {
assert_eq!(
startup_order("btcpay-server"),
&["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"]
);
}
#[test]
fn unpruned_bitcoin_required_for_electrum_indexers_and_mempool() {
for package_id in [
"electrumx",
"mempool-electrs",
"electrs",
"mempool",
"mempool-web",
] {
assert!(requires_unpruned_bitcoin(package_id), "{package_id}");
}
for package_id in ["bitcoin-knots", "btcpay-server", "lnd", "fedimint"] {
assert!(!requires_unpruned_bitcoin(package_id), "{package_id}");
}
}
}
File diff suppressed because it is too large Load Diff
+686 -67
View File
@@ -1,13 +1,21 @@
use super::config::{get_containers_for_app, get_data_dirs_for_app, is_valid_docker_image};
use super::config::{
get_app_capabilities, get_containers_for_app, get_data_dirs_for_app, get_health_check_args,
get_memory_limit, is_valid_docker_image,
};
use super::dependencies::ordered_containers_for_start;
use super::install::install_log;
use super::validation::validate_app_id;
use crate::api::rpc::RpcHandler;
use crate::data_model::PackageState;
use anyhow::{Context, Result};
use std::process::Output;
use std::sync::Arc;
use std::time::Duration;
use tracing::warn;
const PODMAN_CONTROL_TIMEOUT: Duration = Duration::from_secs(30);
const PODMAN_LOG_TIMEOUT: Duration = Duration::from_secs(15);
/// Per-container graceful shutdown timeout in seconds.
/// Bitcoin Core needs 600s to flush UTXO set, LND 330s for channel state,
/// indexers 300s for index flush, databases 120s for WAL/transaction commit.
@@ -59,6 +67,8 @@ impl RpcHandler {
}
let package_id_owned = package_id.to_string();
let companion_app_id = package_id_owned.clone();
let orchestrator = self.orchestrator.clone();
let state_manager = Arc::clone(&self.state_manager);
let pre_state =
flip_package_state(&state_manager, &package_id_owned, PackageState::Starting).await;
@@ -70,8 +80,14 @@ impl RpcHandler {
.await;
tokio::spawn(async move {
match do_package_start(&to_start).await {
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_start(orchestrator.as_ref(), &to_start).await
} else {
do_package_start(&to_start).await
};
match result {
Ok(()) => {
reconcile_companions_for(&companion_app_id).await;
set_package_state(&state_manager, &package_id_owned, PackageState::Running)
.await;
}
@@ -123,6 +139,8 @@ impl RpcHandler {
}
let package_id_owned = package_id.to_string();
let to_stop = containers.clone();
let orchestrator = self.orchestrator.clone();
let state_manager = Arc::clone(&self.state_manager);
let pre_state =
flip_package_state(&state_manager, &package_id_owned, PackageState::Stopping).await;
@@ -134,7 +152,12 @@ impl RpcHandler {
.await;
tokio::spawn(async move {
match do_package_stop(&containers).await {
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_stop(orchestrator.as_ref(), &to_stop).await
} else {
do_package_stop(&containers).await
};
match result {
Ok(()) => {
set_package_state(&state_manager, &package_id_owned, PackageState::Stopped)
.await;
@@ -182,7 +205,10 @@ impl RpcHandler {
}
let package_id_owned = package_id.to_string();
let companion_app_id = package_id_owned.clone();
let to_restart = ordered_containers_for_start(package_id).await?;
let state_manager = Arc::clone(&self.state_manager);
let orchestrator = self.orchestrator.clone();
let pre_state =
flip_package_state(&state_manager, &package_id_owned, PackageState::Restarting).await;
@@ -193,8 +219,14 @@ impl RpcHandler {
.await;
tokio::spawn(async move {
match do_package_restart(&containers).await {
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_restart(orchestrator.as_ref(), &to_restart).await
} else {
do_package_restart(&containers).await
};
match result {
Ok(()) => {
reconcile_companions_for(&companion_app_id).await;
set_package_state(&state_manager, &package_id_owned, PackageState::Running)
.await;
}
@@ -227,6 +259,20 @@ impl RpcHandler {
.and_then(|v| v.as_bool())
.unwrap_or(false);
// Disable + remove Quadlet companion units BEFORE the rm loop.
// Otherwise systemd's Restart=always will respawn each companion
// within ~10s of `podman rm`, leaving them orphaned post-uninstall.
crate::container::companion::remove_for(package_id).await;
// Keep the production reconciler from recreating an app immediately
// after uninstall. The reconciler owns a manifest map independent of
// podman state, so a raw `podman rm` alone is not enough.
if let Some(orchestrator) = &self.orchestrator {
for app_id in orchestrator_uninstall_app_ids(package_id) {
let _ = orchestrator.remove(&app_id, preserve_data).await;
}
}
let containers_to_remove = get_containers_for_app(package_id).await?;
if containers_to_remove.is_empty() {
tracing::warn!("Uninstall {}: no containers found", package_id);
@@ -254,10 +300,7 @@ impl RpcHandler {
)
.await;
tracing::info!("Uninstall {}: stopping container {}", package_id, name);
let stop_out = tokio::process::Command::new("podman")
.args(["stop", "-t", stop_timeout_secs(name), name])
.output()
.await;
let stop_out = podman_control(&["stop", "-t", stop_timeout_secs(name), name]).await;
match stop_out {
Ok(o) if o.status.success() => stopped += 1,
Ok(o) => {
@@ -276,10 +319,7 @@ impl RpcHandler {
// Remove container (without -f to respect graceful shutdown above)
tracing::info!("Uninstall {}: removing container {}", package_id, name);
let rm_out = tokio::process::Command::new("podman")
.args(["rm", name])
.output()
.await;
let rm_out = podman_control(&["rm", name]).await;
match rm_out {
Ok(o) if o.status.success() => removed += 1,
Ok(o) => {
@@ -291,10 +331,7 @@ impl RpcHandler {
name,
stderr.trim()
);
let force_rm = tokio::process::Command::new("podman")
.args(["rm", "-f", name])
.output()
.await;
let force_rm = podman_control(&["rm", "-f", name]).await;
match force_rm {
Ok(o2) if o2.status.success() => removed += 1,
_ => {
@@ -315,10 +352,7 @@ impl RpcHandler {
self.set_uninstall_stage(package_id, "Cleaning up volumes")
.await;
// Clean up dangling volumes associated with removed containers
let _ = tokio::process::Command::new("podman")
.args(["volume", "prune", "-f"])
.output()
.await;
let _ = podman_control(&["volume", "prune", "-f"]).await;
// Clean up app-specific networks (only if no other containers use them)
let app_networks: Vec<&str> = match package_id {
@@ -328,16 +362,17 @@ impl RpcHandler {
_ => vec![],
};
for net in &app_networks {
let _ = tokio::process::Command::new("podman")
.args(["network", "rm", net])
.output()
.await;
let _ = podman_control(&["network", "rm", net]).await;
}
// Release port allocation
{
let mut allocator = self.port_allocator.lock().await;
let _ = allocator.release(package_id).await;
if package_id == "nginx-proxy-manager" {
let _ = allocator.release("nginx-proxy-manager-http").await;
let _ = allocator.release("nginx-proxy-manager-https").await;
}
}
// Clean data directories unless preserve_data
@@ -447,18 +482,16 @@ impl RpcHandler {
.and_then(|v| v.as_array())
.ok_or_else(|| anyhow::anyhow!("Missing volumes"))?;
let check_output = tokio::process::Command::new("podman")
.args([
"ps",
"-a",
"--format",
"{{.Names}}",
"--filter",
&format!("name={}", app_id),
])
.output()
.await
.context("Failed to check container")?;
let check_output = podman_control(&[
"ps",
"-a",
"--format",
"{{.Names}}",
"--filter",
&format!("name={}", app_id),
])
.await
.context("Failed to check container")?;
let existing = String::from_utf8_lossy(&check_output.stdout);
@@ -503,16 +536,14 @@ impl RpcHandler {
cmd.arg(image);
let output = cmd.output().await.context("Failed to create container")?;
let output = command_with_timeout(cmd, PODMAN_CONTROL_TIMEOUT, "podman run").await?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(anyhow::anyhow!("Failed to create container: {}", stderr));
}
} else {
let output = tokio::process::Command::new("podman")
.args(["start", app_id])
.output()
let output = podman_control(&["start", app_id])
.await
.context("Failed to start container")?;
@@ -537,9 +568,7 @@ impl RpcHandler {
.ok_or_else(|| anyhow::anyhow!("Missing app_id"))?;
validate_app_id(app_id)?;
let output = tokio::process::Command::new("podman")
.args(["stop", "-t", stop_timeout_secs(app_id), app_id])
.output()
let output = podman_control(&["stop", "-t", stop_timeout_secs(app_id), app_id])
.await
.context("Failed to stop container")?;
@@ -571,15 +600,15 @@ async fn do_package_start(to_start: &[String]) -> Result<()> {
if i > 0 {
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
repair_before_package_start(name).await;
tracing::info!("Starting container: {}", name);
let out = tokio::process::Command::new("podman")
.args(["start", name])
.output()
let out = podman_control(&["start", name])
.await
.context(format!("Failed to exec podman start {}", name))?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
tracing::error!("Failed to start {}: {}", name, stderr);
cleanup_start_conflict(name, &stderr).await;
install_log(&format!("START FAIL: {}{}", name, stderr)).await;
errors.push(format!("{}: {}", name, stderr));
}
@@ -592,17 +621,12 @@ async fn do_package_start(to_start: &[String]) -> Result<()> {
// container exits immediately after).
tokio::time::sleep(std::time::Duration::from_secs(3)).await;
for name in to_start {
let status = tokio::process::Command::new("podman")
.args(["inspect", name, "--format", "{{.State.Status}}"])
.output()
.await;
let status = podman_control(&["inspect", name, "--format", "{{.State.Status}}"]).await;
if let Ok(o) = status {
let state = String::from_utf8_lossy(&o.stdout).trim().to_string();
if state == "exited" {
let logs = tokio::process::Command::new("podman")
.args(["logs", "--tail", "5", name])
.output()
.await;
let logs =
podman_with_timeout(&["logs", "--tail", "5", name], PODMAN_LOG_TIMEOUT).await;
let log_text = logs
.map(|o| {
let combined = format!(
@@ -625,9 +649,123 @@ async fn do_package_start(to_start: &[String]) -> Result<()> {
errors.join("; ")
));
}
for name in to_start {
ensure_runtime_host_port_listener(name).await?;
}
if to_start.iter().any(|name| name == "indeedhub") {
super::install::patch_indeedhub_nostr_provider().await;
}
Ok(())
}
async fn do_orchestrator_package_start(
orchestrator: &dyn crate::container::traits::ContainerOrchestrator,
to_start: &[String],
) -> Result<()> {
let mut errors = Vec::new();
for (i, name) in to_start.iter().enumerate() {
if i > 0 {
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
repair_before_package_start(name).await;
match orchestrator.start(name).await {
Ok(()) => wait_after_orchestrator_start(name).await,
Err(e) if is_unknown_app_id_error(&e) => {
do_package_start(&[name.clone()]).await?;
}
Err(e) => {
tracing::error!(container = %name, error = %e, "orchestrator start failed");
install_log(&format!("START FAIL: {}{:#}", name, e)).await;
errors.push(format!("{}: {:#}", name, e));
}
}
}
if errors.is_empty() {
Ok(())
} else {
Err(anyhow::anyhow!("Start failed: {}", errors.join("; ")))
}
}
async fn wait_after_orchestrator_start(name: &str) {
let delay = match name {
"archy-btcpay-db" => 5,
"archy-nbxplorer" => 8,
_ => 0,
};
if delay > 0 {
tokio::time::sleep(std::time::Duration::from_secs(delay)).await;
}
}
async fn podman_control(args: &[&str]) -> Result<Output> {
podman_with_timeout(args, podman_control_timeout(args)).await
}
fn podman_control_timeout(args: &[&str]) -> Duration {
args.windows(2)
.find_map(|pair| {
(pair[0] == "-t")
.then(|| pair[1].parse::<u64>().ok())
.flatten()
})
.map(|secs| Duration::from_secs(secs.saturating_add(30)))
.unwrap_or(PODMAN_CONTROL_TIMEOUT)
}
async fn podman_with_timeout(args: &[&str], timeout: Duration) -> Result<Output> {
let mut cmd = tokio::process::Command::new("podman");
cmd.args(args);
command_with_timeout(cmd, timeout, &format!("podman {}", args.join(" "))).await
}
async fn command_with_timeout(
mut cmd: tokio::process::Command,
timeout: Duration,
description: &str,
) -> Result<Output> {
cmd.kill_on_drop(true);
tokio::time::timeout(timeout, cmd.output())
.await
.with_context(|| format!("{} timed out after {}s", description, timeout.as_secs()))?
.with_context(|| format!("Failed to exec {}", description))
}
async fn do_orchestrator_package_stop(
orchestrator: &dyn crate::container::traits::ContainerOrchestrator,
containers: &[String],
) -> Result<()> {
let mut errors = Vec::new();
for name in containers.iter().rev() {
match orchestrator.stop(name).await {
Ok(()) => {}
Err(e) if is_unknown_app_id_error(&e) => {
if let Err(e) = do_package_stop(&[name.clone()]).await {
errors.push(format!("{}: {:#}", name, e));
}
}
Err(e) => {
tracing::error!(container = %name, error = %e, "orchestrator stop failed");
errors.push(format!("{}: {:#}", name, e));
}
}
}
if errors.is_empty() {
Ok(())
} else {
Err(anyhow::anyhow!("Stop failed: {}", errors.join("; ")))
}
}
async fn do_orchestrator_package_restart(
orchestrator: &dyn crate::container::traits::ContainerOrchestrator,
to_restart: &[String],
) -> Result<()> {
do_orchestrator_package_stop(orchestrator, to_restart).await?;
do_orchestrator_package_start(orchestrator, to_restart).await
}
/// Stop all containers with their per-container graceful-shutdown timeout.
async fn do_package_stop(containers: &[String]) -> Result<()> {
let mut errors = Vec::new();
@@ -637,13 +775,15 @@ async fn do_package_stop(containers: &[String]) -> Result<()> {
name,
stop_timeout_secs(name)
);
let out = tokio::process::Command::new("podman")
.args(["stop", "-t", stop_timeout_secs(name), name])
.output()
let out = podman_control(&["stop", "-t", stop_timeout_secs(name), name])
.await
.context(format!("Failed to exec podman stop {}", name))?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
if is_missing_companion_ok(name, &stderr) {
tracing::debug!(container = %name, "companion already absent during stop");
continue;
}
tracing::error!("Failed to stop {}: {}", name, stderr);
errors.push(format!("{}: {}", name, stderr));
}
@@ -660,39 +800,46 @@ async fn do_package_restart(containers: &[String]) -> Result<()> {
let mut errors = Vec::new();
for name in containers {
tracing::info!("Restarting container: {}", name);
let out = tokio::process::Command::new("podman")
.args(["restart", "-t", stop_timeout_secs(name), name])
.output()
repair_before_package_start(name).await;
let out = podman_control(&["restart", "-t", stop_timeout_secs(name), name])
.await
.context(format!("Failed to exec podman restart {}", name))?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
if is_missing_companion_ok(name, &stderr) {
tracing::debug!(container = %name, "companion absent during restart; reconcile will recreate it");
continue;
}
tracing::warn!(
"podman restart {} failed: {}, trying stop+start",
name,
stderr
);
// Fallback: stop then start
let _ = tokio::process::Command::new("podman")
.args(["stop", "-t", stop_timeout_secs(name), name])
.output()
.await;
let start_out = tokio::process::Command::new("podman")
.args(["start", name])
.output()
let _ = podman_control(&["stop", "-t", stop_timeout_secs(name), name]).await;
let start_out = podman_control(&["start", name])
.await
.context(format!("Failed to exec podman start {}", name))?;
if !start_out.status.success() {
let start_err = String::from_utf8_lossy(&start_out.stderr)
.trim()
.to_string();
cleanup_start_conflict(name, &start_err).await;
if is_missing_companion_ok(name, &start_err) {
tracing::debug!(container = %name, "companion absent during restart fallback; reconcile will recreate it");
continue;
}
tracing::error!("stop+start {} also failed: {}", name, start_err);
errors.push(format!("{}: {}", name, start_err));
} else {
tracing::info!("Restarted {} via stop+start fallback", name);
}
}
ensure_runtime_host_port_listener(name).await?;
}
if containers.iter().any(|name| name == "indeedhub") {
super::install::patch_indeedhub_nostr_provider().await;
}
if !errors.is_empty() {
return Err(anyhow::anyhow!("Restart failed: {}", errors.join("; ")));
@@ -700,6 +847,440 @@ async fn do_package_restart(containers: &[String]) -> Result<()> {
Ok(())
}
fn is_unknown_app_id_error(err: &anyhow::Error) -> bool {
err.chain()
.any(|cause| cause.to_string().contains("unknown app_id"))
}
async fn repair_before_package_start(container_name: &str) {
match container_name {
"btcpay-server" | "archy-nbxplorer" => repair_btcpay_dirs().await,
"indeedhub-postgres" | "indeedhub-redis" | "indeedhub-minio" | "indeedhub-relay"
| "indeedhub-api" | "indeedhub-ffmpeg" | "indeedhub" => repair_indeedhub_network().await,
"grafana" => {
repair_grafana_dirs().await;
cleanup_stale_pasta_port("3000").await;
}
"vaultwarden" => cleanup_stale_pasta_port("8082").await,
"homeassistant" | "home-assistant" => cleanup_stale_pasta_port("8123").await,
"nextcloud" => {
repair_nextcloud_dirs().await;
cleanup_stale_pasta_port("8085").await;
}
"nginx-proxy-manager" => repair_nginx_proxy_manager_container().await,
"gitea" => cleanup_gitea_stale_ports().await,
_ => {}
}
}
async fn repair_nginx_proxy_manager_container() {
repair_nginx_proxy_manager_dirs().await;
if !nginx_proxy_manager_has_legacy_admin_port().await {
cleanup_nginx_proxy_manager_ports().await;
return;
}
install_log(
"START REPAIR: nginx-proxy-manager - recreating stale container using host port 8081",
)
.await;
let _ = podman_control(&["rm", "-f", "nginx-proxy-manager"]).await;
cleanup_nginx_proxy_manager_ports().await;
if let Err(err) = recreate_nginx_proxy_manager_container().await {
tracing::warn!(error = %err, "failed to recreate stale nginx-proxy-manager container");
}
}
async fn repair_nginx_proxy_manager_dirs() {
let _ = tokio::process::Command::new("sudo")
.args([
"mkdir",
"-p",
"/var/lib/archipelago/nginx-proxy-manager/data/letsencrypt-acme-challenge/.well-known/acme-challenge",
"/var/lib/archipelago/nginx-proxy-manager/letsencrypt",
])
.output()
.await;
let _ = tokio::process::Command::new("sudo")
.args([
"chown",
"-R",
"1000:1000",
"/var/lib/archipelago/nginx-proxy-manager",
])
.output()
.await;
}
async fn nginx_proxy_manager_has_legacy_admin_port() -> bool {
if let Ok(output) = podman_control(&["port", "nginx-proxy-manager", "81/tcp"]).await {
if output.status.success()
&& String::from_utf8_lossy(&output.stdout).lines().any(|line| {
line.rsplit(':')
.next()
.is_some_and(|port| port.trim() == "81")
})
{
return true;
}
}
let Ok(output) = podman_control(&[
"ps",
"-a",
"--format",
"{{.Ports}}",
"--filter",
"name=^nginx-proxy-manager$",
])
.await
else {
return false;
};
if !output.status.success() {
return false;
}
let ports = String::from_utf8_lossy(&output.stdout);
ports.contains(":81->81/tcp") || ports.contains(":8443->443/tcp")
}
async fn recreate_nginx_proxy_manager_container() -> Result<()> {
tokio::process::Command::new("sudo")
.args([
"mkdir",
"-p",
"/var/lib/archipelago/nginx-proxy-manager/data/letsencrypt-acme-challenge/.well-known/acme-challenge",
"/var/lib/archipelago/nginx-proxy-manager/letsencrypt",
])
.output()
.await
.context("failed to create nginx-proxy-manager data directories")?;
let _ = tokio::process::Command::new("sudo")
.args([
"chown",
"-R",
"1000:1000",
"/var/lib/archipelago/nginx-proxy-manager",
])
.output()
.await;
let image = crate::container::image_versions::pinned_image_for_app("nginx-proxy-manager")
.unwrap_or_else(|| "docker.io/jc21/nginx-proxy-manager:latest".to_string());
let mut args = vec![
"run".to_string(),
"-d".to_string(),
"--name".to_string(),
"nginx-proxy-manager".to_string(),
"--restart=unless-stopped".to_string(),
"--network=slirp4netns:allow_host_loopback=true".to_string(),
"--cap-drop=ALL".to_string(),
"--security-opt=no-new-privileges:true".to_string(),
"--pids-limit=4096".to_string(),
];
args.extend(get_app_capabilities("nginx-proxy-manager"));
args.extend([
"-p".to_string(),
"8081:81".to_string(),
"-p".to_string(),
"8084:80".to_string(),
"-p".to_string(),
"8444:443".to_string(),
"-v".to_string(),
"/var/lib/archipelago/nginx-proxy-manager/data:/data".to_string(),
"-v".to_string(),
"/var/lib/archipelago/nginx-proxy-manager/letsencrypt:/etc/letsencrypt".to_string(),
"--memory".to_string(),
get_memory_limit("nginx-proxy-manager").to_string(),
"--cpus=2".to_string(),
]);
args.extend(get_health_check_args("nginx-proxy-manager", ""));
args.push(image);
let refs = args.iter().map(String::as_str).collect::<Vec<_>>();
let output = podman_control(&refs).await?;
if !output.status.success() {
anyhow::bail!(
"podman run nginx-proxy-manager failed: {}",
String::from_utf8_lossy(&output.stderr).trim()
);
}
Ok(())
}
async fn ensure_runtime_host_port_listener(container_name: &str) -> Result<()> {
let Some(port) = runtime_required_host_port(container_name) else {
return Ok(());
};
if wait_for_runtime_host_port(port, 10).await {
return Ok(());
}
install_log(&format!(
"START REPAIR: {} — host port {} missing after start; restarting container",
container_name, port
))
.await;
let output = podman_control(&["restart", container_name])
.await
.context("failed to restart container after missing host port")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(anyhow::anyhow!(
"Container {} host port {} was not listening and restart failed: {}",
container_name,
port,
stderr.trim()
));
}
if wait_for_runtime_host_port(port, 60).await {
install_log(&format!(
"START REPAIR OK: {} — host port {} is listening after restart",
container_name, port
))
.await;
return Ok(());
}
Err(anyhow::anyhow!(
"Container {} is running but host port {} is not listening",
container_name,
port
))
}
fn runtime_required_host_port(container_name: &str) -> Option<u16> {
match container_name {
"grafana" => Some(3000),
"homeassistant" | "home-assistant" => Some(8123),
"searxng" => Some(8888),
"uptime-kuma" => Some(3002),
"vaultwarden" => Some(8082),
"gitea" => Some(3001),
"nextcloud" => Some(8085),
"nginx-proxy-manager" => Some(8081),
_ => None,
}
}
async fn wait_for_runtime_host_port(port: u16, timeout_secs: u64) -> bool {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(timeout_secs);
loop {
if tokio::net::TcpStream::connect(("127.0.0.1", port))
.await
.is_ok()
{
return true;
}
if std::time::Instant::now() >= deadline {
return false;
}
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
}
async fn repair_btcpay_dirs() {
let _ = tokio::process::Command::new("sudo")
.args([
"mkdir",
"-p",
"/var/lib/archipelago/btcpay/Main",
"/var/lib/archipelago/nbxplorer/Main",
])
.output()
.await;
for dir in [
"/var/lib/archipelago/btcpay",
"/var/lib/archipelago/nbxplorer",
] {
let _ = tokio::process::Command::new("sudo")
.args(["chown", "-R", "1000:1000", dir])
.output()
.await;
}
repair_btcpay_database_password().await;
}
async fn repair_grafana_dirs() {
let _ = tokio::process::Command::new("sudo")
.args(["mkdir", "-p", "/var/lib/archipelago/grafana"])
.output()
.await;
let podman_chown = podman_control(&[
"unshare",
"chown",
"-R",
"472:472",
"/var/lib/archipelago/grafana",
])
.await;
if !podman_chown.as_ref().is_ok_and(|o| o.status.success()) {
let _ = tokio::process::Command::new("sudo")
.args([
"chown",
"-R",
"100471:100471",
"/var/lib/archipelago/grafana",
])
.output()
.await;
}
}
async fn repair_nextcloud_dirs() {
let _ = tokio::process::Command::new("sudo")
.args(["mkdir", "-p", "/var/lib/archipelago/nextcloud"])
.output()
.await;
let podman_chown = podman_control(&[
"unshare",
"chown",
"-R",
"33:33",
"/var/lib/archipelago/nextcloud",
])
.await;
if !podman_chown.as_ref().is_ok_and(|o| o.status.success()) {
let _ = tokio::process::Command::new("sudo")
.args([
"chown",
"-R",
"100032:100032",
"/var/lib/archipelago/nextcloud",
])
.output()
.await;
}
}
async fn repair_btcpay_database_password() {
let Ok(db_pass) =
tokio::fs::read_to_string("/var/lib/archipelago/secrets/btcpay-db-password").await
else {
return;
};
let db_pass = db_pass.trim();
if db_pass.is_empty() {
return;
}
let _ = podman_control(&["start", "archy-btcpay-db"]).await;
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
let escaped = db_pass.replace('\'', "''");
let sql = format!("ALTER USER btcpay WITH PASSWORD '{}';", escaped);
let _ = podman_control(&[
"exec",
"archy-btcpay-db",
"psql",
"-U",
"btcpay",
"-d",
"btcpay",
"-c",
&sql,
])
.await;
let _ = podman_control(&[
"exec",
"archy-btcpay-db",
"createdb",
"-U",
"btcpay",
"nbxplorer",
])
.await;
}
async fn repair_indeedhub_network() {
super::stacks::repair_indeedhub_network_aliases().await;
}
async fn cleanup_start_conflict(container_name: &str, stderr: &str) {
if !stderr.contains("address already in use") && !stderr.contains("pasta failed") {
return;
}
if container_name == "gitea" {
cleanup_gitea_stale_ports().await;
return;
}
match container_name {
"grafana" => cleanup_stale_pasta_port("3000").await,
"homeassistant" | "home-assistant" => cleanup_stale_pasta_port("8123").await,
"vaultwarden" => cleanup_stale_pasta_port("8082").await,
"nextcloud" => cleanup_stale_pasta_port("8085").await,
"nginx-proxy-manager" => cleanup_nginx_proxy_manager_ports().await,
_ => {}
}
}
async fn cleanup_nginx_proxy_manager_ports() {
cleanup_stale_pasta_port("8081").await;
cleanup_stale_pasta_port("8084").await;
cleanup_stale_pasta_port("8444").await;
}
async fn cleanup_stale_pasta_port(port: &str) {
let kill_listener = format!(
"ss -ltnp 'sport = :{}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | xargs -r kill 2>/dev/null || true",
port
);
let _ = tokio::process::Command::new("sh")
.args(["-c", &kill_listener])
.output()
.await;
let pattern = format!("pasta.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
let pattern = format!("rootlessport.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
async fn cleanup_gitea_stale_ports() {
for port in ["3001", "2222", "3000"] {
let kill_listener = format!(
"ss -ltnp 'sport = :{}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | xargs -r kill 2>/dev/null || true",
port
);
let _ = tokio::process::Command::new("sh")
.args(["-c", &kill_listener])
.output()
.await;
let pattern = format!("pasta.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
let pattern = format!("rootlessport.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
}
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
pub(super) fn is_missing_companion_ok(name: &str, stderr: &str) -> bool {
matches!(
name,
"archy-bitcoin-ui" | "archy-lnd-ui" | "archy-electrs-ui"
) && stderr.contains("no container with name or ID")
}
/// Flip the primary package entry's state and return the pre-transition
/// state for revert on error. Mirrors `transitional::flip_to_transitional`
/// but lives here because the package path keys by `package_id` (which may
@@ -733,3 +1314,41 @@ async fn set_package_state(
}
}
}
pub(super) async fn reconcile_companions_for(package_id: &str) {
let app_ids = match package_id {
"bitcoin" | "bitcoin-core" => vec!["bitcoin-core".to_string(), "bitcoin-ui".to_string()],
"bitcoin-knots" => vec!["bitcoin-knots".to_string(), "bitcoin-ui".to_string()],
"lnd" => vec!["lnd".to_string(), "lnd-ui".to_string()],
"electrumx" | "electrs" | "mempool-electrs" => {
vec!["electrumx".to_string(), "electrs-ui".to_string()]
}
_ => return,
};
for (companion, err) in crate::container::companion::reconcile(&app_ids).await {
tracing::warn!(companion = %companion, error = %err, "companion reconcile failed");
}
}
pub(super) fn orchestrator_uninstall_app_ids(package_id: &str) -> Vec<String> {
match package_id {
"bitcoin" | "bitcoin-core" => vec!["bitcoin-core".into(), "bitcoin-ui".into()],
"bitcoin-knots" => vec!["bitcoin-knots".into(), "bitcoin-ui".into()],
"lnd" => vec!["lnd".into(), "lnd-ui".into()],
"electrumx" | "electrs" | "mempool-electrs" => {
vec!["electrumx".into(), "electrs-ui".into()]
}
"mempool" | "mempool-web" => vec![
"mempool-api".into(),
"archy-mempool-web".into(),
"archy-mempool-db".into(),
],
"btcpay-server" | "btcpayserver" | "btcpay" => vec![
"btcpay-server".into(),
"archy-nbxplorer".into(),
"archy-btcpay-db".into(),
],
"fedimint" => vec!["fedimint".into(), "fedimint-gateway".into()],
_ => vec![package_id.to_string()],
}
}
+260 -50
View File
@@ -8,7 +8,7 @@ use crate::data_model::InstallPhase;
use anyhow::{Context, Result};
use tracing::info;
use super::install::install_log;
use super::install::{install_log, patch_indeedhub_nostr_provider};
/// Adopt an existing container stack: start all named containers and return success.
/// Returns `Ok(Some(json))` if the primary container was found (adopted),
@@ -40,6 +40,8 @@ async fn adopt_stack_if_exists(
))
.await;
repair_stack_before_adopt(stack_name).await;
for container in all_containers {
if names.iter().any(|n| n == container) {
let _ = tokio::process::Command::new("podman")
@@ -48,6 +50,16 @@ async fn adopt_stack_if_exists(
.await;
}
}
let existing: Vec<&str> = all_containers
.iter()
.copied()
.filter(|container| names.iter().any(|n| n == container))
.collect();
wait_for_stack_containers(stack_name, &existing, 60).await?;
if stack_name == "indeedhub" {
patch_indeedhub_nostr_provider().await;
}
install_log(&format!(
"INSTALL ADOPT OK: {} — started existing containers",
@@ -61,6 +73,177 @@ async fn adopt_stack_if_exists(
})))
}
async fn repair_stack_before_adopt(stack_name: &str) {
match stack_name {
"btcpay" | "btcpay-server" => {
let _ = tokio::process::Command::new("sudo")
.args([
"mkdir",
"-p",
"/var/lib/archipelago/btcpay/Main",
"/var/lib/archipelago/nbxplorer/Main",
])
.output()
.await;
let user = std::env::var("USER").unwrap_or_else(|_| "archipelago".to_string());
for dir in [
"/var/lib/archipelago/btcpay",
"/var/lib/archipelago/nbxplorer",
] {
let _ = tokio::process::Command::new("sudo")
.args(["chown", "-R", &format!("{}:{}", user, user), dir])
.output()
.await;
}
}
"indeedhub" => repair_indeedhub_network_aliases().await,
_ => {}
}
}
pub(in crate::api::rpc::package) async fn repair_indeedhub_network_aliases() {
let _ = tokio::process::Command::new("podman")
.args(["network", "create", "indeedhub-net"])
.output()
.await;
for (container, alias) in [
("indeedhub-postgres", "postgres"),
("indeedhub-redis", "redis"),
("indeedhub-minio", "minio"),
("indeedhub-relay", "relay"),
("indeedhub-api", "api"),
("indeedhub", "indeedhub"),
] {
let exists = tokio::process::Command::new("podman")
.args(["container", "exists", container])
.status()
.await
.map(|s| s.success())
.unwrap_or(false);
if !exists {
continue;
}
let _ = tokio::process::Command::new("podman")
.args(["network", "disconnect", "-f", "indeedhub-net", container])
.output()
.await;
let _ = tokio::process::Command::new("podman")
.args([
"network",
"connect",
"--alias",
alias,
"indeedhub-net",
container,
])
.output()
.await;
}
}
async fn run_required_stack_command(
stack_name: &str,
label: &str,
cmd: &mut tokio::process::Command,
) -> Result<()> {
let output = cmd
.output()
.await
.with_context(|| format!("{}: failed to run {}", stack_name, label))?;
if output.status.success() {
return Ok(());
}
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
let msg = format!("{} failed: {}{}", label, stdout, stderr);
install_log(&format!("INSTALL FAIL: {} - {}", stack_name, msg.trim())).await;
Err(anyhow::anyhow!("{} {}", stack_name, msg.trim()))
}
async fn wait_for_stack_containers(
stack_name: &str,
containers: &[&str],
timeout_secs: u64,
) -> Result<()> {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(timeout_secs);
loop {
let mut pending = Vec::new();
for container in containers {
let status = tokio::process::Command::new("podman")
.args(["inspect", container, "--format", "{{.State.Status}}"])
.output()
.await;
match status {
Ok(output) if output.status.success() => {
let state = String::from_utf8_lossy(&output.stdout).trim().to_string();
match state.as_str() {
"running" => {}
"exited" | "dead" => {
let logs = stack_container_logs(container, 40).await;
install_log(&format!(
"INSTALL CRASH: {} - container {} exited. Logs:\n{}",
stack_name,
container,
logs.chars().take(1000).collect::<String>()
))
.await;
return Err(anyhow::anyhow!(
"{} container {} exited after install. Logs: {}",
stack_name,
container,
logs.chars().take(500).collect::<String>()
));
}
other => pending.push(format!("{}={}", container, other)),
}
}
Ok(output) => {
pending.push(format!(
"{}=missing({})",
container,
String::from_utf8_lossy(&output.stderr).trim()
));
}
Err(e) => pending.push(format!("{}=inspect-error({})", container, e)),
}
}
if pending.is_empty() {
return Ok(());
}
if std::time::Instant::now() >= deadline {
install_log(&format!(
"INSTALL TIMEOUT: {} - containers not running: {}",
stack_name,
pending.join(", ")
))
.await;
return Err(anyhow::anyhow!(
"{} containers did not reach running state: {}",
stack_name,
pending.join(", ")
));
}
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
}
async fn stack_container_logs(container: &str, lines: u32) -> String {
tokio::process::Command::new("podman")
.args(["logs", "--tail", &lines.to_string(), container])
.output()
.await
.map(|o| {
let stdout = String::from_utf8_lossy(&o.stdout);
let stderr = String::from_utf8_lossy(&o.stderr);
format!("{}{}", stdout, stderr)
})
.unwrap_or_default()
}
async fn install_stack_via_orchestrator(
handler: &RpcHandler,
stack_name: &str,
@@ -133,8 +316,13 @@ async fn pull_image_with_retry(image: &str) -> Result<()> {
const BACKOFF_SECS: [u64; 3] = [5, 15, 45];
for attempt in 1..=MAX_ATTEMPTS {
let output = tokio::process::Command::new("podman")
.args(["pull", image])
let mut cmd = tokio::process::Command::new("podman");
cmd.arg("pull");
if archipelago_container::image_uses_insecure_registry(image) {
cmd.arg("--tls-verify=false");
}
let output = cmd
.arg(image)
.output()
.await
.context("Failed to execute podman pull")?;
@@ -208,10 +396,12 @@ impl RpcHandler {
.await;
let n_images = images.len() as u64;
for (i, img) in images.iter().enumerate() {
self.set_install_progress("immich", i as u64, n_images).await;
self.set_install_progress("immich", i as u64, n_images)
.await;
pull_image_with_retry(img).await?;
}
self.set_install_progress("immich", n_images, n_images).await;
self.set_install_progress("immich", n_images, n_images)
.await;
self.set_install_phase("immich", InstallPhase::CreatingContainer)
.await;
@@ -260,7 +450,7 @@ impl RpcHandler {
"--cap-add=SETGID",
"--cap-add=SETUID",
"--security-opt=no-new-privileges:true",
"--memory=512m",
"--memory=2g",
"--pids-limit=4096",
"--health-cmd=pg_isready -U postgres || exit 1",
"--health-interval=30s",
@@ -366,6 +556,12 @@ impl RpcHandler {
/// Install BTCPay stack (postgres + nbxplorer + btcpay-server).
pub(super) async fn install_btcpay_stack(&self) -> Result<serde_json::Value> {
if let Some(orchestrated) =
install_stack_via_orchestrator(self, "btcpay-server", btcpay_stack_app_ids()).await?
{
return Ok(orchestrated);
}
if let Some(adopted) = adopt_stack_if_exists(
"btcpay-server",
"btcpay",
@@ -376,12 +572,6 @@ impl RpcHandler {
return Ok(adopted);
}
if let Some(orchestrated) =
install_stack_via_orchestrator(self, "btcpay-server", btcpay_stack_app_ids()).await?
{
return Ok(orchestrated);
}
// Dependency check: Bitcoin must be running
let deps = super::dependencies::detect_running_deps().await?;
super::dependencies::check_install_deps("btcpay-server", &deps)?;
@@ -397,7 +587,7 @@ impl RpcHandler {
let images = [
&format!("{}/postgres:15.17", REGISTRY),
&format!("{}/nbxplorer:2.6.0", REGISTRY),
&format!("{}/btcpayserver:1.13.7", REGISTRY),
"docker.io/btcpayserver/btcpayserver:2.3.9",
];
self.set_install_phase("btcpay-server", InstallPhase::PullingImage)
.await;
@@ -553,7 +743,7 @@ impl RpcHandler {
"BTCPAY_POSTGRES=User ID=btcpay;Password={};Host=archy-btcpay-db;Port=5432;Database=btcpay;Include Error Detail=true",
db_pass
),
&format!("{}/btcpayserver:1.13.7", REGISTRY),
"docker.io/btcpayserver/btcpayserver:2.3.9",
])
.output()
.await
@@ -585,6 +775,8 @@ impl RpcHandler {
/// Install Mempool stack (mariadb + mempool-api + mempool-web).
pub(super) async fn install_mempool_stack(&self) -> Result<serde_json::Value> {
super::dependencies::check_bitcoin_pruning_compatibility("mempool").await?;
if let Some(adopted) = adopt_stack_if_exists(
"mempool",
"mempool",
@@ -626,10 +818,12 @@ impl RpcHandler {
.await;
let n_images = images.len() as u64;
for (i, img) in images.iter().enumerate() {
self.set_install_progress("mempool", i as u64, n_images).await;
self.set_install_progress("mempool", i as u64, n_images)
.await;
pull_image_with_retry(img).await?;
}
self.set_install_progress("mempool", n_images, n_images).await;
self.set_install_progress("mempool", n_images, n_images)
.await;
self.set_install_phase("mempool", InstallPhase::CreatingContainer)
.await;
@@ -901,7 +1095,8 @@ impl RpcHandler {
let minio_pass = super::config::read_or_generate_secret("indeedhub-minio-password").await;
// 1. Postgres
let _ = tokio::process::Command::new("podman")
let mut postgres_cmd = tokio::process::Command::new("podman");
postgres_cmd
.args([
"run",
"-d",
@@ -923,12 +1118,12 @@ impl RpcHandler {
"indeedhub-postgres-data:/var/lib/postgresql/data",
&format!("{}/postgres:16.13-alpine", registry),
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create postgres", &mut postgres_cmd).await?;
// 2. Redis
let _ = tokio::process::Command::new("podman")
let mut redis_cmd = tokio::process::Command::new("podman");
redis_cmd
.args([
"run",
"-d",
@@ -944,12 +1139,12 @@ impl RpcHandler {
"indeedhub-redis-data:/data",
&format!("{}/redis:7.4.8-alpine", registry),
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create redis", &mut redis_cmd).await?;
// 3. MinIO
let _ = tokio::process::Command::new("podman")
let mut minio_cmd = tokio::process::Command::new("podman");
minio_cmd
.args([
"run",
"-d",
@@ -971,12 +1166,12 @@ impl RpcHandler {
"server",
"/data",
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create minio", &mut minio_cmd).await?;
// 4. Nostr relay
let _ = tokio::process::Command::new("podman")
let mut relay_cmd = tokio::process::Command::new("podman");
relay_cmd
.args([
"run",
"-d",
@@ -992,12 +1187,12 @@ impl RpcHandler {
"indeedhub-relay-data:/usr/src/app/db",
&format!("{}/nostr-rs-relay:0.9.0", registry),
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create relay", &mut relay_cmd).await?;
// 5. API
let _ = tokio::process::Command::new("podman")
let mut api_cmd = tokio::process::Command::new("podman");
api_cmd
.args([
"run",
"-d",
@@ -1049,12 +1244,12 @@ impl RpcHandler {
"ENVIRONMENT=production",
&format!("{}/indeedhub-api:1.0.0", registry),
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create api", &mut api_cmd).await?;
// 6. FFmpeg worker
let _ = tokio::process::Command::new("podman")
let mut ffmpeg_cmd = tokio::process::Command::new("podman");
ffmpeg_cmd
.args([
"run",
"-d",
@@ -1096,15 +1291,15 @@ impl RpcHandler {
"AES_MASTER_SECRET=0123456789abcdef0123456789abcdef",
&format!("{}/indeedhub-ffmpeg:1.0.0", registry),
])
.env("TMPDIR", &user_tmp)
.status()
.await;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create ffmpeg worker", &mut ffmpeg_cmd).await?;
// Wait for backend services to start
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
// 7. Frontend (nginx)
let run = tokio::process::Command::new("podman")
let mut frontend_cmd = tokio::process::Command::new("podman");
frontend_cmd
.args([
"run",
"-d",
@@ -1114,19 +1309,31 @@ impl RpcHandler {
"indeedhub-net",
"--restart",
"unless-stopped",
"--tmpfs",
"/run:rw,nosuid,nodev,size=16m",
"--tmpfs",
"/var/cache/nginx:rw,nosuid,nodev,size=32m",
"-p",
"7778:7777",
&format!("{}/indeedhub:1.0.0", registry),
])
.env("TMPDIR", &user_tmp)
.output()
.await
.context("Failed to create indeedhub container")?;
.env("TMPDIR", &user_tmp);
run_required_stack_command("indeedhub", "create frontend", &mut frontend_cmd).await?;
if !run.status.success() {
let err = String::from_utf8_lossy(&run.stderr);
return Err(anyhow::anyhow!("IndeedHub frontend failed: {}", err));
}
wait_for_stack_containers(
"indeedhub",
&[
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
],
60,
)
.await?;
// Phase: WaitingHealthy → PostInstall → clear. The actual readiness
// gate is the package scanner's next sweep; this just gives the UI a
@@ -1136,9 +1343,12 @@ impl RpcHandler {
.await;
self.set_install_phase("indeedhub", InstallPhase::PostInstall)
.await;
self.set_install_phase("indeedhub", InstallPhase::Done).await;
self.set_install_phase("indeedhub", InstallPhase::Done)
.await;
self.clear_install_progress("indeedhub").await;
patch_indeedhub_nostr_provider().await;
install_log("INSTALL OK: indeedhub stack").await;
info!("IndeedHub stack installed");
Ok(serde_json::json!({
@@ -322,8 +322,13 @@ impl RpcHandler {
async fn pull_update_image(&self, package_id: &str, image: &str) -> Result<()> {
self.set_install_progress(package_id, 0, 0).await;
let mut child = tokio::process::Command::new("podman")
.args(["pull", image])
let mut cmd = tokio::process::Command::new("podman");
cmd.arg("pull");
if archipelago_container::image_uses_insecure_registry(image) {
cmd.arg("--tls-verify=false");
}
let mut child = cmd
.arg(image)
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
+2 -2
View File
@@ -353,9 +353,9 @@ pub(in crate::api::rpc) fn known_service_port(name: &str) -> u16 {
"immich" => 2283,
"photoprism" => 2342,
"penpot" => 9001,
"nginx-proxy-manager" => 81,
"nginx-proxy-manager" => 8081,
"vaultwarden" => 8343,
"indeedhub" => 7777,
"indeedhub" => 7778,
_ => 0,
}
}
+23 -20
View File
@@ -4,26 +4,9 @@ use anyhow::{Context, Result};
impl RpcHandler {
/// Check for available system updates.
/// Tries git-based check first (if repo exists), falls back to manifest-based.
/// Prefer manifest-based OTA so installed nodes with a checked-out repo do
/// not depend on a potentially stale git remote. Git remains a dev fallback.
pub(super) async fn handle_update_check(&self) -> Result<serde_json::Value> {
// Manifest override: when ARCHIPELAGO_UPDATE_URL is explicitly set,
// the operator wants OTA via manifest — typically a dev box where
// ~/archy/.git exists but isn't the intended update surface.
// Without this short-circuit the dev box always advertises "Pull
// & Rebuild" and can never exercise the manifest OTA path.
let manifest_override = std::env::var("ARCHIPELAGO_UPDATE_URL").is_ok();
let repo_dir = std::path::PathBuf::from(
std::env::var("HOME").unwrap_or_else(|_| "/home/archipelago".to_string()),
)
.join("archy");
if !manifest_override && repo_dir.join(".git").exists() {
if let Ok(git_status) = self.git_check_update(&repo_dir).await {
return Ok(git_status);
}
}
// Fall back to manifest-based check
let state = update::check_for_updates(&self.config.data_dir).await?;
let update_info = state.available_update.as_ref().map(|u| {
@@ -35,10 +18,30 @@ impl RpcHandler {
})
});
if update_info.is_some() {
return Ok(serde_json::json!({
"current_version": state.current_version,
"last_check": state.last_check,
"update_available": true,
"update": update_info,
"manifest_mirror": state.manifest_mirror,
}));
}
let repo_dir = std::path::PathBuf::from(
std::env::var("HOME").unwrap_or_else(|_| "/home/archipelago".to_string()),
)
.join("archy");
if std::env::var("ARCHIPELAGO_GIT_UPDATES").is_ok() && repo_dir.join(".git").exists() {
if let Ok(git_status) = self.git_check_update(&repo_dir).await {
return Ok(git_status);
}
}
Ok(serde_json::json!({
"current_version": state.current_version,
"last_check": state.last_check,
"update_available": update_info.is_some(),
"update_available": false,
"update": update_info,
"manifest_mirror": state.manifest_mirror,
}))
+3 -3
View File
@@ -381,14 +381,14 @@ async fn change_ssh_password(new_password: &str) -> Result<()> {
// usermod -p writes directly to /etc/shadow, bypassing PAM
// Use /usr/sbin/usermod - not always in systemd's PATH
let status = tokio::process::Command::new("/usr/sbin/usermod")
.args(["-p", &hash, &ssh_user])
let status = tokio::process::Command::new("/usr/bin/sudo")
.args(["-n", "/usr/sbin/usermod", "-p", &hash, &ssh_user])
.output()
.await?;
if !status.status.success() {
let stderr = String::from_utf8_lossy(&status.stderr);
anyhow::bail!("usermod failed: {}", stderr);
anyhow::bail!("sudo usermod failed: {}", stderr);
}
tracing::info!("SSH password updated for user {}", ssh_user);
+185
View File
@@ -0,0 +1,185 @@
//! Cached Bitcoin node status for browser UIs.
//!
//! The bitcoin-ui should not poll Bitcoin RPC directly for display state.
//! During container restarts, reindexing, and IBD, direct browser RPC polling
//! turns short RPC gaps into visible UI failures. This module owns the RPC
//! polling loop, caches the last successful snapshot, and serves stale-but-known
//! state while the node is reconnecting.
use anyhow::{Context, Result};
use serde::Serialize;
use std::sync::OnceLock;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use tokio::sync::RwLock;
use tracing::{debug, warn};
const CACHE_REFRESH_SECS: u64 = 5;
#[derive(Debug, Clone, Serialize)]
pub struct BitcoinNodeStatus {
pub ok: bool,
pub stale: bool,
pub updated_at_ms: u64,
pub error: Option<String>,
pub blockchain_info: Option<serde_json::Value>,
pub network_info: Option<serde_json::Value>,
pub index_info: Option<serde_json::Value>,
pub zmq_notifications: Option<serde_json::Value>,
}
impl Default for BitcoinNodeStatus {
fn default() -> Self {
Self {
ok: false,
stale: false,
updated_at_ms: 0,
error: Some("Connecting to Bitcoin node...".to_string()),
blockchain_info: None,
network_info: None,
index_info: None,
zmq_notifications: None,
}
}
}
static STATUS_CACHE: OnceLock<RwLock<BitcoinNodeStatus>> = OnceLock::new();
fn cache() -> &'static RwLock<BitcoinNodeStatus> {
STATUS_CACHE.get_or_init(|| RwLock::new(BitcoinNodeStatus::default()))
}
fn now_ms() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_millis() as u64
}
fn transient_error(err_msg: &str) -> bool {
let lower = err_msg.to_lowercase();
lower.contains("connect")
|| lower.contains("reset")
|| lower.contains("refused")
|| lower.contains("timed out")
|| lower.contains("timeout")
|| lower.contains("broken pipe")
|| lower.contains("eof")
|| lower.contains("500 internal server error")
}
pub fn spawn_status_cache() {
tokio::spawn(async {
loop {
let fresh = fetch_bitcoin_status().await;
let mut cached = cache().write().await;
match fresh {
Ok(mut status) => {
status.ok = true;
status.stale = false;
status.error = None;
*cached = status;
}
Err(e) => {
let err_msg = e.to_string();
if transient_error(&err_msg) {
debug!("Bitcoin status: transient RPC failure: {}", err_msg);
} else {
warn!("Bitcoin status: RPC failure: {}", err_msg);
}
if cached.blockchain_info.is_some() {
cached.ok = false;
cached.stale = true;
cached.error = Some(format!(
"Bitcoin node is reconnecting; showing last known state: {}",
err_msg
));
} else {
*cached = BitcoinNodeStatus {
ok: false,
stale: false,
updated_at_ms: now_ms(),
error: Some(format!("Connecting to Bitcoin node: {}", err_msg)),
..BitcoinNodeStatus::default()
};
}
}
}
drop(cached);
tokio::time::sleep(Duration::from_secs(CACHE_REFRESH_SECS)).await;
}
});
}
pub async fn get_bitcoin_status() -> BitcoinNodeStatus {
cache().read().await.clone()
}
async fn fetch_bitcoin_status() -> Result<BitcoinNodeStatus> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(8))
.build()
.context("build Bitcoin status HTTP client")?;
let blockchain_info = bitcoin_rpc_call(&client, "getblockchaininfo", serde_json::json!([]))
.await
.context("getblockchaininfo")?;
let network_info = bitcoin_rpc_call(&client, "getnetworkinfo", serde_json::json!([]))
.await
.context("getnetworkinfo")
.ok();
let index_info = bitcoin_rpc_call(&client, "getindexinfo", serde_json::json!([]))
.await
.context("getindexinfo")
.ok();
let zmq_notifications = bitcoin_rpc_call(&client, "getzmqnotifications", serde_json::json!([]))
.await
.context("getzmqnotifications")
.ok();
Ok(BitcoinNodeStatus {
ok: true,
stale: false,
updated_at_ms: now_ms(),
error: None,
blockchain_info: Some(blockchain_info),
network_info,
index_info,
zmq_notifications,
})
}
async fn bitcoin_rpc_call(
client: &reqwest::Client,
method: &str,
params: serde_json::Value,
) -> Result<serde_json::Value> {
let (rpc_user, rpc_pass) = crate::bitcoin_rpc::bitcoin_rpc_credentials().await;
let body = serde_json::json!({
"jsonrpc": "1.0",
"id": "bitcoin-status",
"method": method,
"params": params,
});
let resp = client
.post(crate::constants::BITCOIN_RPC_URL)
.basic_auth(rpc_user, Some(rpc_pass))
.header("Content-Type", "application/json")
.json(&body)
.send()
.await
.context("Bitcoin RPC request failed")?;
let status = resp.status();
let json: serde_json::Value = resp.json().await.context("decode Bitcoin RPC JSON")?;
if !status.is_success() {
anyhow::bail!("Bitcoin RPC returned {}: {}", status, json);
}
if let Some(error) = json.get("error").filter(|e| !e.is_null()) {
anyhow::bail!("Bitcoin RPC {} error: {}", method, error);
}
json.get("result")
.cloned()
.context("missing Bitcoin RPC result")
}
+413 -55
View File
@@ -8,14 +8,14 @@
//!
//! Two things are synced on startup:
//! 1. Doctor artifacts (container-doctor.sh + service + timer).
//! 2. An nginx `location /api/app-catalog` proxy block required for
//! the App Store catalog proxy to actually reach the backend.
//! 2. Missing nginx backend proxy blocks required for frontend fetches to
//! reach the backend instead of the SPA fallback.
//!
//! Idempotent: no-ops on boxes that are already in sync. All work is
//! best-effort — failures are logged but never abort the backend.
use anyhow::{Context, Result};
use std::path::Path;
use std::path::{Path, PathBuf};
use tokio::fs;
use tracing::{debug, info, warn};
@@ -31,25 +31,352 @@ const DOCTOR_SERVICE_PATH: &str = "/etc/systemd/system/archipelago-doctor.servic
const DOCTOR_TIMER_PATH: &str = "/etc/systemd/system/archipelago-doctor.timer";
const NGINX_CONF_PATH: &str = "/etc/nginx/sites-available/archipelago";
const NGINX_ENABLED_CONF_PATH: &str = "/etc/nginx/sites-enabled/archipelago";
const RUNTIME_ASSETS_DIR: &str = "/opt/archipelago/web-ui/archipelago-runtime";
/// Inserted into every server block of the nginx config that lacks the
/// `/api/app-catalog` proxy. Kept in sync with the canonical block in
/// image-recipe/configs/nginx-archipelago.conf.
const NGINX_APP_CATALOG_BLOCK: &str = "\n # App Store catalog proxy — backend fetches from configured registries\n # so the browser doesn't hit CORS/CSP. Without this block nginx falls\n # through to the SPA index.html and the frontend gets HTML back instead\n # of JSON.\n location /api/app-catalog {\n proxy_pass http://127.0.0.1:5678;\n proxy_http_version 1.1;\n proxy_set_header Host $host;\n proxy_set_header X-Real-IP $remote_addr;\n proxy_set_header Cookie $http_cookie;\n proxy_connect_timeout 15s;\n proxy_read_timeout 30s;\n proxy_send_timeout 15s;\n error_page 502 503 = @backend_unavailable;\n error_page 504 = @backend_timeout;\n }\n\n";
const NGINX_BITCOIN_STATUS_BLOCK: &str = "\n location /bitcoin-status {\n proxy_pass http://127.0.0.1:5678/bitcoin-status;\n proxy_http_version 1.1;\n proxy_set_header Host $host;\n proxy_connect_timeout 10s;\n proxy_read_timeout 10s;\n proxy_send_timeout 5s;\n error_page 502 503 = @backend_unavailable;\n error_page 504 = @backend_timeout;\n }\n";
/// Entry point called from main startup. Never returns an error to the caller —
/// failing to bootstrap host artifacts must not prevent the backend from serving.
pub async fn ensure_doctor_installed() {
match run_service_override_repair().await {
Ok(true) => info!("Removed stale Archipelago dev-mode service override"),
Ok(false) => debug!("No stale Archipelago dev-mode service override found"),
Err(e) => warn!("Service override repair failed (non-fatal): {:#}", e),
}
match run_runtime_assets().await {
Ok(changed) if changed => info!("Runtime assets synchronized from OTA payload"),
Ok(_) => debug!("No OTA runtime payload to synchronize"),
Err(e) => warn!("Runtime asset bootstrap failed (non-fatal): {:#}", e),
}
match run().await {
Ok(changed) if changed => info!("Doctor artifacts synchronized with binary"),
Ok(_) => debug!("Doctor artifacts already in sync"),
Err(e) => warn!("Doctor bootstrap failed (non-fatal): {:#}", e),
}
match run_nginx().await {
Ok(true) => info!("Patched nginx config to proxy /api/app-catalog"),
Ok(false) => debug!("Nginx already has /api/app-catalog block"),
Ok(true) => info!("Patched nginx config to proxy missing backend endpoints"),
Ok(false) => debug!("Nginx backend endpoint proxy blocks already present"),
Err(e) => warn!("Nginx bootstrap failed (non-fatal): {:#}", e),
}
match run_bitcoin_rpc_repair().await {
Ok(true) => {
info!("Repaired Bitcoin RPC bind settings; running Bitcoin containers left untouched")
}
Ok(false) => debug!("Bitcoin RPC bind settings already usable"),
Err(e) => warn!("Bitcoin RPC repair failed (non-fatal): {:#}", e),
}
match tighten_secrets_dir().await {
Ok(n) if n > 0 => info!(tightened = n, "Tightened mode on secret files"),
Ok(_) => debug!("Secrets directory already at expected mode"),
Err(e) => warn!("Secrets dir tightening failed (non-fatal): {:#}", e),
}
// Podman probing MUST be the last bootstrap stage. We used to delete
// transient runroot state here when `podman info` failed, but live nodes
// can still have rootlessport/conmon processes holding that state. Removing
// it automatically makes failures worse: containers lose `.containerenv`,
// ports stay bound, and later starts fail. Report the fault instead; repair
// must be deliberate/operator-driven.
match heal_podman_state().await {
Ok(PodmanHealOutcome::Healthy) => debug!("podman runtime state healthy"),
Ok(PodmanHealOutcome::Unhealthy) => warn!(
"podman runtime state is unhealthy at startup — skipping automatic runroot cleanup"
),
Err(e) => warn!(
"podman self-heal failed (non-fatal, will retry next boot): {:#}",
e
),
}
}
#[derive(Debug, PartialEq, Eq)]
enum PodmanHealOutcome {
Healthy,
Unhealthy,
}
async fn heal_podman_state() -> Result<PodmanHealOutcome> {
if probe_podman_ok().await {
return Ok(PodmanHealOutcome::Healthy);
}
Ok(PodmanHealOutcome::Unhealthy)
}
/// True iff `podman info` returns 0 within 5s. Any timeout, spawn
/// failure, or non-zero exit reads as "wedged" and triggers cleanup.
async fn probe_podman_ok() -> bool {
use std::time::Duration;
let probe = tokio::time::timeout(
Duration::from_secs(5),
tokio::process::Command::new("podman")
.arg("info")
.arg("--format=json")
.output(),
)
.await;
match probe {
Ok(Ok(out)) => out.status.success(),
Ok(Err(_)) | Err(_) => false,
}
}
/// Make sure /var/lib/archipelago/secrets/ stays 0700 owned by archipelago,
/// and every file inside is 0600. The parent dir mode is the load-bearing
/// boundary against host-side reads from other UIDs (rootless container
/// escapes get mapped to UID >= 100000 and can't traverse a 0700/uid=1000
/// directory). The per-file 0600 sweep is defense-in-depth in case some
/// installer wrote a 0644 file.
async fn tighten_secrets_dir() -> Result<u32> {
let dir = Path::new("/var/lib/archipelago/secrets");
if !dir.exists() {
return Ok(0);
}
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(dir, std::fs::Permissions::from_mode(0o700))
.await
.with_context(|| format!("chmod 0700 {}", dir.display()))?;
let mut entries = fs::read_dir(dir)
.await
.with_context(|| format!("read_dir {}", dir.display()))?;
let mut tightened = 0u32;
while let Some(entry) = entries.next_entry().await? {
let path = entry.path();
let meta = match entry.metadata().await {
Ok(m) => m,
Err(_) => continue,
};
if !meta.is_file() {
continue;
}
if meta.permissions().mode() & 0o777 != 0o600 {
fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))
.await
.with_context(|| format!("chmod 0600 {}", path.display()))?;
tightened += 1;
}
}
Ok(tightened)
}
async fn run_service_override_repair() -> Result<bool> {
let override_path = Path::new("/etc/systemd/system/archipelago.service.d/override.conf");
let Ok(content) = fs::read_to_string(override_path).await else {
return Ok(false);
};
if !content.contains("ARCHIPELAGO_DEV_MODE=true") {
return Ok(false);
}
let only_dev_mode_override = content
.lines()
.map(str::trim)
.filter(|line| !line.is_empty() && !line.starts_with('#'))
.all(|line| line == "[Service]" || line == "Environment=ARCHIPELAGO_DEV_MODE=true");
if !only_dev_mode_override {
warn!(
path = %override_path.display(),
"Archipelago service override contains ARCHIPELAGO_DEV_MODE=true plus other settings; leaving it untouched"
);
return Ok(false);
}
let path_s = override_path.to_string_lossy().to_string();
let status = host_sudo(&["rm", "-f", &path_s])
.await
.with_context(|| format!("remove {}", override_path.display()))?;
if !status.success() {
anyhow::bail!("remove {} exited with {}", override_path.display(), status);
}
let _ = host_sudo(&["systemctl", "daemon-reload"]).await;
Ok(true)
}
async fn run_runtime_assets() -> Result<bool> {
// The v1.7.50 OTA bridge puts scripts/apps/docker assets inside the
// frontend tarball because older binaries only know how to apply the
// backend binary and frontend archive. Once the new backend starts, it
// promotes that payload into /opt so app installs use the matching specs.
let runtime_dir = Path::new(RUNTIME_ASSETS_DIR);
if !runtime_dir.exists() {
return Ok(false);
}
let mut changed = false;
for (relative, dest) in [
("apps", "/opt/archipelago/apps"),
("scripts", "/opt/archipelago/scripts"),
("docker", "/opt/archipelago/docker"),
] {
let src = runtime_dir.join(relative);
if src.exists() {
replace_dir_from_runtime(&src, dest).await?;
if relative == "scripts" {
let _ = host_sudo(&[
"find", dest, "-type", "f", "-name", "*.sh", "-exec", "chmod", "755", "{}", "+",
])
.await;
let image_versions = format!("{}/image-versions.sh", dest);
if Path::new(&image_versions).exists() {
let _ =
host_sudo(&["cp", &image_versions, "/opt/archipelago/image-versions.sh"])
.await;
}
}
changed = true;
}
}
let configs = runtime_dir.join("image-recipe/configs");
let nginx_src = configs.join("nginx-archipelago.conf");
if nginx_src.exists() {
let src_s = nginx_src.to_string_lossy().to_string();
let status = host_sudo(&[
"install",
"-m",
"644",
&src_s,
"/etc/nginx/sites-available/archipelago",
])
.await
.context("install nginx-archipelago.conf")?;
if !status.success() {
anyhow::bail!("install nginx-archipelago.conf exited with {}", status);
}
changed = true;
}
for unit in ["archipelago-doctor.service", "archipelago-doctor.timer"] {
let src = configs.join(unit);
if src.exists() {
let src_s = src.to_string_lossy().to_string();
let dest = format!("/etc/systemd/system/{}", unit);
let status = host_sudo(&["install", "-m", "644", &src_s, &dest])
.await
.with_context(|| format!("install {}", unit))?;
if !status.success() {
anyhow::bail!("install {} exited with {}", unit, status);
}
changed = true;
}
}
if changed {
let _ = host_sudo(&["systemctl", "daemon-reload"]).await;
if nginx_src.exists() {
match host_sudo(&["nginx", "-t"]).await {
Ok(status) if status.success() => {
let _ = host_sudo(&["systemctl", "reload", "nginx"]).await;
}
Ok(status) => {
tracing::warn!("nginx config test failed after runtime sync: {}", status);
}
Err(e) => {
tracing::warn!("failed to test nginx config after runtime sync: {}", e);
}
}
}
}
Ok(changed)
}
async fn replace_dir_from_runtime(src: &Path, dest: &str) -> Result<()> {
let tmp = format!("{}.new.{}", dest, chrono::Utc::now().timestamp_millis());
let src_dot = path_dot(src);
let mkdir = host_sudo(&["mkdir", "-p", &tmp])
.await
.with_context(|| format!("mkdir {}", tmp))?;
if !mkdir.success() {
anyhow::bail!("mkdir {} exited with {}", tmp, mkdir);
}
let copy = host_sudo(&["cp", "-a", &src_dot, &tmp])
.await
.with_context(|| format!("copy runtime {} -> {}", src.display(), tmp))?;
if !copy.success() {
let _ = host_sudo(&["rm", "-rf", &tmp]).await;
anyhow::bail!("copy runtime {} exited with {}", src.display(), copy);
}
let _ = host_sudo(&["mkdir", "-p", dest]).await;
let cleanup = host_sudo(&[
"find",
dest,
"-mindepth",
"1",
"-maxdepth",
"1",
"-exec",
"rm",
"-rf",
"{}",
"+",
])
.await
.with_context(|| format!("clean {}", dest))?;
if !cleanup.success() {
let _ = host_sudo(&["rm", "-rf", &tmp]).await;
anyhow::bail!("clean {} exited with {}", dest, cleanup);
}
let tmp_dot = format!("{}/.", tmp);
let promote = host_sudo(&["cp", "-a", &tmp_dot, dest])
.await
.with_context(|| format!("promote {} -> {}", tmp, dest))?;
let _ = host_sudo(&["rm", "-rf", &tmp]).await;
if !promote.success() {
anyhow::bail!("promote {} exited with {}", dest, promote);
}
Ok(())
}
fn path_dot(path: &Path) -> String {
let mut p = PathBuf::from(path);
p.push(".");
p.to_string_lossy().to_string()
}
async fn run_bitcoin_rpc_repair() -> Result<bool> {
// Older installs can have a container-owned bitcoin.conf with only rpcauth
// and printtoconsole. Repair it at startup so OTA fixes existing nodes
// without a manual uninstall/reinstall. Bind/port stay in the container
// command line to avoid duplicate RPC endpoint definitions.
let script = r#"
set -eu
conf=/var/lib/archipelago/bitcoin/bitcoin.conf
[ -f "$conf" ] || exit 0
changed=0
ensure_line() {
line="$1"
key="${line%%=*}"
if ! grep -q "^${key}=" "$conf"; then
printf '%s\n' "$line" >> "$conf"
changed=1
fi
}
ensure_line server=1
ensure_line rpcallowip=0.0.0.0/0
ensure_line listen=1
[ "$changed" -eq 0 ] && exit 0
exit 2
"#;
let status = host_sudo(&["sh", "-lc", script])
.await
.context("repair bitcoin.conf RPC bind settings")?;
match status.code() {
Some(0) => Ok(false),
// Do not restart Bitcoin from bootstrap. During IBD, an automatic
// restart can cost hours of progress. The repaired file is only a
// fallback for future starts; current containers keep their command-line
// RPC args until an operator or update intentionally restarts them.
Some(2) => Ok(true),
_ => {
warn!("Bitcoin RPC repair helper exited with {}", status);
Ok(false)
}
}
}
async fn run() -> Result<bool> {
@@ -101,21 +428,14 @@ async fn run() -> Result<bool> {
let timer_changed = write_root_if_needed(DOCTOR_TIMER_PATH, DOCTOR_TIMER).await?;
changed = changed || service_changed || timer_changed;
// 3. Reload + enable. Only when we actually touched units, or when the
// timer isn't enabled yet (catches fresh upgrades of boxes that predate
// the doctor entirely).
let timer_enabled = is_timer_enabled().await;
if service_changed || timer_changed || !timer_enabled {
// 3. Reload if units changed. Do not enable/start the timer here: lifecycle
// qualification and explicit app operations need deterministic Podman
// ownership, and the doctor can race those flows. Operators can enable it
// separately when they want periodic host repair.
if service_changed || timer_changed {
if let Err(e) = host_sudo(&["systemctl", "daemon-reload"]).await {
warn!("daemon-reload failed: {:#}", e);
}
if let Err(e) =
host_sudo(&["systemctl", "enable", "--now", "archipelago-doctor.timer"]).await
{
warn!("enable archipelago-doctor.timer failed: {:#}", e);
} else if !timer_enabled {
info!("Enabled archipelago-doctor.timer");
}
}
Ok(changed)
@@ -156,22 +476,10 @@ async fn write_root_if_needed(path: &str, content: &str) -> Result<bool> {
Ok(true)
}
async fn is_timer_enabled() -> bool {
tokio::process::Command::new("systemctl")
.args(["is-enabled", "--quiet", "archipelago-doctor.timer"])
.status()
.await
.map(|s| s.success())
.unwrap_or(false)
}
/// Patch the nginx site config to add a `/api/app-catalog` proxy block if
/// it's missing. The original ISO shipped individual per-endpoint `location`
/// blocks and no catch-all `/api/`, so `/api/app-catalog` silently fell
/// through to the SPA `index.html` and the frontend got HTML instead of
/// JSON. We anchor the insert to the DWN comment that already sits right
/// after the `/api/blob` block, so the new block lands in both the HTTP
/// and HTTPS server blocks.
/// Patch the nginx site config to add missing backend proxy blocks. Older ISO
/// configs shipped individual per-endpoint `location` blocks, so missing
/// endpoints silently fell through to the SPA `index.html` and the frontend
/// got HTML instead of JSON.
///
/// Validates via `nginx -t` before reloading. On failure the patch is
/// rolled back from a backup written just before the write.
@@ -186,51 +494,90 @@ async fn run_nginx() -> Result<bool> {
return Ok(false);
}
if !Path::new(NGINX_CONF_PATH).exists() {
debug!("{} missing — skipping nginx bootstrap", NGINX_CONF_PATH);
return Ok(false);
let mut changed = false;
let mut patched_paths = Vec::<PathBuf>::new();
for path in [NGINX_CONF_PATH, NGINX_ENABLED_CONF_PATH] {
let candidate = Path::new(path);
if !candidate.exists() {
debug!("{} missing — skipping nginx bootstrap", path);
continue;
}
let canonical = fs::canonicalize(candidate)
.await
.unwrap_or_else(|_| candidate.to_path_buf());
if patched_paths.iter().any(|p| p == &canonical) {
continue;
}
patched_paths.push(canonical);
changed |= patch_nginx_conf(path).await?;
}
Ok(changed)
}
let content = fs::read_to_string(NGINX_CONF_PATH)
async fn patch_nginx_conf(path: &str) -> Result<bool> {
let content = fs::read_to_string(path)
.await
.with_context(|| format!("read {}", NGINX_CONF_PATH))?;
if content.contains("location /api/app-catalog") {
.with_context(|| format!("read {}", path))?;
let missing_app_catalog = !content.contains("location /api/app-catalog");
let missing_bitcoin_status = !content.contains("location /bitcoin-status");
if !missing_app_catalog && !missing_bitcoin_status {
return Ok(false);
}
// The DWN comment sits at the same indent right after the `/api/blob`
// block in both server blocks — a stable anchor that existed on every
// ISO shipped to date. If it's absent (config got heavily customized),
// we bail rather than guess where to splice.
let anchor = " # DWN endpoints — peer access over Tor (no auth)";
if !content.contains(anchor) {
warn!("nginx conf missing DWN anchor — skipping /api/app-catalog patch");
return Ok(false);
let mut patched = content.clone();
if missing_bitcoin_status {
let anchor = " location /electrs-status {";
if !patched.contains(anchor) {
warn!("nginx conf missing electrs-status anchor — skipping /bitcoin-status patch");
} else {
let replacement = format!("{}{}", NGINX_BITCOIN_STATUS_BLOCK, anchor);
patched = patched.replace(anchor, &replacement);
}
}
let replacement = format!("{}{}", NGINX_APP_CATALOG_BLOCK, anchor);
let patched = content.replace(anchor, &replacement);
if missing_app_catalog {
// The DWN comment sits at the same indent right after the `/api/blob`
// block in both server blocks — a stable anchor that existed on every
// ISO shipped to date. If it's absent (config got heavily customized),
// skip rather than guess where to splice.
let anchor = " # DWN endpoints — peer access over Tor (no auth)";
if !patched.contains(anchor) {
warn!("nginx conf missing DWN anchor — skipping /api/app-catalog patch");
} else {
let replacement = format!("{}{}", NGINX_APP_CATALOG_BLOCK, anchor);
patched = patched.replace(anchor, &replacement);
}
}
if patched == content {
return Ok(false);
}
// Write patched config via a user-owned tmp + sudo mv, after stashing
// a backup so we can revert if `nginx -t` hates what we produced.
// a backup outside nginx include dirs so validation cannot load it too.
let pid = std::process::id();
let tmp = format!("/tmp/archipelago-nginx-{}.conf", pid);
fs::write(&tmp, &patched)
.await
.with_context(|| format!("write {}", tmp))?;
let backup = format!("/tmp/archipelago-nginx-backup-{}.conf", pid);
if let Err(e) = host_sudo(&["cp", NGINX_CONF_PATH, &backup]).await {
let backup = format!(
"/tmp/archipelago-nginx-backup-{}-{}.conf",
pid,
patched.len()
);
if let Err(e) = host_sudo(&["cp", path, &backup]).await {
let _ = fs::remove_file(&tmp).await;
return Err(e.context("backup nginx conf"));
}
let mv = host_sudo(&["mv", &tmp, NGINX_CONF_PATH]).await;
let mv = host_sudo(&["mv", &tmp, path]).await;
match mv {
Ok(s) if s.success() => {}
Ok(s) => {
let _ = fs::remove_file(&tmp).await;
anyhow::bail!("sudo mv nginx conf exited with {}", s);
anyhow::bail!("sudo mv nginx conf to {} exited with {}", path, s);
}
Err(e) => {
let _ = fs::remove_file(&tmp).await;
@@ -243,7 +590,7 @@ async fn run_nginx() -> Result<bool> {
let valid = matches!(&test, Ok(s) if s.success());
if !valid {
warn!("nginx -t failed after patch — reverting");
let _ = host_sudo(&["mv", &backup, NGINX_CONF_PATH]).await;
let _ = host_sudo(&["mv", &backup, path]).await;
if let Err(e) = test {
return Err(e.context("nginx -t"));
}
@@ -258,3 +605,14 @@ async fn run_nginx() -> Result<bool> {
let _ = host_sudo(&["rm", "-f", &backup]).await;
Ok(true)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn podman_heal_outcome_no_longer_has_cleanup_variant() {
let outcome = PodmanHealOutcome::Unhealthy;
assert_ne!(outcome, PodmanHealOutcome::Healthy);
}
}
+58 -3
View File
@@ -62,6 +62,14 @@ pub struct Config {
/// Tor SOCKS5 proxy (e.g. 127.0.0.1:9050). When set, ALL Nostr traffic routes through Tor.
#[serde(default)]
pub nostr_tor_proxy: Option<String>,
/// Phase 3.2 of v1.7.52: route orchestrator-managed backend installs
/// through Quadlet (`.container` units in ~/.config/containers/systemd
/// + systemctl --user start) instead of `podman create + start`. Default
/// off so the legacy path stays the production path until the harness
/// at tests/lifecycle/run-20x.sh has gone green against the new path
/// on .228 + .198. See `project_v1_7_52_phase3_quadlet_design`.
#[serde(default)]
pub use_quadlet_backends: bool,
}
impl Config {
@@ -132,9 +140,12 @@ impl Config {
config.log_level = level;
}
// Dev mode configuration
if let Ok(dev_mode) = std::env::var("ARCHIPELAGO_DEV_MODE") {
config.dev_mode = dev_mode.parse().unwrap_or(false);
// Production binaries must not be switched into dev orchestration by
// host environment. Several live nodes carried a stale systemd
// ARCHIPELAGO_DEV_MODE override, which rewrote production volume
// mounts into /tmp and prevented real installs from starting.
if std::env::var("ARCHIPELAGO_DEV_MODE").is_ok() {
tracing::warn!("Ignoring ARCHIPELAGO_DEV_MODE in production config");
}
if let Ok(runtime) = std::env::var("ARCHIPELAGO_CONTAINER_RUNTIME") {
@@ -171,6 +182,17 @@ impl Config {
config.nostr_tor_proxy = if s.is_empty() { None } else { Some(s) };
}
// Phase 3.2 of v1.7.52. Truthy values (1, true, yes, on — case-insensitive)
// route backend installs through the Quadlet path without requiring a
// config.json edit + archipelago.service restart (which would trigger
// FM3 cgroup cascade until 3.5 ships). Anything else (or unset) leaves
// the config.json value untouched.
if let Ok(v) = std::env::var("ARCHIPELAGO_USE_QUADLET_BACKENDS") {
if parse_truthy_env(&v) {
config.use_quadlet_backends = true;
}
}
// Host IP for container env vars (detect if not set)
if let Ok(ip) = std::env::var("ARCHIPELAGO_HOST_IP") {
config.host_ip = ip;
@@ -218,10 +240,23 @@ impl Default for Config {
"wss://relay.nostr.info".into(),
],
nostr_tor_proxy: Some("127.0.0.1:9050".into()),
use_quadlet_backends: false,
}
}
}
/// Recognise the canonical "the user meant true" forms for boolean env
/// vars: 1, true, yes, on (case-insensitive, surrounding whitespace
/// trimmed). Anything else — including the typo'd "ture" or the empty
/// string — counts as false. Centralised so future env flags stay
/// consistent with each other.
fn parse_truthy_env(raw: &str) -> bool {
matches!(
raw.trim().to_ascii_lowercase().as_str(),
"1" | "true" | "yes" | "on"
)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -425,4 +460,24 @@ mod tests {
assert!(!config.nostr_relays.is_empty());
assert!(config.nostr_relays.iter().all(|r| r.starts_with("wss://")));
}
#[test]
fn parse_truthy_env_recognises_canonical_forms() {
for t in ["1", "true", "TRUE", "yes", "Yes", "on", "ON", " true "] {
assert!(parse_truthy_env(t), "{t:?} should parse truthy");
}
for f in ["", "0", "false", "no", "off", "ture", "anything else", " "] {
assert!(!parse_truthy_env(f), "{f:?} should NOT parse truthy");
}
}
#[test]
fn test_config_use_quadlet_backends_defaults_off() {
// Phase 3.2 of v1.7.52 — the new path stays gated until the 20×
// harness goes green on .228 and .198. Flipping this default
// ahead of that would route every backend install through code
// we haven't fleet-validated yet.
let config = Config::default();
assert!(!config.use_quadlet_backends);
}
}
@@ -15,5 +15,13 @@ server {
add_header Access-Control-Allow-Headers "Content-Type, Authorization";
if ($request_method = OPTIONS) { return 204; }
}
location /bitcoin-status {
proxy_pass http://127.0.0.1:5678/bitcoin-status;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
add_header Cache-Control "no-store";
}
location / { try_files $uri $uri/ /index.html; }
}
@@ -3,8 +3,8 @@
//!
//! Step 5 of the rust-orchestrator migration. Spawned once from `main.rs`
//! (Step 6) after the initial `adopt_existing()` pass. Every `interval` it
//! calls `ProdContainerOrchestrator::reconcile_all()`, which ensures every
//! loaded manifest has a running container, installing fresh ones as needed.
//! calls `ProdContainerOrchestrator::reconcile_existing()`, which repairs
//! containers that already exist without installing every catalog manifest.
//!
//! Per answered design Q3, `interval` defaults to 30 seconds.
//!
@@ -29,6 +29,11 @@ pub struct BootReconciler {
orchestrator: Arc<ProdContainerOrchestrator>,
interval: Duration,
shutdown: Arc<Notify>,
/// Run the companion-unit repair stage each tick. Default true.
/// Tests disable this — companion reconcile shells out to
/// `systemctl --user` and `podman`, which both block real time
/// and would race the paused-clock test fixtures.
companion_stage: bool,
}
impl BootReconciler {
@@ -41,28 +46,46 @@ impl BootReconciler {
orchestrator,
interval,
shutdown,
companion_stage: true,
}
}
/// Disable the companion-unit reconcile stage. Used by unit tests
/// that exercise loop cadence without the real systemd / podman
/// surface. Production must not call this.
#[cfg(test)]
pub fn without_companion_stage(mut self) -> Self {
self.companion_stage = false;
self
}
/// Run the reconcile loop until `shutdown` is notified.
///
/// Does one reconcile immediately, then sleeps `interval` between
/// subsequent passes. A `shutdown.notify_one()` call unblocks the sleep
/// and the task returns after the *next* pass completes.
///
/// Never panics: per-app failures are absorbed into `ReconcileReport` by
/// the orchestrator, and `reconcile_all` itself returns infallibly.
/// Each pass is two stages:
/// 1. App reconcile: `reconcile_all()` keeps every loaded manifest's
/// container running.
/// 2. Companion reconcile: any expected Quadlet companion unit that
/// is missing or inactive is repaired (writes the unit, daemon-
/// reloads, starts the service). This is the safety net for the
/// "someone deleted my unit file" / "systemd lost the service"
/// failure modes.
///
/// Never panics: per-app failures are absorbed into `ReconcileReport`
/// by the orchestrator, and companion failures are logged but never
/// propagated.
pub async fn run_forever(self) {
// Initial pass: no delay.
let report = self.orchestrator.reconcile_all().await;
Self::log_report(&report);
self.tick().await;
loop {
let deadline = Instant::now() + self.interval;
tokio::select! {
_ = time::sleep_until(deadline) => {
let report = self.orchestrator.reconcile_all().await;
Self::log_report(&report);
self.tick().await;
}
_ = self.shutdown.notified() => {
tracing::info!("boot reconciler: shutdown requested, exiting loop");
@@ -72,6 +95,23 @@ impl BootReconciler {
}
}
async fn tick(&self) {
let report = self.orchestrator.reconcile_existing().await;
Self::log_report(&report);
if !self.companion_stage {
return;
}
let installed = self.orchestrator.manifest_ids().await;
for (companion, err) in crate::container::companion::reconcile(&installed).await {
tracing::warn!(
companion = %companion,
error = %err,
"companion reconcile failed"
);
}
}
fn log_report(report: &ReconcileReport) {
for (app_id, action) in &report.actions {
tracing::debug!(app_id = %app_id, action = ?action, "reconcile action");
@@ -200,10 +240,11 @@ mod tests {
async fn orch_with_one_running_manifest(
rt: Arc<CountingRuntime>,
) -> Arc<ProdContainerOrchestrator> {
let orch = Arc::new(ProdContainerOrchestrator::with_runtime(
rt,
PathBuf::from("/nonexistent-for-tests"),
));
let mut orch =
ProdContainerOrchestrator::with_runtime(rt, PathBuf::from("/nonexistent-for-tests"));
let tmp = tempfile::tempdir().unwrap().keep();
orch.set_data_dir(tmp);
let orch = Arc::new(orch);
orch.insert_manifest_for_test(
pull_manifest("bitcoin-knots", "docker.io/bitcoin/knots:28"),
PathBuf::from("/tmp/bk"),
@@ -218,7 +259,8 @@ mod tests {
let orch = orch_with_one_running_manifest(rt.clone()).await;
let shutdown = Arc::new(Notify::new());
let reconciler =
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone())
.without_companion_stage();
let handle = tokio::spawn(reconciler.run_forever());
// Yield so the spawned task gets CPU to run its initial reconcile.
@@ -242,7 +284,8 @@ mod tests {
let orch = orch_with_one_running_manifest(rt.clone()).await;
let shutdown = Arc::new(Notify::new());
let reconciler =
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone())
.without_companion_stage();
let handle = tokio::spawn(reconciler.run_forever());
tokio::task::yield_now().await;
@@ -271,7 +314,8 @@ mod tests {
let orch = orch_with_one_running_manifest(rt.clone()).await;
let shutdown = Arc::new(Notify::new());
let reconciler =
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone())
.without_companion_stage();
let handle = tokio::spawn(reconciler.run_forever());
tokio::task::yield_now().await;
tokio::task::yield_now().await;
@@ -294,10 +338,13 @@ mod tests {
// will run, and the next pass will see a new state. We care about
// "loop keeps ticking even when the report has actions".
let rt = Arc::new(CountingRuntime::default());
let orch = Arc::new(ProdContainerOrchestrator::with_runtime(
let mut orch = ProdContainerOrchestrator::with_runtime(
rt.clone(),
PathBuf::from("/nonexistent-for-tests"),
));
);
let tmp = tempfile::tempdir().unwrap().keep();
orch.set_data_dir(tmp);
let orch = Arc::new(orch);
orch.insert_manifest_for_test(
pull_manifest("bitcoin-knots", "docker.io/bitcoin/knots:28"),
PathBuf::from("/tmp/bk"),
@@ -305,7 +352,8 @@ mod tests {
.await;
let shutdown = Arc::new(Notify::new());
let reconciler =
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone())
.without_companion_stage();
let handle = tokio::spawn(reconciler.run_forever());
tokio::task::yield_now().await;
+401
View File
@@ -0,0 +1,401 @@
//! Companion UI container lifecycle, entirely Quadlet-managed.
//!
//! A "companion" is a small nginx-based container that exposes a
//! browser-friendly UI on top of a headless backend service:
//!
//! | Backend | Companion | Purpose |
//! |------------------|--------------------|--------------------------|
//! | bitcoin-knots | archy-bitcoin-ui | RPC viewer |
//! | bitcoin-core | archy-bitcoin-ui | RPC viewer |
//! | lnd | archy-lnd-ui | wallet/channel UI |
//! | electrumx | archy-electrs-ui | indexer status UI |
//!
//! Lifecycle: `install` writes a Quadlet `.container` unit to
//! `~/.config/containers/systemd/`, daemon-reloads, then starts the
//! generated `.service`. systemd owns supervision from that point on
//! — archipelago can crash, restart, or be uninstalled without
//! touching the companion.
//!
//! This replaces the old `tokio::spawn { podman run }` block in
//! `install.rs` (~165 lines of fire-and-forget shellouts) with a
//! single declarative call.
use anyhow::{Context, Result};
use std::path::PathBuf;
use tokio::fs;
use tokio::process::Command;
use tracing::{info, warn};
use crate::container::quadlet::{self, BindMount, NetworkMode, QuadletUnit};
use archipelago_container::image_uses_insecure_registry;
const COMPANION_REGISTRY: &str = "146.59.87.168:3000/lfg2025";
/// Static description of one companion. The full list per backend
/// app_id lives in `companions_for`.
#[derive(Debug, Clone)]
pub struct CompanionSpec {
/// Container + unit name (e.g. "archy-bitcoin-ui").
pub name: &'static str,
/// Image base name in the lfg2025 registry namespace
/// (e.g. "bitcoin-ui" → "146.59.87.168:3000/lfg2025/bitcoin-ui:latest").
pub image_base: &'static str,
/// Filesystem locations to look for a local Dockerfile (build wins
/// over registry pull). Searched in order; first hit wins.
pub build_dir_candidates: &'static [&'static str],
/// Optional pre-start hook that renders config files referenced
/// by `bind_mounts`. Returns Ok(()) on success; bind-mount must
/// be present at start time or the companion will 502.
pub pre_start: Option<PreStartHook>,
/// Bind mounts. Always read-only — companions don't write to
/// host paths.
pub bind_mounts: &'static [(&'static str, &'static str)],
/// Host-to-container TCP ports for non-host-network companions.
pub ports: &'static [(u16, u16)],
/// Whether the companion must share the host network namespace.
pub host_network: bool,
}
pub type PreStartHook = fn() -> futures_util::future::BoxFuture<'static, Result<()>>;
/// Companions to install when `package_id` lands. Empty for apps
/// without a companion UI.
pub fn companions_for(package_id: &str) -> &'static [CompanionSpec] {
match package_id {
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => BITCOIN_UI,
"lnd" => LND_UI,
"electrumx" | "electrs" | "mempool-electrs" => ELECTRS_UI,
_ => &[],
}
}
const BITCOIN_UI: &[CompanionSpec] = &[CompanionSpec {
name: "archy-bitcoin-ui",
image_base: "bitcoin-ui",
build_dir_candidates: &[
"/opt/archipelago/docker/bitcoin-ui",
"/home/archipelago/archy/docker/bitcoin-ui",
"/home/archipelago/Projects/archy/docker/bitcoin-ui",
],
pre_start: Some(render_bitcoin_ui),
bind_mounts: &[(
"/var/lib/archipelago/bitcoin-ui/nginx.conf",
"/etc/nginx/conf.d/default.conf",
)],
ports: &[],
host_network: true,
}];
const LND_UI: &[CompanionSpec] = &[CompanionSpec {
name: "archy-lnd-ui",
image_base: "lnd-ui",
build_dir_candidates: &[
"/opt/archipelago/docker/lnd-ui",
"/home/archipelago/archy/docker/lnd-ui",
"/home/archipelago/Projects/archy/docker/lnd-ui",
],
pre_start: None,
bind_mounts: &[],
ports: &[(18083, 80)],
host_network: false,
}];
const ELECTRS_UI: &[CompanionSpec] = &[CompanionSpec {
name: "archy-electrs-ui",
image_base: "electrs-ui",
build_dir_candidates: &[
"/opt/archipelago/docker/electrs-ui",
"/home/archipelago/archy/docker/electrs-ui",
"/home/archipelago/Projects/archy/docker/electrs-ui",
],
pre_start: None,
bind_mounts: &[],
ports: &[],
host_network: true,
}];
fn render_bitcoin_ui() -> futures_util::future::BoxFuture<'static, Result<()>> {
Box::pin(async {
let paths = crate::container::bitcoin_ui::RenderPaths::default();
crate::container::bitcoin_ui::render(&paths)
.await
.map(|_| ())
.context("render bitcoin-ui nginx.conf")
})
}
/// Provision and start every companion for `package_id`. Each
/// companion is independent — a failure in one is logged but does
/// not abort the others.
pub async fn install_for(package_id: &str) -> Vec<(String, anyhow::Error)> {
let mut failures = Vec::new();
for spec in companions_for(package_id) {
if let Err(e) = install_one(spec).await {
warn!(companion = spec.name, error = %e, "companion install failed");
failures.push((spec.name.to_string(), e));
}
}
failures
}
/// Stop and remove every companion for `package_id`. Best effort:
/// errors are logged but do not abort the sequence.
pub async fn remove_for(package_id: &str) {
let dir = match quadlet::unit_dir().await {
Ok(d) => d,
Err(e) => {
warn!("companion remove: cannot resolve quadlet dir: {e:#}");
return;
}
};
for spec in companions_for(package_id) {
if let Err(e) = quadlet::disable_remove(spec.name, &dir).await {
warn!(companion = spec.name, error = %e, "companion remove failed");
}
}
}
/// Provision one companion: pre-start hook → image present → write
/// quadlet → daemon-reload → start.
pub async fn install_one(spec: &CompanionSpec) -> Result<()> {
if let Some(hook) = spec.pre_start {
hook().await.with_context(|| {
format!(
"pre-start hook failed for {} — companion will not start",
spec.name
)
})?;
}
let image = ensure_image_present(spec).await?;
let unit = build_unit(spec, &image);
let dir = quadlet::unit_dir().await?;
let changed = quadlet::write_if_changed(&unit, &dir).await?;
if changed {
info!(companion = spec.name, "wrote quadlet unit");
quadlet::daemon_reload_user().await?;
}
// Start is idempotent — if already running, systemctl returns 0.
quadlet::enable_now(&unit.service_name()).await?;
info!(companion = spec.name, "companion started");
Ok(())
}
/// Build companion image locally if a Dockerfile exists, otherwise
/// pull from the lfg2025 registry. Returns the image ref the quadlet
/// should reference (`localhost/<base>:latest` for build, registry
/// URL for pull).
async fn ensure_image_present(spec: &CompanionSpec) -> Result<String> {
let local_image = format!("localhost/{}:latest", spec.image_base);
let local_image_compat = format!("localhost/{}:local", spec.image_base);
let registry_image = format!("{}/{}:latest", COMPANION_REGISTRY, spec.image_base);
// Prefer local build — companions can carry build-time customizations
// (e.g. nginx.conf templates baked in). Search known candidates.
for dir in spec.build_dir_candidates {
let dockerfile = PathBuf::from(dir).join("Dockerfile");
if fs::try_exists(&dockerfile).await.unwrap_or(false) {
if image_exists(&local_image_compat).await {
return Ok(local_image_compat);
}
if image_exists(&local_image).await {
return Ok(local_image);
}
info!(companion = spec.name, "building locally from {dir}");
let out = Command::new("podman")
.args(["build", "-t", &local_image, dir])
.output()
.await
.context("spawn podman build")?;
if out.status.success() {
return Ok(local_image);
}
warn!(
companion = spec.name,
"local build failed: {}",
String::from_utf8_lossy(&out.stderr).trim()
);
// Fall through to registry pull rather than fail outright.
break;
}
}
// Registry pull. Use insecure flag only for whitelisted hosts.
let mut cmd = Command::new("podman");
cmd.arg("pull");
if image_uses_insecure_registry(&registry_image) {
cmd.arg("--tls-verify=false");
}
cmd.arg(&registry_image);
let out = cmd.output().await.context("spawn podman pull")?;
if !out.status.success() {
anyhow::bail!(
"no local Dockerfile and registry pull failed for {}: {}",
spec.name,
String::from_utf8_lossy(&out.stderr).trim()
);
}
Ok(registry_image)
}
async fn image_exists(image: &str) -> bool {
Command::new("podman")
.args(["image", "exists", image])
.status()
.await
.is_ok_and(|status| status.success())
}
fn build_unit(spec: &CompanionSpec, image: &str) -> QuadletUnit {
QuadletUnit {
name: spec.name.into(),
description: format!("Archipelago companion UI: {}", spec.name),
image: image.into(),
network: if spec.host_network {
NetworkMode::Host
} else {
NetworkMode::Bridge("bridge".into())
},
// Run as root inside the container so nginx can chown its
// worker dirs. Rootless podman maps this to a high host UID,
// so it is unprivileged on the host.
user: Some("0:0".into()),
memory_mb: Some(128),
cap_drop_all: true,
cap_add: vec![
"CHOWN".into(),
"DAC_OVERRIDE".into(),
"NET_BIND_SERVICE".into(),
"SETUID".into(),
"SETGID".into(),
],
bind_mounts: spec
.bind_mounts
.iter()
.map(|(host, container)| BindMount {
host: PathBuf::from(*host),
container: PathBuf::from(*container),
read_only: true,
})
.collect(),
ports: spec
.ports
.iter()
.map(|(host, container)| (*host, *container, "tcp".into()))
.collect(),
extra_podman_args: vec![],
depends_on: vec![],
// Companions don't use the backend-manifest extension fields;
// the renderer skips empty/false directives so the rendered
// bytes are unchanged from before quadlet.rs grew the new fields.
..QuadletUnit::default()
}
}
/// Is a user systemd manager reachable? In production archipelago.service
/// inherits XDG_RUNTIME_DIR from systemd; in unit tests / CI sandboxes it
/// is unset, in which case `systemctl --user` would fail and write to
/// HOME would be an unwanted side effect. The reconciler skips its
/// companion stage when this is false.
fn user_systemd_available() -> bool {
std::env::var_os("XDG_RUNTIME_DIR")
.map(|v| !v.is_empty())
.unwrap_or(false)
}
/// Reconcile companion presence: every expected companion for the
/// given installed apps must have its quadlet unit on disk and its
/// service active. Returns a list of (companion, error) for anything
/// that needed correction and failed.
///
/// Called from `boot_reconciler` so a deleted unit file or a stopped
/// service is repaired within one tick. No-ops if the user systemd
/// manager is not reachable (CI / test environments).
pub async fn reconcile(installed_apps: &[String]) -> Vec<(String, anyhow::Error)> {
if !user_systemd_available() {
return Vec::new();
}
let mut failures = Vec::new();
for app_id in installed_apps {
for spec in companions_for(app_id) {
match needs_repair(spec).await {
Ok(false) => {}
Ok(true) => {
info!(
companion = spec.name,
"reconcile: companion not active, repairing"
);
if let Err(e) = install_one(spec).await {
failures.push((spec.name.to_string(), e));
}
}
Err(e) => {
warn!(companion = spec.name, error = %e, "reconcile probe failed");
failures.push((spec.name.to_string(), e));
}
}
}
}
failures
}
/// Does this companion need install_one to be re-run? Returns true if
/// the unit file is missing, stale, or the service is not active.
async fn needs_repair(spec: &CompanionSpec) -> Result<bool> {
let dir = quadlet::unit_dir().await?;
let unit_path = dir.join(format!("{}.container", spec.name));
if !fs::try_exists(&unit_path).await.unwrap_or(false) {
return Ok(true);
}
let expected_image = ensure_image_present(spec).await?;
let expected_unit = build_unit(spec, &expected_image);
if expected_unit.render() != fs::read_to_string(&unit_path).await.unwrap_or_default() {
return Ok(true);
}
let svc = format!("{}.service", spec.name);
Ok(!quadlet::is_active(&svc).await)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn companions_for_known_apps_returns_expected_set() {
assert_eq!(companions_for("bitcoin-knots").len(), 1);
assert_eq!(companions_for("bitcoin-core").len(), 1);
assert_eq!(companions_for("bitcoin").len(), 1);
assert_eq!(companions_for("lnd").len(), 1);
assert_eq!(companions_for("electrumx").len(), 1);
assert_eq!(companions_for("electrs").len(), 1);
assert_eq!(companions_for("mempool-electrs").len(), 1);
assert_eq!(companions_for("nextcloud").len(), 0);
assert_eq!(companions_for("not-a-real-app").len(), 0);
}
#[test]
fn build_unit_uses_host_network_and_drops_caps() {
let spec = &BITCOIN_UI[0];
let u = build_unit(spec, "localhost/bitcoin-ui:latest");
assert_eq!(u.name, "archy-bitcoin-ui");
assert!(matches!(u.network, NetworkMode::Host));
assert!(u.cap_drop_all);
assert!(u.cap_add.iter().any(|c| c == "NET_BIND_SERVICE"));
assert_eq!(u.user.as_deref(), Some("0:0"));
assert_eq!(u.memory_mb, Some(128));
assert_eq!(u.bind_mounts.len(), 1);
assert_eq!(
u.bind_mounts[0].container,
PathBuf::from("/etc/nginx/conf.d/default.conf")
);
assert!(u.bind_mounts[0].read_only);
}
#[test]
fn lnd_ui_uses_port_mapping_not_host_port_80() {
let spec = &LND_UI[0];
let u = build_unit(spec, "localhost/lnd-ui:latest");
assert_eq!(u.name, "archy-lnd-ui");
assert!(matches!(u.network, NetworkMode::Bridge(ref n) if n == "bridge"));
assert_eq!(u.ports, vec![(18083, 80, "tcp".into())]);
}
}
@@ -252,7 +252,8 @@ impl DevContainerOrchestrator {
match status.state {
archipelago_container::ContainerState::Running => Ok("healthy".to_string()),
archipelago_container::ContainerState::Stopped
| archipelago_container::ContainerState::Exited => Ok("unhealthy".to_string()),
| archipelago_container::ContainerState::Exited
| archipelago_container::ContainerState::Stopping => Ok("unhealthy".to_string()),
archipelago_container::ContainerState::Created => Ok("starting".to_string()),
archipelago_container::ContainerState::Paused => Ok("paused".to_string()),
archipelago_container::ContainerState::Unknown(_) => Ok("unknown".to_string()),
+115 -29
View File
@@ -61,12 +61,17 @@ impl DockerPackageScanner {
"indeedhub-build_minio-init_1",
"indeedhub-build_relay_1",
"indeedhub-build_ffmpeg-worker_1",
"buildx_buildkit_default",
];
// First pass: collect UI containers
// First pass: collect running UI containers. Custom UI-backed apps must
// not advertise a launch URL unless their companion is actually alive.
let mut ui_containers: HashMap<String, String> = HashMap::new();
for container in &containers {
if container.name.ends_with("-ui") {
if !matches!(container.state, ContainerState::Running) {
continue;
}
// Map fedimint-ui -> fedimint, lnd-ui -> lnd (normalize archy- prefix for lookup)
let parent_app = container
.name
@@ -76,10 +81,10 @@ impl DockerPackageScanner {
.strip_prefix("archy-")
.unwrap_or(parent_app)
.to_string();
if !container.ports.is_empty() {
if let Some(ui_address) = extract_lan_address(&container.ports) {
ui_containers.insert(canonical_id, ui_address);
}
let ui_address = extract_lan_address(&container.ports)
.or_else(|| companion_lan_address(&canonical_id));
if let Some(ui_address) = ui_address {
ui_containers.insert(canonical_id, ui_address);
}
}
}
@@ -119,6 +124,11 @@ impl DockerPackageScanner {
continue;
}
if app_id.starts_with("buildx_buildkit") {
debug!("Skipping BuildKit helper container: {}", app_id);
continue;
}
// Skip UI containers (they're merged with their parent apps)
if app_id.ends_with("-ui") {
debug!("Skipping UI container: {}", app_id);
@@ -132,17 +142,18 @@ impl DockerPackageScanner {
let lan_address = if let Some(ui_address) = ui_containers.get(&app_id) {
// Apps with separate UI containers (e.g. archy-bitcoin-ui, archy-lnd-ui)
debug!("Using UI container for {}: {}", app_id, ui_address);
Some(ui_address.clone())
} else if app_id == "bitcoin-knots" {
Some("http://localhost:8334".to_string())
} else if app_id == "lnd" {
Some("http://localhost:8081".to_string())
} else if app_id == "electrumx" || app_id == "mempool-electrs" || app_id == "electrs" {
Some("http://localhost:50002".to_string())
reachable_lan_address(&app_id, Some(ui_address.clone())).await
} else {
// Dynamic: use actual port bindings from container, fall back to static map
extract_lan_address(&container.ports)
.or_else(|| PodmanClient::lan_address_for(&app_id))
// Prefer the known web UI port over arbitrary first binding
// (for example Gitea exposes SSH on 2222 before web on 3001).
let candidate = if uses_allocated_launch_port(&app_id) {
extract_lan_address(&container.ports)
.or_else(|| PodmanClient::lan_address_for(&app_id))
} else {
PodmanClient::lan_address_for(&app_id)
.or_else(|| extract_lan_address(&container.ports))
};
reachable_lan_address(&app_id, candidate).await
};
debug!(
@@ -158,21 +169,8 @@ impl DockerPackageScanner {
// Extract actual version from container image tag
let running_version = image_versions::extract_version_from_image(&container.image);
// Check for available update by comparing running image vs pinned image
let available_update =
image_versions::pinned_image_for_app(&app_id).and_then(|pinned| {
if pinned != container.image {
let pinned_version = image_versions::extract_version_from_image(&pinned);
// Don't flag if both are "latest" — no meaningful diff
if pinned_version != "latest" || running_version != "latest" {
Some(pinned_version)
} else {
None
}
} else {
None
}
});
image_versions::available_update_for_app(&app_id, &container.image);
let package = PackageDataEntry {
state: package_state.clone(),
@@ -559,10 +557,37 @@ fn get_app_metadata(app_id: &str) -> AppMetadata {
tier: "",
},
};
apply_dynamic_metadata(app_id, &mut meta);
meta.tier = get_app_tier(app_id);
meta
}
fn apply_dynamic_metadata(app_id: &str, meta: &mut AppMetadata) {
let config_path = format!("/var/lib/archipelago/app-configs/{}.json", app_id);
let Ok(data) = std::fs::read_to_string(config_path) else {
return;
};
let Ok(cfg) = serde_json::from_str::<serde_json::Value>(&data) else {
return;
};
if let Some(title) = cfg
.get("title")
.and_then(|v| v.as_str())
.map(str::trim)
.filter(|s| !s.is_empty() && s.len() <= 80)
{
meta.title = title.to_string();
}
if let Some(description) = cfg
.get("description")
.and_then(|v| v.as_str())
.map(str::trim)
.filter(|s| !s.is_empty() && s.len() <= 240)
{
meta.description = description.to_string();
}
}
/// Map app_id to Tor hidden service directory name.
/// "archipelago" is the main web UI (nginx port 80).
/// Supports container names from deploy (archy-*, btcpay-server, etc.).
@@ -633,9 +658,70 @@ fn extract_lan_address(ports: &[String]) -> Option<String> {
None
}
async fn reachable_lan_address(app_id: &str, candidate: Option<String>) -> Option<String> {
let url = candidate?;
if !requires_reachable_launch(app_id) {
return Some(url);
}
let Some(port) = url.rsplit(':').next().and_then(|p| p.parse::<u16>().ok()) else {
return None;
};
match tokio::time::timeout(
std::time::Duration::from_secs(2),
tokio::net::TcpStream::connect(("127.0.0.1", port)),
)
.await
{
Ok(Ok(_)) => Some(url),
_ => {
debug!(app_id = %app_id, port, "suppressing unreachable launch URL");
None
}
}
}
fn requires_reachable_launch(app_id: &str) -> bool {
matches!(
app_id,
"botfights"
| "btcpay-server"
| "fedimint"
| "filebrowser"
| "grafana"
| "homeassistant"
| "home-assistant"
| "jellyfin"
| "mempool"
| "nginx-proxy-manager"
| "uptime-kuma"
| "gitea"
| "nextcloud"
| "portainer"
| "tailscale"
| "immich"
| "searxng"
)
}
fn companion_lan_address(app_id: &str) -> Option<String> {
match app_id {
"bitcoin" | "bitcoin-knots" | "bitcoin-core" => Some("http://localhost:8334".to_string()),
"electrumx" | "mempool-electrs" | "electrs" => Some("http://localhost:50002".to_string()),
_ => None,
}
}
fn uses_allocated_launch_port(app_id: &str) -> bool {
matches!(
app_id,
"filebrowser" | "nextcloud" | "nginx-proxy-manager" | "vaultwarden"
)
}
fn convert_state(container_state: &ContainerState) -> (PackageState, ServiceStatus) {
match container_state {
ContainerState::Running => (PackageState::Running, ServiceStatus::Running),
ContainerState::Stopping => (PackageState::Stopping, ServiceStatus::Stopped),
ContainerState::Stopped => (PackageState::Stopped, ServiceStatus::Stopped),
ContainerState::Exited => (PackageState::Exited, ServiceStatus::Stopped),
ContainerState::Created => (PackageState::Stopped, ServiceStatus::Stopped),
@@ -205,6 +205,30 @@ pub fn pinned_image_for_app(app_id: &str) -> Option<String> {
images.get(var).cloned()
}
/// Return the pinned tag only when the running image is genuinely behind.
/// Registry host changes alone are not app updates, and floating tags are not
/// explicit versions we should advertise to users as available updates.
pub fn available_update_for_app(app_id: &str, running_image: &str) -> Option<String> {
let pinned = pinned_image_for_app(app_id)?;
let pinned_version = extract_version_from_image(&pinned);
if is_floating_tag(&pinned_version) {
return None;
}
let running_version = extract_version_from_image(running_image);
if pinned_version == running_version {
return None;
}
let pinned_repo = image_without_registry_or_tag(&pinned);
let running_repo = image_without_registry_or_tag(running_image);
if pinned_repo != running_repo {
return None;
}
Some(pinned_version)
}
/// Extract version tag from a full image reference.
/// e.g. "git.tx1138.com/lfg2025/lnd:v0.18.4-beta" → "v0.18.4-beta"
/// Returns "latest" if no tag or tag is empty.
@@ -223,6 +247,32 @@ pub fn extract_version_from_image(image: &str) -> String {
"latest".to_string()
}
fn is_floating_tag(tag: &str) -> bool {
matches!(tag, "latest" | "stable" | "release" | "main")
}
fn image_without_registry_or_tag(image: &str) -> &str {
let without_tag = strip_tag(image);
match without_tag.split_once('/') {
Some((first, rest))
if first.contains('.') || first.contains(':') || first == "localhost" =>
{
rest
}
_ => without_tag,
}
}
fn strip_tag(image: &str) -> &str {
if let Some(slash_pos) = image.rfind('/') {
let after_slash = &image[slash_pos..];
if let Some(colon_pos) = after_slash.rfind(':') {
return &image[..slash_pos + colon_pos];
}
}
image
}
/// Container names and their image variable names for multi-container stacks.
/// Returns empty vec for single-container apps.
pub fn containers_for_stack(app_id: &str) -> Vec<(&'static str, &'static str)> {
@@ -286,6 +336,25 @@ mod tests {
);
}
#[test]
fn strips_registry_and_tag_for_image_identity() {
assert_eq!(
image_without_registry_or_tag("146.59.87.168:3000/lfg2025/lnd:v0.18.4-beta"),
"lfg2025/lnd"
);
assert_eq!(
image_without_registry_or_tag("git.tx1138.com/lfg2025/lnd:v0.18.4-beta"),
"lfg2025/lnd"
);
}
#[test]
fn floating_tags_are_not_explicit_updates() {
assert!(is_floating_tag("latest"));
assert!(is_floating_tag("stable"));
assert!(!is_floating_tag("v0.18.4-beta"));
}
#[test]
fn test_parse_image_versions() {
let content = r#"
+453
View File
@@ -0,0 +1,453 @@
//! lnd config bootstrap helper.
use anyhow::{Context, Result};
use base64::Engine;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
use tokio::fs;
use crate::update::host_sudo;
pub const DEFAULT_DATA_DIR: &str = "/var/lib/archipelago/lnd";
pub const DEFAULT_CONF_PATH: &str = "/var/lib/archipelago/lnd/lnd.conf";
const LND_REST_BASE_URL: &str = "https://127.0.0.1:18080";
pub const WALLET_PASSWORD: &str = "hellohello";
#[derive(Debug, Clone)]
pub struct EnsurePaths {
pub data_dir: PathBuf,
pub conf_path: PathBuf,
}
impl Default for EnsurePaths {
fn default() -> Self {
Self {
data_dir: PathBuf::from(DEFAULT_DATA_DIR),
conf_path: PathBuf::from(DEFAULT_CONF_PATH),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EnsureOutcome {
Written,
Unchanged,
}
pub async fn ensure_config(paths: &EnsurePaths, rpc_pass: &str) -> Result<EnsureOutcome> {
fs::create_dir_all(&paths.data_dir)
.await
.with_context(|| format!("creating {}", paths.data_dir.display()))?;
if paths.conf_path.exists() {
let existing = fs::read_to_string(&paths.conf_path)
.await
.with_context(|| format!("reading {}", paths.conf_path.display()))?;
if has_required_lnd_flags(&existing, rpc_pass) {
return Ok(EnsureOutcome::Unchanged);
}
}
let conf = format!(
"debuglevel=info\n\
maxpendingchannels=10\n\
alias=Archipelago Node\n\
color=#f7931a\n\
listen=0.0.0.0:9735\n\
rpclisten=0.0.0.0:10009\n\
restlisten=0.0.0.0:8080\n\
bitcoin.active=true\n\
bitcoin.mainnet=true\n\
bitcoin.node=bitcoind\n\
bitcoind.rpchost=bitcoin-knots:8332\n\
bitcoind.rpcuser=archipelago\n\
bitcoind.rpcpass={}\n\
bitcoind.rpcpolling=true\n\
bitcoind.estimatemode=ECONOMICAL\n",
rpc_pass
);
write_config_atomically(paths, &conf).await?;
Ok(EnsureOutcome::Written)
}
pub async fn ensure_wallet_initialized() -> Result<()> {
let admin_macaroon = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
let wallet_db = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/wallet.db";
if file_exists_as_root(wallet_db).await {
if file_exists_as_root(admin_macaroon).await && lnd_getinfo_ready(admin_macaroon).await {
return Ok(());
}
unlock_existing_wallet().await?;
wait_for_admin_macaroon(admin_macaroon).await?;
return Ok(());
}
init_wallet_via_rest().await?;
wait_for_admin_macaroon(admin_macaroon).await
}
async fn file_exists_as_root(path: &str) -> bool {
if std::path::Path::new(path).exists() {
return true;
}
tokio::process::Command::new("sudo")
.args(["test", "-f", path])
.status()
.await
.map(|status| status.success())
.unwrap_or(false)
}
async fn read_file_as_root(path: &str) -> Result<Vec<u8>> {
match fs::read(path).await {
Ok(bytes) => Ok(bytes),
Err(direct_err) => {
let out = tokio::process::Command::new("sudo")
.args(["cat", path])
.output()
.await
.with_context(|| format!("reading {path} via sudo"))?;
if out.status.success() {
Ok(out.stdout)
} else {
anyhow::bail!(
"reading {path} failed (direct: {direct_err}; sudo: {})",
String::from_utf8_lossy(&out.stderr).trim()
)
}
}
}
}
async fn unlock_existing_wallet() -> Result<()> {
unlock_existing_wallet_via_rest().await
}
async fn unlock_existing_wallet_via_rest() -> Result<()> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(20))
.danger_accept_invalid_certs(true)
.build()
.context("building LND REST client")?;
let wallet_password = base64::engine::general_purpose::STANDARD.encode(WALLET_PASSWORD);
match post_lnd_unlocker_json::<serde_json::Value>(
&client,
"/v1/unlockwallet",
serde_json::json!({ "wallet_password": wallet_password }),
)
.await
.context("unlocking existing LND wallet")?
{
UnlockerResponse::Value(_) | UnlockerResponse::WalletAlreadyExists => Ok(()),
}
}
#[allow(dead_code)]
async fn unlock_existing_wallet_via_lncli() -> Result<()> {
let mut last_err = None;
for _ in 0..60 {
let mut cmd = tokio::process::Command::new("podman");
cmd.args(["exec", "-i", "lnd", "lncli", "unlock", "--stdin"]);
cmd.stdin(std::process::Stdio::piped());
cmd.stdout(std::process::Stdio::piped());
cmd.stderr(std::process::Stdio::piped());
let mut child = cmd.spawn().context("spawning lncli wallet unlock")?;
if let Some(mut stdin) = child.stdin.take() {
use tokio::io::AsyncWriteExt;
stdin
.write_all(format!("{}\n", WALLET_PASSWORD).as_bytes())
.await
.context("writing lncli password")?;
}
let out = child
.wait_with_output()
.await
.context("waiting for lncli")?;
if out.status.success() {
return Ok(());
}
let stderr = String::from_utf8_lossy(&out.stderr);
let stdout = String::from_utf8_lossy(&out.stdout);
let msg = format!("{stderr}{stdout}");
if msg.contains("wallet already unlocked") || msg.contains("already unlocked") {
return Ok(());
}
last_err = Some(msg);
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
anyhow::bail!(
"lncli wallet unlock failed: {}",
last_err.unwrap_or_else(|| "unknown error".to_string())
)
}
#[derive(Debug, Deserialize)]
struct GenSeedResponse {
cipher_seed_mnemonic: Vec<String>,
}
#[derive(Debug)]
enum UnlockerResponse<T> {
Value(T),
WalletAlreadyExists,
}
#[derive(Debug, Serialize)]
struct InitWalletRequest {
wallet_password: String,
cipher_seed_mnemonic: Vec<String>,
}
async fn init_wallet_via_rest() -> Result<()> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(20))
.danger_accept_invalid_certs(true)
.build()
.context("building LND REST client")?;
let seed: GenSeedResponse = match get_lnd_unlocker_json(&client, "/v1/genseed")
.await
.context("generating LND wallet seed")?
{
UnlockerResponse::Value(seed) => seed,
UnlockerResponse::WalletAlreadyExists => {
unlock_existing_wallet().await?;
return Ok(());
}
};
if seed.cipher_seed_mnemonic.is_empty() {
anyhow::bail!("LND genseed returned no seed words");
}
let wallet_password = base64::engine::general_purpose::STANDARD.encode(WALLET_PASSWORD);
let req = InitWalletRequest {
wallet_password,
cipher_seed_mnemonic: seed.cipher_seed_mnemonic,
};
match post_lnd_unlocker_json::<serde_json::Value>(
&client,
"/v1/initwallet",
serde_json::to_value(req)?,
)
.await
.context("initializing LND wallet")?
{
UnlockerResponse::Value(_) => {}
UnlockerResponse::WalletAlreadyExists => unlock_existing_wallet().await?,
}
Ok(())
}
async fn get_lnd_unlocker_json<T: for<'de> Deserialize<'de>>(
client: &reqwest::Client,
path: &str,
) -> Result<UnlockerResponse<T>> {
let url = format!("{LND_REST_BASE_URL}{path}");
let mut last_err = None;
for _ in 0..60 {
match client.get(&url).send().await {
Ok(resp) => match decode_lnd_unlocker_response(resp, path).await {
Ok(value) => return Ok(value),
Err(e) => last_err = Some(e.to_string()),
},
Err(e) => last_err = Some(e.to_string()),
}
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
anyhow::bail!(
"LND REST {path} unavailable: {}",
last_err.unwrap_or_else(|| "unknown error".to_string())
)
}
async fn post_lnd_unlocker_json<T: for<'de> Deserialize<'de>>(
client: &reqwest::Client,
path: &str,
body: serde_json::Value,
) -> Result<UnlockerResponse<T>> {
let url = format!("{LND_REST_BASE_URL}{path}");
let mut last_err = None;
for _ in 0..60 {
match client.post(&url).json(&body).send().await {
Ok(resp) => match decode_lnd_unlocker_response(resp, path).await {
Ok(value) => return Ok(value),
Err(e) => last_err = Some(e.to_string()),
},
Err(e) => last_err = Some(e.to_string()),
}
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
anyhow::bail!(
"LND REST {path} unavailable: {}",
last_err.unwrap_or_else(|| "unknown error".to_string())
)
}
async fn decode_lnd_unlocker_response<T: for<'de> Deserialize<'de>>(
resp: reqwest::Response,
path: &str,
) -> Result<UnlockerResponse<T>> {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
if status.is_success() {
let value = serde_json::from_str(&text)
.with_context(|| format!("parsing LND REST response from {path}"))?;
return Ok(UnlockerResponse::Value(value));
}
if text.contains("wallet already exists") {
return Ok(UnlockerResponse::WalletAlreadyExists);
}
anyhow::bail!("LND REST {path} returned {status}: {text}")
}
async fn lnd_getinfo_ready(admin_macaroon: &str) -> bool {
let Ok(macaroon) = read_file_as_root(admin_macaroon).await else {
return false;
};
let Ok(client) = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(5))
.danger_accept_invalid_certs(true)
.build()
else {
return false;
};
client
.get(format!("{LND_REST_BASE_URL}/v1/getinfo"))
.header("Grpc-Metadata-macaroon", hex::encode(macaroon))
.send()
.await
.map(|resp| resp.status().is_success())
.unwrap_or(false)
}
async fn wait_for_admin_macaroon(admin_macaroon: &str) -> Result<()> {
for _ in 0..60 {
if file_exists_as_root(admin_macaroon).await {
return Ok(());
}
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
anyhow::bail!("LND admin macaroon not created after wallet init")
}
async fn write_config_atomically(paths: &EnsurePaths, conf: &str) -> Result<()> {
let tmp = paths.conf_path.with_extension("tmp");
match fs::write(&tmp, conf).await {
Ok(()) => {
fs::rename(&tmp, &paths.conf_path).await.with_context(|| {
format!(
"renaming {} -> {}",
tmp.display(),
paths.conf_path.display()
)
})?;
Ok(())
}
Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => {
let script = format!(
"set -eu\ncat > '{}' <<'LNDCONF'\n{}LNDCONF\n",
shell_quote(&paths.conf_path.to_string_lossy()),
conf
);
let status = host_sudo(&["sh", "-lc", &script])
.await
.context("writing lnd.conf via sudo")?;
if !status.success() {
anyhow::bail!("writing lnd.conf via sudo exited with {status}");
}
Ok(())
}
Err(e) => Err(e).with_context(|| format!("writing tmp {}", tmp.display())),
}
}
fn shell_quote(s: &str) -> String {
s.replace('\'', "'\\''")
}
fn has_required_lnd_flags(conf: &str, rpc_pass: &str) -> bool {
let rpc_pass_line = format!("bitcoind.rpcpass={rpc_pass}");
[
"bitcoin.active=true",
"bitcoin.mainnet=true",
"bitcoin.node=bitcoind",
"bitcoind.rpchost=bitcoin-knots:8332",
rpc_pass_line.as_str(),
]
.iter()
.all(|needle| conf.lines().any(|line| line.trim() == *needle))
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn ensure_config_writes_required_bitcoin_network_flags() {
let tmp = tempfile::TempDir::new().unwrap();
let paths = EnsurePaths {
data_dir: tmp.path().join("lnd"),
conf_path: tmp.path().join("lnd/lnd.conf"),
};
let out = ensure_config(&paths, "secret").await.unwrap();
assert_eq!(out, EnsureOutcome::Written);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoin.active=true"));
assert!(conf.contains("bitcoin.mainnet=true"));
assert!(conf.contains("bitcoin.node=bitcoind"));
assert!(conf.contains("bitcoind.rpchost=bitcoin-knots:8332"));
assert!(conf.contains("bitcoind.rpcpass=secret"));
}
#[tokio::test]
async fn ensure_config_repairs_rpc_password_drift() {
let tmp = tempfile::TempDir::new().unwrap();
let paths = EnsurePaths {
data_dir: tmp.path().join("lnd"),
conf_path: tmp.path().join("lnd/lnd.conf"),
};
assert_eq!(
ensure_config(&paths, "first").await.unwrap(),
EnsureOutcome::Written
);
assert_eq!(
ensure_config(&paths, "second").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoind.rpcpass=second"));
}
#[tokio::test]
async fn ensure_config_repairs_incomplete_existing_config() {
let tmp = tempfile::TempDir::new().unwrap();
let paths = EnsurePaths {
data_dir: tmp.path().join("lnd"),
conf_path: tmp.path().join("lnd/lnd.conf"),
};
fs::create_dir_all(&paths.data_dir).await.unwrap();
fs::write(&paths.conf_path, "debuglevel=info\n")
.await
.unwrap();
assert_eq!(
ensure_config(&paths, "repaired").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoin.mainnet=true"));
assert!(conf.contains("bitcoind.rpcpass=repaired"));
}
#[test]
fn wallet_password_is_valid_for_lncli() {
assert!(WALLET_PASSWORD.len() > 8);
}
}
+4 -4
View File
@@ -1,19 +1,19 @@
pub mod bitcoin_ui;
pub mod boot_reconciler;
pub mod companion;
pub mod data_manager;
pub mod dev_orchestrator;
pub mod docker_packages;
pub mod filebrowser;
pub mod image_versions;
pub mod lnd;
pub mod prod_orchestrator;
pub mod quadlet;
pub mod registry;
pub mod traits;
pub use boot_reconciler::{BootReconciler, DEFAULT_INTERVAL as RECONCILER_DEFAULT_INTERVAL};
pub use dev_orchestrator::DevContainerOrchestrator;
pub use docker_packages::DockerPackageScanner;
pub use prod_orchestrator::{
compute_container_name, AdoptionReport, ProdContainerOrchestrator, ReconcileAction,
ReconcileReport,
};
pub use prod_orchestrator::ProdContainerOrchestrator;
pub use traits::ContainerOrchestrator;
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+48 -3
View File
@@ -8,9 +8,10 @@ use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use std::path::Path;
use tokio::fs;
use tracing::{debug, info};
const REGISTRY_FILE: &str = "config/registries.json";
const OVH_REGISTRY_URL: &str = "146.59.87.168:3000/lfg2025";
const TX1138_REGISTRY_URL: &str = "git.tx1138.com/lfg2025";
/// A single container registry.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -44,14 +45,14 @@ impl Default for RegistryConfig {
Self {
registries: vec![
Registry {
url: "146.59.87.168:3000/lfg2025".to_string(),
url: OVH_REGISTRY_URL.to_string(),
name: "Server 1 (OVH)".to_string(),
tls_verify: false,
enabled: true,
priority: 0,
},
Registry {
url: "git.tx1138.com/lfg2025".to_string(),
url: TX1138_REGISTRY_URL.to_string(),
name: "Server 2 (tx1138)".to_string(),
tls_verify: true,
enabled: true,
@@ -139,6 +140,19 @@ pub async fn load_registries(data_dir: &Path) -> Result<RegistryConfig> {
changed = true;
}
}
let before_order: Vec<(String, bool, u32)> = config
.registries
.iter()
.map(|r| (r.url.clone(), r.enabled, r.priority))
.collect();
force_ovh_registry_primary(&mut config);
changed = changed
|| before_order
!= config
.registries
.iter()
.map(|r| (r.url.clone(), r.enabled, r.priority))
.collect::<Vec<_>>();
if changed {
// Persist so the next load doesn't have to re-merge.
let _ = save_registries(data_dir, &config).await;
@@ -146,6 +160,37 @@ pub async fn load_registries(data_dir: &Path) -> Result<RegistryConfig> {
Ok(config)
}
fn force_ovh_registry_primary(config: &mut RegistryConfig) {
let defaults = RegistryConfig::default();
for def in defaults.registries {
if !config.registries.iter().any(|r| r.url == def.url) {
config.registries.push(def);
}
}
for registry in config.registries.iter_mut() {
match registry.url.as_str() {
OVH_REGISTRY_URL => {
registry.name = "Server 1 (OVH)".to_string();
registry.tls_verify = false;
registry.enabled = true;
registry.priority = 0;
}
TX1138_REGISTRY_URL => {
registry.name = "Server 2 (tx1138)".to_string();
registry.tls_verify = true;
registry.enabled = true;
registry.priority = 10;
}
_ => {
if registry.priority <= 10 {
registry.priority = registry.priority.saturating_add(20);
}
}
}
}
}
/// Save registry config to disk.
pub async fn save_registries(data_dir: &Path, config: &RegistryConfig) -> Result<()> {
let dir = data_dir.join("config");
+83 -60
View File
@@ -11,8 +11,9 @@
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::process::Output;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
use std::time::{Duration, Instant};
use tokio::fs;
use tracing::{info, warn};
@@ -189,15 +190,9 @@ pub async fn remove_pid_marker(data_dir: &Path) {
/// Save a snapshot of currently running containers to disk.
/// Called periodically so we know what to restart after a crash.
pub async fn save_container_snapshot(data_dir: &Path) -> Result<()> {
let output = tokio::time::timeout(
std::time::Duration::from_secs(30),
tokio::process::Command::new("podman")
.args(["ps", "--format", "json"])
.output(),
)
.await
.context("podman ps timed out (30s)")?
.context("Failed to run podman ps")?;
let mut cmd = tokio::process::Command::new("podman");
cmd.args(["ps", "--format", "json"]);
let output = command_with_timeout(cmd, Duration::from_secs(30), "podman ps").await?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
@@ -277,22 +272,23 @@ pub async fn recover_containers(containers: &[RunningContainerRecord]) -> Recove
);
tokio::time::sleep(std::time::Duration::from_secs(10)).await;
}
let result = tokio::time::timeout(
std::time::Duration::from_secs(timeout_secs),
tokio::process::Command::new("podman")
.args(["start", &record.name])
.output(),
let mut cmd = tokio::process::Command::new("podman");
cmd.args(["start", &record.name]);
let result = command_with_timeout(
cmd,
Duration::from_secs(timeout_secs),
&format!("podman start {}", record.name),
)
.await;
match result {
Ok(Ok(output)) if output.status.success() => {
Ok(output) if output.status.success() => {
info!("Successfully restarted container: {}", record.name);
report.recovered += 1;
started = true;
break;
}
Ok(Ok(output)) => {
Ok(output) => {
let stderr = String::from_utf8_lossy(&output.stderr);
warn!(
"Failed to restart container {} (attempt {}): {}",
@@ -301,20 +297,13 @@ pub async fn recover_containers(containers: &[RunningContainerRecord]) -> Recove
stderr.trim()
);
}
Ok(Err(e)) => {
Err(e) => {
warn!(
"Failed to execute podman start for {} (attempt {}): {}",
record.name,
attempt + 1,
e
);
}
Err(_) => {
warn!(
"Timeout starting container {} ({}s, attempt {})",
"Failed to start container {} ({}s, attempt {}): {}",
record.name,
timeout_secs,
attempt + 1
attempt + 1,
e
);
}
}
@@ -345,26 +334,22 @@ fn is_process_running(pid: u32) -> bool {
/// The crash recovery (PID-based) handles dirty shutdowns; this handles clean ones.
/// Skips containers that the user intentionally stopped via the UI.
pub async fn start_stopped_containers(data_dir: &Path) -> RecoveryReport {
let output = match tokio::time::timeout(
std::time::Duration::from_secs(60),
tokio::process::Command::new("podman")
.args([
"ps",
"-a",
"--filter",
"status=exited",
"--filter",
"status=created",
"--format",
"{{.Names}}",
])
.output(),
)
.await
let mut cmd = tokio::process::Command::new("podman");
cmd.args([
"ps",
"-a",
"--filter",
"status=exited",
"--filter",
"status=created",
"--format",
"{{.Names}}",
]);
let output = match command_with_timeout(cmd, Duration::from_secs(60), "podman ps stopped").await
{
Ok(result) => result,
Err(_) => {
warn!("Timeout listing stopped containers (60s)");
Ok(output) => output,
Err(e) => {
warn!("Failed listing stopped containers: {}", e);
return RecoveryReport {
total: 0,
recovered: 0,
@@ -373,13 +358,14 @@ pub async fn start_stopped_containers(data_dir: &Path) -> RecoveryReport {
}
};
let all_names: Vec<String> = match output {
Ok(o) if o.status.success() => String::from_utf8_lossy(&o.stdout)
let all_names: Vec<String> = if output.status.success() {
String::from_utf8_lossy(&output.stdout)
.lines()
.filter(|l| !l.is_empty())
.map(|s| s.to_string())
.collect(),
_ => Vec::new(),
.collect()
} else {
Vec::new()
};
if all_names.is_empty() {
@@ -412,6 +398,19 @@ pub async fn start_stopped_containers(data_dir: &Path) -> RecoveryReport {
};
}
let names: Vec<String> = names
.into_iter()
.filter(|n| should_auto_start_stopped_container(n))
.collect();
if names.is_empty() {
return RecoveryReport {
total: 0,
recovered: 0,
failed: Vec::new(),
};
}
// Sort by startup tier: databases first, then core, then dependent services, then apps
let mut records: Vec<RunningContainerRecord> = names
.iter()
@@ -430,6 +429,13 @@ pub async fn start_stopped_containers(data_dir: &Path) -> RecoveryReport {
recover_containers(&records).await
}
fn should_auto_start_stopped_container(name: &str) -> bool {
// Keep generic boot recovery narrow. The Rust manifest reconciler owns
// managed app stacks; starting every exited Podman container here races
// it and resurrects legacy/orphan helper containers.
matches!(name, "filebrowser" | "nostr-rs-relay")
}
/// Simple tier ordering for boot recovery (mirrors health_monitor tiers).
fn container_boot_tier(name: &str) -> u8 {
let id = name.strip_prefix("archy-").unwrap_or(name);
@@ -461,27 +467,35 @@ pub async fn run_boot_reconciliation() {
return;
}
info!("Running boot reconciliation...");
let result = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new(script).output(),
)
.await;
let cmd = tokio::process::Command::new(script);
let result = command_with_timeout(cmd, Duration::from_secs(300), script).await;
match result {
Ok(Ok(output)) if output.status.success() => {
Ok(output) if output.status.success() => {
info!("Boot reconciliation complete");
}
Ok(Ok(output)) => {
Ok(output) => {
let stderr = String::from_utf8_lossy(&output.stderr);
warn!(
"Boot reconciliation had failures: {}",
stderr.chars().take(500).collect::<String>()
);
}
Ok(Err(e)) => warn!("Boot reconciliation failed to run: {}", e),
Err(_) => warn!("Boot reconciliation timed out (300s)"),
Err(e) => warn!("Boot reconciliation failed: {}", e),
}
}
async fn command_with_timeout(
mut cmd: tokio::process::Command,
timeout: Duration,
description: &str,
) -> Result<Output> {
cmd.kill_on_drop(true);
tokio::time::timeout(timeout, cmd.output())
.await
.with_context(|| format!("{} timed out after {}s", description, timeout.as_secs()))?
.with_context(|| format!("Failed to run {}", description))
}
/// Spawn a background task that periodically saves the container snapshot.
pub fn spawn_snapshot_task(data_dir: PathBuf) {
tokio::spawn(async move {
@@ -603,4 +617,13 @@ mod tests {
let result = check_for_crash(tmp.path()).await.unwrap();
assert!(result.is_none());
}
#[test]
fn generic_boot_recovery_skips_manifest_owned_and_legacy_stacks() {
assert!(should_auto_start_stopped_container("filebrowser"));
assert!(should_auto_start_stopped_container("nostr-rs-relay"));
assert!(!should_auto_start_stopped_container("bitcoin-knots"));
assert!(!should_auto_start_stopped_container("lnd"));
assert!(!should_auto_start_stopped_container("indeedhub-postgres"));
}
}
+181 -42
View File
@@ -30,9 +30,11 @@ async fn bitcoin_rpc_auth() -> String {
#[derive(Debug, Clone, Serialize)]
pub struct ElectrsSyncStatus {
pub indexed_height: u64,
pub bitcoin_height: u64,
pub network_height: u64,
pub progress_pct: f64,
pub status: String,
pub stale: bool,
pub error: Option<String>,
/// Index data size in human-readable format (e.g. "11.2 GB")
pub index_size: Option<String>,
@@ -44,9 +46,11 @@ impl Default for ElectrsSyncStatus {
fn default() -> Self {
Self {
indexed_height: 0,
bitcoin_height: 0,
network_height: 0,
progress_pct: 0.0,
status: "starting".to_string(),
stale: false,
error: None,
index_size: None,
tor_onion: None,
@@ -64,15 +68,31 @@ fn cache() -> &'static RwLock<ElectrsSyncStatus> {
/// Spawn background task that refreshes ElectrumX status every CACHE_REFRESH_SECS.
pub fn spawn_status_cache() {
tokio::spawn(async {
// Initial delay — let services start up before first query
tokio::time::sleep(Duration::from_secs(5)).await;
let mut interval = tokio::time::interval(Duration::from_secs(CACHE_REFRESH_SECS));
loop {
interval.tick().await;
let fresh = fetch_electrs_sync_status().await;
let mut fresh = fetch_electrs_sync_status().await;
let mut cached = cache().write().await;
if fresh.indexed_height == 0
&& cached.indexed_height > 0
&& matches!(fresh.status.as_str(), "indexing" | "waiting")
{
fresh.indexed_height = cached.indexed_height;
if fresh.network_height == 0 {
fresh.network_height = cached.network_height;
}
if fresh.bitcoin_height == 0 {
fresh.bitcoin_height = cached.bitcoin_height;
}
if fresh.progress_pct <= 0.0 {
fresh.progress_pct = cached.progress_pct;
}
fresh.stale = true;
fresh.error = Some(fresh.error.unwrap_or_else(|| {
"ElectrumX is reconnecting; showing last known indexed height.".to_string()
}));
}
*cached = fresh;
drop(cached);
tokio::time::sleep(Duration::from_secs(CACHE_REFRESH_SECS)).await;
}
});
}
@@ -187,13 +207,69 @@ async fn electrumx_indexed_height() -> Result<u64> {
Ok(height)
}
/// Fetch Bitcoin network height via JSON-RPC.
async fn bitcoin_network_height() -> Result<u64> {
fn parse_electrumx_height_from_logs(logs: &str) -> Option<u64> {
let mut height = None;
for line in logs.lines() {
if let Some(idx) = line.find("BlockProcessor:our height:") {
let rest = &line[idx + "BlockProcessor:our height:".len()..];
if let Some(parsed) = parse_first_u64_token(rest) {
height = Some(parsed);
}
continue;
}
if let Some(idx) = line.find("DB:height:") {
let rest = &line[idx + "DB:height:".len()..];
if let Some(parsed) = parse_first_u64_token(rest) {
height = Some(parsed);
}
}
}
height
}
fn parse_first_u64_token(input: &str) -> Option<u64> {
let token: String = input
.trim_start()
.chars()
.take_while(|c| c.is_ascii_digit() || *c == ',')
.filter(|c| *c != ',')
.collect();
if token.is_empty() {
None
} else {
token.parse().ok()
}
}
async fn electrumx_log_indexed_height() -> Result<u64> {
let output = tokio::process::Command::new("podman")
.args(["logs", "--tail", "500", "electrumx"])
.output()
.await
.context("Failed to read ElectrumX logs")?;
if !output.status.success() {
anyhow::bail!(
"podman logs electrumx failed: {}",
String::from_utf8_lossy(&output.stderr).trim()
);
}
let logs = String::from_utf8_lossy(&output.stdout);
parse_electrumx_height_from_logs(&logs).context("No ElectrumX indexed height in logs")
}
/// Fetch Bitcoin local block height and best-known network header height via JSON-RPC.
async fn bitcoin_chain_heights() -> Result<(u64, u64)> {
let client = reqwest::Client::new();
let body = serde_json::json!({
"jsonrpc": "1.0",
"id": "electrs-status",
"method": "getblockcount",
"method": "getblockchaininfo",
"params": []
});
let resp = client
@@ -211,11 +287,18 @@ async fn bitcoin_network_height() -> Result<u64> {
}
let json: serde_json::Value = resp.json().await?;
let height = json
let result = json
.get("result")
.and_then(|r| r.as_u64())
.context("Missing result in Bitcoin RPC")?;
Ok(height)
let blocks = result
.get("blocks")
.and_then(|h| h.as_u64())
.context("Missing blocks in Bitcoin RPC")?;
let headers = result
.get("headers")
.and_then(|h| h.as_u64())
.unwrap_or(blocks);
Ok((blocks, headers.max(blocks)))
}
/// Fetch fresh ElectrumX sync status (called by background cache task).
@@ -260,8 +343,8 @@ async fn fetch_electrs_sync_status() -> ElectrsSyncStatus {
onion
};
let network_height = match bitcoin_network_height().await {
Ok(h) => h,
let (bitcoin_blocks, network_height) = match bitcoin_chain_heights().await {
Ok(heights) => heights,
Err(e) => {
let err_msg = e.to_string();
if is_transient_error(&err_msg) {
@@ -271,9 +354,11 @@ async fn fetch_electrs_sync_status() -> ElectrsSyncStatus {
}
return ElectrsSyncStatus {
indexed_height: 0,
bitcoin_height: 0,
network_height: 0,
progress_pct: 0.0,
status: "waiting".to_string(),
stale: false,
error: Some("Waiting for Bitcoin node...".to_string()),
index_size,
tor_onion,
@@ -283,21 +368,25 @@ async fn fetch_electrs_sync_status() -> ElectrsSyncStatus {
let indexed_height = match electrumx_indexed_height().await {
Ok(h) => h,
Err(e) => {
let err_msg = e.to_string();
if is_transient_error(&err_msg) {
// ElectrumX is starting up or busy — estimate from data size
let progress_pct = if data_bytes > 0 {
((data_bytes as f64 / ESTIMATED_FULL_INDEX_BYTES) * 100.0).min(99.0)
} else {
0.0
};
let size_str = index_size.clone().unwrap_or_else(|| "0 MB".to_string());
return ElectrsSyncStatus {
Err(e) => match electrumx_log_indexed_height().await {
Ok(h) if h > 0 => h,
_ => {
let err_msg = e.to_string();
if is_transient_error(&err_msg) {
// ElectrumX is starting up or busy — estimate from data size
let progress_pct = if data_bytes > 0 {
((data_bytes as f64 / ESTIMATED_FULL_INDEX_BYTES) * 100.0).min(99.0)
} else {
0.0
};
let size_str = index_size.clone().unwrap_or_else(|| "0 MB".to_string());
return ElectrsSyncStatus {
indexed_height: 0,
bitcoin_height: bitcoin_blocks,
network_height,
progress_pct,
status: "indexing".to_string(),
stale: false,
error: Some(format!(
"Building index ({} / ~130 GB estimated). Electrum RPC will be available when complete.",
size_str
@@ -305,40 +394,90 @@ async fn fetch_electrs_sync_status() -> ElectrsSyncStatus {
index_size,
tor_onion,
};
}
// Genuine unexpected error
warn!("ElectrumX status: unexpected error: {}", err_msg);
return ElectrsSyncStatus {
indexed_height: 0,
bitcoin_height: bitcoin_blocks,
network_height,
progress_pct: 0.0,
status: "error".to_string(),
stale: false,
error: Some(format!("ElectrumX: {}", err_msg)),
index_size,
tor_onion,
};
}
// Genuine unexpected error
warn!("ElectrumX status: unexpected error: {}", err_msg);
return ElectrsSyncStatus {
indexed_height: 0,
network_height,
progress_pct: 0.0,
status: "error".to_string(),
error: Some(format!("ElectrumX: {}", err_msg)),
index_size,
tor_onion,
};
}
},
};
let progress_pct = if network_height > 0 {
(indexed_height as f64 / network_height as f64) * 100.0
let observed_header_height = network_height.max(indexed_height);
let bitcoin_catching_up = bitcoin_blocks > 0 && bitcoin_blocks < observed_header_height;
let electrum_waiting_on_bitcoin =
bitcoin_catching_up && indexed_height >= bitcoin_blocks.saturating_sub(1);
let sync_target_height = if bitcoin_blocks > 0 {
bitcoin_blocks
} else {
observed_header_height
};
let progress_pct = if electrum_waiting_on_bitcoin && observed_header_height > 0 {
((bitcoin_blocks as f64 / observed_header_height as f64) * 100.0).min(99.9)
} else if sync_target_height > 0 {
((indexed_height as f64 / sync_target_height as f64) * 100.0).min(100.0)
} else {
0.0
};
let status = if indexed_height >= network_height.saturating_sub(1) {
let status = if sync_target_height == 0 {
"waiting"
} else if electrum_waiting_on_bitcoin {
"waiting"
} else if indexed_height >= sync_target_height.saturating_sub(1) {
"synced"
} else {
"syncing"
};
let error = if electrum_waiting_on_bitcoin {
Some(format!(
"ElectrumX is indexed to {:}; waiting for the local Bitcoin node to catch up from {:} to known header {:}.",
indexed_height, bitcoin_blocks, observed_header_height
))
} else if status == "syncing" && bitcoin_blocks < observed_header_height {
Some(format!(
"Indexing local Bitcoin node height {:} of {:}. Bitcoin node is still catching up to known header {:}.",
indexed_height, bitcoin_blocks, observed_header_height
))
} else {
None
};
ElectrsSyncStatus {
indexed_height,
network_height,
bitcoin_height: bitcoin_blocks,
network_height: observed_header_height,
progress_pct,
status: status.to_string(),
error: None,
stale: false,
error,
index_size,
tor_onion,
}
}
#[cfg(test)]
mod tests {
use super::parse_electrumx_height_from_logs;
#[test]
fn parses_latest_electrumx_progress_height_from_logs() {
let logs = r#"
INFO:DB:height: 228,238
INFO:BlockProcessor:our height: 228,248 daemon: 731,568 UTXOs 1MB hist 1MB
INFO:BlockProcessor:our height: 232,117 daemon: 732,108 UTXOs 281MB hist 83MB
"#;
assert_eq!(parse_electrumx_height_from_logs(logs), Some(232_117));
}
}
+389 -12
View File
@@ -4,11 +4,11 @@
// handles "created" state containers, resets dependent counters when deps recover,
// and sends WebSocket notifications to the UI on failure.
use crate::data_model::{Notification, NotificationLevel};
use crate::data_model::{Notification, NotificationLevel, PackageState};
use crate::state::StateManager;
use crate::webhooks::{self, WebhookEvent};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Instant;
@@ -67,14 +67,14 @@ fn container_dependencies(name: &str) -> &'static [&'static str] {
let id = name.strip_prefix("archy-").unwrap_or(name);
match id {
// Bitcoin-dependent chain
"lnd" => &["bitcoin-knots"],
"electrumx" | "mempool-electrs" | "electrs" => &["bitcoin-knots"],
"nbxplorer" => &["bitcoin-knots"],
"lnd" => &["bitcoin"],
"electrumx" | "mempool-electrs" | "electrs" => &["bitcoin"],
"nbxplorer" => &["bitcoin"],
"btcpay-server" => &["btcpay-db", "nbxplorer"],
"mempool-api" => &["mempool-db", "electrumx"],
"mempool-web" => &["mempool-api"],
"fedimint" => &["bitcoin-knots"],
"fedimint-gateway" => &["bitcoin-knots", "fedimint"],
"fedimint" => &["bitcoin"],
"fedimint-gateway" => &["bitcoin", "fedimint"],
// IndeedHub stack
"indeedhub-api" => &["indeedhub-postgres", "indeedhub-redis"],
@@ -88,7 +88,7 @@ fn container_dependencies(name: &str) -> &'static [&'static str] {
"penpot-frontend" => &["penpot-backend"],
// UI containers
"bitcoin-ui" => &["bitcoin-knots"],
"bitcoin-ui" => &["bitcoin"],
"lnd-ui" => &["lnd"],
"electrs-ui" => &["electrumx"],
@@ -103,6 +103,16 @@ fn deps_are_running(name: &str, containers: &[ContainerHealth]) -> bool {
return true;
}
for dep in deps {
if *dep == "bitcoin" {
let bitcoin_running = containers.iter().any(|c| {
let c_id = c.name.strip_prefix("archy-").unwrap_or(&c.name);
matches!(c_id, "bitcoin" | "bitcoin-knots" | "bitcoin-core") && c.state == "running"
});
if !bitcoin_running {
return false;
}
continue;
}
// Check both plain name and archy- prefixed name
let dep_running = containers.iter().any(|c| {
let c_id = c.name.strip_prefix("archy-").unwrap_or(&c.name);
@@ -115,6 +125,24 @@ fn deps_are_running(name: &str, containers: &[ContainerHealth]) -> bool {
true
}
fn conflicting_bitcoin_variant(name: &str) -> Option<&'static str> {
match name.strip_prefix("archy-").unwrap_or(name) {
"bitcoin-core" => Some("bitcoin-knots"),
"bitcoin-knots" | "bitcoin" => Some("bitcoin-core"),
_ => None,
}
}
fn has_running_bitcoin_conflict(name: &str, containers: &[ContainerHealth]) -> bool {
let Some(conflict) = conflicting_bitcoin_variant(name) else {
return false;
};
containers.iter().any(|c| {
let id = c.name.strip_prefix("archy-").unwrap_or(&c.name);
id == conflict && c.state == "running"
})
}
/// Track restart attempts per container with exponential backoff and stability reset.
struct RestartTracker {
attempts: HashMap<String, u32>,
@@ -188,6 +216,7 @@ struct ContainerHealth {
name: String,
app_id: String,
state: String,
podman_health: Option<String>,
healthy: bool,
}
@@ -391,6 +420,9 @@ async fn check_containers() -> Vec<ContainerHealth> {
let stdout = String::from_utf8_lossy(&output.stdout);
let containers: Vec<serde_json::Value> = serde_json::from_str(&stdout).unwrap_or_default();
let live_container_ids = live_container_ids(&containers);
cleanup_stale_podman_healthcheck_units(&live_container_ids).await;
// Monitor ALL long-running containers for health — backend services (databases,
// nbxplorer, mempool-api) and UI containers need auto-restart too.
// Only skip ephemeral containers (build infrastructure, init one-shots).
@@ -419,18 +451,195 @@ async fn check_containers() -> Vec<ContainerHealth> {
.unwrap_or("unknown")
.to_lowercase();
let healthy = state == "running";
let podman_health = parse_podman_health(c, &state);
let healthy = state == "running" && podman_health.as_deref() != Some("unhealthy");
Some(ContainerHealth {
name,
app_id,
state,
podman_health,
healthy,
})
})
.collect()
}
fn live_container_ids(containers: &[serde_json::Value]) -> HashSet<String> {
containers
.iter()
.filter_map(|c| {
c.get("Id")
.or_else(|| c.get("ID"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
})
.collect()
}
async fn cleanup_stale_podman_healthcheck_units(live_container_ids: &HashSet<String>) {
if live_container_ids.is_empty() {
return;
}
let mut units = stale_healthcheck_units_from_systemd(live_container_ids).await;
if units.is_empty() {
return;
}
units.sort();
units.dedup();
let mut cleaned = 0;
for unit in units {
let Some(container_id) = parse_podman_healthcheck_unit(&unit) else {
continue;
};
let service = format!("{}.service", unit.trim_end_matches(".timer"));
if stop_user_unit(&unit).await {
cleaned += 1;
}
let _ = stop_user_unit(&service).await;
let _ = reset_failed_user_unit(&service).await;
debug!(
"Stopped stale Podman healthcheck unit {} for removed container {}",
unit, container_id
);
}
if cleaned > 0 {
info!("Cleaned {} stale Podman healthcheck timer(s)", cleaned);
}
}
async fn stale_healthcheck_units_from_systemd(live_container_ids: &HashSet<String>) -> Vec<String> {
let mut units = Vec::new();
for args in [
[
"--user",
"list-timers",
"--all",
"--no-legend",
"--no-pager",
]
.as_slice(),
["--user", "list-units", "--all", "--no-legend", "--no-pager"].as_slice(),
] {
let output = match tokio::time::timeout(
std::time::Duration::from_secs(20),
tokio::process::Command::new("systemctl")
.args(args.iter().copied())
.output(),
)
.await
{
Ok(Ok(output)) if output.status.success() => output,
Ok(Ok(output)) => {
let stderr = String::from_utf8_lossy(&output.stderr);
debug!("systemctl {} failed: {}", args.join(" "), stderr.trim());
continue;
}
Ok(Err(e)) => {
debug!("Failed to run systemctl {}: {}", args.join(" "), e);
continue;
}
Err(_) => {
debug!("systemctl {} timed out", args.join(" "));
continue;
}
};
let stdout = String::from_utf8_lossy(&output.stdout);
units.extend(stale_healthcheck_units(&stdout, live_container_ids));
}
units
}
fn stale_healthcheck_units(output: &str, live_container_ids: &HashSet<String>) -> Vec<String> {
output
.lines()
.flat_map(|line| line.split_whitespace())
.filter_map(|token| {
let unit = token.trim_start_matches('●');
let id = parse_podman_healthcheck_unit(unit)?;
(!live_container_ids.contains(id)).then(|| unit.to_string())
})
.collect()
}
fn parse_podman_healthcheck_unit(unit: &str) -> Option<&str> {
let unit = unit
.strip_suffix(".timer")
.or_else(|| unit.strip_suffix(".service"))?;
let (container_id, _suffix) = unit.split_once('-')?;
if container_id.len() == 64 && container_id.bytes().all(|b| b.is_ascii_hexdigit()) {
Some(container_id)
} else {
None
}
}
async fn stop_user_unit(unit: &str) -> bool {
run_systemctl_user(["stop", unit]).await
}
async fn reset_failed_user_unit(unit: &str) -> bool {
run_systemctl_user(["reset-failed", unit]).await
}
async fn run_systemctl_user<const N: usize>(args: [&str; N]) -> bool {
let output = match tokio::time::timeout(
std::time::Duration::from_secs(10),
tokio::process::Command::new("systemctl")
.arg("--user")
.args(args.iter().copied())
.output(),
)
.await
{
Ok(Ok(output)) => output,
Ok(Err(e)) => {
debug!("Failed to run systemctl --user {}: {}", args.join(" "), e);
return false;
}
Err(_) => {
debug!("systemctl --user {} timed out", args.join(" "));
return false;
}
};
if output.status.success() {
true
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
debug!(
"systemctl --user {} failed: {}",
args.join(" "),
stderr.trim()
);
false
}
}
fn parse_podman_health(c: &serde_json::Value, state: &str) -> Option<String> {
c.get("Status")
.and_then(|v| v.as_str())
.and_then(parse_health_from_status)
.or_else(|| {
c.get("State")
.and_then(|v| v.get("Health"))
.and_then(|v| v.get("Status"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
})
.or_else(|| state.contains("unhealthy").then(|| "unhealthy".to_string()))
}
fn parse_health_from_status(status: &str) -> Option<String> {
let start = status.rfind('(')?;
let end = status.rfind(')')?;
(start < end).then(|| status[start + 1..end].to_string())
}
/// Try to restart a container.
async fn restart_container(name: &str) -> bool {
info!("Auto-restarting unhealthy container: {}", name);
@@ -539,6 +748,16 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
debug!("Skipping uninstalled container: {}", container.name);
continue;
}
if matches!(
pkg.state,
PackageState::Starting | PackageState::Stopping | PackageState::Restarting
) {
debug!(
"Skipping container during package lifecycle transition: {} ({:?})",
container.name, pkg.state
);
continue;
}
} else {
// Orphan: container exists in podman but archipelago has
// no package_data entry for it. Common after a variant
@@ -552,10 +771,26 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
"Skipping orphan container (not in package_data): {}",
container.name
);
let before = data.notifications.len();
data.notifications.retain(|n| {
n.app_id.as_deref() != Some(&container.app_id)
&& !n.title.contains(&container.app_id)
});
if data.notifications.len() != before {
state_changed = true;
}
continue;
}
if container.healthy {
let before = data.notifications.len();
data.notifications.retain(|n| {
n.app_id.as_deref() != Some(&container.app_id)
&& !n.title.contains(&container.app_id)
});
if data.notifications.len() != before {
state_changed = true;
}
if tracker.attempt_count(&container.name) > 0 {
info!(
"Container {} is healthy again after restart",
@@ -590,8 +825,9 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
}
continue;
}
// Handle exited, stopped, AND created state containers
if container.state == "exited"
// Handle exited, stopped, created, and Podman-unhealthy running containers.
if container.podman_health.as_deref() == Some("unhealthy")
|| container.state == "exited"
|| container.state == "stopped"
|| container.state == "created"
{
@@ -604,6 +840,21 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
}
}
let unhealthy_app_ids: HashSet<&str> = unhealthy
.iter()
.map(|container| container.app_id.as_str())
.collect();
let before = data.notifications.len();
data.notifications.retain(|n| {
!n.id.starts_with("health-")
|| n.app_id
.as_deref()
.is_some_and(|app_id| unhealthy_app_ids.contains(app_id))
});
if data.notifications.len() != before {
state_changed = true;
}
// Sort by startup tier: databases first, then core, then dependent, then apps, then UIs
unhealthy.sort_by_key(|c| container_tier(&c.name));
@@ -650,6 +901,14 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
continue;
}
if has_running_bitcoin_conflict(&container.name, &containers) {
debug!(
"Skipping auto-restart for {} because the other Bitcoin implementation is running",
container.name
);
continue;
}
// When transitioning to a higher tier, wait briefly for previous tier to stabilize
if let Some(prev) = prev_tier {
if tier > prev {
@@ -828,6 +1087,7 @@ mod tests {
name: "archy-bitcoin-knots".to_string(),
app_id: "bitcoin-knots".to_string(),
state: "running".to_string(),
podman_health: Some("healthy".to_string()),
healthy: true,
};
assert!(health.healthy);
@@ -842,6 +1102,7 @@ mod tests {
name: "archy-mempool-web".to_string(),
app_id: "mempool-web".to_string(),
state: "exited".to_string(),
podman_health: None,
healthy: false,
};
assert!(!health.healthy);
@@ -916,7 +1177,7 @@ mod tests {
#[test]
fn test_container_dependencies() {
assert!(container_dependencies("lnd").contains(&"bitcoin-knots"));
assert!(container_dependencies("lnd").contains(&"bitcoin"));
assert!(container_dependencies("indeedhub-api").contains(&"indeedhub-postgres"));
assert!(container_dependencies("indeedhub-api").contains(&"indeedhub-redis"));
assert!(container_dependencies("mempool-api").contains(&"mempool-db"));
@@ -931,18 +1192,21 @@ mod tests {
name: "indeedhub-postgres".into(),
app_id: "indeedhub-postgres".into(),
state: "running".into(),
podman_health: None,
healthy: true,
},
ContainerHealth {
name: "indeedhub-redis".into(),
app_id: "indeedhub-redis".into(),
state: "running".into(),
podman_health: None,
healthy: true,
},
ContainerHealth {
name: "indeedhub-api".into(),
app_id: "indeedhub-api".into(),
state: "exited".into(),
podman_health: None,
healthy: false,
},
];
@@ -952,11 +1216,70 @@ mod tests {
name: "indeedhub-redis".into(),
app_id: "indeedhub-redis".into(),
state: "running".into(),
podman_health: None,
healthy: true,
}];
assert!(!deps_are_running("indeedhub-api", &partial));
}
#[test]
fn test_bitcoin_dependency_accepts_core_or_knots() {
let core = vec![ContainerHealth {
name: "bitcoin-core".into(),
app_id: "bitcoin-core".into(),
state: "running".into(),
podman_health: None,
healthy: true,
}];
assert!(deps_are_running("lnd", &core));
let knots = vec![ContainerHealth {
name: "bitcoin-knots".into(),
app_id: "bitcoin-knots".into(),
state: "running".into(),
podman_health: None,
healthy: true,
}];
assert!(deps_are_running("fedimint", &knots));
let stopped = vec![ContainerHealth {
name: "bitcoin-core".into(),
app_id: "bitcoin-core".into(),
state: "stopped".into(),
podman_health: None,
healthy: false,
}];
assert!(!deps_are_running("electrumx", &stopped));
}
#[test]
fn test_bitcoin_conflict_detection() {
let containers = vec![ContainerHealth {
name: "bitcoin-core".into(),
app_id: "bitcoin-core".into(),
state: "running".into(),
podman_health: None,
healthy: true,
}];
assert!(has_running_bitcoin_conflict("bitcoin-knots", &containers));
assert!(!has_running_bitcoin_conflict("bitcoin-core", &containers));
assert!(!has_running_bitcoin_conflict("lnd", &containers));
}
#[test]
fn test_bitcoin_conflict_ignores_stopped_sibling() {
let containers = vec![ContainerHealth {
name: "bitcoin-core".into(),
app_id: "bitcoin-core".into(),
state: "stopped".into(),
podman_health: None,
healthy: false,
}];
assert!(!has_running_bitcoin_conflict("bitcoin-knots", &containers));
}
#[test]
fn test_container_tier_core() {
assert_eq!(container_tier("bitcoin-knots"), StartupTier::CoreInfra);
@@ -1022,4 +1345,58 @@ mod tests {
tracker.record("test", 100_000_000);
assert!(tracker.check_leak("test").is_none());
}
#[test]
fn podman_unhealthy_makes_running_container_unhealthy() {
let c = serde_json::json!({ "Status": "Up 5 minutes (unhealthy)" });
assert_eq!(
parse_podman_health(&c, "running").as_deref(),
Some("unhealthy")
);
}
#[test]
fn parses_podman_healthcheck_systemd_units() {
let id = "c1f44a6369c91d65f9e9f6134a5591aa02792cff2f1a4e0f689b5a6c03b6c77c";
assert_eq!(
parse_podman_healthcheck_unit(&format!("{}-15c66ddfefa8a763.timer", id)),
Some(id)
);
assert_eq!(
parse_podman_healthcheck_unit(&format!("{}-15c66ddfefa8a763.service", id)),
Some(id)
);
assert_eq!(parse_podman_healthcheck_unit("grafana.service"), None);
assert_eq!(
parse_podman_healthcheck_unit(
"nothexzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz-x.timer"
),
None
);
}
#[test]
fn stale_healthcheck_units_filters_only_removed_container_ids() {
let live = "6467e25fd87d791a63fe9dbf6e2fabc7bf26533aa2c402b1089effeacf7ebbba";
let stale = "c1f44a6369c91d65f9e9f6134a5591aa02792cff2f1a4e0f689b5a6c03b6c77c";
let mut live_ids = HashSet::new();
live_ids.insert(live.to_string());
let output = format!(
" {live}-6fdc497fd3ba3b62.timer loaded active waiting\n\
{stale}-15c66ddfefa8a763.service loaded failed failed\n\
grafana.service loaded active running\n\
{stale}-1898d85de0bb707f.timer loaded active waiting\n"
);
let mut units = stale_healthcheck_units(&output, &live_ids);
units.sort();
assert_eq!(
units,
vec![
format!("{stale}-15c66ddfefa8a763.service"),
format!("{stale}-1898d85de0bb707f.timer"),
]
);
}
}
+33 -36
View File
@@ -29,6 +29,7 @@ mod auth;
mod avatar;
mod backup;
mod bitcoin_rpc;
mod bitcoin_status;
mod blobs;
mod bootstrap;
mod config;
@@ -136,44 +137,39 @@ async fn main() -> Result<()> {
// Write PID marker early so we can detect crashes on next startup
crash_recovery::write_pid_marker(&config.data_dir).await?;
// Crash recovery runs in background so health endpoint is available immediately
{
let data_dir = config.data_dir.clone();
tokio::spawn(async move {
// Check if previous instance shut down cleanly
match crash_recovery::check_for_crash(&data_dir).await {
Ok(Some(containers)) => {
info!(
"🔧 Recovering {} containers from previous crash...",
containers.len()
);
let report = crash_recovery::recover_containers(&containers).await;
info!(
"🔧 Recovery complete: {}/{} containers restarted (failed: {:?})",
report.recovered, report.total, report.failed
);
}
Ok(None) => {}
Err(e) => {
tracing::warn!("Crash recovery check failed: {}", e);
}
}
// Start any stopped containers (handles clean reboot)
// Skips user-stopped containers, uses tier ordering
let boot_report = crash_recovery::start_stopped_containers(&data_dir).await;
if boot_report.total > 0 {
info!(
"🔄 Boot startup: {}/{} containers started (failed: {:?})",
boot_report.recovered, boot_report.total, boot_report.failed
);
}
// Signal to health monitor that boot recovery is done
crash_recovery::mark_recovery_complete();
});
// Run crash recovery before starting the manifest reconciler. Both paths
// mutate Podman; running them concurrently can corrupt transient runtime
// state and leave netavark/conmon unable to start containers.
match crash_recovery::check_for_crash(&config.data_dir).await {
Ok(Some(containers)) => {
info!(
"🔧 Recovering {} containers from previous crash...",
containers.len()
);
let report = crash_recovery::recover_containers(&containers).await;
info!(
"🔧 Recovery complete: {}/{} containers restarted (failed: {:?})",
report.recovered, report.total, report.failed
);
}
Ok(None) => {}
Err(e) => {
tracing::warn!("Crash recovery check failed: {}", e);
}
}
// Start any stopped containers (handles clean reboot). This remains
// synchronous for the same reason: no concurrent reconciler during Podman
// startup/recovery operations.
let boot_report = crash_recovery::start_stopped_containers(&config.data_dir).await;
if boot_report.total > 0 {
info!(
"🔄 Boot startup: {}/{} containers started (failed: {:?})",
boot_report.recovered, boot_report.total, boot_report.failed
);
}
crash_recovery::mark_recovery_complete();
// Construct the container orchestrator once. In prod mode we load the
// on-disk app manifests, do an initial adoption pass, and spawn the
// BootReconciler loop (Step 5/6 of the rust-orchestrator migration).
@@ -289,6 +285,7 @@ async fn main() -> Result<()> {
// Spawn ElectrumX status cache (refreshes every 15s, serves cached data to avoid race conditions)
electrs_status::spawn_status_cache();
bitcoin_status::spawn_status_cache();
let startup_ms = startup_start.elapsed().as_millis();
info!(
+113 -24
View File
@@ -1,42 +1,140 @@
//! Mesh session lifecycle: connect, initialize, main loop.
use super::super::meshtastic::MeshtasticDevice;
use super::super::serial::MeshcoreDevice;
use super::super::types::*;
use super::{
frames, MeshCommand, MeshState, ADVERT_INTERVAL, MAX_CONSECUTIVE_WRITE_FAILURES, SYNC_INTERVAL,
};
use anyhow::Result;
use anyhow::{Context, Result};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc;
use tracing::{debug, error, info, warn};
/// Scan all candidate serial ports and open the first Meshcore device found.
async fn auto_detect_and_open() -> Result<(String, MeshcoreDevice, DeviceInfo)> {
enum MeshRadioDevice {
Meshcore(MeshcoreDevice),
Meshtastic(MeshtasticDevice),
}
impl MeshRadioDevice {
fn device_type(&self) -> DeviceType {
match self {
Self::Meshcore(_) => DeviceType::Meshcore,
Self::Meshtastic(_) => DeviceType::Meshtastic,
}
}
fn advert_name(&self) -> Option<String> {
match self {
Self::Meshcore(device) => device.advert_name.clone(),
Self::Meshtastic(device) => device.advert_name(),
}
}
async fn set_advert_name(&mut self, name: &str) -> Result<()> {
match self {
Self::Meshcore(device) => device.set_advert_name(name).await,
Self::Meshtastic(device) => device.set_advert_name(name).await,
}
}
async fn send_self_advert(&mut self) -> Result<()> {
match self {
Self::Meshcore(device) => device.send_self_advert().await,
Self::Meshtastic(device) => device.send_self_advert().await,
}
}
async fn send_channel_text(&mut self, channel: u8, payload: &[u8]) -> Result<()> {
match self {
Self::Meshcore(device) => device.send_channel_text(channel, payload).await,
Self::Meshtastic(device) => device.send_channel_text(channel, payload).await,
}
}
async fn get_contacts(&mut self) -> Result<Vec<super::super::protocol::ParsedContact>> {
match self {
Self::Meshcore(device) => device.get_contacts().await,
Self::Meshtastic(device) => device.get_contacts().await,
}
}
async fn reset_contact_path(&mut self, pubkey: &[u8; 32]) -> Result<()> {
match self {
Self::Meshcore(device) => device.reset_contact_path(pubkey).await,
Self::Meshtastic(device) => device.reset_contact_path(pubkey).await,
}
}
async fn sync_messages(&mut self) -> Result<Vec<super::super::protocol::InboundFrame>> {
match self {
Self::Meshcore(device) => device.sync_messages().await,
Self::Meshtastic(device) => device.sync_messages().await,
}
}
async fn try_recv_frame(&mut self) -> Result<Option<super::super::protocol::InboundFrame>> {
match self {
Self::Meshcore(device) => device.try_recv_frame().await,
Self::Meshtastic(device) => device.try_recv_frame().await,
}
}
}
/// Scan all candidate serial ports and open the first supported mesh device found.
async fn auto_detect_and_open() -> Result<(String, MeshRadioDevice, DeviceInfo)> {
let paths = super::super::serial::detect_serial_devices().await;
if paths.is_empty() {
anyhow::bail!("No serial devices found in /dev");
}
for path in &paths {
debug!(path = %path, "Probing for Meshcore device");
debug!(path = %path, "Probing for mesh radio device");
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => {
info!(path = %path, firmware = %info.firmware_version, "Found Meshcore device via auto-detect");
return Ok((path.clone(), dev, info));
return Ok((path.clone(), MeshRadioDevice::Meshcore(dev), info));
}
Err(e) => debug!(path = %path, error = %e, "Not a Meshcore device"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open serial port"),
}
match MeshtasticDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => {
info!(path = %path, firmware = %info.firmware_version, "Found Meshtastic device via auto-detect");
return Ok((path.clone(), MeshRadioDevice::Meshtastic(dev), info));
}
Err(e) => debug!(path = %path, error = %e, "Not a Meshtastic device"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open serial port for Meshtastic"),
}
}
anyhow::bail!(
"No Meshcore device found on {} candidate ports: {:?}",
"No supported mesh radio found on {} candidate ports: {:?}",
paths.len(),
paths
)
}
async fn open_preferred_path(path: &str) -> Result<(MeshRadioDevice, DeviceInfo)> {
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => return Ok((MeshRadioDevice::Meshcore(dev), info)),
Err(e) => debug!(path = %path, error = %e, "Preferred path is not Meshcore"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open preferred path as Meshcore"),
}
match MeshtasticDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => Ok((MeshRadioDevice::Meshtastic(dev), info)),
Err(e) => Err(e).context("Preferred path is not Meshtastic"),
},
Err(e) => Err(e).context("Could not open preferred path as Meshtastic"),
}
}
/// ASCII marker for the original DM-via-channel format:
/// `@DM:` + base64(`[dest_prefix(6)][inner…]`). No sender info on the wire,
/// so the receiver has to guess the sender from its contact table — which
@@ -90,7 +188,7 @@ fn our_sender_prefix(state: &Arc<MeshState>) -> [u8; 6] {
/// with both the recipient and sender prefixes so attribution works on
/// the receiver side.
async fn send_dm_via_channel(
device: &mut MeshcoreDevice,
device: &mut MeshRadioDevice,
state: &Arc<MeshState>,
dest_pubkey_prefix: &[u8; 6],
payload: &[u8],
@@ -166,7 +264,7 @@ async fn send_dm_via_channel(
}
/// Fetch the contacts list from the device and update the peer cache.
async fn refresh_contacts(device: &mut MeshcoreDevice, state: &Arc<MeshState>) {
async fn refresh_contacts(device: &mut MeshRadioDevice, state: &Arc<MeshState>) {
match device.get_contacts().await {
Ok(contacts) => {
// Skip firmware contacts the user has explicitly wiped via
@@ -239,7 +337,7 @@ async fn refresh_contacts(device: &mut MeshcoreDevice, state: &Arc<MeshState>) {
/// Drain any queued messages from the device.
/// Returns `true` if a write/communication error occurred (for failure tracking).
async fn sync_queued_messages(
device: &mut MeshcoreDevice,
device: &mut MeshRadioDevice,
state: &Arc<MeshState>,
our_x25519_secret: &[u8; 32],
) -> bool {
@@ -274,20 +372,11 @@ pub(super) async fn run_mesh_session(
) -> Result<()> {
// Detect device — try preferred path first, fall back to auto-detect
let (device_path, mut device, device_info) = if let Some(path) = preferred_path {
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => (path.to_string(), dev, info),
Err(e) => {
warn!(
"Preferred path {} handshake failed: {} — trying auto-detect",
path, e
);
auto_detect_and_open().await?
}
},
match open_preferred_path(path).await {
Ok((dev, info)) => (path.to_string(), dev, info),
Err(e) => {
warn!(
"Preferred path {} open failed: {} — trying auto-detect",
"Preferred path {} probe failed: {} — trying auto-detect",
path, e
);
auto_detect_and_open().await?
@@ -301,11 +390,11 @@ pub(super) async fn run_mesh_session(
{
let mut status = state.status.write().await;
status.device_connected = true;
status.device_type = DeviceType::Meshcore;
status.device_type = device.device_type();
status.device_path = Some(device_path.clone());
status.firmware_version = Some(device_info.firmware_version.clone());
status.self_node_id = Some(device_info.node_id);
status.self_advert_name = device.advert_name.clone();
status.self_advert_name = device.advert_name();
}
let _ = state.event_tx.send(MeshEvent::DeviceConnected(device_info));
@@ -434,7 +523,7 @@ pub(super) async fn run_mesh_session(
/// Process a single outbound command from MeshService.
async fn handle_send_command(
cmd: MeshCommand,
device: &mut MeshcoreDevice,
device: &mut MeshRadioDevice,
state: &Arc<MeshState>,
consecutive_write_failures: &mut u32,
) {
+621
View File
@@ -0,0 +1,621 @@
//! Async serial driver for Meshtastic devices.
//!
//! Meshtastic uses protobuf payloads over a SLIP-like serial stream. This
//! module implements only the small subset Archipelago needs: connect,
//! discover the local node, send/receive text packets, and provide synthetic
//! contacts to the existing mesh listener.
use super::protocol::{InboundFrame, ParsedContact};
use super::types::{DeviceInfo, DeviceType};
use anyhow::{Context, Result};
use std::collections::HashMap;
use std::time::Duration;
use tracing::{debug, info, warn};
const BAUD_RATE: u32 = 115200;
const READ_TIMEOUT: Duration = Duration::from_secs(5);
const WRITE_TIMEOUT: Duration = Duration::from_secs(2);
const READ_BUF_SIZE: usize = 512;
const START1: u8 = 0x94;
const START2: u8 = 0xc3;
const TO_RADIO_MAX: usize = 512;
const BROADCAST_NUM: u32 = 0xffff_ffff;
const TEXT_MESSAGE_APP: u32 = 1;
const TO_RADIO_PACKET: u64 = 1;
const TO_RADIO_WANT_CONFIG_ID: u64 = 3;
const TO_RADIO_HEARTBEAT: u64 = 7;
const FROM_RADIO_PACKET: u64 = 2;
const FROM_RADIO_MY_INFO: u64 = 3;
const FROM_RADIO_NODE_INFO: u64 = 4;
const FROM_RADIO_CONFIG_COMPLETE_ID: u64 = 7;
const FROM_RADIO_REBOOTED: u64 = 8;
/// Async Meshtastic device handle.
pub struct MeshtasticDevice {
port: serial2_tokio::SerialPort,
read_buf: Vec<u8>,
node_num: Option<u32>,
user_id: Option<String>,
long_name: Option<String>,
short_name: Option<String>,
contacts: HashMap<u32, ParsedContact>,
device_path: String,
}
impl MeshtasticDevice {
pub async fn open(path: &str) -> Result<Self> {
match tokio::fs::metadata(path).await {
Ok(meta) => {
debug!(path = %path, permissions = ?meta.permissions(), "Device node exists")
}
Err(e) => anyhow::bail!("Serial device {} not accessible: {}", path, e),
}
let port = serial2_tokio::SerialPort::open(path, BAUD_RATE).context(format!(
"Failed to open serial port {} (permission denied? device busy?)",
path
))?;
info!(path = %path, baud = BAUD_RATE, "Opened Meshtastic serial port");
Ok(Self {
port,
read_buf: Vec::with_capacity(READ_BUF_SIZE),
node_num: None,
user_id: None,
long_name: None,
short_name: None,
contacts: HashMap::new(),
device_path: path.to_string(),
})
}
pub async fn initialize(&mut self) -> Result<DeviceInfo> {
info!(path = %self.device_path, "Starting Meshtastic handshake");
self.send_to_radio(&encode_want_config()).await?;
let deadline = tokio::time::Instant::now() + READ_TIMEOUT;
let mut saw_meshtastic_frame = false;
let mut saw_config_complete = false;
loop {
let remaining = deadline.saturating_duration_since(tokio::time::Instant::now());
if remaining.is_zero() {
break;
}
match tokio::time::timeout(
remaining.min(Duration::from_millis(250)),
self.read_from_radio(),
)
.await
{
Ok(Ok(Some(frame))) => {
saw_meshtastic_frame = true;
if matches!(
decode_top_level_variant(&frame),
Some((FROM_RADIO_CONFIG_COMPLETE_ID, _))
) {
saw_config_complete = true;
}
self.handle_from_radio(&frame);
if saw_config_complete && self.node_num.is_some() {
break;
}
}
Ok(Ok(None)) | Err(_) => {}
Ok(Err(e)) => return Err(e),
}
}
if !saw_meshtastic_frame {
anyhow::bail!("No Meshtastic serial API response");
}
let node_id = self
.node_num
.ok_or_else(|| anyhow::anyhow!("Meshtastic serial API did not provide MyInfo"))?;
if self.user_id.is_none() && self.long_name.is_none() && self.short_name.is_none() {
anyhow::bail!("Meshtastic serial API did not provide node identity");
}
let firmware_version = self
.long_name
.clone()
.or_else(|| self.user_id.clone())
.unwrap_or_else(|| "Meshtastic".to_string());
info!(node_id, name = %firmware_version, "Meshtastic identity");
Ok(DeviceInfo {
firmware_version,
node_id,
max_contacts: 200,
device_type: DeviceType::Meshtastic,
})
}
pub async fn set_advert_name(&mut self, name: &str) -> Result<()> {
self.long_name = Some(name.to_string());
Ok(())
}
pub async fn send_self_advert(&mut self) -> Result<()> {
self.send_to_radio(&encode_heartbeat()).await
}
pub async fn send_channel_text(&mut self, _channel: u8, msg: &[u8]) -> Result<()> {
let text = String::from_utf8_lossy(msg);
let packet = encode_mesh_packet(BROADCAST_NUM, TEXT_MESSAGE_APP, text.as_bytes());
self.send_to_radio(&encode_to_radio_variant(TO_RADIO_PACKET, &packet))
.await
}
pub async fn get_contacts(&mut self) -> Result<Vec<ParsedContact>> {
if self.contacts.is_empty() {
self.send_to_radio(&encode_want_config()).await?;
let deadline = tokio::time::Instant::now() + Duration::from_secs(2);
while tokio::time::Instant::now() < deadline {
match self.read_from_radio().await? {
Some(frame) => {
let config_complete = matches!(
decode_top_level_variant(&frame),
Some((FROM_RADIO_CONFIG_COMPLETE_ID, _))
);
self.handle_from_radio(&frame);
if config_complete || !self.contacts.is_empty() {
break;
}
}
None => tokio::time::sleep(Duration::from_millis(50)).await,
}
}
}
Ok(self.contacts.values().cloned().collect())
}
pub async fn reset_contact_path(&mut self, _pubkey: &[u8; 32]) -> Result<()> {
Ok(())
}
pub async fn sync_messages(&mut self) -> Result<Vec<InboundFrame>> {
Ok(Vec::new())
}
pub async fn try_recv_frame(&mut self) -> Result<Option<InboundFrame>> {
let Some(frame) = self.read_from_radio().await? else {
return Ok(None);
};
Ok(self.handle_from_radio(&frame))
}
pub fn advert_name(&self) -> Option<String> {
self.long_name
.clone()
.or_else(|| self.short_name.clone())
.or_else(|| self.user_id.clone())
}
async fn send_to_radio(&mut self, payload: &[u8]) -> Result<()> {
if payload.len() > TO_RADIO_MAX {
anyhow::bail!("Meshtastic payload too large: {} bytes", payload.len());
}
let mut frame = Vec::with_capacity(4 + payload.len());
frame.push(START1);
frame.push(START2);
frame.extend_from_slice(&(payload.len() as u16).to_be_bytes());
frame.extend_from_slice(payload);
tokio::time::timeout(WRITE_TIMEOUT, self.port.write_all(&frame))
.await
.context("Meshtastic serial write timed out")?
.context("Meshtastic serial write failed")?;
Ok(())
}
async fn read_from_radio(&mut self) -> Result<Option<Vec<u8>>> {
if let Some(frame) = decode_serial_frame(&mut self.read_buf) {
return Ok(Some(frame));
}
let mut tmp = [0u8; READ_BUF_SIZE];
match tokio::time::timeout(Duration::from_millis(50), self.port.read(&mut tmp)).await {
Ok(Ok(0)) => anyhow::bail!("Meshtastic serial port closed"),
Ok(Ok(n)) => self.read_buf.extend_from_slice(&tmp[..n]),
Ok(Err(e)) => return Err(e).context("Meshtastic serial read error"),
Err(_) => return Ok(None),
}
Ok(decode_serial_frame(&mut self.read_buf))
}
fn handle_from_radio(&mut self, frame: &[u8]) -> Option<InboundFrame> {
let Some((field, value)) = decode_top_level_variant(frame) else {
return None;
};
match field {
FROM_RADIO_MY_INFO => {
if let Some((node_num, user_id)) = parse_my_info(value) {
self.node_num = Some(node_num);
if let Some(user_id) = user_id {
self.user_id = Some(user_id);
}
}
None
}
FROM_RADIO_NODE_INFO => {
self.update_node_info(value);
None
}
FROM_RADIO_PACKET => self.packet_to_inbound_frame(value),
FROM_RADIO_CONFIG_COMPLETE_ID | FROM_RADIO_REBOOTED => None,
other => {
debug!(
field = other,
len = value.len(),
"Unhandled Meshtastic FromRadio field"
);
None
}
}
}
fn update_node_info(&mut self, data: &[u8]) {
if let Some(node) = parse_node_info(data) {
let key = synthetic_pubkey(node.num);
let name = node
.long_name
.or(node.short_name)
.or(node.id)
.unwrap_or_else(|| format!("Meshtastic !{:08x}", node.num));
if Some(node.num) == self.node_num {
self.long_name = Some(name.clone());
}
self.contacts.insert(
node.num,
ParsedContact {
public_key_hex: hex::encode(key),
advert_name: name,
last_advert: node.last_heard.unwrap_or_default(),
contact_type: 1,
path_len: 0xff,
flags: 0,
},
);
}
}
fn packet_to_inbound_frame(&mut self, data: &[u8]) -> Option<InboundFrame> {
let packet = parse_mesh_packet(data)?;
if packet.portnum != TEXT_MESSAGE_APP || packet.payload.is_empty() {
return None;
}
let from = packet.from.unwrap_or(0);
if Some(from) == self.node_num {
return None;
}
let from_key = synthetic_pubkey(from);
self.contacts.entry(from).or_insert_with(|| ParsedContact {
public_key_hex: hex::encode(synthetic_pubkey(from)),
advert_name: format!("Meshtastic !{:08x}", from),
last_advert: 0,
contact_type: 1,
path_len: 0xff,
flags: 0,
});
let mut payload = Vec::with_capacity(15 + packet.payload.len());
payload.push(0); // SNR unknown
payload.extend_from_slice(&[0, 0]); // reserved
payload.extend_from_slice(&from_key[..6]);
payload.push(0xff); // unknown/flood path
payload.push(0); // text type
payload.extend_from_slice(&0u32.to_le_bytes());
payload.extend_from_slice(&packet.payload);
Some(InboundFrame {
code: super::protocol::RESP_CONTACT_MSG_V3,
data: payload,
bytes_consumed: 0,
})
}
}
fn decode_serial_frame(buf: &mut Vec<u8>) -> Option<Vec<u8>> {
let start = buf.windows(2).position(|w| w == [START1, START2])?;
if start > 0 {
buf.drain(..start);
}
if buf.len() < 4 {
return None;
}
let len = u16::from_be_bytes([buf[2], buf[3]]) as usize;
if buf.len() < 4 + len {
return None;
}
let payload = buf[4..4 + len].to_vec();
buf.drain(..4 + len);
Some(payload)
}
fn encode_want_config() -> Vec<u8> {
encode_varint_field(TO_RADIO_WANT_CONFIG_ID, 1)
}
fn encode_heartbeat() -> Vec<u8> {
encode_to_radio_variant(TO_RADIO_HEARTBEAT, &[])
}
fn encode_to_radio_variant(field: u64, bytes: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
encode_len_field(field, bytes, &mut out);
out
}
fn encode_mesh_packet(to: u32, portnum: u32, payload: &[u8]) -> Vec<u8> {
let mut decoded = Vec::new();
encode_varint_field_into(1, portnum as u64, &mut decoded);
encode_len_field(2, payload, &mut decoded);
let mut packet = Vec::new();
encode_fixed32_field(2, to, &mut packet);
encode_len_field(4, &decoded, &mut packet);
packet
}
fn decode_top_level_variant(buf: &[u8]) -> Option<(u64, &[u8])> {
let mut idx = 0;
while idx < buf.len() {
let (key, n) = read_varint(&buf[idx..])?;
idx += n;
let field = key >> 3;
match key & 0x07 {
0 => {
let (_, n) = read_varint(&buf[idx..])?;
idx += n;
if matches!(field, FROM_RADIO_CONFIG_COMPLETE_ID | FROM_RADIO_REBOOTED) {
return Some((field, &[]));
}
}
2 => {
let (len, n) = read_varint(&buf[idx..])?;
idx += n;
let end = idx.checked_add(len as usize)?;
if end > buf.len() {
return None;
}
if matches!(
field,
FROM_RADIO_PACKET | FROM_RADIO_MY_INFO | FROM_RADIO_NODE_INFO
) {
return Some((field, &buf[idx..end]));
}
idx = end;
}
_ => return None,
}
}
None
}
fn parse_my_info(data: &[u8]) -> Option<(u32, Option<String>)> {
let mut idx = 0;
let mut node_num = None;
let mut user_id = None;
while idx < data.len() {
let (field, value, next) = next_field(data, idx)?;
idx = next;
match (field, value) {
(1, FieldValue::Varint(v)) => node_num = Some(v as u32),
(1, FieldValue::Fixed32(v)) => node_num = Some(v),
(3, FieldValue::Bytes(b)) => user_id = parse_user(b).and_then(|u| u.id),
_ => {}
}
}
node_num.map(|n| (n, user_id))
}
struct ParsedNode {
num: u32,
id: Option<String>,
long_name: Option<String>,
short_name: Option<String>,
last_heard: Option<u32>,
}
fn parse_node_info(data: &[u8]) -> Option<ParsedNode> {
let mut idx = 0;
let mut node = ParsedNode {
num: 0,
id: None,
long_name: None,
short_name: None,
last_heard: None,
};
while idx < data.len() {
let (field, value, next) = next_field(data, idx)?;
idx = next;
match (field, value) {
(1, FieldValue::Varint(v)) => node.num = v as u32,
(1, FieldValue::Fixed32(v)) => node.num = v,
(2, FieldValue::Bytes(b)) => {
if let Some(user) = parse_user(b) {
node.id = user.id;
node.long_name = user.long_name;
node.short_name = user.short_name;
}
}
(5, FieldValue::Fixed32(v)) => node.last_heard = Some(v),
_ => {}
}
}
if node.num == 0 {
None
} else {
Some(node)
}
}
struct ParsedUser {
id: Option<String>,
long_name: Option<String>,
short_name: Option<String>,
}
fn parse_user(data: &[u8]) -> Option<ParsedUser> {
let mut idx = 0;
let mut user = ParsedUser {
id: None,
long_name: None,
short_name: None,
};
while idx < data.len() {
let (field, value, next) = next_field(data, idx)?;
idx = next;
match (field, value) {
(1, FieldValue::Bytes(b)) => user.id = string_field(b),
(2, FieldValue::Bytes(b)) => user.long_name = string_field(b),
(3, FieldValue::Bytes(b)) => user.short_name = string_field(b),
_ => {}
}
}
Some(user)
}
struct ParsedPacket {
from: Option<u32>,
portnum: u32,
payload: Vec<u8>,
}
fn parse_mesh_packet(data: &[u8]) -> Option<ParsedPacket> {
let mut idx = 0;
let mut from = None;
let mut decoded = None;
while idx < data.len() {
let (field, value, next) = next_field(data, idx)?;
idx = next;
match (field, value) {
(1, FieldValue::Fixed32(v)) => from = Some(v),
(4, FieldValue::Bytes(b)) => decoded = Some(b),
_ => {}
}
}
let decoded = decoded?;
let mut didx = 0;
let mut portnum = 0;
let mut payload = Vec::new();
while didx < decoded.len() {
let (field, value, next) = next_field(decoded, didx)?;
didx = next;
match (field, value) {
(1, FieldValue::Varint(v)) => portnum = v as u32,
(2, FieldValue::Bytes(b)) => payload = b.to_vec(),
_ => {}
}
}
Some(ParsedPacket {
from,
portnum,
payload,
})
}
enum FieldValue<'a> {
Varint(u64),
Fixed32(u32),
Bytes(&'a [u8]),
}
fn next_field(buf: &[u8], idx: usize) -> Option<(u64, FieldValue<'_>, usize)> {
let (key, n) = read_varint(&buf[idx..])?;
let field = key >> 3;
let mut pos = idx + n;
match key & 0x07 {
0 => {
let (v, n) = read_varint(&buf[pos..])?;
pos += n;
Some((field, FieldValue::Varint(v), pos))
}
2 => {
let (len, n) = read_varint(&buf[pos..])?;
pos += n;
let end = pos.checked_add(len as usize)?;
if end > buf.len() {
return None;
}
Some((field, FieldValue::Bytes(&buf[pos..end]), end))
}
5 => {
let end = pos.checked_add(4)?;
if end > buf.len() {
None
} else {
let value =
u32::from_le_bytes([buf[pos], buf[pos + 1], buf[pos + 2], buf[pos + 3]]);
Some((field, FieldValue::Fixed32(value), end))
}
}
1 => {
let end = pos.checked_add(8)?;
if end > buf.len() {
None
} else {
Some((field, FieldValue::Bytes(&buf[pos..end]), end))
}
}
wire => {
warn!(wire, "Unsupported Meshtastic protobuf wire type");
None
}
}
}
fn read_varint(buf: &[u8]) -> Option<(u64, usize)> {
let mut out = 0u64;
for (i, b) in buf.iter().copied().enumerate().take(10) {
out |= ((b & 0x7f) as u64) << (7 * i);
if b & 0x80 == 0 {
return Some((out, i + 1));
}
}
None
}
fn encode_varint_field(field: u64, value: u64) -> Vec<u8> {
let mut out = Vec::new();
encode_varint_field_into(field, value, &mut out);
out
}
fn encode_varint_field_into(field: u64, value: u64, out: &mut Vec<u8>) {
write_varint((field << 3) | 0, out);
write_varint(value, out);
}
fn encode_len_field(field: u64, bytes: &[u8], out: &mut Vec<u8>) {
write_varint((field << 3) | 2, out);
write_varint(bytes.len() as u64, out);
out.extend_from_slice(bytes);
}
fn encode_fixed32_field(field: u64, value: u32, out: &mut Vec<u8>) {
write_varint((field << 3) | 5, out);
out.extend_from_slice(&value.to_le_bytes());
}
fn write_varint(mut value: u64, out: &mut Vec<u8>) {
while value >= 0x80 {
out.push((value as u8 & 0x7f) | 0x80);
value >>= 7;
}
out.push(value as u8);
}
fn string_field(bytes: &[u8]) -> Option<String> {
std::str::from_utf8(bytes).ok().map(|s| s.to_string())
}
fn synthetic_pubkey(node_num: u32) -> [u8; 32] {
let mut out = [0u8; 32];
out[..4].copy_from_slice(&node_num.to_le_bytes());
out[4..15].copy_from_slice(b"meshtastic:");
out
}
+5 -3
View File
@@ -1,13 +1,15 @@
//! Mesh networking: Meshcore LoRa radio integration for offline peer discovery
//! Mesh networking: LoRa radio integration for offline peer discovery
//! and encrypted messaging between Archipelago nodes.
//!
//! Supports Meshcore firmware on Heltec V3, T-Beam, RAK WisBlock, Station G2,
//! and other ESP32/nRF52-based LoRa boards via USB serial (Companion USB mode).
//! Supports Meshcore firmware via Companion USB and Meshtastic firmware via
//! the Meshtastic serial API on Heltec, T-Beam, RAK WisBlock, Station G2,
//! and other ESP32/nRF52-based LoRa boards.
pub mod alerts;
pub mod bitcoin_relay;
pub mod crypto;
pub mod listener;
pub mod meshtastic;
pub mod message_types;
pub mod outbox;
pub mod protocol;
+1
View File
@@ -323,6 +323,7 @@ pub fn parse_self_info(data: &[u8]) -> Result<(u32, String)> {
}
/// Parsed contact from RESP_CONTACT (0x03).
#[derive(Clone)]
pub struct ParsedContact {
pub public_key_hex: String,
pub advert_name: String,
+32
View File
@@ -8,6 +8,7 @@
use super::protocol::{self, InboundFrame};
use super::types::DeviceInfo;
use anyhow::{Context, Result};
use std::path::Path;
use std::time::Duration;
use tracing::{debug, info, warn};
@@ -400,12 +401,43 @@ const SERIAL_CANDIDATES: &[&str] = &[
"/dev/ttyACM2",
];
const SKIP_SERIAL_MODEL_SUBSTRINGS: &[&str] = &["Sierra_Wireless", "Z-Wave", "Zooz"];
fn likely_non_mesh_serial_device(path: &str) -> bool {
let Some(name) = Path::new(path).file_name().and_then(|s| s.to_str()) else {
return false;
};
let by_id = Path::new("/dev/serial/by-id");
let Ok(entries) = std::fs::read_dir(by_id) else {
return false;
};
for entry in entries.flatten() {
let file_name = entry.file_name().to_string_lossy().to_string();
if !SKIP_SERIAL_MODEL_SUBSTRINGS
.iter()
.any(|needle| file_name.contains(needle))
{
continue;
}
if let Ok(target) = std::fs::read_link(entry.path()) {
if target.file_name().and_then(|s| s.to_str()) == Some(name) {
return true;
}
}
}
false
}
/// Scan for serial devices that could be Meshcore radios.
/// Returns paths to existing serial device files.
pub async fn detect_serial_devices() -> Vec<String> {
let mut devices = Vec::new();
for path in SERIAL_CANDIDATES {
if tokio::fs::metadata(path).await.is_ok() {
if likely_non_mesh_serial_device(path) {
debug!(path = %path, "Skipping known non-mesh serial device");
continue;
}
devices.push(path.to_string());
}
}
+8 -4
View File
@@ -13,8 +13,8 @@ use std::path::Path;
const RESERVED_PORTS: &[u16] = &[
80, 443, 81, // HTTP/HTTPS
8332, 8333, 8334, // Bitcoin RPC/P2P
9735, 10009, 8080, // LND P2P, gRPC, REST
8081, // LND UI (archy-lnd-ui)
9735, 10009, 8080, // LND P2P, gRPC, REST
18083, // LND UI (archy-lnd-ui)
4080, 8999, 50001, // Mempool stack
23000, // BTCPay
8173, 8174, 8175, // Fedimint
@@ -27,7 +27,7 @@ const RESERVED_PORTS: &[u16] = &[
3001, 3002, // Gitea, Uptime Kuma
8888, // SearXNG
8096, 2342, 2283, // Jellyfin, Photoprism, Immich
8443, 8084, // NPM
8443, // FIPS TCP fallback
];
/// Start of range for allocating web app ports when preferred is taken.
@@ -93,8 +93,12 @@ impl PortAllocator {
.any(|m| m.host_port == port)
}
fn can_bind(port: u16) -> bool {
std::net::TcpListener::bind(("0.0.0.0", port)).is_ok()
}
fn is_available(&self, port: u16) -> bool {
!self.is_reserved(port) && !self.is_allocated(port)
!self.is_reserved(port) && !self.is_allocated(port) && Self::can_bind(port)
}
/// Allocate a host port for an app. Uses preferred_port if available, else finds next free.
+4 -1
View File
@@ -80,7 +80,10 @@ impl EndpointRateLimiter {
limits.insert("backup.upload-s3".to_string(), (3, 600));
limits.insert("backup.download-s3".to_string(), (3, 600));
// System operations
limits.insert("update.apply".to_string(), (2, 600));
// Update apply is an authenticated local admin action. Keep a guard
// against accidental button storms without locking operators out for
// ten minutes during OTA troubleshooting.
limits.insert("update.apply".to_string(), (10, 60));
limits.insert("system.reboot".to_string(), (2, 300));
limits.insert("system.shutdown".to_string(), (2, 300));
// Password and TOTP changes
+83 -5
View File
@@ -313,6 +313,7 @@ impl Server {
let scanner = create_docker_scanner(&config).await?;
let state = state_manager.clone();
let identity_clone = identity.clone();
let data_dir = config.data_dir.clone();
let scan_kick = api_handler.rpc_handler().scan_kick();
let scan_tick = api_handler.rpc_handler().scan_tick();
@@ -334,6 +335,7 @@ impl Server {
&scanner,
&state,
identity_clone.as_ref(),
&data_dir,
&mut absence_tracker,
&mut transitional_since,
)
@@ -371,6 +373,7 @@ impl Server {
&scanner,
&state,
identity_clone.as_ref(),
&data_dir,
&mut absence_tracker,
&mut transitional_since,
)
@@ -835,7 +838,21 @@ const CONTAINER_ABSENCE_THRESHOLD: u32 = 3;
/// 600s. 2× that gives the spawned task ample margin before we assume it
/// died (panic, OOM, process restart mid-stop) and fall back to the
/// scanner's authoritative view. Applies to all transitional variants.
const TRANSITIONAL_STUCK_TIMEOUT: Duration = Duration::from_secs(1200);
const TRANSITIONAL_STUCK_TIMEOUT: Duration = Duration::from_secs(120);
/// Multi-container installs can legitimately spend several minutes before the
/// primary user-facing container exists. BTCPay, for example, pulls/starts
/// Postgres and NBXplorer before `btcpay-server`; do not erase its installing
/// card just because the primary container is absent during that setup window.
const INSTALLING_STUCK_TIMEOUT: Duration = Duration::from_secs(20 * 60);
fn transitional_stuck_timeout(state: &crate::data_model::PackageState) -> Duration {
if *state == crate::data_model::PackageState::Installing {
INSTALLING_STUCK_TIMEOUT
} else {
TRANSITIONAL_STUCK_TIMEOUT
}
}
/// Returns true if `state` is one of the transitional variants that a
/// `spawn_transitional`-style background task owns. While such a state is
@@ -865,8 +882,19 @@ fn merge_preserving_transitional(
existing: &crate::data_model::PackageDataEntry,
fresh: &crate::data_model::PackageDataEntry,
) -> crate::data_model::PackageDataEntry {
let state = match (&existing.state, &fresh.state) {
// Removing with a live running container is stale: uninstall either
// failed or Archipelago restarted before the spawned task could revert
// state. Let the scanner recover the UI immediately instead of
// keeping the app wedged in Removing for 20 minutes.
(crate::data_model::PackageState::Removing, crate::data_model::PackageState::Running) => {
fresh.state.clone()
}
_ => existing.state.clone(),
};
crate::data_model::PackageDataEntry {
state: existing.state.clone(),
state,
// install_progress and uninstall_stage are also owned by the
// initiating op (same reason as state) — keep them.
install_progress: existing.install_progress.clone(),
@@ -885,10 +913,18 @@ async fn scan_and_update_packages(
scanner: &DockerPackageScanner,
state: &StateManager,
identity: &NodeIdentity,
data_dir: &std::path::Path,
absence_tracker: &mut HashMap<String, u32>,
transitional_since: &mut HashMap<String, Instant>,
) -> Result<()> {
let packages = scanner.scan_containers().await?;
let mut packages = scanner.scan_containers().await?;
let user_stopped = crate::crash_recovery::load_user_stopped(data_dir).await;
for (id, pkg) in packages.iter_mut() {
if pkg.state == crate::data_model::PackageState::Exited && user_stopped.contains(id) {
pkg.state = crate::data_model::PackageState::Stopped;
pkg.exit_code = None;
}
}
let (current_data, _) = state.get_snapshot().await;
let tor_addr = docker_packages::read_tor_address("archipelago").await;
@@ -939,13 +975,14 @@ async fn scan_and_update_packages(
let overwrite = match existing {
Some(existing_entry) if is_transitional(&existing_entry.state) => {
let entered = *transitional_since.entry(id.clone()).or_insert(now);
let stuck = now.duration_since(entered) > TRANSITIONAL_STUCK_TIMEOUT;
let timeout = transitional_stuck_timeout(&existing_entry.state);
let stuck = now.duration_since(entered) > timeout;
if stuck {
warn!(
"Container {} stuck in {:?} for >{}s; overriding with scan state {:?}",
id,
existing_entry.state,
TRANSITIONAL_STUCK_TIMEOUT.as_secs(),
timeout.as_secs(),
pkg.state
);
transitional_since.remove(id);
@@ -992,6 +1029,29 @@ async fn scan_and_update_packages(
// owner (spawn_task) is responsible for clearing state, not us.
if let Some(entry) = merged.get(&id) {
if is_transitional(&entry.state) {
let entered = *transitional_since.entry(id.clone()).or_insert(now);
let timeout = transitional_stuck_timeout(&entry.state);
if now.duration_since(entered) > timeout {
warn!(
"Container {} stuck in {:?} and absent for >{}s; removing stale transitional state",
id,
entry.state,
timeout.as_secs()
);
merged.remove(&id);
transitional_since.remove(&id);
changed = true;
}
absence_tracker.remove(&id);
continue;
}
// Quadlet-generated units run containers with `--rm`, so a
// clean user stop removes the Podman record. Keep the package
// visible as Stopped while the user-stopped marker exists so
// package.start can recreate it via systemd/Quadlet.
if entry.state == crate::data_model::PackageState::Stopped
&& user_stopped.contains(&id)
{
absence_tracker.remove(&id);
continue;
}
@@ -1170,6 +1230,15 @@ mod merge_tests {
assert_eq!(merged.exit_code, Some(0));
}
#[test]
fn stale_removing_recovers_when_container_is_running() {
let existing = make_entry(PackageState::Removing, Some("unknown"));
let fresh = make_entry(PackageState::Running, Some("healthy"));
let merged = merge_preserving_transitional(&existing, &fresh);
assert_eq!(merged.state, PackageState::Running);
assert_eq!(merged.health.as_deref(), Some("healthy"));
}
#[test]
fn is_transitional_covers_all_variants() {
for s in [
@@ -1194,4 +1263,13 @@ mod merge_tests {
assert!(!is_transitional(&s), "{:?} should NOT be transitional", s);
}
}
#[test]
fn installing_uses_longer_stale_timeout_than_other_transitions() {
assert!(transitional_stuck_timeout(&PackageState::Installing) > TRANSITIONAL_STUCK_TIMEOUT);
assert_eq!(
transitional_stuck_timeout(&PackageState::Stopping),
TRANSITIONAL_STUCK_TIMEOUT
);
}
}
+154 -9
View File
@@ -163,12 +163,40 @@ pub async fn load_mirrors(data_dir: &Path) -> Result<Vec<UpdateMirror>> {
changed = true;
}
}
let before_order: Vec<String> = list.iter().map(|m| m.url.clone()).collect();
force_ovh_update_primary(&mut list);
changed = changed || before_order != list.iter().map(|m| m.url.clone()).collect::<Vec<_>>();
if changed {
let _ = save_mirrors(data_dir, &list).await;
}
Ok(list)
}
fn force_ovh_update_primary(list: &mut Vec<UpdateMirror>) {
let defaults = default_mirrors();
for def in &defaults {
if !list.iter().any(|m| m.url == def.url) {
list.push(def.clone());
}
}
for mirror in list.iter_mut() {
if mirror.url == DEFAULT_UPDATE_MANIFEST_URL {
mirror.label = "Server 1 (OVH)".to_string();
} else if mirror.url == DEFAULT_SECONDARY_MIRROR_URL {
mirror.label = "Server 2 (tx1138)".to_string();
}
}
list.sort_by_key(|m| {
if m.url == DEFAULT_UPDATE_MANIFEST_URL {
0
} else if m.url == DEFAULT_SECONDARY_MIRROR_URL {
1
} else {
2
}
});
}
pub async fn save_mirrors(data_dir: &Path, mirrors: &[UpdateMirror]) -> Result<()> {
fs::create_dir_all(data_dir)
.await
@@ -571,15 +599,10 @@ pub async fn check_for_updates(data_dir: &Path) -> Result<UpdateState> {
mirror = %manifest_url,
"No newer version in manifest"
);
if state.available_update.is_some() {
// A later mirror might still have a
// newer version — don't clobber what an
// earlier mirror told us. But also don't
// break: another mirror could be ahead.
continue 'mirrors;
}
state.manifest_mirror = None;
state.available_update = None;
handled = true;
continue 'mirrors;
}
handled = true;
break 'mirrors;
@@ -690,11 +713,14 @@ pub async fn dismiss_update(data_dir: &Path) -> Result<()> {
/// verified over the complete file at the end of each component, so a
/// partially-corrupt resume still fails cleanly.
pub async fn download_update(data_dir: &Path) -> Result<DownloadProgress> {
let state = load_state(data_dir).await?;
let mut state = load_state(data_dir).await?;
if state.available_update.is_none() {
state = check_for_updates(data_dir).await?;
}
let manifest = state
.available_update
.as_ref()
.ok_or_else(|| anyhow::anyhow!("No update available to download"))?;
.ok_or_else(|| anyhow::anyhow!("No update is available to download"))?;
let staging_dir = data_dir.join("update-staging");
fs::create_dir_all(&staging_dir)
@@ -1173,6 +1199,125 @@ pub async fn apply_update(data_dir: &Path) -> Result<()> {
}
info!(name = %name, "Frontend archive extracted to /opt/archipelago/web-ui");
}
_ if name.contains("runtime") && name.ends_with(".tar.gz") => {
let ts = chrono::Utc::now().timestamp_millis();
let staging_new = format!("/opt/archipelago/runtime.new.{}", ts);
let archive = src.to_string_lossy().to_string();
let mk = host_sudo(&["mkdir", "-p", &staging_new])
.await
.context("Failed to create runtime staging dir")?;
if !mk.success() {
anyhow::bail!("mkdir {} failed", staging_new);
}
let extract = host_sudo(&["tar", "-xzf", &archive, "-C", &staging_new])
.await
.with_context(|| format!("Failed to extract {}", name))?;
if !extract.success() {
let _ = host_sudo(&["rm", "-rf", &staging_new]).await;
anyhow::bail!("tar extraction failed for {}", name);
}
let runtime_paths = [
("apps", "apps"),
("scripts", "scripts"),
("docker", "docker"),
(
"image-recipe/configs/archipelago-doctor.service",
"archipelago-doctor.service",
),
(
"image-recipe/configs/archipelago-doctor.timer",
"archipelago-doctor.timer",
),
];
for (relative, label) in runtime_paths {
let staged_path = format!("{}/{}", staging_new, relative);
if !Path::new(&staged_path).exists() {
tracing::debug!(path = %relative, "Runtime artifact path absent, skipping");
continue;
}
match label {
"apps" | "scripts" | "docker" => {
let dest = format!("/opt/archipelago/{}", label);
let tmp_dest =
format!("{}.new.{}", dest, chrono::Utc::now().timestamp_millis());
let _ = host_sudo(&["mkdir", "-p", &tmp_dest]).await;
let staged_dot = format!("{}/.", staged_path);
let copy = host_sudo(&["cp", "-a", &staged_dot, &tmp_dest])
.await
.with_context(|| format!("Failed to copy runtime {}", label))?;
if !copy.success() {
let _ = host_sudo(&["rm", "-rf", &tmp_dest]).await;
anyhow::bail!("runtime copy failed for {}", label);
}
let _ = host_sudo(&["mkdir", "-p", &dest]).await;
let clean = host_sudo(&[
"find",
&dest,
"-mindepth",
"1",
"-maxdepth",
"1",
"-exec",
"rm",
"-rf",
"{}",
"+",
])
.await
.with_context(|| format!("Failed to clean runtime {}", label))?;
if !clean.success() {
let _ = host_sudo(&["rm", "-rf", &tmp_dest]).await;
anyhow::bail!("runtime clean failed for {}", label);
}
let tmp_dot = format!("{}/.", tmp_dest);
let promote = host_sudo(&["cp", "-a", &tmp_dot, &dest])
.await
.with_context(|| format!("Failed to promote runtime {}", label))?;
let _ = host_sudo(&["rm", "-rf", &tmp_dest]).await;
if !promote.success() {
anyhow::bail!("runtime promote failed for {}", label);
}
if label == "scripts" {
let _ = host_sudo(&[
"find", &dest, "-type", "f", "-name", "*.sh", "-exec", "chmod",
"755", "{}", "+",
])
.await;
}
}
"archipelago-doctor.service" | "archipelago-doctor.timer" => {
let dest = format!("/etc/systemd/system/{}", label);
let install = host_sudo(&["install", "-m", "644", &staged_path, &dest])
.await
.with_context(|| format!("Failed to install {}", label))?;
if !install.success() {
anyhow::bail!("runtime unit install failed for {}", label);
}
}
_ => {}
}
}
if Path::new(&format!("{}/scripts/image-versions.sh", staging_new)).exists() {
let _ = host_sudo(&[
"cp",
&format!("{}/scripts/image-versions.sh", staging_new),
"/opt/archipelago/image-versions.sh",
])
.await;
}
let _ = host_sudo(&["systemctl", "daemon-reload"]).await;
let _ =
host_sudo(&["systemctl", "enable", "--now", "archipelago-doctor.timer"]).await;
let _ = host_sudo(&["rm", "-rf", &staging_new]).await;
info!(name = %name, "Runtime assets applied to /opt/archipelago");
}
_ => {
debug!(name = %name, "Unknown component, skipping");
}
-268
View File
@@ -1,268 +0,0 @@
use crate::manifest::{AppManifest, Dependency};
use indexmap::IndexMap;
use std::collections::{HashMap, HashSet};
use thiserror::Error;
#[derive(Debug, Error)]
pub enum DependencyError {
#[error("Circular dependency detected: {0}")]
CircularDependency(String),
#[error("Missing dependency: {0}")]
MissingDependency(String),
#[error("Version conflict: {0}")]
VersionConflict(String),
}
pub struct DependencyResolver {
manifests: IndexMap<String, AppManifest>,
}
impl DependencyResolver {
pub fn new() -> Self {
Self {
manifests: IndexMap::new(),
}
}
pub fn add_manifest(&mut self, manifest: AppManifest) {
self.manifests.insert(manifest.app.id.clone(), manifest);
}
pub fn resolve_dependencies(&self, app_id: &str) -> Result<Vec<String>, DependencyError> {
let mut visited = HashSet::new();
let mut visiting = HashSet::new();
let mut result = Vec::new();
self.resolve_recursive(app_id, &mut visited, &mut visiting, &mut result)?;
// Result is already in installation order (dependencies first)
Ok(result)
}
fn resolve_recursive(
&self,
app_id: &str,
visited: &mut HashSet<String>,
visiting: &mut HashSet<String>,
result: &mut Vec<String>,
) -> Result<(), DependencyError> {
if visited.contains(app_id) {
return Ok(());
}
if visiting.contains(app_id) {
return Err(DependencyError::CircularDependency(format!(
"Circular dependency detected involving: {}",
app_id
)));
}
visiting.insert(app_id.to_string());
let manifest = self.manifests.get(app_id).ok_or_else(|| {
DependencyError::MissingDependency(format!("App not found: {}", app_id))
})?;
// Resolve all dependencies first
for dep in &manifest.app.dependencies {
match dep {
Dependency::App {
app_id: dep_id,
version: _,
} => {
self.resolve_recursive(dep_id, visited, visiting, result)?;
}
Dependency::Storage { storage: _ } => {
// Storage dependencies are checked but don't require other apps
}
Dependency::Simple(dep_id) => {
self.resolve_recursive(dep_id, visited, visiting, result)?;
}
}
}
visiting.remove(app_id);
visited.insert(app_id.to_string());
if !result.contains(&app_id.to_string()) {
result.push(app_id.to_string());
}
Ok(())
}
pub fn check_conflicts(&self, app_id: &str) -> Result<(), DependencyError> {
let manifest = self.manifests.get(app_id).ok_or_else(|| {
DependencyError::MissingDependency(format!("App not found: {}", app_id))
})?;
// Check for port conflicts
let mut port_usage: HashMap<u16, String> = HashMap::new();
for (id, m) in &self.manifests {
if id == app_id {
continue;
}
for port in &m.app.ports {
if let Some(existing) = port_usage.get(&port.host) {
return Err(DependencyError::VersionConflict(format!(
"Port {} already used by {}",
port.host, existing
)));
}
port_usage.insert(port.host, id.clone());
}
}
// Check for new app's ports
for port in &manifest.app.ports {
if let Some(existing) = port_usage.get(&port.host) {
return Err(DependencyError::VersionConflict(format!(
"Port {} already used by {}",
port.host, existing
)));
}
}
Ok(())
}
pub fn calculate_resources(&self, app_ids: &[String]) -> ResourceRequirements {
let mut total = ResourceRequirements {
cpu: 0,
memory_mb: 0,
disk_gb: 0,
};
for app_id in app_ids {
if let Some(manifest) = self.manifests.get(app_id) {
if let Some(cpu) = manifest.app.resources.cpu_limit {
total.cpu += cpu;
}
if let Some(memory) = &manifest.app.resources.memory_limit {
// Parse memory string (e.g., "1Gi", "512Mi")
if let Ok(mb) = parse_memory(memory) {
total.memory_mb += mb;
}
}
if let Some(disk) = &manifest.app.resources.disk_limit {
// Parse disk string (e.g., "10Gi", "500Mi")
if let Ok(gb) = parse_disk(disk) {
total.disk_gb += gb;
}
}
}
}
total
}
}
fn parse_memory(s: &str) -> Result<u32, ()> {
let s = s.trim().to_lowercase();
if s.ends_with("gi") {
let num: f64 = s.trim_end_matches("gi").parse().map_err(|_| ())?;
Ok((num * 1024.0) as u32)
} else if s.ends_with("mi") {
let num: f64 = s.trim_end_matches("mi").parse().map_err(|_| ())?;
Ok(num as u32)
} else {
Err(())
}
}
fn parse_disk(s: &str) -> Result<u32, ()> {
let s = s.trim().to_lowercase();
if s.ends_with("gi") {
let num: f64 = s.trim_end_matches("gi").parse().map_err(|_| ())?;
Ok(num as u32)
} else if s.ends_with("ti") {
let num: f64 = s.trim_end_matches("ti").parse().map_err(|_| ())?;
Ok((num * 1024.0) as u32)
} else {
Err(())
}
}
#[derive(Debug, Clone)]
pub struct ResourceRequirements {
pub cpu: u32,
pub memory_mb: u32,
pub disk_gb: u32,
}
impl Default for DependencyResolver {
fn default() -> Self {
Self::new()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::{AppDefinition, AppManifest, ContainerConfig};
fn create_test_manifest(id: &str, deps: Vec<Dependency>) -> AppManifest {
AppManifest {
app: AppDefinition {
id: id.to_string(),
name: format!("Test {}", id),
version: "1.0.0".to_string(),
description: None,
container: ContainerConfig {
image: Some(format!("test/{}:latest", id)),
image_signature: None,
pull_policy: "if-not-present".to_string(),
build: None,
network: None,
custom_args: vec![],
entrypoint: None,
derived_env: vec![],
secret_env: vec![],
data_uid: None,
},
dependencies: deps,
resources: Default::default(),
security: Default::default(),
ports: vec![],
volumes: vec![],
environment: vec![],
health_check: None,
devices: vec![],
extensions: Default::default(),
},
}
}
#[test]
fn test_simple_dependency() {
let mut resolver = DependencyResolver::new();
resolver.add_manifest(create_test_manifest("app1", vec![]));
resolver.add_manifest(create_test_manifest(
"app2",
vec![Dependency::Simple("app1".to_string())],
));
let deps = resolver.resolve_dependencies("app2").unwrap();
assert_eq!(deps, vec!["app1", "app2"]);
}
#[test]
fn test_circular_dependency() {
let mut resolver = DependencyResolver::new();
resolver.add_manifest(create_test_manifest(
"app1",
vec![Dependency::Simple("app2".to_string())],
));
resolver.add_manifest(create_test_manifest(
"app2",
vec![Dependency::Simple("app1".to_string())],
));
let result = resolver.resolve_dependencies("app1");
assert!(result.is_err());
}
}
+3 -3
View File
@@ -1,5 +1,4 @@
pub mod bitcoin_simulator;
pub mod dependency_resolver;
pub mod health_monitor;
pub mod manifest;
pub mod podman_client;
@@ -7,13 +6,14 @@ pub mod port_manager;
pub mod runtime;
pub use bitcoin_simulator::{BitcoinSimulationMode, BitcoinSimulator};
pub use dependency_resolver::DependencyResolver;
pub use health_monitor::HealthMonitor;
pub use manifest::{
AppManifest, BuildConfig, ContainerConfig, Dependency, DerivedEnv, HealthCheck, HostFacts,
ManifestError, ResolvedSource, ResourceLimits, SecretEnv, SecretsProvider, SecurityPolicy,
Volume,
};
pub use podman_client::{ContainerState, ContainerStatus, PodmanClient};
pub use podman_client::{
image_uses_insecure_registry, ContainerState, ContainerStatus, PodmanClient,
};
pub use port_manager::{PortError, PortManager};
pub use runtime::{AutoRuntime, ContainerRuntime, DockerRuntime, PodmanRuntime};
+38
View File
@@ -619,6 +619,14 @@ impl ContainerConfig {
let mut out = Vec::with_capacity(self.secret_env.len());
for e in &self.secret_env {
let v = provider.read(&e.secret_file)?;
// An empty secret produces e.g. `-rpcpassword=` and crashes
// the container on auth before logs are useful. Fail loud.
if v.trim().is_empty() {
return Err(ManifestError::Invalid(format!(
"secret_env {} resolved to empty value (file: {})",
e.key, e.secret_file
)));
}
out.push(format!("{}={}", e.key, v));
}
Ok(out)
@@ -1051,6 +1059,36 @@ app:
assert_eq!(out[1], "FM_GATEWAY_PASSWORD=supersecret2");
}
#[test]
fn resolve_secret_env_rejects_empty_value() {
let c = ContainerConfig {
image: Some("x:latest".to_string()),
image_signature: None,
pull_policy: "if-not-present".to_string(),
build: None,
network: None,
custom_args: vec![],
entrypoint: None,
derived_env: vec![],
secret_env: vec![SecretEnv {
key: "BITCOIN_RPC_PASS".to_string(),
secret_file: "bitcoin-rpc-password".to_string(),
}],
data_uid: None,
};
let p = MapSecretsProvider {
data: HashMap::from([("bitcoin-rpc-password".to_string(), " \n".to_string())]),
};
let err = c.resolve_secret_env(&p).unwrap_err();
match err {
ManifestError::Invalid(msg) => assert!(
msg.contains("BITCOIN_RPC_PASS") && msg.contains("bitcoin-rpc-password"),
"msg should name the env key + file: {msg}"
),
other => panic!("expected Invalid, got {other:?}"),
}
}
#[test]
fn parse_every_real_manifest() {
let app_manifests = list_repo_manifests();
+100 -18
View File
@@ -46,6 +46,7 @@ pub struct ContainerStatus {
pub enum ContainerState {
Created,
Running,
Stopping,
Stopped,
Exited,
Paused,
@@ -57,6 +58,7 @@ impl From<&str> for ContainerState {
match s.to_lowercase().as_str() {
"created" => ContainerState::Created,
"running" => ContainerState::Running,
"stopping" => ContainerState::Stopping,
"stopped" => ContainerState::Stopped,
"exited" => ContainerState::Exited,
"paused" => ContainerState::Paused,
@@ -109,7 +111,7 @@ impl PodmanClient {
pub fn lan_address_for(name: &str) -> Option<String> {
let url = match name {
"bitcoin-knots" | "bitcoin-ui" => "http://localhost:8334",
"lnd" | "archy-lnd-ui" => "http://localhost:8081",
"lnd" | "archy-lnd-ui" => "http://localhost:18083",
"homeassistant" => "http://localhost:8123",
"archy-mempool-web" | "mempool" => "http://localhost:4080",
"btcpay-server" => "http://localhost:23000",
@@ -120,17 +122,18 @@ impl PodmanClient {
"penpot" => "http://localhost:9001",
"nextcloud" => "http://localhost:8085",
"vaultwarden" => "http://localhost:8082",
"gitea" => "http://localhost:3001",
"jellyfin" => "http://localhost:8096",
"photoprism" => "http://localhost:2342",
"immich_server" | "immich" => "http://localhost:2283",
"filebrowser" => "http://localhost:8083",
"nginx-proxy-manager" => "http://localhost:81",
"nginx-proxy-manager" => "http://localhost:8081",
"portainer" => "http://localhost:9000",
"uptime-kuma" => "http://localhost:3002",
"fedimint" | "fedimintd" => "http://localhost:8175",
"fedimint-gateway" => "http://localhost:8176",
"nostr-rs-relay" => "http://localhost:18081",
"indeedhub" => "http://localhost:7777",
"indeedhub" => "http://localhost:7778",
"dwn" => "http://localhost:3100",
"endurain" => "http://localhost:8080",
"electrs" | "archy-electrs-ui" => "http://localhost:50002",
@@ -257,7 +260,11 @@ impl PodmanClient {
pub async fn pull_image(&self, image: &str, _signature: Option<&str>) -> Result<()> {
// Image pull uses CLI — it's a streaming operation that the API handles differently
let mut cmd = tokio::process::Command::new("podman");
cmd.arg("pull").arg(image);
cmd.arg("pull");
if image_uses_insecure_registry(image) {
cmd.arg("--tls-verify=false");
}
cmd.arg(image);
let output = tokio::time::timeout(
std::time::Duration::from_secs(600), // 10 min for large images
@@ -357,20 +364,12 @@ impl PodmanClient {
);
}
let net_mode = if let Some(n) = manifest.app.container.network.as_ref() {
if n.is_empty() {
"bridge"
} else {
n.as_str()
}
} else {
match manifest.app.security.network_policy.as_str() {
"host" => "host",
_ => "bridge",
}
};
let (net_mode, custom_network) = podman_network_settings(
manifest.app.container.network.as_deref(),
manifest.app.security.network_policy.as_str(),
);
let body = serde_json::json!({
let mut body = serde_json::json!({
"name": name,
"image": image_ref,
"portmappings": port_mappings,
@@ -378,7 +377,10 @@ impl PodmanClient {
"env": env_map,
"entrypoint": manifest.app.container.entrypoint.clone(),
"command": manifest.app.container.custom_args.clone(),
"hostadd": ["host.containers.internal:host-gateway"],
"hostadd": [
"host.containers.internal:host-gateway",
"host.archipelago:10.89.0.1",
],
"devices": manifest.app.devices.iter().map(|d| {
serde_json::json!({"path": d})
}).collect::<Vec<_>>(),
@@ -393,6 +395,14 @@ impl PodmanClient {
"nsmode": net_mode
},
});
if let Some(network) = custom_network {
body.as_object_mut()
.expect("container create body is a JSON object")
.insert(
"networks".to_string(),
serde_json::json!({ network: { "aliases": [name] } }),
);
}
let result = self
.api_request("POST", "libpod/containers/create", Some(body), LONG_TIMEOUT)
@@ -601,6 +611,35 @@ impl PodmanClient {
}
}
/// Registries we ship with as `--tls-verify=false` because they're internal
/// HTTP mirrors. Add a host:port here only if it's a controlled mirror that
/// the fleet trusts and operators won't ever paste a malicious URL into.
const INSECURE_REGISTRY_HOSTS: &[&str] = &["146.59.87.168:3000"];
pub fn image_uses_insecure_registry(image: &str) -> bool {
image
.split('/')
.next()
.is_some_and(|host| INSECURE_REGISTRY_HOSTS.contains(&host))
}
fn podman_network_settings(
network: Option<&str>,
network_policy: &str,
) -> (&'static str, Option<String>) {
match network {
Some("") => ("bridge", None),
Some("host") => ("host", None),
Some("bridge") => ("bridge", None),
Some("none") => ("none", None),
Some("slirp4netns") => ("slirp4netns", None),
Some("private") => ("private", None),
Some(custom) => ("bridge", Some(custom.to_string())),
None if network_policy == "host" => ("host", None),
None => ("bridge", None),
}
}
// ─── Helpers ─────────────────────────────────────────────────────
fn parse_port_bindings(bindings: &serde_json::Value) -> Vec<String> {
@@ -673,6 +712,49 @@ fn parse_memory_limit(limit: &str) -> Option<i64> {
mod tests {
use super::*;
#[test]
fn insecure_registry_detection_matches_http_mirrors_only() {
assert!(image_uses_insecure_registry(
"146.59.87.168:3000/lfg2025/bitcoin-knots:latest"
));
// The legacy Hetzner mirror at 23.182.128.160 was decommissioned and
// is no longer trusted — it must NOT bypass TLS even if a stale
// registry config still references it.
assert!(!image_uses_insecure_registry(
"23.182.128.160:3000/lfg2025/filebrowser:v2.27.0"
));
assert!(!image_uses_insecure_registry(
"git.tx1138.com/lfg2025/bitcoin-knots:latest"
));
assert!(!image_uses_insecure_registry(
"docker.io/library/nginx:latest"
));
// Spoofing immune: an attacker host that prefixes the trusted IP
// string into its own URL still has the attacker host in the
// registry-host slot, so it does NOT match.
assert!(!image_uses_insecure_registry(
"evil.example:80/146.59.87.168:3000/lfg2025/x:latest"
));
}
#[test]
fn podman_network_settings_uses_networks_map_for_custom_networks() {
assert_eq!(
podman_network_settings(Some("archy-net"), "isolated"),
("bridge", Some("archy-net".to_string()))
);
assert_eq!(
podman_network_settings(Some("host"), "isolated"),
("host", None)
);
assert_eq!(
podman_network_settings(Some(""), "isolated"),
("bridge", None)
);
assert_eq!(podman_network_settings(None, "host"), ("host", None));
assert_eq!(podman_network_settings(None, "isolated"), ("bridge", None));
}
#[test]
fn parse_memory_limit_iec_binary_suffixes() {
// Kubernetes-style — this is what apps/*/manifest.yml uses.
+312 -8
View File
@@ -3,8 +3,12 @@ use crate::podman_client::{ContainerState, ContainerStatus, PodmanClient};
use anyhow::{Context, Result};
use async_trait::async_trait;
use std::process::Command;
use std::time::Duration;
use tokio::process::Command as TokioCommand;
const PODMAN_CLI_DEFAULT_TIMEOUT: Duration = Duration::from_secs(30);
const PODMAN_CLI_BUILD_TIMEOUT: Duration = Duration::from_secs(900);
#[async_trait]
pub trait ContainerRuntime: Send + Sync {
async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()>;
@@ -52,13 +56,31 @@ impl PodmanRuntime {
/// Run `podman <args>`, returning an error with captured stderr on non-zero
/// exit. Used for operations (build, image inspect) that are awkward over the
/// HTTP API. The daemon runs as the target user already, so no sudo hop.
async fn podman_cli(&self, args: &[&str]) -> Result<std::process::Output> {
async fn podman_cli_timeout(
&self,
args: &[&str],
timeout: Duration,
) -> Result<std::process::Output> {
let mut cmd = TokioCommand::new("podman");
cmd.args(args);
cmd.output()
cmd.kill_on_drop(true);
tokio::time::timeout(timeout, cmd.output())
.await
.with_context(|| {
format!(
"podman {} timed out after {}s",
args.join(" "),
timeout.as_secs()
)
})?
.with_context(|| format!("failed to execute podman {}", args.join(" ")))
}
/// Run `podman <args>` with a short timeout for control-plane operations.
async fn podman_cli(&self, args: &[&str]) -> Result<std::process::Output> {
self.podman_cli_timeout(args, PODMAN_CLI_DEFAULT_TIMEOUT)
.await
}
}
#[async_trait]
@@ -84,19 +106,72 @@ impl ContainerRuntime for PodmanRuntime {
}
async fn start_container(&self, name: &str) -> Result<()> {
self.client.start_container(name).await
match self.client.start_container(name).await {
Ok(()) => Ok(()),
Err(api_err) => {
let output = self.podman_cli(&["start", name]).await?;
if output.status.success() {
Ok(())
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
Err(api_err.context(format!("podman start fallback failed: {}", stderr.trim())))
}
}
}
}
async fn stop_container(&self, name: &str) -> Result<()> {
self.client.stop_container(name).await
match self.client.stop_container(name).await {
Ok(()) => Ok(()),
Err(api_err) => {
let output = self.podman_cli(&["stop", "-t", "30", name]).await?;
if output.status.success() {
Ok(())
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
if is_missing_container_error(&stderr) {
return Ok(());
}
Err(api_err.context(format!("podman stop fallback failed: {}", stderr.trim())))
}
}
}
}
async fn remove_container(&self, name: &str) -> Result<()> {
self.client.remove_container(name).await
match self.client.remove_container(name).await {
Ok(()) => Ok(()),
Err(api_err) => {
let output = self.podman_cli(&["rm", "-f", name]).await?;
if output.status.success() {
Ok(())
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
if is_missing_container_error(&stderr) {
return Ok(());
}
Err(api_err.context(format!("podman rm fallback failed: {}", stderr.trim())))
}
}
}
}
async fn get_container_status(&self, name: &str) -> Result<ContainerStatus> {
self.client.get_container_status(name).await
match self.client.get_container_status(name).await {
Ok(status) => Ok(status),
Err(api_err) => {
let output = self
.podman_cli(&["container", "inspect", "--format", "json", name])
.await?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(api_err
.context(format!("podman inspect fallback failed: {}", stderr.trim())));
}
parse_podman_inspect_json(&output.stdout, name)
.with_context(|| format!("podman API inspect failed: {api_err}"))
}
}
}
async fn get_container_logs(&self, name: &str, lines: u32) -> Result<Vec<String>> {
@@ -104,7 +179,20 @@ impl ContainerRuntime for PodmanRuntime {
}
async fn list_containers(&self) -> Result<Vec<ContainerStatus>> {
self.client.list_containers().await
match self.client.list_containers().await {
Ok(containers) => Ok(containers),
Err(api_err) => {
let output = self.podman_cli(&["ps", "-a", "--format", "json"]).await?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(
api_err.context(format!("podman ps fallback failed: {}", stderr.trim()))
);
}
parse_podman_ps_json(&output.stdout)
.with_context(|| format!("podman API list failed: {api_err}"))
}
}
}
async fn image_exists(&self, image_ref: &str) -> Result<bool> {
@@ -129,7 +217,9 @@ impl ContainerRuntime for PodmanRuntime {
async fn build_image(&self, config: &BuildConfig) -> Result<()> {
let args = build_args_for_podman(config);
let borrowed: Vec<&str> = args.iter().map(|s| s.as_str()).collect();
let output = self.podman_cli(&borrowed).await?;
let output = self
.podman_cli_timeout(&borrowed, PODMAN_CLI_BUILD_TIMEOUT)
.await?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
let stdout = String::from_utf8_lossy(&output.stdout);
@@ -147,6 +237,188 @@ impl ContainerRuntime for PodmanRuntime {
}
}
fn parse_podman_ps_json(stdout: &[u8]) -> Result<Vec<ContainerStatus>> {
let text = String::from_utf8_lossy(stdout);
if text.trim().is_empty() {
return Ok(Vec::new());
}
let containers: Vec<serde_json::Value> = serde_json::from_str(&text)?;
Ok(containers
.into_iter()
.map(|c| {
let name = c
.get("Names")
.and_then(|v| v.as_array())
.and_then(|a| a.first())
.and_then(|v| v.as_str())
.or_else(|| c.get("Names").and_then(|v| v.as_str()))
.unwrap_or("")
.to_string();
let status = c.get("Status").and_then(|v| v.as_str()).unwrap_or("");
let state = c.get("State").and_then(|v| v.as_str()).unwrap_or("unknown");
ContainerStatus {
id: c
.get("Id")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
name: name.clone(),
state: ContainerState::from(state),
health: parse_health_from_status(status),
exit_code: c.get("ExitCode").and_then(|v| v.as_i64()).map(|c| c as i32),
started_at: c
.get("StartedAt")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
image: c
.get("Image")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
created: c
.get("Created")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
ports: parse_podman_ps_ports(c.get("Ports")),
lan_address: PodmanClient::lan_address_for(&name),
}
})
.collect())
}
fn parse_podman_inspect_json(stdout: &[u8], requested_name: &str) -> Result<ContainerStatus> {
let text = String::from_utf8_lossy(stdout);
let containers: Vec<serde_json::Value> = serde_json::from_str(&text)?;
let c = containers
.first()
.ok_or_else(|| anyhow::anyhow!("podman inspect returned no containers"))?;
if c.get("State").is_none() {
return Err(anyhow::anyhow!(
"podman inspect returned non-container object for {requested_name}"
));
}
let name = c
.get("Name")
.and_then(|v| v.as_str())
.map(|s| s.trim_start_matches('/'))
.unwrap_or(requested_name)
.to_string();
let state = c
.get("State")
.and_then(|v| v.get("Status"))
.and_then(|v| v.as_str())
.unwrap_or("unknown");
Ok(ContainerStatus {
id: c
.get("Id")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
name: name.clone(),
state: ContainerState::from(state),
health: c
.get("State")
.and_then(|v| v.get("Health"))
.and_then(|v| v.get("Status"))
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
exit_code: c
.get("State")
.and_then(|v| v.get("ExitCode"))
.and_then(|v| v.as_i64())
.map(|c| c as i32),
started_at: c
.get("State")
.and_then(|v| v.get("StartedAt"))
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
image: c
.get("ImageName")
.and_then(|v| v.as_str())
.or_else(|| {
c.get("Config")
.and_then(|v| v.get("Image"))
.and_then(|v| v.as_str())
})
.unwrap_or("")
.to_string(),
created: c
.get("Created")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
ports: parse_inspect_ports(c),
lan_address: PodmanClient::lan_address_for(&name),
})
}
fn parse_inspect_ports(c: &serde_json::Value) -> Vec<String> {
let Some(bindings) = c
.get("HostConfig")
.and_then(|v| v.get("PortBindings"))
.and_then(|v| v.as_object())
else {
return Vec::new();
};
let mut ports = Vec::new();
for (container_port, host_bindings) in bindings {
let Some(host_bindings) = host_bindings.as_array() else {
continue;
};
for binding in host_bindings {
let host_ip = binding
.get("HostIp")
.and_then(|v| v.as_str())
.unwrap_or("0.0.0.0");
let host_port = binding
.get("HostPort")
.and_then(|v| v.as_str())
.unwrap_or("");
if !host_port.is_empty() {
ports.push(format!("{host_ip}:{host_port}->{container_port}"));
}
}
}
ports
}
fn parse_podman_ps_ports(ports: Option<&serde_json::Value>) -> Vec<String> {
ports
.and_then(|v| v.as_array())
.map(|ports| {
ports
.iter()
.filter_map(|port| {
let host = port.get("host_port").and_then(|v| v.as_u64())?;
let container = port.get("container_port").and_then(|v| v.as_u64())?;
let proto = port
.get("protocol")
.and_then(|v| v.as_str())
.unwrap_or("tcp");
Some(format!("0.0.0.0:{host}->{container}/{proto}"))
})
.collect()
})
.unwrap_or_default()
}
fn parse_health_from_status(status: &str) -> Option<String> {
let start = status.rfind('(')?;
let end = status.rfind(')')?;
(start < end).then(|| status[start + 1..end].to_string())
}
fn is_missing_container_error(stderr: &str) -> bool {
let stderr = stderr.to_ascii_lowercase();
stderr.contains("no container with name or id")
|| stderr.contains("no such container")
|| stderr.contains("does not exist")
|| stderr.contains("not found")
}
/// Build the argv for `podman build` from a BuildConfig.
///
/// Extracted so it can be unit-tested without actually invoking podman.
@@ -646,4 +918,36 @@ mod tests {
let args = build_args_for_podman(&c);
assert_eq!(args.last().unwrap(), "/final/context");
}
#[test]
fn parse_podman_ps_json_handles_cli_output() {
let stdout = br#"[
{
"Id": "abc123",
"Names": ["mempool"],
"Image": "docker.io/mempool/frontend:latest",
"State": "running",
"Status": "Up 2 minutes (healthy)",
"Created": "2026-05-03T00:00:00Z",
"StartedAt": "2026-05-03T00:01:00Z",
"ExitCode": 0,
"Ports": [
{
"host_port": 4080,
"container_port": 8080,
"protocol": "tcp"
}
]
}
]"#;
let containers = parse_podman_ps_json(stdout).unwrap();
assert_eq!(containers.len(), 1);
assert_eq!(containers[0].id, "abc123");
assert_eq!(containers[0].name, "mempool");
assert_eq!(containers[0].state, ContainerState::Running);
assert_eq!(containers[0].health.as_deref(), Some("healthy"));
assert_eq!(containers[0].exit_code, Some(0));
assert_eq!(containers[0].ports, vec!["0.0.0.0:4080->8080/tcp"]);
}
}
+121 -43
View File
@@ -606,7 +606,8 @@
console.log('[Bitcoin UI] Script loaded, initializing...');
// RPC Configuration - Use local Nginx proxy within container
const RPC_ENDPOINT = '/bitcoin-rpc/';
const RPC_ENDPOINT = 'bitcoin-rpc/';
const STATUS_ENDPOINT = 'bitcoin-status';
console.log('[Bitcoin UI] RPC Endpoint:', RPC_ENDPOINT);
// Make RPC call to Bitcoin node via local proxy
@@ -645,6 +646,14 @@
}
}
async function fetchBitcoinStatus() {
const response = await fetch(STATUS_ENDPOINT, { cache: 'no-store' });
if (!response.ok) {
throw new Error(`status HTTP ${response.status}`);
}
return response.json();
}
// Implementation branding — detected from getnetworkinfo.subversion.
// Bitcoin Knots identifies as "/Satoshi:<ver>/Knots:<date>/", Bitcoin Core as "/Satoshi:<ver>/".
let brandingApplied = false;
@@ -672,22 +681,62 @@
// Track last block count for animations
let lastBlockCount = 0;
let consecutiveRpcFailures = 0;
let lastSuccessfulUpdateAt = 0;
function formatPercent(value) {
if (!Number.isFinite(value) || value <= 0) return '0.00';
if (value < 0.01) return '<0.01';
return value.toFixed(2);
}
function formatBytes(bytes) {
if (!Number.isFinite(bytes) || bytes <= 0) return null;
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
let value = bytes;
let unit = 0;
while (value >= 1000 && unit < units.length - 1) {
value /= 1000;
unit += 1;
}
return `${value.toFixed(unit >= 3 ? 1 : 0)} ${units[unit]}`;
}
// Update blockchain info
async function updateBlockchainInfo() {
console.log('[Bitcoin UI] updateBlockchainInfo() called');
try {
const blockchainInfo = await callRPC('getblockchaininfo');
const status = await fetchBitcoinStatus();
const blockchainInfo = status.blockchain_info;
console.log('[Bitcoin UI] blockchainInfo:', blockchainInfo);
if (!blockchainInfo) {
console.error('[Bitcoin UI] No blockchain info received');
document.getElementById('syncStatusText').textContent = 'Unable to connect to Bitcoin node';
document.getElementById('syncStatusText').className = 'text-red-400 text-sm';
consecutiveRpcFailures += 1;
const syncStatusText = document.getElementById('syncStatusText');
const syncIcon = document.getElementById('syncIcon');
if (syncStatusText) {
if (status.stale) {
syncStatusText.textContent = status.error || 'Bitcoin node is reconnecting... showing last known values';
syncStatusText.className = 'text-yellow-300 text-sm font-medium';
} else if (consecutiveRpcFailures < 6) {
syncStatusText.textContent = status.error || 'Connecting to Bitcoin node...';
syncStatusText.className = 'text-yellow-300 text-sm font-medium';
} else {
syncStatusText.textContent = status.error || 'Bitcoin node is not responding yet';
syncStatusText.className = 'text-red-400 text-sm font-medium';
}
}
if (syncIcon) {
syncIcon.classList.add('animate-spin-slow');
syncIcon.classList.remove('text-green-500');
}
return;
}
consecutiveRpcFailures = 0;
lastSuccessfulUpdateAt = Date.now();
const networkInfo = await callRPC('getnetworkinfo');
const networkInfo = status.network_info;
applyImplBranding(networkInfo && networkInfo.subversion);
@@ -743,44 +792,51 @@
}
// Populate Settings — Transaction Index, ZMQ, RPC (fire-and-forget)
(async () => {
const txIndexEl = document.getElementById('settingsTxIndex');
if (txIndexEl) {
const idx = await callRPC('getindexinfo');
if (idx && typeof idx === 'object') {
const names = Object.keys(idx);
txIndexEl.textContent = names.length
? `Enabled: ${names.join(', ')}`
: 'Disabled';
} else {
txIndexEl.textContent = 'Disabled';
}
const txIndexEl = document.getElementById('settingsTxIndex');
if (txIndexEl) {
const idx = status.index_info;
if (idx && typeof idx === 'object') {
const names = Object.keys(idx);
txIndexEl.textContent = names.length
? `Enabled: ${names.join(', ')}`
: 'Disabled';
} else {
txIndexEl.textContent = 'Unavailable while node starts';
}
const zmqEl = document.getElementById('settingsZmq');
if (zmqEl) {
const zmq = await callRPC('getzmqnotifications');
if (Array.isArray(zmq) && zmq.length) {
zmqEl.textContent = zmq.map(z => `${z.type}@${z.address}`).join('; ');
} else {
zmqEl.textContent = 'Not enabled';
}
}
const zmqEl = document.getElementById('settingsZmq');
if (zmqEl) {
const zmq = status.zmq_notifications;
if (Array.isArray(zmq) && zmq.length) {
zmqEl.textContent = zmq.map(z => `${z.type}@${z.address}`).join('; ');
} else if (Array.isArray(zmq)) {
zmqEl.textContent = 'Not enabled';
} else {
zmqEl.textContent = 'Unavailable while node starts';
}
const rpcEl = document.getElementById('settingsRpc');
if (rpcEl && networkInfo) {
const port = chain === 'main' ? 8332 : (chain === 'test' ? 18332 : (chain === 'signet' ? 38332 : 18443));
rpcEl.textContent = `Reachable on port ${port}`;
}
})();
}
const rpcEl = document.getElementById('settingsRpc');
if (rpcEl) {
const port = chain === 'main' ? 8332 : (chain === 'test' ? 18332 : (chain === 'signet' ? 38332 : 18443));
rpcEl.textContent = status.stale
? `Reconnecting on port ${port}`
: `Reachable on port ${port}`;
}
// Update sync status
const blocks = blockchainInfo.blocks || 0;
const headers = blockchainInfo.headers || 0;
const verificationProgress = blockchainInfo.verificationprogress || 0;
const isSynced = blocks >= headers - 1;
const initialBlockDownload = blockchainInfo.initialblockdownload === true;
const isSynced = headers > 0 && blocks >= headers - 1 && !initialBlockDownload;
const diskSize = formatBytes(blockchainInfo.size_on_disk || 0);
const appearsToBeReindexing = initialBlockDownload && blocks === 0 && headers > 0 && (blockchainInfo.size_on_disk || 0) > 1024 * 1024 * 1024;
// Calculate actual sync percentage based on blocks/headers
const actualSyncPercentage = headers > 0 ? ((blocks / headers) * 100).toFixed(2) : '0.00';
const verificationPercentage = (verificationProgress * 100).toFixed(2);
const actualSyncValue = headers > 0 ? (blocks / headers) * 100 : 0;
const actualSyncPercentage = formatPercent(actualSyncValue);
const progressWidth = Math.max(0, Math.min(100, actualSyncValue));
const verificationPercentage = formatPercent(verificationProgress * 100);
// Animate block count if it changed
const currentHeightElem = document.getElementById('currentHeight');
@@ -795,16 +851,27 @@
document.getElementById('headers').textContent = headers.toLocaleString();
document.getElementById('verificationProgress').textContent = `${verificationPercentage}%`;
document.getElementById('syncPercentage').textContent = `${actualSyncPercentage}%`;
document.getElementById('currentBlock').textContent = `Block ${blocks.toLocaleString()}`;
document.getElementById('syncProgressBar').style.width = `${actualSyncPercentage}%`;
document.getElementById('currentBlock').textContent = appearsToBeReindexing
? 'Reindexing from disk'
: `Block ${blocks.toLocaleString()}`;
document.getElementById('syncProgressBar').style.width = `${progressWidth}%`;
// Update sync status text and icon
const syncStatusText = document.getElementById('syncStatusText');
const syncIcon = document.getElementById('syncIcon');
if (isSynced) {
syncStatusText.textContent = '✓ Fully synchronized with the network';
syncStatusText.className = 'text-green-400 text-sm font-medium';
if (appearsToBeReindexing) {
syncStatusText.textContent = `Reindexing local block files${diskSize ? ` (${diskSize} on disk)` : ''}`;
syncStatusText.className = 'text-orange-400 text-sm font-medium';
if (syncIcon) {
syncIcon.classList.add('animate-spin-slow');
syncIcon.classList.remove('text-green-500');
}
} else if (isSynced) {
syncStatusText.textContent = status.stale
? 'Bitcoin node is reconnecting... showing last known synchronized state'
: '✓ Fully synchronized with the network';
syncStatusText.className = status.stale ? 'text-yellow-300 text-sm font-medium' : 'text-green-400 text-sm font-medium';
// Stop spinning when synced
if (syncIcon) {
syncIcon.classList.remove('animate-spin-slow');
@@ -812,8 +879,12 @@
}
} else {
const remaining = headers - blocks;
syncStatusText.textContent = `Syncing... ${remaining.toLocaleString()} blocks remaining`;
syncStatusText.className = 'text-orange-400 text-sm font-medium';
syncStatusText.textContent = status.stale
? 'Bitcoin node is reconnecting... showing last known sync state'
: initialBlockDownload
? `Initial block download... ${remaining.toLocaleString()} blocks remaining`
: `Syncing... ${remaining.toLocaleString()} blocks remaining`;
syncStatusText.className = status.stale ? 'text-yellow-300 text-sm font-medium' : 'text-orange-400 text-sm font-medium';
// Keep spinning while syncing
if (syncIcon) {
syncIcon.classList.add('animate-spin-slow');
@@ -834,8 +905,15 @@
} catch (error) {
console.error('Failed to update blockchain info:', error);
document.getElementById('syncStatusText').textContent = 'Unable to fetch blockchain data';
document.getElementById('syncStatusText').className = 'text-red-400 text-sm';
consecutiveRpcFailures += 1;
const syncStatusText = document.getElementById('syncStatusText');
if (syncStatusText) {
const hasRecentData = lastSuccessfulUpdateAt > 0 && Date.now() - lastSuccessfulUpdateAt < 120000;
syncStatusText.textContent = hasRecentData
? 'Bitcoin status bridge is reconnecting... keeping last known values'
: 'Connecting to Bitcoin status bridge...';
syncStatusText.className = 'text-yellow-300 text-sm font-medium';
}
}
}
+44 -14
View File
@@ -68,6 +68,7 @@
@media (min-width: 768px) {
.md-flex-row { flex-direction: row; }
.md-grid-cols-4 { grid-template-columns: repeat(4, 1fr); }
.md-grid-cols-5 { grid-template-columns: repeat(5, 1fr); }
}
/* Connection details */
@@ -147,13 +148,17 @@
</div>
</div>
<div class="grid grid-cols-2 md-grid-cols-4 gap-3">
<div class="grid grid-cols-2 md-grid-cols-5 gap-3">
<div class="info-card">
<p class="text-xs text-white-60 mb-1">Indexed Height</p>
<p class="text-xs text-white-60 mb-1">Electrum Indexed</p>
<p class="text-lg font-semibold text-white" id="indexedHeight">-</p>
</div>
<div class="info-card">
<p class="text-xs text-white-60 mb-1">Network Height</p>
<p class="text-xs text-white-60 mb-1">Bitcoin Node</p>
<p class="text-lg font-semibold text-white" id="bitcoinHeight">-</p>
</div>
<div class="info-card">
<p class="text-xs text-white-60 mb-1">Known Headers</p>
<p class="text-lg font-semibold text-white" id="networkHeight">-</p>
</div>
<div class="info-card">
@@ -370,15 +375,36 @@
}
var indexedH = data.indexed_height || 0;
var networkH = data.network_height || 0;
var bitcoinH = data.bitcoin_height || 0;
var reportedNetworkH = data.network_height || 0;
var knownHeaderH = Math.max(reportedNetworkH, indexedH, bitcoinH);
var targetH = bitcoinH > 0 ? bitcoinH : knownHeaderH;
var pct = data.progress_pct || 0;
var hasIndexedHeight = indexedH > 0 || data.stale;
var indexedLabel = hasIndexedHeight
? indexedH.toLocaleString()
: (data.status === 'indexing' ? 'Pending' : '-');
var currentBlockLabel;
if (hasIndexedHeight && bitcoinH > 0 && indexedH > bitcoinH) {
currentBlockLabel = 'Bitcoin node ' + bitcoinH.toLocaleString()
+ (knownHeaderH > 0 ? ' of known headers ' + knownHeaderH.toLocaleString() : '')
+ '; Electrum index ' + indexedH.toLocaleString();
} else if (hasIndexedHeight) {
currentBlockLabel = 'Indexed ' + indexedH.toLocaleString() + ' of '
+ (targetH > 0 ? targetH.toLocaleString() : 'Bitcoin node height');
} else {
currentBlockLabel = data.index_size
? 'Index building from disk (' + data.index_size + ')'
: 'Waiting for Electrum index height';
}
document.getElementById('indexedHeight').textContent = indexedH > 0 ? indexedH.toLocaleString() : (data.status === 'indexing' ? 'Building...' : '-');
document.getElementById('networkHeight').textContent = networkH > 0 ? networkH.toLocaleString() : '-';
document.getElementById('indexedHeight').textContent = indexedLabel;
document.getElementById('bitcoinHeight').textContent = bitcoinH > 0 ? bitcoinH.toLocaleString() : 'Checking...';
document.getElementById('networkHeight').textContent = knownHeaderH > 0 ? knownHeaderH.toLocaleString() : 'Checking...';
document.getElementById('indexSize').textContent = data.index_size || '-';
document.getElementById('progressPct').textContent = pct > 0 ? pct.toFixed(1) + '%' : '-';
document.getElementById('currentBlock').textContent = indexedH > 0 ? 'Block ' + indexedH.toLocaleString() : (data.index_size ? 'Index: ' + data.index_size : 'Block 0');
document.getElementById('syncPercentage').textContent = pct > 0 ? pct.toFixed(1) + '%' : '0%';
document.getElementById('progressPct').textContent = (knownHeaderH > 0 || pct > 0) ? pct.toFixed(1) + '%' : '-';
document.getElementById('currentBlock').textContent = currentBlockLabel;
document.getElementById('syncPercentage').textContent = (knownHeaderH > 0 || pct > 0) ? pct.toFixed(1) + '%' : '0%';
document.getElementById('syncProgressBar').style.width = Math.max(pct, 0.5) + '%';
var statusTextEl = document.getElementById('syncStatusText');
@@ -389,14 +415,16 @@
statusTextEl.textContent = data.error || 'Starting up...';
statusTextEl.style.color = '#fbbf24';
statusDot.className = 'status-dot bg-yellow animate-pulse';
document.getElementById('statusText').textContent = 'Starting';
document.getElementById('statusText').textContent = data.status === 'waiting' ? 'Waiting' : 'Starting';
syncIcon.classList.add('animate-spin-slow');
document.getElementById('connSubtitle').textContent = 'Connections will be available once ElectrumX has completed syncing.';
} else if (data.status === 'indexing') {
statusTextEl.textContent = data.error || 'Building index...';
statusTextEl.textContent = data.stale
? (data.error || 'ElectrumX is reconnecting; showing last known indexed height.')
: (data.error || 'Building index. Indexed height will appear when Electrum RPC is ready.');
statusTextEl.style.color = '#fbbf24';
statusDot.className = 'status-dot bg-amber animate-pulse';
document.getElementById('statusText').textContent = 'Indexing';
document.getElementById('statusText').textContent = data.stale ? 'Reconnecting' : 'Indexing';
syncIcon.classList.add('animate-spin-slow');
document.getElementById('connSubtitle').textContent = 'Connections will be available once ElectrumX has completed syncing.';
} else if (data.status === 'error') {
@@ -414,8 +442,10 @@
syncIcon.style.color = '#4ade80';
document.getElementById('connSubtitle').textContent = 'Use the following details to connect your wallet or application to ElectrumX.';
} else {
var remaining = networkH - indexedH;
statusTextEl.textContent = 'Syncing... ' + remaining.toLocaleString() + ' blocks remaining';
var remaining = Math.max(targetH - indexedH, 0);
statusTextEl.textContent = data.error || (targetH > 0
? 'Syncing... ' + remaining.toLocaleString() + ' blocks remaining'
: 'Waiting for Bitcoin network height...');
statusTextEl.style.color = '#fb923c';
statusDot.className = 'status-dot bg-yellow';
document.getElementById('statusText').textContent = 'Syncing';
+317
View File
@@ -0,0 +1,317 @@
# Chat Transcript And Working Notes
Date: 2026-05-02
This file captures the current chat context, decisions, progress, and next steps so work can continue from another device/session.
## User Request
The user asked to continue hardening Archipelago app/container lifecycle, then asked multiple times to save the plan/progress/next steps and finally to save the entire chat to Markdown.
Key user constraints and corrections:
- Continue if next steps are clear; ask only if blocked.
- Exhaustively harden app/container lifecycle before release.
- Preserve data during destructive lifecycle testing unless explicitly instructed otherwise.
- Do not rely on `/app/...` proxy paths for app launch/testing. The user corrected: “we never use paths only ports.”
- LND/Electrum wallet-connect tests must validate real connection details and QR, including Tor.
## Earlier Progress Summary
Before the latest work, the project already had substantial lifecycle hardening in progress:
- Remote lifecycle harness exists at `tests/lifecycle/remote-lifecycle.sh`.
- `.198` SSH works with `/home/archipelago/.ssh/id_ed25519`.
- `.228` RPC works, but SSH is blocked with `Permission denied (publickey,password)`.
- Multiple backend release binaries were built and deployed to `.198` with backups in `/usr/local/bin/archipelago.bak-*`.
- Fixed stale package scanner state recovery from `Removing -> Running` when a container is actually live.
- Fixed startup ordering so crash recovery runs before BootReconciler.
- Removed dangerous automatic Podman runtime directory deletion on `podman info` failure.
- Narrowed generic crash recovery to safe legacy containers.
- Fixed companion reconciliation on install/start/restart.
- Fixed uninstall/reinstall behavior so uninstall disables manifest apps instead of deleting manifest availability, and reinstall re-enables them.
- Fixed LND config generation/repair:
- `bitcoin.active=true`
- `bitcoin.mainnet=true`
- `bitcoin.node=bitcoind`
- `bitcoind.rpchost=bitcoin-knots:8332`
- sudo fallback for writing container-owned config paths.
- `.198` had previously passed focused lifecycle for `filebrowser`, `bitcoin-knots`, and a looser LND launch test.
## Major Files Touched In This Session
- `docs/CONTAINER_LIFECYCLE_HANDOFF.md`
- `docs/CHAT_TRANSCRIPT_2026-05-02.md`
- `tests/lifecycle/remote-lifecycle.sh`
- `core/archipelago/src/container/lnd.rs`
- `core/archipelago/src/container/companion.rs`
- `core/archipelago/src/container/prod_orchestrator.rs`
- `core/archipelago/src/container/docker_packages.rs`
- `core/container/src/podman_client.rs`
- `core/archipelago/src/port_allocator.rs`
- `apps/lnd-ui/manifest.yml`
- `neode-ui/src/views/appSession/appSessionConfig.ts`
- `neode-ui/src/stores/container.ts`
- `neode-ui/src/stores/appLauncher.ts`
- `neode-ui/src/views/appDetails/appDetailsData.ts`
- nginx config/snippet files under `scripts/` and `image-recipe/`
## LND Wallet Bootstrap Investigation
Initial strict LND probe failed because `/lnd-connect-info` could not read `admin.macaroon`:
```text
Failed to read LND admin macaroon — is LND installed?
direct: Permission denied (os error 13)
sudo: cat: /var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon: No such file or directory
```
LND logs showed the wallet was uninitialized/locked:
```text
Waiting for wallet encryption password. Use lncli create...
```
Tests showed `lncli create` is interactive and does not support `--stdin`:
```text
[lncli] flag provided but not defined: -stdin
```
`lncli unlock --stdin` is supported, so the final approach was:
- Use LND REST unlocker endpoints for new wallet creation.
- Use `lncli unlock --stdin` only for an existing wallet.
- Treat “wallet already exists” from REST as a signal to unlock.
- Use sudo-aware checks/reads for wallet artifacts because LND data directories are container-owned and `0700`.
Implemented in `core/archipelago/src/container/lnd.rs`:
- `ensure_wallet_initialized()`
- `file_exists_as_root()`
- `read_file_as_root()`
- `init_wallet_via_rest()`
- `get_lnd_unlocker_json()`
- `post_lnd_unlocker_json()`
- `unlock_existing_wallet()`
- `wait_for_admin_macaroon()`
- `lnd_getinfo_ready()`
Focused Rust test passes:
```bash
cd /home/archipelago/Projects/archy/core
cargo test -p archipelago --bin archipelago lnd
```
Result:
```text
7 passed; 0 failed
```
## LND UI Port Collision
The strict LND UI test then failed with `502`.
Investigation found a real port collision:
- `nostr-rs-relay` uses host `8081`.
- Old `archy-lnd-ui` also used host `8081`.
- nginx `/app/lnd/` proxy also pointed at `8081`.
Fix implemented:
- Move LND UI companion to host port `18083`, container port `80`.
- Keep `nostr-rs-relay` on `8081`.
- Update app metadata/routing to `18083`.
- Update tests to expect direct port launch.
Important correction from user:
```text
we never use paths only ports, how many times do you need to be told
```
Action taken after correction:
- Stop validating through `/app/lnd/` and `/app/electrumx/` in the lifecycle harness.
- Switch `launch_url_for()` to direct app ports.
- Switch app session resolver to direct `http://host:port` launch, even from HTTPS parent pages.
- Remove use of `HTTPS_PROXY_PATHS[id]` in `resolveAppUrl()`.
Direct-port LND audit command:
```bash
ARCHY_HOST=192.168.1.198 ARCHY_PASSWORD=password123 ARCHY_APPS=lnd tests/lifecycle/remote-lifecycle.sh
```
Result:
```text
### 192.168.1.198 iteration 1 / 1 ###
lnd state=running
all checks passed
```
The audit now validates `http://192.168.1.198:18083/`, not `/app/lnd/`.
## Lifecycle Harness Changes
`tests/lifecycle/remote-lifecycle.sh` changes made:
- Normalize package states with `ascii_downcase` because API returned `Running`.
- Direct port launch URLs:
- LND: `http://${ARCHY_HOST}:18083/`
- Electrum/Electrs: `http://${ARCHY_HOST}:50002/`
- Bitcoin UI: `http://${ARCHY_HOST}:8334/`
- Other apps mapped to direct ports where known.
- LND probe checks:
- `Connect Your Wallet`
- `id="lndQrBox"`
- `id="connHost"`
- `value="rest-tor"`
- `value="grpc-tor"`
- `value="rest-local"`
- `value="grpc-local"`
- `Copy lndconnect URI`
- `/lnd-connect-info` cert, macaroon, ports, and Tor onion.
- Electrum probe checks:
- local QR container and address field
- Tor QR container and onion field
- port `50001`
- QR renderer
- direct `http://${ARCHY_HOST}:50002/qrcode.js`
- `/electrs-status` Tor onion.
- Full lifecycle now fails immediately on any failed phase with `|| return 1` so a later reinstall cannot mask a failed restart/probe.
## Deployments To `.198`
Several release builds were made and deployed:
```bash
cd /home/archipelago/Projects/archy/core
cargo build -p archipelago --bin archipelago --release
```
Deploy pattern:
```bash
scp -i /home/archipelago/.ssh/id_ed25519 -o StrictHostKeyChecking=no \
/home/archipelago/Projects/archy/core/target/release/archipelago \
archipelago@192.168.1.198:/tmp/archipelago.new
ssh -i /home/archipelago/.ssh/id_ed25519 -o StrictHostKeyChecking=no \
archipelago@192.168.1.198 \
"sudo cp /usr/local/bin/archipelago /usr/local/bin/archipelago.bak-<timestamp> && \
sudo install -m 0755 /tmp/archipelago.new /usr/local/bin/archipelago && \
sudo systemctl restart archipelago.service && \
systemctl is-active archipelago.service"
```
Latest deploy returned:
```text
active
```
## `.198` Current Observations
After forcing LND package restart, companion reconciliation succeeded:
```text
nostr-rs-relay Up ... 0.0.0.0:8081->8080/tcp
lnd Up ... 0.0.0.0:8080->8080/tcp, 0.0.0.0:9735->9735/tcp, 0.0.0.0:10009->10009/tcp
archy-lnd-ui Up ... 0.0.0.0:18083->80/tcp
```
Direct UI test from `.198` returned `200`:
```bash
curl -i http://127.0.0.1:18083/
```
Strict direct-port LND audit is green:
```text
lnd state=running
all checks passed
```
## Full LND Lifecycle Status
Full direct-port lifecycle was started:
```bash
ARCHY_HOST=192.168.1.198 ARCHY_PASSWORD=password123 ARCHY_APPS=lnd ARCHY_FULL_LIFECYCLE=1 tests/lifecycle/remote-lifecycle.sh
```
It reached:
```text
### 192.168.1.198 iteration 1 / 1 ###
== lnd: install ==
== lnd: stop ==
```
Then the user aborted the command while asking to save memory/transcript.
The next continuation point is to rerun full LND direct-port lifecycle from scratch and inspect the stop phase if it hangs/fails.
## Handoff File
A durable handoff file was also created:
```text
docs/CONTAINER_LIFECYCLE_HANDOFF.md
```
It contains the plan, progress, current blockers, and next steps.
## Immediate Next Steps
1. Rerun full strict LND direct-port lifecycle:
```bash
ARCHY_HOST=192.168.1.198 ARCHY_PASSWORD=password123 ARCHY_APPS=lnd ARCHY_FULL_LIFECYCLE=1 tests/lifecycle/remote-lifecycle.sh
```
2. If it hangs/fails at `stop`, inspect package runtime stop path and logs:
```bash
ssh -i /home/archipelago/.ssh/id_ed25519 -o StrictHostKeyChecking=no archipelago@192.168.1.198 \
'journalctl -u archipelago.service -n 260 --no-pager | egrep -i "package\.(stop|start|restart|install|uninstall)|lnd|companion|error|failed" | sed -n "1,220p"; podman ps -a --format "{{.Names}} {{.Status}} {{.Ports}}" | egrep "lnd|nostr" || true'
```
3. If stop is unreliable, inspect/fix:
- `core/archipelago/src/api/rpc/package/runtime.rs`
- `core/archipelago/src/container/prod_orchestrator.rs`
Likely causes to check:
- Reconciler restarting LND while stop is expected.
- State scanner reporting stale `running`.
- Companion handling interfering with parent app state.
- Async lifecycle returning before actual stop completes.
4. Once LND full lifecycle is green, run Electrum strict lifecycle with direct port `50002`:
```bash
ARCHY_HOST=192.168.1.198 ARCHY_PASSWORD=password123 ARCHY_APPS=electrumx ARCHY_FULL_LIFECYCLE=1 tests/lifecycle/remote-lifecycle.sh
```
5. Continue with app groups after LND/Electrum:
- `filebrowser`
- `bitcoin-knots`
- `lnd`
- `electrumx`
- `mempool`
- `btcpay-server`
- `fedimint`
- remaining catalog apps.
## Important Instruction To Preserve
Use ports only for app launch/testing. Do not add or rely on `/app/...` path proxy launch behavior unless the user explicitly changes this requirement.
File diff suppressed because it is too large Load Diff
+15
View File
@@ -0,0 +1,15 @@
# Release Notes Backlog
## Next Release Required Work
- Backfill missing or thin historical release notes before cutting the next release.
- Audit every `CHANGELOG.md` section from `v1.7.44-alpha` through the current release.
- Replace raw commit-hash entries with user/operator-facing bullets that explain behavior changes, operational impact, validation, and known limitations.
- Ensure `releases/manifest.json` changelog entries come from curated `CHANGELOG.md` notes only.
## Release Note Policy
- Every release must have at least three curated bullets.
- Raw `git log --oneline` output is not acceptable release documentation.
- Notes should answer what changed, why it matters, what operators should expect, and any known limitations.
- `scripts/check-release-manifest.sh` is the enforcement gate before publishing artifacts.
+3
View File
@@ -648,3 +648,6 @@ Go/no-go verdict:
> please do not miss AIUI in the release build or remove it from the nodes whatever you do
- Critical release constraint: AIUI must remain bundled in release artifacts and must never be removed from existing nodes during update/deploy.
> please check the resume files for our latest plan and resume the work.
- Current directive: read the resume/plan files, resume the latest active work, and continue from the recorded release/ISO lane while preserving the AIUI release constraint above.
+1 -1
View File
@@ -1169,7 +1169,7 @@
<div class="card-header"><span class="card-name">btcpay-server</span><span class="card-badge badge-red">archy-net</span></div>
<div class="card-desc">Self-hosted Bitcoin payment processor. Accept Bitcoin payments with invoices, checkout pages, and POS.</div>
<div class="card-layman">Your own payment terminal for Bitcoin. Create invoices, get paid, no middleman taking a cut.</div>
<div class="card-image">btcpayserver:1.13.7</div>
<div class="card-image">btcpayserver:2.3.9</div>
<div class="card-ports">Ports: <span>23000</span></div>
<div class="card-details">
<div class="detail-row"><span class="detail-label">Memory</span><span class="detail-value">1 GB</span></div>
+2 -2
View File
@@ -35,7 +35,7 @@ See the Architecture documentation for detailed system information.
## What's Included
- **Debian Linux Base**: Stable Debian 13 (Trixie) distribution
- **Debian Linux Base**: Debian 13 (Trixie) with security updates applied during ISO/install creation
- **Podman**: Container runtime for apps (rootless by default)
- **Archipelago Backend**: Rust-based API server
- **Archipelago Frontend**: Vue.js web interface
@@ -44,7 +44,7 @@ See the Architecture documentation for detailed system information.
## Build Output
- `results/archipelago-debian-13-x86_64.iso` - Bootable hybrid ISO image
- `results/archipelago-installer-x86_64.iso` - Bootable hybrid ISO image
## Supported Platforms
@@ -211,10 +211,15 @@ check_tools() {
fi
fi
# Ensure insecure registry config for Archipelago app registry (HTTP)
if [ "$CONTAINER_CMD" = "podman" ]; then
# Ensure insecure registry config for Archipelago app registries that are
# intentionally served over HTTP during ISO builds.
if [[ "$CONTAINER_CMD" == podman* ]]; then
mkdir -p /etc/containers/registries.conf.d
cat > /etc/containers/registries.conf.d/archipelago.conf <<'REGCONF'
[[registry]]
location = "146.59.87.168:3000"
insecure = true
[[registry]]
location = "git.tx1138.com"
insecure = true
@@ -227,6 +232,15 @@ check_tools
mkdir -p "$WORK_DIR"
mkdir -p "$OUTPUT_DIR"
container_pull() {
local image="$1"
if [[ "$CONTAINER_CMD" == podman* && "$image" == 146.59.87.168:3000/* ]]; then
$CONTAINER_CMD pull --tls-verify=false --platform "$CONTAINER_PLATFORM" "$image"
else
$CONTAINER_CMD pull --platform "$CONTAINER_PLATFORM" "$image"
fi
}
# =============================================================================
# STEP 1: Build complete root filesystem using Docker
# =============================================================================
@@ -292,7 +306,7 @@ RUN echo "deb http://deb.debian.org/debian trixie main non-free-firmware" > /etc
rm -f /etc/apt/sources.list.d/debian.sources
# Install all packages we need including nginx, podman, tor, and openssl (for self-signed certs)
RUN apt-get update && apt-get install -y --no-install-recommends \
RUN apt-get update && apt-get -y full-upgrade && apt-get install -y --no-install-recommends \
${LINUX_IMAGE_PKG} \
${GRUB_EFI_PKG} \
${GRUB_EFI_SIGNED_PKG} \
@@ -302,9 +316,15 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
dbus \
sudo \
network-manager \
wpasupplicant \
wireless-regdb \
iw \
rfkill \
polkitd \
openssh-server \
nginx \
podman \
catatonit \
uidmap \
slirp4netns \
passt \
@@ -359,13 +379,13 @@ RUN find /usr/share/doc -depth -type f ! -name copyright -delete 2>/dev/null ||
# Install Tailscale from official repo
RUN curl -fsSL https://pkgs.tailscale.com/stable/debian/trixie.noarmor.gpg | tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null && \
curl -fsSL https://pkgs.tailscale.com/stable/debian/trixie.tailscale-keyring.list | tee /etc/apt/sources.list.d/tailscale.list && \
apt-get update && apt-get install -y --no-install-recommends tailscale && \
apt-get update && apt-get -y full-upgrade && apt-get install -y --no-install-recommends tailscale && \
apt-get clean && rm -rf /var/lib/apt/lists/*
# Install FIPS mesh daemon from the .deb built in stage 1. apt-get install
# resolves dependencies from trixie so a cross-dist build still lands cleanly.
COPY --from=fips-builder /tmp/fips.deb /tmp/fips.deb
RUN apt-get update && apt-get install -y --no-install-recommends /tmp/fips.deb && \
RUN apt-get update && apt-get -y full-upgrade && apt-get install -y --no-install-recommends /tmp/fips.deb && \
apt-get clean && rm -rf /var/lib/apt/lists/* && rm /tmp/fips.deb
# Configure locale
@@ -441,8 +461,21 @@ RUN mkdir -p /etc/systemd/system/user@.service.d && \
# Allow unprivileged ping inside rootless containers
RUN printf 'net.ipv4.ping_group_range=0 2147483647\n' > /etc/sysctl.d/90-podman-ping.conf
# Archipelago's web UI manages Wi-Fi through the backend service, not a local
# desktop seat. Allow the dedicated system user to control NetworkManager.
RUN mkdir -p /etc/polkit-1/rules.d && \
printf '%s\n' \
'polkit.addRule(function(action, subject) {' \
' if (subject.user == "archipelago" && action.id.indexOf("org.freedesktop.NetworkManager.") == 0) {' \
' return polkit.Result.YES;' \
' }' \
'});' \
> /etc/polkit-1/rules.d/49-archipelago-networkmanager.rules && \
chmod 644 /etc/polkit-1/rules.d/49-archipelago-networkmanager.rules
# Enable services
RUN systemctl enable NetworkManager || true && \
systemctl enable polkit || systemctl enable polkit.service || true && \
systemctl enable ssh || true && \
systemctl enable nginx || true && \
systemctl enable archipelago || true && \
@@ -682,6 +715,7 @@ kmod,procps,iproute2,ca-certificates,gdisk,\
cryptsetup,cryptsetup-initramfs,parted,dosfstools,e2fsprogs,\
linux-image-${DEB_ARCH},grub-efi-${DEB_ARCH},grub-pc-bin,\
ifupdown,isc-dhcp-client,\
wpasupplicant,wireless-regdb,iw,rfkill,\
pciutils,usbutils,less,nano \
trixie /installer http://deb.debian.org/debian
@@ -693,6 +727,7 @@ mount --bind /proc /installer/proc
mount --bind /sys /installer/sys
mount --bind /dev /installer/dev
chroot /installer apt-get update -qq
chroot /installer apt-get -y -qq full-upgrade
chroot /installer apt-get install -y --no-install-recommends live-boot live-boot-initramfs-tools
chroot /installer apt-get clean
umount /installer/dev 2>/dev/null || true
@@ -1085,12 +1120,15 @@ if [ "$BACKEND_CAPTURED" = "0" ]; then
FROM rust:1.93-trixie as builder
WORKDIR /build
COPY core ./core
COPY scripts ./scripts
COPY image-recipe/configs ./image-recipe/configs
RUN cd core && cargo build --release --bin archipelago
BACKENDFILE
if $CONTAINER_CMD build --platform $CONTAINER_PLATFORM -t archipelago-backend -f "$BACKEND_DOCKERFILE" "$SCRIPT_DIR/.." 2>&1 | tail -20; then
BACKEND_IMAGE="localhost/archipelago-backend:iso"
if $CONTAINER_CMD build --platform $CONTAINER_PLATFORM -t "$BACKEND_IMAGE" -f "$BACKEND_DOCKERFILE" "$SCRIPT_DIR/.."; then
echo " Extracting backend binary..."
BACKEND_CONTAINER=$($CONTAINER_CMD create --platform $CONTAINER_PLATFORM archipelago-backend)
BACKEND_CONTAINER=$($CONTAINER_CMD create --platform $CONTAINER_PLATFORM "$BACKEND_IMAGE")
$CONTAINER_CMD cp "$BACKEND_CONTAINER:/build/core/target/release/archipelago" "$ARCH_DIR/bin/" && \
echo " ✅ Backend binary built ($(du -h "$ARCH_DIR/bin/archipelago" | cut -f1))"
$CONTAINER_CMD rm "$BACKEND_CONTAINER"
@@ -1288,7 +1326,7 @@ if [ "$UNBUNDLED" = "1" ]; then
echo " ✅ Using cached: $CORE_FILE"
else
echo " Pulling $CORE_IMAGE ($CONTAINER_PLATFORM)..."
if $CONTAINER_CMD pull --platform $CONTAINER_PLATFORM "$CORE_IMAGE"; then
if container_pull "$CORE_IMAGE"; then
$CONTAINER_CMD save "$CORE_IMAGE" -o "$IMAGES_DIR/$CORE_FILE" 2>/dev/null && \
echo " ✅ Saved core: $CORE_FILE ($(du -h "$IMAGES_DIR/$CORE_FILE" | cut -f1))" || \
echo " ⚠️ Failed to save $CORE_IMAGE"
@@ -1366,7 +1404,7 @@ echo "$CONTAINER_IMAGES" | while read -r image filename; do
echo " ✅ Using cached: $filename"
else
echo " Pulling $image ($CONTAINER_PLATFORM)..."
if $CONTAINER_CMD pull --platform $CONTAINER_PLATFORM "$image"; then
if container_pull "$image"; then
echo " Saving $filename..."
if $CONTAINER_CMD save "$image" -o "$tarpath" 2>/dev/null; then
echo " ✅ Saved: $(du -h "$tarpath" | cut -f1)"
@@ -3455,9 +3493,9 @@ echo ""
echo "Step 6: Creating bootable ISO..."
if [ "$UNBUNDLED" = "1" ]; then
OUTPUT_ISO="$OUTPUT_DIR/archipelago-installer-unbundled-${ARCH}.iso"
OUTPUT_ISO="$OUTPUT_DIR/archipelago-installer-${BUILD_VERSION}-unbundled-${ARCH}.iso"
else
OUTPUT_ISO="$OUTPUT_DIR/archipelago-installer-${ARCH}.iso"
OUTPUT_ISO="$OUTPUT_DIR/archipelago-installer-${BUILD_VERSION}-${ARCH}.iso"
fi
# Use the proven MBR code for hybrid USB boot
@@ -157,13 +157,23 @@ EOF
echo "📥 Updating package lists..."
chroot /mnt/archipelago apt-get update
echo "🔒 Applying Debian security updates..."
chroot /mnt/archipelago apt-get -y full-upgrade
echo "📦 Installing kernel and bootloader..."
chroot /mnt/archipelago apt-get install -y linux-image-amd64 grub-efi-amd64 grub-efi-amd64-signed shim-signed
echo "📦 Installing essential packages..."
chroot /mnt/archipelago apt-get install -y \
sudo \
networkmanager \
network-manager \
wpasupplicant \
wireless-regdb \
iw \
rfkill \
pciutils \
usbutils \
polkitd \
openssh-server \
curl \
wget \
@@ -173,7 +183,7 @@ chroot /mnt/archipelago apt-get install -y \
chrony
echo "📦 Installing container tools..."
chroot /mnt/archipelago apt-get install -y podman || echo "⚠️ Podman not available in base repos, will use containers.io later"
chroot /mnt/archipelago apt-get install -y podman catatonit || echo "⚠️ Podman/catatonit not available in base repos, will use containers.io later"
echo "🔧 Installing GRUB bootloader..."
# Need to run grub-install inside chroot with proper environment
@@ -195,9 +205,20 @@ echo "archipelago:archipelago" | chroot /mnt/archipelago chpasswd
echo "⚙️ Enabling services..."
chroot /mnt/archipelago systemctl enable NetworkManager || true
chroot /mnt/archipelago systemctl enable polkit || chroot /mnt/archipelago systemctl enable polkit.service || true
chroot /mnt/archipelago systemctl enable ssh || chroot /mnt/archipelago systemctl enable sshd || true
chroot /mnt/archipelago systemctl enable chrony || true
mkdir -p /mnt/archipelago/etc/polkit-1/rules.d
cat > /mnt/archipelago/etc/polkit-1/rules.d/49-archipelago-networkmanager.rules <<'EOF'
polkit.addRule(function(action, subject) {
if (subject.user == "archipelago" && action.id.indexOf("org.freedesktop.NetworkManager.") == 0) {
return polkit.Result.YES;
}
});
EOF
chmod 644 /mnt/archipelago/etc/polkit-1/rules.d/49-archipelago-networkmanager.rules
# Remove policy-rc.d so services can start on first boot
rm -f /mnt/archipelago/usr/sbin/policy-rc.d
@@ -56,7 +56,6 @@ listen=1
# RPC
rpcuser=$RPC_USER
rpcpassword=$RPC_PASS
rpcbind=0.0.0.0
rpcallowip=10.0.0.0/8
rpcallowip=172.16.0.0/12
rpcallowip=192.168.0.0/16
@@ -110,7 +110,7 @@ ExecStart=/usr/bin/podman run --name lnd \\
-v ${LND_DATA}:/data/.lnd:Z \\
-p 9735:9735 \\
-p 10009:10009 \\
-p 8080:8080 \\
-p 18080:8080 \\
docker.io/lightninglabs/lnd:v0.18.0-beta \\
--configfile=/data/.lnd/lnd.conf
ExecStop=/usr/bin/podman stop lnd
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# Build the Archipelago Debian installer ISO.
#
# The historical ISO builder remains archived because OTA tarballs are the
# normal release path. This wrapper keeps the documented ISO command working
# by running a temporary active-layout copy of that builder with fixed paths.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ARCHIVED_BUILDER="$SCRIPT_DIR/_archived/build-auto-installer-iso.sh"
TMP_DIR="$(mktemp -d -t archipelago-iso-builder.XXXXXX)"
cleanup() {
rm -rf "$TMP_DIR"
}
trap cleanup EXIT
if [ ! -f "$ARCHIVED_BUILDER" ]; then
echo "Archived ISO builder not found: $ARCHIVED_BUILDER" >&2
exit 1
fi
TMP_BUILDER="$TMP_DIR/build-auto-installer-iso.sh"
cp "$ARCHIVED_BUILDER" "$TMP_BUILDER"
# The archived builder lived one directory deeper at image-recipe/_archived/.
# Rewrite only path expressions that were relative to that old location.
perl -0pi -e 's#SCRIPT_DIR="\$\(cd "\$\(dirname "\$0"\)" && pwd\)"#SCRIPT_DIR="__ARCHIPELAGO_IMAGE_RECIPE_DIR__"#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./\.\./scripts#\$SCRIPT_DIR/../scripts#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./configs#\$SCRIPT_DIR/configs#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./docker#\$SCRIPT_DIR/../docker#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./neode-ui#\$SCRIPT_DIR/../neode-ui#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./web#\$SCRIPT_DIR/../web#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./demo#\$SCRIPT_DIR/../demo#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./apps#\$SCRIPT_DIR/../apps#g' "$TMP_BUILDER"
perl -0pi -e 's#\$SCRIPT_DIR/\.\./core#\$SCRIPT_DIR/../core#g' "$TMP_BUILDER"
perl -0pi -e 's#"\$\(dirname "\$0"\)/\.\./\.\./scripts#"$(dirname "$0")/../scripts#g' "$TMP_BUILDER"
perl -0pi -e "s#__ARCHIPELAGO_IMAGE_RECIPE_DIR__#${SCRIPT_DIR}#g" "$TMP_BUILDER"
chmod +x "$TMP_BUILDER"
exec bash "$TMP_BUILDER" "$@"
+4 -1
View File
@@ -1,6 +1,9 @@
# Stable symlink for USB serial adapters used as mesh radios.
# Creates /dev/mesh-radio pointing to the underlying ttyUSB device.
# Supports: CP2102 (Heltec V3), CH340 (T-Beam), FTDI (RAK WisBlock).
# Supports MeshCore and Meshtastic radios using CP2102 (Heltec V3),
# CH340 (T-Beam), FTDI (RAK WisBlock), and known USB CDC ACM radios.
SUBSYSTEM=="tty", ATTRS{idVendor}=="10c4", ATTRS{idProduct}=="ea60", SYMLINK+="mesh-radio", MODE="0660", GROUP="dialout"
SUBSYSTEM=="tty", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="7523", SYMLINK+="mesh-radio", MODE="0660", GROUP="dialout"
SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", SYMLINK+="mesh-radio", MODE="0660", GROUP="dialout"
SUBSYSTEM=="tty", ATTRS{idVendor}=="239a", KERNEL=="ttyACM[0-9]*", SYMLINK+="mesh-radio", MODE="0660", GROUP="dialout"
SUBSYSTEM=="tty", ATTRS{idVendor}=="2e8a", KERNEL=="ttyACM[0-9]*", SYMLINK+="mesh-radio", MODE="0660", GROUP="dialout"
@@ -7,6 +7,6 @@ Type=oneshot
# Runs as root: needs to kill orphaned conmon processes, fix permissions
User=root
ExecStart=/home/archipelago/archy/scripts/container-doctor.sh --local
TimeoutStartSec=120
TimeoutStartSec=300
StandardOutput=journal
StandardError=journal
@@ -2,9 +2,11 @@
Description=Archipelago container doctor (periodic)
[Timer]
# First run 5 minutes after boot, then every 30 minutes
OnBootSec=5min
OnUnitActiveSec=30min
# First run 2 minutes after boot, then every 5 minutes. The doctor is
# idempotent and exits quickly when no drift exists; this keeps vanished
# rootless port listeners and stopped containers from remaining broken.
OnBootSec=2min
OnUnitActiveSec=5min
# Jitter to avoid load spikes
RandomizedDelaySec=60
+4 -3
View File
@@ -7,10 +7,11 @@ Wants=network-online.target
Type=notify
User=archipelago
Environment="ARCHIPELAGO_BIND=127.0.0.1:5678"
Environment="ARCHIPELAGO_USE_QUADLET_BACKENDS=true"
# DEV_MODE disabled in production — enabled via override.conf on dev servers
Environment="XDG_RUNTIME_DIR=/run/user/1000"
# + prefix runs these as root (needed for chown/mkdir outside ReadWritePaths)
ExecStartPre=+/bin/bash -c 'mkdir -p /run/user/1000 /run/containers /var/lib/containers && chown archipelago:archipelago /run/user/1000 && chmod 700 /run/user/1000'
ExecStartPre=+/bin/bash -c 'mkdir -p /run/user/1000 /var/lib/containers && chown archipelago:archipelago /run/user/1000 && chmod 700 /run/user/1000'
ExecStartPre=+/bin/bash -c 'mkdir -p /var/lib/archipelago && chown archipelago:archipelago /var/lib/archipelago && echo "ARCHIPELAGO_HOST_IP=$(hostname -I 2>/dev/null | awk "{print $$1}")" > /var/lib/archipelago/host-ip.env && chown archipelago:archipelago /var/lib/archipelago/host-ip.env'
ExecStart=/usr/local/bin/archipelago
Restart=on-failure
@@ -26,13 +27,13 @@ ProtectSystem=strict
ProtectHome=no
# PrivateTmp disabled: rootless podman runtime lives in /tmp/podman-run-UID/
# and must be shared between the service and SSH-created containers
ReadWritePaths=/var/lib/archipelago /etc/containers /var/lib/containers /run/containers /run/user /tmp /home/archipelago/.local/share/containers /home/archipelago/.config/containers /etc
ReadWritePaths=/var/lib/archipelago /etc/containers /var/lib/containers /run/user /tmp /home/archipelago/.local/share/containers /home/archipelago/.config/containers /etc
# Privilege restriction — NoNewPrivileges=no required for sudo archipelago-wg
# (WireGuard peer management). Scoped via sudoers to only archipelago-wg.
NoNewPrivileges=no
PrivateDevices=no
SupplementaryGroups=dialout debian-tor
SupplementaryGroups=dialout debian-tor fips
# Syscall and network restrictions — safe on Debian 13 (systemd 256+)
# which respects NoNewPrivileges=no as an explicit override for seccomp filters
+64 -4
View File
@@ -8,7 +8,7 @@ resolver 1.1.1.1 8.8.8.8 valid=300s ipv6=off;
resolver_timeout 5s;
server {
listen 80;
listen 80 default_server;
server_name _;
root /opt/archipelago/web-ui;
@@ -23,6 +23,13 @@ server {
add_header X-DNS-Prefetch-Control "off" always;
add_header Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https://*.basemaps.cartocdn.com https://tile.openstreetmap.org; font-src 'self' data:; connect-src 'self' ws: wss: http://$host:* https:; frame-src 'self' http://$host:* https:; frame-ancestors 'self'; base-uri 'self'; form-action 'self';" always;
# Serve Nginx Proxy Manager HTTP-01 challenge files before the SPA fallback.
location ^~ /.well-known/acme-challenge/ {
default_type text/plain;
root /var/lib/archipelago/nginx-proxy-manager/data/letsencrypt-acme-challenge;
try_files $uri =404;
}
# AIUI SPA (Chat mode iframe) — SPA fallback for client-side routing
location /aiui/ {
try_files $uri $uri/ /aiui/index.html;
@@ -105,9 +112,25 @@ server {
try_files $uri =404;
}
# Versioned Vite assets must never fall through to index.html. During OTA
# a browser can keep an old HTML shell that references now-removed hashed
# chunks; returning HTML for /assets/*.js triggers strict MIME failures.
# A real 404 plus immutable/no-cache split lets the app/browser recover on
# refresh without caching the wrong content type.
location /assets/ {
try_files $uri =404;
add_header Cache-Control "public, max-age=31536000, immutable" always;
}
location ~* ^/(registerSW\.js|sw\.js|workbox-[^/]+\.js)$ {
try_files $uri =404;
add_header Cache-Control "no-cache, must-revalidate" always;
}
# Serve static files (Vue.js SPA)
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control "no-cache, must-revalidate";
}
# Peer-to-peer node messaging (receives from other nodes over Tor)
@@ -156,6 +179,16 @@ server {
error_page 502 503 = @backend_unavailable;
error_page 504 = @backend_timeout;
}
location /bitcoin-status {
proxy_pass http://127.0.0.1:5678/bitcoin-status;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_connect_timeout 10s;
proxy_read_timeout 10s;
proxy_send_timeout 5s;
error_page 502 503 = @backend_unavailable;
error_page 504 = @backend_timeout;
}
location /electrs-status {
proxy_pass http://127.0.0.1:5678/electrs-status;
proxy_http_version 1.1;
@@ -544,7 +577,7 @@ server {
add_header Referrer-Policy strict-origin-when-cross-origin always;
}
location /app/lnd/ {
proxy_pass http://127.0.0.1:8081/;
proxy_pass http://127.0.0.1:18083/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
@@ -731,7 +764,7 @@ server {
sub_filter '</head>' '<script src="/nostr-provider.js"></script></head>';
}
location /app/nginx-proxy-manager/ {
proxy_pass http://127.0.0.1:81/;
proxy_pass http://127.0.0.1:8081/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
@@ -863,9 +896,26 @@ server {
}
}
# Compatibility proxy for cached PWA bundles that still launch Nginx Proxy
# Manager on :81. Rootless Podman cannot bind host ports below 1024, so the
# container admin UI runs on :8081 and host nginx owns the old :81 entrypoint.
server {
listen 81;
server_name _;
location / {
proxy_pass http://127.0.0.1:8081/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# HTTPS - required for PWA install (Add to Home Screen) from dev servers
server {
listen 443 ssl;
listen 443 ssl default_server;
server_name _;
ssl_certificate /etc/archipelago/ssl/archipelago.crt;
@@ -969,6 +1019,16 @@ server {
error_page 502 503 = @backend_unavailable;
error_page 504 = @backend_timeout;
}
location /bitcoin-status {
proxy_pass http://127.0.0.1:5678/bitcoin-status;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_connect_timeout 10s;
proxy_read_timeout 10s;
proxy_send_timeout 5s;
error_page 502 503 = @backend_unavailable;
error_page 504 = @backend_timeout;
}
location /electrs-status {
proxy_pass http://127.0.0.1:5678/electrs-status;
proxy_http_version 1.1;
@@ -102,7 +102,7 @@ location /app/endurain/ {
sub_filter '</head>' '<script src="/nostr-provider.js"></script></head>';
}
location /app/lnd/ {
proxy_pass http://127.0.0.1:8081/;
proxy_pass http://127.0.0.1:18083/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
@@ -358,7 +358,7 @@ location /app/indeedhub/ {
sub_filter '</head>' '<script src="/nostr-provider.js"></script></head>';
}
location /app/nginx-proxy-manager/ {
proxy_pass http://127.0.0.1:81/;
proxy_pass http://127.0.0.1:8081/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
+6 -1
View File
@@ -20,13 +20,18 @@ if [ -z "$1" ]; then
fi
USB_DISK="$1"
ISO_FILE="$SCRIPT_DIR/results/archipelago-debian-13-x86_64.iso"
ISO_FILE="${ARCHIPELAGO_ISO:-}"
if [ -z "$ISO_FILE" ]; then
ISO_FILE="$SCRIPT_DIR/results/archipelago-installer-x86_64.iso"
[ -f "$ISO_FILE" ] || ISO_FILE="$SCRIPT_DIR/results/archipelago-installer-unbundled-x86_64.iso"
fi
WORK_DIR="$SCRIPT_DIR/build/usb-extract"
if [ ! -f "$ISO_FILE" ]; then
echo "❌ ISO not found: $ISO_FILE"
echo ""
echo "Build the ISO first with: ./build-debian-iso.sh"
echo "Or set ARCHIPELAGO_ISO=/path/to/archipelago-installer-x86_64.iso"
exit 1
fi

Some files were not shown because too many files have changed in this diff Show More