Commit Graph
52 Commits
Author SHA1 Message Date
archipelagoandClaude Fable 5 ada59acdd5 fix(appgate+container): stop stripping app cookies; create named volumes correctly
Two daemon bugs, one debugging arc (2026-08-05, operator-reported):

1. The app gate removed the ENTIRE Cookie header before proxying. That
   broke the data plane of every first-party companion UI behind the gate
   (lnd-ui/bitcoin-ui/electrs-ui/fips-ui render their shell, then every
   /proxy/* and /lnd-connect-info call 401s — observed as "LND UI
   unreachable"), and silently logged users out of every gated app with
   its own cookie login (vaultwarden, nextcloud, gitea) on each request.
   The gate now strips only its own cookie pairs (session, csrf_token);
   a new per-port manifest opt-in `session_passthrough: true` forwards
   the node session to first-party UIs whose nginx proxies the daemon's
   authenticated endpoints. Undeclared ports never get passthrough.

2. podman_client::create_container sent named volumes to the libpod API
   as bind mounts with the bare volume name as source, so creating any
   manifest app with a `type: volume` mount failed. On .38 the reconciler
   removed indeedhub-postgres/-minio for env drift and then could never
   create their replacements, leaving the stack half-missing forever.
   Named volumes now ride the spec's `volumes` field ({Name, Dest,
   Options}). Also: the reconcile-failure log now prints the full anyhow
   chain — `%e` showed only "create_container X" and hid the real error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 21:21:52 -04:00
archipelagoandClaude Opus 5 ab2c8b6e96 fix(security): silence is not consent — undeclared ports are never acted on
Two live incidents on archi-dev-box today, one bug. Both times a safety
decision read an ABSENT manifest field as if it were a value, and a
node's installed manifests always lag the binary — so "absent" is the
state of essentially every port on every node.

  1. Gating any `session` port regardless of `bind` published Bitcoin's
     loopback-only RPC 8332 on the LAN, Tailscale and IPv6 within seconds
     of deploy.
  2. The `bind`-keyed replacement looked safe because it protected
     `bind: 127.0.0.1` ports — but LND's gRPC 10009 and REST 18080 carry
     an EMPTY bind, so they fell through. One container recreate from
     pinning them to loopback and breaking Zeus and every remote wallet.

`auth` is now `Option<PortAuth>`, separating two questions that were
conflated:

  * `auth_policy()` — what to CLASSIFY the port as. Undeclared reports as
    Session, i.e. shows in the audit as something that should be behind
    the gate. Reporting is always safe.
  * `auth_is_declared()` — whether the daemon may ACT. Only an explicit
    declaration authorises changing how a port is published.

Also reverts the daemon-side publish rewriting entirely. The node proved
it wrong twice over: the recreate path that actually ran was in
package::install, not podman_client, so the pin never fired; and even
`bind: 127.0.0.1` written directly into the node's manifest was
overridden by the signed catalog. Publishes are built in several places
and all of them already honour `bind`, so the migration belongs in the
catalog as data — not in daemon-side inference that can only ever cover
one path and guess wrong on the rest.

Tests: 75/75 container, incl. the LND wallet-port shape (`host: 10009`,
empty bind, no auth) asserted to be non-actionable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 00:36:43 -04:00
archipelagoandClaude Fable 5 e5c7210663 fix(install): stall-aware, retried image pulls in the orchestrator runtime
PodmanClient::pull_image had a single attempt with a hard 600s wall
clock — on a fresh node, any image that legitimately takes >10 min to
download got killed mid-pull and failed the install. Podman keeps the
completed layers, so the user's retry started half-done and succeeded:
the exact "first install fails, second works" pattern reported for
btcpay and indeedhub (and earlier for LND, whose fix in 72d7fa07 only
covered the legacy install path — this is the same cure ported to the
orchestrator path both fresh installs and upgrades use).

A pull is now only killed when nothing is observably happening — no
stderr output AND no staged-byte growth in TMPDIR for 180s — or at a
1800s absolute ceiling, with 3 attempts and backoff on top. Dead
registries still fail fast (no bytes ever land). Failures now surface
podman's actual stderr tail instead of a bare exit status, and the
pull stages via the user's containers tmp like every other pull path
(rootless /var/tmp is read-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:02:53 -04:00
DorianandClaude Fable 5 4b0e44fc6c chore: purge retired git.tx1138.com registry host from the codebase
The tx1138 Gitea was retired as a release server 2026-06-13 and its registry
frontend is fully dead (500 on every /v2 manifest read, observed 2026-07-10).
Nothing may reference it anymore:

- registry.rs: no longer a default registry, no longer force-enabled on
  load; saved configs are stripped of it on load (same one-time migration
  treatment as the decommissioned Hetzner mirror), with a regression test.
- image_policy.rs: removed from TRUSTED_REGISTRIES — refs through the dead
  host are now refused at the pull site (rejection test added).
- api/handler: dropped the legacy catalog-proxy fallback URL.
- .gitmodules: indeedhub submodule repointed to the OVH Gitea.
- scripts, image-recipe, app-catalog data, neode-ui strings, docs, and all
  test fixtures repointed to 146.59.87.168:3000 (or neutral example hosts).
- image-versions.sh: ARCHY_REGISTRY_FALLBACK emptied (guarded consumers
  skip it); reconcile-containers.sh candidate guard hardened.

The only remaining occurrences of the host string are the strip/reject
enforcement paths and their regression tests — the code that guarantees it
is never used again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 18:55:32 +01:00
archipelagoandClaude Fable 5 eb6ec71a56 fix(security): drop unbindable publish binds instead of crash-looping the app
The dd61a204 bind hardening published Bitcoin RPC on the archy-net gateway
10.89.0.1 — but rootlessport binds in the HOST netns, where that address
does not exist. First real deploy (.228, 2026-07-09) crash-looped
bitcoin-knots AND bitcoin-core the moment the gate's stop→start regenerated
the unit from the re-signed catalog: 'rootlessport listen tcp
10.89.0.1:8332: bind: cannot assign requested address', restart counter 132.
Hand-edits to the unit don't survive — the orchestrator regenerates it from
the signed catalog manifest within seconds.

- New archipelago_container::manifest::host_can_bind_publish_ip(): empty/
  wildcard/loopback accepted without probing, anything else ephemeral-bind
  probed. Applied at all three publish paths — quadlet from_manifest
  (PublishPort), podman API create (host_ip), and the legacy -p string
  table loop — each dropping the publish with a warn instead of taking the
  container down. This neutralizes the bad binds already in the SIGNED
  catalog, so nodes recover on binary deploy alone (no ceremony needed).
- Remove the gateway publishes from bitcoin-knots/-core manifests and the
  legacy config.rs tables: verified on .228 that every in-node consumer
  (lnd, btcpay/nbxplorer, fedimint, mempool-api) dials the container's
  archy-net alias directly (bitcoin-knots:8332) and lnd uses RPC polling
  (no ZMQ) — the gateway publish had zero consumers. Loopback-only RPC/ZMQ
  (the approved LAN lockdown) stands; P2P 8333 stays public.

Catalog still ships the gateway binds until the next signing ceremony
regenerates it from these manifests; the guard makes that non-urgent.

Tests: container crate 65/65 (2 new guard tests), quadlet 40/40.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:41:39 -04:00
archipelagoandClaude Fable 5 dd61a20413 feat(security): bind Bitcoin RPC/ZMQ publishes to loopback + archy-net gateway (§C)
USER DECISION 2026-07-08: accept breaking external wallets pointed at
nodeIP:8332. The 0.0.0.0 publish exposed auth-only RPC (and unauthenticated
ZMQ 28332/28333 on the legacy path) to the whole LAN.

- New `bind` field on manifest port mappings (validated as an IP; the same
  host port may repeat with distinct binds). Rendered as
  PublishPort=<ip>:<host>:<container> in quadlet units and host_ip in the
  podman API create — unbound ports render byte-identical to before, so no
  fleet-wide false-drift wave.
- bitcoin-knots/-core manifests publish 8332 on 127.0.0.1 + 10.89.0.1 only.
  In-node consumers (lnd, fedimint-gateway, btcpay/nbxplorer) are unaffected:
  they dial host.archipelago / host.containers.internal, which the
  orchestrator pins to the archy-net gateway 10.89.0.1. P2P 8333 stays public.
- Legacy config.rs port strings get the same treatment incl. ZMQ.

Tests: new quadlet bind-render + manifest bind-validation tests;
container:: suite 167/167, archipelago-container 63/63.

DEPLOY NOTE: PublishPort strings change for bitcoin containers → one
planned recreate per node; restart lnd afterwards (it caches the backend
IP — see the backend-recreate cascade tracker item). Catalog manifests for
bitcoin apps must be regenerated + re-signed for catalog-covered nodes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 20:29:02 -04:00
archipelagoandClaude Fable 5 4665e497d7 feat(security): move secret env out of podman inspect and Quadlet unit files
Secret env used to merge into manifest.app.environment, landing in
'podman inspect' Config.Env on the API backend and — worse — as
plaintext Environment= lines in Quadlet unit files on disk. Now:

- expand_and_partition_env (container crate, pure + tested) expands
  ${KEY} placeholders and splits env into plain entries and
  secret-bearing pairs. Plain entries that interpolate a secret
  (btcpay's Password=${BTCPAY_DB_PASS} connection strings) are
  tainted and travel as secrets too. Secret values themselves are
  never expanded (a generated value containing '${' passes verbatim).
- values register as podman secrets: stdin (never argv/tempfile),
  --replace, content-hash label to skip no-op rewrites; a per-app hash
  cache in the orchestrator makes steady-state reconciles free of
  podman secret calls. Registration goes through the runtime trait
  (default no-op keeps mocks/docker inert).
- containers reference secrets by name: secret_env map in the libpod
  create spec, Secret=<name>,type=env,target=<KEY> in Quadlet units.
  Verified empirically on fleet podman 5.4.2: value absent from
  inspect Config.Env, runtime injection works rootless.
- rotation detection: io.archipelago.secret-env-hash container label
  (API) / the changed unit bytes (Quadlet). Pre-upgrade containers
  lack the label, so every secret-bearing app recreates ONCE on the
  first reconcile after deploy — deliberate, it scrubs the plaintext
  secrets out of existing container configs. Data dirs untouched.
- docker dev fallback keeps plain -e injection (no secret store);
  podman secrets persist across uninstall, matching the
  preserve-credentials invariant (reinstall re-registers by hash).

In-container /proc/<pid>/environ is unchanged — env remains the
app-compat contract; the closed leaks are inspect output and unit
files on disk.

Tests: archipelago-container 61/61 (3 new: taint partition, verbatim
secrets, hash order-independence), archipelago container:: 160/160
(fedimint install test now asserts the secret arrives as a ref, not
env; quadlet render test asserts Secret=/Label= lines). NEEDS the
on-node gate re-run before the item counts as verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 13:55:15 -04:00
archipelagoandClaude Fable 5 eed830e1ee feat(security): enforce declared cosign image signatures at the pull sites
New container::image_verify gates PodmanClient::pull_image and the
dev-only DockerRuntime::pull_image. Signature claims classify three
ways: absent/empty (pull unverified, logged), the literal
'cosign://...' placeholder every fleet manifest carries today (same —
enforcement stays dormant until the signing ceremony ships real
values), or a declared signature, which must verify via
'cosign verify --key /etc/archipelago/cosign.pub
--insecure-ignore-tlog=true' (plus --allow-insecure-registry
--allow-http-registry for the HTTP mirror; flags checked against
cosign's own docs) before anything is fetched. Missing key, missing
cosign binary, timeout, or verification failure all hard-fail the
pull — a declared signature cannot be skipped on either runtime. Key
path overridable via ARCHIPELAGO_COSIGN_PUBKEY for tests/staging.

Deletes security::ImageVerifier: zero callers, blocking
std::process::Command on would-be async paths, and a fantasy
'cosign verify --signature' invocation (that flag belongs to
verify-blob).

Activation ships with the Workstream B ceremony, in order: pin
cosign.pub on nodes + install cosign, then publish real
image_signature values in the catalog.

Tests: archipelago-container 58/58 (5 new), archipelago container::
159/159, security check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 18:11:32 -04:00
archipelagoandClaude Opus 4.8 670ebb0666 fix(launcher): pin Gitea launch URL to web port 3001 (not SSH 2222)
Gitea publishes two host ports — SSH on 2222 and the web UI on 3001.
The launch URL comes from manifest_lan_address_for() (the manifest's
interfaces.main → 3001), but Gitea had no entry in the static
lan_address_for() fallback map. On a node where the gitea manifest is
absent or stale (no interfaces block), the lookup returns None and the
code falls through to extract_lan_address(), which returns whichever
port podman lists first — frequently the SSH port. Result: the app
launched at :2222 instead of :3001 (observed on tailscale node
100.82.34.38).

Add the canonical "gitea" => http://localhost:3001 entry to the static
map, matching every other core app, so the web UI is pinned regardless
of manifest presence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 03:16:41 -04:00
archipelagoandClaude Opus 4.8 a8b9b0f5e8 feat(netbird): manifest-driven migration via reusable orchestrator primitives
Migrate the netbird stack (server/dashboard/proxy) off ~500 lines of per-app Rust
to 3 declarative manifests, adding 4 reusable primitives:
- SecretGenKind::Base64 (netbird relay authSecret + sqlite store encryptionKey)
- GeneratedCert schema + ensure_manifest_certs (self-signed TLS so the dashboard
  gets a secure context for OIDC PKCE — issue #15; https proxy on 8087 preserved)
- templated GeneratedFile render: {{HOST_IP}}/{{HOST_MDNS}}/{{NETWORK_GATEWAY}}
  (aardvark resolver for the #15 stale-IP fix) /{{secret:NAME}} (never logged)
- legacy create_container now honours port.protocol (3478/udp STUN)
install_netbird_stack routes via the orchestrator first (legacy kept as fallback,
mirroring indeedhub); launch URL derives https://{host_ip}:8087 from host facts.
Legacy Rust deletion deferred to post-live-verify.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 13:39:53 -04:00
archipelagoandClaude Opus 4.8 2dad64b2ee fix(stop): honour per-app graceful-stop grace in orchestrator stop path
package.stop left slow-to-SIGTERM apps (fedimint/electrumx/bitcoin/btcpay/immich)
running: the orchestrator path hardcoded podman API ?t=10 / CLI -t 30 and the CLI
wrapper deadline (30s) equalled the -t grace, so the await fired exactly as podman
SIGKILLed -> stop reported failed -> state reverted to running. Reproduced live on
clean .198 (fedimint).

- container/runtime.rs: add ContainerRuntime::stop_container_with_grace (defaulted
  so mock/dev impls are unchanged); PodmanRuntime honours grace for API + CLI with
  deadline = grace + 15s buffer; AutoRuntime delegates. New canonical per-app table
  stop_grace_secs_for() + DEFAULT_STOP_GRACE_SECS / STOP_GRACE_DEADLINE_BUFFER_SECS.
- podman_client.rs: stop_container_with_grace uses ?t=<grace> + longer HTTP deadline.
- prod_orchestrator::stop: resolve grace = manifest stop_grace_secs (north-star) else
  the table; pass to quadlet::stop_service_with_timeout AND stop_container_with_grace.
- quadlet.rs: stop_service_with_timeout so slow apps aren't SIGKILLed at 45s.
- rpc/package/runtime.rs: doc-note its &str stop_timeout_secs mirrors the canonical table.
- tests: resolve_stop_grace_secs (manifest field wins / table fallback / default 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 06:59:40 -04:00
archipelagoandClaude Opus 4.8 b94b61f640 feat(manifest): network_aliases — extra DNS aliases on a container's network
Add `container.network_aliases: Vec<String>` (serde default, DNS-label
validated) so a stack member can answer to short hostnames its peers bake
in, beyond its own container name. Rendered in both runtime paths:
- podman_client: merged (deduped) into the custom-network aliases array.
- quadlet from_manifest: appended after the container name; emitted only
  for Bridge networks (slirp/pasta reject aliases).

Needed for the indeedhub migration: its frontend nginx proxies to
`api:4000` / `minio:9000` / `relay:8080`, so those members declare
`network_aliases: [api|minio|relay]` to keep the short names resolvable on
the dedicated indeedhub-net (vs. colliding generic aliases on archy-net).

Also fixes 4 pre-existing from_manifest test failures (unrelated to this
change, surfaced now that the quadlet suite runs green): test manifests
used the long-invalid `network_policy: archy-net` (allowlist is
isolated/bridge/host → moved to network_policy: isolated + container.network)
and bind sources outside /var/lib/archipelago.

Tests: container crate 53 pass; archipelago quadlet+alias 47 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 15:45:11 -04:00
archipelago 63b98599e8 Revert "fix(fedimint): run fmcd with seccomp=unconfined so its DHT can start (#7)"
This reverts commit 409543c41e.
2026-06-20 14:37:24 -04:00
archipelagoandClaude Opus 4.8 409543c41e fix(fedimint): run fmcd with seccomp=unconfined so its DHT can start (#7)
fmcd crash-looped "Operation not permitted (os error 1)" on .116 (kernel
6.12.74): the default rootless seccomp profile blocks a syscall its Mainline-DHT
/ iroh transport needs, so the REST API never came up (:8178 → HTTP 000) and
federations couldn't be joined. Verified: with seccomp=unconfined fmcd boots and
answers /v2/* (HTTP 401 instead of dead). fmcd works on other nodes, so this is
kernel/seccomp-specific — but the relaxation is safe for an outbound-networking
daemon and harmless where not needed.

- new `security.seccomp_unconfined` manifest flag (SecurityPolicy);
- libpod backend sets `seccomp_profile_path: "unconfined"` (== --security-opt
  seccomp=unconfined); quadlet backend emits `SeccompProfile=unconfined`;
- enabled in apps/fedimint-clientd/manifest.yml.

NOTE: manifests live on-disk at /opt/archipelago/apps/<id>/manifest.yml, so the
node needs the updated manifest deployed + the fmcd container recreated to apply.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:08:13 -04:00
archipelago d6f108d818 chore: snapshot release workspace 2026-06-12 03:00:15 -04:00
archipelago 881478a873 app-platform: type manifest launch interfaces 2026-06-11 00:52:16 -04:00
archipelago 755ba5562d app-platform: derive launch URLs from manifests 2026-06-11 00:33:24 -04:00
archipelago c393b96da3 backend: harden rootless app lifecycle orchestration 2026-06-11 00:24:32 -04:00
archipelago 34c4e87d14 feat(apps): add saleor storefront 2026-05-20 23:02:57 -04:00
archipelago 92c58141af fix(apps): stabilize saleor and netbird launch 2026-05-19 21:45:17 -04:00
archipelago 7804223152 chore: release v1.7.57-alpha 2026-05-17 17:30:04 -04:00
Dorian 835c525218 chore(release): stage v1.7.55-alpha 2026-05-13 15:09:22 -04:00
archipelago 745cb1c626 chore(release): stage v1.7.52-alpha 2026-05-05 11:29:18 -04:00
archipelagoandClaude Opus 4.7 2bf8181110 refactor(security): tighten capability + TLS-bypass surface
Three small, focused tightenings:

- core/container/src/podman_client.rs: drop the legacy Hetzner
  23.182.128.160:3000 mirror from image_uses_insecure_registry().
  It was decommissioned in v1.7.x and is stripped from active
  registry config at load time; leaving it in the bypass list let
  a stale config still skip TLS. Replace the inline match with a
  named INSECURE_REGISTRY_HOSTS slice so future entries are one
  line. Test now also pins the spoofing-immune semantics
  ("evil.example/146.59.87.168:3000/x" must NOT match).

- core/archipelago/src/api/rpc/package/config.rs: split bitcoin
  from lnd in get_app_capabilities(). bitcoind never opens raw
  sockets — drop CAP_NET_RAW from bitcoin/bitcoin-core/bitcoin-knots.
  lnd/fedimint/fedimint-gateway keep it because they enumerate
  network interfaces during cert generation.

- core/archipelago/src/bootstrap.rs: tighten_secrets_dir()
  enforces 0700 on /var/lib/archipelago/secrets and 0600 on every
  file inside on each startup. The dir-mode is the load-bearing
  isolation boundary against rootless container escapes (their UID
  maps to >=100000, can't traverse uid=1000/0700). The per-file
  sweep is defense-in-depth against any installer that wrote 0644.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 08:59:11 -04:00
archipelagoandClaude Opus 4.7 0684491072 chore: baseline codex hardening before lifecycle refactor
Snapshots the in-flight hardening work so subsequent reconcile/Quadlet
phases land on a clean before/after diff.

Changes:
- core/container/src/podman_client.rs: image_uses_insecure_registry()
  whitelist for the OVH (146.59.87.168:3000) and legacy Hetzner
  (23.182.128.160:3000) HTTP mirrors; podman_network_settings() lifts
  custom networks into the Networks map so containers can join them.
- core/archipelago/src/container/prod_orchestrator.rs:
  ensure_container_network() creates per-manifest networks on demand;
  apply_data_uid() now goes through host_sudo for mkdir -p + chown so
  bind-mount roots get created and chowned without password prompts.
- core/archipelago/src/api/rpc/package/{install,update,stacks}.rs:
  podman pull adds --tls-verify=false only for whitelisted registries.
- core/archipelago/src/bootstrap.rs: removes stale dev-mode systemd
  override on startup (live nodes carried it from old installers).
- core/archipelago/src/config.rs: ignore ARCHIPELAGO_DEV_MODE in prod
  binaries — it had been silently rerouting volumes to /tmp.
- apps/bitcoin-{core,knots}/manifest.yml: locate bitcoind at runtime
  so image-layout differences don't break entrypoint.
- scripts/app-catalog-image-smoke-test.py: production catalog/image
  smoke test that probes a target node before users click Install.
- .gitignore: cover .codex, .pnpm-store, __pycache__, *.bak.

Removes filebrowser.rs.bak and two stale catalog.json.bak files
(verified identical to live counterparts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 08:52:29 -04:00
archipelago 8f83b37d51 feat(orchestrator): complete container migration and release hardening 2026-04-28 15:00:58 -04:00
archipelago 8acf7d1112 fix: parse_memory_limit accepts Ki/Mi/Gi IEC binary suffixes
The libpod HTTP API path (PodmanClient::create_container) ran manifest
memory_limit values like "128Mi" through parse_memory_limit which
lowercased+trim_end_matches("m"), leaving "128i" which parse::<f64>()
rejected. The resulting None became 0 via .unwrap_or(0), and podman
serialised that into the OCI config as memory.limit:0. At container
start time systemd then rejected MemoryMax=0 with "Value specified in
MemoryMax is out of range".

Silently wrong for every manifest in apps/ that uses Kubernetes-style
suffixes (all of them). Became visible on .228 when Step 9 first
exercised the ProdContainerOrchestrator path for bitcoin-ui and lnd-ui
installs \u2014 the old first-boot-containers.sh bash script used podman
run --memory 128m directly, which podman-the-CLI parses correctly, so
the bug never surfaced before.

Two parts:
- parse_memory_limit now recognises Ki/Mi/Gi/Ti (IEC binary, what k8s
  and our manifests use), kB/MB/GB/TB (SI decimal), k/K/m/M/g/G/t/T
  (docker shorthand, treated as IEC binary for backwards compat), and
  bare byte integers. Filters out zero/negative results.
- create_container omits the memory/cpu fields entirely when the
  manifest has no limit or parsing fails, rather than emitting 0. The
  libpod API treats absent as unlimited; 0 is "set MemoryMax=0" which
  systemd rightly rejects. Defence in depth against the next weird
  suffix someone puts in a manifest.

Six regression tests in the new tests module cover IEC, SI, shorthand,
raw bytes, invalid input (empty/garbage/0/negative), and whitespace.
2026-04-23 03:44:23 -04:00
archipelago 919055f3f1 feat(container): add build source to manifest schema
ContainerConfig.image is now Option<String>, mutually exclusive with a new
optional ContainerConfig.build: Option<BuildConfig>. Exactly one of image
or build must be present, enforced in AppManifest::validate.

Adds ResolvedSource enum (Pull | Build) and ContainerConfig::resolve +
::image_ref helpers so the orchestrator can treat pull and build uniformly.
All 26 existing pull-only manifests continue to parse unchanged
(covered by existing_pull_only_manifests_still_parse test).

Call sites updated: podman_client, runtime::DockerRuntime, dev_orchestrator.
Dev orchestrator errors out cleanly on Build sources until Step 2 lands
build_image support on the runtime trait.

Step 1 of docs/rust-orchestrator-migration.md. 10 new unit tests, all pass.

Also includes: docs/rust-orchestrator-migration.md (design spec) and
docs/STATUS.md resume section for the next session.
2026-04-22 17:46:36 -04:00
DorianandClaude Opus 4.7 b614c5c694 chore(ci): rustfmt + clippy clean-up to unblock the Rust CI job
The .github/workflows/ci.yml Rust job runs cargo fmt --check, clippy
with -D warnings, and tests. All three were failing. This commit:

- Applies rustfmt across the tree (the bulk of the diff — untouched
  since the last toolchain bump, so a wide sweep was unavoidable).
- Fixes the correctness-level clippy errors:
    container/bitcoin_simulator.rs wildcard-in-or-pattern
    container/manifest.rs from_str rename to parse (reserved name)
    container/podman_client.rs .get(0) -> .first()
    container/runtime.rs manual += collapse
    archipelago/src/constants.rs doc-comment → module-doc
    api/rpc/package/install.rs stray /// comment above a non-item
    container/docker_packages.rs redundant field init
    streaming/advertisement.rs missing Metric import in tests
    tests/orchestration_tests.rs `vec!` in non-Vec contexts
    mesh/listener/dispatch.rs unused store_plain_message import
    api/rpc/tor/mod.rs and mesh/steganography.rs: push-after-new → vec!
- Quiets wide legacy surfaces with crate-level allows in main.rs for
  stylistic lints (too_many_arguments, type_complexity, doc indent,
  enum variant prefix, wildcard-in-or, assertions-on-constants,
  drop_non_drop, unused_io_amount, ptr_arg) — these fired in dozens
  of places with no correctness payoff and have been churning every
  toolchain bump.
- Tags intentional-dead-code helpers: wallet/ and streaming/ modules
  are WIP, mesh::send_chunked_payload and DM_V1_MARKER are kept for
  rollback compatibility, vpn::get_nostr_vpn_status is surface-area
  for a not-yet-landed RPC.

cargo fmt --check, cargo clippy --all-targets --all-features
-- -D warnings, and cargo test --all-features now all pass locally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:23:46 -04:00
DorianandClaude Opus 4.6 b3a6d2103a fix: container stability, OnlyOffice removal, node bootstrapping, UI fixes
Container orchestration:
- Add --network-alias to all archy-net containers (fixes Podman DNS)
- Fix bitcoin-knots health check: expand $BITCOIN_RPC_PASS at creation
- Increase bitcoin-knots memory limit to 4g, reduce dbcache to 2048
- Enable podman-restart.service in ISO for auto-start on boot
- Fix UI container Dockerfiles: ENTRYPOINT [], user root for rootless

App changes:
- Remove OnlyOffice (incompatible with rootless Podman)
- Replace with CryptPad reference (single-process, e2e encrypted)
- Fix NPM port mapping: 8181 → 81
- Fix OnlyOffice port mapping: 8044 → 9980 (now CryptPad: 3003)

AIUI & proxy:
- Add MODEL_MAP to claude-api-proxy (ISO + deploy)
- Map legacy model IDs (claude-haiku-4.5 → claude-haiku-4-5-20251001)

Kiosk:
- Move chromium-kiosk data dir to /var/lib/archipelago (data partition)
- Remove --metrics-recording-only (contradicted --disable-metrics)

Node bootstrapping:
- Add bootstrap-switchover.sh for live node updates
- ElectrumX UI improvements and nginx proxy fixes
- LND UI nginx config updates

Backend:
- Bitcoin health check uses .cookie auth (no plaintext creds)
- ElectrumX status endpoint improvements
- Network alias flag in install.rs for DNS reliability

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:15:04 +01:00
DorianandClaude Opus 4.6 5c003daafa fix: stale rootfs container cleanup, OnlyOffice/NPM port corrections
ISO build:
- Remove stale archipelago-rootfs-tmp container before creating new one
  (previous failed builds leave it behind, blocking subsequent builds)

Container ports:
- OnlyOffice: fix LAN address from 8044 to 9980 (actual mapped port)
- Nginx Proxy Manager: fix from 8181 to 81 (correct admin port)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:16:24 +01:00
DorianandClaude Opus 4.6 c62d7f77b5 fix: container orchestration stability, AIUI inclusion, lnd-ui port, version 1.3.0
Container stability:
- Merge scan results instead of full replacement (prevents UI flapping)
- Absence threshold: 3 consecutive missed scans before removing from state
- container-list RPC uses cached scanner state for consistency
- Increased Podman API timeout 30s → 60s (scanner + health monitor)
- Keep crashed containers visible as "exited" instead of podman rm -f
- Resolve host-gateway IP via ip route (podman 4.3.x compatibility)

ISO build fixes:
- AIUI web app inclusion: searches 5 paths + CI step to copy from build server
- Claude API proxy: systemctl enable with symlink fallback
- AIUI nginx: try_files =404 (was /aiui/index.html redirect loop)
- Build version set to 1.3.0

Container fixes:
- lnd-ui: nginx listens on 8080 (was 80, Permission denied in rootless)
- first-boot: image-versions.sh sourced from correct path with validation
- first-boot: host-gateway resolved to actual gateway IP

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 01:28:11 +01:00
DorianandClaude Opus 4.6 dad184cfc6 fix: container DNS, nginx chown, onboarding guard, seed UX, install flow
Backend:
- Add --add-host host.containers.internal:host-gateway to LND and Bitcoin
  Knots containers (fixes DNS resolution failure in rootless podman)
- Add --user 0:0 and DAC_OVERRIDE to nginx UI sidecar containers
  (fixes chown crash in rootless podman for bitcoin-ui, electrs-ui, lnd-ui)
- Add hostadd to Rust Podman API client for web UI container installs
- Add Chromium privacy flags to kiosk launcher (disable telemetry)

Frontend:
- Fix onboarding reset on raw IP visits (trust localStorage as first-class
  signal, skip boot screen when server is up but not onboarded)
- Fix seed regression: persist challenge indices in sessionStorage so going
  back from Verify doesn't change which words are asked
- Remove glass container from seed Generate/Verify/Restore screens
- Add Back button to Restore from Seed screen
- Replace Network card: Tor (purple), VPN status (orange), Bitcoin sync (orange)
- Add ElectrumX to curated app list with correct .webp icon
- Install flow: navigate to My Apps immediately with toast, hide
  installed/installing apps from marketplace and discover views

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 13:06:57 +01:00
DorianandClaude Opus 4.6 1e283daf13 fix: overhaul container lifecycle — recovery, health, uninstall, UI state
Container recovery:
- Health monitor: MAX_RESTART_ATTEMPTS 3→10, interval 60s→120s
- Dependency-aware restarts: won't restart services before their deps
- Reset dependent counters when a dependency recovers
- Handle "created" state containers (were invisible to health monitor)
- Added IndeedHub, mempool-api, mysql to tier system
- Crash recovery: podman start timeout 30s→120s with retry
- Podman client: socket timeout 5s→30s, added restart policy

UI state representation:
- Exit code 0 shows "stopped" (gray), not "crashed" (red)
- Exit code 137 shows "killed (OOM)"
- Non-zero exit shows "crashed" (red)
- Added exit_code field to PackageDataEntry

Install/uninstall fixes:
- Install returns error when container doesn't start (was silent success)
- Post-install hooks awaited instead of fire-and-forget tokio::spawn
- Uninstall: graceful rm before force, volume prune, network cleanup
- Uninstall returns error on partial failure (was 200 OK)

Config consistency:
- DB passwords read from /var/lib/archipelago/secrets/ (was hardcoded)
- Bitcoin: added ZMQ ports 28332/28333 for LND block notifications
- IndeedHub port 7777→8190 (was conflicting with strfry)
- Marketplace versions: LND 0.17.4→0.18.4, Mempool 2.5.0→3.0.0

Performance:
- Metrics collector interval 60s→300s (was duplicating health monitor)
- Podman client: proper error propagation instead of unwrap_or_default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:03:57 +01:00
Dorian e4e0ef4f11 bug fixing and deploy and build diagnostics 2026-03-22 03:30:21 +00:00
DorianandClaude Opus 4.6 1f3c86687d refactor: PodmanClient uses REST API socket instead of CLI
Replace all `podman` CLI shell-outs with HTTP requests to the rootless
Podman API unix socket (/run/user/{UID}/podman/podman.sock).

Benefits:
- No process spawning overhead — direct HTTP over unix socket
- Structured JSON responses — no string parsing fragility
- Proper timeouts on all operations (5s connect, 30s default, 120s create)
- Health check method to verify socket availability
- Restart container as first-class operation

Still uses CLI for:
- Image pulls (streaming operation better suited to CLI)
- Container logs (raw text stream, not JSON)

The Podman socket is rootless (runs as archipelago user), local-only
(unix socket), and already behind our session auth in the backend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 10:13:49 +00:00
DorianandClaude Opus 4.6 84a56c80de security+feat: v1.3.0 — pentest remediation, container reliability, UI overhaul
Security (33 pentest findings addressed):
- CRITICAL: backend binds 127.0.0.1, path traversal in tor.rs/dwn fixed
- HIGH: federation requires signatures, XSS login redirect, RBAC viewer restricted
- HIGH: tar slip prevention, S3 SSRF validation, backup ID validation
- MEDIUM: remember-me random secret, TOTP session rotation, password re-auth
- LOW: CSP unsafe-inline removed, CORS dev-only, onion/webhook validation

Container reliability:
- Memory limits on all 37 containers (OOM prevention)
- Exited vs stopped state distinction with health-aware status badges
- Crash recovery coordination (no more restart cascade)
- User-stopped tracking survives reboots
- Tiered boot recovery (databases → core → services → apps)

UI:
- Wallet TransactionsModal, health-aware app status badges
- Restart button on containers, exited/crashed red state
- Mesh view overhaul, glass button updates, BaseModal/ToggleSwitch
- Apps sticky header removed, dev faucet, mutable mock wallet

Infrastructure:
- LND REST port 8080 exposed over Tor (LND Connect fix)
- Nginx cookie_session fix, deploy script Tor config updated
- Dev environment: podman auto-start, boot mode simulation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:44:31 +00:00
DorianandClaude Opus 4.6 9d3a59e156 fix: Fedimint Guardian UI on port 8175 (not 8174 API)
Fedimintd serves JSON-RPC API on 8174 and Guardian web UI on 8175.
Updated all port mappings: frontend AppSession, nginx HTTP/HTTPS
proxies, PodmanClient static map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 17:31:07 +00:00
DorianandClaude Opus 4.6 ccffaa3562 fix: use PodmanClient::lan_address_for as static fallback for port mapping
Dynamic port extraction from container bindings, falling back to the
static PodmanClient address map for apps without port bindings (e.g.
host-network containers).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 16:32:39 +00:00
DorianandClaude Opus 4.6 d05ed33528 fix: dynamic port detection + electrumx sync + rootless infra
Backend:
- Remove most hardcoded port overrides from docker_packages.rs, use
  dynamic port extraction from actual container bindings with fallback
  to static map in PodmanClient
- Fix OnlyOffice (8044), NginxPM (8181), Fedimint (8174) port mappings
- Remove Tailscale fake web UI port (no web UI)
- ElectrumX: detect "Connection reset" as syncing state (not error)

Deploy script:
- Auto-configure sysctl unprivileged_port_start=80 for rootless
- Auto-enable loginctl linger for container persistence
- Auto-enable podman.socket for Portainer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 16:29:03 +00:00
DorianandClaude Opus 4.6 39c7ac1924 feat: rootless podman, session hardening, boot stability, sidebar fix
Rootless podman migration (TASK-11):
- Remove sudo from all podman calls in PodmanClient + 8 backend files
- Remove sudo from all podman/docker calls in deploy script
- Restore full systemd security hardening: NoNewPrivileges,
  RestrictAddressFamilies, MemoryDenyWriteExecute, RestrictRealtime,
  RestrictNamespaces, RestrictSUIDSGID, SystemCallFilter, ProtectSystem=strict
- Enable loginctl linger for rootless container persistence
- Remove Ollama from auto-deploy (marketplace-only)

Session & auth hardening:
- Increase MAX_CONCURRENT_SESSIONS 20→50 (prevents eviction storms)
- Debounced 401 redirect in rpc-client.ts (prevents redirect storms)

Boot stability:
- optimize-debian.sh: adds chrony, swap, removes policy-rc.d
- deploy script: pre-restart chrony + swap setup
- ISO build: chrony package, swap file creation
- BootScreen: no longer clears localStorage (prevents splash replay)
- RootRedirect: sole owner of localStorage clearing on server ready

UI fixes:
- Sidebar opacity default changed from 0→visible (fixes missing sidebar
  after page-persistence login without entrance animation)
- Console.log/error wrapped in import.meta.env.DEV guards
- Remove unused route import from RootRedirect

Beta tracking:
- CLAUDE.md: beta freeze protocol added
- MASTER_PLAN.md: TASK-11, TASK-17, phase structure
- BETA-PROGRESS.md: initial tracking doc
- Tagged v1.2.0-alpha.1 as pre-rootless baseline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 13:53:27 +00:00
DorianandClaude Opus 4.6 d37ec1dea5 feat: v1.2.0-alpha — E2E encrypted mesh relay, steganography, relay status polling
Phase 5 mesh networking:
- E2E encrypted TX relay (X25519 + ChaCha20-Poly1305) — non-Archy nodes
  relay encrypted blobs transparently via Meshcore native routing
- Steganographic encoding modes (WeatherStation, SensorNetwork) — traffic
  looks like sensor data on the wire, 0xAA marker, configurable per-node
- Pre-flight Bitcoin Core health check on relay node — specific error codes
  (bitcoin_unreachable, bitcoin_syncing, tx_rejected) instead of generic fails
- mesh.relay-status RPC endpoint — frontend polls for relay result every 3s
- On-Chain / Lightning tabs in Off-Grid Bitcoin panel
- Archy Peers vs Mesh Broadcast relay mode selector
- Mesh view fills viewport (no page scroll), internal panel scrolling
- Version bump to 1.2.0-alpha

Also includes: deploy hardening, container fixes, IndeedHub updates,
boot screen, dashboard improvements, MASTER_PLAN task tracking

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 23:56:37 +00:00
DorianandClaude Opus 4.6 367b483a72 feat: bitcoin-ui CSS fix, HTTPS proxy support, deploy script improvements
Bitcoin UI:
- Replace cdn.tailwindcss.com with locally bundled tailwind.css (CSP blocks external scripts)
- Make all asset paths relative for nginx proxy compatibility
- Add bitcoin-ui build/deploy to deploy-to-target.sh (was missing entirely)
- Use --network host (bitcoin-ui proxies Bitcoin RPC at 127.0.0.1:8332)

HTTPS mixed content fix:
- Add HTTPS_PROXY_PATHS in AppSession.vue — when parent page is HTTPS,
  iframe loads through nginx proxy instead of direct HTTP port
- Prevents browser blocking HTTP iframes inside HTTPS pages
- All Tailscale servers use HTTPS, this was breaking all app iframes

Deploy & first-boot improvements:
- first-boot-containers.sh auto-detects disk size for pruning vs txindex
- first-boot-containers.sh checks fallback source path for UI containers
- Added mempool-electrs to APP_PORTS mapping
- ElectrumX container creation in first-boot
- Podman doctor/fix/uptime skills added

Also includes: session persistence, identity management, LND transactions,
ElectrumX status UI, nostr-provider improvements, Web5 enhancements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 12:58:35 +00:00
DorianandClaude Opus 4.6 0b3bf5b635 refactor: remove dead code and #[allow(dead_code)] annotations
Removed unused sync podman_command/docker_command methods.
Removed dead_code annotations from User and AuthManager (now actively used).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 04:34:14 +00:00
DorianandClaude Opus 4.6 2a867b32a8 fix: enforce no-new-privileges on all container creation
The manifest field was validated but never applied to the podman create
command. Now passes --security-opt no-new-privileges=true for all containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:06:59 +00:00
Dorian d988396111 Add lan_address support in RPC and container management
- Introduced a new `lan_address` field in the RPC response for containers, allowing for easier access to UI launch URLs based on container names.
- Updated the `ContainerStatus` struct to include `lan_address`, ensuring it is initialized and passed through relevant methods in both Podman and Docker runtimes.
- Enhanced the UI store to compute enriched bundled apps with their respective `lan_address`, improving the user experience for accessing containerized applications.
- Modified the `ContainerApps` view to utilize the enriched data, ensuring the correct launch URLs are displayed for bundled apps.
2026-02-04 16:20:09 +00:00
Dorian 337ebee510 Revise BUILD-GUIDE and enhance ISO build process
- Updated BUILD-GUIDE.md to streamline instructions for building the Archipelago Auto-Installer ISO, including prerequisites and post-installation steps.
- Added detailed sections on capturing the live server state and building from source.
- Enhanced Docker and Podman integration in build scripts for improved backend and web UI capture.
- Introduced new app metadata for "IndeedHub" in the Docker package scanner and updated UI components for better installation progress tracking.
- Improved styling and functionality in the Bitcoin UI for a more cohesive user experience.
2026-02-03 21:43:33 +00:00
Dorian 34fc06726e Enhance development workflow and deployment practices for Archipelago
- Updated the Development-Workflow documentation to clarify deployment strategy, emphasizing direct deployment to the live system for testing.
- Added detailed instructions for the deployment command, including syncing code, building frontend and backend, and restarting services.
- Improved SSH key management section to assist with authentication issues.
- Expanded the testing workflow to include steps for checking logs and syncing changes back to the ISO build.
- Updated the ISO build integration section to ensure system-level changes are captured for future builds.
- Refactored various sections for clarity and completeness, including deployment paths and system configuration files.
2026-02-01 13:24:03 +00:00
Dorian 66c823e2fd Refactor configuration and scripts for Archipelago backend and ISO build
- Updated Cargo.toml to remove unnecessary package backtrace optimizations.
- Changed default bind host and port in config.rs for broader accessibility.
- Renamed state_manager to _state_manager in server.rs for clarity.
- Updated user field to _user in PodmanClient and DockerRuntime for consistency.
- Modified build-debian-iso.sh to enhance welcome message and backend startup instructions.
- Improved archipelago-menu.sh to display backend status and updated Web UI URL.
- Enhanced install-to-disk.sh for better package management and user creation during installation.
2026-02-01 05:42:05 +00:00
Dorian 30ed48ad1b Enhance Docker integration and API for container management
- Implemented Docker container scanning and periodic updates in the Server initialization.
- Added new RPC endpoints for managing Docker containers, including start, stop, and restart functionalities.
- Updated the API to handle package management for Docker-based applications.
- Improved environment variable handling for user-specific configurations in Podman and Docker clients.
- Enhanced the development startup script to include Docker container management and provide clearer instructions for full stack setup.
2026-01-27 23:21:26 +00:00