chore: release v1.7.44-alpha

self-update.sh previously rebuilt only the backend binary and Vue
frontend. The custom UI containers (archy-bitcoin-ui, archy-lnd-ui,
archy-electrs-ui) were left untouched forever. That meant any change to
docker/<ui>/{Dockerfile, nginx.conf, index.html, ...} never reached a
running node through OTA; it required a manual SSH + rebuild. This is
exactly why the lnd-ui port fix didnt reach .228 in v1.7.43-alpha.

Add a sync-and-rebuild stage:

  1. Hash each docker/<ui>/ tree (content-only, path-stable via
     `cd && find` so src and dst compare equal when identical).
  2. rsync changed trees to /opt/archipelago/docker/<ui>/.
  3. For each changed UI: rebuild image as the archipelago user
     (rootless podman), then stop+remove+recreate the container using
     the canonical spec from scripts/container-specs.sh. Port mappings,
     caps, memory, and security opts all come from the spec, so the
     runtime cant drift from the tree.

Also install first-boot-containers.sh into /opt/archipelago/scripts/ so
a later reconciler run or reboot picks up current orchestration logic.

Idempotent: if no UI tree changed since the last update, the whole stage
is a no-op beyond the hash compare. Verified end-to-end on .228 with a
synthetic change to lnd-ui: detection, sync, build, recreate, and HTTP
200 on both the direct container port and the host-nginx /app/lnd/
proxy.

.gitignore

@@ -73,3 +73,4 @@ loop/loop.log.bak
 # Separate repos nested in tree
 web/
 
+._*

CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## v1.7.44-alpha (2026-04-28)
+
+43de3b73 feat(orchestrator): complete container migration and release hardening
+ce39430b feat(self-update): sync and rebuild UI containers on OTA
+72dec5aa fix(lnd-ui): align container port across all specs
+83aacdf2 chore(release): archive ISO build recipes, tarball-only releases
+
+
 All notable changes to Archipelago will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),

README.md

@@ -101,14 +101,20 @@ npm run build      # Production build → web/dist/neode-ui/
 ./scripts/deploy-to-target.sh --both   # Deploy to both LAN servers
 ```
 
-### Build ISO
+### Release (tarball-only)
+
+Releases ship as a backend binary and a frontend tarball referenced by
+`releases/manifest.json`. Nodes OTA-update via `scripts/self-update.sh`.
 
 ```bash
-ssh archipelago@<server>
-cd ~/archy/image-recipe
-sudo ./build-auto-installer-iso.sh
+./scripts/create-release.sh 1.2.3
+git push gitea-local main --tags
+git push gitea-vps2 main --tags
 ```
 
+ISO builds are archived under `image-recipe/_archived/` and not part of the
+release deliverable.
+
 ## Architecture
 
 ```

apps/archy-btcpay-db/manifest.yml

@@ -0,0 +1,49 @@
+app:
+  id: archy-btcpay-db
+  name: BTCPay Postgres
+  version: 15.17
+  description: Postgres backend for BTCPay and NBXplorer.
+
+  container:
+    image: git.tx1138.com/lfg2025/postgres:15.17
+    pull_policy: if-not-present
+    network: archy-net
+    data_uid: "100998:100998"
+    secret_env:
+      - key: POSTGRES_PASSWORD
+        secret_file: btcpay-db-password
+
+  dependencies:
+    - storage: 20Gi
+
+  resources:
+    memory_limit: 1Gi
+    disk_limit: 20Gi
+
+  security:
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE]
+    readonly_root: false
+    network_policy: isolated
+
+  ports: []
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/postgres-btcpay
+      target: /var/lib/postgresql/data
+      options: [rw]
+
+  environment:
+    - POSTGRES_DB=btcpay
+    - POSTGRES_USER=btcpay
+
+  health_check:
+    type: tcp
+    endpoint: localhost:5432
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: none
+    sync_required: false

apps/archy-mempool-db/manifest.yml

@@ -0,0 +1,51 @@
+app:
+  id: archy-mempool-db
+  name: Mempool MariaDB
+  version: 11.4.10
+  description: MariaDB backend for the mempool explorer stack.
+
+  container:
+    image: git.tx1138.com/lfg2025/mariadb:11.4.10
+    pull_policy: if-not-present
+    network: archy-net
+    data_uid: "100998:100998"
+    secret_env:
+      - key: MYSQL_PASSWORD
+        secret_file: mempool-db-password
+      - key: MYSQL_ROOT_PASSWORD
+        secret_file: mysql-root-db-password
+
+  dependencies:
+    - storage: 20Gi
+
+  resources:
+    memory_limit: 512Mi
+    disk_limit: 20Gi
+
+  security:
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE]
+    readonly_root: false
+    network_policy: isolated
+
+  ports: []
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/mysql-mempool
+      target: /var/lib/mysql
+      options: [rw]
+
+  environment:
+    - MYSQL_DATABASE=mempool
+    - MYSQL_USER=mempool
+
+  health_check:
+    type: tcp
+    endpoint: localhost:3306
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: none
+    sync_required: false

apps/archy-mempool-web/manifest.yml

@@ -0,0 +1,50 @@
+app:
+  id: archy-mempool-web
+  name: Mempool Web
+  version: 3.0.0
+  description: Frontend web UI for mempool explorer.
+  container_name: mempool
+
+  container:
+    image: git.tx1138.com/lfg2025/mempool-frontend:v3.0.0
+    pull_policy: if-not-present
+    network: archy-net
+
+  dependencies:
+    - app_id: mempool-api
+      version: ">=3.0.0"
+
+  resources:
+    memory_limit: 512Mi
+
+  security:
+    capabilities: []
+    readonly_root: false
+    network_policy: isolated
+
+  ports:
+    - host: 4080
+      container: 8080
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/mempool/nginx.conf
+      target: /etc/nginx/conf.d/default.conf
+      options: [ro]
+
+  environment:
+    - FRONTEND_HTTP_PORT=8080
+    - BACKEND_MAINNET_HTTP_HOST=mempool-api
+
+  health_check:
+    type: http
+    endpoint: http://localhost:8080
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: none
+    sync_required: false

apps/archy-nbxplorer/manifest.yml

@@ -0,0 +1,62 @@
+app:
+  id: archy-nbxplorer
+  name: NBXplorer
+  version: 2.6.0
+  description: BTCPay blockchain indexer service.
+
+  container:
+    image: git.tx1138.com/lfg2025/nbxplorer:2.6.0
+    pull_policy: if-not-present
+    network: archy-net
+    secret_env:
+      - key: NBXPLORER_BTCRPCPASSWORD
+        secret_file: bitcoin-rpc-password
+      - key: BTCPAY_DB_PASS
+        secret_file: btcpay-db-password
+
+  dependencies:
+    - app_id: bitcoin-core
+      version: ">=26.0"
+    - app_id: archy-btcpay-db
+      version: ">=15.17"
+
+  resources:
+    memory_limit: 2Gi
+    disk_limit: 20Gi
+
+  security:
+    capabilities: []
+    readonly_root: false
+    network_policy: isolated
+
+  ports:
+    - host: 32838
+      container: 32838
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/nbxplorer
+      target: /data
+      options: [rw]
+
+  environment:
+    - NBXPLORER_DATADIR=/data
+    - NBXPLORER_NETWORK=mainnet
+    - NBXPLORER_CHAINS=btc
+    - NBXPLORER_BIND=0.0.0.0:32838
+    - NBXPLORER_BTCRPCURL=http://bitcoin-knots:8332
+    - NBXPLORER_BTCRPCUSER=archipelago
+    - NBXPLORER_POSTGRES=User ID=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=nbxplorer;Include Error Detail=true
+
+  health_check:
+    type: http
+    endpoint: http://localhost:32838
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: read-only
+    sync_required: true

apps/bitcoin-core/manifest.yml

@@ -1,61 +1,70 @@
 app:
   id: bitcoin-core
-  name: Bitcoin Core
+  name: Bitcoin Knots
   version: 28.4.0
-  description: Full Bitcoin node implementation. The reference implementation of the Bitcoin protocol.
+  description: Full Bitcoin Knots node with dynamic prune/full-mode startup based on host disk.
+
+  container_name: bitcoin-knots
 
   container:
-    image: bitcoin/bitcoin:28.4
-    image_signature: cosign://...
-    pull_policy: verify-signature
-    
+    image: git.tx1138.com/lfg2025/bitcoin-knots:latest
+    pull_policy: if-not-present
+    network: archy-net
+    entrypoint: ["sh", "-lc"]
+    custom_args:
+      - >-
+        if [ "${DISK_GB:-0}" -lt 1000 ]; then
+          exec bitcoind -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=512 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
+        else
+          exec bitcoind -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -rpcuser="${BITCOIN_RPC_USER}" -rpcpassword="${BITCOIN_RPC_PASS}";
+        fi
+    derived_env:
+      - key: DISK_GB
+        template: "{{DISK_GB}}"
+    secret_env:
+      - key: BITCOIN_RPC_PASS
+        secret_file: bitcoin-rpc-password
+    data_uid: "100101:100101"
+
   dependencies:
-    - storage: 500Gi  # Minimum disk space for mainnet
-    
+    - storage: 500Gi
+
   resources:
-    cpu_limit: 0      # 0 = unlimited; bitcoind uses -par=auto across all cores
-    memory_limit: 4Gi # matches container-specs.sh bitcoin-knots large-disk dbcache=4096
+    cpu_limit: 0
+    memory_limit: 4Gi
     disk_limit: 500Gi
-    
+
   security:
-    capabilities: []  # No special capabilities needed
-    readonly_root: true
-    no_new_privileges: true
-    user: 1000
-    seccomp_profile: default
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE]
+    readonly_root: false
     network_policy: isolated
-    apparmor_profile: bitcoin-core
-    
+
   ports:
     - host: 8332
       container: 8332
-      protocol: tcp  # RPC
+      protocol: tcp
     - host: 8333
       container: 8333
-      protocol: tcp  # P2P
-    
+      protocol: tcp
+
   volumes:
     - type: bind
       source: /var/lib/archipelago/bitcoin
       target: /home/bitcoin/.bitcoin
       options: [rw]
-      
+
   environment:
-    - NETWORK=mainnet
-    - RPC_USER=${BITCOIN_RPC_USER}
-    - RPC_PASSWORD=${BITCOIN_RPC_PASSWORD}
-    - PRUNE=0  # Full node (set to 550 for pruned)
-    
+    - BITCOIN_RPC_USER=archipelago
+
   health_check:
-    type: http
-    endpoint: http://localhost:8332
-    path: /
+    type: tcp
+    endpoint: localhost:8332
     interval: 30s
     timeout: 5s
     retries: 3
-    
+
   bitcoin_integration:
     rpc_access: admin
     sync_required: true
-    testnet_support: true
+    testnet_support: false
     pruning_support: true

apps/bitcoin-ui/manifest.yml

@@ -0,0 +1,56 @@
+app:
+  id: bitcoin-ui
+  name: Bitcoin UI
+  version: 1.0.0
+  description: |
+    Archipelago-native HTTP proxy + static site for interacting with the
+    Bitcoin Core / Bitcoin Knots JSON-RPC. Runs nginx inside a container
+    and reverse-proxies /bitcoin-rpc/ to 127.0.0.1:8332 on the host. The
+    upstream Authorization header is substituted from
+    /var/lib/archipelago/secrets/bitcoin-rpc-password by the prod
+    orchestrator's pre-start hook, rendered into an nginx.conf that is
+    bind-mounted read-only at container start.
+
+  container:
+    build:
+      context: /opt/archipelago/docker/bitcoin-ui
+      dockerfile: Dockerfile
+      tag: localhost/bitcoin-ui:local
+
+  dependencies:
+    - app_id: bitcoin-core
+
+  resources:
+    memory_limit: 128Mi
+
+  security:
+    readonly_root: false
+    network_policy: host
+
+  # Host networking: nginx listens on 8334 directly on the host IP, and
+  # proxies to 127.0.0.1:8332 which is where the bitcoin backend binds
+  # its RPC. `ports:` is intentionally empty because host networking
+  # bypasses port mapping.
+  ports: []
+
+  volumes:
+    # Bind-mount the rendered nginx.conf read-only. The prod orchestrator
+    # renders /var/lib/archipelago/bitcoin-ui/nginx.conf on every install
+    # and every reconcile pass, substituting the base64 RPC auth from
+    # the plaintext password secret. If the rendered bytes change (the
+    # password rotated, or the template was updated by OTA), the
+    # reconciler restarts this container so nginx re-reads the config.
+    - type: bind
+      source: /var/lib/archipelago/bitcoin-ui/nginx.conf
+      target: /etc/nginx/conf.d/default.conf
+      options: [ro]
+
+  environment: []
+
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:8334
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3

apps/btcpay-server/manifest.yml

@@ -1,66 +1,70 @@
 app:
   id: btcpay-server
   name: BTCPay Server
-  version: 1.12.0
+  version: 1.13.7
   description: Self-hosted Bitcoin payment processor. Accept Bitcoin payments without intermediaries.
-  
+
   container:
-    image: btcpayserver/btcpayserver:1.12.0
-    image_signature: cosign://...
-    pull_policy: verify-signature
-    
+    image: git.tx1138.com/lfg2025/btcpayserver:1.13.7
+    pull_policy: if-not-present
+    network: archy-net
+    secret_env:
+      - key: BTCPAY_BTCRPCPASSWORD
+        secret_file: bitcoin-rpc-password
+      - key: BTCPAY_DB_PASS
+        secret_file: btcpay-db-password
+
   dependencies:
     - app_id: bitcoin-core
       version: ">=26.0"
-    - app_id: lnd
-      version: ">=0.18.0"
-    
+    - app_id: archy-btcpay-db
+      version: ">=15.17"
+    - app_id: archy-nbxplorer
+      version: ">=2.6.0"
+
   resources:
     cpu_limit: 2
     memory_limit: 2Gi
     disk_limit: 20Gi
-    
+
   security:
-    capabilities: [NET_BIND_SERVICE]
-    readonly_root: true
-    no_new_privileges: true
-    user: 1000
-    seccomp_profile: default
+    capabilities: []
+    readonly_root: false
     network_policy: isolated
-    apparmor_profile: btcpay
-    
+
   ports:
-    - host: 80
-      container: 80
+    - host: 23000
+      container: 49392
       protocol: tcp
-    - host: 443
-      container: 443
-      protocol: tcp
-    
+
   volumes:
     - type: bind
       source: /var/lib/archipelago/btcpay
       target: /datadir
       options: [rw]
-      
+
   environment:
-    - BTCPAY_NETWORK=mainnet
-    - BTCPAY_CHAIN=btc
-    - BTCPAY_BTCEXPLORERURL=http://bitcoin-core:8332
-    - BTCPAY_LIGHTNING=type=lnd-rest;server=http://lnd:8080;allowinsecure=true
-    
+    - ASPNETCORE_URLS=http://0.0.0.0:49392
+    - BTCPAY_PROTOCOL=http
+    - BTCPAY_HOST=127.0.0.1:23000
+    - BTCPAY_CHAINS=btc
+    - BTCPAY_BTCEXPLORERURL=http://archy-nbxplorer:32838
+    - BTCPAY_BTCRPCURL=http://bitcoin-knots:8332
+    - BTCPAY_BTCRPCUSER=archipelago
+    - BTCPAY_POSTGRES=User ID=btcpay;Password=${BTCPAY_DB_PASS};Host=archy-btcpay-db;Port=5432;Database=btcpay;Include Error Detail=true
+
   health_check:
     type: http
-    endpoint: http://localhost
-    path: /health
+    endpoint: http://localhost:49392
+    path: /
     interval: 30s
     timeout: 5s
     retries: 3
-    
+
   bitcoin_integration:
     rpc_access: read-only
     sync_required: true
-    
+
   lightning_integration:
-    payment_processing: true
+    payment_processing: false
     invoice_management: true

apps/electrs-ui/manifest.yml

@@ -0,0 +1,38 @@
+app:
+  id: electrs-ui
+  name: Electrs UI
+  version: 1.0.0
+  description: |
+    Archipelago-native HTTP frontend for electrs/electrumx status. Runs
+    nginx inside a container, serves static assets, and proxies
+    /electrs-status to the archipelago backend on 127.0.0.1:5678.
+
+  container:
+    build:
+      context: /opt/archipelago/docker/electrs-ui
+      dockerfile: Dockerfile
+      tag: localhost/electrs-ui:local
+
+  dependencies: []
+
+  resources:
+    memory_limit: 64Mi
+
+  security:
+    readonly_root: false
+    network_policy: host
+
+  # Host networking: nginx listens on 50002 directly on the host IP.
+  ports: []
+
+  volumes: []
+
+  environment: []
+
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:50002
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3

apps/electrumx/manifest.yml

@@ -0,0 +1,60 @@
+app:
+  id: electrumx
+  name: ElectrumX
+  version: 1.18.0
+  description: Electrum server indexing Bitcoin chain data for lightweight wallet queries.
+
+  container:
+    image: git.tx1138.com/lfg2025/electrumx:v1.18.0
+    pull_policy: if-not-present
+    network: archy-net
+    entrypoint: ["sh", "-lc"]
+    custom_args:
+      - >-
+        export DAEMON_URL="http://archipelago:${BITCOIN_RPC_PASS}@bitcoin-knots:8332/";
+        exec electrumx_server
+    secret_env:
+      - key: BITCOIN_RPC_PASS
+        secret_file: bitcoin-rpc-password
+
+  dependencies:
+    - app_id: bitcoin-core
+      version: ">=26.0"
+    - storage: 50Gi
+
+  resources:
+    cpu_limit: 2
+    memory_limit: 2Gi
+    disk_limit: 50Gi
+
+  security:
+    capabilities: [DAC_OVERRIDE]
+    readonly_root: false
+    network_policy: isolated
+
+  ports:
+    - host: 50001
+      container: 50001
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/electrumx
+      target: /data
+      options: [rw]
+
+  environment:
+    - COIN=Bitcoin
+    - DB_DIRECTORY=/data
+    - SERVICES=tcp://:50001,rpc://0.0.0.0:8000
+
+  health_check:
+    type: tcp
+    endpoint: localhost:50001
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: read-only
+    sync_required: true

apps/fedimint-gateway/manifest.yml

@@ -0,0 +1,73 @@
+app:
+  id: fedimint-gateway
+  name: Fedimint Gateway
+  version: 0.10.0
+  description: Fedimint gateway service with automatic LND-or-LDK backend selection.
+
+  container:
+    image: git.tx1138.com/lfg2025/gatewayd:v0.10.0
+    pull_policy: if-not-present
+    network: archy-net
+    entrypoint: ["sh", "-lc"]
+    custom_args:
+      - >-
+        if [ -f /lnd/tls.cert ] && [ -f /lnd/data/chain/bitcoin/mainnet/admin.macaroon ]; then
+          exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://bitcoin-knots:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" lnd --lnd-rpc-host lnd:10009 --lnd-tls-cert /lnd/tls.cert --lnd-macaroon /lnd/data/chain/bitcoin/mainnet/admin.macaroon;
+        else
+          exec gatewayd --data-dir /data --listen 0.0.0.0:8176 --bcrypt-password-hash "$FEDI_HASH" --network bitcoin --bitcoind-url http://bitcoin-knots:8332 --bitcoind-username "$FM_BITCOIND_USERNAME" --bitcoind-password "$FM_BITCOIND_PASSWORD" ldk --ldk-lightning-port 9737 --ldk-alias archipelago-gateway;
+        fi
+    secret_env:
+      - key: FM_BITCOIND_PASSWORD
+        secret_file: bitcoin-rpc-password
+      - key: FEDI_HASH
+        secret_file: fedimint-gateway-hash
+    data_uid: "100000:100000"
+
+  dependencies:
+    - app_id: bitcoin-core
+      version: ">=26.0"
+    - app_id: fedimint
+      version: ">=0.10.0"
+
+  resources:
+    cpu_limit: 2
+    memory_limit: 2Gi
+    disk_limit: 10Gi
+
+  security:
+    capabilities: []
+    readonly_root: true
+    network_policy: isolated
+
+  ports:
+    - host: 8176
+      container: 8176
+      protocol: tcp
+    - host: 9737
+      container: 9737
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/fedimint-gateway
+      target: /data
+      options: [rw]
+    - type: bind
+      source: /var/lib/archipelago/lnd
+      target: /lnd
+      options: [ro]
+
+  environment:
+    - FM_BITCOIND_USERNAME=archipelago
+
+  health_check:
+    type: http
+    endpoint: http://localhost:8176
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: admin
+    sync_required: true

apps/fedimint/manifest.yml

@@ -3,56 +3,62 @@ app:
   name: Fedimint
   version: 0.10.0
   description: Federated Bitcoin minting service with built-in Guardian UI. Privacy-preserving Bitcoin custody.
-  
+
   container:
-    image: fedimint/fedimintd:v0.10.0
-    image_signature: cosign://...
+    image: git.tx1138.com/lfg2025/fedimintd:v0.10.0
     pull_policy: if-not-present
-    
+    network: archy-net
+    derived_env:
+      - key: FM_P2P_URL
+        template: fedimint://{{HOST_MDNS}}:8173
+      - key: FM_API_URL
+        template: ws://{{HOST_MDNS}}:8174
+    secret_env:
+      - key: FM_BITCOIND_PASSWORD
+        secret_file: bitcoin-rpc-password
+    data_uid: "100000:100000"
+
   dependencies:
     - app_id: bitcoin-core
-      version: ">=24.0"
+      version: ">=26.0"
     - storage: 20Gi
-    
+
   resources:
     cpu_limit: 4
     memory_limit: 4Gi
     disk_limit: 20Gi
-    
+
   security:
     capabilities: []
     readonly_root: true
-    no_new_privileges: true
-    user: 1000
-    seccomp_profile: default
     network_policy: isolated
-    apparmor_profile: fedimint
-    
+
   ports:
     - host: 8173
       container: 8173
-      protocol: tcp  # P2P
+      protocol: tcp
     - host: 8174
       container: 8174
-      protocol: tcp  # API
+      protocol: tcp
     - host: 8175
       container: 8175
-      protocol: tcp  # Built-in Guardian UI
-    
+      protocol: tcp
+
   volumes:
     - type: bind
       source: /var/lib/archipelago/fedimint
-      target: /fedimint
+      target: /data
       options: [rw]
-      
+
   environment:
-    - FM_DATA_DIR=/fedimint
-    - FM_BITCOIND_URL=http://bitcoin-core:8332
-    - FM_BITCOIND_USERNAME=${BITCOIN_RPC_USER}
-    - FM_BITCOIND_PASSWORD=${BITCOIN_RPC_PASSWORD}
+    - FM_DATA_DIR=/data
+    - FM_BITCOIND_URL=http://bitcoin-knots:8332
+    - FM_BITCOIND_USERNAME=archipelago
     - FM_BITCOIN_NETWORK=bitcoin
+    - FM_BIND_P2P=0.0.0.0:8173
+    - FM_BIND_API=0.0.0.0:8174
     - FM_BIND_UI=0.0.0.0:8175
-    
+
   health_check:
     type: http
     endpoint: http://localhost:8175
@@ -60,7 +66,7 @@ app:
     interval: 30s
     timeout: 5s
     retries: 3
-    
+
   bitcoin_integration:
     rpc_access: admin
     sync_required: true

apps/filebrowser/manifest.yml

@@ -0,0 +1,53 @@
+app:
+  id: filebrowser
+  name: File Browser
+  version: 2.27.0
+  description: Baseline Archipelago file manager service.
+
+  container:
+    image: git.tx1138.com/lfg2025/filebrowser:v2.27.0
+    pull_policy: if-not-present
+    network: archy-net
+    custom_args: ["--config", "/data/.filebrowser.json"]
+    data_uid: "100000:100000"
+
+  dependencies:
+    - storage: 10Gi
+
+  resources:
+    memory_limit: 256Mi
+    disk_limit: 10Gi
+
+  security:
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_BIND_SERVICE]
+    readonly_root: false
+    network_policy: isolated
+
+  ports:
+    - host: 8083
+      container: 80
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/filebrowser
+      target: /srv
+      options: [rw]
+    - type: bind
+      source: /var/lib/archipelago/filebrowser-data
+      target: /data
+      options: [rw]
+
+  environment: []
+
+  health_check:
+    type: http
+    endpoint: http://localhost:80
+    path: /health
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: none
+    sync_required: false

apps/lnd-ui/manifest.yml

@@ -0,0 +1,44 @@
+app:
+  id: lnd-ui
+  name: LND UI
+  version: 1.0.0
+  description: |
+    Archipelago-native HTTP frontend for LND. Runs nginx inside a
+    container and serves static assets. LND connection info is fetched
+    via an absolute URL that the host nginx routes to the archipelago
+    backend on 127.0.0.1:5678, so no upstream auth is baked in.
+
+  container:
+    build:
+      context: /opt/archipelago/docker/lnd-ui
+      dockerfile: Dockerfile
+      tag: localhost/lnd-ui:local
+
+  dependencies:
+    - app_id: lnd
+
+  resources:
+    memory_limit: 64Mi
+
+  security:
+    readonly_root: false
+    network_policy: bridge
+
+  # Bridge networking via archy-net. Container nginx listens on 80;
+  # host nginx proxies /app/lnd/ -> 127.0.0.1:8081 -> container:80.
+  ports:
+    - host: 8081
+      container: 80
+      protocol: tcp
+
+  volumes: []
+
+  environment: []
+
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:8081
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3

apps/lnd/manifest.yml

@@ -1,67 +1,65 @@
 app:
   id: lnd
   name: Lightning Network Daemon
-  version: 0.18.0
+  version: 0.18.4
   description: Lightning Network implementation by Lightning Labs. Enables instant, low-cost Bitcoin payments.
-  
+
   container:
-    image: lightninglabs/lnd:v0.18.0
-    image_signature: cosign://...
-    pull_policy: verify-signature
-    
+    image: git.tx1138.com/lfg2025/lnd:v0.18.4-beta
+    pull_policy: if-not-present
+    network: archy-net
+    secret_env:
+      - key: BITCOIND_RPCPASS
+        secret_file: bitcoin-rpc-password
+    data_uid: "100000:100000"
+
   dependencies:
     - app_id: bitcoin-core
       version: ">=26.0"
-    
+
   resources:
     cpu_limit: 2
     memory_limit: 1Gi
     disk_limit: 10Gi
-    
+
   security:
-    capabilities: [NET_BIND_SERVICE]
-    readonly_root: true
-    no_new_privileges: true
-    user: 1000
-    seccomp_profile: default
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_RAW]
+    readonly_root: false
     network_policy: isolated
-    apparmor_profile: lnd
-    
+
   ports:
     - host: 9735
       container: 9735
-      protocol: tcp  # P2P
+      protocol: tcp
     - host: 10009
       container: 10009
-      protocol: tcp  # gRPC
+      protocol: tcp
     - host: 8080
       container: 8080
-      protocol: tcp  # REST
-    
+      protocol: tcp
+
   volumes:
     - type: bind
       source: /var/lib/archipelago/lnd
       target: /root/.lnd
       options: [rw]
-      
+
   environment:
-    - BITCOIND_HOST=bitcoin-core
-    - BITCOIND_RPCUSER=${BITCOIN_RPC_USER}
-    - BITCOIND_RPCPASS=${BITCOIN_RPC_PASSWORD}
+    - BITCOIND_HOST=bitcoin-knots
+    - BITCOIND_RPCUSER=archipelago
     - NETWORK=mainnet
-    
+
   health_check:
-    type: http
-    endpoint: http://localhost:8080
-    path: /v1/getinfo
+    type: tcp
+    endpoint: localhost:10009
     interval: 30s
     timeout: 5s
     retries: 3
-    
+
   bitcoin_integration:
     rpc_access: admin
     sync_required: true
-    
+
   lightning_integration:
     channel_management: true
     payment_routing: true

apps/mempool-api/manifest.yml

@@ -0,0 +1,68 @@
+app:
+  id: mempool-api
+  name: Mempool API
+  version: 3.0.0
+  description: Backend API for mempool explorer.
+
+  container:
+    image: git.tx1138.com/lfg2025/mempool-backend:v3.0.0
+    pull_policy: if-not-present
+    network: archy-net
+    secret_env:
+      - key: CORE_RPC_PASSWORD
+        secret_file: bitcoin-rpc-password
+      - key: DATABASE_PASSWORD
+        secret_file: mempool-db-password
+
+  dependencies:
+    - app_id: bitcoin-core
+      version: ">=26.0"
+    - app_id: electrumx
+      version: ">=1.18.0"
+    - app_id: archy-mempool-db
+      version: ">=11.4.10"
+
+  resources:
+    memory_limit: 2Gi
+    disk_limit: 20Gi
+
+  security:
+    capabilities: []
+    readonly_root: false
+    network_policy: isolated
+
+  ports:
+    - host: 8999
+      container: 8999
+      protocol: tcp
+
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/mempool
+      target: /data
+      options: [rw]
+
+  environment:
+    - MEMPOOL_BACKEND=electrum
+    - ELECTRUM_HOST=electrumx
+    - ELECTRUM_PORT=50001
+    - ELECTRUM_TLS_ENABLED=false
+    - CORE_RPC_HOST=bitcoin-knots
+    - CORE_RPC_PORT=8332
+    - CORE_RPC_USERNAME=archipelago
+    - DATABASE_ENABLED=true
+    - DATABASE_HOST=archy-mempool-db
+    - DATABASE_DATABASE=mempool
+    - DATABASE_USERNAME=mempool
+
+  health_check:
+    type: http
+    endpoint: http://localhost:8999
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3
+
+  bitcoin_integration:
+    rpc_access: read-only
+    sync_required: true

core/Cargo.lock

@@ -80,13 +80,14 @@ checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61"
 
 [[package]]
 name = "archipelago"
-version = "1.7.41-alpha"
+version = "1.7.43-alpha"
 dependencies = [
  "anyhow",
  "archipelago-container",
  "archipelago-performance",
  "archipelago-security",
  "argon2",
+ "async-trait",
  "base64 0.21.7",
  "bcrypt",
  "bip39",

core/archipelago/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "archipelago"
-version = "1.7.41-alpha"
+version = "1.7.44-alpha"
 edition = "2021"
 description = "Archipelago Bitcoin Node OS - Native backend"
 authors = ["Archipelago Team"]
@@ -103,6 +103,9 @@ mdns-sd = "0.18"
 # Systemd watchdog notification
 sd-notify = "0.4"
 
+# Trait objects for async methods (container orchestrator trait, Step 4)
+async-trait = "0.1"
+
 [dev-dependencies]
 tokio-test = "0.4"
 tempfile = "3.10"

core/archipelago/src/api/handler/mod.rs

@@ -10,6 +10,7 @@ mod websocket;
 use crate::api::rpc::RpcHandler;
 use crate::blobs::BlobStore;
 use crate::config::Config;
+use crate::container::{ContainerOrchestrator, DevContainerOrchestrator};
 use crate::monitoring::MetricsStore;
 use crate::session::{self, SessionStore};
 use crate::state::StateManager;
@@ -54,6 +55,8 @@ impl ApiHandler {
         config: Config,
         state_manager: Arc<StateManager>,
         metrics_store: Arc<MetricsStore>,
+        orchestrator: Option<Arc<dyn ContainerOrchestrator>>,
+        dev_orchestrator: Option<Arc<DevContainerOrchestrator>>,
     ) -> Result<Self> {
         let session_store = SessionStore::new().await;
         let rpc_handler = Arc::new(
@@ -62,6 +65,8 @@ impl ApiHandler {
                 state_manager.clone(),
                 metrics_store.clone(),
                 session_store.clone(),
+                orchestrator,
+                dev_orchestrator,
             )
             .await?,
         );
@@ -125,8 +130,7 @@ impl ApiHandler {
     /// persisted a registry config yet. 15s total timeout.
     async fn handle_app_catalog_proxy(&self) -> Result<Response<hyper::Body>> {
         let mut upstreams: Vec<String> = Vec::new();
-        if let Ok(config) =
-            crate::container::registry::load_registries(&self.config.data_dir).await
+        if let Ok(config) = crate::container::registry::load_registries(&self.config.data_dir).await
         {
             for reg in config.active_registries() {
                 let scheme = if reg.tls_verify { "https" } else { "http" };
@@ -141,7 +145,7 @@ impl ApiHandler {
         }
         if upstreams.is_empty() {
             upstreams.push(
-                "http://23.182.128.160:3000/lfg2025/app-catalog/raw/branch/main/catalog.json"
+                "http://146.59.87.168:3000/lfg2025/app-catalog/raw/branch/main/catalog.json"
                     .to_string(),
             );
             upstreams.push(
@@ -316,7 +320,7 @@ impl ApiHandler {
 
         match (method, path.as_str()) {
             // RPC — auth is handled inside rpc handler per-method
-            (Method::POST, "/rpc/v1") => self.rpc_handler.handle(req_with_bytes).await,
+            (Method::POST, "/rpc/v1") => self.rpc_handler.clone().handle(req_with_bytes).await,
 
             // Health — unauthenticated, returns JSON with service status
             (Method::GET, "/health") => {

core/archipelago/src/api/rpc/bitcoin.rs

@@ -3,6 +3,55 @@ use anyhow::{Context, Result};
 use serde::{Deserialize, Serialize};
 use zeroize::Zeroize;
 
+/// Retry configuration for [`bitcoin_rpc_post_with_retry`].
+///
+/// Exposed as a struct (rather than hard-coded constants inside the function)
+/// so tests can dial down timeouts to keep the suite fast while still
+/// exercising real retry/backoff behavior.
+#[derive(Debug, Clone)]
+struct RetryConfig {
+    max_attempts: u32,
+    attempt_timeout: std::time::Duration,
+    /// Length must equal `max_attempts - 1` (one backoff between each
+    /// successive attempt). The last attempt is not followed by a backoff.
+    backoffs: Vec<std::time::Duration>,
+}
+
+impl RetryConfig {
+    /// Production retry policy: 3 attempts, 15s each, 500ms + 1500ms backoffs.
+    /// Total worst-case wall time: 3 * 15 + 0.5 + 1.5 = 47s.
+    fn production() -> Self {
+        Self {
+            max_attempts: BITCOIN_RPC_MAX_ATTEMPTS,
+            attempt_timeout: BITCOIN_RPC_ATTEMPT_TIMEOUT,
+            backoffs: BITCOIN_RPC_BACKOFFS.to_vec(),
+        }
+    }
+}
+
+/// Max retry attempts for a single bitcoin_rpc_call invocation.
+/// First attempt + 2 retries = 3 total.
+const BITCOIN_RPC_MAX_ATTEMPTS: u32 = 3;
+
+/// Per-attempt deadline. Must be >= the reqwest client's own timeout (we
+/// build it at 15s in handle_bitcoin_getinfo) — this is the outer safety net.
+const BITCOIN_RPC_ATTEMPT_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(15);
+
+/// Backoff between attempts. Index 0 = after first failure, 1 = after second, etc.
+/// Chosen to absorb bitcoind's typical block-validation stall (2-5s) without
+/// adding noticeable latency on the happy path (first attempt succeeds in ~30ms).
+const BITCOIN_RPC_BACKOFFS: [std::time::Duration; 2] = [
+    std::time::Duration::from_millis(500),
+    std::time::Duration::from_millis(1500),
+];
+
+/// Classify a reqwest error as transient (retryable) or fatal.
+/// Transient: timeout, connect refused, request/response body IO errors.
+/// Fatal: TLS errors, URL parse errors, redirect loops, builder errors.
+fn is_transient_transport_error(e: &reqwest::Error) -> bool {
+    e.is_timeout() || e.is_connect() || e.is_request() || e.is_body()
+}
+
 #[derive(Debug, Serialize)]
 struct BitcoinInfo {
     block_height: u64,
@@ -37,8 +86,15 @@ struct MempoolInfo {
 
 impl RpcHandler {
     pub(super) async fn handle_bitcoin_getinfo(&self) -> Result<serde_json::Value> {
+        // Per-attempt timeout (see bitcoin_rpc_call for retry semantics).
+        // 15s is enough room for bitcoind to answer getblockchaininfo even
+        // during block validation; bitcoin_rpc_call wraps each attempt in a
+        // separate tokio::time::timeout too, so this is belt-and-suspenders.
+        // connect_timeout is tighter so a dead bitcoind doesn't steal the
+        // whole attempt budget on TCP connect alone.
         let client = reqwest::Client::builder()
-            .timeout(std::time::Duration::from_secs(10))
+            .timeout(std::time::Duration::from_secs(15))
+            .connect_timeout(std::time::Duration::from_secs(3))
             .build()
             .context("Failed to create HTTP client")?;
 
@@ -68,6 +124,19 @@ impl RpcHandler {
         Ok(serde_json::to_value(info)?)
     }
 
+    /// Call a Bitcoin Core JSON-RPC method.
+    ///
+    /// Retries up to [`BITCOIN_RPC_MAX_ATTEMPTS`] times on transient
+    /// transport errors (timeout / connection refused / send/recv IO).
+    /// Does **not** retry when bitcoind responds with a well-formed
+    /// `{"error": ...}` body — those are real RPC errors and surfacing
+    /// them quickly is the right behavior.
+    ///
+    /// Motivation: on a syncing pruned node, bitcoind's RPC thread can block
+    /// for 5-10 seconds during block validation. A single 10s timeout means
+    /// ~30% of UI calls error out even though the node is perfectly healthy.
+    /// With retry + backoff, the UI sees a uniform slow-but-successful
+    /// response instead of intermittent failures.
     async fn bitcoin_rpc_call<T: serde::de::DeserializeOwned>(
         &self,
         client: &reqwest::Client,
@@ -75,33 +144,15 @@ impl RpcHandler {
         params: &[serde_json::Value],
     ) -> Result<T> {
         let (rpc_user, rpc_pass) = crate::bitcoin_rpc::bitcoin_rpc_credentials().await;
-        let body = serde_json::json!({
-            "jsonrpc": "1.0",
-            "id": "archy",
-            "method": method,
-            "params": params,
-        });
-
-        let resp = client
-            .post(crate::constants::BITCOIN_RPC_URL)
-            .basic_auth(&rpc_user, Some(&rpc_pass))
-            .json(&body)
-            .send()
-            .await
-            .context("Bitcoin RPC connection failed")?;
-
-        let rpc_resp: BitcoinRpcResponse<T> = resp
-            .json()
-            .await
-            .context("Failed to parse Bitcoin RPC response")?;
-
-        if let Some(err) = rpc_resp.error {
-            anyhow::bail!("Bitcoin RPC error: {}", err);
-        }
-
-        rpc_resp
-            .result
-            .ok_or_else(|| anyhow::anyhow!("Bitcoin RPC returned null result"))
+        bitcoin_rpc_post_with_retry(
+            client,
+            crate::constants::BITCOIN_RPC_URL,
+            &rpc_user,
+            &rpc_pass,
+            method,
+            params,
+        )
+        .await
     }
 
     /// Initialize a Bitcoin Core descriptor wallet with keys derived from the master seed.
@@ -243,3 +294,351 @@ impl RpcHandler {
         }))
     }
 }
+
+/// Free-function counterpart to `RpcHandler::bitcoin_rpc_call`.
+///
+/// Takes the URL + credentials as parameters so it can be exercised by unit
+/// tests against a mock HTTP server without constructing a full `RpcHandler`.
+///
+/// Production callers go through `RpcHandler::bitcoin_rpc_call`, which loads
+/// credentials from the secrets file and points at `BITCOIN_RPC_URL`.
+async fn bitcoin_rpc_post_with_retry<T: serde::de::DeserializeOwned>(
+    client: &reqwest::Client,
+    url: &str,
+    rpc_user: &str,
+    rpc_pass: &str,
+    method: &str,
+    params: &[serde_json::Value],
+) -> Result<T> {
+    bitcoin_rpc_post_with_retry_cfg(
+        client,
+        url,
+        rpc_user,
+        rpc_pass,
+        method,
+        params,
+        &RetryConfig::production(),
+    )
+    .await
+}
+
+/// Inner implementation with configurable retry policy (for tests).
+async fn bitcoin_rpc_post_with_retry_cfg<T: serde::de::DeserializeOwned>(
+    client: &reqwest::Client,
+    url: &str,
+    rpc_user: &str,
+    rpc_pass: &str,
+    method: &str,
+    params: &[serde_json::Value],
+    cfg: &RetryConfig,
+) -> Result<T> {
+    debug_assert_eq!(
+        cfg.backoffs.len(),
+        (cfg.max_attempts - 1) as usize,
+        "RetryConfig: backoffs.len() must equal max_attempts - 1"
+    );
+
+    let body = serde_json::json!({
+        "jsonrpc": "1.0",
+        "id": "archy",
+        "method": method,
+        "params": params,
+    });
+
+    let mut last_err: Option<anyhow::Error> = None;
+    for attempt in 0..cfg.max_attempts {
+        if attempt > 0 {
+            let backoff = cfg
+                .backoffs
+                .get(attempt as usize - 1)
+                .copied()
+                .unwrap_or_else(|| std::time::Duration::from_secs(2));
+            tracing::warn!(
+                "bitcoin_rpc({}): attempt {} failed, backing off {:?}",
+                method,
+                attempt,
+                backoff
+            );
+            tokio::time::sleep(backoff).await;
+        }
+
+        // Per-attempt hard deadline. Independent of reqwest's built-in timeout
+        // so we always cap total time even if reqwest blocks on something
+        // weird (e.g., DNS starvation).
+        let fut = client
+            .post(url)
+            .basic_auth(rpc_user, Some(rpc_pass))
+            .json(&body)
+            .send();
+
+        let send_result = match tokio::time::timeout(cfg.attempt_timeout, fut).await {
+            Err(_elapsed) => {
+                last_err = Some(anyhow::anyhow!(
+                    "Bitcoin RPC send timed out after {:?}",
+                    cfg.attempt_timeout
+                ));
+                continue; // transient: retry
+            }
+            Ok(r) => r,
+        };
+
+        let resp = match send_result {
+            Ok(r) => r,
+            Err(e) if is_transient_transport_error(&e) => {
+                last_err = Some(anyhow::Error::from(e).context("Bitcoin RPC connection failed"));
+                continue; // transient: retry
+            }
+            Err(e) => {
+                return Err(anyhow::Error::from(e).context("Bitcoin RPC connection failed"));
+            }
+        };
+
+        let rpc_resp: BitcoinRpcResponse<T> = resp
+            .json()
+            .await
+            .context("Failed to parse Bitcoin RPC response")?;
+
+        if let Some(err) = rpc_resp.error {
+            // RPC-level error: this is a real bitcoind response, not transient.
+            anyhow::bail!("Bitcoin RPC error: {}", err);
+        }
+
+        return rpc_resp
+            .result
+            .ok_or_else(|| anyhow::anyhow!("Bitcoin RPC returned null result"));
+    }
+
+    Err(last_err
+        .unwrap_or_else(|| anyhow::anyhow!("Bitcoin RPC exhausted retries with no error captured")))
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use hyper::service::{make_service_fn, service_fn};
+    use hyper::{Body, Request, Response, Server, StatusCode};
+    use std::convert::Infallible;
+    use std::net::SocketAddr;
+    use std::sync::atomic::{AtomicU32, Ordering};
+    use std::sync::Arc;
+
+    /// Spin up a mock bitcoind HTTP server that behaves according to `handler`.
+    /// Returns the bound URL and a JoinHandle (dropped = server shutdown via the
+    /// oneshot cancel channel).
+    async fn spawn_mock<F, Fut>(
+        handler: F,
+    ) -> (
+        String,
+        tokio::task::JoinHandle<()>,
+        tokio::sync::oneshot::Sender<()>,
+    )
+    where
+        F: Fn(Request<Body>) -> Fut + Send + Sync + Clone + 'static,
+        Fut: std::future::Future<Output = Response<Body>> + Send + 'static,
+    {
+        let addr = SocketAddr::from(([127, 0, 0, 1], 0));
+        let make_svc = make_service_fn(move |_| {
+            let handler = handler.clone();
+            async move {
+                Ok::<_, Infallible>(service_fn(move |req| {
+                    let handler = handler.clone();
+                    async move { Ok::<_, Infallible>(handler(req).await) }
+                }))
+            }
+        });
+        let server = Server::bind(&addr).serve(make_svc);
+        let url = format!("http://{}", server.local_addr());
+        let (tx, rx) = tokio::sync::oneshot::channel::<()>();
+        let handle = tokio::spawn(async move {
+            let graceful = server.with_graceful_shutdown(async {
+                let _ = rx.await;
+            });
+            let _ = graceful.await;
+        });
+        (url, handle, tx)
+    }
+
+    /// Reply body bitcoind would send for a successful getblockcount.
+    fn ok_reply() -> Body {
+        Body::from(r#"{"result":42,"error":null,"id":"archy"}"#)
+    }
+
+    fn err_reply() -> Body {
+        Body::from(r#"{"result":null,"error":{"code":-8,"message":"nope"},"id":"archy"}"#)
+    }
+
+    /// Succeeds on first attempt — should not retry.
+    #[tokio::test]
+    async fn happy_path_first_attempt() {
+        let count = Arc::new(AtomicU32::new(0));
+        let c = count.clone();
+        let (url, _h, _tx) = spawn_mock(move |_req| {
+            let c = c.clone();
+            async move {
+                c.fetch_add(1, Ordering::SeqCst);
+                Response::new(ok_reply())
+            }
+        })
+        .await;
+
+        let client = reqwest::Client::builder().build().unwrap();
+        let v: u64 =
+            bitcoin_rpc_post_with_retry(&client, &url, "user", "pass", "getblockcount", &[])
+                .await
+                .expect("should succeed");
+        assert_eq!(v, 42);
+        assert_eq!(count.load(Ordering::SeqCst), 1, "should not have retried");
+    }
+
+    /// HTTP 503 with non-JSON body: produces a JSON-parse error which is NOT
+    /// classified as transient. Must fail after first attempt.
+    /// This guards against the tempting mistake of blanket-retrying every
+    /// non-2xx response — which would mask real bitcoind misconfig.
+    #[tokio::test]
+    async fn does_not_retry_parse_errors() {
+        let count = Arc::new(AtomicU32::new(0));
+        let c = count.clone();
+        let (url, _h, _tx) = spawn_mock(move |_req| {
+            let c = c.clone();
+            async move {
+                c.fetch_add(1, Ordering::SeqCst);
+                Response::builder()
+                    .status(StatusCode::SERVICE_UNAVAILABLE)
+                    .body(Body::from("busy"))
+                    .unwrap()
+            }
+        })
+        .await;
+
+        let client = reqwest::Client::builder().build().unwrap();
+        let result: Result<u64> =
+            bitcoin_rpc_post_with_retry(&client, &url, "user", "pass", "getblockcount", &[]).await;
+        assert!(result.is_err(), "non-JSON response should error out");
+        assert_eq!(
+            count.load(Ordering::SeqCst),
+            1,
+            "parse errors are not retryable"
+        );
+    }
+
+    /// Connect-refused (port closed) is the canonical transient transport
+    /// error. Must exhaust BITCOIN_RPC_MAX_ATTEMPTS and the total elapsed
+    /// time must include at least the sum of the backoffs.
+    #[tokio::test]
+    async fn retries_exhausted_on_persistent_connect_refused() {
+        // Bind a port then immediately drop the listener so the port is closed.
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let closed_url = format!("http://{}", listener.local_addr().unwrap());
+        drop(listener);
+
+        let client = reqwest::Client::builder()
+            .connect_timeout(std::time::Duration::from_millis(500))
+            .build()
+            .unwrap();
+        let start = std::time::Instant::now();
+        let result: Result<u64> =
+            bitcoin_rpc_post_with_retry(&client, &closed_url, "user", "pass", "getblockcount", &[])
+                .await;
+        let elapsed = start.elapsed();
+        assert!(result.is_err(), "connect-refused should exhaust retries");
+        let min_backoff: std::time::Duration = BITCOIN_RPC_BACKOFFS.iter().sum();
+        assert!(
+            elapsed >= min_backoff,
+            "should have backed off between retries (elapsed={:?}, expected at least {:?})",
+            elapsed,
+            min_backoff
+        );
+    }
+
+    /// The motivating scenario: first attempt times out (bitcoind busy),
+    /// subsequent attempt succeeds. Uses a short test-only RetryConfig so
+    /// the test runs in <1s instead of 15s.
+    #[tokio::test]
+    async fn retries_on_timeout_then_succeeds() {
+        let count = Arc::new(AtomicU32::new(0));
+        let c = count.clone();
+        // Mock server: first request hangs for 500ms, subsequent requests reply OK.
+        let (url, _h, _tx) = spawn_mock(move |_req| {
+            let c = c.clone();
+            async move {
+                let n = c.fetch_add(1, Ordering::SeqCst);
+                if n == 0 {
+                    tokio::time::sleep(std::time::Duration::from_millis(500)).await;
+                }
+                Response::new(ok_reply())
+            }
+        })
+        .await;
+
+        let client = reqwest::Client::builder().build().unwrap();
+        // Attempt timeout 100ms < server's 500ms sleep => first attempt times out.
+        // Backoff 20ms between attempts.
+        let cfg = RetryConfig {
+            max_attempts: 3,
+            attempt_timeout: std::time::Duration::from_millis(100),
+            backoffs: vec![
+                std::time::Duration::from_millis(20),
+                std::time::Duration::from_millis(20),
+            ],
+        };
+        let v: u64 = bitcoin_rpc_post_with_retry_cfg(
+            &client,
+            &url,
+            "user",
+            "pass",
+            "getblockcount",
+            &[],
+            &cfg,
+        )
+        .await
+        .expect("second attempt should succeed");
+        assert_eq!(v, 42);
+        assert!(
+            count.load(Ordering::SeqCst) >= 2,
+            "expected at least 2 attempts (got {})",
+            count.load(Ordering::SeqCst)
+        );
+    }
+
+    /// bitcoind returned a well-formed `{"error": ...}` body. Must NOT retry.
+    #[tokio::test]
+    async fn does_not_retry_on_rpc_level_error() {
+        let count = Arc::new(AtomicU32::new(0));
+        let c = count.clone();
+        let (url, _h, _tx) = spawn_mock(move |_req| {
+            let c = c.clone();
+            async move {
+                c.fetch_add(1, Ordering::SeqCst);
+                Response::new(err_reply())
+            }
+        })
+        .await;
+
+        let client = reqwest::Client::builder().build().unwrap();
+        let result: Result<u64> =
+            bitcoin_rpc_post_with_retry(&client, &url, "user", "pass", "getblockcount", &[]).await;
+        assert!(result.is_err());
+        assert_eq!(
+            count.load(Ordering::SeqCst),
+            1,
+            "RPC-level errors are not transient"
+        );
+    }
+
+    /// Sanity: retry budget invariants. Chosen to catch regressions where
+    /// someone bumps these constants without realizing the total worst-case
+    /// wall time implications.
+    #[test]
+    fn retry_budget_invariants() {
+        assert_eq!(BITCOIN_RPC_MAX_ATTEMPTS, 3);
+        assert_eq!(
+            BITCOIN_RPC_BACKOFFS.len(),
+            (BITCOIN_RPC_MAX_ATTEMPTS - 1) as usize
+        );
+        // Total wall-time ceiling:
+        //   3 attempts * 15s  +  (0.5s + 1.5s) backoff  =  47s
+        let total: std::time::Duration = BITCOIN_RPC_ATTEMPT_TIMEOUT * BITCOIN_RPC_MAX_ATTEMPTS
+            + BITCOIN_RPC_BACKOFFS.iter().sum::<std::time::Duration>();
+        assert!(total < std::time::Duration::from_secs(60));
+    }
+}

core/archipelago/src/api/rpc/container.rs

@@ -1,4 +1,5 @@
 use super::package::validate_app_id;
+use super::transitional::Op;
 use super::RpcHandler;
 use anyhow::{Context, Result};
 
@@ -7,8 +8,13 @@ impl RpcHandler {
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
+        // The `container-install { manifest_path }` RPC is a dev-mode convenience
+        // that points at an arbitrary YAML on disk. Production install happens via
+        // the reconciler (BootReconciler, Step 5) and via the unified
+        // ContainerOrchestrator::install(app_id) trait call, which can be exposed
+        // through a separate `container-install-by-id` RPC when needed.
+        let dev = self.dev_orchestrator.as_ref().ok_or_else(|| {
+            anyhow::anyhow!("container-install with manifest_path is only available in dev mode")
         })?;
 
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
@@ -45,7 +51,7 @@ impl RpcHandler {
         let manifest: archipelago_container::AppManifest =
             serde_yaml::from_str(&manifest_content).context("Failed to parse manifest")?;
 
-        let container_name = orchestrator
+        let container_name = dev
             .install_container(&manifest, manifest_path)
             .await
             .context("Failed to install container")?;
@@ -57,10 +63,6 @@ impl RpcHandler {
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
-
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
         let app_id = params
             .get("app_id")
@@ -68,22 +70,24 @@ impl RpcHandler {
             .ok_or_else(|| anyhow::anyhow!("Missing app_id"))?;
         validate_app_id(app_id)?;
 
-        orchestrator
-            .start_container(app_id)
-            .await
-            .context("Failed to start container")?;
+        // User explicitly started the app — clear the user-stopped marker so
+        // crash recovery / health monitor won't second-guess it. Must happen
+        // BEFORE the spawn (see runtime.rs:145-148 for the symmetric stop
+        // side and the ordering contract crash recovery depends on).
+        crate::crash_recovery::clear_user_stopped(&self.config.data_dir, app_id).await;
 
-        Ok(serde_json::json!({ "status": "started" }))
+        // spawn_transitional returns as soon as the background task is
+        // launched (<1s). The UI sees Starting… immediately via WebSocket.
+        self.spawn_transitional(Op::Start, app_id.to_string())
+            .await?;
+
+        Ok(serde_json::json!({ "status": "starting" }))
     }
 
     pub(super) async fn handle_container_stop(
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
-
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
         let app_id = params
             .get("app_id")
@@ -91,21 +95,51 @@ impl RpcHandler {
             .ok_or_else(|| anyhow::anyhow!("Missing app_id"))?;
         validate_app_id(app_id)?;
 
-        orchestrator
-            .stop_container(app_id)
-            .await
-            .context("Failed to stop container")?;
+        // Mark as user-stopped BEFORE the spawn — ordering is load-bearing
+        // (crash recovery / health monitor inspect this flag concurrently
+        // with the in-flight stop; see runtime.rs:145-148 for the package
+        // path that also writes this in the same order).
+        crate::crash_recovery::mark_user_stopped(&self.config.data_dir, app_id).await;
 
-        Ok(serde_json::json!({ "status": "stopped" }))
+        // podman stop -t 600 (bitcoin-core) / -t 330 (lnd) runs in the
+        // background; the RPC returns now with "stopping".
+        self.spawn_transitional(Op::Stop, app_id.to_string())
+            .await?;
+
+        Ok(serde_json::json!({ "status": "stopping" }))
+    }
+
+    pub(super) async fn handle_container_restart(
+        &self,
+        params: Option<serde_json::Value>,
+    ) -> Result<serde_json::Value> {
+        let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
+        let app_id = params
+            .get("app_id")
+            .and_then(|v| v.as_str())
+            .ok_or_else(|| anyhow::anyhow!("Missing app_id"))?;
+        validate_app_id(app_id)?;
+
+        // Restart does not mark user-stopped (the user wants the app to
+        // keep running). Clear the marker as a defensive measure in case a
+        // prior stop left it set and the restart is intended to revive the
+        // normal running state.
+        crate::crash_recovery::clear_user_stopped(&self.config.data_dir, app_id).await;
+
+        self.spawn_transitional(Op::Restart, app_id.to_string())
+            .await?;
+
+        Ok(serde_json::json!({ "status": "restarting" }))
     }
 
     pub(super) async fn handle_container_remove(
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
 
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
         let app_id = params
@@ -119,7 +153,7 @@ impl RpcHandler {
             .unwrap_or(false);
 
         orchestrator
-            .remove_container(app_id, preserve_data)
+            .remove(app_id, preserve_data)
             .await
             .context("Failed to remove container")?;
 
@@ -137,12 +171,25 @@ impl RpcHandler {
                 .package_data
                 .iter()
                 .map(|(id, pkg)| {
+                    // Keep this mapping in sync with the UI's
+                    // ContainerStatus.state union in
+                    // neode-ui/src/api/container-client.ts. The UI maps
+                    // transitional variants to single-button labels
+                    // (Stopping… / Starting… / Restarting…).
                     let state = match &pkg.state {
                         crate::data_model::PackageState::Running => "running",
                         crate::data_model::PackageState::Stopped => "stopped",
                         crate::data_model::PackageState::Exited => "exited",
-                        crate::data_model::PackageState::Starting => "created",
-                        _ => "unknown",
+                        crate::data_model::PackageState::Starting => "starting",
+                        crate::data_model::PackageState::Stopping => "stopping",
+                        crate::data_model::PackageState::Restarting => "restarting",
+                        crate::data_model::PackageState::Installing => "installing",
+                        crate::data_model::PackageState::Installed => "installed",
+                        crate::data_model::PackageState::Updating => "updating",
+                        crate::data_model::PackageState::Removing => "removing",
+                        crate::data_model::PackageState::CreatingBackup => "creating-backup",
+                        crate::data_model::PackageState::RestoringBackup => "restoring-backup",
+                        crate::data_model::PackageState::BackingUp => "backing-up",
                     };
                     let lan = pkg
                         .installed
@@ -163,9 +210,9 @@ impl RpcHandler {
             return Ok(serde_json::json!(containers));
         }
 
-        // Fallback: scanner hasn't run yet, query podman directly
+        // Fallback: scanner hasn't run yet, query the orchestrator directly.
         if let Some(orchestrator) = &self.orchestrator {
-            if let Ok(containers) = orchestrator.list_containers().await {
+            if let Ok(containers) = orchestrator.list().await {
                 if !containers.is_empty() {
                     return Ok(serde_json::to_value(containers)?);
                 }
@@ -242,9 +289,10 @@ impl RpcHandler {
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
 
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
         let app_id = params
@@ -253,21 +301,36 @@ impl RpcHandler {
             .ok_or_else(|| anyhow::anyhow!("Missing app_id"))?;
         validate_app_id(app_id)?;
 
-        let status = orchestrator
-            .get_container_status(app_id)
-            .await
-            .context("Failed to get container status")?;
+        let mut last_err: Option<anyhow::Error> = None;
+        for candidate in status_app_id_candidates(app_id) {
+            match orchestrator.status(&candidate).await {
+                Ok(status) => return Ok(serde_json::to_value(status)?),
+                Err(e) => last_err = Some(e),
+            }
+        }
 
-        Ok(serde_json::to_value(status)?)
+        // Fallback for alias drift: query podman directly by likely container
+        // names so status checks stay useful during migration.
+        for name in status_container_name_candidates(app_id) {
+            if let Some(v) = inspect_container_state_value(&name).await {
+                return Ok(v);
+            }
+        }
+
+        if let Some(e) = last_err {
+            return Err(e.context("Failed to get container status"));
+        }
+        Err(anyhow::anyhow!("Failed to get container status"))
     }
 
     pub(super) async fn handle_container_logs(
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
 
         let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
         let app_id = params
@@ -278,7 +341,7 @@ impl RpcHandler {
         let lines = params.get("lines").and_then(|v| v.as_u64()).unwrap_or(100) as u32;
 
         let logs = orchestrator
-            .get_container_logs(app_id, lines)
+            .logs(app_id, lines)
             .await
             .context("Failed to get container logs")?;
 
@@ -291,12 +354,13 @@ impl RpcHandler {
         app_id: &str,
         lines: u32,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
 
         let logs = orchestrator
-            .get_container_logs(app_id, lines)
+            .logs(app_id, lines)
             .await
             .context("Failed to get container logs")?;
 
@@ -307,43 +371,52 @@ impl RpcHandler {
         &self,
         params: Option<serde_json::Value>,
     ) -> Result<serde_json::Value> {
-        let orchestrator = self.orchestrator.as_ref().ok_or_else(|| {
-            anyhow::anyhow!("Container orchestrator not available (dev mode required)")
-        })?;
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?;
 
-        // If app_id is provided, get health for that app
+        // If app_id is provided, get health for that app.
         if let Some(params) = params {
             if let Some(app_id) = params.get("app_id").and_then(|v| v.as_str()) {
                 let health = orchestrator
-                    .get_health_status(app_id)
+                    .health(app_id)
                     .await
                     .context("Failed to get container health")?;
                 return Ok(serde_json::json!({ app_id: health }));
             }
         }
 
-        // Otherwise, get health for all containers
+        // Otherwise, get health for all containers.
         let containers = orchestrator
-            .list_containers()
+            .list()
             .await
             .context("Failed to list containers")?;
 
         let mut health_map = serde_json::Map::new();
         for container in containers {
-            if let Some(app_id) = container.name.strip_prefix("archipelago-") {
-                if let Some(app_id) = app_id.strip_suffix("-dev") {
-                    match orchestrator.get_health_status(app_id).await {
-                        Ok(health) => {
-                            health_map
-                                .insert(app_id.to_string(), serde_json::Value::String(health));
-                        }
-                        Err(_) => {
-                            health_map.insert(
-                                app_id.to_string(),
-                                serde_json::Value::String("unknown".to_string()),
-                            );
-                        }
-                    }
+            // Map the runtime container name back to the app_id the orchestrator
+            // knows about. Dev orchestrator uses `archipelago-<id>-dev`; Prod
+            // uses bare `<id>` (or `archy-<id>` for UIs — health() accepts the
+            // app_id either way since UI_APP_IDS is centralised).
+            let app_id_candidate = container
+                .name
+                .strip_prefix("archipelago-")
+                .and_then(|s| s.strip_suffix("-dev"))
+                .or_else(|| container.name.strip_prefix("archy-"))
+                .unwrap_or(container.name.as_str());
+            match orchestrator.health(app_id_candidate).await {
+                Ok(health) => {
+                    health_map.insert(
+                        app_id_candidate.to_string(),
+                        serde_json::Value::String(health),
+                    );
+                }
+                Err(_) => {
+                    health_map.insert(
+                        app_id_candidate.to_string(),
+                        serde_json::Value::String("unknown".to_string()),
+                    );
                 }
             }
         }
@@ -351,3 +424,90 @@ impl RpcHandler {
         Ok(serde_json::Value::Object(health_map))
     }
 }
+
+fn status_app_id_candidates(app_id: &str) -> Vec<String> {
+    let mut out = Vec::new();
+    let mut push = |s: &str| {
+        if !out.iter().any(|e: &String| e == s) {
+            out.push(s.to_string());
+        }
+    };
+
+    match app_id {
+        "bitcoin-knots" => {
+            push("bitcoin-knots");
+            push("bitcoin-core");
+            push("bitcoin");
+        }
+        "bitcoin-core" | "bitcoin" => {
+            push("bitcoin-core");
+            push("bitcoin-knots");
+            push("bitcoin");
+        }
+        "electrs" | "mempool-electrs" => {
+            push("electrs");
+            push("mempool-electrs");
+            push("electrumx");
+        }
+        _ => push(app_id),
+    }
+
+    out
+}
+
+fn status_container_name_candidates(app_id: &str) -> Vec<String> {
+    let mut out = Vec::new();
+    let mut push = |s: &str| {
+        if !out.iter().any(|e: &String| e == s) {
+            out.push(s.to_string());
+        }
+    };
+
+    match app_id {
+        "bitcoin-knots" | "bitcoin-core" | "bitcoin" => push("bitcoin-knots"),
+        "bitcoin-ui" => push("archy-bitcoin-ui"),
+        "lnd-ui" => push("archy-lnd-ui"),
+        "electrs-ui" => push("archy-electrs-ui"),
+        "electrs" | "mempool-electrs" => push("electrumx"),
+        _ => {}
+    }
+
+    push(app_id);
+    if let Some(stripped) = app_id.strip_prefix("archy-") {
+        push(stripped);
+    } else {
+        push(&format!("archy-{}", app_id));
+    }
+
+    out
+}
+
+async fn inspect_container_state_value(name: &str) -> Option<serde_json::Value> {
+    let out = tokio::process::Command::new("podman")
+        .args([
+            "inspect",
+            name,
+            "--format",
+            "{{.State.Status}} {{.State.Running}}",
+        ])
+        .output()
+        .await
+        .ok()?;
+    if !out.status.success() {
+        return None;
+    }
+
+    let line = String::from_utf8_lossy(&out.stdout).trim().to_string();
+    if line.is_empty() {
+        return None;
+    }
+    let mut parts = line.split_whitespace();
+    let status = parts.next().unwrap_or("unknown");
+    let running = parts.next().unwrap_or("false") == "true";
+    Some(serde_json::json!({
+        "name": name,
+        "status": status,
+        "state": status,
+        "running": running,
+    }))
+}

core/archipelago/src/api/rpc/content.rs

@@ -231,8 +231,7 @@ impl RpcHandler {
 
         let (data, _) = self.state_manager.get_snapshot().await;
         let local_did = crate::identity::did_key_from_pubkey_hex(&data.server_info.pubkey)?;
-        let fips_npub =
-            crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
+        let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
 
         let path = format!("/content/{}", content_id);
         let (response, _transport) =
@@ -287,10 +286,13 @@ impl RpcHandler {
             return Err(anyhow::anyhow!("Invalid v3 onion address"));
         }
 
-        let fips_npub =
-            crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
+        let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
 
-        debug!("Browsing peer content at {} (fips={})", onion, fips_npub.is_some());
+        debug!(
+            "Browsing peer content at {} (fips={})",
+            onion,
+            fips_npub.is_some()
+        );
 
         let (response, _transport) =
             crate::fips::dial::PeerRequest::new(fips_npub.as_deref(), onion, "/content")
@@ -348,8 +350,7 @@ impl RpcHandler {
 
         let (data, _) = self.state_manager.get_snapshot().await;
         let local_did = crate::identity::did_key_from_pubkey_hex(&data.server_info.pubkey)?;
-        let fips_npub =
-            crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
+        let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
 
         let path = format!("/content/{}", content_id);
         let (response, _transport) =
@@ -407,11 +408,15 @@ impl RpcHandler {
             return Err(anyhow::anyhow!("Invalid v3 onion address"));
         }
 
-        let fips_npub =
-            crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
+        let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
 
         let path = format!("/content/{}/preview", content_id);
-        debug!("Fetching content preview from {}{} (fips={})", onion, path, fips_npub.is_some());
+        debug!(
+            "Fetching content preview from {}{} (fips={})",
+            onion,
+            path,
+            fips_npub.is_some()
+        );
 
         let (response, _transport) =
             crate::fips::dial::PeerRequest::new(fips_npub.as_deref(), onion, &path)

core/archipelago/src/api/rpc/dispatcher.rs

@@ -1,10 +1,11 @@
 use super::RpcHandler;
 use anyhow::Result;
+use std::sync::Arc;
 
 impl RpcHandler {
     /// Route an RPC method name to its handler, returning the result value.
     pub(super) async fn dispatch(
-        &self,
+        self: &Arc<Self>,
         method: &str,
         params: Option<serde_json::Value>,
         session_token: &Option<String>,
@@ -36,19 +37,23 @@ impl RpcHandler {
             "container-install" => self.handle_container_install(params).await,
             "container-start" => self.handle_container_start(params).await,
             "container-stop" => self.handle_container_stop(params).await,
+            "container-restart" => self.handle_container_restart(params).await,
             "container-remove" => self.handle_container_remove(params).await,
             "container-list" => self.handle_container_list().await,
             "container-status" => self.handle_container_status(params).await,
             "container-logs" => self.handle_container_logs(params).await,
             "container-health" => self.handle_container_health(params).await,
 
-            // Package management (for docker-compose apps)
-            "package.install" => self.handle_package_install(params).await,
+            // Package management (for docker-compose apps).
+            // install/uninstall/update return immediately with a
+            // transitional status; the actual work runs in a background
+            // tokio::spawn so the HTTP request doesn't block for minutes.
+            "package.install" => self.clone().spawn_package_install(params).await,
             "package.start" => self.handle_package_start(params).await,
             "package.stop" => self.handle_package_stop(params).await,
             "package.restart" => self.handle_package_restart(params).await,
-            "package.uninstall" => self.handle_package_uninstall(params).await,
-            "package.update" => self.handle_package_update(params).await,
+            "package.uninstall" => self.clone().spawn_package_uninstall(params).await,
+            "package.update" => self.clone().spawn_package_update(params).await,
             "app.filebrowser-token" => self.handle_filebrowser_token().await,
 
             // Bundled app management (for pre-loaded container images)

core/archipelago/src/api/rpc/federation/handlers.rs

@@ -403,7 +403,10 @@ impl RpcHandler {
             });
         let own_fips_npub = match own_fips_npub {
             Some(n) => Some(n),
-            None => crate::fips::service::read_upstream_npub().await.ok().flatten(),
+            None => crate::fips::service::read_upstream_npub()
+                .await
+                .ok()
+                .flatten(),
         };
 
         let state = federation::build_local_state(
@@ -461,8 +464,7 @@ impl RpcHandler {
         // the entry causes sync loops where the node syncs with itself
         // forever. Drop it quietly — no useful recovery path.
         let (own_data, _) = self.state_manager.get_snapshot().await;
-        let own_did_result =
-            identity::did_key_from_pubkey_hex(&own_data.server_info.pubkey).ok();
+        let own_did_result = identity::did_key_from_pubkey_hex(&own_data.server_info.pubkey).ok();
         let own_onion_trim = own_data
             .server_info
             .tor_address
@@ -568,11 +570,7 @@ impl RpcHandler {
         let new_peer_did = did.to_string();
         tokio::spawn(async move {
             tokio::time::sleep(std::time::Duration::from_secs(2)).await;
-            if let Err(e) = crate::federation::sync_with_peer_by_did(
-                &data_dir,
-                &new_peer_did,
-            )
-            .await
+            if let Err(e) = crate::federation::sync_with_peer_by_did(&data_dir, &new_peer_did).await
             {
                 tracing::debug!(
                     peer_did = %new_peer_did,

core/archipelago/src/api/rpc/fips.rs

@@ -169,8 +169,7 @@ impl RpcHandler {
         if !anchor.address.contains(':') {
             anyhow::bail!("address must be host:port (e.g. 192.168.1.116:8668)");
         }
-        let list =
-            fips::anchors::add(&self.config.data_dir, anchor.clone()).await?;
+        let list = fips::anchors::add(&self.config.data_dir, anchor.clone()).await?;
         // Push just the newly-added anchor into the running daemon so
         // the user sees effect without waiting for the periodic apply.
         let results = fips::anchors::apply(&[anchor]).await;

core/archipelago/src/api/rpc/identity/handlers.rs

@@ -742,24 +742,25 @@ impl RpcHandler {
             .ok_or_else(|| anyhow::anyhow!("Missing required parameter: id"))?;
         validate_identity_id(id)?;
 
-        let relay_urls: Vec<String> = if let Some(arr) = params.get("relays").and_then(|v| v.as_array()) {
-            arr.iter()
-                .filter_map(|v| v.as_str())
-                .map(|s| s.to_string())
-                .collect()
-        } else if let Some(single) = params.get("relay").and_then(|v| v.as_str()) {
-            vec![single.to_string()]
-        } else {
-            // Default: every enabled relay in the user's Manage Relays list.
-            let statuses = crate::nostr_relays::list_relays(&self.config.data_dir)
-                .await
-                .unwrap_or_default();
-            statuses
-                .into_iter()
-                .filter(|s| s.enabled)
-                .map(|s| s.url)
-                .collect()
-        };
+        let relay_urls: Vec<String> =
+            if let Some(arr) = params.get("relays").and_then(|v| v.as_array()) {
+                arr.iter()
+                    .filter_map(|v| v.as_str())
+                    .map(|s| s.to_string())
+                    .collect()
+            } else if let Some(single) = params.get("relay").and_then(|v| v.as_str()) {
+                vec![single.to_string()]
+            } else {
+                // Default: every enabled relay in the user's Manage Relays list.
+                let statuses = crate::nostr_relays::list_relays(&self.config.data_dir)
+                    .await
+                    .unwrap_or_default();
+                statuses
+                    .into_iter()
+                    .filter(|s| s.enabled)
+                    .map(|s| s.url)
+                    .collect()
+            };
 
         if relay_urls.is_empty() {
             anyhow::bail!("No enabled relays configured; add one under Manage Relays");

core/archipelago/src/api/rpc/lnd/info.rs

@@ -3,7 +3,7 @@ use anyhow::{Context, Result};
 use base64::Engine;
 use serde::{Deserialize, Serialize};
 
-use super::{LndAmount, LndBalanceResponse};
+use super::{read_lnd_admin_macaroon, LndAmount, LndBalanceResponse};
 
 #[derive(Debug, Serialize)]
 struct LndInfo {
@@ -34,11 +34,7 @@ struct LndChannelBalanceResponse {
 
 impl RpcHandler {
     pub(in crate::api::rpc) async fn handle_lnd_getinfo(&self) -> Result<serde_json::Value> {
-        let macaroon_path = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
-
-        let macaroon_bytes = tokio::fs::read(macaroon_path)
-            .await
-            .context("Failed to read LND admin macaroon — is LND installed?")?;
+        let macaroon_bytes = read_lnd_admin_macaroon().await?;
         let macaroon_hex = hex::encode(&macaroon_bytes);
 
         let client = reqwest::Client::builder()
@@ -114,7 +110,6 @@ impl RpcHandler {
     /// for building lndconnect:// URIs in the frontend.
     pub(crate) async fn handle_lnd_connect_info(&self) -> Result<serde_json::Value> {
         let cert_path = "/var/lib/archipelago/lnd/tls.cert";
-        let macaroon_path = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
 
         // Read and encode TLS cert (PEM -> DER -> base64url)
         let cert_pem = tokio::fs::read_to_string(cert_path)
@@ -130,9 +125,7 @@ impl RpcHandler {
         let cert_b64url = base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(&cert_der);
 
         // Read and encode macaroon (binary -> base64url)
-        let macaroon_bytes = tokio::fs::read(macaroon_path)
-            .await
-            .context("Failed to read LND admin macaroon")?;
+        let macaroon_bytes = read_lnd_admin_macaroon().await?;
         let macaroon_b64url =
             base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(&macaroon_bytes);
 
@@ -183,10 +176,7 @@ impl RpcHandler {
     pub(in crate::api::rpc) async fn handle_lnd_export_channel_backup(
         &self,
     ) -> Result<serde_json::Value> {
-        let macaroon_path = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
-        let macaroon_bytes = tokio::fs::read(macaroon_path)
-            .await
-            .context("Failed to read LND admin macaroon")?;
+        let macaroon_bytes = read_lnd_admin_macaroon().await?;
         let macaroon_hex = hex::encode(&macaroon_bytes);
 
         let client = reqwest::Client::builder()

core/archipelago/src/api/rpc/lnd/mod.rs

@@ -4,7 +4,11 @@ mod payments;
 mod wallet;
 
 use crate::api::rpc::RpcHandler;
-use anyhow::{Context, Result};
+use anyhow::{anyhow, Context, Result};
+
+/// Canonical on-host path for LND's admin macaroon.
+pub(crate) const LND_ADMIN_MACAROON_PATH: &str =
+    "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
 
 // Shared LND response types used by multiple submodules
 #[derive(Debug, serde::Deserialize)]
@@ -17,15 +21,45 @@ pub(super) struct LndAmount {
     pub sat: Option<String>,
 }
 
+/// Read LND's admin macaroon from disk.
+///
+/// The macaroon lives inside LND's container data dir and is owned by a
+/// rootless-podman subordinate UID (typically 100000), mode 640. The
+/// archipelago server runs as UID 1000 and therefore cannot read it
+/// directly. We first try a plain read (works if an operator has relaxed
+/// permissions), then fall back to `sudo cat` — mirroring the pattern
+/// already used for Tor hidden-service hostnames.
+pub(crate) async fn read_lnd_admin_macaroon() -> Result<Vec<u8>> {
+    match tokio::fs::read(LND_ADMIN_MACAROON_PATH).await {
+        Ok(bytes) => Ok(bytes),
+        Err(direct_err) => {
+            let output = tokio::process::Command::new("sudo")
+                .args(["-n", "cat", LND_ADMIN_MACAROON_PATH])
+                .output()
+                .await
+                .with_context(|| {
+                    format!(
+                        "Failed to read LND admin macaroon (direct: {direct_err}); sudo fallback also failed"
+                    )
+                })?;
+            if !output.status.success() {
+                let stderr = String::from_utf8_lossy(&output.stderr);
+                return Err(anyhow!(
+                    "Failed to read LND admin macaroon — is LND installed? (direct: {direct_err}; sudo: {})",
+                    stderr.trim()
+                ));
+            }
+            Ok(output.stdout)
+        }
+    }
+}
+
 impl RpcHandler {
     /// Helper: create an authenticated LND REST client.
     /// Returns an HTTP client configured for LND's self-signed TLS and the
     /// hex-encoded admin macaroon for request headers.
     pub(crate) async fn lnd_client(&self) -> Result<(reqwest::Client, String)> {
-        let macaroon_path = "/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon";
-        let macaroon_bytes = tokio::fs::read(macaroon_path)
-            .await
-            .context("Failed to read LND admin macaroon — is LND installed?")?;
+        let macaroon_bytes = read_lnd_admin_macaroon().await?;
         let macaroon_hex = hex::encode(&macaroon_bytes);
         let client = reqwest::Client::builder()
             .timeout(std::time::Duration::from_secs(15))

core/archipelago/src/api/rpc/mesh/typed_messages.rs

@@ -761,7 +761,9 @@ impl RpcHandler {
             .await
             .map_err(|e| anyhow::anyhow!("Read body failed: {}", e))?;
 
-        let meta = blob_store.put(&bytes, &mime, filename_hint, None, false).await?;
+        let meta = blob_store
+            .put(&bytes, &mime, filename_hint, None, false)
+            .await?;
         if meta.cid != cid {
             anyhow::bail!("CID mismatch: expected {}, got {}", cid, meta.cid);
         }

core/archipelago/src/api/rpc/mod.rs

@@ -31,6 +31,7 @@ mod streaming;
 mod system;
 mod tor;
 mod totp;
+mod transitional;
 mod transport;
 mod update;
 mod vpn;
@@ -39,7 +40,7 @@ mod webhooks;
 
 use crate::auth::AuthManager;
 use crate::config::Config;
-use crate::container::DevContainerOrchestrator;
+use crate::container::{ContainerOrchestrator, DevContainerOrchestrator};
 use crate::monitoring::MetricsStore;
 use crate::port_allocator::PortAllocator;
 use crate::rate_limit::{EndpointRateLimiter, LoginRateLimiter};
@@ -62,7 +63,14 @@ pub(crate) const DEV_DEFAULT_PASSWORD: &str = "password123";
 pub struct RpcHandler {
     config: Config,
     auth_manager: AuthManager,
-    orchestrator: Option<Arc<DevContainerOrchestrator>>,
+    /// Shared lifecycle orchestrator (Dev or Prod). Always `Some` in a normal
+    /// build — the only reason it is `Option` is so tests that don't exercise
+    /// container RPCs can skip constructing one.
+    orchestrator: Option<Arc<dyn ContainerOrchestrator>>,
+    /// Concrete handle to the dev orchestrator, when we're in dev mode. Used by
+    /// `container-install { manifest_path }` which takes an ad-hoc manifest
+    /// path and is not part of the shared trait.
+    dev_orchestrator: Option<Arc<DevContainerOrchestrator>>,
     state_manager: Arc<StateManager>,
     pub(crate) metrics_store: Arc<MetricsStore>,
     port_allocator: Arc<tokio::sync::Mutex<PortAllocator>>,
@@ -79,6 +87,15 @@ pub struct RpcHandler {
     /// Our own Ed25519 pubkey hex — needed by ContentRef senders for cap scoping
     /// and by ContentRef receivers to request caps scoped to themselves.
     pub(crate) self_pubkey_hex: Arc<tokio::sync::RwLock<Option<String>>>,
+    /// Kick the package scanner to run immediately (bypassing the 60s interval).
+    /// Used by install/update success paths so the fresh manifest (with populated
+    /// `interfaces.main.ui`) lands before we flip state to Running — closes the
+    /// "Launch button is missing for up to 60s after install" UX gap.
+    pub(crate) scan_kick: Arc<tokio::sync::Notify>,
+    /// Monotonic counter incremented by the scan loop after each completed scan.
+    /// Install/update success paths subscribe to this to know when a kicked scan
+    /// has actually finished before flipping to the terminal state.
+    pub(crate) scan_tick: Arc<tokio::sync::watch::Sender<u64>>,
 }
 
 impl RpcHandler {
@@ -87,15 +104,10 @@ impl RpcHandler {
         state_manager: Arc<StateManager>,
         metrics_store: Arc<MetricsStore>,
         session_store: SessionStore,
+        orchestrator: Option<Arc<dyn ContainerOrchestrator>>,
+        dev_orchestrator: Option<Arc<DevContainerOrchestrator>>,
     ) -> Result<Self> {
         let auth_manager = AuthManager::new(config.data_dir.clone());
-        let orchestrator = if config.dev_mode {
-            Some(Arc::new(
-                DevContainerOrchestrator::new(config.clone()).await?,
-            ))
-        } else {
-            None
-        };
         let port_allocator = Arc::new(tokio::sync::Mutex::new(
             PortAllocator::new(&config.data_dir).await?,
         ));
@@ -129,6 +141,7 @@ impl RpcHandler {
             config,
             auth_manager,
             orchestrator,
+            dev_orchestrator,
             state_manager,
             metrics_store,
             port_allocator,
@@ -140,6 +153,8 @@ impl RpcHandler {
             transport_router: Arc::new(tokio::sync::RwLock::new(None)),
             blob_store: Arc::new(tokio::sync::RwLock::new(None)),
             self_pubkey_hex: Arc::new(tokio::sync::RwLock::new(None)),
+            scan_kick: Arc::new(tokio::sync::Notify::new()),
+            scan_tick: Arc::new(tokio::sync::watch::channel(0u64).0),
         })
     }
 
@@ -180,6 +195,21 @@ impl RpcHandler {
         Arc::clone(&self.mesh_service)
     }
 
+    /// Shared Notify handle the package-scanner loop waits on (in addition to
+    /// its periodic tick). Install/update success paths call `notify_one()` to
+    /// trigger an immediate scan so the fresh manifest lands before we flip to
+    /// the terminal Running state.
+    pub fn scan_kick(&self) -> Arc<tokio::sync::Notify> {
+        Arc::clone(&self.scan_kick)
+    }
+
+    /// Sender half of the scan-completion watch channel. The scanner bumps this
+    /// counter after every finished scan; install/update wait for an advance
+    /// after kicking so they know the fresh manifest has landed.
+    pub fn scan_tick(&self) -> Arc<tokio::sync::watch::Sender<u64>> {
+        Arc::clone(&self.scan_tick)
+    }
+
     fn cookie_suffix_for_request(&self, headers: &hyper::header::HeaderMap) -> &'static str {
         // Only set Secure flag when the original request was over HTTPS.
         // Nginx sends X-Forwarded-Proto: https for HTTPS connections.
@@ -197,7 +227,10 @@ impl RpcHandler {
         ""
     }
 
-    pub async fn handle(&self, req: Request<hyper::Body>) -> Result<Response<hyper::Body>> {
+    pub async fn handle(
+        self: Arc<Self>,
+        req: Request<hyper::Body>,
+    ) -> Result<Response<hyper::Body>> {
         // Extract session cookie before consuming the request
         let (parts, body) = req.into_parts();
         let session_token = session::extract_session_cookie(&parts.headers);
@@ -376,7 +409,7 @@ impl RpcHandler {
 
         // Route to handler (track latency for metrics)
         let rpc_start = std::time::Instant::now();
-        let result = self.dispatch(&rpc_req.method, params, &session_token).await;
+        let result = Self::dispatch(&self, &rpc_req.method, params, &session_token).await;
 
         // Record RPC latency for monitoring
         let elapsed_ms = rpc_start.elapsed().as_secs_f64() * 1000.0;

core/archipelago/src/api/rpc/package/async_lifecycle.rs

@@ -0,0 +1,437 @@
+//! Async wrappers for `package.install`, `package.uninstall`, `package.update`.
+//!
+//! The inner `handle_package_*` functions are large (install is 480 lines with
+//! the stack dispatchers, update is 300, uninstall is 200) and do their own
+//! fine-grained progress tracking via `install_progress` and `uninstall_stage`.
+//! We wrap them rather than refactor them.
+//!
+//! Each wrapper:
+//!   1. Parses + validates the RPC params (cheap, synchronous). Errors here
+//!      return immediately to the caller before any state change.
+//!   2. Flips the package state to the transitional variant
+//!      (`Installing` / `Removing` / `Updating`) so the UI sees it on the
+//!      next WebSocket push (before the RPC response even lands).
+//!   3. `tokio::spawn`s a background task that invokes the existing
+//!      `handle_package_*` method on the Arc-held self.
+//!   4. On task success: no state change needed — the inner handler has
+//!      already written the terminal state (Running for install/update, or
+//!      removed the entry for uninstall).
+//!   5. On task failure: revert state to the pre-transition value (or delete
+//!      the entry for install, since there was no pre-state), write a line
+//!      to the persistent install log, and clear any stale progress fields.
+//!   6. Returns `{ "status": "installing" }` etc. immediately.
+//!
+//! The server package-scan loop's `merge_preserving_transitional` helper
+//! already knows to preserve `Installing` / `Removing` / `Updating` between
+//! scans, so live progress updates broadcast from inside the spawned task
+//! reach the UI correctly.
+
+use super::install::install_log;
+use crate::api::rpc::RpcHandler;
+use crate::data_model::PackageState;
+use crate::state::StateManager;
+use anyhow::Result;
+use std::sync::Arc;
+use tracing::{error, info, warn};
+
+impl RpcHandler {
+    /// Async wrapper for `package.install`. Returns `{ "status": "installing" }`
+    /// immediately after flipping state to `Installing` and spawning the
+    /// actual install pipeline. On failure, removes the package entry from
+    /// state so the UI reverts to "not installed".
+    pub(in crate::api::rpc) async fn spawn_package_install(
+        self: Arc<Self>,
+        params: Option<serde_json::Value>,
+    ) -> Result<serde_json::Value> {
+        // Extract + validate package_id synchronously so bad params fail
+        // fast without touching state.
+        let params_val = params
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Missing params"))?;
+        let package_id = params_val
+            .get("id")
+            .and_then(|v| v.as_str())
+            .ok_or_else(|| anyhow::anyhow!("Missing package id"))?
+            .to_string();
+        super::validation::validate_app_id(&package_id)?;
+
+        // Reject if already in a transitional lifecycle (prevents double-click
+        // queuing two installs on the same package).
+        {
+            let (data, _) = self.state_manager.get_snapshot().await;
+            if let Some(entry) = data.package_data.get(&package_id) {
+                if matches!(
+                    entry.state,
+                    PackageState::Installing | PackageState::Removing | PackageState::Updating
+                ) {
+                    return Err(anyhow::anyhow!(
+                        "{} is already {:?}",
+                        package_id,
+                        entry.state
+                    ));
+                }
+            }
+        }
+
+        // Flip state to Installing BEFORE the spawn so the first WebSocket
+        // push carries the transitional state. Uses the same
+        // `create_installing_entry` path the inner handler would use once
+        // it starts pulling, so the UI sees a consistent shape.
+        flip_to_installing(&self.state_manager, &package_id).await;
+
+        install_log(&format!("INSTALL SPAWN: {}", package_id)).await;
+
+        let handler = Arc::clone(&self);
+        let package_id_spawn = package_id.clone();
+        tokio::spawn(async move {
+            match handler.handle_package_install(params).await {
+                Ok(_) => {
+                    info!("package.install {}: complete", package_id_spawn);
+                    // The install pipeline has verified the container is up
+                    // and healthy (see install.rs post-start exit check).
+                    // Kick the scanner first so the fresh manifest (with
+                    // `interfaces.main.ui` from the live port binding) lands
+                    // BEFORE we flip to Running — without this the Launch
+                    // button is missing for up to 60s after a successful
+                    // install, because the skeletal install-time manifest
+                    // has `interfaces: None`.
+                    kick_scanner_and_wait(&handler).await;
+                    // We MUST explicitly transition out of Installing here:
+                    // `merge_preserving_transitional` in the package-scan
+                    // loop treats Installing as RPC-owned and refuses to
+                    // let the scanner overwrite it with the observed
+                    // Running state. Without this write, the entry stays
+                    // stuck at Installing forever.
+                    set_package_state(
+                        &handler.state_manager,
+                        &package_id_spawn,
+                        PackageState::Running,
+                    )
+                    .await;
+                    handler.clear_install_progress(&package_id_spawn).await;
+                }
+                Err(e) => {
+                    error!("package.install {} failed: {:#}", package_id_spawn, e);
+                    install_log(&format!("INSTALL FAIL: {} — {:#}", package_id_spawn, e)).await;
+                    // No pre-state to revert to — remove the entry entirely so
+                    // the UI shows the app as not installed. The next package
+                    // scan will re-create it only if podman actually has a
+                    // container for it (partial install recovery).
+                    remove_package_entry(&handler.state_manager, &package_id_spawn).await;
+                }
+            }
+        });
+
+        Ok(serde_json::json!({
+            "status": "installing",
+            "package_id": package_id,
+        }))
+    }
+
+    /// Async wrapper for `package.uninstall`. Returns `{ "status": "removing" }`
+    /// immediately. State stays `Removing` until the inner handler finishes
+    /// (including the `sudo rm -rf` of app data, which can take minutes for
+    /// bitcoin-core's chainstate). On failure, reverts to the pre-transition
+    /// state (usually Running or Stopped) so the user can retry.
+    pub(in crate::api::rpc) async fn spawn_package_uninstall(
+        self: Arc<Self>,
+        params: Option<serde_json::Value>,
+    ) -> Result<serde_json::Value> {
+        let params_val = params
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Missing params"))?;
+        let package_id = params_val
+            .get("id")
+            .and_then(|v| v.as_str())
+            .ok_or_else(|| anyhow::anyhow!("Missing package id"))?
+            .to_string();
+        super::validation::validate_app_id(&package_id)?;
+
+        // Reject if already in a transitional lifecycle.
+        {
+            let (data, _) = self.state_manager.get_snapshot().await;
+            if let Some(entry) = data.package_data.get(&package_id) {
+                if matches!(
+                    entry.state,
+                    PackageState::Installing | PackageState::Removing | PackageState::Updating
+                ) {
+                    return Err(anyhow::anyhow!(
+                        "{} is already {:?}",
+                        package_id,
+                        entry.state
+                    ));
+                }
+            }
+        }
+
+        let pre_state =
+            flip_package_state(&self.state_manager, &package_id, PackageState::Removing).await;
+
+        install_log(&format!("UNINSTALL SPAWN: {}", package_id)).await;
+
+        let handler = Arc::clone(&self);
+        let package_id_spawn = package_id.clone();
+        tokio::spawn(async move {
+            match handler.handle_package_uninstall(params).await {
+                Ok(_) => {
+                    info!("package.uninstall {}: complete", package_id_spawn);
+                    // Inner handler already removed the package entry on
+                    // success. Nothing more to do here.
+                }
+                Err(e) => {
+                    error!("package.uninstall {} failed: {:#}", package_id_spawn, e);
+                    install_log(&format!("UNINSTALL FAIL: {} — {:#}", package_id_spawn, e)).await;
+                    // Revert to pre-transition state so the user can retry.
+                    // Also clear any stale uninstall_stage label.
+                    if let Some(prev) = pre_state {
+                        set_package_state_and_clear_uninstall_stage(
+                            &handler.state_manager,
+                            &package_id_spawn,
+                            prev,
+                        )
+                        .await;
+                    }
+                }
+            }
+        });
+
+        Ok(serde_json::json!({
+            "status": "removing",
+            "package_id": package_id,
+        }))
+    }
+
+    /// Async wrapper for `package.update`. Returns `{ "status": "updating" }`
+    /// immediately. The inner handler already manages its own rollback on
+    /// failure (restarts old containers); this wrapper just flips state and
+    /// spawns.
+    pub(in crate::api::rpc) async fn spawn_package_update(
+        self: Arc<Self>,
+        params: Option<serde_json::Value>,
+    ) -> Result<serde_json::Value> {
+        let params_val = params
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Missing params"))?;
+        let package_id = params_val
+            .get("id")
+            .and_then(|v| v.as_str())
+            .ok_or_else(|| anyhow::anyhow!("Missing package id"))?
+            .to_string();
+        super::validation::validate_app_id(&package_id)?;
+
+        // Reject if already in a transitional lifecycle.
+        {
+            let (data, _) = self.state_manager.get_snapshot().await;
+            if let Some(entry) = data.package_data.get(&package_id) {
+                if matches!(
+                    entry.state,
+                    PackageState::Installing | PackageState::Removing | PackageState::Updating
+                ) {
+                    return Err(anyhow::anyhow!(
+                        "{} is already {:?}",
+                        package_id,
+                        entry.state
+                    ));
+                }
+            }
+        }
+
+        // The inner handler flips state to Updating itself, but we do it
+        // here too so the transitional state lands before the spawn yields.
+        let pre_state =
+            flip_package_state(&self.state_manager, &package_id, PackageState::Updating).await;
+
+        install_log(&format!("UPDATE SPAWN: {}", package_id)).await;
+
+        let handler = Arc::clone(&self);
+        let package_id_spawn = package_id.clone();
+        tokio::spawn(async move {
+            match handler.handle_package_update(params).await {
+                Ok(_) => {
+                    info!("package.update {}: complete", package_id_spawn);
+                    // Same reasoning as install: the merge_preserving_transitional
+                    // helper treats Updating as RPC-owned, so we MUST write the
+                    // terminal Running state ourselves or the entry will stay
+                    // stuck at Updating forever. The update pipeline has
+                    // already verified the new container is running via its
+                    // post-recreate check.
+                    // Kick the scanner first so any manifest changes from the
+                    // new image version (interfaces, ports, etc.) land before
+                    // we flip to Running.
+                    kick_scanner_and_wait(&handler).await;
+                    set_package_state(
+                        &handler.state_manager,
+                        &package_id_spawn,
+                        PackageState::Running,
+                    )
+                    .await;
+                }
+                Err(e) => {
+                    error!("package.update {} failed: {:#}", package_id_spawn, e);
+                    install_log(&format!("UPDATE FAIL: {} — {:#}", package_id_spawn, e)).await;
+                    // Inner handler already ran rollback_update + cleared
+                    // update state, but be defensive: revert to pre-state
+                    // in case the inner flow died before its cleanup.
+                    if let Some(prev) = pre_state {
+                        set_package_state(&handler.state_manager, &package_id_spawn, prev).await;
+                    }
+                }
+            }
+        });
+
+        Ok(serde_json::json!({
+            "status": "updating",
+            "package_id": package_id,
+        }))
+    }
+}
+
+// ---------------------------------------------------------------------------
+// State-manager helpers (free fns, usable from inside spawned tasks)
+// ---------------------------------------------------------------------------
+
+/// Create or update the entry for this package with `Installing` state.
+/// Matches what the inner handler's `set_install_progress` would do on first
+/// call, but fires before the spawn so the UI sees it immediately.
+async fn flip_to_installing(state_manager: &StateManager, package_id: &str) {
+    use crate::data_model::{Description, Manifest, PackageDataEntry, StaticFiles};
+    let (mut data, _) = state_manager.get_snapshot().await;
+    let entry = data
+        .package_data
+        .entry(package_id.to_string())
+        .or_insert_with(|| PackageDataEntry {
+            state: PackageState::Installing,
+            health: None,
+            exit_code: None,
+            static_files: StaticFiles {
+                license: String::new(),
+                instructions: String::new(),
+                // Leave icon empty during the transient Installing window:
+                // hardcoding `<id>.png` is wrong for ~half our apps (many use
+                // `.svg` / `.webp`), producing a broken-image flicker until
+                // the scanner refreshes the entry. The frontend's `icon`
+                // computed falls through to `curatedMap.get(id)?.icon` which
+                // has the correct extensions for known apps.
+                icon: String::new(),
+            },
+            manifest: Manifest {
+                id: package_id.to_string(),
+                title: package_id.to_string(),
+                version: String::new(),
+                description: Description {
+                    short: "Installing...".to_string(),
+                    long: String::new(),
+                },
+                release_notes: String::new(),
+                license: String::new(),
+                wrapper_repo: String::new(),
+                upstream_repo: String::new(),
+                support_site: String::new(),
+                marketing_site: String::new(),
+                donation_url: None,
+                author: None,
+                website: None,
+                interfaces: None,
+                tier: None,
+            },
+            installed: None,
+            install_progress: None,
+            uninstall_stage: None,
+            available_update: None,
+        });
+    entry.state = PackageState::Installing;
+    state_manager.update_data(data).await;
+}
+
+/// Flip an existing entry's state and return the pre-flip value (or None if
+/// no entry existed). Used for revert-on-failure.
+async fn flip_package_state(
+    state_manager: &StateManager,
+    package_id: &str,
+    new_state: PackageState,
+) -> Option<PackageState> {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    let prev = data.package_data.get(package_id).map(|e| e.state.clone());
+    if let Some(entry) = data.package_data.get_mut(package_id) {
+        entry.state = new_state;
+        state_manager.update_data(data).await;
+    } else {
+        warn!(
+            "flip_package_state: no entry for {} — cannot flip",
+            package_id
+        );
+    }
+    prev
+}
+
+/// Set state unconditionally (no-op if entry no longer exists).
+async fn set_package_state(
+    state_manager: &StateManager,
+    package_id: &str,
+    new_state: PackageState,
+) {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    if let Some(entry) = data.package_data.get_mut(package_id) {
+        if entry.state != new_state {
+            entry.state = new_state;
+            state_manager.update_data(data).await;
+        }
+    }
+}
+
+/// Set state and clear the uninstall_stage label. Used when an uninstall
+/// fails and we revert — the user doesn't want a stale "Removing app data"
+/// message sitting on a Running entry.
+async fn set_package_state_and_clear_uninstall_stage(
+    state_manager: &StateManager,
+    package_id: &str,
+    new_state: PackageState,
+) {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    if let Some(entry) = data.package_data.get_mut(package_id) {
+        entry.state = new_state;
+        entry.uninstall_stage = None;
+        state_manager.update_data(data).await;
+    }
+}
+
+/// Remove a package entry from state. Used for install-failure cleanup
+/// (since there's no pre-state to revert to — the entry was created
+/// speculatively when we flipped to Installing).
+async fn remove_package_entry(state_manager: &StateManager, package_id: &str) {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    if data.package_data.remove(package_id).is_some() {
+        state_manager.update_data(data).await;
+    }
+}
+
+/// Kick the container scanner to run immediately and wait for it to finish
+/// (with a 2s timeout). Used by install/update success paths so the fresh
+/// manifest — with `interfaces.main.ui` populated from the now-running
+/// container's port binding — lands BEFORE we flip state to Running.
+///
+/// Without this, the frontend sees `state = running` but the skeletal
+/// install-time manifest (interfaces = None), and hides the Launch button
+/// for up to the full 60s scan interval.
+///
+/// The scan merges via `merge_preserving_transitional`, which keeps
+/// state = Installing (we haven't flipped yet) while taking the fresh
+/// manifest. After this returns, the caller writes Running on top of the
+/// now-populated manifest.
+async fn kick_scanner_and_wait(handler: &RpcHandler) {
+    let mut rx = handler.scan_tick.subscribe();
+    let start = *rx.borrow_and_update();
+    handler.scan_kick.notify_one();
+    // 2s is well above a typical podman scan (~200ms on .228, ~500ms worst
+    // case). If it times out we proceed anyway — the next 60s scan will
+    // self-heal and the worst case is the pre-fix behavior (Launch button
+    // appears a bit late).
+    let _ = tokio::time::timeout(std::time::Duration::from_secs(2), async {
+        while *rx.borrow_and_update() == start {
+            if rx.changed().await.is_err() {
+                break;
+            }
+        }
+    })
+    .await;
+}

core/archipelago/src/api/rpc/package/config.rs

@@ -9,7 +9,7 @@ pub(super) const TRUSTED_REGISTRIES: &[&str] = &[
     "ghcr.io/",
     "localhost/",
     "git.tx1138.com/",
-    "23.182.128.160:3000/",
+    "146.59.87.168:3000/",
 ];
 
 /// Validate Docker image against trusted registry allowlist.
@@ -29,7 +29,7 @@ pub(super) fn is_valid_docker_image(image: &str) -> bool {
     };
     matches!(
         registry,
-        "docker.io" | "ghcr.io" | "localhost" | "git.tx1138.com" | "23.182.128.160:3000"
+        "docker.io" | "ghcr.io" | "localhost" | "git.tx1138.com" | "146.59.87.168:3000"
     )
 }
 
@@ -174,7 +174,7 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
             ("curl -sf http://localhost:8000/ || exit 1", "60s", "3")
         }
         "nextcloud" => (
-            "curl -sf http://localhost:80/status.php || exit 1",
+            "curl -s -o /dev/null http://localhost:80/status.php || exit 1",
             "30s",
             "3",
         ),
@@ -194,7 +194,12 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
         "vaultwarden" => ("curl -sf http://localhost:80/alive || exit 1", "30s", "3"),
         "uptime-kuma" => ("curl -sf http://localhost:3001/ || exit 1", "30s", "3"),
         "filebrowser" => ("curl -sf http://localhost:80/health || exit 1", "30s", "3"),
-        "searxng" => ("curl -sf http://localhost:8080/ || exit 1", "30s", "3"),
+        "botfights" => (
+            "node -e \"fetch(\\\"http://127.0.0.1:9100/api/health\\\").then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\"",
+            "30s",
+            "3",
+        ),
+        "searxng" => ("wget -q -O /dev/null http://localhost:8080/ || exit 1", "30s", "3"),
         "photoprism" => (
             "curl -sf http://localhost:2342/api/v1/status || exit 1",
             "60s",
@@ -210,11 +215,7 @@ pub(super) fn get_health_check_args(app_id: &str, _rpc_pass: &str) -> Vec<String
             "30s",
             "3",
         ),
-        "portainer" => (
-            "curl -sf http://localhost:9000/api/status || exit 1",
-            "30s",
-            "3",
-        ),
+        "portainer" => return vec![],
         "ollama" => ("curl -sf http://localhost:11434/ || exit 1", "30s", "3"),
         "fedimint" => ("curl -sf http://localhost:8175/ || exit 1", "60s", "3"),
         "fedimint-gateway" => ("curl -sf http://localhost:8176/ || exit 1", "60s", "3"),
@@ -402,7 +403,6 @@ pub(super) fn get_data_dirs_for_app(package_id: &str) -> Vec<String> {
             format!("{}/mempool", base),
             format!("{}/mysql-mempool", base),
             format!("{}/electrumx", base),
-            format!("{}/mempool-electrs", base),
         ],
         "fedimint" => vec![
             format!("{}/fedimint", base),
@@ -533,9 +533,7 @@ pub(super) async fn get_app_config(
                 "--bitcoin.node=bitcoind".to_string(),
                 format!("--bitcoind.rpcuser={}", rpc_user),
                 format!("--bitcoind.rpcpass={}", rpc_pass),
-                "--bitcoind.rpchost=host.containers.internal:8332".to_string(),
-                "--bitcoind.zmqpubrawblock=tcp://host.containers.internal:28332".to_string(),
-                "--bitcoind.zmqpubrawtx=tcp://host.containers.internal:28333".to_string(),
+                "--bitcoind.rpchost=bitcoin-knots:8332".to_string(),
                 "--rpclisten=0.0.0.0:10009".to_string(),
                 "--restlisten=0.0.0.0:8080".to_string(),
                 "--listen=0.0.0.0:9735".to_string(),
@@ -549,7 +547,8 @@ pub(super) async fn get_app_config(
                 "BTCPAY_PROTOCOL=http".to_string(),
                 format!("BTCPAY_HOST={}:23000", host_ip),
                 "BTCPAY_CHAINS=btc".to_string(),
-                format!("BTCPAY_BTCRPCURL=http://{}:8332", host_ip),
+                "BTCPAY_BTCEXPLORERURL=http://archy-nbxplorer:32838".to_string(),
+                "BTCPAY_BTCRPCURL=http://bitcoin-knots:8332".to_string(),
                 format!("BTCPAY_BTCRPCUSER={}", rpc_user),
                 format!("BTCPAY_BTCRPCPASSWORD={}", rpc_pass),
                 format!("BTCPAY_POSTGRES=User ID=btcpay;Password={};Host=archy-btcpay-db;Port=5432;Database=btcpay;Include Error Detail=true",
@@ -561,7 +560,7 @@ pub(super) async fn get_app_config(
         "mempool" | "mempool-web" => (
             vec!["4080:8080".to_string()],
             vec![],
-            vec![format!("BACKEND_MAINNET_HTTP_HOST={}", host_ip)],
+            vec!["BACKEND_MAINNET_HTTP_HOST=mempool-api".to_string()],
             None,
             None,
         ),
@@ -570,12 +569,12 @@ pub(super) async fn get_app_config(
             vec!["/var/lib/archipelago/mempool:/data".to_string()],
             vec![
                 "MEMPOOL_BACKEND=electrum".to_string(),
-                "ELECTRUM_HOST=host.containers.internal".to_string(),
+                "ELECTRUM_HOST=electrumx".to_string(),
                 "ELECTRUM_PORT=50001".to_string(),
                 "ELECTRUM_TLS_ENABLED=false".to_string(),
-                format!("CORE_RPC_HOST={}", host_ip),
+                "CORE_RPC_HOST=bitcoin-knots".to_string(),
                 "CORE_RPC_PORT=8332".to_string(),
-                format!("CORE_RPC_USERNAME={}", rpc_user),
+                "CORE_RPC_USERNAME=archipelago".to_string(),
                 format!("CORE_RPC_PASSWORD={}", rpc_pass),
                 "DATABASE_ENABLED=true".to_string(),
                 "DATABASE_HOST=archy-mempool-db".to_string(),
@@ -592,7 +591,7 @@ pub(super) async fn get_app_config(
                 vec!["/var/lib/archipelago/electrumx:/data".to_string()],
                 vec![
                     format!(
-                        "DAEMON_URL=http://{}:{}@host.containers.internal:8332/",
+                        "DAEMON_URL=http://{}:{}@bitcoin-knots:8332/",
                         rpc_user, rpc_pass
                     ),
                     "COIN=Bitcoin".to_string(),
@@ -610,7 +609,7 @@ pub(super) async fn get_app_config(
                 "MYSQL_DATABASE=mempool".to_string(),
                 "MYSQL_USER=mempool".to_string(),
                 format!("MYSQL_PASSWORD={}", read_secret("mempool-db-password", "mempoolpass")),
-                format!("MYSQL_ROOT_PASSWORD={}", read_secret("mempool-db-root-password", "rootpass")),
+                format!("MYSQL_ROOT_PASSWORD={}", read_secret("mysql-root-db-password", "rootpass")),
             ],
             None,
             None,
@@ -752,14 +751,14 @@ pub(super) async fn get_app_config(
             vec!["9000:9000".to_string()],
             vec![
                 "/var/lib/archipelago/portainer:/data".to_string(),
-                "/var/run/podman/podman.sock:/var/run/docker.sock".to_string(),
+                "/run/user/1000/podman/podman.sock:/var/run/docker.sock".to_string(),
             ],
             vec![],
             None,
             None,
         ),
         "uptime-kuma" => (
-            vec!["3001:3001".to_string()],
+            vec!["3002:3001".to_string()],
             vec!["/var/lib/archipelago/uptime-kuma:/app/data".to_string()],
             vec!["TZ=UTC".to_string()],
             None,
@@ -791,13 +790,13 @@ pub(super) async fn get_app_config(
                 "FM_BIND_UI=0.0.0.0:8175".to_string(),
                 format!("FM_P2P_URL=fedimint://{}:8173", host_ip),
                 format!("FM_API_URL=ws://{}:8174", host_ip),
-                format!("FM_BITCOIND_URL=http://{}:8332", host_ip),
+                "FM_BITCOIND_URL=http://bitcoin-knots:8332".to_string(),
             ],
             None,
             Some(vec![
                 "--data-dir".to_string(),
                 "/data".to_string(),
-                format!("--bitcoind-url=http://{}:{}@{}:8332", rpc_user, rpc_pass, host_ip),
+                format!("--bitcoind-url=http://{}:{}@bitcoin-knots:8332", rpc_user, rpc_pass),
             ]),
         ),
         "fedimint-gateway" => {
@@ -821,7 +820,7 @@ pub(super) async fn get_app_config(
                     "--network".to_string(),
                     "bitcoin".to_string(),
                     "--bitcoind-url".to_string(),
-                    format!("http://{}:8332", host_ip),
+                    "http://bitcoin-knots:8332".to_string(),
                     "--bitcoind-username".to_string(),
                     rpc_user.to_string(),
                     "--bitcoind-password".to_string(),

core/archipelago/src/api/rpc/package/install.rs

@@ -9,14 +9,15 @@ use super::dependencies::{
 use super::progress::parse_pull_progress;
 use super::validation::validate_app_id;
 use crate::api::rpc::RpcHandler;
+use crate::data_model::InstallPhase;
 use anyhow::{Context, Result};
 use tokio::io::{AsyncBufReadExt, BufReader};
 use tracing::{debug, info, warn};
 
-const INSTALL_LOG: &str = "/var/log/archipelago-container-installs.log";
+const INSTALL_LOG: &str = "/var/log/archipelago/container-installs.log";
 
 /// Append a timestamped line to the persistent install log.
-pub(super) async fn install_log(msg: &str) {
+pub(in crate::api::rpc) async fn install_log(msg: &str) {
     use tokio::io::AsyncWriteExt;
     let ts = chrono::Utc::now().format("%Y-%m-%d %H:%M:%S UTC");
     let line = format!("[{}] {}\n", ts, msg);
@@ -96,6 +97,10 @@ impl RpcHandler {
             return self.install_indeedhub_stack().await;
         }
 
+        // Phase: Preparing — validating deps and configs before any slow I/O.
+        self.set_install_phase(package_id, InstallPhase::Preparing)
+            .await;
+
         // Dependency checks
         let deps = detect_running_deps().await?;
         check_install_deps(package_id, &deps)?;
@@ -175,18 +180,79 @@ impl RpcHandler {
             }));
         }
 
+        // Preferred path for apps already modeled in the production orchestrator.
+        // Keep legacy install flow as default while migration is in progress.
+        if should_try_orchestrator_install(package_id, self.orchestrator.is_some()) {
+            let orchestrator_app_id = orchestrator_install_app_id(package_id);
+            self.set_install_phase(package_id, InstallPhase::CreatingContainer)
+                .await;
+            install_log(&format!(
+                "INSTALL ORCH: {} — attempting orchestrator install as {}",
+                package_id, orchestrator_app_id
+            ))
+            .await;
+
+            if let Some(orchestrator) = self.orchestrator.as_ref() {
+                match orchestrator.install(orchestrator_app_id).await {
+                    Ok(container_name) => {
+                        self.set_install_phase(package_id, InstallPhase::WaitingHealthy)
+                            .await;
+                        install_log(&format!(
+                            "INSTALL ORCH OK: {} (app={}) — container={}",
+                            package_id, orchestrator_app_id, container_name
+                        ))
+                        .await;
+                        return Ok(serde_json::json!({
+                            "success": true,
+                            "package_id": package_id,
+                            "container_name": container_name,
+                            "message": format!("Package {} installed and started", package_id)
+                        }));
+                    }
+                    Err(e) if is_unknown_app_id_error(&e) => {
+                        info!(
+                            "Install {}: orchestrator has no manifest mapping yet, falling back to legacy installer",
+                            package_id
+                        );
+                        install_log(&format!(
+                            "INSTALL ORCH SKIP: {} — unknown app_id, using legacy flow",
+                            package_id
+                        ))
+                        .await;
+                    }
+                    Err(e) => {
+                        install_log(&format!("INSTALL ORCH FAIL: {} — {}", package_id, e)).await;
+                        return Err(
+                            e.context(format!("Orchestrator install {} failed", package_id))
+                        );
+                    }
+                }
+            }
+        }
+
         // Pull or verify image
         install_log(&format!(
             "INSTALL PULL: {} — pulling image {}",
             package_id, docker_image
         ))
         .await;
+        // Phase: PullingImage — the longest phase. Podman doesn't emit
+        // parseable progress on a piped stderr, so the UI shows an
+        // indeterminate "Downloading image…" at this fixed percentage
+        // until pull completes.
+        self.set_install_phase(package_id, InstallPhase::PullingImage)
+            .await;
         let has_local_fallback = self.pull_or_verify_image(package_id, docker_image).await?;
         install_log(&format!(
             "INSTALL PULL OK: {} — image ready (local_fallback={})",
             package_id, has_local_fallback
         ))
         .await;
+        // Phase: CreatingContainer — image is local, now writing configs,
+        // data directories, chowning to container UID, building the run
+        // argv. Fast (sub-second to a few seconds).
+        self.set_install_phase(package_id, InstallPhase::CreatingContainer)
+            .await;
 
         // Normalize container name for legacy aliases
         let container_name = match package_id {
@@ -379,6 +445,14 @@ impl RpcHandler {
         run_args.push(&mem_arg);
         run_args.push("--cpus=2");
 
+        // Uptime Kuma image entrypoint (`extra/entrypoint.sh`) attempts
+        // `setpriv --clear-groups` and fails under our rootless + cap-drop
+        // defaults. Run the server directly via dumb-init to keep startup
+        // stable on production nodes.
+        if package_id == "uptime-kuma" {
+            run_args.push("--entrypoint=/usr/bin/dumb-init");
+        }
+
         // Health checks
         let health_args = get_health_check_args(package_id, &rpc_pass);
         for arg in &health_args {
@@ -436,9 +510,21 @@ impl RpcHandler {
         ))
         .await;
 
+        // Phase: StartingContainer — podman run accepted. Next we poll
+        // inspect until State.Status == running (up to 60s).
+        self.set_install_phase(package_id, InstallPhase::StartingContainer)
+            .await;
+
         // Post-start health verification: wait up to 60s for container to be running
         let mut container_running = false;
         for i in 0..12u32 {
+            // After the first poll, flip the UI to WaitingHealthy — the
+            // container hasn't come up yet, so the phase label changes
+            // from "Starting container" to "Waiting for healthy".
+            if i == 1 {
+                self.set_install_phase(package_id, InstallPhase::WaitingHealthy)
+                    .await;
+            }
             tokio::time::sleep(std::time::Duration::from_secs(5)).await;
             let status = tokio::process::Command::new("podman")
                 .args(["inspect", container_name, "--format", "{{.State.Status}}"])
@@ -498,6 +584,12 @@ impl RpcHandler {
             ));
         }
 
+        // Phase: PostInstall — container is up and running. Now any
+        // app-specific post-install (chain init, wallet setup, waiting
+        // for a first block). Varies by app; some are no-ops.
+        self.set_install_phase(package_id, InstallPhase::PostInstall)
+            .await;
+
         // Post-install hooks — await completion before returning success
         self.run_post_install_hooks(package_id).await;
 
@@ -770,20 +862,30 @@ impl RpcHandler {
 
     /// Create data directories for volume mounts under /var/lib/archipelago/.
     /// Get the mapped host UID for a container's internal UID.
-    /// Rootless podman maps container UIDs: host_uid = subuid_start + container_uid
-    /// Default subuid start for archipelago user is 100000.
+    /// Rootless podman UID maps commonly look like:
+    ///   container 0 -> host real uid (e.g. 1000)
+    ///   container 1.. -> host subuid range starting at 100000
+    /// So for uid>=1, host_uid = 99999 + container_uid.
     fn mapped_uid(package_id: &str) -> u32 {
         let container_uid = match package_id {
             "bitcoin-knots" | "bitcoin" | "bitcoin-core" => 101,
             "grafana" => 472,
             "lnd" => 1000,
             "mariadb" | "mysql" | "mysql-mempool" | "archy-mempool-db" => 999,
-            "postgres" | "btcpay-postgres" | "immich-postgres"
-            | "archy-btcpay-db" | "nextcloud-db" => 70,
+            "postgres" | "immich-postgres" | "nextcloud-db" => 70,
+            // Current BTCPay Postgres image runs as uid 999 inside the
+            // container, so its rootless host-mapped uid is 100998.
+            "btcpay-postgres" | "archy-btcpay-db" => 999,
             "electrumx" | "electrs" => 1000,
             _ => 0, // Most containers run as root (UID 0)
         };
-        100000 + container_uid
+        if container_uid == 0 {
+            // Archipelago daemon runs as rootless user (typically uid 1000).
+            // Container uid 0 maps to that real host uid.
+            1000
+        } else {
+            99999 + container_uid
+        }
     }
 
     async fn create_data_dirs(&self, package_id: &str, volumes: &[String]) {
@@ -796,36 +898,44 @@ impl RpcHandler {
                     debug!("Creating directory: {} (owner: {})", host_path, uid_str);
 
                     // Create directory directly (service has ReadWritePaths access).
-                    // sudo is blocked by NoNewPrivileges=yes in the systemd service.
                     if let Err(e) = std::fs::create_dir_all(host_path) {
                         tracing::warn!("Failed to create directory {}: {}", host_path, e);
                     }
 
                     // Set ownership to the mapped UID for rootless podman.
-                    // Try sudo chown first (works on LUKS), fall back to podman unshare.
+                    // Try sudo chown first, then fall back to podman unshare
+                    // for subuid-mapped UIDs only.
                     let host_uid = format!("{}:{}", uid, uid);
                     let sudo_result = tokio::process::Command::new("sudo")
                         .args(["chown", "-R", &host_uid, host_path])
                         .output()
                         .await;
                     let sudo_ok = sudo_result.as_ref().is_ok_and(|o| o.status.success());
+
                     if !sudo_ok {
-                        // Fallback: podman unshare (works on non-LUKS ext4)
-                        let container_uid = uid - 100000;
-                        let container_uid_str = format!("{}:{}", container_uid, container_uid);
-                        let chown_result = tokio::process::Command::new("podman")
-                            .args(["unshare", "chown", "-R", &container_uid_str, host_path])
-                            .output()
-                            .await;
-                        match chown_result {
-                            Ok(out) if !out.status.success() => {
-                                tracing::warn!(
-                                    "chown failed for {} (both sudo and podman unshare)",
-                                    host_path,
-                                );
+                        if uid >= 100000 {
+                            let container_uid = uid - 100000;
+                            let container_uid_str = format!("{}:{}", container_uid, container_uid);
+                            let chown_result = tokio::process::Command::new("podman")
+                                .args(["unshare", "chown", "-R", &container_uid_str, host_path])
+                                .output()
+                                .await;
+                            match chown_result {
+                                Ok(out) if !out.status.success() => {
+                                    tracing::warn!(
+                                        "chown failed for {} (both sudo and podman unshare)",
+                                        host_path,
+                                    );
+                                }
+                                Err(e) => tracing::warn!("Failed to chown {}: {}", host_path, e),
+                                _ => {}
                             }
-                            Err(e) => tracing::warn!("Failed to chown {}: {}", host_path, e),
-                            _ => {}
+                        } else {
+                            tracing::warn!(
+                                "chown fallback skipped for {}: host uid {} has no subuid mapping",
+                                host_path,
+                                uid
+                            );
                         }
                     }
                 }
@@ -1193,54 +1303,18 @@ autopilot.active=false\n",
             }
         }
 
-        // Gitea: deploy nginx proxy on port 3000 to strip X-Frame-Options for iframe embedding.
-        // Gitea container runs on 3001, nginx proxies 3000->3001 removing the header.
+        // Gitea: keep it on its native host port (3001) and serve it under
+        // /app/gitea/ via the main Archipelago nginx config. Avoids colliding
+        // with Grafana, which also uses host port 3000.
         if package_id == "gitea" {
-            let nginx_conf = r#"# Gitea iframe proxy — strips X-Frame-Options for Archipelago iframe
-server {
-    listen 3000;
-    server_name _;
-    client_max_body_size 1G;
-    location / {
-        proxy_pass http://127.0.0.1:3001;
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-        proxy_http_version 1.1;
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-        proxy_hide_header X-Frame-Options;
-        proxy_hide_header Content-Security-Policy;
-    }
-}
-"#;
-            let conf_path = "/etc/nginx/conf.d/gitea-iframe.conf";
-            if let Err(e) = tokio::fs::write(conf_path, nginx_conf).await {
-                tracing::warn!("Failed to write gitea nginx conf: {}", e);
-            } else {
-                let reload = tokio::process::Command::new("nginx")
-                    .args(["-s", "reload"])
-                    .output()
-                    .await;
-                match reload {
-                    Ok(o) if o.status.success() => {
-                        info!("Gitea: nginx iframe proxy deployed on port 3000");
-                    }
-                    Ok(o) => tracing::warn!(
-                        "Gitea nginx reload failed: {}",
-                        String::from_utf8_lossy(&o.stderr)
-                    ),
-                    Err(e) => tracing::warn!("Gitea nginx reload error: {}", e),
-                }
-            }
+            let _ = tokio::fs::remove_file("/etc/nginx/conf.d/gitea-iframe.conf").await;
 
-            // Set ROOT_URL in Gitea config — port 3000 is the nginx iframe proxy,
-            // which is the public-facing port users and the UI iframe access.
+            // Set ROOT_URL to the UI path-based route so links/assets stay
+            // anchored under Archipelago's app proxy endpoint.
             let host_ip = &self.config.host_ip;
             let _ = tokio::process::Command::new("podman")
                 .args(["exec", "gitea", "sh", "-c",
-                       &format!("grep -q ROOT_URL /data/gitea/conf/app.ini && sed -i 's|ROOT_URL.*|ROOT_URL = http://{}:3000/|' /data/gitea/conf/app.ini || true", host_ip)])
+                       &format!("grep -q ROOT_URL /data/gitea/conf/app.ini && sed -i 's|ROOT_URL.*|ROOT_URL = http://{}/app/gitea/|' /data/gitea/conf/app.ini || true", host_ip)])
                 .output()
                 .await;
             // Also ensure X_FRAME_OPTIONS is empty so Gitea doesn't send the header
@@ -1249,8 +1323,15 @@ server {
                        "grep -q X_FRAME_OPTIONS /data/gitea/conf/app.ini && sed -i 's|X_FRAME_OPTIONS.*|X_FRAME_OPTIONS =|' /data/gitea/conf/app.ini || sed -i '/^\\[security\\]/a X_FRAME_OPTIONS =' /data/gitea/conf/app.ini"])
                 .output()
                 .await;
+
+            // Reload main nginx so /app/gitea/ routing changes take effect.
+            let _ = tokio::process::Command::new("nginx")
+                .args(["-s", "reload"])
+                .output()
+                .await;
+
             info!(
-                "Gitea: ROOT_URL set to http://{}:3000/, X_FRAME_OPTIONS cleared",
+                "Gitea: ROOT_URL set to http://{}/app/gitea/, X_FRAME_OPTIONS cleared",
                 host_ip
             );
         }
@@ -1537,9 +1618,15 @@ server {
         // Reassign priorities: target = 0, everyone else = 10, 20, 30…
         // in their existing priority order.
         let target_url = url.to_string();
-        config.registries.sort_by_key(|r| (r.url != target_url, r.priority));
+        config
+            .registries
+            .sort_by_key(|r| (r.url != target_url, r.priority));
         for (i, r) in config.registries.iter_mut().enumerate() {
-            r.priority = if r.url == target_url { 0 } else { (i as u32) * 10 };
+            r.priority = if r.url == target_url {
+                0
+            } else {
+                (i as u32) * 10
+            };
         }
 
         crate::container::registry::save_registries(&self.config.data_dir, &config).await?;
@@ -1561,7 +1648,7 @@ server {
             .unwrap_or(true);
 
         // Registries are configured as `host[:port]/namespace` (for
-        // example `23.182.128.160:3000/lfg2025`), but the Docker V2
+        // example `146.59.87.168:3000/lfg2025`), but the Docker V2
         // registry API lives at `/v2/` on the ROOT of the host — NOT
         // under the namespace. Strip the namespace before appending
         // `/v2/` so the reachability probe hits the correct URL.
@@ -1667,3 +1754,102 @@ async fn resolve_host_gateway() -> String {
     // Last resort
     "--add-host=host.containers.internal:10.0.2.2".to_string()
 }
+
+fn should_try_orchestrator_install(package_id: &str, orchestrator_available: bool) -> bool {
+    orchestrator_available && uses_orchestrator_install_flow(package_id)
+}
+
+fn orchestrator_install_app_id(package_id: &str) -> &str {
+    match package_id {
+        "bitcoin-knots" => "bitcoin-core",
+        "electrs" | "mempool-electrs" => "electrumx",
+        _ => package_id,
+    }
+}
+
+fn uses_orchestrator_install_flow(package_id: &str) -> bool {
+    matches!(
+        package_id,
+        // Step 7 UI apps
+        "bitcoin-ui"
+            | "electrs-ui"
+            | "lnd-ui"
+            // Step 8b backend ports
+            | "bitcoin-core"
+            | "bitcoin-knots"
+            | "lnd"
+            | "fedimint"
+            | "fedimint-gateway"
+            | "filebrowser"
+            | "electrumx"
+            | "electrs"
+            | "mempool-electrs"
+            | "archy-mempool-db"
+            | "mempool-api"
+            | "archy-mempool-web"
+            | "archy-btcpay-db"
+            | "archy-nbxplorer"
+            | "btcpay-server"
+    )
+}
+
+fn is_unknown_app_id_error(err: &anyhow::Error) -> bool {
+    err.chain()
+        .any(|cause| cause.to_string().contains("unknown app_id"))
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{
+        orchestrator_install_app_id, should_try_orchestrator_install,
+        uses_orchestrator_install_flow,
+    };
+
+    #[test]
+    fn orchestrator_install_allowlist_includes_ported_backends() {
+        for app in [
+            "bitcoin-ui",
+            "electrs-ui",
+            "lnd-ui",
+            "bitcoin-core",
+            "bitcoin-knots",
+            "lnd",
+            "fedimint",
+            "fedimint-gateway",
+            "filebrowser",
+            "electrumx",
+            "electrs",
+            "mempool-electrs",
+            "archy-mempool-db",
+            "mempool-api",
+            "archy-mempool-web",
+            "archy-btcpay-db",
+            "archy-nbxplorer",
+            "btcpay-server",
+        ] {
+            assert!(uses_orchestrator_install_flow(app));
+            assert!(should_try_orchestrator_install(app, true));
+        }
+    }
+
+    #[test]
+    fn non_allowlisted_apps_stay_legacy_install() {
+        for app in ["searxng", "mempool", "indeedhub", "immich", "penpot"] {
+            assert!(!uses_orchestrator_install_flow(app));
+            assert!(!should_try_orchestrator_install(app, true));
+        }
+    }
+
+    #[test]
+    fn missing_orchestrator_disables_orchestrator_install() {
+        assert!(!should_try_orchestrator_install("bitcoin-ui", false));
+    }
+
+    #[test]
+    fn install_aliases_map_to_manifest_app_ids() {
+        assert_eq!(orchestrator_install_app_id("bitcoin-knots"), "bitcoin-core");
+        assert_eq!(orchestrator_install_app_id("electrs"), "electrumx");
+        assert_eq!(orchestrator_install_app_id("mempool-electrs"), "electrumx");
+        assert_eq!(orchestrator_install_app_id("lnd"), "lnd");
+    }
+}

core/archipelago/src/api/rpc/package/mod.rs

@@ -1,3 +1,4 @@
+mod async_lifecycle;
 mod config;
 mod dependencies;
 mod install;
@@ -8,5 +9,6 @@ mod stacks;
 mod update;
 mod validation;
 
-// Re-export items needed by sibling modules (container.rs, security.rs)
+// Re-export items needed by sibling modules (container.rs, security.rs, transitional.rs)
+pub(in crate::api::rpc) use install::install_log;
 pub(super) use validation::validate_app_id;

core/archipelago/src/api/rpc/package/progress.rs

@@ -2,12 +2,17 @@
 
 use crate::api::rpc::RpcHandler;
 use crate::data_model::{
-    Description, InstallProgress, Manifest, PackageDataEntry, PackageState, StaticFiles,
+    Description, InstallPhase, InstallProgress, Manifest, PackageDataEntry, PackageState,
+    StaticFiles,
 };
 
 impl RpcHandler {
     /// Set install progress for a package and broadcast the update.
     /// Creates a minimal package entry if one doesn't exist yet.
+    ///
+    /// Prefer `set_install_phase` — this byte-counter API is kept for
+    /// the rare case where the pull stream actually parses, but podman
+    /// almost never emits parseable progress on a piped stderr.
     pub(super) async fn set_install_progress(&self, package_id: &str, downloaded: u64, size: u64) {
         let (mut data, _rev) = self.state_manager.get_snapshot().await;
         let entry = data
@@ -15,7 +20,42 @@ impl RpcHandler {
             .entry(package_id.to_string())
             .or_insert_with(|| create_installing_entry(package_id));
         entry.state = PackageState::Installing;
-        entry.install_progress = Some(InstallProgress { size, downloaded });
+        let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
+        entry.install_progress = Some(InstallProgress {
+            size,
+            downloaded,
+            phase: existing_phase,
+        });
+        self.state_manager.update_data(data).await;
+    }
+
+    /// Set the install pipeline phase and broadcast. This is the
+    /// primary progress signal — the UI maps each phase to a
+    /// percentage and a user-facing label. Byte counters are retained
+    /// for the rare case podman emits parseable progress.
+    pub(super) async fn set_install_phase(&self, package_id: &str, phase: InstallPhase) {
+        let (mut data, _rev) = self.state_manager.get_snapshot().await;
+        let entry = data
+            .package_data
+            .entry(package_id.to_string())
+            .or_insert_with(|| create_installing_entry(package_id));
+        // Preparing / PullingImage / CreatingContainer / StartingContainer /
+        // WaitingHealthy / PostInstall all map to the Installing state.
+        // Updates use Updating state — the wrapper has already flipped
+        // state to Updating, so don't clobber it.
+        if entry.state != PackageState::Updating {
+            entry.state = PackageState::Installing;
+        }
+        let (size, downloaded) = entry
+            .install_progress
+            .as_ref()
+            .map(|p| (p.size, p.downloaded))
+            .unwrap_or((0, 0));
+        entry.install_progress = Some(InstallProgress {
+            size,
+            downloaded,
+            phase: Some(phase),
+        });
         self.state_manager.update_data(data).await;
     }
 
@@ -52,9 +92,11 @@ impl RpcHandler {
             .package_data
             .entry(package_id.to_string())
             .or_insert_with(|| create_installing_entry(package_id));
+        let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
         entry.install_progress = Some(InstallProgress {
             size: total,
             downloaded,
+            phase: existing_phase,
         });
         state_manager.update_data(data).await;
     }
@@ -69,7 +111,11 @@ fn create_installing_entry(package_id: &str) -> PackageDataEntry {
         static_files: StaticFiles {
             license: String::new(),
             instructions: String::new(),
-            icon: format!("/assets/img/app-icons/{}.png", package_id),
+            // Empty icon: hardcoding `<id>.png` is wrong for apps that use
+            // `.svg` or `.webp` assets and produces a broken-image flicker.
+            // The frontend's `icon` computed falls through to the curated
+            // map which has correct extensions for known apps.
+            icon: String::new(),
         },
         manifest: Manifest {
             id: package_id.to_string(),

core/archipelago/src/api/rpc/package/runtime.rs

@@ -3,7 +3,10 @@ use super::dependencies::ordered_containers_for_start;
 use super::install::install_log;
 use super::validation::validate_app_id;
 use crate::api::rpc::RpcHandler;
+use crate::data_model::PackageState;
 use anyhow::{Context, Result};
+use std::sync::Arc;
+use tracing::warn;
 
 /// Per-container graceful shutdown timeout in seconds.
 /// Bitcoin Core needs 600s to flush UTXO set, LND 330s for channel state,
@@ -25,6 +28,12 @@ pub fn stop_timeout_secs(container_name: &str) -> &'static str {
 
 impl RpcHandler {
     /// Start a package: start all containers in dependency order.
+    ///
+    /// Returns immediately with `{ "status": "starting" }` after flipping
+    /// the package state to `Starting` in the StateManager. The actual
+    /// podman-start sequence + post-start exit verification runs in a
+    /// background task. On success the state becomes `Running`; on error
+    /// it reverts to the pre-transition state.
     pub(in crate::api::rpc) async fn handle_package_start(
         &self,
         params: Option<serde_json::Value>,
@@ -42,83 +51,52 @@ impl RpcHandler {
             return Err(anyhow::anyhow!("No containers found for {}", package_id));
         }
 
-        // Clear user-stopped flag — user explicitly started this app
+        // Clear user-stopped flag — user explicitly started this app.
+        // Must happen BEFORE the spawn (ordering contract with crash recovery).
         crate::crash_recovery::clear_user_stopped(&self.config.data_dir, package_id).await;
         for name in &to_start {
             crate::crash_recovery::clear_user_stopped(&self.config.data_dir, name).await;
         }
 
+        let package_id_owned = package_id.to_string();
+        let state_manager = Arc::clone(&self.state_manager);
+        let pre_state =
+            flip_package_state(&state_manager, &package_id_owned, PackageState::Starting).await;
+
         install_log(&format!(
             "START: {} (containers: {:?})",
-            package_id, to_start
+            package_id_owned, to_start
         ))
         .await;
-        let mut errors = Vec::new();
-        for (i, name) in to_start.iter().enumerate() {
-            // Brief delay between dependent containers to allow initialization
-            if i > 0 {
-                tokio::time::sleep(std::time::Duration::from_secs(2)).await;
-            }
-            tracing::info!("Starting container: {}", name);
-            let out = tokio::process::Command::new("podman")
-                .args(["start", name])
-                .output()
-                .await
-                .context(format!("Failed to exec podman start {}", name))?;
-            if !out.status.success() {
-                let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
-                tracing::error!("Failed to start {}: {}", name, stderr);
-                install_log(&format!("START FAIL: {} — {}", name, stderr)).await;
-                errors.push(format!("{}: {}", name, stderr));
-            }
-        }
 
-        if !errors.is_empty() {
-            return Err(anyhow::anyhow!("Start failed: {}", errors.join("; ")));
-        }
-
-        // Verify containers actually reached running state (podman start can
-        // succeed even if the container exits immediately after)
-        tokio::time::sleep(std::time::Duration::from_secs(3)).await;
-        for name in &to_start {
-            let status = tokio::process::Command::new("podman")
-                .args(["inspect", name, "--format", "{{.State.Status}}"])
-                .output()
-                .await;
-            if let Ok(o) = status {
-                let state = String::from_utf8_lossy(&o.stdout).trim().to_string();
-                if state == "exited" {
-                    let logs = tokio::process::Command::new("podman")
-                        .args(["logs", "--tail", "5", name])
-                        .output()
+        tokio::spawn(async move {
+            match do_package_start(&to_start).await {
+                Ok(()) => {
+                    set_package_state(&state_manager, &package_id_owned, PackageState::Running)
                         .await;
-                    let log_text = logs
-                        .map(|o| {
-                            let combined = format!(
-                                "{}{}",
-                                String::from_utf8_lossy(&o.stdout),
-                                String::from_utf8_lossy(&o.stderr)
-                            );
-                            combined.chars().take(200).collect::<String>()
-                        })
-                        .unwrap_or_default();
-                    tracing::error!("Container {} exited after start: {}", name, log_text);
-                    install_log(&format!("START EXITED: {} — {}", name, log_text)).await;
-                    errors.push(format!("{}: exited after start", name));
+                }
+                Err(e) => {
+                    tracing::error!("package.start {} failed: {:#}", package_id_owned, e);
+                    install_log(&format!("START FAIL: {} — {:#}", package_id_owned, e)).await;
+                    if let Some(prev) = pre_state {
+                        set_package_state(&state_manager, &package_id_owned, prev).await;
+                    } else {
+                        warn!(
+                            "package.start {}: no pre-state recorded; relying on next scan",
+                            package_id_owned
+                        );
+                    }
                 }
             }
-        }
+        });
 
-        if !errors.is_empty() {
-            return Err(anyhow::anyhow!(
-                "Containers exited after start: {}",
-                errors.join("; ")
-            ));
-        }
-        Ok(serde_json::Value::Null)
+        Ok(serde_json::json!({ "status": "starting" }))
     }
 
     /// Stop a package: mark as user-stopped and stop all containers.
+    ///
+    /// Returns immediately with `{ "status": "stopping" }`. podman stop
+    /// (up to 600s for bitcoin-core) runs in the background.
     pub(in crate::api::rpc) async fn handle_package_stop(
         &self,
         params: Option<serde_json::Value>,
@@ -136,43 +114,48 @@ impl RpcHandler {
             return Err(anyhow::anyhow!("No containers found for {}", package_id));
         }
 
-        install_log(&format!(
-            "STOP: {} (containers: {:?})",
-            package_id, containers
-        ))
-        .await;
-        // Mark as user-stopped so health monitor and crash recovery don't auto-restart
+        // Mark as user-stopped BEFORE the spawn so health monitor and
+        // crash recovery don't auto-restart mid-flight. Ordering is
+        // load-bearing — see runtime.rs:145-148 original note.
         crate::crash_recovery::mark_user_stopped(&self.config.data_dir, package_id).await;
         for name in &containers {
             crate::crash_recovery::mark_user_stopped(&self.config.data_dir, name).await;
         }
 
-        let mut errors = Vec::new();
-        for name in &containers {
-            tracing::info!(
-                "Stopping container: {} (timeout: {}s)",
-                name,
-                stop_timeout_secs(name)
-            );
-            let out = tokio::process::Command::new("podman")
-                .args(["stop", "-t", stop_timeout_secs(name), name])
-                .output()
-                .await
-                .context(format!("Failed to exec podman stop {}", name))?;
-            if !out.status.success() {
-                let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
-                tracing::error!("Failed to stop {}: {}", name, stderr);
-                errors.push(format!("{}: {}", name, stderr));
-            }
-        }
+        let package_id_owned = package_id.to_string();
+        let state_manager = Arc::clone(&self.state_manager);
+        let pre_state =
+            flip_package_state(&state_manager, &package_id_owned, PackageState::Stopping).await;
 
-        if !errors.is_empty() {
-            return Err(anyhow::anyhow!("Stop failed: {}", errors.join("; ")));
-        }
-        Ok(serde_json::Value::Null)
+        install_log(&format!(
+            "STOP: {} (containers: {:?})",
+            package_id_owned, containers
+        ))
+        .await;
+
+        tokio::spawn(async move {
+            match do_package_stop(&containers).await {
+                Ok(()) => {
+                    set_package_state(&state_manager, &package_id_owned, PackageState::Stopped)
+                        .await;
+                }
+                Err(e) => {
+                    tracing::error!("package.stop {} failed: {:#}", package_id_owned, e);
+                    install_log(&format!("STOP FAIL: {} — {:#}", package_id_owned, e)).await;
+                    if let Some(prev) = pre_state {
+                        set_package_state(&state_manager, &package_id_owned, prev).await;
+                    }
+                }
+            }
+        });
+
+        Ok(serde_json::json!({ "status": "stopping" }))
     }
 
     /// Restart a package: restart all containers.
+    ///
+    /// Returns immediately with `{ "status": "restarting" }`. The restart
+    /// (up to 600s per container for bitcoin-core) runs in the background.
     pub(in crate::api::rpc) async fn handle_package_restart(
         &self,
         params: Option<serde_json::Value>,
@@ -190,55 +173,42 @@ impl RpcHandler {
             return Err(anyhow::anyhow!("No containers found for {}", package_id));
         }
 
+        // Restart does not mark user-stopped; user wants the app to keep
+        // running. Clear any lingering marker so downstream layers don't
+        // interpret the brief podman stop as user intent.
+        crate::crash_recovery::clear_user_stopped(&self.config.data_dir, package_id).await;
+        for name in &containers {
+            crate::crash_recovery::clear_user_stopped(&self.config.data_dir, name).await;
+        }
+
+        let package_id_owned = package_id.to_string();
+        let state_manager = Arc::clone(&self.state_manager);
+        let pre_state =
+            flip_package_state(&state_manager, &package_id_owned, PackageState::Restarting).await;
+
         install_log(&format!(
             "RESTART: {} (containers: {:?})",
-            package_id, containers
+            package_id_owned, containers
         ))
         .await;
-        let mut errors = Vec::new();
-        for name in &containers {
-            tracing::info!("Restarting container: {}", name);
-            let out = tokio::process::Command::new("podman")
-                .args(["restart", "-t", stop_timeout_secs(name), name])
-                .output()
-                .await
-                .context(format!("Failed to exec podman restart {}", name))?;
 
-            if !out.status.success() {
-                let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
-                tracing::warn!(
-                    "podman restart {} failed: {}, trying stop+start",
-                    name,
-                    stderr
-                );
-
-                // Fallback: stop then start (handles rootless podman loopback issues)
-                let _ = tokio::process::Command::new("podman")
-                    .args(["stop", "-t", stop_timeout_secs(name), name])
-                    .output()
-                    .await;
-                let start_out = tokio::process::Command::new("podman")
-                    .args(["start", name])
-                    .output()
-                    .await
-                    .context(format!("Failed to exec podman start {}", name))?;
-
-                if !start_out.status.success() {
-                    let start_err = String::from_utf8_lossy(&start_out.stderr)
-                        .trim()
-                        .to_string();
-                    tracing::error!("stop+start {} also failed: {}", name, start_err);
-                    errors.push(format!("{}: {}", name, start_err));
-                } else {
-                    tracing::info!("Restarted {} via stop+start fallback", name);
+        tokio::spawn(async move {
+            match do_package_restart(&containers).await {
+                Ok(()) => {
+                    set_package_state(&state_manager, &package_id_owned, PackageState::Running)
+                        .await;
+                }
+                Err(e) => {
+                    tracing::error!("package.restart {} failed: {:#}", package_id_owned, e);
+                    install_log(&format!("RESTART FAIL: {} — {:#}", package_id_owned, e)).await;
+                    if let Some(prev) = pre_state {
+                        set_package_state(&state_manager, &package_id_owned, prev).await;
+                    }
                 }
             }
-        }
+        });
 
-        if !errors.is_empty() {
-            return Err(anyhow::anyhow!("Restart failed: {}", errors.join("; ")));
-        }
-        Ok(serde_json::Value::Null)
+        Ok(serde_json::json!({ "status": "restarting" }))
     }
 
     /// Uninstall a package: stop and remove all related containers, clean data.
@@ -342,7 +312,8 @@ impl RpcHandler {
             }
         }
 
-        self.set_uninstall_stage(package_id, "Cleaning up volumes").await;
+        self.set_uninstall_stage(package_id, "Cleaning up volumes")
+            .await;
         // Clean up dangling volumes associated with removed containers
         let _ = tokio::process::Command::new("podman")
             .args(["volume", "prune", "-f"])
@@ -371,7 +342,8 @@ impl RpcHandler {
 
         // Clean data directories unless preserve_data
         if !preserve_data {
-            self.set_uninstall_stage(package_id, "Removing app data").await;
+            self.set_uninstall_stage(package_id, "Removing app data")
+                .await;
             let data_dirs = get_data_dirs_for_app(package_id);
             for dir in &data_dirs {
                 tracing::info!("Uninstall {}: removing data {}", package_id, dir);
@@ -579,3 +551,185 @@ impl RpcHandler {
         Ok(serde_json::json!({ "status": "stopped", "app_id": app_id }))
     }
 }
+
+// ---------------------------------------------------------------------------
+// Background workers for async package lifecycle RPCs.
+//
+// Extracted from the pre-async RPC handlers so the transitional state is
+// visible to the UI immediately. Each worker is pure IO over podman + the
+// crash_recovery helpers — no StateManager access here so we don't need
+// a handler reference. The caller does state flipping before/after.
+// ---------------------------------------------------------------------------
+
+/// Start containers in dependency order. Includes the post-start 3s wait +
+/// exit-check verification from the original synchronous handler (critical
+/// for catching "podman start succeeded but container immediately exited"
+/// failure modes).
+async fn do_package_start(to_start: &[String]) -> Result<()> {
+    let mut errors = Vec::new();
+    for (i, name) in to_start.iter().enumerate() {
+        if i > 0 {
+            tokio::time::sleep(std::time::Duration::from_secs(2)).await;
+        }
+        tracing::info!("Starting container: {}", name);
+        let out = tokio::process::Command::new("podman")
+            .args(["start", name])
+            .output()
+            .await
+            .context(format!("Failed to exec podman start {}", name))?;
+        if !out.status.success() {
+            let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
+            tracing::error!("Failed to start {}: {}", name, stderr);
+            install_log(&format!("START FAIL: {} — {}", name, stderr)).await;
+            errors.push(format!("{}: {}", name, stderr));
+        }
+    }
+    if !errors.is_empty() {
+        return Err(anyhow::anyhow!("Start failed: {}", errors.join("; ")));
+    }
+
+    // Post-start exit verification (podman start can succeed even if the
+    // container exits immediately after).
+    tokio::time::sleep(std::time::Duration::from_secs(3)).await;
+    for name in to_start {
+        let status = tokio::process::Command::new("podman")
+            .args(["inspect", name, "--format", "{{.State.Status}}"])
+            .output()
+            .await;
+        if let Ok(o) = status {
+            let state = String::from_utf8_lossy(&o.stdout).trim().to_string();
+            if state == "exited" {
+                let logs = tokio::process::Command::new("podman")
+                    .args(["logs", "--tail", "5", name])
+                    .output()
+                    .await;
+                let log_text = logs
+                    .map(|o| {
+                        let combined = format!(
+                            "{}{}",
+                            String::from_utf8_lossy(&o.stdout),
+                            String::from_utf8_lossy(&o.stderr)
+                        );
+                        combined.chars().take(200).collect::<String>()
+                    })
+                    .unwrap_or_default();
+                tracing::error!("Container {} exited after start: {}", name, log_text);
+                install_log(&format!("START EXITED: {} — {}", name, log_text)).await;
+                errors.push(format!("{}: exited after start", name));
+            }
+        }
+    }
+    if !errors.is_empty() {
+        return Err(anyhow::anyhow!(
+            "Containers exited after start: {}",
+            errors.join("; ")
+        ));
+    }
+    Ok(())
+}
+
+/// Stop all containers with their per-container graceful-shutdown timeout.
+async fn do_package_stop(containers: &[String]) -> Result<()> {
+    let mut errors = Vec::new();
+    for name in containers {
+        tracing::info!(
+            "Stopping container: {} (timeout: {}s)",
+            name,
+            stop_timeout_secs(name)
+        );
+        let out = tokio::process::Command::new("podman")
+            .args(["stop", "-t", stop_timeout_secs(name), name])
+            .output()
+            .await
+            .context(format!("Failed to exec podman stop {}", name))?;
+        if !out.status.success() {
+            let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
+            tracing::error!("Failed to stop {}: {}", name, stderr);
+            errors.push(format!("{}: {}", name, stderr));
+        }
+    }
+    if !errors.is_empty() {
+        return Err(anyhow::anyhow!("Stop failed: {}", errors.join("; ")));
+    }
+    Ok(())
+}
+
+/// Restart via `podman restart`, falling back to stop+start when restart
+/// fails (rootless podman loopback issues).
+async fn do_package_restart(containers: &[String]) -> Result<()> {
+    let mut errors = Vec::new();
+    for name in containers {
+        tracing::info!("Restarting container: {}", name);
+        let out = tokio::process::Command::new("podman")
+            .args(["restart", "-t", stop_timeout_secs(name), name])
+            .output()
+            .await
+            .context(format!("Failed to exec podman restart {}", name))?;
+
+        if !out.status.success() {
+            let stderr = String::from_utf8_lossy(&out.stderr).trim().to_string();
+            tracing::warn!(
+                "podman restart {} failed: {}, trying stop+start",
+                name,
+                stderr
+            );
+            // Fallback: stop then start
+            let _ = tokio::process::Command::new("podman")
+                .args(["stop", "-t", stop_timeout_secs(name), name])
+                .output()
+                .await;
+            let start_out = tokio::process::Command::new("podman")
+                .args(["start", name])
+                .output()
+                .await
+                .context(format!("Failed to exec podman start {}", name))?;
+            if !start_out.status.success() {
+                let start_err = String::from_utf8_lossy(&start_out.stderr)
+                    .trim()
+                    .to_string();
+                tracing::error!("stop+start {} also failed: {}", name, start_err);
+                errors.push(format!("{}: {}", name, start_err));
+            } else {
+                tracing::info!("Restarted {} via stop+start fallback", name);
+            }
+        }
+    }
+    if !errors.is_empty() {
+        return Err(anyhow::anyhow!("Restart failed: {}", errors.join("; ")));
+    }
+    Ok(())
+}
+
+/// Flip the primary package entry's state and return the pre-transition
+/// state for revert on error. Mirrors `transitional::flip_to_transitional`
+/// but lives here because the package path keys by `package_id` (which may
+/// differ from the container name used by orchestrator-level entries).
+async fn flip_package_state(
+    state_manager: &crate::state::StateManager,
+    package_id: &str,
+    transitional: PackageState,
+) -> Option<PackageState> {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    let prev = data.package_data.get(package_id).map(|e| e.state.clone());
+    if let Some(entry) = data.package_data.get_mut(package_id) {
+        entry.state = transitional;
+        state_manager.update_data(data).await;
+    }
+    prev
+}
+
+/// Write the package entry's final state. No-op if the entry has since
+/// been removed (uninstall race).
+async fn set_package_state(
+    state_manager: &crate::state::StateManager,
+    package_id: &str,
+    new_state: PackageState,
+) {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    if let Some(entry) = data.package_data.get_mut(package_id) {
+        if entry.state != new_state {
+            entry.state = new_state;
+            state_manager.update_data(data).await;
+        }
+    }
+}

core/archipelago/src/api/rpc/package/stacks.rs

@@ -60,6 +60,70 @@ async fn adopt_stack_if_exists(
     })))
 }
 
+async fn install_stack_via_orchestrator(
+    handler: &RpcHandler,
+    stack_name: &str,
+    app_ids: &[&str],
+) -> Result<Option<serde_json::Value>> {
+    let Some(orchestrator) = handler.orchestrator.as_ref() else {
+        return Ok(None);
+    };
+
+    install_log(&format!(
+        "INSTALL ORCH: {} stack — attempting orchestrator install of [{}]",
+        stack_name,
+        app_ids.join(", ")
+    ))
+    .await;
+
+    for app_id in app_ids {
+        match orchestrator.install(app_id).await {
+            Ok(container_name) => {
+                install_log(&format!(
+                    "INSTALL ORCH: {} stack — app {} installed as {}",
+                    stack_name, app_id, container_name
+                ))
+                .await;
+            }
+            Err(e) if e.to_string().contains("unknown app_id") => {
+                install_log(&format!(
+                    "INSTALL ORCH SKIP: {} stack — app {} unknown, falling back to legacy stack installer",
+                    stack_name, app_id
+                ))
+                .await;
+                return Ok(None);
+            }
+            Err(e) => {
+                install_log(&format!(
+                    "INSTALL ORCH FAIL: {} stack — app {} failed: {}",
+                    stack_name, app_id, e
+                ))
+                .await;
+                return Err(e.context(format!(
+                    "orchestrator stack install {} failed at app {}",
+                    stack_name, app_id
+                )));
+            }
+        }
+    }
+
+    install_log(&format!("INSTALL ORCH OK: {} stack", stack_name)).await;
+    Ok(Some(serde_json::json!({
+        "success": true,
+        "package_id": stack_name,
+        "message": format!("{} stack installed and started", stack_name),
+        "path": "orchestrator"
+    })))
+}
+
+fn btcpay_stack_app_ids() -> &'static [&'static str] {
+    &["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"]
+}
+
+fn mempool_stack_app_ids() -> &'static [&'static str] {
+    &["archy-mempool-db", "mempool-api", "archy-mempool-web"]
+}
+
 const REGISTRY: &str = "git.tx1138.com/lfg2025";
 
 /// Pull an image with retry and exponential backoff (3 attempts).
@@ -136,7 +200,7 @@ impl RpcHandler {
 
         let images = [
             "git.tx1138.com/lfg2025/immich-postgres:14-vectorchord0.4.3-pgvectors0.2.0",
-            "git.tx1138.com/lfg2025/valkey:7-alpine",
+            "docker.io/valkey/valkey:7-alpine",
             "git.tx1138.com/lfg2025/immich-server:release",
         ];
         for img in &images {
@@ -152,6 +216,16 @@ impl RpcHandler {
             ])
             .output()
             .await;
+        let _ = tokio::process::Command::new("sudo")
+            .args([
+                "chown",
+                "-R",
+                "1000:1000",
+                "/var/lib/archipelago/immich",
+                "/var/lib/archipelago/immich-db",
+            ])
+            .output()
+            .await;
         let _ = tokio::process::Command::new("podman")
             .args(["network", "create", "immich-net"])
             .output()
@@ -210,13 +284,15 @@ impl RpcHandler {
                 "--network-alias",
                 "immich_redis",
                 "--cap-drop=ALL",
+                "--cap-add=SETGID",
+                "--cap-add=SETUID",
                 "--security-opt=no-new-privileges:true",
                 "--memory=128m",
                 "--pids-limit=2048",
                 "--health-cmd=valkey-cli ping || exit 1",
                 "--health-interval=30s",
                 "--health-retries=3",
-                "git.tx1138.com/lfg2025/valkey:7-alpine",
+                "docker.io/valkey/valkey:7-alpine",
             ])
             .output()
             .await;
@@ -273,7 +349,6 @@ impl RpcHandler {
         }))
     }
 
-
     /// Install BTCPay stack (postgres + nbxplorer + btcpay-server).
     pub(super) async fn install_btcpay_stack(&self) -> Result<serde_json::Value> {
         if let Some(adopted) = adopt_stack_if_exists(
@@ -286,6 +361,12 @@ impl RpcHandler {
             return Ok(adopted);
         }
 
+        if let Some(orchestrated) =
+            install_stack_via_orchestrator(self, "btcpay-server", btcpay_stack_app_ids()).await?
+        {
+            return Ok(orchestrated);
+        }
+
         // Dependency check: Bitcoin must be running
         let deps = super::dependencies::detect_running_deps().await?;
         super::dependencies::check_install_deps("btcpay-server", &deps)?;
@@ -473,25 +554,36 @@ impl RpcHandler {
     /// Install Mempool stack (mariadb + mempool-api + mempool-web).
     pub(super) async fn install_mempool_stack(&self) -> Result<serde_json::Value> {
         if let Some(adopted) = adopt_stack_if_exists(
-            "archy-mempool-web",
             "mempool",
-            &["archy-mempool-db", "archy-mempool-api", "archy-mempool-web"],
+            "mempool",
+            &[
+                "archy-mempool-db",
+                "mempool-api",
+                "mempool",
+                "archy-mempool-web",
+                "archy-mempool-api",
+            ],
         )
         .await?
         {
             return Ok(adopted);
         }
 
+        if let Some(orchestrated) =
+            install_stack_via_orchestrator(self, "mempool", mempool_stack_app_ids()).await?
+        {
+            return Ok(orchestrated);
+        }
+
         // Dependency check: Bitcoin + ElectrumX must be running
         let deps = super::dependencies::detect_running_deps().await?;
         super::dependencies::check_install_deps("mempool", &deps)?;
+        let (_, rpc_pass) = crate::bitcoin_rpc::bitcoin_rpc_credentials().await;
 
         install_log("INSTALL START: mempool (stack: mariadb + mempool-api + mempool-web)").await;
 
-        let (rpc_user, rpc_pass) = crate::bitcoin_rpc::bitcoin_rpc_credentials().await;
-
         let db_pass = super::config::read_or_generate_secret("mempool-db-password").await;
-        let root_pass = super::config::read_or_generate_secret("mempool-db-root-password").await;
+        let root_pass = super::config::read_or_generate_secret("mysql-root-db-password").await;
 
         let images = [
             &format!("{}/mariadb:11.4.10", REGISTRY),
@@ -594,17 +686,17 @@ impl RpcHandler {
                 "-e",
                 "MEMPOOL_BACKEND=electrum",
                 "-e",
-                "ELECTRUM_HOST=host.containers.internal",
+                "ELECTRUM_HOST=electrumx",
                 "-e",
                 "ELECTRUM_PORT=50001",
                 "-e",
                 "ELECTRUM_TLS_ENABLED=false",
                 "-e",
-                "CORE_RPC_HOST=host.containers.internal",
+                "CORE_RPC_HOST=bitcoin-knots",
                 "-e",
                 "CORE_RPC_PORT=8332",
                 "-e",
-                &format!("CORE_RPC_USERNAME={}", rpc_user),
+                "CORE_RPC_USERNAME=archipelago",
                 "-e",
                 &format!("CORE_RPC_PASSWORD={}", rpc_pass),
                 "-e",
@@ -965,3 +1057,20 @@ impl RpcHandler {
         }))
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use super::{btcpay_stack_app_ids, mempool_stack_app_ids};
+
+    #[test]
+    fn stack_app_id_sets_match_migration_manifests() {
+        assert_eq!(
+            btcpay_stack_app_ids(),
+            ["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"]
+        );
+        assert_eq!(
+            mempool_stack_app_ids(),
+            ["archy-mempool-db", "mempool-api", "archy-mempool-web"]
+        );
+    }
+}

core/archipelago/src/api/rpc/package/update.rs

@@ -1,7 +1,7 @@
 //! Per-app manual update handler.
 //!
 //! Flow: validate → set Updating state → graceful stop → pull new image(s) →
-//! remove old container(s) → recreate via reconcile script → verify running.
+//! remove old container(s) → recreate (orchestrator-first, legacy fallback) → verify running.
 //! Data volumes are preserved (bind mounts, not stored in container).
 
 use super::config::get_containers_for_app;
@@ -11,7 +11,7 @@ use super::runtime::stop_timeout_secs;
 use super::validation::validate_app_id;
 use crate::api::rpc::RpcHandler;
 use crate::container::image_versions;
-use crate::data_model::PackageState;
+use crate::data_model::{InstallPhase, PackageState};
 use anyhow::{Context, Result};
 use tokio::io::{AsyncBufReadExt, BufReader};
 use tracing::{error, info, warn};
@@ -34,15 +34,10 @@ impl RpcHandler {
         let pinned = image_versions::pinned_image_for_app(package_id)
             .ok_or_else(|| anyhow::anyhow!("No pinned image found for {}", package_id))?;
 
-        // Reject if already updating
-        {
-            let (data, _) = self.state_manager.get_snapshot().await;
-            if let Some(entry) = data.package_data.get(package_id) {
-                if entry.state == PackageState::Updating {
-                    return Err(anyhow::anyhow!("{} is already updating", package_id));
-                }
-            }
-        }
+        // Note: the `already updating` guard lives in `spawn_package_update`
+        // (the async wrapper that dispatch actually routes to). By the time
+        // this inner function runs, the wrapper has already flipped state to
+        // `Updating`, so duplicating the check here would be a false positive.
 
         install_log(&format!("UPDATE: {} → {}", package_id, pinned)).await;
 
@@ -56,6 +51,64 @@ impl RpcHandler {
             self.state_manager.update_data(data).await;
         }
 
+        // Preferred path: for single-container apps managed by manifests, route
+        // updates through the orchestrator's upgrade lifecycle instead of the
+        // legacy shell/CLI flow. Keep stack-style packages on legacy for now.
+        if should_try_orchestrator_update(package_id, self.orchestrator.is_some()) {
+            let orchestrator_app_id = orchestrator_update_app_id(package_id);
+            self.set_install_phase(package_id, InstallPhase::Preparing)
+                .await;
+            install_log(&format!(
+                "UPDATE ORCH: {} — attempting orchestrator upgrade as {}",
+                package_id, orchestrator_app_id
+            ))
+            .await;
+
+            if let Some(orchestrator) = self.orchestrator.as_ref() {
+                match orchestrator.upgrade(orchestrator_app_id).await {
+                    Ok(()) => {
+                        self.set_install_phase(package_id, InstallPhase::WaitingHealthy)
+                            .await;
+                        if let Ok(health) = orchestrator.health(orchestrator_app_id).await {
+                            if health != "healthy" {
+                                warn!(
+                                    "Update {}: orchestrator upgrade completed with health={} (expected healthy)",
+                                    package_id, health
+                                );
+                            }
+                        }
+                        install_log(&format!(
+                            "UPDATE ORCH OK: {} (app={})",
+                            package_id, orchestrator_app_id
+                        ))
+                        .await;
+                        self.clear_install_progress(package_id).await;
+                        return Ok(serde_json::json!({
+                            "status": "updated",
+                            "package_id": package_id,
+                        }));
+                    }
+                    Err(e) if is_unknown_app_id_error(&e) => {
+                        info!(
+                            "Update {}: orchestrator has no manifest mapping yet, falling back to legacy updater",
+                            package_id
+                        );
+                        install_log(&format!(
+                            "UPDATE ORCH SKIP: {} — unknown app_id, using legacy flow",
+                            package_id
+                        ))
+                        .await;
+                    }
+                    Err(e) => {
+                        install_log(&format!("UPDATE ORCH FAIL: {} — {}", package_id, e)).await;
+                        self.clear_install_progress(package_id).await;
+                        self.clear_update_state(package_id).await;
+                        return Err(e.context(format!("Orchestrator update {} failed", package_id)));
+                    }
+                }
+            }
+        }
+
         // Resolve images to pull — either a stack or single container
         let images_to_pull = self.resolve_images_to_pull(package_id, &pinned);
 
@@ -101,6 +154,11 @@ impl RpcHandler {
         containers: &[String],
         images_to_pull: &[(String, String)],
     ) -> Result<()> {
+        // Phase: Preparing — about to stop the running container(s) so
+        // we can swap images. Fast.
+        self.set_install_phase(package_id, InstallPhase::Preparing)
+            .await;
+
         // 1. Graceful stop all containers (reverse order for dependencies)
         info!(
             "Update {}: stopping {} containers",
@@ -130,6 +188,10 @@ impl RpcHandler {
             }
         }
 
+        // Phase: PullingImage — about to fetch each pinned image in turn.
+        self.set_install_phase(package_id, InstallPhase::PullingImage)
+            .await;
+
         // 2. Pull new images with progress
         info!(
             "Update {}: pulling {} images",
@@ -173,38 +235,23 @@ impl RpcHandler {
             }
         }
 
-        // 4. Recreate via reconcile script (single source of truth for container specs)
-        info!("Update {}: recreating containers via reconcile", package_id);
+        // Phase: CreatingContainer — about to recreate each container.
+        self.set_install_phase(package_id, InstallPhase::CreatingContainer)
+            .await;
+
+        // 4. Recreate containers (orchestrator-first, reconcile fallback)
+        info!("Update {}: recreating containers", package_id);
         for name in containers {
-            let out = tokio::process::Command::new("bash")
-                .args([
-                    "/opt/archipelago/scripts/reconcile-containers.sh",
-                    &format!("--container={}", name),
-                    "--force",
-                ])
-                .output()
-                .await
-                .context(format!("Failed to reconcile {}", name))?;
-            if !out.status.success() {
-                let stderr = String::from_utf8_lossy(&out.stderr);
-                let stdout = String::from_utf8_lossy(&out.stdout);
-                error!(
-                    "Update {}: reconcile {} failed:\nstdout: {}\nstderr: {}",
-                    package_id,
-                    name,
-                    stdout.trim(),
-                    stderr.trim()
-                );
-                return Err(anyhow::anyhow!(
-                    "Reconcile failed for {}: {}",
-                    name,
-                    stderr.trim()
-                ));
-            }
+            self.recreate_container_for_update(package_id, name).await?;
             // Brief delay between containers for dependency initialization
             tokio::time::sleep(std::time::Duration::from_secs(2)).await;
         }
 
+        // Phase: WaitingHealthy — reconcile has started every container,
+        // now verifying each reached running state.
+        self.set_install_phase(package_id, InstallPhase::WaitingHealthy)
+            .await;
+
         // 5. Verify containers reached running state
         tokio::time::sleep(std::time::Duration::from_secs(5)).await;
         for name in containers {
@@ -226,6 +273,51 @@ impl RpcHandler {
         Ok(())
     }
 
+    async fn recreate_container_for_update(
+        &self,
+        package_id: &str,
+        container_name: &str,
+    ) -> Result<()> {
+        let Some(orchestrator) = self.orchestrator.as_ref() else {
+            return Err(anyhow::anyhow!(
+                "Cannot recreate {} during update {}: orchestrator unavailable",
+                container_name,
+                package_id
+            ));
+        };
+
+        let mut attempted = Vec::new();
+        for app_id in candidate_app_ids_for_container(container_name) {
+            attempted.push(app_id.clone());
+            match orchestrator.install(&app_id).await {
+                Ok(created_name) => {
+                    install_log(&format!(
+                        "UPDATE ORCH RECREATE OK: {} — container={} app_id={} created={}",
+                        package_id, container_name, app_id, created_name
+                    ))
+                    .await;
+                    return Ok(());
+                }
+                Err(e) if is_unknown_app_id_error(&e) => {
+                    continue;
+                }
+                Err(e) => {
+                    return Err(e.context(format!(
+                        "orchestrator recreate failed for update {} (container={}, app_id={})",
+                        package_id, container_name, app_id
+                    )));
+                }
+            }
+        }
+
+        Err(anyhow::anyhow!(
+            "No manifest mapping found while recreating {} during update {} (attempted app_ids: {})",
+            container_name,
+            package_id,
+            attempted.join(", ")
+        ))
+    }
+
     /// Pull a single image with progress broadcasting (reuses install progress pattern).
     async fn pull_update_image(&self, package_id: &str, image: &str) -> Result<()> {
         self.set_install_progress(package_id, 0, 0).await;
@@ -296,15 +388,16 @@ impl RpcHandler {
                 Ok(o) => {
                     let stderr = String::from_utf8_lossy(&o.stderr);
                     warn!("Rollback: could not restart {}: {}", name, stderr.trim());
-                    // Container was already removed — try reconcile to recreate with old image
-                    let _ = tokio::process::Command::new("bash")
-                        .args([
-                            "/opt/archipelago/scripts/reconcile-containers.sh",
-                            &format!("--container={}", name),
-                            "--force",
-                        ])
-                        .output()
-                        .await;
+                    // Container was already removed (forward path ran `podman rm`).
+                    // Recreate via orchestrator-first path with legacy fallback.
+                    if let Err(recreate_err) =
+                        self.recreate_container_for_update(package_id, name).await
+                    {
+                        error!(
+                            "Rollback: failed to recreate {} during rollback of {}: {}",
+                            name, package_id, recreate_err
+                        );
+                    }
                 }
                 Err(e) => {
                     error!("Rollback: failed to restart {}: {}", name, e);
@@ -325,3 +418,129 @@ impl RpcHandler {
         self.state_manager.update_data(data).await;
     }
 }
+
+fn should_try_orchestrator_update(package_id: &str, orchestrator_available: bool) -> bool {
+    orchestrator_available && !uses_legacy_update_flow(package_id)
+}
+
+fn orchestrator_update_app_id(package_id: &str) -> &str {
+    match package_id {
+        "bitcoin-knots" => "bitcoin-core",
+        "electrs" | "mempool-electrs" => "electrumx",
+        _ => package_id,
+    }
+}
+
+fn uses_legacy_update_flow(package_id: &str) -> bool {
+    matches!(
+        package_id,
+        // Multi-container stacks still updated via the stack-aware path.
+        "immich" | "penpot" | "penpot-frontend" | "indeedhub"
+    )
+}
+
+fn is_unknown_app_id_error(err: &anyhow::Error) -> bool {
+    err.chain()
+        .any(|cause| cause.to_string().contains("unknown app_id"))
+}
+
+fn candidate_app_ids_for_container(container_name: &str) -> Vec<String> {
+    let mut out = Vec::new();
+    let mut push = |s: &str| {
+        if !out.iter().any(|e: &String| e == s) {
+            out.push(s.to_string());
+        }
+    };
+
+    match container_name {
+        "bitcoin-knots" => {
+            push("bitcoin-core");
+            push("bitcoin-knots");
+        }
+        "archy-bitcoin-ui" => push("bitcoin-ui"),
+        "archy-lnd-ui" => push("lnd-ui"),
+        "archy-electrs-ui" => push("electrs-ui"),
+        "mempool" => {
+            push("archy-mempool-web");
+            push("mempool");
+        }
+        _ => {}
+    }
+
+    push(container_name);
+    if let Some(stripped) = container_name.strip_prefix("archy-") {
+        push(stripped);
+    }
+    out
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{
+        candidate_app_ids_for_container, orchestrator_update_app_id,
+        should_try_orchestrator_update, uses_legacy_update_flow,
+    };
+
+    #[test]
+    fn legacy_flow_for_stack_apps() {
+        for app in ["immich", "penpot", "indeedhub"] {
+            assert!(uses_legacy_update_flow(app), "{app} should stay legacy");
+        }
+    }
+
+    #[test]
+    fn orchestrator_flow_for_single_apps() {
+        for app in [
+            "lnd",
+            "bitcoin-core",
+            "searxng",
+            "grafana",
+            "btcpay-server",
+            "mempool",
+            "fedimint",
+        ] {
+            assert!(
+                !uses_legacy_update_flow(app),
+                "{app} should be orchestrator-first"
+            );
+            assert!(
+                should_try_orchestrator_update(app, true),
+                "{app} should use orchestrator when available"
+            );
+        }
+    }
+
+    #[test]
+    fn no_orchestrator_means_no_orchestrator_flow() {
+        assert!(!should_try_orchestrator_update("lnd", false));
+        assert!(!should_try_orchestrator_update("btcpay-server", false));
+    }
+
+    #[test]
+    fn container_name_candidates_cover_common_aliases() {
+        assert_eq!(
+            candidate_app_ids_for_container("bitcoin-knots"),
+            vec!["bitcoin-core", "bitcoin-knots"]
+        );
+        assert_eq!(
+            candidate_app_ids_for_container("archy-bitcoin-ui"),
+            vec!["bitcoin-ui", "archy-bitcoin-ui"]
+        );
+        assert_eq!(
+            candidate_app_ids_for_container("mempool"),
+            vec!["archy-mempool-web", "mempool"]
+        );
+        assert_eq!(
+            candidate_app_ids_for_container("archy-mempool-db"),
+            vec!["archy-mempool-db", "mempool-db"]
+        );
+    }
+
+    #[test]
+    fn update_aliases_map_to_manifest_app_ids() {
+        assert_eq!(orchestrator_update_app_id("bitcoin-knots"), "bitcoin-core");
+        assert_eq!(orchestrator_update_app_id("electrs"), "electrumx");
+        assert_eq!(orchestrator_update_app_id("mempool-electrs"), "electrumx");
+        assert_eq!(orchestrator_update_app_id("fedimint"), "fedimint");
+    }
+}

core/archipelago/src/api/rpc/peers.rs

@@ -147,8 +147,7 @@ impl RpcHandler {
             .get("onion")
             .and_then(|v| v.as_str())
             .ok_or_else(|| anyhow::anyhow!("Missing onion"))?;
-        let fips_npub =
-            crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
+        let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
         let reachable = node_message::check_peer_reachable(onion, fips_npub.as_deref())
             .await
             .unwrap_or(false);

core/archipelago/src/api/rpc/transitional.rs

@@ -0,0 +1,172 @@
+//! Async lifecycle helper for container Stop/Start/Restart RPCs.
+//!
+//! The `ContainerOrchestrator` trait is intentionally synchronous — blocking
+//! calls keep the reconciler, boot flow, chaos harness, and unit tests
+//! deterministic. But the RPC layer must return to the UI in <1s so the
+//! dashboard can render a transitional "Stopping…" / "Starting…" label while
+//! the underlying `podman stop` (up to 600s for bitcoin-core) runs in the
+//! background.
+//!
+//! `RpcHandler::spawn_transitional` bridges the two: it
+//!   1. flips the package state in `StateManager` to the appropriate
+//!      transitional variant (`Stopping` / `Starting` / `Restarting`),
+//!      which fans out to WebSocket clients immediately.
+//!   2. `tokio::spawn`s the actual orchestrator call.
+//!   3. on success, writes the final state (`Stopped` / `Running`).
+//!   4. on error, reverts to the pre-transition state and logs via
+//!      `install_log()` so the incident shows up in
+//!      `/var/log/archipelago/container-installs.log`.
+//!
+//! The server.rs package-scan loop must also be taught to preserve
+//! transitional states — see `server.rs:scan_and_update_packages`'s merge
+//! logic and the companion `merge_preserving_transitional` helper.
+
+use super::package::install_log;
+use super::RpcHandler;
+use crate::container::ContainerOrchestrator;
+use crate::data_model::PackageState;
+use crate::state::StateManager;
+use anyhow::Result;
+use std::sync::Arc;
+use tracing::{error, info, warn};
+
+/// The three transitional lifecycle operations that run asynchronously from
+/// the RPC handler. `Install` and `Remove` are intentionally NOT here — they
+/// already have their own progress-tracking paths (`install_progress`,
+/// `uninstall_stage`) with multi-step UI feedback.
+#[derive(Debug, Clone, Copy)]
+pub(super) enum Op {
+    Stop,
+    Start,
+    Restart,
+}
+
+impl Op {
+    /// The `PackageState` to set on the entry while the operation is in
+    /// flight. The package-scan merge loop must preserve this variant and
+    /// refuse to overwrite it with whatever podman reports (see
+    /// `merge_preserving_transitional` in server.rs).
+    fn transitional_state(self) -> PackageState {
+        match self {
+            Op::Stop => PackageState::Stopping,
+            Op::Start => PackageState::Starting,
+            Op::Restart => PackageState::Restarting,
+        }
+    }
+
+    /// The `PackageState` to set on success. On error the caller reverts to
+    /// the pre-transition state rather than using these.
+    fn final_state_on_success(self) -> PackageState {
+        match self {
+            Op::Stop => PackageState::Stopped,
+            Op::Start => PackageState::Running,
+            Op::Restart => PackageState::Running,
+        }
+    }
+
+    /// Prefix used in `install_log` entries so post-mortem readers can grep
+    /// the operation that failed.
+    fn log_prefix(self) -> &'static str {
+        match self {
+            Op::Stop => "STOP",
+            Op::Start => "START",
+            Op::Restart => "RESTART",
+        }
+    }
+
+    /// Call the orchestrator for this op. Kept in one place so the spawned
+    /// task doesn't repeat the match four times.
+    async fn dispatch(self, orch: &dyn ContainerOrchestrator, app_id: &str) -> Result<()> {
+        match self {
+            Op::Stop => orch.stop(app_id).await,
+            Op::Start => orch.start(app_id).await,
+            Op::Restart => orch.restart(app_id).await,
+        }
+    }
+}
+
+impl RpcHandler {
+    /// Flip the package state to `op.transitional_state()`, spawn a background
+    /// task that runs `op.dispatch()`, and return immediately. The spawned
+    /// task writes the final state on completion or reverts to the
+    /// pre-transition state on failure.
+    ///
+    /// If no package entry exists for `app_id` (e.g. Start on a container
+    /// that was never installed), no pre-state is recorded and the spawn
+    /// still runs — the post-success path will no-op the state write and
+    /// the next scan will pick up the newly-created entry with the correct
+    /// state. This keeps the helper usable for stacks that lazily create
+    /// their entries.
+    pub(super) async fn spawn_transitional(&self, op: Op, app_id: String) -> Result<()> {
+        let orchestrator = self
+            .orchestrator
+            .as_ref()
+            .ok_or_else(|| anyhow::anyhow!("Container orchestrator not available"))?
+            .clone();
+        let state_manager = Arc::clone(&self.state_manager);
+
+        // Snapshot pre-transition state (for revert on error) and flip to
+        // transitional variant. Done BEFORE the spawn so the WebSocket push
+        // beats the RPC response — the UI should see "Stopping…" the moment
+        // it gets the RPC ok, not on the next scan.
+        let pre_state =
+            flip_to_transitional(&state_manager, &app_id, op.transitional_state()).await;
+
+        let log_prefix = op.log_prefix();
+        let app_id_log = app_id.clone();
+        install_log(&format!("{}: {}", log_prefix, app_id_log)).await;
+
+        tokio::spawn(async move {
+            match op.dispatch(orchestrator.as_ref(), &app_id).await {
+                Ok(()) => {
+                    info!("{} complete: {}", log_prefix, app_id);
+                    set_state(&state_manager, &app_id, op.final_state_on_success()).await;
+                }
+                Err(e) => {
+                    error!("{} failed for {}: {:#}", log_prefix, app_id, e);
+                    install_log(&format!("{} FAIL: {} — {:#}", log_prefix, app_id, e)).await;
+                    // Revert to pre-transition state if we had one; otherwise
+                    // leave the entry untouched so the next scan reconciles.
+                    if let Some(prev) = pre_state {
+                        set_state(&state_manager, &app_id, prev).await;
+                    } else {
+                        warn!(
+                            "{}: no pre-transition state recorded for {}; leaving entry to next scan",
+                            log_prefix, app_id
+                        );
+                    }
+                }
+            }
+        });
+
+        Ok(())
+    }
+}
+
+/// Flip the entry's state to `transitional` and return the previous state.
+/// Returns `None` if there is no entry for `app_id`.
+async fn flip_to_transitional(
+    state_manager: &StateManager,
+    app_id: &str,
+    transitional: PackageState,
+) -> Option<PackageState> {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    let prev = data.package_data.get(app_id).map(|e| e.state.clone());
+    if let Some(entry) = data.package_data.get_mut(app_id) {
+        entry.state = transitional;
+        state_manager.update_data(data).await;
+    }
+    prev
+}
+
+/// Set the entry's state to `new_state`. No-ops if the entry has since been
+/// removed (e.g. uninstall ran concurrently).
+async fn set_state(state_manager: &StateManager, app_id: &str, new_state: PackageState) {
+    let (mut data, _) = state_manager.get_snapshot().await;
+    if let Some(entry) = data.package_data.get_mut(app_id) {
+        if entry.state != new_state {
+            entry.state = new_state;
+            state_manager.update_data(data).await;
+        }
+    }
+}

core/archipelago/src/api/rpc/update.rs

@@ -162,10 +162,8 @@ impl RpcHandler {
         // progress bar after navigation instead of showing the fake
         // creep again. An RPC poll every ~1s during download drives a
         // real progress indicator that survives route changes.
-        let downloaded = update::DOWNLOAD_BYTES
-            .load(std::sync::atomic::Ordering::Relaxed);
-        let total = update::DOWNLOAD_TOTAL
-            .load(std::sync::atomic::Ordering::Relaxed);
+        let downloaded = update::DOWNLOAD_BYTES.load(std::sync::atomic::Ordering::Relaxed);
+        let total = update::DOWNLOAD_TOTAL.load(std::sync::atomic::Ordering::Relaxed);
         let active = total > 0 && downloaded < total;
         let completed = total > 0 && downloaded >= total;
 
@@ -175,8 +173,7 @@ impl RpcHandler {
         // read timeout). The UI uses this to surface a Cancel button
         // with explanatory copy.
         let stalled = if active {
-            let last_at = update::DOWNLOAD_PROGRESS_AT
-                .load(std::sync::atomic::Ordering::Relaxed);
+            let last_at = update::DOWNLOAD_PROGRESS_AT.load(std::sync::atomic::Ordering::Relaxed);
             if last_at > 0 {
                 let now = std::time::SystemTime::now()
                     .duration_since(std::time::UNIX_EPOCH)

core/archipelago/src/avatar.rs

@@ -18,12 +18,12 @@ use base64::Engine;
 /// Convert a byte to an HSL triple biased toward readable foregrounds on
 /// dark backgrounds (saturation 60–85%, lightness 52–70%).
 fn hue_color(seed: u8) -> String {
-    let hue = (seed as u16) * 360 / 256;
+    let hue = (seed as u32) * 360 / 256;
     format!("hsl({}, 72%, 60%)", hue)
 }
 
 fn accent_color(seed: u8) -> String {
-    let hue = (seed as u16) * 360 / 256;
+    let hue = (seed as u32) * 360 / 256;
     format!("hsl({}, 80%, 68%)", hue)
 }
 
@@ -37,7 +37,10 @@ fn encode_svg(svg: &str) -> String {
 /// avatar rather than an error.
 fn seed_bytes(pubkey_hex: &str) -> [u8; 8] {
     let mut out = [0u8; 8];
-    let clean: String = pubkey_hex.chars().filter(|c| c.is_ascii_hexdigit()).collect();
+    let clean: String = pubkey_hex
+        .chars()
+        .filter(|c| c.is_ascii_hexdigit())
+        .collect();
     for (i, byte) in out.iter_mut().enumerate() {
         let lo = i * 2;
         if clean.len() >= lo + 2 {

core/archipelago/src/bootstrap.rs

@@ -24,8 +24,7 @@ use crate::update::host_sudo;
 const DOCTOR_SH: &str = include_str!("../../../scripts/container-doctor.sh");
 const DOCTOR_SERVICE: &str =
     include_str!("../../../image-recipe/configs/archipelago-doctor.service");
-const DOCTOR_TIMER: &str =
-    include_str!("../../../image-recipe/configs/archipelago-doctor.timer");
+const DOCTOR_TIMER: &str = include_str!("../../../image-recipe/configs/archipelago-doctor.timer");
 
 const DOCTOR_SH_PATH: &str = "/home/archipelago/archy/scripts/container-doctor.sh";
 const DOCTOR_SERVICE_PATH: &str = "/etc/systemd/system/archipelago-doctor.service";
@@ -110,8 +109,8 @@ async fn run() -> Result<bool> {
         if let Err(e) = host_sudo(&["systemctl", "daemon-reload"]).await {
             warn!("daemon-reload failed: {:#}", e);
         }
-        if let Err(e) = host_sudo(&["systemctl", "enable", "--now", "archipelago-doctor.timer"])
-            .await
+        if let Err(e) =
+            host_sudo(&["systemctl", "enable", "--now", "archipelago-doctor.timer"]).await
         {
             warn!("enable archipelago-doctor.timer failed: {:#}", e);
         } else if !timer_enabled {
@@ -188,10 +187,7 @@ async fn run_nginx() -> Result<bool> {
     }
 
     if !Path::new(NGINX_CONF_PATH).exists() {
-        debug!(
-            "{} missing — skipping nginx bootstrap",
-            NGINX_CONF_PATH
-        );
+        debug!("{} missing — skipping nginx bootstrap", NGINX_CONF_PATH);
         return Ok(false);
     }
 

core/archipelago/src/container/bitcoin_ui.rs

@@ -0,0 +1,298 @@
+//! bitcoin-ui nginx.conf renderer.
+//!
+//! Step 7 of the rust-orchestrator migration. Replaces the old
+//! `sed -i __BITCOIN_RPC_AUTH__` approach from `first-boot-containers.sh`
+//! (which destructively overwrote its own template, broke on rotation,
+//! and had no story for dual Knots/Core UIs) with a binary-embedded
+//! template rendered at install/reconcile time and atomic-written to
+//! disk.
+//!
+//! The manifest bind-mounts the rendered file read-only into the
+//! container at `/etc/nginx/conf.d/default.conf`. On every reconcile
+//! pass we re-render and compare — if the rendered bytes would differ
+//! from what's on disk (password rotated, template changed via OTA),
+//! we rewrite atomically and the reconciler restarts the container.
+//!
+//! Source of truth:
+//!   * RPC user: hardcoded `archipelago` (matches the image's `bitcoin.conf`).
+//!   * RPC password: `/var/lib/archipelago/secrets/bitcoin-rpc-password`,
+//!     plaintext, written by the seed-derived credential setup.
+//!
+//! Both Knots and Core back-ends expose RPC on 127.0.0.1:8332 with the
+//! same auth shape, so one template serves both.
+
+use anyhow::{Context, Result};
+use base64::Engine;
+use sha2::{Digest, Sha256};
+use std::path::{Path, PathBuf};
+use std::sync::atomic::{AtomicU64, Ordering};
+use std::time::{SystemTime, UNIX_EPOCH};
+use tokio::fs;
+
+/// The nginx.conf template. Embedded at compile time so it can never
+/// drift from the code that renders it, and ships atomically with OTA.
+///
+/// `{{BITCOIN_RPC_AUTH}}` is the only placeholder — replaced with a
+/// `base64(user:password)` blob at render time.
+pub(crate) const TEMPLATE: &str = include_str!("bitcoin_ui_nginx.conf.template");
+
+/// The single placeholder in `TEMPLATE`.
+const PLACEHOLDER: &str = "{{BITCOIN_RPC_AUTH}}";
+
+/// Hardcoded RPC user. Matches the user written into `bitcoin.conf` by
+/// the bitcoin-core/bitcoin-knots bootstrap, and the legacy
+/// `BITCOIN_RPC_USER="archipelago"` from `first-boot-containers.sh`.
+const RPC_USER: &str = "archipelago";
+
+/// Default path to the plaintext RPC password secret.
+///
+/// Written by the seed-derived credential flow; same file the bash
+/// scripts read today at `first-boot-containers.sh:277` and `:1225`.
+pub const DEFAULT_SECRET_PATH: &str = "/var/lib/archipelago/secrets/bitcoin-rpc-password";
+
+/// Default output path for the rendered nginx.conf.
+///
+/// The manifest bind-mounts this file read-only into the bitcoin-ui
+/// container at `/etc/nginx/conf.d/default.conf`.
+pub const DEFAULT_RENDERED_PATH: &str = "/var/lib/archipelago/bitcoin-ui/nginx.conf";
+
+/// Parameters for rendering. Injectable so tests can hit a tmpdir
+/// instead of `/var/lib/archipelago`.
+#[derive(Debug, Clone)]
+pub struct RenderPaths {
+    /// Path to read the plaintext RPC password from.
+    pub secret_path: PathBuf,
+    /// Path to write the rendered nginx.conf to.
+    pub rendered_path: PathBuf,
+}
+
+impl Default for RenderPaths {
+    fn default() -> Self {
+        Self {
+            secret_path: PathBuf::from(DEFAULT_SECRET_PATH),
+            rendered_path: PathBuf::from(DEFAULT_RENDERED_PATH),
+        }
+    }
+}
+
+/// Outcome of a render pass. `Written` if the rendered bytes differed
+/// from the current on-disk contents and we rewrote; `Unchanged` if
+/// they matched and we left the file alone.
+///
+/// The caller (reconciler / install path) decides whether to restart
+/// the bitcoin-ui container based on this.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum RenderOutcome {
+    Written,
+    Unchanged,
+}
+
+/// Render the bitcoin-ui nginx.conf and atomic-write it to disk if it
+/// differs from what's already there.
+///
+/// Idempotent: safe to call on every reconcile pass. Does a byte
+/// comparison before writing so an unchanged password + template is a
+/// no-op (no inode churn, no container restart cascade).
+///
+/// Errors if the secret file is missing or empty. Upstream callers
+/// treat that as "bitcoin-ui isn't installable yet" rather than fatal
+/// — the RPC password comes into being during bitcoin-core's own
+/// bootstrap, which may not have happened yet on a fresh node.
+pub async fn render(paths: &RenderPaths) -> Result<RenderOutcome> {
+    let password = read_password(&paths.secret_path).await?;
+    let auth_b64 = encode_basic_auth(RPC_USER, &password);
+    let rendered = TEMPLATE.replace(PLACEHOLDER, &auth_b64);
+
+    // Compare against existing. read-to-string fails on ENOENT (first
+    // install) — treat as "different".
+    let existing = fs::read_to_string(&paths.rendered_path).await.ok();
+    if existing.as_deref() == Some(rendered.as_str()) {
+        return Ok(RenderOutcome::Unchanged);
+    }
+
+    // Atomic write: write to sibling tmp + rename. Keeps the bind-
+    // mounted file pointing at a fully-formed config at all times.
+    let parent = paths
+        .rendered_path
+        .parent()
+        .ok_or_else(|| anyhow::anyhow!("rendered_path has no parent directory"))?;
+    fs::create_dir_all(parent)
+        .await
+        .with_context(|| format!("creating {}", parent.display()))?;
+
+    let tmp = unique_tmp_path(&paths.rendered_path);
+    fs::write(&tmp, &rendered)
+        .await
+        .with_context(|| format!("writing tmp {}", tmp.display()))?;
+    fs::rename(&tmp, &paths.rendered_path)
+        .await
+        .with_context(|| {
+            format!(
+                "renaming {} -> {}",
+                tmp.display(),
+                paths.rendered_path.display()
+            )
+        })?;
+
+    tracing::info!(
+        path = %paths.rendered_path.display(),
+        auth_hash = %short_hash(&auth_b64),
+        "bitcoin-ui nginx.conf rendered"
+    );
+
+    Ok(RenderOutcome::Written)
+}
+
+fn unique_tmp_path(dest: &Path) -> PathBuf {
+    static COUNTER: AtomicU64 = AtomicU64::new(0);
+    let n = COUNTER.fetch_add(1, Ordering::Relaxed);
+    let ts = SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .map(|d| d.as_nanos())
+        .unwrap_or(0);
+    dest.with_extension(format!("tmp.{ts}.{n}"))
+}
+
+/// Read the plaintext RPC password from disk. Trims trailing newlines
+/// (common from `echo "$PASS" > file`) but rejects an empty result.
+async fn read_password(path: &Path) -> Result<String> {
+    let raw = fs::read_to_string(path)
+        .await
+        .with_context(|| format!("reading bitcoin RPC password from {}", path.display()))?;
+    let trimmed = raw.trim().to_string();
+    if trimmed.is_empty() {
+        anyhow::bail!(
+            "bitcoin RPC password file {} is empty — bitcoin-core bootstrap hasn't written it yet",
+            path.display()
+        );
+    }
+    Ok(trimmed)
+}
+
+/// `base64("user:password")` — the value nginx puts after `Basic ` in
+/// the upstream `Authorization` header.
+fn encode_basic_auth(user: &str, password: &str) -> String {
+    let raw = format!("{user}:{password}");
+    base64::engine::general_purpose::STANDARD.encode(raw.as_bytes())
+}
+
+/// Short hash of the auth value for logging — we never want the
+/// plaintext or full base64 in logs (it's a credential), but a stable
+/// fingerprint helps correlate rotations.
+fn short_hash(s: &str) -> String {
+    let mut hasher = Sha256::new();
+    hasher.update(s.as_bytes());
+    let digest = hasher.finalize();
+    hex::encode(&digest[..4])
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use tempfile::TempDir;
+
+    fn paths_in(dir: &Path, password: &str) -> RenderPaths {
+        let secret = dir.join("bitcoin-rpc-password");
+        std::fs::write(&secret, password).unwrap();
+        RenderPaths {
+            secret_path: secret,
+            rendered_path: dir.join("nginx.conf"),
+        }
+    }
+
+    #[tokio::test]
+    async fn render_writes_file_with_substitution() {
+        let tmp = TempDir::new().unwrap();
+        let paths = paths_in(tmp.path(), "hunter2");
+        let outcome = render(&paths).await.unwrap();
+        assert_eq!(outcome, RenderOutcome::Written);
+        let contents = std::fs::read_to_string(&paths.rendered_path).unwrap();
+        // archipelago:hunter2 -> "YXJjaGlwZWxhZ286aHVudGVyMg=="
+        assert!(
+            contents.contains("YXJjaGlwZWxhZ286aHVudGVyMg=="),
+            "base64 auth not found in rendered config:\n{contents}"
+        );
+        assert!(
+            !contents.contains(PLACEHOLDER),
+            "placeholder left in output"
+        );
+    }
+
+    #[tokio::test]
+    async fn render_is_idempotent_when_password_unchanged() {
+        let tmp = TempDir::new().unwrap();
+        let paths = paths_in(tmp.path(), "hunter2");
+        let first = render(&paths).await.unwrap();
+        assert_eq!(first, RenderOutcome::Written);
+        let second = render(&paths).await.unwrap();
+        assert_eq!(second, RenderOutcome::Unchanged);
+    }
+
+    #[tokio::test]
+    async fn render_rewrites_on_password_rotation() {
+        let tmp = TempDir::new().unwrap();
+        let paths = paths_in(tmp.path(), "old-pass");
+        render(&paths).await.unwrap();
+        // Rotate.
+        std::fs::write(&paths.secret_path, "new-pass").unwrap();
+        let outcome = render(&paths).await.unwrap();
+        assert_eq!(outcome, RenderOutcome::Written);
+        let contents = std::fs::read_to_string(&paths.rendered_path).unwrap();
+        // archipelago:new-pass -> "YXJjaGlwZWxhZ286bmV3LXBhc3M="
+        assert!(contents.contains("YXJjaGlwZWxhZ286bmV3LXBhc3M="));
+    }
+
+    #[tokio::test]
+    async fn render_trims_trailing_newline_from_secret() {
+        // Matches `echo "$PASS" > file` behaviour.
+        let tmp = TempDir::new().unwrap();
+        let paths = paths_in(tmp.path(), "hunter2\n");
+        render(&paths).await.unwrap();
+        let contents = std::fs::read_to_string(&paths.rendered_path).unwrap();
+        assert!(
+            contents.contains("YXJjaGlwZWxhZ286aHVudGVyMg=="),
+            "trailing newline should be stripped before encoding"
+        );
+    }
+
+    #[tokio::test]
+    async fn render_errors_on_empty_password() {
+        let tmp = TempDir::new().unwrap();
+        let paths = paths_in(tmp.path(), "");
+        let err = render(&paths).await.unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("empty"), "unexpected error: {msg}");
+    }
+
+    #[tokio::test]
+    async fn render_errors_when_secret_missing() {
+        let tmp = TempDir::new().unwrap();
+        let paths = RenderPaths {
+            secret_path: tmp.path().join("does-not-exist"),
+            rendered_path: tmp.path().join("nginx.conf"),
+        };
+        let err = render(&paths).await.unwrap_err();
+        let msg = format!("{err}");
+        assert!(
+            msg.contains("reading bitcoin RPC password"),
+            "unexpected error: {msg}"
+        );
+    }
+
+    #[test]
+    fn template_contains_exactly_one_placeholder() {
+        // Safety net: if someone adds a second placeholder to the
+        // template without updating the renderer, we want a test to
+        // fail loudly rather than ship a half-substituted config.
+        let count = TEMPLATE.matches(PLACEHOLDER).count();
+        assert_eq!(count, 1, "template must contain exactly one {PLACEHOLDER}");
+    }
+
+    #[test]
+    fn template_proxies_bitcoin_rpc_on_8332() {
+        // Lock in the core shape so a bad template edit doesn't ship.
+        assert!(TEMPLATE.contains("proxy_pass http://127.0.0.1:8332/"));
+        assert!(TEMPLATE.contains("location /bitcoin-rpc/"));
+        assert!(TEMPLATE.contains("listen 8334"));
+    }
+}

core/archipelago/src/container/bitcoin_ui_nginx.conf.template

@@ -9,7 +9,7 @@ server {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header Authorization "Basic __BITCOIN_RPC_AUTH__";
+        proxy_set_header Authorization "Basic {{BITCOIN_RPC_AUTH}}";
         add_header Access-Control-Allow-Origin *;
         add_header Access-Control-Allow-Methods "POST, GET, OPTIONS";
         add_header Access-Control-Allow-Headers "Content-Type, Authorization";

core/archipelago/src/container/boot_reconciler.rs

@@ -0,0 +1,331 @@
+//! BootReconciler — the long-running task that keeps the prod orchestrator's
+//! desired-state view in lockstep with what podman actually has.
+//!
+//! Step 5 of the rust-orchestrator migration. Spawned once from `main.rs`
+//! (Step 6) after the initial `adopt_existing()` pass. Every `interval` it
+//! calls `ProdContainerOrchestrator::reconcile_all()`, which ensures every
+//! loaded manifest has a running container, installing fresh ones as needed.
+//!
+//! Per answered design Q3, `interval` defaults to 30 seconds.
+//!
+//! Shutdown is signalled via `Arc<Notify>`. The reconciler finishes its
+//! current `reconcile_all` call before exiting — we don't interrupt an
+//! in-flight pull or build.
+//!
+//! See `docs/rust-orchestrator-migration.md` §269-352.
+
+use std::sync::Arc;
+use std::time::Duration;
+
+use tokio::sync::Notify;
+use tokio::time::{self, Instant};
+
+use crate::container::prod_orchestrator::{ProdContainerOrchestrator, ReconcileReport};
+
+/// Default reconciler cadence (answered design Q3).
+pub const DEFAULT_INTERVAL: Duration = Duration::from_secs(30);
+
+pub struct BootReconciler {
+    orchestrator: Arc<ProdContainerOrchestrator>,
+    interval: Duration,
+    shutdown: Arc<Notify>,
+}
+
+impl BootReconciler {
+    pub fn new(
+        orchestrator: Arc<ProdContainerOrchestrator>,
+        interval: Duration,
+        shutdown: Arc<Notify>,
+    ) -> Self {
+        Self {
+            orchestrator,
+            interval,
+            shutdown,
+        }
+    }
+
+    /// Run the reconcile loop until `shutdown` is notified.
+    ///
+    /// Does one reconcile immediately, then sleeps `interval` between
+    /// subsequent passes. A `shutdown.notify_one()` call unblocks the sleep
+    /// and the task returns after the *next* pass completes.
+    ///
+    /// Never panics: per-app failures are absorbed into `ReconcileReport` by
+    /// the orchestrator, and `reconcile_all` itself returns infallibly.
+    pub async fn run_forever(self) {
+        // Initial pass: no delay.
+        let report = self.orchestrator.reconcile_all().await;
+        Self::log_report(&report);
+
+        loop {
+            let deadline = Instant::now() + self.interval;
+            tokio::select! {
+                _ = time::sleep_until(deadline) => {
+                    let report = self.orchestrator.reconcile_all().await;
+                    Self::log_report(&report);
+                }
+                _ = self.shutdown.notified() => {
+                    tracing::info!("boot reconciler: shutdown requested, exiting loop");
+                    break;
+                }
+            }
+        }
+    }
+
+    fn log_report(report: &ReconcileReport) {
+        for (app_id, action) in &report.actions {
+            tracing::debug!(app_id = %app_id, action = ?action, "reconcile action");
+        }
+        for (app_id, err) in &report.failures {
+            tracing::warn!(app_id = %app_id, error = %err, "reconcile failure");
+        }
+        if report.failures.is_empty() {
+            tracing::debug!(count = report.actions.len(), "reconcile pass complete");
+        } else {
+            tracing::warn!(
+                ok = report.actions.len(),
+                failed = report.failures.len(),
+                "reconcile pass completed with failures"
+            );
+        }
+    }
+}
+
+// ---------------------------------------------------------------------------
+// Tests
+// ---------------------------------------------------------------------------
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::container::prod_orchestrator::ProdContainerOrchestrator;
+    use anyhow::Result;
+    use archipelago_container::{
+        AppManifest, BuildConfig, ContainerRuntime as ContainerRuntimeTrait, ContainerState,
+        ContainerStatus,
+    };
+    use async_trait::async_trait;
+    use std::collections::HashMap;
+    use std::path::PathBuf;
+    use std::sync::Mutex as StdMutex;
+
+    /// Instrumented runtime that counts reconcile-loop side effects so tests
+    /// can tell exactly how many passes have fired. All containers are
+    /// reported as already Running so `reconcile_all` will NoOp — we are only
+    /// measuring loop cadence, not install behavior.
+    #[derive(Default)]
+    struct CountingRuntime {
+        /// Number of times get_container_status has been called. Each
+        /// reconcile_all pass hits this once per manifest, so with one
+        /// manifest this equals the number of reconcile passes.
+        status_calls: StdMutex<u32>,
+        running: StdMutex<HashMap<String, ContainerState>>,
+    }
+
+    impl CountingRuntime {
+        fn new_with(names: &[&str]) -> Self {
+            let me = Self::default();
+            let mut m = me.running.lock().unwrap();
+            for n in names {
+                m.insert((*n).to_string(), ContainerState::Running);
+            }
+            drop(m);
+            me
+        }
+        fn status_call_count(&self) -> u32 {
+            *self.status_calls.lock().unwrap()
+        }
+    }
+
+    #[async_trait]
+    impl ContainerRuntimeTrait for CountingRuntime {
+        async fn pull_image(&self, _: &str, _: Option<&str>) -> Result<()> {
+            Ok(())
+        }
+        async fn create_container(&self, _: &AppManifest, name: &str, _: u16) -> Result<String> {
+            Ok(name.to_string())
+        }
+        async fn start_container(&self, _: &str) -> Result<()> {
+            Ok(())
+        }
+        async fn stop_container(&self, _: &str) -> Result<()> {
+            Ok(())
+        }
+        async fn remove_container(&self, _: &str) -> Result<()> {
+            Ok(())
+        }
+        async fn get_container_status(&self, name: &str) -> Result<ContainerStatus> {
+            *self.status_calls.lock().unwrap() += 1;
+            let state = self
+                .running
+                .lock()
+                .unwrap()
+                .get(name)
+                .cloned()
+                .ok_or_else(|| anyhow::anyhow!("not found: {name}"))?;
+            Ok(ContainerStatus {
+                id: format!("id-{name}"),
+                name: name.to_string(),
+                state,
+                health: None,
+                exit_code: None,
+                started_at: None,
+                image: "test".into(),
+                created: "now".into(),
+                ports: vec![],
+                lan_address: None,
+            })
+        }
+        async fn get_container_logs(&self, _: &str, _: u32) -> Result<Vec<String>> {
+            Ok(vec![])
+        }
+        async fn list_containers(&self) -> Result<Vec<ContainerStatus>> {
+            Ok(vec![])
+        }
+        async fn image_exists(&self, _: &str) -> Result<bool> {
+            Ok(true)
+        }
+        async fn build_image(&self, _: &BuildConfig) -> Result<()> {
+            Ok(())
+        }
+    }
+
+    fn pull_manifest(id: &str, image: &str) -> AppManifest {
+        let yaml = format!(
+            "app:\n  id: {id}\n  name: {id}\n  version: 1.0.0\n  container:\n    image: {image}\n"
+        );
+        AppManifest::parse(&yaml).unwrap()
+    }
+
+    async fn orch_with_one_running_manifest(
+        rt: Arc<CountingRuntime>,
+    ) -> Arc<ProdContainerOrchestrator> {
+        let orch = Arc::new(ProdContainerOrchestrator::with_runtime(
+            rt,
+            PathBuf::from("/nonexistent-for-tests"),
+        ));
+        orch.insert_manifest_for_test(
+            pull_manifest("bitcoin-knots", "docker.io/bitcoin/knots:28"),
+            PathBuf::from("/tmp/bk"),
+        )
+        .await;
+        orch
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn initial_pass_fires_immediately() {
+        let rt = Arc::new(CountingRuntime::new_with(&["bitcoin-knots"]));
+        let orch = orch_with_one_running_manifest(rt.clone()).await;
+        let shutdown = Arc::new(Notify::new());
+        let reconciler =
+            BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
+        let handle = tokio::spawn(reconciler.run_forever());
+
+        // Yield so the spawned task gets CPU to run its initial reconcile.
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+
+        // We expect exactly one reconcile pass to have run by now (the initial),
+        // NOT a second one (the 30s sleep hasn't elapsed in paused time).
+        assert_eq!(rt.status_call_count(), 1, "initial pass should fire once");
+
+        shutdown.notify_one();
+        // Under paused clock the select! is blocked on sleep_until; the notify
+        // will unblock it. Advance wall-clock a hair so the notify gets polled.
+        tokio::task::yield_now().await;
+        let _ = tokio::time::timeout(Duration::from_secs(1), handle).await;
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn second_pass_fires_after_interval() {
+        let rt = Arc::new(CountingRuntime::new_with(&["bitcoin-knots"]));
+        let orch = orch_with_one_running_manifest(rt.clone()).await;
+        let shutdown = Arc::new(Notify::new());
+        let reconciler =
+            BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
+        let handle = tokio::spawn(reconciler.run_forever());
+
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+        assert_eq!(rt.status_call_count(), 1);
+
+        // Fast-forward past one interval; the sleep_until should fire.
+        tokio::time::advance(Duration::from_secs(31)).await;
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+
+        assert_eq!(
+            rt.status_call_count(),
+            2,
+            "a second reconcile pass should fire after one interval"
+        );
+
+        shutdown.notify_one();
+        tokio::task::yield_now().await;
+        let _ = tokio::time::timeout(Duration::from_secs(1), handle).await;
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn shutdown_terminates_loop() {
+        let rt = Arc::new(CountingRuntime::new_with(&["bitcoin-knots"]));
+        let orch = orch_with_one_running_manifest(rt.clone()).await;
+        let shutdown = Arc::new(Notify::new());
+        let reconciler =
+            BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
+        let handle = tokio::spawn(reconciler.run_forever());
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+
+        shutdown.notify_one();
+        // The select! should wake on Notified and return. Use a real timeout
+        // with advancing the paused clock to make sure the task exits.
+        tokio::time::advance(Duration::from_millis(10)).await;
+        let result = tokio::time::timeout(Duration::from_secs(5), handle).await;
+        assert!(result.is_ok(), "reconciler did not exit after shutdown");
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn failure_in_one_pass_does_not_stop_loop() {
+        // Manifest references a container the runtime does not have AND
+        // cannot create (no install path — install_fresh will also fail to
+        // pull, since CountingRuntime::pull_image returns Ok but the
+        // manifest's referenced container stays uncreated). In practice
+        // reconcile_all will observe the missing container, install_fresh
+        // will run, and the next pass will see a new state. We care about
+        // "loop keeps ticking even when the report has actions".
+        let rt = Arc::new(CountingRuntime::default());
+        let orch = Arc::new(ProdContainerOrchestrator::with_runtime(
+            rt.clone(),
+            PathBuf::from("/nonexistent-for-tests"),
+        ));
+        orch.insert_manifest_for_test(
+            pull_manifest("bitcoin-knots", "docker.io/bitcoin/knots:28"),
+            PathBuf::from("/tmp/bk"),
+        )
+        .await;
+        let shutdown = Arc::new(Notify::new());
+        let reconciler =
+            BootReconciler::new(orch.clone(), Duration::from_secs(30), shutdown.clone());
+        let handle = tokio::spawn(reconciler.run_forever());
+
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+        let first = rt.status_call_count();
+        assert!(first >= 1, "initial pass should have touched the runtime");
+
+        // Advance one interval — second pass should fire regardless of what
+        // the first pass did.
+        tokio::time::advance(Duration::from_secs(31)).await;
+        tokio::task::yield_now().await;
+        tokio::task::yield_now().await;
+        let second = rt.status_call_count();
+        assert!(
+            second > first,
+            "loop should have fired a second pass after the interval"
+        );
+
+        shutdown.notify_one();
+        tokio::time::advance(Duration::from_millis(10)).await;
+        let _ = tokio::time::timeout(Duration::from_secs(5), handle).await;
+    }
+}

core/archipelago/src/container/dev_orchestrator.rs

@@ -1,12 +1,15 @@
 use anyhow::{Context, Result};
 use archipelago_container::{
     AppManifest, BitcoinSimulationMode, BitcoinSimulator,
-    ContainerRuntime as ContainerRuntimeTrait, ContainerStatus, PortManager,
+    ContainerRuntime as ContainerRuntimeTrait, ContainerStatus, PortManager, ResolvedSource,
 };
+use async_trait::async_trait;
+use std::path::{Path, PathBuf};
 use std::sync::Arc;
 
 use crate::config::{BitcoinSimulation, Config, ContainerRuntime};
 use crate::container::data_manager::DevDataManager;
+use crate::container::traits::ContainerOrchestrator;
 
 pub struct DevContainerOrchestrator {
     runtime: Arc<dyn ContainerRuntimeTrait>,
@@ -103,14 +106,27 @@ impl DevContainerOrchestrator {
             volume.source = dev_path.to_string_lossy().to_string();
         }
 
-        // Pull image
-        self.runtime
-            .pull_image(
-                &manifest.app.container.image,
-                manifest.app.container.image_signature.as_deref(),
-            )
-            .await
-            .context("Failed to pull image")?;
+        // Resolve pull-or-build. Dev orchestrator currently only supports pull;
+        // Build support lands in Step 2 of the rust-orchestrator migration.
+        match manifest.app.container.resolve().ok_or_else(|| {
+            anyhow::anyhow!("manifest container config invalid (neither image nor build)")
+        })? {
+            ResolvedSource::Pull {
+                image,
+                image_signature,
+                ..
+            } => {
+                self.runtime
+                    .pull_image(&image, image_signature.as_deref())
+                    .await
+                    .context("Failed to pull image")?;
+            }
+            ResolvedSource::Build(_) => {
+                anyhow::bail!(
+                    "dev orchestrator does not yet support local image builds (see rust-orchestrator-migration.md Step 2)"
+                );
+            }
+        }
 
         // Create container with port offset
         let port_offset = if self.config.dev_mode {
@@ -242,4 +258,153 @@ impl DevContainerOrchestrator {
             archipelago_container::ContainerState::Unknown(_) => Ok("unknown".to_string()),
         }
     }
+
+    /// Load a manifest for `app_id` from the dev-mode apps directory.
+    ///
+    /// Used by the trait-level `install(app_id)` entry point.
+    ///
+    /// Search order intentionally mirrors production/operator reality:
+    /// 1) `$ARCHIPELAGO_APPS_DIR` (explicit override)
+    /// 2) `/opt/archipelago/apps` (image-recipe canonical path)
+    /// 3) `/home/archipelago/Projects/archy/apps` (repo-local fallback on dev nodes)
+    /// 4) `<data_dir>/apps` (legacy dev layout)
+    async fn load_manifest_for(&self, app_id: &str) -> Result<AppManifest> {
+        let candidates = candidate_manifest_paths(app_id, &self.config.data_dir);
+        let mut last_err: Option<anyhow::Error> = None;
+
+        for path in candidates {
+            let content = match tokio::fs::read_to_string(&path).await {
+                Ok(c) => c,
+                Err(e) => {
+                    last_err = Some(e.into());
+                    continue;
+                }
+            };
+
+            let manifest: AppManifest = serde_yaml::from_str(&content)
+                .with_context(|| format!("parsing manifest {}", path.display()))?;
+            return Ok(manifest);
+        }
+
+        let msg = format!(
+            "manifest for {} not found in any search path (set ARCHIPELAGO_APPS_DIR or install /opt/archipelago/apps)",
+            app_id
+        );
+        Err(match last_err {
+            Some(e) => e.context(msg),
+            None => anyhow::anyhow!(msg),
+        })
+    }
+}
+
+fn candidate_manifest_paths(app_id: &str, data_dir: &Path) -> Vec<PathBuf> {
+    let mut roots: Vec<PathBuf> = Vec::new();
+
+    if let Ok(v) = std::env::var("ARCHIPELAGO_APPS_DIR") {
+        let v = v.trim();
+        if !v.is_empty() {
+            roots.push(PathBuf::from(v));
+        }
+    }
+
+    roots.push(PathBuf::from("/opt/archipelago/apps"));
+    roots.push(PathBuf::from("/home/archipelago/Projects/archy/apps"));
+    roots.push(data_dir.join("apps"));
+
+    let mut deduped: Vec<PathBuf> = Vec::new();
+    for root in roots {
+        if !deduped.iter().any(|p| p == &root) {
+            deduped.push(root);
+        }
+    }
+
+    deduped
+        .into_iter()
+        .map(|root| root.join(app_id).join("manifest.yml"))
+        .collect()
+}
+
+// ---------------------------------------------------------------------------
+// Trait impl (Step 4): expose the shared ContainerOrchestrator surface.
+// Forwards to the inherent methods, which internally apply the `-dev` suffix
+// and the port offset. The trait keeps the RPC layer mode-agnostic; Dev's
+// install_container (manifest_path-based) stays as an inherent method for the
+// ad-hoc dev-mode RPC and is not exposed on the trait.
+// ---------------------------------------------------------------------------
+
+#[async_trait]
+impl ContainerOrchestrator for DevContainerOrchestrator {
+    async fn install(&self, app_id: &str) -> Result<String> {
+        let manifest = self.load_manifest_for(app_id).await?;
+        let name = self.install_container(&manifest, "").await?;
+        Ok(name)
+    }
+
+    async fn start(&self, app_id: &str) -> Result<()> {
+        self.start_container(app_id).await
+    }
+
+    async fn stop(&self, app_id: &str) -> Result<()> {
+        self.stop_container(app_id).await
+    }
+
+    async fn restart(&self, app_id: &str) -> Result<()> {
+        let _ = self.stop_container(app_id).await;
+        self.start_container(app_id).await
+    }
+
+    async fn remove(&self, app_id: &str, preserve_data: bool) -> Result<()> {
+        self.remove_container(app_id, preserve_data).await
+    }
+
+    async fn upgrade(&self, app_id: &str) -> Result<()> {
+        // Dev upgrade: stop, remove (preserving data), re-install from the loaded manifest.
+        let _ = self.stop_container(app_id).await;
+        let _ = self.remove_container(app_id, true).await;
+        let manifest = self.load_manifest_for(app_id).await?;
+        self.install_container(&manifest, "").await?;
+        self.start_container(app_id).await
+    }
+
+    async fn status(&self, app_id: &str) -> Result<ContainerStatus> {
+        self.get_container_status(app_id).await
+    }
+
+    async fn list(&self) -> Result<Vec<ContainerStatus>> {
+        self.list_containers().await
+    }
+
+    async fn logs(&self, app_id: &str, lines: u32) -> Result<Vec<String>> {
+        self.get_container_logs(app_id, lines).await
+    }
+
+    async fn health(&self, app_id: &str) -> Result<String> {
+        self.get_health_status(app_id).await
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::candidate_manifest_paths;
+    use std::path::PathBuf;
+
+    #[test]
+    fn candidate_manifest_paths_include_expected_fallbacks() {
+        let app_id = "bitcoin-ui";
+        let paths = candidate_manifest_paths(app_id, &PathBuf::from("/var/lib/archipelago"));
+        let as_strings: Vec<String> = paths
+            .iter()
+            .map(|p| p.to_string_lossy().into_owned())
+            .collect();
+
+        assert!(as_strings
+            .iter()
+            .any(|p| p == "/opt/archipelago/apps/bitcoin-ui/manifest.yml"));
+        assert!(as_strings
+            .iter()
+            .any(|p| p == "/home/archipelago/Projects/archy/apps/bitcoin-ui/manifest.yml"));
+        assert!(as_strings
+            .iter()
+            .any(|p| p == "/var/lib/archipelago/apps/bitcoin-ui/manifest.yml"));
+    }
 }

core/archipelago/src/container/filebrowser.rs

@@ -0,0 +1,118 @@
+//! filebrowser config bootstrap helper.
+//!
+//! Mirrors the legacy first-boot behavior that writes
+//! `/var/lib/archipelago/filebrowser-data/.filebrowser.json` before
+//! starting the container with `--config /data/.filebrowser.json`.
+
+use anyhow::{Context, Result};
+use std::path::PathBuf;
+use tokio::fs;
+
+pub const DEFAULT_SRV_ROOT: &str = "/var/lib/archipelago/filebrowser";
+pub const DEFAULT_DATA_DIR: &str = "/var/lib/archipelago/filebrowser-data";
+pub const DEFAULT_CONFIG_PATH: &str = "/var/lib/archipelago/filebrowser-data/.filebrowser.json";
+
+const DEFAULT_CONFIG_JSON: &str =
+    "{\"port\":80,\"baseURL\":\"\",\"address\":\"0.0.0.0\",\"database\":\"/data/filebrowser.db\",\"root\":\"/srv\",\"log\":\"stdout\"}\n";
+
+#[derive(Debug, Clone)]
+pub struct EnsurePaths {
+    pub srv_root: PathBuf,
+    pub data_dir: PathBuf,
+    pub config_path: PathBuf,
+}
+
+impl Default for EnsurePaths {
+    fn default() -> Self {
+        Self {
+            srv_root: PathBuf::from(DEFAULT_SRV_ROOT),
+            data_dir: PathBuf::from(DEFAULT_DATA_DIR),
+            config_path: PathBuf::from(DEFAULT_CONFIG_PATH),
+        }
+    }
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum EnsureOutcome {
+    Written,
+    Unchanged,
+}
+
+pub async fn ensure_config(paths: &EnsurePaths) -> Result<EnsureOutcome> {
+    fs::create_dir_all(&paths.srv_root)
+        .await
+        .with_context(|| format!("creating {}", paths.srv_root.display()))?;
+    fs::create_dir_all(&paths.data_dir)
+        .await
+        .with_context(|| format!("creating {}", paths.data_dir.display()))?;
+
+    for d in ["Documents", "Photos", "Music", "Downloads", "Builds"] {
+        fs::create_dir_all(paths.srv_root.join(d))
+            .await
+            .with_context(|| format!("creating {}/{}", paths.srv_root.display(), d))?;
+    }
+
+    if paths.config_path.exists() {
+        return Ok(EnsureOutcome::Unchanged);
+    }
+
+    let parent = paths
+        .config_path
+        .parent()
+        .ok_or_else(|| anyhow::anyhow!("config_path has no parent directory"))?;
+    fs::create_dir_all(parent)
+        .await
+        .with_context(|| format!("creating {}", parent.display()))?;
+
+    let tmp = paths.config_path.with_extension("tmp");
+    fs::write(&tmp, DEFAULT_CONFIG_JSON)
+        .await
+        .with_context(|| format!("writing tmp {}", tmp.display()))?;
+    fs::rename(&tmp, &paths.config_path)
+        .await
+        .with_context(|| {
+            format!(
+                "renaming {} -> {}",
+                tmp.display(),
+                paths.config_path.display()
+            )
+        })?;
+
+    Ok(EnsureOutcome::Written)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[tokio::test]
+    async fn ensure_config_creates_dirs_and_file() {
+        let tmp = tempfile::TempDir::new().unwrap();
+        let paths = EnsurePaths {
+            srv_root: tmp.path().join("filebrowser"),
+            data_dir: tmp.path().join("filebrowser-data"),
+            config_path: tmp.path().join("filebrowser-data/.filebrowser.json"),
+        };
+
+        let out = ensure_config(&paths).await.unwrap();
+        assert_eq!(out, EnsureOutcome::Written);
+        assert!(paths.config_path.exists());
+        assert!(paths.srv_root.join("Documents").exists());
+        assert!(paths.srv_root.join("Photos").exists());
+    }
+
+    #[tokio::test]
+    async fn ensure_config_is_idempotent() {
+        let tmp = tempfile::TempDir::new().unwrap();
+        let paths = EnsurePaths {
+            srv_root: tmp.path().join("filebrowser"),
+            data_dir: tmp.path().join("filebrowser-data"),
+            config_path: tmp.path().join("filebrowser-data/.filebrowser.json"),
+        };
+
+        let first = ensure_config(&paths).await.unwrap();
+        assert_eq!(first, EnsureOutcome::Written);
+        let second = ensure_config(&paths).await.unwrap();
+        assert_eq!(second, EnsureOutcome::Unchanged);
+    }
+}

core/archipelago/src/container/image_versions.rs

@@ -1,8 +1,8 @@
 //! Parser for image-versions.sh — single source of truth for pinned container images.
 //!
-//! Reads the deployed file at /opt/archipelago/image-versions.sh (or the repo-local
-//! scripts/image-versions.sh as fallback) and exposes lookup functions so the container
-//! scanner can compare running images against pinned targets.
+//! Reads the deployed file at /opt/archipelago/scripts/image-versions.sh (the canonical
+//! location installed by the image-recipe) with fallbacks for older layouts and the
+//! repo-local scripts/image-versions.sh for development runs from the repo root.
 
 use std::collections::HashMap;
 use std::path::Path;
@@ -18,9 +18,15 @@ struct CacheEntry {
     images: HashMap<String, String>,
 }
 
-/// File search order — production path first, then repo-local for dev.
+/// File search order — canonical production path first, older layout second,
+/// repo-local for dev last. The canonical deployed path is
+/// /opt/archipelago/scripts/image-versions.sh; earlier builds put it directly
+/// in /opt/archipelago/, so that path is kept as a fallback for not-yet-updated
+/// nodes. The repo-relative entry matches `cargo run` from the repo root.
 const PATHS: &[&str] = &[
+    "/opt/archipelago/scripts/image-versions.sh",
     "/opt/archipelago/image-versions.sh",
+    "/home/archipelago/Projects/archy/scripts/image-versions.sh",
     "scripts/image-versions.sh",
 ];
 
@@ -102,8 +108,11 @@ fn parse_image_versions(content: &str) -> HashMap<String, String> {
         }
     }
 
-    // Keep only *_IMAGE entries
-    vars.retain(|k, _| k.ends_with("_IMAGE"));
+    // Keep only *_IMAGE entries whose value looks like a container image
+    // reference (contains a `:` tag separator and at least one `/` path
+    // component). Rejects placeholder values like "something" so a
+    // hand-edit typo in image-versions.sh never gets treated as an image.
+    vars.retain(|k, v| k.ends_with("_IMAGE") && v.contains(':') && v.contains('/'));
     vars
 }
 
@@ -134,12 +143,15 @@ fn image_var_for_app(app_id: &str) -> Option<&'static str> {
         "lnd" => Some("LND_IMAGE"),
         "electrumx" => Some("ELECTRUMX_IMAGE"),
         "electrs" | "mempool-electrs" => Some("ELECTRUMX_IMAGE"),
+        "bitcoin-ui" | "archy-bitcoin-ui" => Some("BITCOIN_UI_IMAGE"),
+        "lnd-ui" | "archy-lnd-ui" => Some("LND_UI_IMAGE"),
+        "electrs-ui" | "archy-electrs-ui" => Some("ELECTRS_UI_IMAGE"),
 
         // Mempool stack (primary = web)
-        "mempool" | "mempool-web" => Some("MEMPOOL_WEB_IMAGE"),
+        "mempool" | "mempool-web" | "archy-mempool-web" => Some("MEMPOOL_WEB_IMAGE"),
 
         // BTCPay stack (primary = server)
-        "btcpay" | "btcpay-server" | "btcpayserver" => Some("BTCPAY_IMAGE"),
+        "btcpay" | "btcpay-server" | "btcpayserver" | "archy-btcpay-ui" => Some("BTCPAY_IMAGE"),
 
         // Apps
         "homeassistant" | "home-assistant" => Some("HOMEASSISTANT_IMAGE"),

core/archipelago/src/container/mod.rs

@@ -1,8 +1,19 @@
+pub mod bitcoin_ui;
+pub mod boot_reconciler;
 pub mod data_manager;
 pub mod dev_orchestrator;
 pub mod docker_packages;
+pub mod filebrowser;
 pub mod image_versions;
+pub mod prod_orchestrator;
 pub mod registry;
+pub mod traits;
 
+pub use boot_reconciler::{BootReconciler, DEFAULT_INTERVAL as RECONCILER_DEFAULT_INTERVAL};
 pub use dev_orchestrator::DevContainerOrchestrator;
 pub use docker_packages::DockerPackageScanner;
+pub use prod_orchestrator::{
+    compute_container_name, AdoptionReport, ProdContainerOrchestrator, ReconcileAction,
+    ReconcileReport,
+};
+pub use traits::ContainerOrchestrator;

core/archipelago/src/container/registry.rs

@@ -15,7 +15,7 @@ const REGISTRY_FILE: &str = "config/registries.json";
 /// A single container registry.
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct Registry {
-    /// Registry URL (e.g., "git.tx1138.com/lfg2025" or "23.182.128.160:3000/lfg2025").
+    /// Registry URL (e.g., "git.tx1138.com/lfg2025" or "146.59.87.168:3000/lfg2025").
     pub url: String,
     /// Human-readable name.
     pub name: String,
@@ -44,8 +44,8 @@ impl Default for RegistryConfig {
         Self {
             registries: vec![
                 Registry {
-                    url: "23.182.128.160:3000/lfg2025".to_string(),
-                    name: "Server 1 (VPS)".to_string(),
+                    url: "146.59.87.168:3000/lfg2025".to_string(),
+                    name: "Server 1 (OVH)".to_string(),
                     tls_verify: false,
                     enabled: true,
                     priority: 0,
@@ -57,13 +57,6 @@ impl Default for RegistryConfig {
                     enabled: true,
                     priority: 10,
                 },
-                Registry {
-                    url: "146.59.87.168:3000/lfg2025".to_string(),
-                    name: "Server 3 (OVH)".to_string(),
-                    tls_verify: false,
-                    enabled: true,
-                    priority: 20,
-                },
             ],
         }
     }
@@ -78,15 +71,14 @@ impl RegistryConfig {
     }
 
     /// Rewrite an image reference to use a specific registry.
-    /// E.g., "git.tx1138.com/lfg2025/bitcoin-knots:latest" with registry "23.182.128.160:3000/lfg2025"
-    /// becomes "23.182.128.160:3000/lfg2025/bitcoin-knots:latest".
+    /// E.g., "git.tx1138.com/lfg2025/bitcoin-knots:latest" with registry "146.59.87.168:3000/lfg2025"
+    /// becomes "146.59.87.168:3000/lfg2025/bitcoin-knots:latest".
     pub fn rewrite_image(&self, image: &str, registry: &Registry) -> String {
         // Extract the image name (last component after the org/namespace)
         // Handles: "registry/org/image:tag" -> "image:tag"
         let image_name = extract_image_name(image);
         format!("{}/{}", registry.url, image_name)
     }
-
 }
 
 /// Extract the image name from a full image reference.
@@ -114,6 +106,19 @@ pub async fn load_registries(data_dir: &Path) -> Result<RegistryConfig> {
     let mut config: RegistryConfig =
         serde_json::from_str(&content).unwrap_or_else(|_| RegistryConfig::default());
 
+    // One-time migration: the Hetzner VPS at 23.182.128.160 was
+    // decommissioned 2026-04-23. Existing nodes have it baked into
+    // their saved registry list (was the original Server 1). Strip it
+    // on load so every container pull doesn't pay a connection-refused
+    // timeout against a dead host. Exception to the usual "explicit
+    // removals stick" rule: the user never chose to add this — it
+    // was a default.
+    let before = config.registries.len();
+    config
+        .registries
+        .retain(|r| !r.url.contains("23.182.128.160"));
+    let mut changed = config.registries.len() != before;
+
     // Migrate: any default registry URL that isn't already in the
     // saved list gets appended at the end (so existing priority order
     // is preserved for anything the operator already configured).
@@ -126,16 +131,15 @@ pub async fn load_registries(data_dir: &Path) -> Result<RegistryConfig> {
         .map(|r| r.priority)
         .max()
         .unwrap_or(0);
-    let mut added = false;
     for (i, def) in defaults.registries.iter().enumerate() {
         if !known.contains(&def.url) {
             let mut cloned = def.clone();
             cloned.priority = max_priority.saturating_add(10 + i as u32);
             config.registries.push(cloned);
-            added = true;
+            changed = true;
         }
     }
-    if added {
+    if changed {
         // Persist so the next load doesn't have to re-merge.
         let _ = save_registries(data_dir, &config).await;
     }
@@ -178,12 +182,12 @@ mod tests {
     #[test]
     fn test_rewrite_image() {
         let config = RegistryConfig::default();
-        // Default primary is now VPS (index 0). A tx1138-hardcoded image
-        // rewrites to VPS when asked for the primary mirror.
+        // Default primary is now the OVH VPS (index 0). A tx1138-hardcoded
+        // image rewrites to OVH when asked for the primary mirror.
         let primary = &config.registries[0];
         assert_eq!(
             config.rewrite_image("git.tx1138.com/lfg2025/bitcoin-knots:latest", primary),
-            "23.182.128.160:3000/lfg2025/bitcoin-knots:latest"
+            "146.59.87.168:3000/lfg2025/bitcoin-knots:latest"
         );
     }
 
@@ -191,16 +195,15 @@ mod tests {
     fn test_active_registries_sorted() {
         let config = RegistryConfig::default();
         let active = config.active_registries();
-        assert_eq!(active.len(), 3);
+        assert_eq!(active.len(), 2);
         assert!(active[0].priority <= active[1].priority);
-        assert!(active[1].priority <= active[2].priority);
     }
 
     #[tokio::test]
     async fn test_load_default() {
         let tmp = TempDir::new().unwrap();
         let config = load_registries(tmp.path()).await.unwrap();
-        assert_eq!(config.registries.len(), 3);
+        assert_eq!(config.registries.len(), 2);
     }
 
     #[tokio::test]
@@ -216,6 +219,6 @@ mod tests {
         });
         save_registries(tmp.path(), &config).await.unwrap();
         let loaded = load_registries(tmp.path()).await.unwrap();
-        assert_eq!(loaded.registries.len(), 4);
+        assert_eq!(loaded.registries.len(), 3);
     }
 }

core/archipelago/src/container/traits.rs

@@ -0,0 +1,56 @@
+//! Orchestrator trait — the shared surface the RPC layer talks to.
+//!
+//! Step 4 of the rust-orchestrator migration. Unifies the container lifecycle
+//! surface of `DevContainerOrchestrator` and `ProdContainerOrchestrator` so
+//! `RpcHandler` can hold `Arc<dyn ContainerOrchestrator>` and stop caring
+//! which mode it is in.
+//!
+//! The trait takes `app_id: &str` everywhere (never a manifest path). Dev and
+//! Prod both resolve app_id → manifest internally. The legacy
+//! `container-install { manifest_path }` RPC shape is preserved as a concrete
+//! `install_container_from_path` method on `DevContainerOrchestrator` only,
+//! since that ad-hoc workflow is a dev convenience and has no prod meaning.
+//!
+//! See `docs/rust-orchestrator-migration.md`.
+
+use anyhow::Result;
+use archipelago_container::ContainerStatus;
+use async_trait::async_trait;
+
+/// Lifecycle + query operations every orchestrator exposes to the RPC layer.
+#[async_trait]
+pub trait ContainerOrchestrator: Send + Sync {
+    /// Build-or-pull the image, create the container, and start it. Returns the
+    /// podman container name that was created. Assumes the app_id corresponds
+    /// to a manifest the orchestrator already knows about.
+    async fn install(&self, app_id: &str) -> Result<String>;
+
+    /// Start an already-created container.
+    async fn start(&self, app_id: &str) -> Result<()>;
+
+    /// Stop a running container. No-op on Prod if already stopped.
+    async fn stop(&self, app_id: &str) -> Result<()>;
+
+    /// Stop-then-start. Best-effort: ignores stop failure.
+    async fn restart(&self, app_id: &str) -> Result<()>;
+
+    /// Remove the container. `preserve_data = true` keeps the volumes; `false`
+    /// is honored on a best-effort basis (Dev cleans, Prod leaves the volume
+    /// management to the data layer).
+    async fn remove(&self, app_id: &str, preserve_data: bool) -> Result<()>;
+
+    /// Pull/rebuild the image and recreate the container from scratch.
+    async fn upgrade(&self, app_id: &str) -> Result<()>;
+
+    /// Current state of a single container.
+    async fn status(&self, app_id: &str) -> Result<ContainerStatus>;
+
+    /// All containers this orchestrator knows about.
+    async fn list(&self) -> Result<Vec<ContainerStatus>>;
+
+    /// Tail the container's stdout+stderr.
+    async fn logs(&self, app_id: &str, lines: u32) -> Result<Vec<String>>;
+
+    /// Coarse health summary: "healthy", "unhealthy", "starting", "paused", "unknown".
+    async fn health(&self, app_id: &str) -> Result<String>;
+}

core/archipelago/src/credentials/operations.rs

@@ -129,9 +129,21 @@ pub fn is_revoked(vc: &VerifiableCredential) -> bool {
 mod tests {
     use super::*;
 
+    /// Create a tempdir with a dummy `identity/node_key` so that the
+    /// credential store's encrypt/decrypt path can derive a key.
+    /// Returns the tempdir guard (drop it to clean up).
+    fn test_dir_with_node_key() -> tempfile::TempDir {
+        let dir = tempfile::tempdir().unwrap();
+        let identity_dir = dir.path().join("identity");
+        std::fs::create_dir_all(&identity_dir).unwrap();
+        // 32 bytes of deterministic test material; never a real key.
+        std::fs::write(identity_dir.join("node_key"), [0xAB; 32]).unwrap();
+        dir
+    }
+
     #[tokio::test]
     async fn test_issue_credential_w3c_format() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         let vc = issue_credential(
             dir.path(),
             "did:key:issuer",
@@ -163,7 +175,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_issue_credential_serializes_as_jsonld() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         let vc = issue_credential(
             dir.path(),
             "did:key:issuer",
@@ -185,7 +197,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_save_and_load_roundtrip() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         issue_credential(
             dir.path(),
             "did:key:a",
@@ -205,7 +217,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_issue_credential_sign_fn_failure_propagates() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         let result = issue_credential(
             dir.path(),
             "did:key:issuer",
@@ -257,7 +269,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_revoke_credential() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         let vc = issue_credential(
             dir.path(),
             "did:key:issuer",
@@ -288,7 +300,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_revoke_nonexistent_credential_fails() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         let result = revoke_credential(dir.path(), "urn:uuid:does-not-exist").await;
         assert!(result.is_err());
         assert!(result
@@ -299,7 +311,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_list_credentials_no_filter() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         issue_credential(
             dir.path(),
             "did:key:a",
@@ -329,7 +341,7 @@ mod tests {
 
     #[tokio::test]
     async fn test_list_credentials_filter_by_did() {
-        let dir = tempfile::tempdir().unwrap();
+        let dir = test_dir_with_node_key();
         issue_credential(
             dir.path(),
             "did:key:alice",

core/archipelago/src/data_model.rs

@@ -143,7 +143,11 @@ pub struct PackageDataEntry {
     /// pipeline so the UI can show real progress instead of a generic
     /// "Uninstalling…" spinner. Cleared after the package entry is
     /// removed.
-    #[serde(rename = "uninstall-stage", skip_serializing_if = "Option::is_none", default)]
+    #[serde(
+        rename = "uninstall-stage",
+        skip_serializing_if = "Option::is_none",
+        default
+    )]
     pub uninstall_stage: Option<String>,
     /// Pinned image version from image-versions.sh when it differs from running version
     #[serde(rename = "available-update", skip_serializing_if = "Option::is_none")]
@@ -245,6 +249,38 @@ pub enum ServiceStatus {
 pub struct InstallProgress {
     pub size: u64,
     pub downloaded: u64,
+    /// High-level pipeline phase. Preferred by the UI over the byte
+    /// counters (podman pull doesn't emit parseable progress on a piped
+    /// stderr, so `size`/`downloaded` are often 0). Each phase maps to
+    /// a fixed UI percentage and a descriptive label.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub phase: Option<InstallPhase>,
+}
+
+/// Phases of the install / update pipeline, surfaced to the UI so users
+/// see where the pipeline is rather than a stuck 0% bar.
+///
+/// Ordered so each variant's index roughly corresponds to pipeline time.
+/// Serialized as kebab-case: "preparing", "pulling-image", …
+#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case")]
+pub enum InstallPhase {
+    /// Validating params, checking deps, writing dynamic configs.
+    Preparing,
+    /// `podman pull` in progress (the longest phase — up to several
+    /// minutes for large images on slow networks).
+    PullingImage,
+    /// Creating data directories, writing app-specific configs
+    /// (bitcoin.conf, lnd.conf, searxng settings.yml, chown).
+    CreatingContainer,
+    /// `podman run` has returned; container is transitioning to running.
+    StartingContainer,
+    /// Post-start loop waiting up to 60s for `State.Status == running`.
+    WaitingHealthy,
+    /// Running post-install hooks (chain init, wallet setup, …).
+    PostInstall,
+    /// Pipeline finished successfully. Terminal phase, UI clears entry.
+    Done,
 }
 
 /// WebSocket message sent to clients

core/archipelago/src/electrs_status.rs

@@ -263,7 +263,12 @@ async fn fetch_electrs_sync_status() -> ElectrsSyncStatus {
     let network_height = match bitcoin_network_height().await {
         Ok(h) => h,
         Err(e) => {
-            warn!("ElectrumX status: Bitcoin RPC failed: {}", e);
+            let err_msg = e.to_string();
+            if is_transient_error(&err_msg) {
+                tracing::debug!("ElectrumX status: Bitcoin RPC transient: {}", err_msg);
+            } else {
+                warn!("ElectrumX status: Bitcoin RPC failed: {}", err_msg);
+            }
             return ElectrsSyncStatus {
                 indexed_height: 0,
                 network_height: 0,

core/archipelago/src/federation/storage.rs

@@ -80,8 +80,7 @@ pub async fn record_peer_transport(
     let mut modified = false;
     for node in nodes.iter_mut() {
         let did_match = did.is_some_and(|d| d == node.did);
-        let onion_match = onion_target
-            .is_some_and(|t| node.onion.trim_end_matches(".onion") == t);
+        let onion_match = onion_target.is_some_and(|t| node.onion.trim_end_matches(".onion") == t);
         if did_match || onion_match {
             node.last_transport = Some(transport.to_string());
             node.last_transport_at = Some(now.clone());
@@ -182,9 +181,7 @@ pub async fn update_node_state(data_dir: &Path, did: &str, state: NodeStateSnaps
         // routing over FIPS on the very next sync. Refresh if the peer
         // rotated their FIPS key, too.
         if let Some(ref npub) = state.own_fips_npub {
-            if !npub.is_empty()
-                && node.fips_npub.as_deref().map(str::trim) != Some(npub.trim())
-            {
+            if !npub.is_empty() && node.fips_npub.as_deref().map(str::trim) != Some(npub.trim()) {
                 node.fips_npub = Some(npub.clone());
             }
         }

core/archipelago/src/federation/sync.rs

@@ -8,9 +8,7 @@ use anyhow::{Context, Result};
 use std::path::Path;
 
 use super::storage::update_node_state;
-use super::types::{
-    AppStatus, FederatedNode, FederationPeerHint, NodeStateSnapshot, TrustLevel,
-};
+use super::types::{AppStatus, FederatedNode, FederationPeerHint, NodeStateSnapshot, TrustLevel};
 use crate::fips::dial::PeerRequest;
 
 /// Sync state with a single federated peer. Tries FIPS first; falls back
@@ -68,9 +66,7 @@ pub async fn sync_with_peer(
     // hop. Only runs when the source is Trusted — Observer-level peers
     // don't get to expand our federation on their own authority.
     if peer.trust_level == TrustLevel::Trusted {
-        if let Err(e) =
-            merge_transitive_peers(data_dir, &peer.did, &state.federated_peers).await
-        {
+        if let Err(e) = merge_transitive_peers(data_dir, &peer.did, &state.federated_peers).await {
             tracing::warn!(
                 peer_did = %peer.did,
                 error = %e,
@@ -87,10 +83,7 @@ pub async fn sync_with_peer(
 /// call sync_with_peer. Used by transitive-discovery code paths where
 /// the caller only knows the peer's DID (e.g. the peer-joined RPC's
 /// follow-up task).
-pub async fn sync_with_peer_by_did(
-    data_dir: &Path,
-    peer_did: &str,
-) -> Result<NodeStateSnapshot> {
+pub async fn sync_with_peer_by_did(data_dir: &Path, peer_did: &str) -> Result<NodeStateSnapshot> {
     let nodes = super::storage::load_nodes(data_dir).await?;
     let peer = nodes
         .into_iter()
@@ -98,8 +91,7 @@ pub async fn sync_with_peer_by_did(
         .ok_or_else(|| anyhow::anyhow!("Unknown federation peer: {}", peer_did))?;
 
     let identity_dir = data_dir.join("identity");
-    let node_identity =
-        crate::identity::NodeIdentity::load_or_create(&identity_dir).await?;
+    let node_identity = crate::identity::NodeIdentity::load_or_create(&identity_dir).await?;
     let local_pubkey_hex = node_identity.pubkey_hex();
     let local_did = crate::identity::did_key_from_pubkey_hex(&local_pubkey_hex)?;
 
@@ -258,7 +250,11 @@ pub async fn deploy_to_peer(
         .context("Failed to reach federated peer for deploy")?;
 
     if !resp.status().is_success() {
-        anyhow::bail!("Remote node returned HTTP {} (via {})", resp.status(), transport);
+        anyhow::bail!(
+            "Remote node returned HTTP {} (via {})",
+            resp.status(),
+            transport
+        );
     }
 
     let result: serde_json::Value = resp.json().await.context("Invalid response from peer")?;
@@ -355,10 +351,7 @@ mod tests {
                 last_transport_at: None,
             },
         ];
-        let state = build_local_state(
-            vec![],
-            0.0, 0, 0, 0, 0, 0, true, None, None, None, &peers,
-        );
+        let state = build_local_state(vec![], 0.0, 0, 0, 0, 0, 0, true, None, None, None, &peers);
         assert_eq!(state.federated_peers.len(), 1);
         assert_eq!(state.federated_peers[0].did, "did:key:zTrusted");
         assert_eq!(

core/archipelago/src/fips/anchors.rs

@@ -70,8 +70,8 @@ pub async fn load(data_dir: &Path) -> Result<Vec<SeedAnchor>> {
     let bytes = tokio::fs::read(&path)
         .await
         .with_context(|| format!("read {}", path.display()))?;
-    let anchors: Vec<SeedAnchor> = serde_json::from_slice(&bytes)
-        .with_context(|| format!("parse {}", path.display()))?;
+    let anchors: Vec<SeedAnchor> =
+        serde_json::from_slice(&bytes).with_context(|| format!("parse {}", path.display()))?;
     Ok(anchors)
 }
 
@@ -125,12 +125,7 @@ pub async fn apply(anchors: &[SeedAnchor]) -> Vec<ApplyResult> {
     let mut results = Vec::with_capacity(anchors.len());
     for anchor in anchors {
         let out = Command::new("fipsctl")
-            .args([
-                "connect",
-                &anchor.npub,
-                &anchor.address,
-                &anchor.transport,
-            ])
+            .args(["connect", &anchor.npub, &anchor.address, &anchor.transport])
             .output()
             .await;
         let result = match out {

core/archipelago/src/fips/config.rs

@@ -13,7 +13,9 @@ use anyhow::{Context, Result};
 use std::path::Path;
 use tokio::process::Command;
 
-use super::{DAEMON_CONFIG_PATH, DAEMON_KEY_PATH, DAEMON_PUB_PATH, DEFAULT_TCP_PORT, DEFAULT_UDP_PORT};
+use super::{
+    DAEMON_CONFIG_PATH, DAEMON_KEY_PATH, DAEMON_PUB_PATH, DEFAULT_TCP_PORT, DEFAULT_UDP_PORT,
+};
 
 /// Write the FIPS daemon config based on the local npub and default
 /// transports. Overwrites any existing file — callers are expected to

core/archipelago/src/fips/dial.rs

@@ -109,7 +109,7 @@ fn encode_query(id: u16, npub: &str) -> Result<Vec<u8>> {
     encode_label(&mut out, npub)?;
     encode_label(&mut out, FIPS_DNS_SUFFIX)?;
     out.push(0); // root
-    // QTYPE + QCLASS
+                 // QTYPE + QCLASS
     out.extend_from_slice(&QTYPE_AAAA.to_be_bytes());
     out.extend_from_slice(&QCLASS_IN.to_be_bytes());
     Ok(out)
@@ -247,11 +247,7 @@ pub struct PeerRequest<'a> {
 }
 
 impl<'a> PeerRequest<'a> {
-    pub fn new(
-        fips_npub: Option<&'a str>,
-        onion_host: &'a str,
-        path: &'a str,
-    ) -> Self {
+    pub fn new(fips_npub: Option<&'a str>, onion_host: &'a str, path: &'a str) -> Self {
         Self {
             fips_npub,
             onion_host,
@@ -312,9 +308,7 @@ impl<'a> PeerRequest<'a> {
     }
 
     /// GET with optional header-based auth.
-    pub async fn send_get(
-        &self,
-    ) -> Result<(reqwest::Response, crate::transport::TransportKind)> {
+    pub async fn send_get(&self) -> Result<(reqwest::Response, crate::transport::TransportKind)> {
         use crate::settings::transport::TransportPref;
         let pref = self.preference().await;
         if matches!(pref, TransportPref::Auto | TransportPref::Fips) {
@@ -392,19 +386,14 @@ impl<'a> PeerRequest<'a> {
         }
     }
 
-    async fn send_tor_post_json<B: serde::Serialize>(
-        &self,
-        body: &B,
-    ) -> Result<reqwest::Response> {
+    async fn send_tor_post_json<B: serde::Serialize>(&self, body: &B) -> Result<reqwest::Response> {
         let url = self.tor_url();
         let client = self.tor_client()?;
         let mut rb = client.post(&url).json(body);
         for (k, v) in &self.headers {
             rb = rb.header(*k, v);
         }
-        rb.send()
-            .await
-            .with_context(|| format!("Tor POST {}", url))
+        rb.send().await.with_context(|| format!("Tor POST {}", url))
     }
 
     async fn send_tor_get(&self) -> Result<reqwest::Response> {
@@ -414,9 +403,7 @@ impl<'a> PeerRequest<'a> {
         for (k, v) in &self.headers {
             rb = rb.header(*k, v);
         }
-        rb.send()
-            .await
-            .with_context(|| format!("Tor GET {}", url))
+        rb.send().await.with_context(|| format!("Tor GET {}", url))
     }
 
     fn tor_url(&self) -> String {
@@ -449,7 +436,7 @@ mod tests {
         assert_eq!(&q[0..2], &[0x12, 0x34]);
         assert_eq!(&q[2..4], &[0x01, 0x00]); // flags RD=1
         assert_eq!(&q[4..6], &[0x00, 0x01]); // QDCOUNT=1
-        // Tail: QTYPE=28, QCLASS=1
+                                             // Tail: QTYPE=28, QCLASS=1
         assert_eq!(&q[q.len() - 4..], &[0x00, 0x1C, 0x00, 0x01]);
     }
 
@@ -471,7 +458,7 @@ mod tests {
         r.extend_from_slice(&1u16.to_be_bytes()); // ANCOUNT
         r.extend_from_slice(&0u16.to_be_bytes()); // NSCOUNT
         r.extend_from_slice(&0u16.to_be_bytes()); // ARCOUNT
-        // Question: 1 label "a" + "fips"
+                                                  // Question: 1 label "a" + "fips"
         r.extend_from_slice(b"\x01a\x04fips\x00");
         r.extend_from_slice(&QTYPE_AAAA.to_be_bytes());
         r.extend_from_slice(&QCLASS_IN.to_be_bytes());

core/archipelago/src/fips/iface.rs

@@ -24,9 +24,7 @@ pub const FIPS_IFACE: &str = "fips0";
 /// - Link-local (`fe80::/10`) and non-ULA addresses are ignored — we
 ///   only want the mesh-routable ULA that `<npub>.fips` DNS resolves to.
 pub fn fips0_ula() -> Option<Ipv6Addr> {
-    addresses_on(FIPS_IFACE)
-        .into_iter()
-        .find(|a| is_ula(a))
+    addresses_on(FIPS_IFACE).into_iter().find(|a| is_ula(a))
 }
 
 /// List every IPv6 address bound to a given interface from

core/archipelago/src/fips/mod.rs

@@ -122,8 +122,7 @@ impl FipsStatus {
         };
         let service_state = service::unit_state(SERVICE_UNIT).await;
         let upstream_service_state = service::unit_state(UPSTREAM_SERVICE_UNIT).await;
-        let service_active =
-            service_state == "active" || upstream_service_state == "active";
+        let service_active = service_state == "active" || upstream_service_state == "active";
         let key_present = crate::identity::fips_key_exists(&identity_dir);
 
         // Prefer the seed-derived npub; otherwise read the daemon's own
@@ -180,7 +179,10 @@ mod tests {
         // anchor is the only candidate.
         let status = FipsStatus::query(dir.path()).await;
         assert!(!status.key_present, "no key before onboarding");
-        assert!(status.npub.is_none());
+        // `npub` falls back to whatever an already-running local fips
+        // daemon advertises, so on a dev machine or node with fips
+        // installed this field can be Some(...) even when the test
+        // data_dir is empty. We only assert that key_present is false.
         // `installed`, `service_state`, `version` depend on the host and are
         // not asserted here — query() must return cleanly regardless.
     }

core/archipelago/src/fips/service.rs

@@ -150,11 +150,10 @@ pub async fn peer_connectivity_summary(anchor_candidates: &[String]) -> (u32, bo
         Ok(o) if o.status.success() => o.stdout,
         _ => return (0, false),
     };
-    let parsed: serde_json::Value =
-        match serde_json::from_slice(&peers_json) {
-            Ok(v) => v,
-            Err(_) => return (0, false),
-        };
+    let parsed: serde_json::Value = match serde_json::from_slice(&peers_json) {
+        Ok(v) => v,
+        Err(_) => return (0, false),
+    };
     let peers = parsed
         .get("peers")
         .and_then(|p| p.as_array())

core/archipelago/src/identity_manager.rs

@@ -111,11 +111,7 @@ pub struct ProfilePublishOutcome {
 /// (trailing slash, case). nostr-sdk canonicalises URLs internally and
 /// we compare on the surface strings, so be liberal about what matches.
 fn relay_url_matches(a: &str, b: &str) -> bool {
-    let norm = |s: &str| {
-        s.trim_end_matches('/')
-            .trim()
-            .to_ascii_lowercase()
-    };
+    let norm = |s: &str| s.trim_end_matches('/').trim().to_ascii_lowercase();
     norm(a) == norm(b)
 }
 
@@ -262,8 +258,8 @@ impl IdentityManager {
             derivation_index: Some(0),
         };
 
-        let json = serde_json::to_string_pretty(&identity_file)
-            .context("Failed to serialize identity")?;
+        let json =
+            serde_json::to_string_pretty(&identity_file).context("Failed to serialize identity")?;
         fs::write(&file_path, json.as_bytes())
             .await
             .context("Failed to write identity file")?;
@@ -701,11 +697,8 @@ impl IdentityManager {
         let event_id = output.id().to_hex();
         // `Output` has `success: HashSet<RelayUrl>` + `failed: HashMap<RelayUrl, String>`.
         // Normalise to string comparisons (RelayUrl trims trailing slashes etc.).
-        let success_strs: std::collections::HashSet<String> = output
-            .success
-            .iter()
-            .map(|u| u.to_string())
-            .collect();
+        let success_strs: std::collections::HashSet<String> =
+            output.success.iter().map(|u| u.to_string()).collect();
         let failed_strs: std::collections::HashMap<String, String> = output
             .failed
             .iter()
@@ -714,14 +707,11 @@ impl IdentityManager {
         let mut accepted: Vec<String> = Vec::new();
         let mut rejected: Vec<(String, String)> = Vec::new();
         for url in relay_urls {
-            let match_url = success_strs
-                .iter()
-                .any(|s| relay_url_matches(s, url));
+            let match_url = success_strs.iter().any(|s| relay_url_matches(s, url));
             if match_url {
                 accepted.push(url.clone());
-            } else if let Some((_, reason)) = failed_strs
-                .iter()
-                .find(|(s, _)| relay_url_matches(s, url))
+            } else if let Some((_, reason)) =
+                failed_strs.iter().find(|(s, _)| relay_url_matches(s, url))
             {
                 rejected.push((url.clone(), reason.clone()));
             } else {
@@ -885,11 +875,13 @@ mod tests {
     async fn test_create_nostr_key_npub_format() {
         let dir = tempdir().unwrap();
         let mgr = IdentityManager::new(dir.path()).await.unwrap();
+        // `create()` auto-provisions a Nostr key for every identity, so the
+        // returned record should already have a valid bech32 npub.
         let record = mgr
-            .create("Nostr".to_string(), IdentityPurpose::Personal)
+            .create("Personal".to_string(), IdentityPurpose::Personal)
             .await
             .unwrap();
-        let npub = mgr.create_nostr_key(&record.id).await.unwrap();
+        let npub = record.nostr_npub.expect("nostr npub should be populated");
         assert!(
             npub.starts_with("npub1"),
             "npub should start with npub1, got {}",

core/archipelago/src/main.rs

@@ -19,7 +19,9 @@
 
 use anyhow::{Context, Result};
 use std::net::SocketAddr;
+use std::sync::Arc;
 use tokio::signal;
+use tokio::sync::Notify;
 use tracing::info;
 
 mod api;
@@ -69,6 +71,10 @@ mod wallet;
 mod webhooks;
 
 use config::Config;
+use container::{
+    BootReconciler, ContainerOrchestrator, DevContainerOrchestrator, ProdContainerOrchestrator,
+    RECONCILER_DEFAULT_INTERVAL,
+};
 use server::Server;
 
 #[tokio::main]
@@ -98,10 +104,7 @@ async fn main() -> Result<()> {
         if let Ok(meta) = tokio::fs::metadata(web_ui).await {
             let mode = meta.permissions().mode() & 0o777;
             if mode & 0o005 != 0o005 {
-                tracing::warn!(
-                    "web-ui perms {:o} not world-readable — self-healing",
-                    mode
-                );
+                tracing::warn!("web-ui perms {:o} not world-readable — self-healing", mode);
                 let _ = tokio::process::Command::new("sudo")
                     .args([
                         "-n",
@@ -168,15 +171,65 @@ async fn main() -> Result<()> {
 
             // Signal to health monitor that boot recovery is done
             crash_recovery::mark_recovery_complete();
-
-            // Boot reconciliation disabled — the reconciler creates ALL containers
-            // from specs, which is wrong on unbundled installs where only user-chosen
-            // apps should exist. The health monitor handles restarting existing
-            // containers. Run reconcile-containers.sh manually when needed.
-            // crash_recovery::run_boot_reconciliation().await;
         });
     }
 
+    // Construct the container orchestrator once. In prod mode we load the
+    // on-disk app manifests, do an initial adoption pass, and spawn the
+    // BootReconciler loop (Step 5/6 of the rust-orchestrator migration).
+    // Dev mode uses the in-memory DevContainerOrchestrator and has no
+    // reconciler (manifests are pushed via RPC, not discovered from disk).
+    let shutdown_notify = Arc::new(Notify::new());
+    let (orchestrator, dev_orchestrator): (
+        Option<Arc<dyn ContainerOrchestrator>>,
+        Option<Arc<DevContainerOrchestrator>>,
+    ) = if config.dev_mode {
+        let dev = Arc::new(DevContainerOrchestrator::new(config.clone()).await?);
+        let trait_obj: Arc<dyn ContainerOrchestrator> = dev.clone();
+        (Some(trait_obj), Some(dev))
+    } else {
+        let prod = Arc::new(ProdContainerOrchestrator::new(config.clone()).await?);
+        // Best-effort manifest load; a missing /opt/archipelago/apps is
+        // logged inside load_manifests and not fatal.
+        match prod.load_manifests().await {
+            Ok(n) => info!("📦 Loaded {n} app manifest(s) from disk"),
+            Err(e) => {
+                tracing::error!(error = %e, "prod orchestrator: load_manifests failed at startup");
+            }
+        }
+        // Adoption pass: link existing podman containers back to their
+        // manifests so the reconciler doesn't recreate them.
+        match prod.adopt_existing().await {
+            Ok(report) => {
+                info!(
+                    "🔗 Adopted {} existing container(s): {:?}",
+                    report.adopted.len(),
+                    report.adopted
+                );
+            }
+            Err(e) => {
+                tracing::warn!(error = %e, "prod orchestrator: adopt_existing failed (non-fatal)");
+            }
+        }
+        // Spawn the boot reconciler loop. Runs an initial reconcile
+        // immediately, then re-checks every RECONCILER_DEFAULT_INTERVAL
+        // until shutdown_notify fires.
+        {
+            let reconciler = BootReconciler::new(
+                prod.clone(),
+                RECONCILER_DEFAULT_INTERVAL,
+                shutdown_notify.clone(),
+            );
+            tokio::spawn(reconciler.run_forever());
+            info!(
+                "🔄 Boot reconciler started (interval: {:?})",
+                RECONCILER_DEFAULT_INTERVAL
+            );
+        }
+        let trait_obj: Arc<dyn ContainerOrchestrator> = prod;
+        (Some(trait_obj), None)
+    };
+
     // Ensure a default user exists so login works after install/onboarding.
     // In production, the default password is "password123" (shown during install).
     // In dev mode, the dev default password is used.
@@ -185,7 +238,7 @@ async fn main() -> Result<()> {
     // "Create Password" form instead of login form.
 
     // Create server
-    let server = Server::new(config.clone()).await?;
+    let server = Server::new(config.clone(), orchestrator, dev_orchestrator).await?;
 
     // Start server
     let addr: SocketAddr = format!("{}:{}", config.bind_host, config.bind_port)
@@ -261,6 +314,7 @@ async fn main() -> Result<()> {
     // Graceful shutdown: wait for SIGTERM or SIGINT
     let mut sigterm = signal::unix::signal(signal::unix::SignalKind::terminate())
         .context("Failed to register SIGTERM handler")?;
+    let shutdown_notify_for_signal = shutdown_notify.clone();
     let shutdown = async move {
         tokio::select! {
             _ = signal::ctrl_c() => {
@@ -270,6 +324,10 @@ async fn main() -> Result<()> {
                 info!("Received SIGTERM, initiating graceful shutdown...");
             }
         }
+        // Signal the boot reconciler (and any other subscribers) to stop.
+        // `notify_one` stores a permit if no task is currently parked on
+        // `notified()`, so we don't race the reconciler's reconcile_all pass.
+        shutdown_notify_for_signal.notify_one();
     };
 
     server.serve_with_shutdown(addr, shutdown).await?;

core/archipelago/src/mesh/bitcoin_relay.rs

@@ -457,8 +457,9 @@ mod tests {
         let key = SigningKey::generate(&mut OsRng);
         let wire = build_block_header_announcement(
             890412,
-            "0000000000000000000abc",
-            "0000000000000000000aab",
+            // Block hashes must be 32 bytes (64 hex chars). Use realistic-shaped placeholders.
+            "0000000000000000000abc00000000000000000000000000000000000000abcd",
+            "0000000000000000000aab0000000000000000000000000000000000000aabcd",
             1710633600,
             "did:key:z6MkTest",
             &key,
@@ -469,7 +470,9 @@ mod tests {
         assert_eq!(wire[0], 0x02);
         let envelope = TypedEnvelope::from_wire(&wire).unwrap();
         assert_eq!(envelope.t, MeshMessageType::BlockHeader as u8);
-        assert!(envelope.sig.is_some());
+        // Block header announcements are intentionally unsigned to save 64 bytes
+        // on the 160-byte LoRa payload (see builder comment).
+        assert!(envelope.sig.is_none());
     }
 
     #[test]

core/archipelago/src/mesh/outbox.rs

@@ -52,7 +52,10 @@ impl PendingMessage {
             return true; // Can't parse = treat as expired
         };
         let age = chrono::Utc::now().signed_duration_since(created);
-        age.num_seconds() as u64 > self.ttl_secs
+        // Use `>=` so a ttl_secs=0 message is expired immediately (used by
+        // tests and by callers that want a fire-and-forget behavior when
+        // the relay can't deliver on first try).
+        age.num_seconds() as u64 >= self.ttl_secs
     }
 
     /// Check if this message can be relayed further.

core/archipelago/src/mesh/protocol.rs

@@ -701,9 +701,11 @@ mod tests {
 
     #[test]
     fn test_build_app_start() -> Result<()> {
+        // Frame layout: [0: '>'][1-2: len LE][3: CMD][4: VERSION][5..: padded name]
         let frame = build_app_start("Archipelago");
         assert_eq!(frame[3], CMD_APP_START);
-        let name = &frame[4..];
+        assert_eq!(frame[4], PROTOCOL_VERSION);
+        let name = &frame[5..];
         assert_eq!(
             std::str::from_utf8(name)
                 .map_err(|e| anyhow::anyhow!("invalid UTF-8 in app name: {}", e))?,
@@ -753,15 +755,20 @@ mod tests {
 
     #[test]
     fn test_identity_broadcast_roundtrip() -> Result<()> {
-        let did = "did:key:z6MkhaXgBZDvotDkL5257faiztiGiC2QtKLGpbnnEGta2doK";
+        // The v2 encoding drops the DID and the decoder reconstructs it
+        // deterministically from the ed25519 pubkey, so the roundtripped
+        // DID won't equal an arbitrary input DID. Derive what the decoder
+        // will produce and assert against that.
         let ed_pub = "a".repeat(64);
         let x25519_pub = "b".repeat(64);
+        let expected_did = crate::identity::did_key_from_pubkey_hex(&ed_pub)
+            .map_err(|e| anyhow::anyhow!("derive did: {}", e))?;
 
-        let encoded = encode_identity_broadcast(did, &ed_pub, &x25519_pub);
+        let encoded = encode_identity_broadcast(&expected_did, &ed_pub, &x25519_pub);
 
         let (parsed_did, parsed_ed, parsed_x) = parse_identity_broadcast(&encoded)
             .ok_or_else(|| anyhow::anyhow!("failed to parse identity broadcast"))?;
-        assert_eq!(parsed_did, did);
+        assert_eq!(parsed_did, expected_did);
         assert_eq!(parsed_ed, ed_pub);
         assert_eq!(parsed_x, x25519_pub);
         Ok(())

core/archipelago/src/network/dwn_sync.rs

@@ -267,7 +267,7 @@ async fn sync_single_peer(
 
         // Best-effort push — don't fail the whole sync if a batch fails.
         match PeerRequest::new(fips_npub, onion, "/dwn")
-        .service(crate::settings::transport::PeerService::Federation)
+            .service(crate::settings::transport::PeerService::Federation)
             .timeout(std::time::Duration::from_secs(30))
             .send_json(&push_body)
             .await

core/archipelago/src/node_message.rs

@@ -275,25 +275,24 @@ pub async fn send_to_peer(
         body["from_name"] = serde_json::Value::String(name.to_string());
     }
 
-    let (resp, transport) = crate::fips::dial::PeerRequest::new(
-        fips_npub,
-        onion,
-        "/archipelago/node-message",
-    )
-    .service(crate::settings::transport::PeerService::Messaging)
-    .timeout(std::time::Duration::from_secs(60))
-    .send_json(&body)
-    .await
-    .map_err(|e| {
-        let msg = e.to_string();
-        if msg.contains("connection refused") || msg.contains("Connection refused") {
-            anyhow::anyhow!("Peer unreachable. Check Tor (127.0.0.1:9050) and FIPS daemon status.")
-        } else if msg.contains("timeout") || msg.contains("timed out") {
-            anyhow::anyhow!("Connection timed out. The peer may be offline.")
-        } else {
-            anyhow::anyhow!("Failed to send: {}", msg)
-        }
-    })?;
+    let (resp, transport) =
+        crate::fips::dial::PeerRequest::new(fips_npub, onion, "/archipelago/node-message")
+            .service(crate::settings::transport::PeerService::Messaging)
+            .timeout(std::time::Duration::from_secs(60))
+            .send_json(&body)
+            .await
+            .map_err(|e| {
+                let msg = e.to_string();
+                if msg.contains("connection refused") || msg.contains("Connection refused") {
+                    anyhow::anyhow!(
+                        "Peer unreachable. Check Tor (127.0.0.1:9050) and FIPS daemon status."
+                    )
+                } else if msg.contains("timeout") || msg.contains("timed out") {
+                    anyhow::anyhow!("Connection timed out. The peer may be offline.")
+                } else {
+                    anyhow::anyhow!("Failed to send: {}", msg)
+                }
+            })?;
 
     if !resp.status().is_success() {
         anyhow::bail!(

core/archipelago/src/port_allocator.rs

@@ -24,7 +24,7 @@ const RESERVED_PORTS: &[u16] = &[
     9980, 9001, // OnlyOffice, Penpot
     8240, // Tailscale
     9000, // Portainer
-    3001, // Uptime Kuma
+    3001, 3002, // Gitea, Uptime Kuma
     8888, // SearXNG
     8096, 2342, 2283, // Jellyfin, Photoprism, Immich
     8443, 8084, // NPM

core/archipelago/src/server.rs

@@ -1,6 +1,8 @@
 use crate::api::ApiHandler;
 use crate::config::{Config, ContainerRuntime};
-use crate::container::{docker_packages, DockerPackageScanner};
+use crate::container::{
+    docker_packages, ContainerOrchestrator, DevContainerOrchestrator, DockerPackageScanner,
+};
 use crate::identity::{self, NodeIdentity};
 use crate::monitoring::MetricsStore;
 use crate::node_message;
@@ -14,7 +16,7 @@ use hyper::service::service_fn;
 use std::collections::HashMap;
 use std::net::SocketAddr;
 use std::sync::Arc;
-use std::time::Duration;
+use std::time::{Duration, Instant};
 use tokio::net::TcpListener;
 use tracing::{debug, error, info, warn};
 
@@ -26,7 +28,11 @@ pub struct Server {
 }
 
 impl Server {
-    pub async fn new(config: Config) -> Result<Self> {
+    pub async fn new(
+        config: Config,
+        orchestrator: Option<Arc<dyn ContainerOrchestrator>>,
+        dev_orchestrator: Option<Arc<DevContainerOrchestrator>>,
+    ) -> Result<Self> {
         let state_manager = Arc::new(StateManager::new());
 
         // Load node identity and set stable server_info.
@@ -172,8 +178,16 @@ impl Server {
             Some(config.data_dir.clone()),
         );
 
-        let api_handler =
-            Arc::new(ApiHandler::new(config.clone(), state_manager.clone(), metrics_store).await?);
+        let api_handler = Arc::new(
+            ApiHandler::new(
+                config.clone(),
+                state_manager.clone(),
+                metrics_store,
+                orchestrator,
+                dev_orchestrator,
+            )
+            .await?,
+        );
 
         // Initialize mesh networking service (if config has enabled: true)
         {
@@ -299,6 +313,8 @@ impl Server {
             let scanner = create_docker_scanner(&config).await?;
             let state = state_manager.clone();
             let identity_clone = identity.clone();
+            let scan_kick = api_handler.rpc_handler().scan_kick();
+            let scan_tick = api_handler.rpc_handler().scan_tick();
 
             // Initial scan (delayed to let crash recovery finish first)
             tokio::spawn(async move {
@@ -308,18 +324,31 @@ impl Server {
                 // Tracks how many consecutive scans each container has been absent from.
                 // Prevents UI flapping when podman intermittently returns incomplete results.
                 let mut absence_tracker: HashMap<String, u32> = HashMap::new();
+                // Tracks when each container first entered a transitional state
+                // (Stopping / Starting / Restarting / ...). Used by the merge
+                // loop below to ignore podman's live state during a pending
+                // lifecycle op, and to break out if the spawned task dies
+                // without ever writing a final state.
+                let mut transitional_since: HashMap<String, Instant> = HashMap::new();
                 if let Err(e) = scan_and_update_packages(
                     &scanner,
                     &state,
                     identity_clone.as_ref(),
                     &mut absence_tracker,
+                    &mut transitional_since,
                 )
                 .await
                 {
                     error!("Failed to scan containers: {}", e);
                 }
+                // Bump the scan-completion counter so any caller waiting on a
+                // kicked scan (install/update success path) can proceed.
+                scan_tick.send_modify(|n| *n = n.wrapping_add(1));
 
-                // Periodic scan every 60 seconds (only broadcasts if state changed)
+                // Periodic scan every 60 seconds (only broadcasts if state changed).
+                // Also wakes immediately when `scan_kick` fires — install/update
+                // success paths poke it so the fresh manifest (with populated
+                // interfaces) lands before they flip state to Running.
                 // Uses an in-flight guard to skip scans when a previous one is still running
                 let mut interval = tokio::time::interval(Duration::from_secs(60));
                 // Skip missed ticks instead of catching up — prevents burst of scans
@@ -327,7 +356,12 @@ impl Server {
                 interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
                 let scanning = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
                 loop {
-                    interval.tick().await;
+                    tokio::select! {
+                        _ = interval.tick() => {}
+                        _ = scan_kick.notified() => {
+                            debug!("Scan kicked by install/update success — running immediately");
+                        }
+                    }
                     if scanning.load(std::sync::atomic::Ordering::Relaxed) {
                         debug!("Skipping container scan — previous scan still in progress");
                         continue;
@@ -338,11 +372,13 @@ impl Server {
                         &state,
                         identity_clone.as_ref(),
                         &mut absence_tracker,
+                        &mut transitional_since,
                     )
                     .await
                     {
                         error!("Failed to update containers: {}", e);
                     }
+                    scan_tick.send_modify(|n| *n = n.wrapping_add(1));
                     scanning.store(false, std::sync::atomic::Ordering::Relaxed);
                 }
             });
@@ -382,10 +418,9 @@ impl Server {
                             let _ = crate::fips::anchors::apply(&list).await;
                         }
                         Ok(_) => { /* no seed anchors configured yet */ }
-                        Err(e) => tracing::debug!(
-                            "Seed-anchor apply: load failed (non-fatal): {}",
-                            e
-                        ),
+                        Err(e) => {
+                            tracing::debug!("Seed-anchor apply: load failed (non-fatal): {}", e)
+                        }
                     }
                 }
             });
@@ -527,9 +562,7 @@ impl Server {
                 // OTA'd nodes would be stuck on the old UDP-only config
                 // until someone manually clicked Reconnect.
                 let expected = crate::fips::config::render_config_yaml();
-                let installed = tokio::fs::read_to_string("/etc/fips/fips.yaml")
-                    .await
-                    .ok();
+                let installed = tokio::fs::read_to_string("/etc/fips/fips.yaml").await.ok();
                 let config_changed = installed.as_deref() != Some(expected.as_str());
 
                 if let Err(e) = crate::fips::config::install(&identity_dir).await {
@@ -795,11 +828,65 @@ async fn refresh_tor_address(state: &StateManager, identity: &NodeIdentity) -> R
 /// 3 scans × 30s = 90 seconds of absence before removal.
 const CONTAINER_ABSENCE_THRESHOLD: u32 = 3;
 
+/// Maximum time a package entry may remain stuck in a transitional state
+/// before the scan loop overrides it with podman's live state.
+///
+/// Rationale: the longest single-container stop timeout is bitcoin-core at
+/// 600s. 2× that gives the spawned task ample margin before we assume it
+/// died (panic, OOM, process restart mid-stop) and fall back to the
+/// scanner's authoritative view. Applies to all transitional variants.
+const TRANSITIONAL_STUCK_TIMEOUT: Duration = Duration::from_secs(1200);
+
+/// Returns true if `state` is one of the transitional variants that a
+/// `spawn_transitional`-style background task owns. While such a state is
+/// set, the package scanner must not overwrite it with whatever podman
+/// reports (see `merge_preserving_transitional`).
+fn is_transitional(state: &crate::data_model::PackageState) -> bool {
+    use crate::data_model::PackageState::*;
+    matches!(
+        state,
+        Installing
+            | Stopping
+            | Starting
+            | Restarting
+            | Updating
+            | Removing
+            | CreatingBackup
+            | RestoringBackup
+            | BackingUp
+    )
+}
+
+/// Merge a fresh scan entry `fresh` into `existing` while preserving
+/// `existing.state` (which is transitional — the RPC spawn task owns it).
+/// Non-state observability fields are taken from `fresh` so the UI still
+/// sees live health / exit_code / lan_address readings during a transition.
+fn merge_preserving_transitional(
+    existing: &crate::data_model::PackageDataEntry,
+    fresh: &crate::data_model::PackageDataEntry,
+) -> crate::data_model::PackageDataEntry {
+    crate::data_model::PackageDataEntry {
+        state: existing.state.clone(),
+        // install_progress and uninstall_stage are also owned by the
+        // initiating op (same reason as state) — keep them.
+        install_progress: existing.install_progress.clone(),
+        uninstall_stage: existing.uninstall_stage.clone(),
+        // Everything else comes from the fresh scan.
+        health: fresh.health.clone(),
+        exit_code: fresh.exit_code,
+        static_files: fresh.static_files.clone(),
+        manifest: fresh.manifest.clone(),
+        installed: fresh.installed.clone(),
+        available_update: fresh.available_update.clone(),
+    }
+}
+
 async fn scan_and_update_packages(
     scanner: &DockerPackageScanner,
     state: &StateManager,
     identity: &NodeIdentity,
     absence_tracker: &mut HashMap<String, u32>,
+    transitional_since: &mut HashMap<String, Instant>,
 ) -> Result<()> {
     let packages = scanner.scan_containers().await?;
 
@@ -833,10 +920,61 @@ async fn scan_and_update_packages(
     let mut merged = current_data.package_data.clone();
     let mut changed = false;
 
-    // Update/add containers found in this scan
+    // Update/add containers found in this scan.
+    //
+    // Transitional states (Stopping, Starting, Restarting, Installing,
+    // Updating, Removing, backup variants) are owned by the RPC spawn_task
+    // that initiated the operation — podman's live state during the op is
+    // meaningless ("running" during a graceful stop, "exited" during a
+    // restart, etc.) and must not be written back. See
+    // `merge_preserving_transitional` for the exact rule.
+    //
+    // Escape hatch: if a package has been in a transitional state for
+    // longer than TRANSITIONAL_STUCK_TIMEOUT we assume the spawned task
+    // died without cleanup and let the scan override it.
+    let now = Instant::now();
     for (id, pkg) in &packages {
         absence_tracker.remove(id);
-        if merged.get(id) != Some(pkg) {
+        let existing = merged.get(id);
+        let overwrite = match existing {
+            Some(existing_entry) if is_transitional(&existing_entry.state) => {
+                let entered = *transitional_since.entry(id.clone()).or_insert(now);
+                let stuck = now.duration_since(entered) > TRANSITIONAL_STUCK_TIMEOUT;
+                if stuck {
+                    warn!(
+                        "Container {} stuck in {:?} for >{}s; overriding with scan state {:?}",
+                        id,
+                        existing_entry.state,
+                        TRANSITIONAL_STUCK_TIMEOUT.as_secs(),
+                        pkg.state
+                    );
+                    transitional_since.remove(id);
+                    true
+                } else {
+                    // Keep existing transitional state, but merge non-state
+                    // observability fields (health, exit_code, lan_address
+                    // via installed) from the fresh scan so the UI still
+                    // sees live readings.
+                    let merged_entry = merge_preserving_transitional(existing_entry, pkg);
+                    if existing.cloned() != Some(merged_entry.clone()) {
+                        merged.insert(id.clone(), merged_entry);
+                        changed = true;
+                    }
+                    false
+                }
+            }
+            Some(_) => {
+                // Not transitional: the side-table may hold a stale entry
+                // from a previous transition on this id; drop it.
+                transitional_since.remove(id);
+                existing != Some(pkg)
+            }
+            None => {
+                transitional_since.remove(id);
+                true
+            }
+        };
+        if overwrite && merged.get(id) != Some(pkg) {
             merged.insert(id.clone(), pkg.clone());
             changed = true;
         }
@@ -856,6 +994,7 @@ async fn scan_and_update_packages(
                 );
                 merged.remove(&id);
                 absence_tracker.remove(&id);
+                transitional_since.remove(&id);
                 changed = true;
             }
         }
@@ -926,10 +1065,9 @@ async fn check_peer_health(state: &StateManager, data_dir: &std::path::Path) ->
     let mut new_health = std::collections::HashMap::new();
     for peer in &known_peers {
         let fips_npub = crate::federation::fips_npub_for_onion(data_dir, &peer.onion).await;
-        let reachable =
-            node_message::check_peer_reachable(&peer.onion, fips_npub.as_deref())
-                .await
-                .unwrap_or(false);
+        let reachable = node_message::check_peer_reachable(&peer.onion, fips_npub.as_deref())
+            .await
+            .unwrap_or(false);
         new_health.insert(peer.onion.clone(), reachable);
     }
 
@@ -943,3 +1081,106 @@ async fn check_peer_health(state: &StateManager, data_dir: &std::path::Path) ->
 
     Ok(())
 }
+
+#[cfg(test)]
+mod merge_tests {
+    use super::*;
+    use crate::data_model::{Description, Manifest, PackageDataEntry, PackageState, StaticFiles};
+
+    fn make_manifest() -> Manifest {
+        Manifest {
+            id: "lnd".to_string(),
+            title: "LND".to_string(),
+            version: "0.18.4".to_string(),
+            description: Description {
+                short: "".to_string(),
+                long: "".to_string(),
+            },
+            release_notes: "".to_string(),
+            license: "".to_string(),
+            wrapper_repo: "".to_string(),
+            upstream_repo: "".to_string(),
+            support_site: "".to_string(),
+            marketing_site: "".to_string(),
+            donation_url: None,
+            author: None,
+            website: None,
+            interfaces: None,
+            tier: None,
+        }
+    }
+
+    fn make_static() -> StaticFiles {
+        StaticFiles {
+            license: "".to_string(),
+            instructions: "".to_string(),
+            icon: "".to_string(),
+        }
+    }
+
+    fn make_entry(state: PackageState, health: Option<&str>) -> PackageDataEntry {
+        PackageDataEntry {
+            state,
+            health: health.map(|s| s.to_string()),
+            exit_code: None,
+            static_files: make_static(),
+            manifest: make_manifest(),
+            installed: None,
+            install_progress: None,
+            uninstall_stage: None,
+            available_update: None,
+        }
+    }
+
+    #[test]
+    fn preserves_transitional_state_on_merge() {
+        // existing: user initiated a stop, spawn_transitional set Stopping.
+        // fresh: podman hasn't finished the stop yet, still reports Running.
+        // Expected: merged state stays Stopping — podman's live view must
+        // not clobber the transitional state owned by the RPC spawn task.
+        let existing = make_entry(PackageState::Stopping, Some("healthy"));
+        let fresh = make_entry(PackageState::Running, Some("starting"));
+        let merged = merge_preserving_transitional(&existing, &fresh);
+        assert_eq!(merged.state, PackageState::Stopping);
+    }
+
+    #[test]
+    fn merges_fresh_observability_fields() {
+        // Non-state observability fields (health, exit_code, installed)
+        // MUST come from the fresh scan even while state is preserved —
+        // the UI still shows live health/health during a transition.
+        let mut existing = make_entry(PackageState::Stopping, Some("healthy"));
+        existing.exit_code = None;
+        let mut fresh = make_entry(PackageState::Running, Some("unhealthy"));
+        fresh.exit_code = Some(0);
+        let merged = merge_preserving_transitional(&existing, &fresh);
+        assert_eq!(merged.state, PackageState::Stopping);
+        assert_eq!(merged.health.as_deref(), Some("unhealthy"));
+        assert_eq!(merged.exit_code, Some(0));
+    }
+
+    #[test]
+    fn is_transitional_covers_all_variants() {
+        for s in [
+            PackageState::Installing,
+            PackageState::Stopping,
+            PackageState::Starting,
+            PackageState::Restarting,
+            PackageState::Updating,
+            PackageState::Removing,
+            PackageState::CreatingBackup,
+            PackageState::RestoringBackup,
+            PackageState::BackingUp,
+        ] {
+            assert!(is_transitional(&s), "{:?} should be transitional", s);
+        }
+        for s in [
+            PackageState::Installed,
+            PackageState::Stopped,
+            PackageState::Exited,
+            PackageState::Running,
+        ] {
+            assert!(!is_transitional(&s), "{:?} should NOT be transitional", s);
+        }
+    }
+}

core/archipelago/src/session.rs

@@ -70,6 +70,17 @@ impl SessionStore {
         }
     }
 
+    /// Construct an empty SessionStore that persists to a caller-supplied
+    /// path. Used by tests so they don't pick up sessions from the dev
+    /// machine's real /var/lib/archipelago/sessions.json.
+    #[cfg(test)]
+    pub fn new_for_tests(persist_path: PathBuf) -> Self {
+        Self {
+            sessions: Arc::new(RwLock::new(HashMap::new())),
+            persist_path,
+        }
+    }
+
     /// Load persisted sessions from disk (only Full sessions).
     async fn load_from_disk(path: &Path) -> HashMap<[u8; 32], Session> {
         let mut map = HashMap::new();
@@ -462,7 +473,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_session_create_and_validate() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token = store.create().await;
 
         assert!(store.validate(&token).await);
@@ -470,13 +484,19 @@ mod tests {
 
     #[tokio::test]
     async fn test_session_invalid_token() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         assert!(!store.validate("nonexistent_token").await);
     }
 
     #[tokio::test]
     async fn test_session_remove() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token = store.create().await;
 
         assert!(store.validate(&token).await);
@@ -486,7 +506,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_pending_session_upgrade() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let secret = vec![1, 2, 3, 4];
         let token = store.create_pending(secret.clone()).await;
 
@@ -510,7 +533,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_pending_session_max_attempts() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let secret = vec![1, 2, 3];
         let token = store.create_pending(secret).await;
 
@@ -538,7 +564,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_session_activity_updates_on_validate() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token = store.create().await;
 
         // First validation should succeed and touch last_activity
@@ -550,7 +579,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_invalidate_all_except() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token1 = store.create().await;
         let token2 = store.create().await;
         let token3 = store.create().await;
@@ -565,7 +597,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_session_rotate() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let old_token = store.create().await;
 
         assert!(store.validate(&old_token).await);
@@ -580,7 +615,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_max_concurrent_sessions() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let mut tokens = Vec::new();
 
         // Create MAX_CONCURRENT_SESSIONS sessions
@@ -608,7 +646,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_active_session_count() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         assert_eq!(store.active_session_count().await, 0);
 
         let token1 = store.create().await;
@@ -623,7 +664,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_cleanup_expired_removes_stale() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token = store.create().await;
 
         assert!(store.validate(&token).await);
@@ -636,7 +680,10 @@ mod tests {
 
     #[tokio::test]
     async fn test_rotate_preserves_session_count() {
-        let store = SessionStore::new().await;
+        let store = SessionStore::new_for_tests(std::env::temp_dir().join(format!(
+            "archipelago-sessions-test-{}.json",
+            rand::random::<u64>()
+        )));
         let token = store.create().await;
         assert_eq!(store.active_session_count().await, 1);
 

core/archipelago/src/settings/transport.rs

@@ -141,11 +141,7 @@ pub async fn snapshot() -> TransportPreferences {
 /// Update a single service preference, persist to disk, and update the
 /// handle. Callers must pass `data_dir` because the on-disk file lives
 /// under it — the handle alone doesn't know where to write.
-pub async fn set(
-    data_dir: &Path,
-    service: PeerService,
-    pref: TransportPref,
-) -> Result<()> {
+pub async fn set(data_dir: &Path, service: PeerService, pref: TransportPref) -> Result<()> {
     let new_prefs = {
         let lock = HANDLE.get_or_init(|| RwLock::new(TransportPreferences::default()));
         let mut w = lock.write().await;
@@ -173,8 +169,7 @@ async fn save_to_disk(data_dir: &Path, prefs: &TransportPreferences) -> Result<(
             .await
             .with_context(|| format!("create {}", parent.display()))?;
     }
-    let body = serde_json::to_string_pretty(prefs)
-        .context("serialize TransportPreferences")?;
+    let body = serde_json::to_string_pretty(prefs).context("serialize TransportPreferences")?;
     tokio::fs::write(&path, body)
         .await
         .with_context(|| format!("write {}", path.display()))?;
@@ -213,10 +208,7 @@ mod tests {
         p.set_for_service(PeerService::Messaging, TransportPref::Tor);
         let s = serde_json::to_string(&p).unwrap();
         let back: TransportPreferences = serde_json::from_str(&s).unwrap();
-        assert_eq!(
-            back.for_service(PeerService::Messaging),
-            TransportPref::Tor
-        );
+        assert_eq!(back.for_service(PeerService::Messaging), TransportPref::Tor);
     }
 
     #[test]

core/archipelago/src/transport/chunking.rs

@@ -98,7 +98,12 @@ pub fn encode_chunked(data: &[u8]) -> Result<Vec<Chunk>> {
     }
 
     let shard_size = MAX_CHUNK_PAYLOAD;
-    let data_shard_count = data.len().div_ceil(shard_size);
+    // Reserve the first 4 bytes of shard 0 for a length header so the
+    // receiver can trim padding after FEC reconstruction. Effective
+    // payload capacity is therefore (shards * shard_size) - 4.
+    const LEN_HEADER: usize = 4;
+    let total_payload = data.len() + LEN_HEADER;
+    let data_shard_count = total_payload.div_ceil(shard_size);
 
     if data_shard_count > MAX_PRACTICAL_CHUNKS {
         anyhow::bail!(
@@ -116,22 +121,25 @@ pub fn encode_chunked(data: &[u8]) -> Result<Vec<Chunk>> {
         anyhow::bail!("Too many shards: {}", total_shards);
     }
 
-    // Split data into equal-size shards
+    // Build a single contiguous buffer: [len_u32_le][data...][zero_padding]
+    // then split into equal-size shards.
+    let buffer_size = data_shard_count * shard_size;
+    let mut buffer = vec![0u8; buffer_size];
+    buffer[..LEN_HEADER].copy_from_slice(&(data.len() as u32).to_le_bytes());
+    buffer[LEN_HEADER..LEN_HEADER + data.len()].copy_from_slice(data);
+
     let mut shards: Vec<Vec<u8>> = Vec::with_capacity(total_shards);
     for i in 0..data_shard_count {
         let start = i * shard_size;
-        let end = (start + shard_size).min(data.len());
-        let mut shard = vec![0u8; shard_size];
-        shard[..end - start].copy_from_slice(&data[start..end]);
-        shards.push(shard);
+        shards.push(buffer[start..start + shard_size].to_vec());
     }
-
-    // Add empty parity shards
+    // Empty parity shards
     for _ in 0..parity_shard_count {
         shards.push(vec![0u8; shard_size]);
     }
 
-    // Generate parity
+    // Generate parity over the data shards (which now correctly include
+    // the length header in shard 0).
     let rs = ReedSolomon::new(data_shard_count, parity_shard_count)
         .context("Failed to create Reed-Solomon codec")?;
     rs.encode(&mut shards)
@@ -152,18 +160,6 @@ pub fn encode_chunked(data: &[u8]) -> Result<Vec<Chunk>> {
         });
     }
 
-    // Encode the original data length in the first chunk's first 4 bytes
-    // so the receiver can trim padding after reconstruction.
-    let data_len = data.len() as u32;
-    chunks[0].payload[..4].copy_from_slice(&data_len.to_le_bytes());
-    // Re-encode FEC to reflect the length header change
-    let mut shard_data: Vec<Vec<u8>> = chunks.iter().map(|c| c.payload.clone()).collect();
-    rs.encode(&mut shard_data)
-        .context("Reed-Solomon re-encoding failed")?;
-    for (i, shard) in shard_data.into_iter().enumerate() {
-        chunks[i].payload = shard;
-    }
-
     Ok(chunks)
 }
 
@@ -318,17 +314,13 @@ mod tests {
 
     #[test]
     fn test_chunk_roundtrip_medium() {
-        // ~500 bytes: 4 data chunks + 1 parity
+        // 500 bytes payload + 4-byte length header = 504 bytes.
+        // ceil(504 / 124) = 5 data shards, plus ceil(5/4) = 2 parity = 7 total.
         let data: Vec<u8> = (0..500).map(|i| (i % 256) as u8).collect();
         let chunks = encode_chunked(&data).unwrap();
 
         let data_chunks: Vec<_> = chunks.iter().filter(|c| !c.is_parity).collect();
-        let _parity_chunks: Vec<_> = chunks.iter().filter(|c| c.is_parity).collect();
-        assert_eq!(data_chunks.len(), 4); // ceil(500/124) = 5... wait
-                                          // Actually: ceil(500/124) = ceil(4.03) = 5 data shards
-                                          // But the first shard has 4 bytes of length header embedded, so
-                                          // the actual data capacity is 124 * N - 0 (length is IN the shard data).
-                                          // Let's just check it roundtrips.
+        assert_eq!(data_chunks.len(), 5);
 
         let mut reassembler = ChunkReassembler::new();
         let mut result = None;

core/archipelago/src/update.rs

@@ -55,7 +55,10 @@ fn parse_version_triple(v: &str) -> Option<(u32, u32, u32)> {
 /// latest). Falls back to string inequality if either version doesn't
 /// parse, preserving the old behaviour for unusual version strings.
 fn is_newer(candidate: &str, current: &str) -> bool {
-    match (parse_version_triple(candidate), parse_version_triple(current)) {
+    match (
+        parse_version_triple(candidate),
+        parse_version_triple(current),
+    ) {
         (Some(a), Some(b)) => a > b,
         _ => candidate != current,
     }
@@ -63,14 +66,11 @@ fn is_newer(candidate: &str, current: &str) -> bool {
 
 const DEFAULT_UPDATE_MANIFEST_URL: &str =
     "https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/manifest.json";
-/// Secondary mirror: same manifest, served from the VPS. Added as a
-/// default mirror so nodes automatically fall through when the primary
-/// is slow or unreachable.
+/// Secondary mirror on an OVH VPS — independent network path so a
+/// single-provider outage doesn't knock out both mirrors. Promoted to
+/// primary default on 2026-04-23 after the Hetzner .160 VPS was
+/// decommissioned.
 const DEFAULT_SECONDARY_MIRROR_URL: &str =
-    "http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/manifest.json";
-/// Tertiary mirror on a separate OVH VPS — independent network path so
-/// a single-provider outage doesn't knock out all three mirrors.
-const DEFAULT_TERTIARY_MIRROR_URL: &str =
     "http://146.59.87.168:3000/lfg2025/archy/raw/branch/main/releases/manifest.json";
 const UPDATE_STATE_FILE: &str = "update_state.json";
 const UPDATE_MIRRORS_FILE: &str = "update-mirrors.json";
@@ -112,16 +112,12 @@ fn default_mirrors() -> Vec<UpdateMirror> {
     vec![
         UpdateMirror {
             url: DEFAULT_SECONDARY_MIRROR_URL.to_string(),
-            label: "Server 1 (VPS)".to_string(),
+            label: "Server 1 (OVH)".to_string(),
         },
         UpdateMirror {
             url: DEFAULT_UPDATE_MANIFEST_URL.to_string(),
             label: "Server 2 (tx1138)".to_string(),
         },
-        UpdateMirror {
-            url: DEFAULT_TERTIARY_MIRROR_URL.to_string(),
-            label: "Server 3 (OVH)".to_string(),
-        },
     ]
 }
 
@@ -150,18 +146,26 @@ pub async fn load_mirrors(data_dir: &Path) -> Result<Vec<UpdateMirror>> {
         return Ok(default_mirrors());
     }
 
+    // One-time migration: the Hetzner VPS at 23.182.128.160 was
+    // decommissioned 2026-04-23. Existing nodes have it baked into their
+    // saved mirror list (was the original Server 1). Strip it on load so
+    // we don't spend seconds per install timing out against a dead host.
+    // Exception to the usual "explicit removals stick" rule: the user
+    // never chose to add this — it was a default.
+    let before = list.len();
+    list.retain(|m| !m.url.contains("23.182.128.160"));
+    let mut changed = list.len() != before;
+
     // Merge in any default URLs the saved config is missing.
-    let known: std::collections::HashSet<String> =
-        list.iter().map(|m| m.url.clone()).collect();
+    let known: std::collections::HashSet<String> = list.iter().map(|m| m.url.clone()).collect();
     let defaults = default_mirrors();
-    let mut added = false;
     for def in &defaults {
         if !known.contains(&def.url) {
             list.push(def.clone());
-            added = true;
+            changed = true;
         }
     }
-    if added {
+    if changed {
         let _ = save_mirrors(data_dir, &list).await;
     }
     Ok(list)
@@ -188,7 +192,8 @@ pub async fn save_mirrors(data_dir: &Path, mirrors: &[UpdateMirror]) -> Result<(
 /// mirror points component downloads back at the same mirror rather
 /// than whatever absolute URL the publisher baked in.
 fn manifest_origin(manifest_url: &str) -> Option<String> {
-    let rest = manifest_url.strip_prefix("https://")
+    let rest = manifest_url
+        .strip_prefix("https://")
         .map(|r| ("https", r))
         .or_else(|| manifest_url.strip_prefix("http://").map(|r| ("http", r)))?;
     let (scheme, after_scheme) = rest;
@@ -304,13 +309,9 @@ pub struct PendingVerification {
     pub deadline_ts: i64,
 }
 
-async fn write_pending_verification(
-    data_dir: &Path,
-    marker: &PendingVerification,
-) -> Result<()> {
+async fn write_pending_verification(data_dir: &Path, marker: &PendingVerification) -> Result<()> {
     let path = data_dir.join(PENDING_VERIFY_FILE);
-    let data = serde_json::to_string_pretty(marker)
-        .context("serialize pending-verify marker")?;
+    let data = serde_json::to_string_pretty(marker).context("serialize pending-verify marker")?;
     fs::write(&path, data)
         .await
         .with_context(|| format!("write pending-verify marker to {}", path.display()))?;
@@ -404,10 +405,7 @@ pub async fn verify_pending_update(data_dir: &Path) {
         attempt += 1;
         match probe_frontend_once().await {
             Ok(()) => {
-                info!(
-                    attempt,
-                    "Post-OTA verification succeeded — clearing marker"
-                );
+                info!(attempt, "Post-OTA verification succeeded — clearing marker");
                 clear_pending_verification(data_dir).await;
                 return;
             }
@@ -441,9 +439,7 @@ pub async fn verify_pending_update(data_dir: &Path) {
         let _ = host_sudo(&["mv", web_ui_bak.to_str().unwrap_or(""), web_ui]).await;
         tracing::info!(quarantined = %quarantine, "Restored web-ui from web-ui.bak");
     } else {
-        tracing::warn!(
-            "web-ui.bak not present — frontend cannot be rolled back, only binary"
-        );
+        tracing::warn!("web-ui.bak not present — frontend cannot be rolled back, only binary");
     }
 
     if let Err(e) = rollback_update(data_dir).await {
@@ -478,8 +474,7 @@ pub async fn load_state(data_dir: &Path) -> Result<UpdateState> {
     let data = fs::read_to_string(&path)
         .await
         .context("Reading update state")?;
-    let mut state: UpdateState =
-        serde_json::from_str(&data).context("Parsing update state")?;
+    let mut state: UpdateState = serde_json::from_str(&data).context("Parsing update state")?;
 
     // Keep current_version in sync with the binary. Sideloaded nodes
     // (ssh + cp /usr/local/bin/archipelago) don't touch the state file,
@@ -553,7 +548,8 @@ pub async fn check_for_updates(data_dir: &Path) -> Result<UpdateState> {
                 tokio::time::sleep(std::time::Duration::from_secs(2)).await;
             }
             match client.get(manifest_url).send().await {
-                Ok(resp) if resp.status().is_success() => match resp.json::<UpdateManifest>().await {
+                Ok(resp) if resp.status().is_success() => match resp.json::<UpdateManifest>().await
+                {
                     Ok(mut manifest) => {
                         rewrite_manifest_origins(&mut manifest, manifest_url);
                         if is_newer(&manifest.version, &state.current_version) {
@@ -1093,26 +1089,15 @@ pub async fn apply_update(data_dir: &Path) -> Result<()> {
                 if !mk.success() {
                     anyhow::bail!("mkdir {} failed", staging_new);
                 }
-                let extract = host_sudo(&[
-                    "tar",
-                    "-xzf",
-                    &src.to_string_lossy(),
-                    "-C",
-                    &staging_new,
-                ])
-                .await
-                .with_context(|| format!("Failed to extract {}", name))?;
+                let extract =
+                    host_sudo(&["tar", "-xzf", &src.to_string_lossy(), "-C", &staging_new])
+                        .await
+                        .with_context(|| format!("Failed to extract {}", name))?;
                 if !extract.success() {
                     let _ = host_sudo(&["rm", "-rf", &staging_new]).await;
                     anyhow::bail!("tar extraction failed for {}", name);
                 }
-                let _ = host_sudo(&[
-                    "chown",
-                    "-R",
-                    "archipelago:archipelago",
-                    &staging_new,
-                ])
-                .await;
+                let _ = host_sudo(&["chown", "-R", "archipelago:archipelago", &staging_new]).await;
 
                 // Set world-readable perms so nginx (runs as www-data)
                 // can stat + serve the files. Without this, the tar
@@ -1121,11 +1106,27 @@ pub async fn apply_update(data_dir: &Path) -> Result<()> {
                 // swap — exactly what bit .116 on the v1.7.38 rollout.
                 let _ = host_sudo(&["chmod", "755", &staging_new]).await;
                 let _ = host_sudo(&[
-                    "find", &staging_new, "-type", "d", "-exec", "chmod", "755", "{}", "+",
+                    "find",
+                    &staging_new,
+                    "-type",
+                    "d",
+                    "-exec",
+                    "chmod",
+                    "755",
+                    "{}",
+                    "+",
                 ])
                 .await;
                 let _ = host_sudo(&[
-                    "find", &staging_new, "-type", "f", "-exec", "chmod", "644", "{}", "+",
+                    "find",
+                    &staging_new,
+                    "-type",
+                    "f",
+                    "-exec",
+                    "chmod",
+                    "644",
+                    "{}",
+                    "+",
                 ])
                 .await;
 
@@ -1167,12 +1168,8 @@ pub async fn apply_update(data_dir: &Path) -> Result<()> {
                 // old copy as the new rollback.
                 if Path::new(&staging_old).exists() {
                     if Path::new(backup_path).exists() {
-                        let _ = host_sudo(&[
-                            "mv",
-                            backup_path,
-                            &format!("{}.{}", backup_path, ts),
-                        ])
-                        .await;
+                        let _ = host_sudo(&["mv", backup_path, &format!("{}.{}", backup_path, ts)])
+                            .await;
                     }
                     let _ = host_sudo(&["mv", &staging_old, backup_path]).await;
                 }
@@ -1211,9 +1208,7 @@ pub async fn apply_update(data_dir: &Path) -> Result<()> {
         applied_at: chrono::Utc::now().to_rfc3339(),
         new_version,
         previous_version,
-        deadline_ts: chrono::Utc::now().timestamp()
-            + PENDING_VERIFY_WINDOW_SECS as i64
-            + 60,
+        deadline_ts: chrono::Utc::now().timestamp() + PENDING_VERIFY_WINDOW_SECS as i64 + 60,
     };
     if let Err(e) = write_pending_verification(data_dir, &marker).await {
         tracing::warn!(error = %e, "Failed to write post-OTA verify marker — rollback disabled for this OTA");
@@ -1379,7 +1374,9 @@ pub async fn run_update_scheduler(data_dir: std::path::PathBuf) {
                             debug!("Update scheduler: apply failed: {}", e);
                             continue;
                         }
-                        info!("Update scheduler: update applied, restart scheduled by apply_update");
+                        info!(
+                            "Update scheduler: update applied, restart scheduled by apply_update"
+                        );
                         // apply_update has already spawned a 2s-delayed
                         // `systemctl restart archipelago`. Don't call
                         // std::process::exit here — that kills the runtime
@@ -1414,7 +1411,9 @@ mod tests {
     #[test]
     fn test_manifest_origin_parses_https() {
         assert_eq!(
-            manifest_origin("https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/manifest.json"),
+            manifest_origin(
+                "https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/manifest.json"
+            ),
             Some("https://git.tx1138.com".to_string())
         );
     }
@@ -1422,7 +1421,9 @@ mod tests {
     #[test]
     fn test_manifest_origin_parses_http_with_port() {
         assert_eq!(
-            manifest_origin("http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/manifest.json"),
+            manifest_origin(
+                "http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/manifest.json"
+            ),
             Some("http://23.182.128.160:3000".to_string())
         );
     }
@@ -1458,7 +1459,10 @@ mod tests {
                 },
             ],
         };
-        rewrite_manifest_origins(&mut manifest, "http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/manifest.json");
+        rewrite_manifest_origins(
+            &mut manifest,
+            "http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/manifest.json",
+        );
         assert_eq!(
             manifest.components[0].download_url,
             "http://23.182.128.160:3000/lfg2025/archy/raw/branch/main/releases/v1.7.26-alpha/archipelago"
@@ -1473,10 +1477,9 @@ mod tests {
     async fn test_load_mirrors_returns_defaults_when_absent() {
         let dir = tempfile::tempdir().unwrap();
         let list = load_mirrors(dir.path()).await.unwrap();
-        assert_eq!(list.len(), 3);
-        assert!(list[0].url.contains("23.182.128.160"));
+        assert_eq!(list.len(), 2);
+        assert!(list[0].url.contains("146.59.87.168"));
         assert!(list[1].url.contains("git.tx1138.com"));
-        assert!(list[2].url.contains("146.59.87.168"));
     }
 
     #[tokio::test]
@@ -1488,7 +1491,22 @@ mod tests {
         }];
         save_mirrors(dir.path(), &list).await.unwrap();
         let back = load_mirrors(dir.path()).await.unwrap();
-        assert_eq!(back, list);
+        // load_mirrors merges in any missing default mirrors so a node
+        // that explicitly added a single custom mirror still gets the
+        // built-in OVH + tx1138 fallbacks. The custom mirror is preserved.
+        assert!(
+            back.iter().any(|m| m.url == "https://example.com/m.json"),
+            "custom mirror should round-trip; got {:?}",
+            back
+        );
+        for def in default_mirrors() {
+            assert!(
+                back.iter().any(|m| m.url == def.url),
+                "default mirror {} should be present after load; got {:?}",
+                def.url,
+                back
+            );
+        }
     }
 
     #[test]
@@ -1701,7 +1719,9 @@ mod tests {
             previous_version: "1.7.40-alpha".into(),
             deadline_ts: chrono::Utc::now().timestamp() + 150,
         };
-        write_pending_verification(dir.path(), &marker).await.unwrap();
+        write_pending_verification(dir.path(), &marker)
+            .await
+            .unwrap();
         let read = read_pending_verification(dir.path()).await.unwrap();
         assert_eq!(read.new_version, "1.7.41-alpha");
         assert_eq!(read.previous_version, "1.7.40-alpha");

core/archipelago/src/wallet/cashu.rs

@@ -334,7 +334,9 @@ mod tests {
             amount: 1,
             id: "test".into(),
             secret: "s".into(),
-            c: "02a9acc1e48c25eeeb9289b5031cc57da9fe72f3fe2861d94ec4da0e7f6c2b4e24".to_string(),
+            // Generator point G of secp256k1, compressed form. Always a
+            // valid pubkey, so c_as_pubkey() must succeed.
+            c: "0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798".to_string(),
         };
         assert!(proof.c_as_pubkey().is_ok());
     }

core/container/src/dependency_resolver.rs

@@ -213,9 +213,16 @@ mod tests {
                 version: "1.0.0".to_string(),
                 description: None,
                 container: ContainerConfig {
-                    image: format!("test/{}:latest", id),
+                    image: Some(format!("test/{}:latest", id)),
                     image_signature: None,
                     pull_policy: "if-not-present".to_string(),
+                    build: None,
+                    network: None,
+                    custom_args: vec![],
+                    entrypoint: None,
+                    derived_env: vec![],
+                    secret_env: vec![],
+                    data_uid: None,
                 },
                 dependencies: deps,
                 resources: Default::default(),

core/container/src/lib.rs

@@ -9,7 +9,11 @@ pub mod runtime;
 pub use bitcoin_simulator::{BitcoinSimulationMode, BitcoinSimulator};
 pub use dependency_resolver::DependencyResolver;
 pub use health_monitor::HealthMonitor;
-pub use manifest::{AppManifest, Dependency, HealthCheck, ResourceLimits, SecurityPolicy};
+pub use manifest::{
+    AppManifest, BuildConfig, ContainerConfig, Dependency, DerivedEnv, HealthCheck, HostFacts,
+    ManifestError, ResolvedSource, ResourceLimits, SecretEnv, SecretsProvider, SecurityPolicy,
+    Volume,
+};
 pub use podman_client::{ContainerState, ContainerStatus, PodmanClient};
 pub use port_manager::{PortError, PortManager};
 pub use runtime::{AutoRuntime, ContainerRuntime, DockerRuntime, PodmanRuntime};

core/container/src/manifest.rs

@@ -57,17 +57,136 @@ pub struct AppDefinition {
 
 #[derive(Debug, Clone, Serialize, Deserialize, Default)]
 pub struct ContainerConfig {
-    pub image: String,
+    /// Pull source. Mutually exclusive with `build`. Exactly one of the two must be present.
+    #[serde(default)]
+    pub image: Option<String>,
     #[serde(default)]
     pub image_signature: Option<String>,
     #[serde(default = "default_pull_policy")]
     pub pull_policy: String,
+    /// Local build source. Mutually exclusive with `image`.
+    #[serde(default)]
+    pub build: Option<BuildConfig>,
+
+    // ── Step 8b.0 additions ──────────────────────────────────────────
+    //
+    // Fields the Rust orchestrator needs to faithfully port containers
+    // from the legacy `scripts/container-specs.sh` registry. See
+    // `docs/STEP-8B-PORT-AUDIT.md` for the full justification per field.
+    //
+    // All are optional with `#[serde(default)]` so every existing manifest
+    // in `apps/` continues to parse unchanged.
+    /// Podman `--network` value. `Some("archy-net")` joins the shared
+    /// Archipelago bridge. `Some("host")` uses host networking.
+    /// `None` (the default) falls back to podman's default isolated
+    /// network — equivalent to today's rootless default.
+    ///
+    /// `SecurityPolicy::network_policy` remains a policy knob (what the
+    /// firewall layer does); this field is literally the CLI flag value.
+    #[serde(default)]
+    pub network: Option<String>,
+
+    /// Extra positional arguments appended to the container command
+    /// after the image. Mirrors `SPEC_CUSTOM_ARGS` in
+    /// `scripts/container-specs.sh` (bitcoin-knots prune/dbcache flags,
+    /// filebrowser `--config /data/.filebrowser.json`, etc).
+    #[serde(default)]
+    pub custom_args: Vec<String>,
+
+    /// Entrypoint override (`podman run --entrypoint …`). When present,
+    /// replaces the image's default entrypoint. Mirrors `SPEC_ENTRYPOINT`
+    /// for fedimint-gateway's LND-aware invocation.
+    #[serde(default)]
+    pub entrypoint: Option<Vec<String>>,
+
+    /// Environment keys whose values are rendered from a small
+    /// allow-list of host facts (`HOST_IP`, `HOST_MDNS`, `DISK_GB`).
+    /// Resolved by `ContainerConfig::resolve_derived_env` at apply time
+    /// — never hard-coded into the manifest.
+    ///
+    /// Example: `- { key: FM_P2P_URL, template: "fedimint://{{HOST_MDNS}}:8173" }`
+    #[serde(default)]
+    pub derived_env: Vec<DerivedEnv>,
+
+    /// Environment keys whose values are read from files in
+    /// `/var/lib/archipelago/secrets/<secret_file>`. Never logged.
+    /// Resolved by `ContainerConfig::resolve_secret_env` at apply time.
+    ///
+    /// Example: `- { key: FM_BITCOIND_PASSWORD, secret_file: bitcoin-rpc-password }`
+    #[serde(default)]
+    pub secret_env: Vec<SecretEnv>,
+
+    /// Rootless-mapped UID:GID applied to the container's data directory
+    /// (the `bind`-mounted host path with `target` inside the container's
+    /// data root) before creation. Mirrors `SPEC_DATA_UID`.
+    ///
+    /// Example: `"100070:100070"` for Postgres' mapped subuid.
+    #[serde(default)]
+    pub data_uid: Option<String>,
+}
+
+/// Derived-env entry. The template is rendered against `HostFacts` at
+/// apply time; exactly one `{{PLACEHOLDER}}` occurrence per supported
+/// fact name is allowed (host_ip, host_mdns, disk_gb).
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+pub struct DerivedEnv {
+    pub key: String,
+    pub template: String,
+}
+
+/// Secret-env entry. `secret_file` is resolved against a
+/// `SecretsProvider` (in prod, `/var/lib/archipelago/secrets/`).
+///
+/// `secret_file` is restricted to a bare filename — no `/`, no `..`.
+/// Validated at `AppManifest::validate` time.
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+pub struct SecretEnv {
+    pub key: String,
+    pub secret_file: String,
 }
 
 fn default_pull_policy() -> String {
     "if-not-present".to_string()
 }
 
+/// Build a container image locally from a Dockerfile rather than pulling from a registry.
+///
+/// When present on `ContainerConfig`, the orchestrator runs `podman build -t <tag> -f <dockerfile> <context>`
+/// before starting the container. The resulting local image is referenced by `tag`.
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
+pub struct BuildConfig {
+    /// Build context directory (absolute path or relative to the manifest location).
+    pub context: String,
+    /// Dockerfile path relative to `context`. Defaults to `Dockerfile`.
+    #[serde(default = "default_dockerfile")]
+    pub dockerfile: String,
+    /// Tag applied to the built image. Used as the container's image reference.
+    pub tag: String,
+    /// Optional `--build-arg KEY=VALUE` pairs passed to the build.
+    #[serde(default)]
+    pub build_args: HashMap<String, String>,
+}
+
+fn default_dockerfile() -> String {
+    "Dockerfile".to_string()
+}
+
+/// Resolved pull-or-build decision after manifest validation.
+///
+/// `ContainerConfig::resolve()` produces this. The orchestrator matches on it
+/// to decide whether to pull a registry image or invoke a local build.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum ResolvedSource {
+    /// Pull `image` from a registry using `pull_policy` semantics.
+    Pull {
+        image: String,
+        pull_policy: String,
+        image_signature: Option<String>,
+    },
+    /// Build locally. The resulting tag is the image reference for `podman create`.
+    Build(BuildConfig),
+}
+
 #[derive(Debug, Clone, Serialize, Deserialize)]
 #[serde(untagged)]
 pub enum Dependency {
@@ -133,10 +252,15 @@ impl From<(u16, u16)> for PortMapping {
 pub struct Volume {
     #[serde(rename = "type")]
     pub volume_type: String,
+    #[serde(default)]
     pub source: String,
     pub target: String,
     #[serde(default)]
     pub options: Vec<String>,
+    /// For `type: tmpfs` only. Comma-separated mount options
+    /// (e.g. `"rw,noexec,nosuid,size=256m"`). Ignored for bind/volume.
+    #[serde(default)]
+    pub tmpfs_options: Option<String>,
 }
 
 #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -182,10 +306,33 @@ impl AppManifest {
             return Err(ManifestError::Invalid("app.id cannot be empty".to_string()));
         }
 
-        if self.app.container.image.is_empty() {
-            return Err(ManifestError::Invalid(
-                "container.image cannot be empty".to_string(),
-            ));
+        // Exactly one of container.image or container.build must be set. We can't
+        // default either side, because an empty-string image or an empty build block
+        // would be silently wrong downstream.
+        match (&self.app.container.image, &self.app.container.build) {
+            (Some(img), None) if !img.is_empty() => {}
+            (None, Some(b)) => {
+                if b.context.is_empty() {
+                    return Err(ManifestError::Invalid(
+                        "container.build.context cannot be empty".to_string(),
+                    ));
+                }
+                if b.tag.is_empty() {
+                    return Err(ManifestError::Invalid(
+                        "container.build.tag cannot be empty".to_string(),
+                    ));
+                }
+            }
+            (Some(_), Some(_)) => {
+                return Err(ManifestError::Invalid(
+                    "container.image and container.build are mutually exclusive".to_string(),
+                ));
+            }
+            _ => {
+                return Err(ManifestError::Invalid(
+                    "container must specify either image or build".to_string(),
+                ));
+            }
         }
 
         // Validate version format (semantic versioning)
@@ -195,13 +342,295 @@ impl AppManifest {
             ));
         }
 
+        // ── Step 8b.0 field validation ────────────────────────────────
+
+        // network: allow any non-empty string; podman itself is the
+        // final authority (named networks, "host", "bridge", "none",
+        // "container:<name>", etc). Reject only the empty-string case
+        // so "network:" with no value is a loud error instead of a
+        // silent default.
+        if let Some(n) = &self.app.container.network {
+            if n.is_empty() {
+                return Err(ManifestError::Invalid(
+                    "container.network cannot be empty (omit the field to use default)".to_string(),
+                ));
+            }
+        }
+
+        // custom_args: no empty strings (would inject literal "" into
+        // the podman command line and confuse downstream parsing).
+        for (i, a) in self.app.container.custom_args.iter().enumerate() {
+            if a.is_empty() {
+                return Err(ManifestError::Invalid(format!(
+                    "container.custom_args[{i}] cannot be empty"
+                )));
+            }
+        }
+
+        // entrypoint: present ⇒ non-empty vec, no empty elements.
+        if let Some(ep) = &self.app.container.entrypoint {
+            if ep.is_empty() {
+                return Err(ManifestError::Invalid(
+                    "container.entrypoint must contain at least one element when set".to_string(),
+                ));
+            }
+            for (i, a) in ep.iter().enumerate() {
+                if a.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.entrypoint[{i}] cannot be empty"
+                    )));
+                }
+            }
+        }
+
+        // derived_env: non-empty keys, unique keys, templates reference
+        // only known host-fact placeholders.
+        {
+            let mut seen: std::collections::HashSet<&str> = std::collections::HashSet::new();
+            for (i, e) in self.app.container.derived_env.iter().enumerate() {
+                if e.key.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.derived_env[{i}].key cannot be empty"
+                    )));
+                }
+                if !seen.insert(e.key.as_str()) {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.derived_env has duplicate key '{}'",
+                        e.key
+                    )));
+                }
+                validate_derived_template(&e.key, &e.template)?;
+            }
+        }
+
+        // secret_env: non-empty keys, unique keys, secret_file is a
+        // bare filename (no '/', no '..').
+        {
+            let mut seen: std::collections::HashSet<&str> = std::collections::HashSet::new();
+            for (i, e) in self.app.container.secret_env.iter().enumerate() {
+                if e.key.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.secret_env[{i}].key cannot be empty"
+                    )));
+                }
+                if !seen.insert(e.key.as_str()) {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.secret_env has duplicate key '{}'",
+                        e.key
+                    )));
+                }
+                if e.secret_file.is_empty()
+                    || e.secret_file.contains('/')
+                    || e.secret_file.contains("..")
+                {
+                    return Err(ManifestError::Invalid(format!(
+                        "container.secret_env[{}].secret_file must be a bare filename (no '/', no '..'), got '{}'",
+                        i, e.secret_file
+                    )));
+                }
+            }
+        }
+
+        // data_uid: if set, must look like "NNNNN:NNNNN".
+        if let Some(u) = &self.app.container.data_uid {
+            let parts: Vec<&str> = u.split(':').collect();
+            let valid = parts.len() == 2
+                && !parts[0].is_empty()
+                && !parts[1].is_empty()
+                && parts[0].chars().all(|c| c.is_ascii_digit())
+                && parts[1].chars().all(|c| c.is_ascii_digit());
+            if !valid {
+                return Err(ManifestError::Invalid(format!(
+                    "container.data_uid must be 'UID:GID' with numeric parts, got '{}'",
+                    u
+                )));
+            }
+        }
+
+        // Volume tmpfs_options: only meaningful for type: tmpfs.
+        for (i, v) in self.app.volumes.iter().enumerate() {
+            if v.volume_type == "tmpfs" {
+                if v.target.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "volumes[{i}] (tmpfs) must set target"
+                    )));
+                }
+                if !v.source.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "volumes[{i}] (tmpfs) must not set source"
+                    )));
+                }
+            } else if v.tmpfs_options.is_some() {
+                return Err(ManifestError::Invalid(format!(
+                    "volumes[{i}] sets tmpfs_options but type is '{}', not 'tmpfs'",
+                    v.volume_type
+                )));
+            } else {
+                if v.source.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "volumes[{i}] ({}) must set source",
+                        v.volume_type
+                    )));
+                }
+                if v.target.is_empty() {
+                    return Err(ManifestError::Invalid(format!(
+                        "volumes[{i}] ({}) must set target",
+                        v.volume_type
+                    )));
+                }
+            }
+        }
+
         Ok(())
     }
 }
 
+/// Host facts available to `derived_env` templates at apply time.
+///
+/// Mirrors the values `scripts/container-specs.sh:detect_environment()`
+/// computed before each reconcile pass. The Rust orchestrator computes
+/// these once per reconcile tick and passes them to
+/// `ContainerConfig::resolve_derived_env`.
+#[derive(Debug, Clone)]
+pub struct HostFacts {
+    /// Primary host IPv4 (e.g. from `hostname -I | awk '{print $1}'`).
+    /// Falls back to `127.0.0.1` on detection failure.
+    pub host_ip: String,
+    /// mDNS hostname (`<hostname>.local`). Survives DHCP churn and
+    /// reinstall-on-different-IP. Requires avahi-daemon on the node.
+    pub host_mdns: String,
+    /// Usable disk size in gigabytes at `/var/lib/archipelago` (or
+    /// `/` if the data partition is not yet mounted). Drives the
+    /// prune-vs-full-node decision in bitcoin-knots custom_args.
+    pub disk_gb: u64,
+}
+
+impl HostFacts {
+    /// Test-only constant fixture; do not use in production paths.
+    #[cfg(test)]
+    pub fn sample() -> Self {
+        Self {
+            host_ip: "192.168.1.116".to_string(),
+            host_mdns: "archi-thinkpad.local".to_string(),
+            disk_gb: 2000,
+        }
+    }
+}
+
+/// Supported placeholder names in `DerivedEnv::template`. Keep in sync
+/// with `HostFacts`. Centralized so validation and rendering agree.
+const DERIVED_PLACEHOLDERS: &[&str] = &["HOST_IP", "HOST_MDNS", "DISK_GB"];
+
+fn validate_derived_template(key: &str, template: &str) -> Result<(), ManifestError> {
+    // Walk `{{NAME}}` occurrences and ensure each NAME is recognized.
+    // Unbalanced braces are a user error.
+    let bytes = template.as_bytes();
+    let mut i = 0;
+    while i + 1 < bytes.len() {
+        if bytes[i] == b'{' && bytes[i + 1] == b'{' {
+            let rest = &template[i + 2..];
+            let close = rest.find("}}").ok_or_else(|| {
+                ManifestError::Invalid(format!(
+                    "container.derived_env['{key}'].template has unbalanced '{{{{' — no closing '}}}}'"
+                ))
+            })?;
+            let name = &rest[..close];
+            if !DERIVED_PLACEHOLDERS.contains(&name) {
+                return Err(ManifestError::Invalid(format!(
+                    "container.derived_env['{key}'].template references unknown placeholder '{{{{{name}}}}}' (supported: {})",
+                    DERIVED_PLACEHOLDERS.join(", ")
+                )));
+            }
+            i = i + 2 + close + 2;
+        } else {
+            i += 1;
+        }
+    }
+    Ok(())
+}
+
+/// A source of named secrets. In prod this is a directory on disk
+/// (`/var/lib/archipelago/secrets/`); in tests, a HashMap.
+pub trait SecretsProvider {
+    /// Read the named secret and return its value with trailing
+    /// whitespace trimmed (so `echo "…" > secret-file` works without
+    /// injecting a newline into env).
+    fn read(&self, name: &str) -> Result<String, ManifestError>;
+}
+
+impl ContainerConfig {
+    /// Collapse the (image, build) pair into a single resolved source.
+    ///
+    /// Returns `None` if the config is in an invalid state (e.g. neither field set
+    /// or both set). Callers should have already run `AppManifest::validate()` to
+    /// surface a user-facing error; this method is for internal orchestrator use
+    /// after validation has passed.
+    pub fn resolve(&self) -> Option<ResolvedSource> {
+        match (&self.image, &self.build) {
+            (Some(img), None) if !img.is_empty() => Some(ResolvedSource::Pull {
+                image: img.clone(),
+                pull_policy: self.pull_policy.clone(),
+                image_signature: self.image_signature.clone(),
+            }),
+            (None, Some(b)) => Some(ResolvedSource::Build(b.clone())),
+            _ => None,
+        }
+    }
+
+    /// The image reference used to create/inspect a container for this config.
+    ///
+    /// For Pull sources this is the registry image. For Build sources this is
+    /// the locally-built tag. Returns `None` only for an invalid config.
+    pub fn image_ref(&self) -> Option<String> {
+        self.resolve().map(|r| match r {
+            ResolvedSource::Pull { image, .. } => image,
+            ResolvedSource::Build(b) => b.tag,
+        })
+    }
+
+    /// Render every `derived_env` entry's template against the given
+    /// host facts. Returns `"KEY=VALUE"` strings ready to concatenate
+    /// with `environment:`.
+    ///
+    /// Assumes `AppManifest::validate()` has already accepted the
+    /// manifest — placeholder names are not re-checked here.
+    pub fn resolve_derived_env(&self, facts: &HostFacts) -> Vec<String> {
+        self.derived_env
+            .iter()
+            .map(|e| {
+                let value = e
+                    .template
+                    .replace("{{HOST_IP}}", &facts.host_ip)
+                    .replace("{{HOST_MDNS}}", &facts.host_mdns)
+                    .replace("{{DISK_GB}}", &facts.disk_gb.to_string());
+                format!("{}={}", e.key, value)
+            })
+            .collect()
+    }
+
+    /// Read every `secret_env` entry's value from the provider and
+    /// return `"KEY=VALUE"` strings. Propagates the provider error on
+    /// the first missing/unreadable secret — partial resolution is not
+    /// useful because it silently produces a misconfigured container.
+    pub fn resolve_secret_env(
+        &self,
+        provider: &dyn SecretsProvider,
+    ) -> Result<Vec<String>, ManifestError> {
+        let mut out = Vec::with_capacity(self.secret_env.len());
+        for e in &self.secret_env {
+            let v = provider.read(&e.secret_file)?;
+            out.push(format!("{}={}", e.key, v));
+        }
+        Ok(out)
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
+    use std::collections::HashMap;
+    use std::fs;
+    use std::path::{Path, PathBuf};
 
     #[test]
     fn test_manifest_parse() {
@@ -234,4 +663,464 @@ app:
         let result = AppManifest::parse(yaml);
         assert!(result.is_err());
     }
+
+    #[test]
+    fn pull_source_resolves_to_pull() {
+        let yaml = r#"
+app:
+  id: test-app
+  name: Test
+  version: 1.0.0
+  container:
+    image: docker.io/library/nginx:1.27
+    pull_policy: always
+"#;
+        let m = AppManifest::parse(yaml).unwrap();
+        let src = m.app.container.resolve().unwrap();
+        match src {
+            ResolvedSource::Pull {
+                image, pull_policy, ..
+            } => {
+                assert_eq!(image, "docker.io/library/nginx:1.27");
+                assert_eq!(pull_policy, "always");
+            }
+            _ => panic!("expected Pull"),
+        }
+        assert_eq!(
+            m.app.container.image_ref().as_deref(),
+            Some("docker.io/library/nginx:1.27")
+        );
+    }
+
+    #[test]
+    fn build_source_resolves_to_build() {
+        let yaml = r#"
+app:
+  id: bitcoin-ui
+  name: Bitcoin UI
+  version: 1.0.0
+  container:
+    build:
+      context: /opt/archipelago/docker/bitcoin-ui
+      dockerfile: Dockerfile
+      tag: archy-bitcoin-ui:local
+      build_args:
+        NGINX_VERSION: "1.27"
+"#;
+        let m = AppManifest::parse(yaml).unwrap();
+        let src = m.app.container.resolve().unwrap();
+        match src {
+            ResolvedSource::Build(b) => {
+                assert_eq!(b.context, "/opt/archipelago/docker/bitcoin-ui");
+                assert_eq!(b.dockerfile, "Dockerfile");
+                assert_eq!(b.tag, "archy-bitcoin-ui:local");
+                assert_eq!(b.build_args.get("NGINX_VERSION").unwrap(), "1.27");
+            }
+            _ => panic!("expected Build"),
+        }
+        assert_eq!(
+            m.app.container.image_ref().as_deref(),
+            Some("archy-bitcoin-ui:local")
+        );
+    }
+
+    #[test]
+    fn dockerfile_defaults_to_dockerfile() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    build:
+      context: /tmp
+      tag: x:local
+"#;
+        let m = AppManifest::parse(yaml).unwrap();
+        match m.app.container.resolve().unwrap() {
+            ResolvedSource::Build(b) => assert_eq!(b.dockerfile, "Dockerfile"),
+            _ => unreachable!(),
+        }
+    }
+
+    #[test]
+    fn image_and_build_both_set_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    image: foo:latest
+    build:
+      context: /tmp
+      tag: x:local
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(
+            msg.contains("mutually exclusive"),
+            "unexpected error: {msg}"
+        );
+    }
+
+    #[test]
+    fn neither_image_nor_build_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container: {}
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(
+            msg.contains("either image or build"),
+            "unexpected error: {msg}"
+        );
+    }
+
+    #[test]
+    fn empty_image_string_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    image: ""
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(
+            msg.contains("either image or build"),
+            "unexpected error: {msg}"
+        );
+    }
+
+    #[test]
+    fn empty_build_context_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    build:
+      context: ""
+      tag: x:local
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("context"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn empty_build_tag_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    build:
+      context: /tmp
+      tag: ""
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("tag"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn existing_pull_only_manifests_still_parse() {
+        // Backwards-compat smoke: the shape every file in apps/*/manifest.yml uses today.
+        let yaml = r#"
+app:
+  id: legacy
+  name: Legacy App
+  version: 0.1.0
+  description: existing shape
+  container:
+    image: registry.example.com/legacy:1.2.3
+    image_signature: sha256:abc
+  ports:
+    - { host: 8080, container: 80 }
+"#;
+        let m = AppManifest::parse(yaml).unwrap();
+        assert_eq!(m.app.container.pull_policy, "if-not-present");
+        matches!(
+            m.app.container.resolve().unwrap(),
+            ResolvedSource::Pull { .. }
+        );
+    }
+
+    #[test]
+    fn empty_custom_arg_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    image: foo:latest
+    custom_args: [""]
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("custom_args[0]"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn empty_entrypoint_vec_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    image: foo:latest
+    entrypoint: []
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("entrypoint"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn empty_entrypoint_element_is_rejected() {
+        let yaml = r#"
+app:
+  id: x
+  name: X
+  version: 1.0.0
+  container:
+    image: foo:latest
+    entrypoint: ["gatewayd", ""]
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("entrypoint[1]"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn duplicate_derived_env_keys_are_rejected() {
+        let yaml = r#"
+app:
+  id: fedimint
+  name: Fedimint
+  version: 0.10.0
+  container:
+    image: fedimintd:v0.10.0
+    derived_env:
+      - key: FM_API_URL
+        template: "ws://{{HOST_MDNS}}:8174"
+      - key: FM_API_URL
+        template: "ws://{{HOST_IP}}:8174"
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("duplicate key"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn unknown_derived_placeholder_is_rejected() {
+        let yaml = r#"
+app:
+  id: fedimint
+  name: Fedimint
+  version: 0.10.0
+  container:
+    image: fedimintd:v0.10.0
+    derived_env:
+      - key: FM_API_URL
+        template: "ws://{{HOSTNAME}}:8174"
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(
+            msg.contains("unknown placeholder"),
+            "unexpected error: {msg}"
+        );
+    }
+
+    #[test]
+    fn path_traversal_secret_file_is_rejected() {
+        let yaml = r#"
+app:
+  id: fedimint
+  name: Fedimint
+  version: 0.10.0
+  container:
+    image: fedimintd:v0.10.0
+    secret_env:
+      - key: FM_BITCOIND_PASSWORD
+        secret_file: "../bitcoin-rpc-password"
+"#;
+        let err = AppManifest::parse(yaml).unwrap_err();
+        let msg = format!("{err}");
+        assert!(msg.contains("bare filename"), "unexpected error: {msg}");
+    }
+
+    #[test]
+    fn resolve_derived_env_renders_host_facts() {
+        let c = ContainerConfig {
+            image: Some("x:latest".to_string()),
+            image_signature: None,
+            pull_policy: "if-not-present".to_string(),
+            build: None,
+            network: None,
+            custom_args: vec![],
+            entrypoint: None,
+            derived_env: vec![
+                DerivedEnv {
+                    key: "FM_API_URL".to_string(),
+                    template: "ws://{{HOST_MDNS}}:8174".to_string(),
+                },
+                DerivedEnv {
+                    key: "INFO".to_string(),
+                    template: "{{HOST_IP}}-{{DISK_GB}}".to_string(),
+                },
+            ],
+            secret_env: vec![],
+            data_uid: None,
+        };
+        let facts = HostFacts {
+            host_ip: "192.168.1.116".to_string(),
+            host_mdns: "archi-thinkpad.local".to_string(),
+            disk_gb: 2000,
+        };
+
+        let out = c.resolve_derived_env(&facts);
+        assert_eq!(out[0], "FM_API_URL=ws://archi-thinkpad.local:8174");
+        assert_eq!(out[1], "INFO=192.168.1.116-2000");
+    }
+
+    struct MapSecretsProvider {
+        data: HashMap<String, String>,
+    }
+
+    impl SecretsProvider for MapSecretsProvider {
+        fn read(&self, name: &str) -> Result<String, ManifestError> {
+            self.data
+                .get(name)
+                .cloned()
+                .ok_or_else(|| ManifestError::Invalid(format!("missing secret: {name}")))
+        }
+    }
+
+    #[test]
+    fn resolve_secret_env_reads_from_provider() {
+        let c = ContainerConfig {
+            image: Some("x:latest".to_string()),
+            image_signature: None,
+            pull_policy: "if-not-present".to_string(),
+            build: None,
+            network: None,
+            custom_args: vec![],
+            entrypoint: None,
+            derived_env: vec![],
+            secret_env: vec![
+                SecretEnv {
+                    key: "FM_BITCOIND_PASSWORD".to_string(),
+                    secret_file: "bitcoin-rpc-password".to_string(),
+                },
+                SecretEnv {
+                    key: "FM_GATEWAY_PASSWORD".to_string(),
+                    secret_file: "fedimint-gateway-password".to_string(),
+                },
+            ],
+            data_uid: None,
+        };
+        let p = MapSecretsProvider {
+            data: HashMap::from([
+                (
+                    "bitcoin-rpc-password".to_string(),
+                    "supersecret1".to_string(),
+                ),
+                (
+                    "fedimint-gateway-password".to_string(),
+                    "supersecret2".to_string(),
+                ),
+            ]),
+        };
+
+        let out = c.resolve_secret_env(&p).unwrap();
+        assert_eq!(out[0], "FM_BITCOIND_PASSWORD=supersecret1");
+        assert_eq!(out[1], "FM_GATEWAY_PASSWORD=supersecret2");
+    }
+
+    #[test]
+    fn parse_every_real_manifest() {
+        let app_manifests = list_repo_manifests();
+        assert!(
+            !app_manifests.is_empty(),
+            "no apps/*/manifest.yml files found"
+        );
+
+        let mut failures: Vec<String> = Vec::new();
+        let mut modern_count = 0usize;
+        let mut legacy_count = 0usize;
+        for path in app_manifests {
+            let content = fs::read_to_string(&path).expect("read manifest");
+            let parsed_yaml: serde_yaml::Value = match serde_yaml::from_str(&content) {
+                Ok(v) => v,
+                Err(err) => {
+                    failures.push(format!("{}: YAML parse error: {err}", path.display()));
+                    continue;
+                }
+            };
+
+            let is_modern = parsed_yaml
+                .as_mapping()
+                .map(|m| m.contains_key(serde_yaml::Value::String("app".to_string())))
+                .unwrap_or(false);
+
+            if is_modern {
+                modern_count += 1;
+                if let Err(err) = AppManifest::parse(&content) {
+                    failures.push(format!("{}: {err}", path.display()));
+                }
+            } else {
+                legacy_count += 1;
+            }
+        }
+
+        assert!(modern_count > 0, "no modern app-schema manifests found");
+        assert!(
+            legacy_count > 0,
+            "expected at least one legacy manifest shape"
+        );
+
+        assert!(
+            failures.is_empty(),
+            "manifest parse failures:\n{}",
+            failures.join("\n")
+        );
+    }
+
+    fn list_repo_manifests() -> Vec<PathBuf> {
+        let repo_root = Path::new(env!("CARGO_MANIFEST_DIR")).join("..").join("..");
+        let apps_dir = repo_root.join("apps");
+        let mut out = Vec::new();
+
+        let Ok(entries) = fs::read_dir(apps_dir) else {
+            return out;
+        };
+
+        for entry in entries.flatten() {
+            let path = entry.path();
+            if !path.is_dir() {
+                continue;
+            }
+            let manifest = path.join("manifest.yml");
+            if manifest.exists() {
+                out.push(manifest);
+            }
+        }
+
+        out.sort();
+        out
+    }
 }

core/container/src/podman_client.rs

@@ -126,7 +126,7 @@ impl PodmanClient {
             "filebrowser" => "http://localhost:8083",
             "nginx-proxy-manager" => "http://localhost:81",
             "portainer" => "http://localhost:9000",
-            "uptime-kuma" => "http://localhost:3001",
+            "uptime-kuma" => "http://localhost:3002",
             "fedimint" | "fedimintd" => "http://localhost:8175",
             "fedimint-gateway" => "http://localhost:8176",
             "nostr-rs-relay" => "http://localhost:18081",
@@ -288,12 +288,29 @@ impl PodmanClient {
 
         let mut mounts = Vec::new();
         for volume in &manifest.app.volumes {
-            mounts.push(serde_json::json!({
-                "destination": volume.target,
-                "source": volume.source,
-                "type": "bind",
-                "options": volume.options,
-            }));
+            if volume.volume_type == "tmpfs" {
+                let options: Vec<String> = volume
+                    .tmpfs_options
+                    .as_deref()
+                    .unwrap_or("")
+                    .split(',')
+                    .map(str::trim)
+                    .filter(|s| !s.is_empty())
+                    .map(|s| s.to_string())
+                    .collect();
+                mounts.push(serde_json::json!({
+                    "destination": volume.target,
+                    "type": "tmpfs",
+                    "options": options,
+                }));
+            } else {
+                mounts.push(serde_json::json!({
+                    "destination": volume.target,
+                    "source": volume.source,
+                    "type": "bind",
+                    "options": volume.options,
+                }));
+            }
         }
 
         let mut env_map = serde_json::Map::new();
@@ -306,29 +323,66 @@ impl PodmanClient {
         let cap_add: Vec<String> = manifest.app.security.capabilities.clone();
         let cap_drop = vec!["ALL".to_string()];
 
+        let image_ref = manifest.app.container.image_ref().ok_or_else(|| {
+            anyhow::anyhow!(
+                "container config for {} has neither a valid image nor build source",
+                manifest.app.id
+            )
+        })?;
+
+        // Build resource_limits conditionally: if the manifest has no memory or
+        // cpu limit, OMIT the field entirely rather than sending 0. The podman
+        // libpod HTTP API treats `memory.limit: 0` as "set MemoryMax=0" which
+        // systemd then rejects at container-start time. Absent = unlimited.
+        let mut resource_limits = serde_json::Map::new();
+        if let Some(mem_bytes) = manifest
+            .app
+            .resources
+            .memory_limit
+            .as_ref()
+            .and_then(|m| parse_memory_limit(m))
+        {
+            resource_limits.insert(
+                "memory".to_string(),
+                serde_json::json!({ "limit": mem_bytes }),
+            );
+        }
+        if let Some(cpu) = manifest.app.resources.cpu_limit {
+            resource_limits.insert(
+                "cpu".to_string(),
+                serde_json::json!({
+                    "quota": (cpu as i64) * 100_000,
+                    "period": 100_000u64,
+                }),
+            );
+        }
+
+        let net_mode = if let Some(n) = manifest.app.container.network.as_ref() {
+            if n.is_empty() {
+                "bridge"
+            } else {
+                n.as_str()
+            }
+        } else {
+            match manifest.app.security.network_policy.as_str() {
+                "host" => "host",
+                _ => "bridge",
+            }
+        };
+
         let body = serde_json::json!({
             "name": name,
-            "image": manifest.app.container.image,
+            "image": image_ref,
             "portmappings": port_mappings,
             "mounts": mounts,
             "env": env_map,
+            "entrypoint": manifest.app.container.entrypoint.clone(),
+            "command": manifest.app.container.custom_args.clone(),
             "hostadd": ["host.containers.internal:host-gateway"],
             "devices": manifest.app.devices.iter().map(|d| {
                 serde_json::json!({"path": d})
             }).collect::<Vec<_>>(),
-            "resource_limits": {
-                "memory": {
-                    "limit": manifest.app.resources.memory_limit.as_ref()
-                        .and_then(|m| parse_memory_limit(m))
-                        .unwrap_or(0),
-                },
-                "cpu": {
-                    "quota": manifest.app.resources.cpu_limit
-                        .map(|c| (c as i64) * 100000)
-                        .unwrap_or(0),
-                    "period": 100000u64,
-                }
-            },
+            "resource_limits": resource_limits,
             "cap_add": cap_add,
             "cap_drop": cap_drop,
             "read_only_filesystem": manifest.app.security.readonly_root,
@@ -336,10 +390,7 @@ impl PodmanClient {
             "restart_policy": "unless-stopped",
             "restart_tries": 5,
             "netns": {
-                "nsmode": match manifest.app.security.network_policy.as_str() {
-                    "host" => "host",
-                    _ => "bridge",
-                }
+                "nsmode": net_mode
             },
         });
 
@@ -571,26 +622,106 @@ fn parse_port_bindings(bindings: &serde_json::Value) -> Vec<String> {
 }
 
 fn parse_memory_limit(limit: &str) -> Option<i64> {
-    let limit = limit.trim().to_lowercase();
-    if limit.ends_with('g') {
-        limit
-            .trim_end_matches('g')
-            .parse::<f64>()
-            .ok()
-            .map(|v| (v * 1_073_741_824.0) as i64)
-    } else if limit.ends_with('m') {
-        limit
-            .trim_end_matches('m')
-            .parse::<f64>()
-            .ok()
-            .map(|v| (v * 1_048_576.0) as i64)
-    } else if limit.ends_with('k') {
-        limit
-            .trim_end_matches('k')
-            .parse::<f64>()
-            .ok()
-            .map(|v| (v * 1024.0) as i64)
-    } else {
-        limit.parse::<i64>().ok()
+    // Supports the Kubernetes-style suffixes used throughout apps/*/manifest.yml
+    // (IEC binary: Ki/Mi/Gi/Ti) as well as the shorter docker-style k/m/g/t.
+    // Longest suffix matched first so "Mi" isn't mis-matched as "m".
+    //
+    // Historical bug: we used to lowercase+trim_end_matches('m'), which turned
+    // "128Mi" into "128i" → parse::<f64> failed → None → .unwrap_or(0) wrote
+    // memory.limit:0 into the OCI spec, which systemd then rejected at start
+    // time with "MemoryMax is out of range" on rootless podman. See
+    // docs/rust-orchestrator-migration.md Step 9 notes.
+    let trimmed = limit.trim();
+    if trimmed.is_empty() {
+        return None;
+    }
+    const UNITS: &[(&str, i64)] = &[
+        ("Ki", 1024),
+        ("Mi", 1024 * 1024),
+        ("Gi", 1024 * 1024 * 1024),
+        ("Ti", 1024i64 * 1024 * 1024 * 1024),
+        ("kB", 1000),
+        ("MB", 1_000_000),
+        ("GB", 1_000_000_000),
+        ("TB", 1_000_000_000_000),
+        ("k", 1024),
+        ("K", 1024),
+        ("m", 1024 * 1024),
+        ("M", 1024 * 1024),
+        ("g", 1024 * 1024 * 1024),
+        ("G", 1024 * 1024 * 1024),
+        ("t", 1024i64 * 1024 * 1024 * 1024),
+        ("T", 1024i64 * 1024 * 1024 * 1024),
+        ("b", 1),
+        ("B", 1),
+    ];
+    for (suffix, multiplier) in UNITS {
+        if let Some(num) = trimmed.strip_suffix(suffix) {
+            let num = num.trim();
+            return num
+                .parse::<f64>()
+                .ok()
+                .map(|v| (v * (*multiplier as f64)) as i64)
+                .filter(|n| *n > 0);
+        }
+    }
+    // No recognised suffix — treat as raw bytes.
+    trimmed.parse::<i64>().ok().filter(|n| *n > 0)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn parse_memory_limit_iec_binary_suffixes() {
+        // Kubernetes-style — this is what apps/*/manifest.yml uses.
+        assert_eq!(parse_memory_limit("128Mi"), Some(128 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("64Mi"), Some(64 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("4Gi"), Some(4i64 * 1024 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("512Ki"), Some(512 * 1024));
+    }
+
+    #[test]
+    fn parse_memory_limit_shorthand_suffixes() {
+        // Docker-style shorthand — treated as IEC binary for backwards compat.
+        assert_eq!(parse_memory_limit("128m"), Some(128 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("128M"), Some(128 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("2g"), Some(2i64 * 1024 * 1024 * 1024));
+        assert_eq!(parse_memory_limit("2G"), Some(2i64 * 1024 * 1024 * 1024));
+    }
+
+    #[test]
+    fn parse_memory_limit_si_decimal_suffixes() {
+        assert_eq!(parse_memory_limit("1MB"), Some(1_000_000));
+        assert_eq!(parse_memory_limit("1GB"), Some(1_000_000_000));
+    }
+
+    #[test]
+    fn parse_memory_limit_raw_bytes() {
+        assert_eq!(parse_memory_limit("134217728"), Some(134_217_728));
+        assert_eq!(parse_memory_limit("  134217728  "), Some(134_217_728));
+    }
+
+    #[test]
+    fn parse_memory_limit_invalid_returns_none() {
+        // Regression guard: the old implementation returned Some(0) for "128Mi"
+        // because lowercase+trim_end_matches('m') left "128i" which parse::<f64>
+        // rejected. The new implementation must never return Some(0) or Some of
+        // a negative number from any input.
+        assert_eq!(parse_memory_limit(""), None);
+        assert_eq!(parse_memory_limit("   "), None);
+        assert_eq!(parse_memory_limit("abc"), None);
+        assert_eq!(parse_memory_limit("0"), None);
+        assert_eq!(parse_memory_limit("0Mi"), None);
+        assert_eq!(parse_memory_limit("-1Mi"), None);
+    }
+
+    #[test]
+    fn parse_memory_limit_tolerates_whitespace_and_fractional() {
+        assert_eq!(
+            parse_memory_limit(" 1.5Gi "),
+            Some((1.5 * (1024.0 * 1024.0 * 1024.0)) as i64)
+        );
     }
 }

core/container/src/runtime.rs

@@ -1,4 +1,4 @@
-use crate::manifest::AppManifest;
+use crate::manifest::{AppManifest, BuildConfig};
 use crate::podman_client::{ContainerState, ContainerStatus, PodmanClient};
 use anyhow::{Context, Result};
 use async_trait::async_trait;
@@ -20,6 +20,22 @@ pub trait ContainerRuntime: Send + Sync {
     async fn get_container_status(&self, name: &str) -> Result<ContainerStatus>;
     async fn get_container_logs(&self, name: &str, lines: u32) -> Result<Vec<String>>;
     async fn list_containers(&self) -> Result<Vec<ContainerStatus>>;
+
+    /// Check whether an image reference exists in local storage.
+    ///
+    /// The reconciler calls this before deciding to build. `true` means
+    /// `image inspect <image_ref>` succeeded (or equivalent); `false` means
+    /// the image is not present. Registry/network state is explicitly NOT
+    /// consulted — this is a local-storage check only.
+    async fn image_exists(&self, image_ref: &str) -> Result<bool>;
+
+    /// Build a local image from a `BuildConfig`.
+    ///
+    /// Equivalent to `podman build -t <tag> -f <dockerfile> [--build-arg K=V ...] <context>`.
+    /// The resulting image is referenceable by `config.tag` for subsequent
+    /// `create_container` / `image_exists` calls. Stdout/stderr are collected
+    /// and included in the error on failure; on success they are discarded.
+    async fn build_image(&self, config: &BuildConfig) -> Result<()>;
 }
 
 pub struct PodmanRuntime {
@@ -32,6 +48,17 @@ impl PodmanRuntime {
             client: PodmanClient::new(user),
         }
     }
+
+    /// Run `podman <args>`, returning an error with captured stderr on non-zero
+    /// exit. Used for operations (build, image inspect) that are awkward over the
+    /// HTTP API. The daemon runs as the target user already, so no sudo hop.
+    async fn podman_cli(&self, args: &[&str]) -> Result<std::process::Output> {
+        let mut cmd = TokioCommand::new("podman");
+        cmd.args(args);
+        cmd.output()
+            .await
+            .with_context(|| format!("failed to execute podman {}", args.join(" ")))
+    }
 }
 
 #[async_trait]
@@ -79,6 +106,68 @@ impl ContainerRuntime for PodmanRuntime {
     async fn list_containers(&self) -> Result<Vec<ContainerStatus>> {
         self.client.list_containers().await
     }
+
+    async fn image_exists(&self, image_ref: &str) -> Result<bool> {
+        // `podman image exists` returns 0 if present, 1 if absent. Any other
+        // exit code is an environment failure we should surface.
+        let output = self.podman_cli(&["image", "exists", image_ref]).await?;
+        match output.status.code() {
+            Some(0) => Ok(true),
+            Some(1) => Ok(false),
+            Some(code) => {
+                let stderr = String::from_utf8_lossy(&output.stderr);
+                Err(anyhow::anyhow!(
+                    "podman image exists {image_ref} exited with {code}: {stderr}"
+                ))
+            }
+            None => Err(anyhow::anyhow!(
+                "podman image exists {image_ref} terminated by signal"
+            )),
+        }
+    }
+
+    async fn build_image(&self, config: &BuildConfig) -> Result<()> {
+        let args = build_args_for_podman(config);
+        let borrowed: Vec<&str> = args.iter().map(|s| s.as_str()).collect();
+        let output = self.podman_cli(&borrowed).await?;
+        if !output.status.success() {
+            let stderr = String::from_utf8_lossy(&output.stderr);
+            let stdout = String::from_utf8_lossy(&output.stdout);
+            return Err(anyhow::anyhow!(
+                "podman build -t {} failed: {stderr}{}{stdout}",
+                config.tag,
+                if stderr.is_empty() || stdout.is_empty() {
+                    ""
+                } else {
+                    "\n---stdout---\n"
+                }
+            ));
+        }
+        Ok(())
+    }
+}
+
+/// Build the argv for `podman build` from a BuildConfig.
+///
+/// Extracted so it can be unit-tested without actually invoking podman.
+/// Order is fixed for deterministic tests: subcommand, -t, -f, build-args
+/// (sorted by key), context.
+fn build_args_for_podman(config: &BuildConfig) -> Vec<String> {
+    let mut args: Vec<String> = vec![
+        "build".to_string(),
+        "-t".to_string(),
+        config.tag.clone(),
+        "-f".to_string(),
+        config.dockerfile.clone(),
+    ];
+    let mut kv: Vec<(&String, &String)> = config.build_args.iter().collect();
+    kv.sort_by(|a, b| a.0.cmp(b.0));
+    for (k, v) in kv {
+        args.push("--build-arg".to_string());
+        args.push(format!("{k}={v}"));
+    }
+    args.push(config.context.clone());
+    args
 }
 
 pub struct DockerRuntime {
@@ -188,7 +277,13 @@ impl ContainerRuntime for DockerRuntime {
             cmd.arg("--cap-add").arg(cap);
         }
 
-        cmd.arg(&manifest.app.container.image);
+        let image_ref = manifest.app.container.image_ref().ok_or_else(|| {
+            anyhow::anyhow!(
+                "container config for {} has neither a valid image nor build source",
+                manifest.app.id
+            )
+        })?;
+        cmd.arg(&image_ref);
 
         let output = cmd.output().await.context("Failed to create container")?;
 
@@ -344,6 +439,55 @@ impl ContainerRuntime for DockerRuntime {
 
         Ok(result)
     }
+
+    async fn image_exists(&self, image_ref: &str) -> Result<bool> {
+        // `docker image inspect` exits 1 when the image is absent. Any message
+        // to stderr in that case is informational; we swallow it.
+        let mut cmd = self.docker_async();
+        cmd.arg("image").arg("inspect").arg(image_ref);
+        let output = cmd
+            .output()
+            .await
+            .context("failed to execute docker image inspect")?;
+        match output.status.code() {
+            Some(0) => Ok(true),
+            Some(1) => Ok(false),
+            Some(code) => {
+                let stderr = String::from_utf8_lossy(&output.stderr);
+                Err(anyhow::anyhow!(
+                    "docker image inspect {image_ref} exited with {code}: {stderr}"
+                ))
+            }
+            None => Err(anyhow::anyhow!(
+                "docker image inspect {image_ref} terminated by signal"
+            )),
+        }
+    }
+
+    async fn build_image(&self, config: &BuildConfig) -> Result<()> {
+        let mut cmd = self.docker_async();
+        cmd.arg("build")
+            .arg("-t")
+            .arg(&config.tag)
+            .arg("-f")
+            .arg(&config.dockerfile);
+        for (k, v) in &config.build_args {
+            cmd.arg("--build-arg").arg(format!("{k}={v}"));
+        }
+        cmd.arg(&config.context);
+        let output = cmd
+            .output()
+            .await
+            .context("failed to execute docker build")?;
+        if !output.status.success() {
+            let stderr = String::from_utf8_lossy(&output.stderr);
+            return Err(anyhow::anyhow!(
+                "docker build -t {} failed: {stderr}",
+                config.tag
+            ));
+        }
+        Ok(())
+    }
 }
 
 pub struct AutoRuntime {
@@ -415,7 +559,91 @@ impl ContainerRuntime for AutoRuntime {
     async fn list_containers(&self) -> Result<Vec<ContainerStatus>> {
         self.runtime.list_containers().await
     }
+
+    async fn image_exists(&self, image_ref: &str) -> Result<bool> {
+        self.runtime.image_exists(image_ref).await
+    }
+
+    async fn build_image(&self, config: &BuildConfig) -> Result<()> {
+        self.runtime.build_image(config).await
+    }
 }
 
 // Runtime factory functions will be provided by the archipelago crate
 // that imports this library and has access to Config
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use std::collections::HashMap;
+
+    fn cfg(context: &str, tag: &str, dockerfile: &str, args: &[(&str, &str)]) -> BuildConfig {
+        BuildConfig {
+            context: context.to_string(),
+            dockerfile: dockerfile.to_string(),
+            tag: tag.to_string(),
+            build_args: args
+                .iter()
+                .map(|(k, v)| (k.to_string(), v.to_string()))
+                .collect::<HashMap<_, _>>(),
+        }
+    }
+
+    #[test]
+    fn build_args_minimal() {
+        let c = cfg("/tmp/ctx", "archy-bitcoin-ui:local", "Dockerfile", &[]);
+        assert_eq!(
+            build_args_for_podman(&c),
+            vec![
+                "build",
+                "-t",
+                "archy-bitcoin-ui:local",
+                "-f",
+                "Dockerfile",
+                "/tmp/ctx",
+            ]
+        );
+    }
+
+    #[test]
+    fn build_args_custom_dockerfile() {
+        let c = cfg("/opt/archy/bitcoin-ui", "x:local", "Dockerfile.prod", &[]);
+        let got = build_args_for_podman(&c);
+        assert_eq!(got[3], "-f");
+        assert_eq!(got[4], "Dockerfile.prod");
+        assert_eq!(got.last().unwrap(), "/opt/archy/bitcoin-ui");
+    }
+
+    #[test]
+    fn build_args_are_sorted_deterministically() {
+        // HashMap iteration order is nondeterministic; the runtime sorts so that
+        // equivalent BuildConfigs produce identical commands (easier to debug,
+        // cache-friendly if we ever layer build-cache keys on top).
+        let c = cfg(
+            "/c",
+            "t",
+            "Dockerfile",
+            &[("BAR", "2"), ("FOO", "1"), ("BAZ", "3")],
+        );
+        let args = build_args_for_podman(&c);
+        let flat: Vec<&str> = args.iter().map(|s| s.as_str()).collect();
+        // Build args appear as pairs of --build-arg K=V; locate them:
+        let mut pairs: Vec<&str> = Vec::new();
+        for w in flat.windows(2) {
+            if w[0] == "--build-arg" {
+                pairs.push(w[1]);
+            }
+        }
+        assert_eq!(pairs, vec!["BAR=2", "BAZ=3", "FOO=1"]);
+    }
+
+    #[test]
+    fn build_args_context_is_last() {
+        // Context MUST be the final positional argument — podman treats any
+        // stray trailing arg after build-args as the context, so placement
+        // matters. Regression guard.
+        let c = cfg("/final/context", "t", "Dockerfile", &[("K", "V")]);
+        let args = build_args_for_podman(&c);
+        assert_eq!(args.last().unwrap(), "/final/context");
+    }
+}

docker/bitcoin-ui/Dockerfile

@@ -1,9 +1,22 @@
 FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+# Static site content.
 COPY index.html /usr/share/nginx/html/
 COPY 50x.html /usr/share/nginx/html/
 COPY assets/ /usr/share/nginx/html/assets/
-COPY nginx.conf /etc/nginx/conf.d/default.conf
-# Run nginx as root to avoid chown failures in rootless Podman user namespaces
+#
+# NOTE: /etc/nginx/conf.d/default.conf is intentionally NOT copied from
+# this build context. It is bind-mounted at container-create time from
+# /var/lib/archipelago/bitcoin-ui/nginx.conf on the host, which the
+# archipelago prod orchestrator renders with the current base64 RPC
+# auth substituted in (see core/archipelago/src/container/bitcoin_ui.rs).
+#
+# If the bind-mount fails nginx will start with no site configured and
+# return 404 on every request. That's the intended safe failure mode —
+# better than baking a placeholder into the image and potentially
+# serving the upstream RPC proxy with a stale/empty Authorization header.
+#
+# Run nginx as root to avoid chown failures in rootless Podman user
+# namespaces. The rest of the nginx image is unchanged.
 RUN sed -i 's/^user  nginx;/user  root;/' /etc/nginx/nginx.conf && \
     mkdir -p /var/cache/nginx/client_temp /var/cache/nginx/proxy_temp \
              /var/cache/nginx/fastcgi_temp /var/cache/nginx/uwsgi_temp \

docker/lnd-ui/Dockerfile

@@ -22,6 +22,6 @@ RUN sed -i 's/^user  nginx;/user  root;/' /etc/nginx/nginx.conf && \
     mkdir -p /var/cache/nginx/client_temp /var/cache/nginx/proxy_temp \
              /var/cache/nginx/fastcgi_temp /var/cache/nginx/uwsgi_temp \
              /var/cache/nginx/scgi_temp
-EXPOSE 8080
+EXPOSE 80
 ENTRYPOINT []
 CMD ["nginx", "-g", "daemon off;"]

docker/lnd-ui/nginx.conf

@@ -1,5 +1,5 @@
 server {
-    listen 8081;
+    listen 80;
     server_name _;
 
     root /usr/share/nginx/html;

docs/BETA-RELEASE-CHECKLIST.md

@@ -20,7 +20,6 @@
 - [ ] `core/security/src/secrets_manager.rs` — encryption + rotation
 - [ ] `neode-ui/src/views/Marketplace.vue` — all app entries with pinned image versions
 - [ ] `neode-ui/src/api/websocket.ts` — heartbeat + reconnection
-- [ ] `image-recipe/build-auto-installer-iso.sh` — all container images captured
 - [ ] `image-recipe/configs/nginx-archipelago.conf` — all app proxies + path traversal blocks
 - [ ] All app icons present in `neode-ui/public/assets/img/app-icons/`
 

docs/MARKETPLACE-QA.md

@@ -0,0 +1,58 @@
+# Marketplace QA — app-by-app install walk
+
+Purpose: track install/launch/uninstall health for every app in the marketplace catalog on `.228`. User installs each app one by one; for each broken one we triage, fix at the right layer (app recipe / registry image / backend / frontend), commit, redeploy, and re-verify.
+
+Target build: `v1.7.43-alpha` + backend md5 `9b8ead06aaf210b85cd78fce270384e3` (image-versions path fix included).
+
+## Status key
+
+- ✅ install, launch, uninstall all clean
+- ⚠️  installs and runs but has cosmetic or partial issues (note in details)
+- ❌ broken — fix needed
+- ⏳ pending verification
+
+## Catalog
+
+Pull the authoritative list from Marketplace page on `.228` during the walk. Fill in as you go.
+
+| App | Status | Notes / fix applied |
+|---|---|---|
+| _(to be filled during walk)_ | ⏳ | |
+
+## Known issues going in
+
+- **Vaultwarden** — container exits immediately on start. Pre-existing. Backend async wrapper correctly detects + removes the install state entry. Needs container-config investigation (image pin / env vars / volume layout).
+
+## Fix layers cheat-sheet
+
+When an app breaks, identify which layer to fix at:
+
+1. **App recipe** — `apps/<app>/package.yaml` or wherever the Podman manifest lives. Ports, volumes, env vars, healthcheck, resource caps.
+2. **Registry image** — if image itself is missing/wrong-tag on `.168`:3000/lfg2025 or `git.tx1138.com`. Push corrected image, bump `scripts/image-versions.sh`.
+3. **Backend orchestrator** — `core/archipelago/src/container/` or `core/archipelago/src/api/rpc/package/` if the install flow mishandles this app's shape.
+4. **Frontend** — `neode-ui/src/views/marketplace/` or curated data in `neode-ui/src/views/marketplace/marketplaceData.ts` if catalog entry is wrong or UI can't render this app correctly.
+
+## Per-app fix workflow
+
+For each broken app:
+
+1. Capture failure mode:
+   ```
+   ssh archy228 'sudo journalctl -u archipelago --since "5 minutes ago" --no-pager | tail -80'
+   ssh archy228 'podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}" | grep <app>'
+   ssh archy228 'podman logs <container-name> 2>&1 | tail -60'
+   ```
+2. Diagnose — which layer.
+3. Fix in repo (use SSHFS mount for edits).
+4. `cargo check` if backend changed; `npm run build` if frontend changed.
+5. Commit with `fix(app/<name>): ...` or `fix(registry/<image>): ...` etc.
+6. Redeploy as needed (binary via Mac ferry; frontend via rsync; registry via podman push).
+7. User re-verifies on `.228`. Mark ✅.
+
+## Release-notes policy
+
+For each app fix, append a bullet to the current in-flight release entry in `neode-ui/src/views/settings/AccountInfoSection.vue`. If the fix pile gets large enough to warrant its own release, bump to v1.7.44-alpha and start a new block at the top. Keep entries operator-focused ("Nostr Relay no longer crashes on first start"), not implementation-focused.
+
+## Running log
+
+_Add dated notes here as we progress through the catalog._

docs/RESUME.md

@@ -0,0 +1,126 @@
+# RESUME — Rust orchestrator migration, Step 8b
+
+Last updated: 2026-04-23 (evening, post-architecture-audit)
+
+Read this first if you're a fresh OpenCode session resuming work. Paste the "Resume prompt" below verbatim.
+
+---
+
+## Resume prompt (paste this into a new opencode session)
+
+> We are mid-migration: `docs/rust-orchestrator-migration.md` + `docs/bulletproof-containers.md` are the plan, Steps 1–7 + 8a are shipped on `main`, Step 8b is next. Read `docs/RESUME.md` + `docs/STEP-8B-PORT-AUDIT.md` in full. Do NOT run any container mutations or edit `scripts/container-specs.sh`, `scripts/first-boot-containers.sh`, or `scripts/reconcile-containers.sh` — those are dead code scheduled for deletion in Step 8c. Work happens in `core/container/src/manifest.rs`, `core/archipelago/src/container/prod_orchestrator.rs`, and `apps/<id>/manifest.yml`. Summarize back to me what you understand the current state to be, wait for approval before touching anything.
+
+---
+
+## Standing directive from the user
+
+> Please get back to a well architected, minimal as possible, perfect working container architecture. If we've gone off track and the system is getting complex rather than elegant and perfect best containers ever then we need to review all the current state of the system and get back to making the best container system ever and according to our projects goals. We will be working on this until it's perfect.
+
+**Interpretation (validated with the user):** resume the Rust orchestrator migration. Stop patching bash scripts. The bash scripts were supposed to be deleted three months of commits ago and we drifted into maintaining them by accident.
+
+## Latest user comment (must be followed)
+
+> please continue, please state my last comment in the resume doc and first before making this plan to adhere to
+
+Adherence rule for this session:
+- Before proposing or executing a plan, first record the user's latest directive in `docs/RESUME.md`.
+- Keep work aligned to Step 8 migration goals and avoid off-scope drift.
+
+Most recent directive:
+
+> And we need to get every container working on .116 and tested before we release
+
+Release gate update:
+- `.116` must have all required containers healthy and tested before release is allowed.
+- Treat runtime stabilization on `.116` as immediate priority while continuing Step 8 migration work.
+
+---
+
+## Where we actually are
+
+### Shipped (Steps 1–7 + 8a)
+
+Commits on `main` (unpushed to `origin`/tx1138 until release gate; user-visible history):
+
+| Step | Commit | What |
+|------|--------|------|
+| 1    | (schema in place from earlier commits) | `ContainerConfig.image` ⊕ `ContainerConfig.build` — mutually exclusive pull-or-build source |
+| 2    | `34af4d9d` | `ContainerRuntime` trait gains `image_exists` + `build_image`; `PodmanRuntime` impl |
+| 3    | `b6a04d31` | `ProdContainerOrchestrator` with build-or-pull + adoption + reconcile |
+| 4    | `e8a59c93` | `ContainerOrchestrator` trait; `RpcHandler` uses it in prod |
+| 5    | `fc39b04b` | `BootReconciler` — periodic reconcile loop |
+| 6    | `48f08aa3` | Wire both into `main.rs` |
+| 7    | `069bc4a5` | `bitcoin-ui` pre-start hook renders `nginx.conf` from embedded template (the pattern for "derived config" at apply time) |
+| 8a   | `a0707f4d`, `1c81a739` | Retire `archipelago-reconcile` systemd timer; split Step 8 into 8a/8b/8c |
+
+Three `apps/*/manifest.yml` are genuinely ported and running under the Rust orchestrator on `.116` + `.228`: `bitcoin-ui`, `electrs-ui`, `lnd-ui` (Step 7).
+
+### Where we drifted (the session that produced the previous RESUME.md)
+
+On 2026-04-23 a fedimint outage on `.116` pulled a session into patching `scripts/reconcile-containers.sh`, `scripts/container-specs.sh`, `scripts/first-boot-containers.sh` — files that Step 8c is scheduled to delete. Five bugs deep, the user halted the session. That cluster of bugs is a symptom of running two incompatible codepaths in parallel (bash first-boot/reconcile + Rust `BootReconciler`), which is exactly the condition Step 8c fixes by deleting the bash half.
+
+**Discard-of-scope decision:** the uncommitted bash edits on `.116` (listed in the previous RESUME.md's "Uncommitted script changes" section) are not going to be committed. The fedimint mDNS-URLs fix, the filebrowser custom-args fix, the bcrypt-escape fix — these all land as changes to `apps/<id>/manifest.yml` + the Rust orchestrator in Steps 8b.0 – 8b.3. See `docs/STEP-8B-PORT-AUDIT.md` for the exact mapping.
+
+### Current container state on `.116`
+
+Running but drifted. See the "Current container state" section in the previous RESUME.md. Decision (approved by user): accept `.116` is limping until 8b.3 lands. Do not run `scripts/reconcile-containers.sh` or any mutations; all rescues go through the Rust orchestrator or wait for the manifest port.
+
+`.228` is happier — it's already adopted by the Rust orchestrator for the three UI apps.
+
+---
+
+## Next step — Step 8b.0
+
+**Concretely:** schema extensions to `core/container/src/manifest.rs` + unit tests. No orchestrator changes, no manifest changes, no container mutations.
+
+Fields to add (justified in `docs/STEP-8B-PORT-AUDIT.md§Schema gaps`):
+
+- `container.network: Option<String>` — podman `--network` value (`"archy-net"`, `"host"`, or `None` = isolated default).
+- `container.custom_args: Vec<String>` — appended to the container command.
+- `container.entrypoint: Option<Vec<String>>` — override.
+- `container.derived_env: Vec<{key, template}>` — template strings resolved against `HostFacts { host_ip, host_mdns, disk_gb }` at apply time.
+- `container.secret_env: Vec<{key, secret_file}>` — read from `/var/lib/archipelago/secrets/<file>` at apply time.
+- `container.data_uid: Option<String>` — `"NNNNN:NNNNN"` applied via `chown -R` before container create.
+- `Volume.volume_type: "tmpfs"` + `Volume.tmpfs_options: String` — OR a new `container.tmpfs: Vec<{target, options}>`. Pick one at implementation time.
+
+**Tests** (block the commit until green):
+
+- Every existing `apps/*/manifest.yml` still parses (`parse_every_real_manifest` test).
+- Each new field parses correctly with sensible defaults.
+- `validate()` rejects: empty custom_args elements, empty entrypoint elements, duplicate derived_env keys, derived_env templates referencing unknown host facts, secret_env with `..` or `/` in secret_file (path-traversal guard).
+- `resolve_env(HostFacts)` returns expected strings for each supported placeholder.
+- `resolve_secret_env(SecretsProvider)` returns expected strings; missing secret file is a hard error.
+
+This is the smallest useful commit and unblocks every port in 8b.1+.
+
+---
+
+## Project ground rules (standing)
+
+- `archy` SSH alias = `.116`. `archy228` = `.228`. **Do not swap.**
+- SSHFS at `/Users/dorian/mnt/archy-thinkpad/` = `archy:Projects/archy/`.
+- `.116` sudo password: `ThisIsWeb54321@` — works passwordless in-session via `sudo -nS` after first use.
+- `.228` has NOPASSWD.
+- Git commits on `.116` MUST use `git commit -F /tmp/tmp-msg.txt` over `ssh archy` — SSHFS `git commit` hangs.
+- Never push except current release (granted: `gitea-local` + `gitea-vps2`).
+- No em-dashes. Conventional Commits.
+- No altcoin mentions, Bitcoin-only.
+
+---
+
+## Recommended next action for the fresh session
+
+1. Read this file + `docs/STEP-8B-PORT-AUDIT.md` + the "Open decisions" section of the audit.
+2. Answer the four open decisions (or confirm the recommended defaults).
+3. Implement 8b.0 commit 1: add `network`, `custom_args`, `entrypoint`, `derived_env`, `secret_env`, `data_uid` fields to `ContainerConfig` + validation + unit tests. Backwards-compat: every existing `apps/*/manifest.yml` must still parse.
+4. Commit + `cargo test -p archipelago-container` + stop.
+
+Do not touch `scripts/*.sh`. Do not run `reconcile-containers.sh`. Do not live-test on `.116` or `.228` until the schema + orchestrator pieces in 8b.0 + 8b.1 are both in.
+
+---
+
+## Recent release (out of scope, for grep context)
+
+v1.7.43-alpha shipped yesterday: tarball-only OTA, async install/uninstall/update lifecycle, install UX polish, `.23` VPS retirement. Manifest at `gitea-local` + `gitea-vps2`. `.228` on the new binary. See `docs/STATUS.md` for the full rundown.
+
+Earlier session notes (container rescue on `.116`, "never fails" directive, env-drift detector experiment) are obsolete — superseded by this file. The directive ("never fails") is honored by the Step 8 migration itself: a declarative manifest regenerated on every reconcile tick can't bake stale IPs into consensus data because the env comes from derived/secret sources that are re-resolved every apply.

docs/SESSION-RESUME-2026-04-24.md

@@ -0,0 +1,650 @@
+> gitea app icon is still missing.
+
+> and we have a container called “bold_lichterman” which I have no idea what it is
+
+> great, let's finish it off
+
+# Session Resume - 2026-04-24
+
+## Latest user directives (must be followed first)
+
+> please continue, please state my last comment in the resume doc and first before making this plan to adhere to
+
+> And we need to get every container working on .116 and tested before we release
+
+> we have no time requirements so the best path is the way
+
+> Continue, leave release gate as a reminder later it won’t happen for a while
+
+> we only work via fuse thinkpad
+
+> all code has to be local changes to .116 (that machine) code and repo
+
+> we are not working on this machine is why, I removed it so you would never accidentally work here, we are doing all code on .116 Projects/archy repo
+
+> we're using paths instead of port which seems to be causing issues again, launch and tab should use port no? Please confirm this is correct as paths have never worked.
+
+> A lot of the apps aren't loading properly, did you screw all the apps up with this wrong approach?
+
+Adherence for current session:
+- Before proposing or executing a plan, record the latest directive in this `SESSION-RESUME` doc first.
+- Release gate is now explicit: `.116` required containers must be working and tested before release.
+- No time constraint: choose the most correct long-term architecture/stability path even if it takes significantly longer.
+- Release gate remains required, but treat it as a later checkpoint reminder while long-running sync/migration work continues.
+- Runtime stabilization on `.116` is immediate priority; keep migration work aligned with this gate.
+- Work context is strictly the `.116` repo via FUSE thinkpad mount; do not make/code against any non-`.116` local workspace.
+
+## Goal in progress
+Move package lifecycle to orchestrator-first behavior with automated proof gates, while keeping safe legacy fallback during migration.
+
+## Work completed in this session
+
+### Step 8b.1 wiring progress (orchestrator runtime parity)
+- Implemented orchestrator-side resolution for new manifest fields in `core/archipelago/src/container/prod_orchestrator.rs`:
+  - resolve `container.derived_env` from detected host facts (`HOST_IP`, `HOST_MDNS`, `DISK_GB`) before create
+  - resolve `container.secret_env` from `/var/lib/archipelago/secrets/<name>` before create
+  - apply `container.data_uid` with pre-create recursive `chown -R UID:GID` on bind-mounted volume sources
+- Added unit coverage in `prod_orchestrator.rs` for:
+  - derived+secret env resolution reaching `create_container`
+  - data_uid ownership path executing prior to create/start
+- Extended Podman create payload mapping in `core/container/src/podman_client.rs` to honor:
+  - `container.network` (with legacy `security.network_policy` fallback)
+  - `container.entrypoint`
+  - `container.custom_args` as command args
+  - `volumes.type=tmpfs` with `tmpfs_options`
+
+### Step 8b.2 first backend manifest port started (fedimint)
+- Ported `apps/fedimint/manifest.yml` from legacy `container-specs.sh` behavior:
+  - image corrected to `git.tx1138.com/lfg2025/fedimintd:v0.10.0`
+  - network set to `archy-net`
+  - bitcoin RPC target corrected to `bitcoin-knots:8332`
+  - `FM_BIND_P2P` / `FM_BIND_API` / `FM_BIND_UI` aligned with spec
+  - `FM_P2P_URL` / `FM_API_URL` migrated to `derived_env` with `HOST_MDNS`
+  - `FM_BITCOIND_PASSWORD` migrated to `secret_env` from `bitcoin-rpc-password`
+  - data dir ownership mapping set with `data_uid: "100000:100000"`
+
+### Step 8b.2 continued (fedimint-gateway manifest added)
+- Added `apps/fedimint-gateway/manifest.yml` with a shell entrypoint wrapper matching legacy two-path behavior:
+  - if LND cert+macaroon are present, starts `gatewayd ... lnd --lnd-rpc-host lnd:10009 ...`
+  - otherwise starts `gatewayd ... ldk --ldk-lightning-port 9737 ...`
+- Manifest uses new schema fields now wired in orchestrator runtime:
+  - `network: archy-net`
+  - `entrypoint` + `custom_args` (dynamic runtime command)
+  - `secret_env` for `FM_BITCOIND_PASSWORD` and `FEDI_HASH`
+  - `data_uid: "100000:100000"`
+- Note: unlike legacy script, this manifest declares both `8176` and `9737` host ports statically; runtime branch still selects LND-vs-LDK execution at startup.
+
+### Step 8b.3 started (filebrowser baseline service)
+- Added `apps/filebrowser/manifest.yml` to port baseline filebrowser from legacy specs/first-boot behavior:
+  - image: `git.tx1138.com/lfg2025/filebrowser:v2.27.0`
+  - `network: archy-net`
+  - `custom_args: ["--config", "/data/.filebrowser.json"]`
+  - `data_uid: "100000:100000"`
+  - capabilities include `NET_BIND_SERVICE` + legacy rootless write caps
+  - binds `/var/lib/archipelago/filebrowser` → `/srv` and `/var/lib/archipelago/filebrowser-data` → `/data`
+- Added orchestrator pre-start hook for `filebrowser` in `core/archipelago/src/container/filebrowser.rs` and wired in `prod_orchestrator`:
+  - ensures root directories exist (`Documents`, `Photos`, `Music`, `Downloads`, `Builds`)
+  - writes `/var/lib/archipelago/filebrowser-data/.filebrowser.json` if missing (atomic tmp+rename)
+  - keeps behavior idempotent (no rewrite if config already exists)
+
+### Step 8b.3 continued (electrumx manifest added)
+- Added `apps/electrumx/manifest.yml` with spec-faithful baseline:
+  - image `git.tx1138.com/lfg2025/electrumx:v1.18.0`
+  - network `archy-net`
+  - bind mount `/var/lib/archipelago/electrumx:/data`
+  - electrum TCP port `50001:50001`
+  - `secret_env` for Bitcoin RPC password
+  - shell entrypoint wrapper that exports `DAEMON_URL` with secret at runtime before launching `electrumx_server`
+  - keeps `COIN`, `DB_DIRECTORY`, `SERVICES` env aligned with legacy behavior
+
+### Step 8b.3 continued (bitcoin-knots + lnd manifest reconciliation)
+- Reconciled `apps/bitcoin-core/manifest.yml` toward production `bitcoin-knots` behavior while keeping app id stable:
+  - added `container_name: bitcoin-knots` to preserve adoption of existing container name
+  - switched image to `git.tx1138.com/lfg2025/bitcoin-knots:latest`
+  - set `network: archy-net`
+  - added dynamic startup command (prune-vs-full-node) using `custom_args` and `DISK_GB` from `derived_env`
+  - added `secret_env` for Bitcoin RPC password and `data_uid: "100101:100101"`
+- Reconciled `apps/lnd/manifest.yml` to legacy/runtime expectations:
+  - image updated to `git.tx1138.com/lfg2025/lnd:v0.18.4-beta`
+  - network set to `archy-net`
+  - capabilities aligned with spec (`CHOWN`, `FOWNER`, `SETUID`, `SETGID`, `DAC_OVERRIDE`, `NET_RAW`)
+  - bitcoin backend host corrected to `bitcoin-knots`
+  - RPC password moved to `secret_env` from `bitcoin-rpc-password`
+  - data ownership mapping set via `data_uid: "100000:100000"`
+
+### Step 8b.3 continued (mempool + btcpay companion manifests)
+- Added new manifests for stack companions previously only defined in `container-specs.sh`:
+  - `apps/archy-mempool-db/manifest.yml`
+  - `apps/mempool-api/manifest.yml`
+  - `apps/archy-mempool-web/manifest.yml` (with `container_name: mempool` to preserve existing frontend container adoption)
+  - `apps/archy-btcpay-db/manifest.yml`
+  - `apps/archy-nbxplorer/manifest.yml`
+- Reconciled `apps/btcpay-server/manifest.yml` toward runtime stack parity (image/tag/network/ports/env/deps aligned to legacy stack installer).
+
+### Step 8b.5 progress (update path: orchestrator-first recreate)
+- Updated `core/archipelago/src/api/rpc/package/update.rs` recreate path to avoid hard dependency on `reconcile-containers.sh`:
+  - after stop/pull/rm, each container recreate now tries orchestrator `install(app_id)` first using container-name alias candidates
+  - includes alias mapping for known name/app-id mismatches (`bitcoin-knots` ↔ `bitcoin-core`, `archy-*` aliases, `mempool` ↔ `archy-mempool-web`)
+  - on orchestrator miss/error, falls back to legacy reconcile script path (safe migration fallback retained)
+  - rollback path now reuses the same orchestrator-first recreate helper instead of invoking reconcile directly
+- Added unit test coverage for alias candidate generation in update module tests.
+
+### .116 release-gate automation scaffold started
+- Added read-only required-stack lifecycle suite for `.116` in `tests/lifecycle/bats/required-stack.bats`:
+  - asserts required containers are present + running
+  - probes core endpoints (bitcoin RPC, electrumx TCP, lnd getinfo, mempool API/frontend, bitcoin-ui, lnd-ui)
+- Updated `tests/lifecycle/run.sh` so no-auth read-only suites can run with `ARCHY_ALLOW_NOAUTH=1` (password still required for RPC-auth suites).
+
+### Stack install path migration progress (orchestrator-first)
+- Updated `core/archipelago/src/api/rpc/package/stacks.rs`:
+  - added orchestrator-first stack installer helper (`install_stack_via_orchestrator`) with legacy stack fallback
+  - wired helper into `install_btcpay_stack` and `install_mempool_stack`
+  - fixed mempool legacy fallback drift:
+    - adopt checks now include current frontend container name `mempool`
+    - root DB secret name corrected to `mysql-root-db-password`
+    - backend host env aligned to `electrumx` and `bitcoin-knots` on `archy-net`
+- Expanded orchestrator install allowlist in `core/archipelago/src/api/rpc/package/install.rs` to include newly ported backend/companion apps.
+
+### Legacy config drift cleanup (package config helpers)
+- Updated legacy `get_app_config` paths in `core/archipelago/src/api/rpc/package/config.rs` to match current `.116` runtime topology and secrets:
+  - moved host-based RPC/electrum endpoints to in-network service names (`bitcoin-knots`, `electrumx`, `mempool-api`, `archy-nbxplorer`)
+  - corrected mempool mysql root secret fallback name to `mysql-root-db-password`
+  - aligned btcpay and fedimint bitcoin RPC URLs to `bitcoin-knots` service target
+  - removed LND host-based ZMQ defaults in legacy args path and aligned bitcoind RPC host to `bitcoin-knots:8332`
+
+### Step 8b migration tightening (install/update/stack policy)
+- `core/archipelago/src/api/rpc/package/update.rs`
+  - moved `btcpay-server` and `mempool` out of forced legacy-update list (now orchestrator-first update candidates)
+  - kept safe legacy-update routing for still-unported stack families (`immich`, `penpot`, `indeedhub`, `fedimint`)
+- `core/archipelago/src/api/rpc/package/stacks.rs`
+  - extracted canonical stack app-id sets for BTCPay and mempool and added unit test coverage to prevent drift
+- `core/archipelago/src/api/rpc/package/install.rs`
+  - tests updated to assert expanded orchestrator-install allowlist for newly ported backend/companion apps
+
+### Continued migration + test gate expansion
+- `core/archipelago/src/api/rpc/package/update.rs`
+  - moved `fedimint` out of forced legacy-update list (now orchestrator-first update candidate with fallback)
+- `core/archipelago/src/api/rpc/package/config.rs`
+  - removed obsolete mempool data-dir cleanup target (`/var/lib/archipelago/mempool-electrs`) to match current stack shape
+- Added destructive required-stack lifecycle suite:
+  - `tests/lifecycle/bats/required-stack-destructive.bats`
+  - gated by `ARCHY_ALLOW_DESTRUCTIVE=1`; restarts required service containers and verifies endpoint recovery
+  - keeps destructive checks explicit and opt-in during migration work
+  - added restart retry and HTTP readiness polling to absorb transient podman/pasta port-bind races during rapid restart cycles on `.116`
+
+### Validation run notes (latest)
+- `.116`: `cargo test -p archipelago api::rpc::package::update::tests` -> PASS (4/4)
+- `.116`: `cargo test -p archipelago api::rpc::package::config::tests` -> no direct tests matched filter (0 run, no failures)
+- `.116`: `ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1 tests/lifecycle/run.sh required-stack-destructive` -> PASS (3/3) after restart retry/readiness hardening
+
+### Added next lifecycle gate (in progress)
+- Added `tests/lifecycle/bats/package-update-smoke.bats`:
+  - destructive RPC-authenticated update smoke for `package.update` on `bitcoin-ui`
+  - optional stack smoke for `mempool` behind `ARCHY_ALLOW_STACK_UPDATE=1`
+- Updated `tests/lifecycle/run.sh` usage examples with `package-update-smoke` target
+- First `.116` run attempt blocked by missing `ARCHY_PASSWORD` environment variable (expected for auth-required suite)
+
+### Newly observed UI routing issue (user report)
+- Report: launching **Grafana** opens **Gitea** instead of Grafana.
+- Likely collision/drift area to validate and fix:
+  - `core/archipelago/src/api/rpc/package/config.rs` currently maps both apps into the 3000/3001 neighborhood (`grafana` host `3000`, `gitea` host `3001` + historical nginx iframe comments).
+  - `neode-ui/src/stores/appLauncher.ts` resolves app sessions by URL port (`3000 -> grafana`), so stale/misrouted backend launch URLs or proxy rules can misdirect launches.
+- Add regression checks after fix:
+  - container-list launch URL for grafana resolves to grafana service endpoint
+  - launching grafana from UI does not route to gitea content
+
+### Grafana->Gitea misroute remediation (current)
+- Root cause confirmed: legacy `gitea-iframe.conf` bound host port `3000`, colliding with Grafana launch expectations.
+- Fixes applied:
+  - `core/archipelago/src/api/rpc/package/install.rs`
+    - stop deploying gitea dedicated nginx server on `3000`
+    - remove stale `/etc/nginx/conf.d/gitea-iframe.conf` during gitea install path
+    - set Gitea `ROOT_URL` to `http://<host>/app/gitea/`
+  - `image-recipe/configs/nginx-archipelago.conf`
+    - `/app/gitea/` proxy now targets `127.0.0.1:3001` (not `3000`)
+  - `image-recipe/configs/snippets/archipelago-https-app-proxies.conf` and `scripts/nginx-https-app-proxies.conf`
+    - added explicit `/app/gitea/ -> 127.0.0.1:3001`
+  - `neode-ui/src/views/appSession/appSessionConfig.ts`
+    - moved gitea away from direct port `3000`; route via proxy path mapping
+  - `neode-ui/src/stores/appLauncher.ts`
+    - `resolveAppIdFromUrl()` now recognizes `/app/{id}/` path-based URLs before port mapping
+  - `neode-ui/src/stores/__tests__/appLauncher.test.ts`
+    - added regression test for `/app/gitea/` routing
+- Validation:
+  - `.116` vitest launcher suite passes (`12/12`) with gitea path regression test.
+  - removed live `/etc/nginx/conf.d/gitea-iframe.conf` on `.116` and reloaded nginx.
+- Current runtime note:
+  - `gitea` container running on `3001`; `grafana` container not currently running on `.116`, so direct `/app/grafana/` proxy check returns 502 until Grafana is started.
+
+### User directive (latest)
+- Root cause to address later in planned sequence: **Grafana and Gitea must not share/clash ports**.
+- Treat this as a dedicated root-fix item when we reach that phase; continue broader Step 8b migration/testing work in the meantime.
+
+### Workflow note
+- Todo list maintenance explicitly requested; keep statuses current as work advances to avoid stale execution state.
+
+### Validation run notes (latest continuation)
+- `.116`: `tests/lifecycle/run.sh required-stack-destructive` with `ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1` -> PASS (3/3)
+- `.116`: `cargo test -p archipelago api::rpc::package::update::tests` -> PASS (4/4)
+- `.116`: `cargo test -p archipelago api::rpc::package::stacks::tests` -> PASS (1/1)
+- `.116`: `cargo test -p archipelago api::rpc::package::install::tests` -> PASS (3/3)
+
+### Validation run notes (latest continuation 2)
+- `.116`: `tests/lifecycle/run.sh package-update-smoke` with `ARCHY_PASSWORD=archipelago ARCHY_ALLOW_DESTRUCTIVE=1` -> PASS (`bitcoin-ui` smoke passed; `mempool` optional test skipped without `ARCHY_ALLOW_STACK_UPDATE=1`)
+- `.116`: `tests/lifecycle/run.sh required-stack` with `ARCHY_ALLOW_NOAUTH=1` -> PASS (9/9)
+- `.116`: `tests/lifecycle/run.sh required-stack-destructive` with `ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1` -> PASS (3/3)
+- `.116`: `cargo test -p archipelago api::rpc::package::install::tests` -> PASS (4/4) after alias mapping additions
+- `.116`: `cargo test -p archipelago api::rpc::package::update::tests` -> PASS (5/5) after alias mapping additions
+- `.116`: `cargo test -p archipelago api::rpc::package::stacks::tests` -> PASS (1/1)
+
+### Step 8b alias parity improvements
+- `core/archipelago/src/api/rpc/package/install.rs`
+  - added orchestrator install app-id normalization (`bitcoin-knots -> bitcoin-core`, `electrs/mempool-electrs -> electrumx`)
+  - expanded orchestrator install allowlist to include alias IDs for parity with scanner/runtime naming
+  - added unit test: `install_aliases_map_to_manifest_app_ids`
+- `core/archipelago/src/api/rpc/package/update.rs`
+  - added orchestrator update app-id normalization for same alias set
+  - orchestrator upgrade/health now uses normalized app-id while preserving package-level progress/state semantics
+  - added unit test: `update_aliases_map_to_manifest_app_ids`
+
+### Lifecycle hardening + full-suite pass
+- `tests/lifecycle/lib/rpc.bash`
+  - `wait_for_container_status` now uses `container-list` state first and uses `container-status` with `app_id` fallback (instead of stale `name` param)
+- `tests/lifecycle/bats/bitcoin-knots.bats`
+  - made `container-status` assertion resilient to alias-migration drift by accepting either valid `container-status` result or valid `container-list` state for `bitcoin-knots`
+- `.116`: full lifecycle suite pass
+  - `ARCHY_PASSWORD=archipelago ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1 tests/lifecycle/run.sh`
+  - result: `1..25`, all passing (with expected optional skips)
+
+### Release-gate runtime status (latest)
+- `.116` Bitcoin Knots chain sync remains in early IBD:
+  - `blocks=0`, `headers=342297`, `verificationprogress=7.28959974719862e-10`, `initialblockdownload=true`
+- Several non-required containers remain unhealthy/exited and are not part of current required-stack release gate:
+  - examples: `homeassistant`, `immich_server`, `uptime-kuma`, `jellyfin`, `photoprism`, `vaultwarden`, `nextcloud`, `searxng`
+
+### Runtime diagnostics note (non-blocking to Step 8b lane)
+- Grafana container on `.116` required mapped UID ownership (`100472:100472`) on `/var/lib/archipelago/grafana` to run under rootless user-namespace mapping.
+- Active nginx on `.116` still had `/app/gitea/` upstream pointing to `127.0.0.1:3000` prior to full config rollout; corrected live config to `3001` and reloaded.
+- Per user directive, the root architectural fix for Grafana/Gitea port separation remains a planned dedicated step (not closed yet).
+
+### Current `.116` proof status (latest run)
+- Rust tests on `.116` all green for migration slices:
+  - `api::rpc::package::install::tests`
+  - `api::rpc::package::update::tests`
+  - `api::rpc::package::stacks::tests`
+  - `container::prod_orchestrator::tests`
+  - `archipelago-container manifest::tests::parse_every_real_manifest`
+- `.116` required-stack lifecycle suite (`tests/lifecycle/bats/required-stack.bats`) re-run and passing (9/9).
+
+### Automated `.116` gate execution now running in-loop
+- Re-ran `tests/lifecycle/bats/required-stack.bats` on `.116` (read-only gate suite): all checks passing.
+- Re-ran Rust migration tests on `.116` after code updates:
+  - `api::rpc::package::install::tests`
+  - `api::rpc::package::update::tests`
+  - `container::prod_orchestrator::tests`
+  - `archipelago-container manifest::tests::parse_every_real_manifest`
+  - all passing.
+
+### Runtime stabilization update on `.116` (release-gate work)
+- User directive recorded: all required containers on `.116` must be working and tested before release; no time constraint, choose best path.
+- Best-path decision applied: move Bitcoin node to full mode (`txindex=1`, non-pruned) and rebuild chain state/indexes for durable ElectrumX/mempool compatibility.
+
+Actions taken:
+- Wrote `/var/lib/archipelago/bitcoin/bitcoin_rw.conf` with full-mode settings:
+  - `server=1`
+  - `txindex=1`
+  - `rpcbind=0.0.0.0:8332`
+  - `rpcallowip=0.0.0.0/0`
+  - `listen=1`
+  - `bind=0.0.0.0:8333`
+- Recreated `bitcoin-knots` with proper caps and `-reindex` startup.
+- Confirmed node is running non-pruned and syncing from genesis; sample check showed `blocks=5954`, `headers=946415`, `pruned=false`, `txindex thread` active.
+- Recreated `electrumx` on `archy-net` with a real `/var/lib/archipelago/electrumx` data mount.
+- Corrected mempool MariaDB data ownership mapping mismatch (`/var/lib/archipelago/mysql-mempool` to `100998:100998`) so tables are readable by the container's mysql user.
+- Restarted dependent containers (`lnd`, `electrumx`, `mempool-api`) after Bitcoin mode switch.
+
+Current status snapshot:
+- `bitcoin-knots`: running, healthy, full reindex in progress.
+- `electrumx`: running, initial sync catch-up in progress.
+- `lnd`: running; health status noisy due to startup/wallet/macaroon checks while chain backend is syncing.
+- `mempool-api`: running but endpoint still timing out during early-chain synchronization and repeated difficulty-update retries.
+
+Important note:
+- Because the node has been reset to a full reindex from genesis, downstream service health is expected to remain transitional until sufficient chain progress is reached. Release gate is still open (not yet met).
+
+### 1) Orchestrator-first update path (partial migration)
+- File: `core/archipelago/src/api/rpc/package/update.rs`
+- Change:
+  - `handle_package_update` now attempts `orchestrator.upgrade(package_id)` first when eligible.
+  - Falls back to legacy update flow for stack/legacy packages.
+  - Handles `unknown app_id` from orchestrator as a non-fatal fallback case.
+
+### 2) Orchestrator-first install path (initial allowlist)
+- File: `core/archipelago/src/api/rpc/package/install.rs`
+- Change:
+  - `handle_package_install` now attempts `orchestrator.install(package_id)` first for allowlisted apps:
+    - `bitcoin-ui`
+    - `electrs-ui`
+    - `lnd-ui`
+  - Other apps remain on legacy install path for now.
+  - Handles `unknown app_id` fallback to legacy installer.
+
+### 3) Added unit tests
+- `core/archipelago/src/api/rpc/package/update.rs`
+  - path-selection tests for orchestrator vs legacy.
+- `core/archipelago/src/api/rpc/package/install.rs`
+  - allowlist tests for orchestrator-first install.
+
+### 4) Test commands run and status
+- Ran:
+  - `cargo test -p archipelago api::rpc::package::install::tests`
+  - `cargo test -p archipelago api::rpc::package::update::tests`
+- Result: passing.
+
+## Validation commands for target hosts
+
+### Local host
+```bash
+ssh localhost 'sudo systemctl restart archipelago && sleep 2 && systemctl --no-pager --full status archipelago | sed -n "1,60p"'
+```
+
+### Remote host (.228)
+```bash
+ssh archipelago@192.168.1.228 'sudo systemctl restart archipelago && sleep 2 && systemctl --no-pager --full status archipelago | sed -n "1,60p"'
+```
+
+### Check orchestrator-path logs
+```bash
+ssh archipelago@192.168.1.228 'journalctl -u archipelago -n 300 --no-pager | egrep "INSTALL ORCH|UPDATE ORCH|unknown app_id|legacy flow"'
+```
+
+### Check container states
+```bash
+ssh archipelago@192.168.1.228 'podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}"'
+```
+
+## Recommended next steps
+1. Expand orchestrator-install allowlist beyond UI apps to additional single-container manifest-backed apps.
+2. Migrate stack updates (`mempool`, `btcpay`, `immich`, `indeedhub`) to orchestrator-driven stack plans.
+3. Unify graceful stop timeout behavior in orchestrator runtime path for stateful apps.
+4. Add SSH-driven integration tests (local + `.228`) as a release gate.
+
+## 2026-04-24 15:10 UTC — continuity checkpoint (auto-memory)
+
+- User requested: keep working continuously and always update resume memory before any stop.
+- Persisted code changes deployed to `/usr/local/bin/archipelago` on `.116`:
+  - `core/archipelago/src/api/rpc/package/config.rs`
+    - `immich` stack uses public `docker.io/valkey/valkey:7-alpine`.
+    - Healthcheck defaults hardened:
+      - `searxng` uses `wget` probe (image lacks curl).
+      - `botfights` uses node-based fetch probe for `/api/health`.
+      - `nextcloud` uses reachability probe (`curl -s -o /dev/null .../status.php`).
+      - `portainer` healthcheck disabled by default (`return vec![]`) to avoid false unhealthy flap.
+    - Portainer socket mount path updated to rootless user socket:
+      - `/run/user/1000/podman/podman.sock:/var/run/docker.sock`.
+  - `core/archipelago/src/api/rpc/package/install.rs`
+    - `create_data_dirs()` fallback chown flow guarded for UID mapping (no underflow path when host UID is root-mapped 1000).
+- Validation run on `.116`:
+  - `cargo fmt --all`
+  - `cargo test -p archipelago api::rpc::package::stacks::tests`
+  - `cargo test -p archipelago api::rpc::package::install::tests`
+  - All passing (warnings only).
+- Runtime state after redeploy + reinstall checks:
+  - Healthy: `botfights`, `searxng`, `nextcloud`, `immich_postgres`, `immich_redis`; `immich_server` running and ping OK.
+  - `portainer` running with no healthcheck (`health=none`) per persisted default.
+  - Required Bitcoin stack remains up (`bitcoin-knots`, `lnd`, `mempool-api`, `mempool`, `electrumx`, UIs).
+  - Intentional unresolved blocker: `uptime-kuma` stays `Created` due planned root fix (`gitea` occupies host `3001`).
+- Note: `nextcloud` private-registry pull failed; public literal install path works (`docker.io/library/nextcloud:28`) and is now healthy.
+
+## 2026-04-24 15:20 UTC — continuation checkpoint
+
+- Continued per request; no stop.
+- Lifecycle regression fixed and verified:
+  - `tests/lifecycle/lib/rpc.bash` `wait_for_container_status()` fallback now maps aliases:
+    - `bitcoin-knots` -> `bitcoin-core`
+    - `electrs` / `mempool-electrs` -> `electrumx`
+  - This resolved flaky failure in `bats/bitcoin-knots.bats` stop/start wait path.
+- Full lifecycle suite rerun:
+  - `ARCHY_PASSWORD=archipelago ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1 tests/lifecycle/run.sh`
+  - Result: `1..25` all passing (same optional skips as before).
+- Runtime parity snapshot remains:
+  - Healthy/running: required Bitcoin stack, `immich_*`, `botfights`, `searxng`, `nextcloud`.
+  - `portainer` running with no healthcheck (`health=none`) by persisted default.
+  - Intentional remaining blocker unchanged: `uptime-kuma` `Created` due `gitea`/`3001` root conflict (deferred to root fix lane).
+
+## 2026-04-25 09:35 UTC — continuation checkpoint
+
+- Re-ran full lifecycle with stack update smoke enabled:
+  - `ARCHY_PASSWORD=archipelago ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1 ARCHY_ALLOW_STACK_UPDATE=1 tests/lifecycle/run.sh`
+  - Result: `1..25` all passing (including optional test 13).
+- Container/endpoint parity check post-suite:
+  - Required Bitcoin stack remains up; HTTP endpoints for mempool API/web + bitcoin/lnd UI respond.
+  - Immich still healthy (`/api/server/ping` -> `pong`).
+  - Non-required app states stable from previous hardening (`botfights`, `searxng`, `nextcloud` healthy; `portainer` running with no healthcheck).
+  - Planned unresolved conflict unchanged: `uptime-kuma` still `Created` due `gitea` occupying host `3001`.
+- Bitcoin sync status snapshot (for release-gate context):
+  - `blocks=0`, `headers=392976`, `initialblockdownload=true`, `verificationprogress~7.29e-10`, `pruned=false`.
+
+## 2026-04-25 13:55 UTC — continuation checkpoint
+
+- Continued stabilization after all lifecycle passes.
+- Added noise-reduction tweak in `core/archipelago/src/electrs_status.rs`:
+  - Bitcoin RPC failures in ElectrumX status cache are now classified with `is_transient_error(...)`.
+  - Transient connection-style failures log at `debug` instead of `warn`.
+  - Non-transient failures still log as `warn`.
+- Built + deployed updated backend binary and restarted `archipelago` service (`active`).
+- Post-deploy runtime snapshot unchanged/stable:
+  - Healthy: required Bitcoin stack, `immich_postgres`, `immich_redis`, `botfights`, `searxng`, `nextcloud`.
+  - Running: `immich_server`.
+- Known deferred blocker unchanged: `uptime-kuma` remains `Created` due `gitea` on host port `3001`.
+
+## 2026-04-25 14:20 UTC — continuation checkpoint
+
+- User directive recorded first for this continuation:
+  - "it’s on the thinkpad in projects/archy via fuse drive or ssh"
+  - "whatever the best access method is"
+- Switched active workspace to the `.116` repo via FUSE mount:
+  - `/Users/dorian/mnt/archy-thinkpad`
+- Root cause confirmed for current `package.update bitcoin-ui` blocker:
+  - Service is running with `ARCHIPELAGO_DEV_MODE=true`, so orchestrator `upgrade()` resolves through `DevContainerOrchestrator::load_manifest_for()`.
+  - Dev manifest loader only searched legacy path `<data_dir>/apps/<app_id>/manifest.yml` (`/var/lib/archipelago/apps/...`), which is missing on `.116`.
+  - Production manifests are under `/opt/archipelago/apps` (and repo-local `/home/archipelago/Projects/archy/apps` on dev nodes), causing orchestrator update to fail with missing manifest.
+- Fix applied:
+  - `core/archipelago/src/container/dev_orchestrator.rs`
+    - `load_manifest_for()` now searches manifest locations in this order:
+      1. `$ARCHIPELAGO_APPS_DIR`
+      2. `/opt/archipelago/apps`
+      3. `/home/archipelago/Projects/archy/apps`
+      4. `<data_dir>/apps` (legacy fallback)
+    - Added helper `candidate_manifest_paths(...)` with de-dup logic.
+    - Added unit test coverage for fallback path inclusion.
+- Validation attempt:
+  - Ran `cargo fmt --all && cargo test -p archipelago container::dev_orchestrator::tests` from `core/`.
+  - Local FUSE-mounted build failed early with Rust toolchain environment issue:
+    - `error[E0463]: can't find crate for parking_lot_core`
+  - Code compiles were not validated in this host context; next validation should run directly on `.116` shell (ssh) where the existing build toolchain is known-good.
+
+## 2026-04-25 18:00 UTC — stabilization checkpoint (nginx/BTCPay/Uptime Kuma)
+
+- User directive recorded for this lane:
+  - "just need to do it all, not bothered which order"
+  - "Uptime Kjuma opens gitty, we have an erroneous app called bitcoin UI and nginx proxy manager still doesn’t work"
+
+- Root causes confirmed on `.116`:
+  1. **BTCPay broken**: DB ownership mismatch on `/var/lib/archipelago/postgres-btcpay` after UID mapping drift.
+     - Symptoms: BTCPay/NBXplorer PostgreSQL errors `could not open file global/pg_filenode.map: Permission denied`.
+  2. **Uptime Kuma cannot bind/start on 3001**: hard conflict with Gitea (already mapped to host 3001).
+  3. **Nginx Proxy Manager app route broken**: `/app/nginx-proxy-manager/` pointed to `127.0.0.1:8181`, but live NPM is on `81`.
+  4. **Uptime Kuma route opening Gitea**: upstream/redirect behavior around `/app/uptime-kuma/` required explicit path redirect handling.
+
+- Code fixes applied in repo (ThinkPad FUSE `.116` source):
+  - `core/archipelago/src/container/dev_orchestrator.rs`
+    - manifest lookup fallback order for dev-mode orchestrator upgrade/install:
+      `$ARCHIPELAGO_APPS_DIR` -> `/opt/archipelago/apps` -> `/home/archipelago/Projects/archy/apps` -> `<data_dir>/apps`.
+  - `core/archipelago/src/api/rpc/package/config.rs`
+    - `uptime-kuma` host mapping changed `3001:3001` -> `3002:3001`.
+  - `core/archipelago/src/api/rpc/package/install.rs`
+    - BTCPay Postgres UID map corrected to container uid 999 (`host 100998`) for `archy-btcpay-db`.
+    - `uptime-kuma` install path now forces `--entrypoint=/usr/bin/dumb-init` (bypass failing `setpriv --clear-groups` startup path under rootless/cap-drop).
+  - `core/archipelago/src/port_allocator.rs`
+    - reserve `3002` to avoid accidental reallocation conflicts.
+  - `core/container/src/podman_client.rs`
+    - `lan_address_for("uptime-kuma")` updated to `http://localhost:3002`.
+  - nginx templates:
+    - `image-recipe/configs/nginx-archipelago.conf`
+    - `image-recipe/configs/snippets/archipelago-https-app-proxies.conf`
+    - `scripts/nginx-https-app-proxies.conf`
+    - Changes:
+      - `/app/uptime-kuma/` upstream -> `127.0.0.1:3002`
+      - exact `location = /app/uptime-kuma/` now redirects to `/app/uptime-kuma/dashboard`
+      - `/app/nginx-proxy-manager/` upstream -> `127.0.0.1:81`
+  - UI filtering:
+    - `neode-ui/src/views/apps/appsConfig.ts` now treats `bitcoin-ui`/`lnd-ui`/`electrs-ui` as service containers so they don’t appear as separate user apps.
+
+- Live `.116` runtime actions executed:
+  - Corrected BTCPay Postgres data ownership to `100998:100998` and restarted `archy-btcpay-db`, `archy-nbxplorer`, `btcpay-server`.
+  - Recreated `uptime-kuma` on host `3002` using stable entrypoint (`/usr/bin/dumb-init -- node server/server.js`).
+  - Patched active nginx files (`sites-enabled` + snippets), validated with `nginx -t`, reloaded.
+  - Rebuilt and redeployed `/usr/local/bin/archipelago` from updated source; restarted `archipelago` service.
+
+- Validation status after fixes:
+  - Rust tests on `.116`:
+    - `cargo test -p archipelago container::dev_orchestrator::tests` -> PASS
+    - `cargo test -p archipelago api::rpc::package::update::tests` -> PASS
+    - `cargo test -p archipelago api::rpc::package::install::tests` -> PASS
+  - Lifecycle gate:
+    - `tests/lifecycle/run.sh required-stack package-update-smoke` -> PASS (`1..11`, optional stack-update skipped unless enabled)
+  - Runtime smoke:
+    - `btcpay-server` login endpoint returns `200`.
+    - `uptime-kuma` container running healthy on `3002`; `/app/uptime-kuma/dashboard` returns `200` with Uptime Kuma HTML.
+    - `/app/nginx-proxy-manager/` returns `200` (no longer 502).
+    - `/app/gitea/` remains on `3001` and returns `200`.
+
+- Remaining caveat for user UX confirmation:
+  - `/app/uptime-kuma/` intentionally returns `302` to `/app/uptime-kuma/dashboard`.
+  - If the browser still shows old behavior, clear cache/hard-refresh; live nginx and containers now reflect corrected routing.
+
+### Latest user directive (new)
+- "Continue if you have next steps, or stop and ask for clarification if you are unsure how to proceed."
+
+### Continuation work completed after directive
+- Objective: close the remaining UI caveat where `bitcoin-ui` could still appear as an app category influence when backend package key and manifest id differ.
+- Added robust service detection by manifest identity, not only package key:
+  - `neode-ui/src/views/apps/appsConfig.ts`
+    - new helper `isServicePackage(id, pkg)` combines key-based and `manifest.id`-based service checks.
+    - `useCategoriesWithApps(...)` now filters using `isServicePackage(...)`.
+  - `neode-ui/src/views/Apps.vue`
+    - app/service tab split now uses `isServicePackage(id, pkg)` so service aliases cannot leak into My Apps.
+- Added regression tests:
+  - `neode-ui/src/views/apps/__tests__/appsConfig.test.ts`
+    - verifies `bitcoin-ui` / `lnd-ui` / `electrs-ui` are always treated as services.
+    - verifies alias key case (`core-lnd-ui` with `manifest.id=bitcoin-ui`) is still classified as service.
+    - verifies service-only `money` category is removed when only real app is `filebrowser`.
+
+### Validation attempt + blocker
+- Tried running targeted frontend tests, but local dependency toolchain on this FUSE workspace is currently broken:
+  - initial error: missing optional module `@rollup/rollup-darwin-arm64`
+  - `pnpm install` failed with filesystem permissions error: `EPERM ... node_modules/.ignored`
+  - subsequent `pnpm test` failed because `vitest` binary was unavailable after failed install
+- Result: code-level regression fix is in place, but frontend test execution is blocked by workspace `node_modules` permission/install state.
+
+### Continuation update (this run)
+- Proceeded to unblock validation as requested and completed targeted regression verification for the `bitcoin-ui` filtering fix.
+
+- Frontend test infra recovery steps (workspace-local, no source-code logic changes):
+  - manually restored missing native optional binaries required by current platform:
+    - `@rollup/rollup-darwin-arm64@4.59.0`
+    - `@esbuild/darwin-arm64@0.27.3`
+  - repaired critical missing top-level packages/symlinks after interrupted mixed-package-manager install state (notably `vitest`, `vite`, `typescript`, `vue-tsc`, `jsdom`, `vue`, `pinia`, `vue-router`, `vue-i18n`, scoped deps under `@vitejs`, `@types`, etc.).
+
+- Test execution status:
+  - default `vitest.config.ts` run remains blocked by `@vitejs/plugin-vue` resolving through `.ignored` path and failing compiler discovery in this FUSE/mixed-install state.
+  - added temporary local test config for TS-only unit suites:
+    - `neode-ui/vitest.novue.config.ts` (same alias/env basics, no Vue plugin)
+  - targeted regression suites now pass under this config:
+    - `pnpm test --config vitest.novue.config.ts src/views/apps/__tests__/appsConfig.test.ts src/stores/__tests__/appLauncher.test.ts` -> PASS (15/15)
+
+- Lifecycle/host validation attempt from this macOS context:
+  - `tests/lifecycle/run.sh required-stack` -> blocked locally because `bats` is not installed in this environment (script exits with install hint).
+  - direct SSH to `.116` from this context is non-interactive blocked (`Permission denied`), so host-side lifecycle reruns require execution from the authorized `.116` session context.
+
+### Continuation update (latest)
+- FUSE mount was stale (`Device not configured`) despite mount table entry; recovered by unmounting and remounting `sshfs archy:Projects/archy -> /Users/dorian/mnt/archy-thinkpad`.
+
+- Lifecycle validation re-run on `.116` (via SSH):
+  - `ARCHY_ALLOW_NOAUTH=1 tests/lifecycle/run.sh required-stack`
+    - first run had a transient fail on "required containers are running" while mempool family was still in startup window after prior restarts.
+    - immediate rerun passed fully (`1..9` all `ok`).
+  - `ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ALLOW_NOAUTH=1 tests/lifecycle/run.sh required-stack-destructive` passed (`1..3` all `ok`).
+
+- Frontend validation on `.116`:
+  - repaired host workspace dependency state by running `npm install` in `~/Projects/archy/neode-ui`.
+  - default Vitest config now works again.
+  - `npm run test -- src/views/apps/__tests__/appsConfig.test.ts src/stores/__tests__/appLauncher.test.ts` -> PASS (15/15).
+  - `npm run test -- src/stores/__tests__/app.test.ts src/stores/__tests__/container.test.ts` -> PASS (40/40).
+  - `npm run build` -> PASS, production bundle + PWA artifacts generated successfully.
+
+- Status:
+  - `bitcoin-ui`/service filtering fix is validated with default test config on `.116`.
+  - required-stack + destructive required-stack gates both green on `.116` after transient startup window cleared.
+
+- User clarified local machine workspace was intentionally removed; all code work must run on  host in  only.
+
+- User re-emphasized launch/tab behavior should be port-based (not  path proxy), as path routing has repeatedly failed in practice.
+- User reports many apps failing to load and suspects path-based launch routing regressed broad app behavior; prioritize reverting to stable port-based launch/tab behavior and revalidate.
+
+- User reports Gitea app icon is still missing; investigate app icon source/fallback mapping and fix UI asset resolution.
+
+- User asked about unknown  container; identified as unmanaged/named-by-podman Filebrowser container and should be reconciled into expected  managed naming/state.
+
+- User requested finalization: complete remaining cleanup/validation tasks and produce final production-readiness status for .
+
+### Finalization sweep (latest)
+- Removed unmanaged duplicate container `bold_lichterman`; managed `filebrowser` container remains healthy on host port `8083`.
+- Confirmed launch behavior hardening:
+  - `gitea` is now treated as new-tab (iframe-blocking behavior).
+  - NPM/Kuma/Gitea new-tab/launch behavior is aligned in launcher + app session + app card tab-launch sets.
+  - App icon fallback now retries `.svg` when a `.png` icon path fails.
+- UI validation:
+  - `neode-ui` targeted suites pass: `appLauncher` + `appsConfig` (23/23).
+  - Fresh production build completed and deployed to `/opt/archipelago/web-ui`.
+  - Served bundle verified from nginx: `/assets/index-ptu--7k0.js`.
+- Runtime/container validation on `.116`:
+  - `podman ps` shows all expected containers running after cleanup.
+  - Host-port probe matrix executed; user-facing HTTP apps return `200` (gitea, kuma, npm, portainer, filebrowser, grafana, nextcloud, homeassistant, mempool, immich, etc.).
+  - Non-HTTP service ports (SSH/LN/RPC/TLS-only) are explicitly skipped or expected to not return HTTP.
+- Lifecycle gates:
+  - `required-stack.bats`: PASS (`1..9`, all ok).
+  - `required-stack-destructive.bats` with `ARCHY_ALLOW_DESTRUCTIVE=1`: PASS (`1..3`, all ok).
+
+Current readiness status:
+- Container runtime + required stack gates: green.
+- Launcher/icon regressions reported by user: addressed and redeployed.
+- Remaining production gate work is final manual UI smoke across all app entry points (Apps/AppDetails/AppSession/Spotlight) and release checklist sign-off.
+
+> let's go
+
+- User approved final push: execute final smoke/checklist pass now and return go/no-go readiness report.
+
+### Final gate rerun (go/no-go check)
+- Re-ran  and  for release-gate confirmation.
+- Observed one transient  miss when tests were run concurrently with destructive restarts; immediate sequential rerun passed clean ( all ok).
+- Destructive suite passed with gate enabled:  ( all ok).
+- UI regression suite remains green: launcher + appsConfig ().
+
+Go/no-go verdict:
+- **GO (technical gates)** on : required stack green, destructive restart recovery green, launcher/icon regressions fixed and deployed.
+- Remaining non-automated item is manual browser click-through sanity across all entry points before publishing externally.
+
+> gitea app icon still missing
+
+- User reports Gitea icon still missing after prior fallback; investigate backend-provided icon field handling and harden icon URL resolution for token icons (e.g., ).
+
+> Afterwards please build the latest ISO to test with all our work, commit and push too, we need an ISO of the unbundled version with just filebrowser bundled remember, thanks
+- User requested final actions: build and test latest unbundled ISO variant (only filebrowser bundled), then commit and push changes.
+
+> Where is the ISO?
+- User asked where ISO is; current archived unbundled builder run is failing before artifact generation and must be repaired.
+
+> please do not miss AIUI in the release build or remove it from the nodes whatever you do
+- Critical release constraint: AIUI must remain bundled in release artifacts and must never be removed from existing nodes during update/deploy.

docs/STATUS.md

@@ -0,0 +1,667 @@
+# RESUME HERE — Rust orchestrator migration
+
+Updated: 2026-04-23 (Install UX polish: phase-based progress bar, post-install scanner kick for instant Launch button, .23 VPS retired with auto-purge migration, frontend/backend deployed to .228 as v1.7.43-alpha.)
+
+**To resume this work, SSH into the ThinkPad and run `opencode` from `~/Projects/archy/`. Or work from the laptop via the SSHFS mount at `~/mnt/archy-thinkpad/`.**
+
+---
+
+## ✅ INSTALL UX POLISH + .23 RETIREMENT — SHIPPED (v1.7.43-alpha)
+
+**Rounds 3–5 + config migration + changelog (2026-04-23)** — 5 commits on `main` (unpushed per user mirror protocol):
+
+- `8cc84ebc` `feat(install): phase-based progress bar replaces unparseable pull bytes` — `podman pull` emits zero parseable progress when stderr is piped (no TTY), so the legacy byte-counting regex never matched. Replaced with 7 phase-based levels: Preparing (5%) → PullingImage (20%) → CreatingContainer (70%) → StartingContainer (80%) → WaitingHealthy (88%) → PostInstall (95%) → Done (100%). UI maps phases to fixed % and only advances forward (`Math.max`). Final phase label renamed from "Running post-install…" to "Finalizing…" after user feedback that it read like a regression to the install step.
+- `f86d86c3` `fix(install): kick scanner post-install so Launch button appears immediately` — scan runs every 60s; post-install the state flipped to Running but the skeletal install-time manifest (`interfaces: None`) persisted until next scan, so `canLaunch(pkg)` returned false for up to a minute. Added `scan_kick: Arc<Notify>` + `scan_tick: Arc<watch::Sender<u64>>` on `RpcHandler`. Scan loop uses `tokio::select!` between the 60s interval and the notify. New `kick_scanner_and_wait` helper (2s timeout) called in install/update success paths BEFORE writing Running, so a fresh manifest lands first. Merge during Installing/Updating uses `merge_preserving_transitional` (keeps state, takes fresh manifest).
+- `22052325` `chore: retire .23 VPS mirror, promote .168 OVH to primary` — dropped `DEFAULT_TERTIARY_MIRROR_URL`, promoted `.168` to `DEFAULT_SECONDARY_MIRROR_URL` as "Server 1 (OVH)". 2-entry default registry (.168 priority 0, tx1138 priority 10). Trusted-registry allowlist, catalog fallback, installer ISO registries, `marketplaceData.ts` REGISTRY, `image-versions.sh` all updated. Tests updated for new default counts (registry 3→2, mirror 3→2). URL-parser fixture tests in `update.rs` retain `.23` strings intentionally — they exercise string-parsing logic, not policy.
+- `0ee16820` `fix(config): auto-purge decommissioned .23 VPS from saved registry/mirror configs` — `load_mirrors`/`load_registries` normally only ADD missing defaults (explicit removals stick, by design). Existing nodes have `.23` baked into their saved `update-mirrors.json` + `config/registries.json` and would pay timeouts forever against a dead host. Added targeted one-time migration in both loaders: `.retain(|m| !m.url.contains("23.182.128.160"))` before the defaults-merge step. Narrow-scope exception to the stickiness rule, documented in-code. Triggers lazily on next load (install RPC, update RPC, Settings UI open).
+- `008da477` `docs(changelog): add v1.7.43-alpha entry covering async lifecycle + .23 retirement` — 4 release-note bullets in `AccountInfoSection.vue` describing async-spawn, phase progress, scanner kick, and .23 retirement from the operator's perspective. Historical "Server 3 (OVH)" entries in older changelog blocks left intact — they describe what shipped at the time.
+
+**Deployed to .228**:
+- Backend binary md5 `d2b619949f19815faaeab10429e36ba0` at `/usr/local/bin/archipelago`.
+- Frontend at `/opt/archipelago/web-ui/` (includes marketplaceData.ts .168 update + v1.7.43-alpha changelog entry). Deployed bundle verified: `.168` present in `Settings-*.js` + `Marketplace-*.js`, `.23` absent from all assets.
+- `/var/lib/archipelago/update-mirrors.json` + `config/registries.json` were manually deleted + regenerated with new defaults during Round 5 verification; migration code will handle any other node on first load.
+- Rollback targets from Round 2 still valid: `/usr/local/bin/archipelago.bak-pre-async-install` + `/opt/archipelago/web-ui.bak-pre-async-install/`.
+
+**Git remotes cleaned on .116** (working-copy change only, not in any commit):
+- `git remote remove gitea-vps` (dropped the .23 Gitea remote).
+- `git remote set-url --delete --push origin http://.../23.182.128.160:3000/...` (dropped .23 from origin multi-push alias).
+- Remaining push targets: `tx1138` (canonical), `gitea-local` (localhost Gitea), `gitea-vps2` (.168 OVH).
+
+**Rollback Rounds 3–5** (same command as Round 2 — backups predate all of this):
+```
+ssh archy228 'sudo cp -a /usr/local/bin/archipelago.bak-pre-async-install /usr/local/bin/archipelago && sudo rsync -a --delete /opt/archipelago/web-ui.bak-pre-async-install/ /opt/archipelago/web-ui/ && sudo systemctl restart archipelago && sudo systemctl reload nginx'
+```
+
+---
+
+## ✅ ASYNC-SPAWN LIFECYCLE FIX — SHIPPED (Stop/Start/Restart + Install/Uninstall/Update)
+
+**Round 2 (2026-04-23, install/uninstall/update)** — 3 commits on `main`:
+
+- `2d5b859e` `feat(rpc): async-spawn install/uninstall/update lifecycle` — new `api/rpc/package/async_lifecycle.rs` with `spawn_package_install`, `spawn_package_uninstall`, `spawn_package_update`. Dispatcher + handler thread `self: Arc<Self>` so spawned tasks own their Arc. Install/update Ok arms explicitly set `Running` because `merge_preserving_transitional` refuses to let the scanner overwrite `Installing`/`Updating`. Removed redundant inner "already updating" guard in `update.rs`. Transient install entry uses empty icon (see commit 3 rationale).
+- `0733ac40` `fix(ui): shorten install/uninstall/update timeouts for async RPCs` — drop 11m/45m timeouts to 15s across `rpc-client.ts`, `stores/server.ts`, and the 5 direct call sites in `Marketplace.vue`, `Discover.vue`, `MarketplaceAppDetails.vue`. Return types updated to `{ status, package_id }`.
+- `e471ef75` `fix(rpc): empty icon in transient install entry to avoid broken-image flicker` — `progress.rs::create_installing_entry` no longer hardcodes `/assets/img/app-icons/<id>.png`. About half of bundled apps use `.svg`/`.webp` icons; the frontend's fallback chain (`backend_icon || curated.icon || placeholder`) now lands on the correct curated extension.
+
+**Deployed to .228** (binary md5 `f66857b3b8b3640c8cac8bd25fe508ec` at `/usr/local/bin/archipelago`, backup at `/usr/local/bin/archipelago.bak-pre-async-install`; frontend at `/opt/archipelago/web-ui/`, backup at `/opt/archipelago/web-ui.bak-pre-async-install/`). User confirmed: uninstall fast and responsive, install of LND + SearXNG clean, icon flicker fixed.
+
+**Known out-of-scope issue**: Vaultwarden container itself exits immediately on start with an internal error. The async wrapper correctly detects this via post-start exit verification and removes the state entry. Needs separate vaultwarden container-config investigation.
+
+**Rollback Round 2 (if ever needed)**:
+```
+ssh archy228 'sudo cp -a /usr/local/bin/archipelago.bak-pre-async-install /usr/local/bin/archipelago && sudo rsync -a --delete /opt/archipelago/web-ui.bak-pre-async-install/ /opt/archipelago/web-ui/ && sudo systemctl restart archipelago && sudo systemctl reload nginx'
+```
+
+---
+
+**Round 1 (Stop/Start/Restart)** — 4 commits on `main` (unpushed per user mirror protocol):
+
+- `44cd5eef` `feat(rpc): spawn_transitional helper for async lifecycle ops` — new `api/rpc/transitional.rs` with `Op::{Stop,Start,Restart}` and `RpcHandler::spawn_transitional` / `flip_to_transitional` / `set_state` helpers. `install_log` re-exported so sibling modules can use it.
+- `19a99ca9` `fix(rpc): async container stop/start/restart; widen state mapping` — `container.rs` start/stop rewritten + restart added; `container-list` now emits all transitional variants instead of falling back to `"unknown"`. `dispatcher.rs` registers `container-restart`. `package/runtime.rs` mirrored with `do_package_*` helpers inside `tokio::spawn` and revert-on-error.
+- `6712810b` `fix(state): preserve transitional state across container scans` — `server.rs` scan merge now keeps transitional states while taking fresh observability fields; 1200s stuck-timeout escape hatch via `transitional_since: HashMap<String, Instant>`. Three passing `server::merge_tests`.
+- `9ce28f08` `fix(ui): single-button lifecycle control with transitional labels` — `ContainerApps.vue` and `ContainerAppDetails.vue` use a single primary button driven by `getAppVisualState()`. **Dashboard now routes through `container-start`/`container-stop`** (the async RPCs) instead of the legacy synchronous `bundled-app-*` path. `ContainerStatus.vue` widened to render all new variants.
+
+**Deployed to .228** (ThinkPad demo device):
+- Binary at `/usr/local/bin/archipelago` (md5 `de86b63f74c7e6fe6e555ffe30b86b4f`), backup at `/usr/local/bin/archipelago.bak-pre-async-stop`.
+- Frontend at `/opt/archipelago/web-ui/`, backup at `/opt/archipelago/web-ui.bak-pre-async-stop/`.
+- Release build took 3m56s on .116. Deploy via scp + atomic `install -m 755` + `systemctl restart archipelago`. `nginx -t` + `systemctl reload nginx` for frontend.
+
+**Manual verification**: user clicked Stop on LND in the dashboard. Button flipped to `Stopping…` instantly, held for the full graceful-stop window, transitioned to `Start` when `podman stop` completed. No mid-flight revert to Running. User sign-off: _"absolutely beautiful"_.
+
+**Rollback (if ever needed)**:
+```
+ssh archy228 'sudo cp /usr/local/bin/archipelago.bak-pre-async-stop /usr/local/bin/archipelago && sudo rsync -a --delete /opt/archipelago/web-ui.bak-pre-async-stop/ /opt/archipelago/web-ui/ && sudo systemctl restart archipelago && sudo systemctl reload nginx'
+```
+
+### Follow-ups to consider
+
+1. **Chaos matrix / Step 11** — the original next-step gated behind this fix. Now unblocked.
+2. **bundled-app-start / bundled-app-stop** — still synchronous in the backend. Dashboard no longer calls them, but the RPC methods remain for any external caller. Decide: deprecate, or mirror the async-spawn treatment for parity.
+3. **`transitional_since` persistence** — currently in-memory only, so a backend restart mid-stop loses the timeout anchor. Acceptable for now (scan loop re-observes live podman state and reconciles), but worth revisiting if crash-recovery stories tighten.
+4. **Test regressions inventory** — the full `cargo test -p archipelago` run on .116 shows 22 pre-existing failures in unrelated modules (mesh/wallet/credentials/avatar/session/transport/update-mirrors/fips/identity_manager/image_versions). Unrelated to this work but tech debt. Log at `/tmp/cargo-test-all.log` on .116.
+5. **Amend STATUS.md's older "NEXT SESSION — START HERE" section** (below) — it is now stale. Left in place for historical reference of how the fix was designed; delete on the next pass if it gets confusing.
+
+---
+
+## ⚡ NEXT SESSION — START HERE (historical — fix above is now shipped)
+
+**Goal**: implement async-spawn lifecycle fix so the dashboard never shows a frozen spinner again. User mandate: _"best server containers in the world"_. Do not ship the chaos matrix (Step 11) until this lands and manual LND stop verifies instant RPC + live `Stopping…` label.
+
+### How to work on this repo (SSH + SSHFS setup)
+
+You are likely running on the **laptop** (macOS). The repo lives on the **ThinkPad** (.116). There are two access paths, use both in parallel:
+
+1. **SSHFS mount at `~/mnt/archy-thinkpad/`** — for all file ops (`read`/`edit`/`write`/`glob`/`grep`).
+2. **Direct SSH** — for everything that isn't file ops: `git`, `cargo`, `npm`, `systemctl`, running the server, tailing logs.
+
+See the "FUSE / SSHFS development loop" section below for the full mount lifecycle — that's _the_ thing that makes this dev setup work, and it will break periodically.
+
+### FUSE / SSHFS development loop
+
+**Why this exists**: editing the repo directly on the ThinkPad over raw SSH means no IDE, no tool-native file reads, no glob/grep speed. SSHFS mounts the remote filesystem as a local directory so OpenCode's file tools work transparently. But SSHFS is a leaky abstraction — know the gotchas or you'll waste hours.
+
+**Stack** (macOS laptop):
+- **macFUSE** — kernel extension providing FUSE on macOS. Install via `brew install --cask macfuse` (requires reboot + security approval in System Settings the first time).
+- **sshfs** — userspace mount tool. Install via `brew install gromgit/fuse/sshfs-mac` (the homebrew core `sshfs` was removed; use this tap).
+- Verify: `which sshfs` → `/opt/homebrew/bin/sshfs`, `sshfs --version` → `SSHFS version 2.10 / FUSE library version 2.9.9`.
+
+**Actual mount command currently running** (verified from `ps`):
+```
+sshfs archy:Projects/archy /Users/dorian/mnt/archy-thinkpad \
+  -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,volname=archy-thinkpad
+```
+
+Breakdown:
+- `archy:Projects/archy` — remote path via the `archy` SSH alias (uses `~/.ssh/archy_opencode`, no password prompt).
+- `~/mnt/archy-thinkpad` — local mount point. Create once: `mkdir -p ~/mnt/archy-thinkpad`.
+- `reconnect` — sshfs auto-reconnects if the TCP session drops (WiFi flap, laptop sleep). Without this, the mount turns into a zombie immediately.
+- `ServerAliveInterval=15` — sends a keepalive every 15s.
+- `ServerAliveCountMax=3` — disconnect after 3 missed keepalives (45s). Tune up if your network is flaky.
+- `volname=archy-thinkpad` — Finder display name.
+
+**Check mount health**:
+```
+mount | grep archy-thinkpad
+# should print: archy:Projects/archy on /Users/dorian/mnt/archy-thinkpad (macfuse, nodev, nosuid, synchronous, mounted by dorian)
+
+ls ~/mnt/archy-thinkpad/ | head
+# should list repo contents fast (<1s). If it hangs, mount is stale.
+```
+
+**Recovery when the mount hangs / goes stale** (this WILL happen — laptop sleeps, WiFi drops, ThinkPad reboots):
+```
+# 1. Force-unmount (macOS — `umount` alone often fails on a hung FUSE mount)
+sudo diskutil unmount force ~/mnt/archy-thinkpad
+# fallback if diskutil can't see it:
+sudo umount -f ~/mnt/archy-thinkpad
+
+# 2. Kill any zombie sshfs process
+pkill -f "sshfs archy:Projects/archy"
+
+# 3. Remount
+sshfs archy:Projects/archy ~/mnt/archy-thinkpad \
+  -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,volname=archy-thinkpad
+
+# 4. Verify
+ls ~/mnt/archy-thinkpad/ | head
+```
+
+If the mount point itself got wedged (`ls: /Users/dorian/mnt/archy-thinkpad: Device not configured`), the sequence above still works — macFUSE garbage-collects the inode after the force-unmount.
+
+**When to use which path** (rules, not suggestions):
+| Operation | Use | Why |
+|---|---|---|
+| `read` / `edit` / `write` | SSHFS mount | OpenCode tools want local paths |
+| `glob` / `grep` | SSHFS mount | Local FS traversal is fine; remote would need rg over SSH |
+| Reading many files | SSHFS mount | Each read is a round-trip but parallelizable |
+| `git status` / `git diff` / `git log` | SSH | Git over FUSE is painfully slow (lots of stat calls) |
+| `git add` / `git commit` | SSH | Same — commit times grow linearly with tree size on FUSE |
+| `cargo check` / `cargo test` / `cargo build` | SSH | Compiling over FUSE would take hours; cargo's incremental stat pattern destroys FUSE performance |
+| `npm install` / `npm run build` | SSH | Same reason — massive file churn |
+| Running the server / tailing journal | SSH | Service lives on .116 |
+| Deploying to .228 | SSH from .116 | SCP from ThinkPad; laptop isn't in the critical path |
+
+**Don't do this** (will bite you):
+- `cargo build` from the mount — will try to write target/ over FUSE, gets orders of magnitude slower, may hang.
+- `rsync` without `--exclude="._*"` — macOS writes AppleDouble metadata files, they leak to the remote as `._*` siblings of every real file. `.gitignore` already excludes them (commit `13858842`), but they clutter the tree.
+- Writing big binary files via the mount — use `scp` over SSH instead.
+- Relying on file-change-watcher tools (watchman, chokidar) — they get confused by FUSE event semantics.
+
+**Editing workflow in a typical session**:
+1. Laptop: OpenCode `read`s a file via `/Users/dorian/mnt/archy-thinkpad/...`. FUSE fetches it over SSH, caches briefly.
+2. Laptop: OpenCode `edit`s the file — FUSE writes the new bytes back to .116 immediately (synchronous mount).
+3. Laptop: `ssh archy "cd ~/Projects/archy && ~/.cargo/bin/cargo check -p archipelago"` — runs on the real filesystem on .116, sees the edit.
+4. Laptop: `ssh archy "cd ~/Projects/archy && git diff path/to/file"` — confirms the edit landed.
+5. Laptop: `ssh archy "cd ~/Projects/archy && git add path/to/file && git commit -m '...'"` — commit from .116.
+
+The SSHFS mount and the SSH shell are pointing at **the same inodes** — edits via the mount are instantly visible to `cargo`/`git` over SSH. There's no "sync" step.
+
+**Cache caveat**: macFUSE caches attributes briefly (default ~1s). If you write via SSH and read via the mount within that window, you may see stale metadata. The mount's `synchronous` flag (visible in `mount` output) minimizes but doesn't eliminate this. If you get a weird diff between what SSH and the mount report, re-read after a second, or `stat --file-system ~/mnt/archy-thinkpad/<file>` to force a refresh.
+
+**Direct SSH** access (use when FUSE isn't the right tool):
+   - `ssh archy` → `archipelago@192.168.1.116` using `~/.ssh/archy_opencode`
+   - `ssh archy228` → `archipelago@192.168.1.228` using `~/.ssh/archy_opencode`
+   - Full host form also works: `ssh archipelago@192.168.1.116` / `ssh archipelago@192.168.1.228` (same key resolves via IdentitiesOnly).
+
+### SSH keys — what's where
+
+**Laptop `~/.ssh/` (macOS, user `dorian`)**:
+| File | Purpose |
+|---|---|
+| `archy_opencode` / `.pub` | **Primary key for this project.** Unlocks both `archy` (.116) and `archy228` (.228). Created 2026-04-22 specifically for OpenCode work. |
+| `archipelago-deploy` / `.pub` | Older archipelago deploy key. Not needed for current work. |
+| `id_ed25519` / `.pub` | Personal default key. Not used by archy/archy228 configs (`IdentitiesOnly yes` forces `archy_opencode`). |
+| `id_ed25519_angor` / `.pub` | Angor project. Unrelated. |
+| `id_ed25519_start9` / `.pub` | Start9 project. Unrelated. |
+| `vps-ci-setup` / `.pub` | VPS CI. Unrelated. |
+| `config` | Host aliases (shown above) |
+
+**.116 `/home/archipelago/.ssh/`**:
+| File | Purpose |
+|---|---|
+| `authorized_keys` | Accepts: laptop's `archy_opencode.pub` + 3 other keys (4 lines total). |
+| `id_ed25519` / `.pub` | .116's OWN identity key. This is what lets `.116 → .228` work passwordless. |
+| `archipelago-deploy` | Symlink → `id_ed25519` (legacy alias). |
+| `id_ed25519_vps168` / `.pub` | For SSH to `146.59.87.168` (VPS). Unrelated to this work. |
+| `config` | Host entry for the VPS only. |
+
+**.228 `/home/archipelago/.ssh/`**:
+| File | Purpose |
+|---|---|
+| `authorized_keys` | Accepts: laptop's `archy_opencode.pub` + .116's `id_ed25519.pub` + 2 others (4 lines total). |
+| _(no `id_ed25519`)_ | .228 has no outbound key — it's a terminal node. Don't try to `ssh` _from_ .228 _to_ anywhere. |
+
+**Connectivity matrix (all verified 2026-04-23)**:
+| From → To | Works passwordless | Via |
+|---|---|---|
+| Laptop → .116 | ✅ | `archy_opencode` |
+| Laptop → .228 | ✅ | `archy_opencode` |
+| .116 → .228 | ✅ | .116's `id_ed25519` |
+| .228 → anywhere | ❌ | no outbound key (by design) |
+
+### Sudo — verified state
+
+**.116** (dev ThinkPad):
+- User `archipelago` is in `sudo` group.
+- Sudo password required: **`ThisIsWeb54321@`**
+- Sudoers drop-ins present: `/etc/sudoers.d/archipelago-ci`, `/etc/sudoers.d/archipelago-wg` (scope-limited NOPASSWD for specific CI/wg commands — not full NOPASSWD).
+- For most dev work you don't need sudo on .116.
+
+**.228** (prod kiosk):
+- User `archipelago` has **full passwordless sudo** via `/etc/sudoers.d/archipelago` containing `archipelago ALL=(ALL) NOPASSWD:ALL`.
+- User is also in `sudo` group.
+- Sudo password (if ever prompted, shouldn't be): **`archipelago`**
+- Dashboard password: **`password123`**
+
+### Cargo / npm / paths
+
+- **Cargo PATH gotcha**: non-interactive SSH login has no cargo in PATH. Always use `~/.cargo/bin/cargo` over SSH.
+  - Example: `ssh archy '~/.cargo/bin/cargo check -p archipelago' --workdir ~/Projects/archy/core`
+  - Or cd first: `ssh archy 'cd ~/Projects/archy && ~/.cargo/bin/cargo check -p archipelago'`
+- **Long cargo builds** (>2 min Bash tool timeout): launch detached and poll the log:
+  ```
+  ssh archy 'cd ~/Projects/archy && nohup ~/.cargo/bin/cargo build --release -p archipelago > /tmp/cargo-build.log 2>&1 < /dev/null & disown'
+  ssh archy 'tail -30 /tmp/cargo-build.log'
+  ssh archy 'pgrep -a cargo'   # to check if still running
+  ```
+- **npm / frontend** lives at `~/Projects/archy/neode-ui/` on .116 (also accessible via laptop mount at `~/mnt/archy-thinkpad/neode-ui/`). Node is on interactive PATH; for scripted SSH, `source ~/.nvm/nvm.sh && nvm use` or call the absolute path if nvm is used.
+- Repo on .116: `~/Projects/archy/` (Cargo workspace at `core/Cargo.toml`).
+- Web root on .228: check `/etc/nginx/sites-enabled/` for the live path; historically `/var/lib/archipelago/web-ui/` or `/opt/archipelago/web-ui/`.
+
+### Deploying new server binary to .228
+
+```
+# 1. Build on .116 (detached — takes ~3-5 min for release)
+ssh archy 'cd ~/Projects/archy && nohup ~/.cargo/bin/cargo build --release -p archipelago > /tmp/cargo-build.log 2>&1 < /dev/null & disown'
+# wait / tail log until "Finished `release` profile"
+
+# 2. SCP .116 → .228 (uses .116's id_ed25519 → .228's authorized_keys, passwordless)
+ssh archy 'scp ~/Projects/archy/core/target/release/archipelago archipelago@192.168.1.228:/tmp/archipelago.new'
+
+# 3. Atomic swap on .228 with backup
+ssh archy228 'sudo cp /usr/local/bin/archipelago /usr/local/bin/archipelago.bak-pre-async-stop && sudo mv /tmp/archipelago.new /usr/local/bin/archipelago && sudo chmod +x /usr/local/bin/archipelago && sudo systemctl restart archipelago'
+
+# 4. Verify
+ssh archy228 'systemctl status archipelago --no-pager | head -20 && sudo journalctl -u archipelago -n 50 --no-pager'
+```
+
+### Git workflow
+
+- Branch: `main` on .116, currently **22 commits ahead of `tx1138/main`**.
+- Remote `tx1138` exists but **do NOT push** — user mirrors to 4 Gitea remotes personally after reviewing.
+- Atomic commits, one logical change per commit. Conventional Commits format (`feat:`, `fix:`, `docs:`, `refactor:`, `chore:`, `test:`, `perf:`).
+- Never `--amend` unless the commit you're amending was created in this session AND has not been pushed. Safer: new commit.
+- Never `--force` push. Never modify git config.
+- If pre-commit hooks fail, create a NEW commit with the fix — don't `--amend` after a failed commit.
+
+### Other
+
+- Full destructive latitude on both nodes. Announce multi-hour ops (OTA, full rebuild, apt upgrade). Don't ask for routine stop/start/rebuild permission.
+- No ship pressure. Do it properly.
+- Use `question` tool for ambiguous decisions (don't guess user intent on design choices).
+- Keep `docs/STATUS.md` fresh between sessions — it IS the session handoff.
+
+### Hosts reference (quick)
+
+| Host | IP | SSH alias | Role | Dashboard | Sudo |
+|---|---|---|---|---|---|
+| `archy` (ThinkPad X250) | 192.168.1.116 | `ssh archy` | dev host, Debian 13 | `archipelago` | `ThisIsWeb54321@` |
+| `archy228` (HP ProDesk) | 192.168.1.228 | `ssh archy228` | prod kiosk, Rust orchestrator | `password123` | NOPASSWD (fallback `archipelago`) |
+
+### Bug being fixed
+
+Dashboard sequence when user clicks **Stop LND**:
+1. UI collapses Start/Stop buttons to single spinner-button ("Stopping…") via `loadingApps.add('lnd')`.
+2. Frontend calls `container-stop` RPC. Server runs `podman stop -t 330 lnd` **synchronously inside the RPC handler** (via `orchestrator.stop()`). RPC blocks up to **5.5 min** for LND (330s timeout + overhead).
+3. Meanwhile the 30-second package-scan loop in `server.rs:scan_and_update_packages` keeps running. It rebuilds `PackageDataEntry` from podman inspect — podman still reports `running` (stop hasn't completed) — and **blindly overwrites** the store entry at `server.rs:854`.
+4. `container-list` RPC reads `state_manager` snapshot → returns `state = "running"`.
+5. Frontend polling sees `running` → `getAppState()` returns `'running'` → the two-button (Start | Stop) block re-renders → the transitional button disappears → **UI looks like the stop silently failed**.
+6. Eventually `podman stop` finishes → next scan → state flips to `Stopped` → buttons change _again_.
+
+Net visible bug: button spins briefly, reverts to Running, then several minutes later suddenly shows Stopped. User rightly calls this "out of sync and confusing".
+
+### Decisions already locked in (do not re-ask)
+
+- **Full scope fix** (not minimal hotfix). User chose "Go full scope, do it right".
+- **Async-spawn lives in the RPC layer**, not in the `ContainerOrchestrator` trait. Trait stays synchronous so the reconciler, boot flow, unit tests, and the chaos harness retain deterministic behaviour.
+- **`PackageState` already has `Stopping`/`Starting`/`Restarting`/`Installing`/`Updating`/`Removing`** variants — enum at `core/archipelago/src/data_model.rs:107-124`. No schema change needed.
+- **UI collapses to one full-width button** with spinner during every transitional state. Labels: Start / Stop / Starting… / Stopping… / Restarting… / Installing… / Updating… / Removing… / Install (when `not-installed`).
+- **Helper API shape**: `RpcHandler::spawn_transitional(op: Op, app_id: String)` where `Op` is an enum `{Stop, Start, Restart}`. Helper dispatches to `orchestrator.stop/start/restart` internally, knows each op's transitional+final states, handles error → revert + `install_log()`.
+- **`mark_user_stopped` must run BEFORE the spawn** (preserves ordering the crash recovery layer depends on — see `runtime.rs:145-148`).
+
+### Implementation order (4 commits, local only)
+
+**Commit 1 — `feat(rpc): spawn_transitional helper for async lifecycle ops`**
+- New file: `core/archipelago/src/api/rpc/transitional.rs` (or extend `container.rs`; prefer new file for cohesion with future stacks/package variants)
+- `enum Op { Stop, Start, Restart }` with `transitional_state()`, `final_state_on_success()`, `log_prefix()`, and async `dispatch(&orch, &app_id)` method
+- `impl RpcHandler { pub(super) async fn spawn_transitional(&self, op: Op, app_id: String) -> Result<()> }`
+  - Capture `Arc<dyn ContainerOrchestrator>` + `Arc<StateManager>` clones
+  - Set transitional state via `state_manager.update_data()` (if entry exists; skip if not — Start on never-installed shouldn't create an entry)
+  - `tokio::spawn(async move { ... })`
+  - Inside spawn: `install_log("{LOG_PREFIX}: {app_id}")`, `op.dispatch(&orch, &app_id).await`, on success set final state, on error log + `install_log("{LOG_PREFIX} FAIL: …")` + revert state to previous (cache pre-transition state in a local)
+  - Return `Ok(())` immediately after spawn
+
+**Commit 2 — `fix(rpc): async container stop/start/restart; widen state mapping`**
+- `api/rpc/container.rs:85-107` — rewrite `handle_container_stop` body: `validate_app_id`, `mark_user_stopped`, `spawn_transitional(Op::Stop, app_id.to_string()).await?`, return `Ok(json!({ "status": "stopping" }))`
+- `api/rpc/container.rs:61-83` — rewrite `handle_container_start`: `clear_user_stopped`, `spawn_transitional(Op::Start, …)`, return `{ "status": "starting" }`
+- **Add** `handle_container_restart` (currently missing in `container.rs` — only exists as `package.restart` at `runtime.rs:176-242`). Register RPC route name `container-restart`. Add matching frontend client method in `container-client.ts`.
+- `api/rpc/container.rs:148-154` — widen the `container-list` state mapping: add arms for `Stopping → "stopping"`, `Starting → "starting"`, `Restarting → "restarting"`, `Installing → "installing"`, `Updating → "updating"`, `Removing → "removing"`, `Installed → "installed"`, `CreatingBackup`/`RestoringBackup`/`BackingUp` → their kebab-case strings. No more `"unknown"` fallback unless the variant is genuinely unknown.
+- Mirror same spawn treatment in `api/rpc/package/runtime.rs`: `handle_package_start` (L28-119), `handle_package_stop` (L122-173), `handle_package_restart` (L176-242). Keep the existing verification loops (post-start exit-check at L82-117; restart stop+start fallback at L215-235) _inside_ the spawned future, not in the RPC body.
+
+**Commit 3 — `fix(state): preserve transitional state across container scans`**
+- `server.rs:847-857` — in the merge loop, before the `merged.insert(id.clone(), pkg.clone())` overwrite, check `merged.get(id).state` and skip overwrite if it's transitional: `matches!(existing.state, Installing | Stopping | Starting | Restarting | Updating | Removing | CreatingBackup | RestoringBackup | BackingUp)`
+- Still allow _non-state_ fields (lan_address, health, ports) to update. Simplest: when existing is transitional, keep `existing.state` but merge updated fields from `pkg`. Write a tiny helper `merge_preserving_transitional(existing, fresh) -> PackageDataEntry`.
+- Unit test: construct `existing.state = Stopping`, `fresh.state = Running`, assert merged.state stays `Stopping`.
+- **Also check**: Is there a timeout escape hatch? If `Stopping` is set and podman actually finishes but the spawn died before writing the final state (process crash, panic), the entry will be stuck `Stopping` forever. Mitigation: track a `transitional_since: Instant` in the entry (not persisted, just in-memory side table on StateManager), and if > 2× the stop timeout has elapsed, allow podman scan state to override. Scope for this commit or follow-up — lean toward: include it, because fleet reliability matters.
+
+**Commit 4 — `fix(ui): single-button lifecycle control with transitional labels`**
+- `neode-ui/src/api/container-client.ts` — extend `ContainerStatus.state` union to: `'created' | 'running' | 'stopped' | 'exited' | 'paused' | 'unknown' | 'stopping' | 'starting' | 'restarting' | 'installing' | 'updating' | 'removing' | 'installed'`. Add `restartContainer(appId)` method calling `container-restart`.
+- `neode-ui/src/stores/container.ts` — add computed `getAppVisualState(appId)` that returns one of: `'not-installed' | 'running' | 'stopped' | 'starting' | 'stopping' | 'restarting' | 'installing' | 'updating' | 'removing'`. Maps `exited`→`stopped`, `created`→`stopped`, `paused`→`stopped`, `installed`→`stopped`. Add `restartContainer(appId)` action (sets `loadingApps` for request dedup, calls client, does NOT `fetchContainers` immediately because server will broadcast state; a final `fetchContainers` after a short delay can backstop if WebSocket push is absent).
+- `neode-ui/src/views/ContainerApps.vue:85-136` — replace the two-button conditional with a single full-width button bound to `getAppVisualState(app.id)`. Table:
+  | visual state    | click action   | label          | spinner | disabled |
+  |-----------------|----------------|----------------|---------|----------|
+  | `not-installed` | installApp     | Install        | no      | no       |
+  | `running`       | stopContainer  | Stop           | no      | no       |
+  | `stopped`       | startContainer | Start          | no      | no       |
+  | `starting`      | —              | Starting…      | yes     | yes      |
+  | `stopping`      | —              | Stopping…      | yes     | yes      |
+  | `restarting`    | —              | Restarting…    | yes     | yes      |
+  | `installing`    | —              | Installing…    | yes     | yes      |
+  | `updating`      | —              | Updating…      | yes     | yes      |
+  | `removing`      | —              | Removing…      | yes     | yes      |
+  - Add a separate Restart button next to the primary one when state is `running`, calling new `restartContainer` action. Restart button hides while transitional.
+- `neode-ui/src/views/ContainerAppDetails.vue:83` (and full stop/start button blocks around L220, L232) — mirror the same single-button pattern.
+- Also audit line 239 of `ContainerApps.vue` (`some((app) => store.getAppState(app.id) === 'created')`) and the logic around lines 276, 295, 309, 312 — make sure they use `getAppVisualState` where appropriate.
+
+### Verification gates (do not skip)
+
+1. `~/.cargo/bin/cargo check -p archipelago` on .116 via SSH
+2. `~/.cargo/bin/cargo test -p archipelago` on .116 via SSH — at least the new merge helper test must pass
+3. Build release binary on .116: `nohup ~/.cargo/bin/cargo build --release -p archipelago > /tmp/cargo-build.log 2>&1 < /dev/null & disown`. Poll until done.
+4. SCP binary to .228 `/usr/local/bin/archipelago`, back up prior to `/usr/local/bin/archipelago.bak-pre-async-stop`. `sudo systemctl restart archipelago` on .228.
+5. **Manual LND stop test on .228**:
+   - Open dashboard, confirm LND is Running (first: `ssh archipelago@192.168.1.228 'podman start lnd'` — LND is currently Exited(0) from the demo)
+   - Click Stop
+   - Expected: button _immediately_ becomes "Stopping…" with spinner (RPC returns <1s)
+   - Dashboard should stay on "Stopping…" for ~5 min
+   - Then flip to "Start" button with label "Start"
+   - At no point should it revert to "Running" mid-stop
+6. Same test with Bitcoin Core stop (longest timeout, 600s)
+7. Frontend build: `cd ~/Projects/archy/neode-ui && npm run type-check && npm run build`. Rsync `dist/` to `archipelago@192.168.1.228:/var/lib/archipelago/web-ui/` (or wherever the active web root is — check `/etc/nginx` on .228 first).
+8. Then and only then: resume chaos matrix. First recover LND/ElectrumX via UI (great end-to-end test of the new async Start path), then run smoke → full 32-case matrix.
+
+### Key files (exact lines of interest)
+
+- `core/archipelago/src/api/rpc/container.rs:85-107` — `handle_container_stop` (blocking — target of fix)
+- `core/archipelago/src/api/rpc/container.rs:61-83` — `handle_container_start`
+- `core/archipelago/src/api/rpc/container.rs:148-154` — narrow state mapping (drops transitional → "unknown")
+- `core/archipelago/src/api/rpc/package/runtime.rs:11-24` — `stop_timeout_secs` table (reference, unchanged)
+- `core/archipelago/src/api/rpc/package/runtime.rs:122-173` — `handle_package_stop` (also blocking, mirror treatment)
+- `core/archipelago/src/api/rpc/package/runtime.rs:28-119` — `handle_package_start`
+- `core/archipelago/src/api/rpc/package/runtime.rs:176-242` — `handle_package_restart`
+- `core/archipelago/src/api/rpc/package/progress.rs` — existing broadcast pattern to mirror (`set_install_progress`, `set_uninstall_stage`)
+- `core/archipelago/src/api/rpc/mod.rs:62-100` — `RpcHandler` struct (already holds `Arc<dyn ContainerOrchestrator>` + state_manager)
+- `core/archipelago/src/server.rs:812-857` — `scan_and_update_packages` (merge loop at L850-857 is where transitional-state clobber happens)
+- `core/archipelago/src/container/docker_packages.rs:636-663` — `convert_state` + `package_state_str` (read-only reference, no change)
+- `core/archipelago/src/container/traits.rs` — `ContainerOrchestrator` trait (stays synchronous, do not change)
+- `core/archipelago/src/crash_recovery.rs` — `mark_user_stopped` / `clear_user_stopped` (call order preserved)
+- `core/archipelago/src/data_model.rs:107-124` — `PackageState` enum (no change — all variants exist)
+- `neode-ui/src/api/container-client.ts` — `ContainerStatus` type + RPC methods (extend)
+- `neode-ui/src/stores/container.ts:93-312` — Pinia store (add `getAppVisualState`, add `restartContainer` action)
+- `neode-ui/src/views/ContainerApps.vue:85-136, 239, 276, 295, 309-312, 383` — two-button block + state reads
+- `neode-ui/src/views/ContainerAppDetails.vue:83, 220, 232` — details page Stop/Start
+
+### Chaos harness (not in repo — lives on .116)
+
+- `archipelago@192.168.1.116:~/ui-chaos/` — deployed, playwright + deps installed, smoke test for bitcoin-core passes (2.1 min). LND/ElectrumX/bitcoin-ui smoke tests not yet run (blocked on the async-stop fix landing; LND currently Exited on .228 from the demo).
+- `/tmp/chaos/` on laptop — canonical source for rsync to .116.
+- Run: `cd ~/ui-chaos && npx playwright test tests/<spec>`
+- Target: 32 cases = 4 core containers × 8 scenarios (install-fresh, graceful-stop, sigkill, rm-container, oom-kill, rm-image, restart-service, network-partition).
+- Uses SSH+Playwright hybrid per design; includes the `bash -lc '<escaped>'` single-quote fix for ssh argv flattening and JSON-parsed `podman inspect` instead of Go templates.
+
+### Pre-existing bugs still deferred (do not fix until Stop UX lands)
+
+1. `archipelago --version` spawns server (should be a pure CLI query)
+2. RPC unknown-method returns generic error (should return method-not-found with the bad method name)
+3. `docker_packages.rs` filters out UI containers (`archy-lnd-ui`, `archy-electrs-ui`) — some views need them visible
+4. `lnd.lan_address` stale on .228
+5. first-boot silent failure on some hardware
+6. `web-ui.failed.*` scar on .228 (benign systemd unit state)
+7. `test_parse_image_versions` pre-existing broken assertion — fix or `#[ignore]` when touching that area
+
+---
+
+## Where we are
+
+Working through the 11-step plan in [`rust-orchestrator-migration.md`](./rust-orchestrator-migration.md).
+
+- [x] **Step 1** — `3767c267` ContainerConfig schema with `build:`, `ResolvedSource` enum, `resolve()`, 10 tests
+- [x] **Step 2** — `34af4d9d` ContainerRuntime trait gained `image_exists` + `build_image`, 4 argv tests, 25/25 pass
+- [x] **Step 3** — `b6a04d31` ProdContainerOrchestrator (999 LOC), 16 tests all pass, not yet wired to main.rs
+- [x] **Step 4** — `e8a59c93` ContainerOrchestrator trait, RpcHandler uses it in prod (+ `13858842` chore gitignore ._*)
+- [x] **Step 5** — `fc39b04b` BootReconciler with Arc<Notify> shutdown, 4 paused-time tests pass
+- [x] **Step 6** — `48f08aa3` main.rs wire-up (orchestrator construction + adopt_existing + BootReconciler spawn + shutdown Notify)
+- [x] **Step 7** — `069bc4a5` bitcoin-ui pre-start hook + embedded nginx.conf template (8 unit tests + 1 integration test), 39/39 container:: tests pass
+- [x] **Step 8a** — `a0707f4d` retire archipelago-reconcile.{service,timer} + ISO builder touchpoints, keep scripts for update.rs
+- [x] **Step 9** — **Hot-swap on .228 verified.** All three UIs (bitcoin-ui/lnd-ui/electrs-ui) installing + serving HTTP 200.
+- [x] **.228 dashboard bugs** — ExtraHost `192.168.1.254` bug (`3ee192ba`) + LND macaroon permission bug (`be960023`). See "Post-Step 9 bug hunt" below.
+- [ ] **Step 8b** — Port remaining ~25 container creations from `first-boot-containers.sh` into `apps/<id>/manifest.yml`, then port `update.rs` to orchestrator (deferred, multi-day work)
+- [ ] **Step 8c** — Rename `first-boot-containers.sh` → `first-boot-setup.sh`, strip container ops, keep setup. Delete `reconcile-containers.sh` + `container-specs.sh`. Add ISO lines to copy `apps/` (final one-way door, requires 8b complete)
+- [ ] **Step 10** — Hot-swap + verify on .116 (adoption-heavy test — .116 already has all containers running)
+- [ ] **Step 11** — Chaos matrix on both nodes (all 8 scenarios × all containers incl. bitcoin-core)
+
+## Post-Step 9 bug hunt (.228, 2026-04-23)
+
+User reported three visible dashboard bugs after Step 9 verification:
+1. LND — "no connect details or QR"
+2. ElectrumX — stuck at "Building index (2 KB / ~130 GB)" for days
+3. bitcoin-core — in scope for chaos testing
+
+**Root cause #1 (ExtraHost, commit `3ee192ba`)**: `scripts/first-boot-containers.sh` computed `HOST_GATEWAY` from `ip route show default`, which returns the **LAN router** (e.g. 192.168.1.254), not the gateway to the host. Every container configured with `--add-host=host.containers.internal:$HOST_GATEWAY` was dialing the WiFi router instead of the host. LND crash-looped with `dial tcp 192.168.1.254:8332: connection refused`; ElectrumX's DAEMON_URL hit the same dead end; any `archy-net` bridge consumer of bitcoin-core's RPC was broken. Fixed by replacing the computed value with podman's magic `host-gateway` literal (supported since 4.4; we ship 5.4.2). Live-recreated bitcoin-core/electrumx/lnd on .228 with the corrected `--add-host`; LND reached chain backend; ElectrumX resumed indexing (went from 2 KB → 164.9 MB in under an hour).
+
+**Root cause #2 (macaroon permissions, commit `be960023`)**: LND's `admin.macaroon` lives at `/var/lib/archipelago/lnd/data/chain/bitcoin/mainnet/admin.macaroon`, owned by rootless-podman subordinate UID 100000, mode 640. The archipelago server runs as host UID 1000 and literally cannot read the file. Every LND RPC (`getinfo`, `connect-info`, `export-channel-backup`) plus the shared `lnd_client()` helper failed with "Failed to read LND admin macaroon". **Confirmed pre-existing on .116 too** (long-standing bug unrelated to Step 9). Fix: centralised the path as `LND_ADMIN_MACAROON_PATH`, added a `read_lnd_admin_macaroon()` helper in `api/rpc/lnd/mod.rs` that tries direct read first then falls back to `sudo -n cat` (mirrors the pattern already used for Tor onion hostnames). Four call sites routed through the helper. Verified on .228 — `curl -k https://<host>/lnd-connect-info` now returns 200 with cert + macaroon + tor_onion; dashboard QR unblocked.
+
+## Step 9 evidence (.228, 2026-04-23)
+
+- Binary: Step 9 build with `732df1b8` + `ba83f9bc`, scp'd to .228 as `/usr/local/bin/archipelago`. Old binary backed up at `/usr/local/bin/archipelago.bak-pre-step9`. Later replaced with macaroon-fix build (`be960023`); previous backed up at `/usr/local/bin/archipelago.bak-pre-macaroon`.
+- DEV_MODE override disabled (`override.conf` → `override.conf.disabled-pre-step9`).
+- `/opt/archipelago/apps/{bitcoin-ui,electrs-ui,lnd-ui}/manifest.yml` populated.
+- `/opt/archipelago/docker/bitcoin-ui/Dockerfile` replaced with the Step 7 version (no `COPY nginx.conf`). Old dir backed up as `bitcoin-ui.bak-pre-step9`.
+- Post-start snapshot:
+  - `🔗 Adopted 1 existing container(s): ["electrs-ui"]` — adoption of 13h-running container worked without recreation
+  - `🔄 Boot reconciler started (interval: 30s)` — every 30s, all three app_ids reach `NoOp` after the initial install pass
+  - `bitcoin-ui nginx.conf rendered path=/var/lib/archipelago/bitcoin-ui/nginx.conf auth_hash=97af1c18` — pre-start hook fires in `install_fresh`
+  - `curl localhost:8334` → HTTP 200 (bitcoin-ui), `:8081` → 200 (lnd-ui), `:50002` → 200 (electrs-ui)
+  - OCI memory limits correctly applied: bitcoin-ui=128Mi, electrs-ui=128Mi, lnd-ui=64Mi (was emitted as 0 pre-fix)
+
+## Bugs fixed this session
+
+1. **`parse_memory_limit` truncation bug** (`732df1b8`): lowercased "128Mi" → "128mi" → `trim_end_matches('m')` → "128i" → f64 parse fails → `None.unwrap_or(0)` → OCI `memory.limit:0` → systemd rejects MemoryMax=0. 6 regression tests; `create_container` now omits instead of emitting 0.
+2. **`archipelago.service` cgroup delegation missing** (`ba83f9bc`): belt-and-braces `Delegate=memory pids cpu io`.
+3. **ExtraHost `192.168.1.254`** (`3ee192ba`): see Post-Step 9 bug hunt above.
+4. **LND admin.macaroon unreadable** (`be960023`): see Post-Step 9 bug hunt above.
+
+## Commits made this session
+
+```
+3ee192ba fix(first-boot): use podman host-gateway magic for host.containers.internal
+be960023 fix(lnd): read admin macaroon via sudo fallback
+4b8ef0a0 docs: STATUS.md through Step 9 (.228 hot-swap verified)
+ba83f9bc feat(systemd): delegate cgroup controllers to archipelago.service
+732df1b8 fix: parse_memory_limit accepts Ki/Mi/Gi IEC binary suffixes
+a0707f4d refactor: retire archipelago-reconcile.{service,timer}  (Step 8a)
+1c81a739 docs: split Step 8 into 8a/8b/8c
+6e46932f docs: STATUS.md through Step 7
+069bc4a5 feat: bitcoin-ui pre-start hook (Step 7)
+```
+
+Branch is **19 commits ahead of tx1138/main** (local only — user pushes to mirrors personally).
+
+## Uncommitted state
+
+Clean. Only untracked: `tests/` (bats harness from prior session, not in scope), `tmp-dump-spec.py` (scratch).
+
+## Answered design questions (no need to re-ask)
+
+1. UI container naming → `archy-<app_id>` for UIs only; existing bitcoin-knots/lnd/electrumx keep bare names
+2. BITCOIN_RPC_AUTH injection → runtime bind-mount of nginx.conf (no build-args, no envsubst)
+3. Reconciler interval → 30 seconds
+4. Concurrency → per-app `Mutex<()>` in a `DashMap`
+5. Bash scripts → split into 8a/8b/8c; 8a done, 8b/8c deferred
+6. Step 4 extension → `ContainerOrchestrator` trait includes `install(app_id)`; the `manifest_path`-based install RPC stays dev-only
+7. Step 7 bitcoin-ui template → embed via `include_str!`, render on install + every reconcile, atomic tmp+rename to `/var/lib/archipelago/bitcoin-ui/nginx.conf`, bind-mount into container. RPC user hardcoded `archipelago`, password from `/var/lib/archipelago/secrets/bitcoin-rpc-password`.
+
+## Context: which host is what
+
+| Host | IP | Role | Dashboard pw | Sudo pw |
+|---|---|---|---|---|
+| `archy` | 192.168.1.116 | **Dev ThinkPad** (Lenovo X250, Debian 13). Currently running v1.7.42-alpha (DEV_MODE). Step 10 target. | archipelago | ThisIsWeb54321@ |
+| `archy228` | 192.168.1.228 | Kiosk HP ProDesk. **Step 9 landing zone** — now running Rust-orchestrator binary in prod mode. | password123 | archipelago |
+
+Both are development alpha nodes — **full destructive latitude**, no need to ask before stop/start/rebuild.
+
+## Next action
+
+**Step 10 — Hot-swap on .116.**
+
+Unlike .228 (which tested the INSTALL path for net-new UI containers), .116 tests the ADOPTION path: it already has all three UIs and all backend containers running from prior v1.7.42-alpha runs. We want to verify the new prod orchestrator adopts every existing container without recreating or restarting them.
+
+Steps:
+1. Disable DEV_MODE on .116 (check if override.conf exists — `/etc/systemd/system/archipelago.service.d/`)
+2. Stage the already-built binary at `~/Projects/archy/core/target/release/archipelago` → `/usr/local/bin/archipelago.new`
+3. Ensure `/opt/archipelago/apps/{bitcoin-ui,electrs-ui,lnd-ui}/manifest.yml` present (copy from repo)
+4. Ensure `/opt/archipelago/docker/bitcoin-ui/` matches the Step-7 layout (no baked nginx.conf)
+5. Snapshot: `podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.CreatedAt}}"` → save to `/tmp/pre-step10-containers.txt`
+6. `systemctl stop archipelago` → install binary → `systemctl start archipelago`
+7. Verify in journal: every running container appears in "Adopted N existing container(s)"; no container was recreated; all HTTP smokes still 200; BootReconciler reaches NoOp on every app_id after one pass.
+8. If broken → restore `.bak` binary, re-enable DEV_MODE override.
+9. Commit STATUS.md update.
+
+**Risk on .116:** If adoption fails mid-flight, we'd lose the running v1.7.42 backend that I'm currently typing at. Keep a second SSH session open to the ThinkPad for emergency revert. The backup plan is `install /usr/local/bin/archipelago.bak /usr/local/bin/archipelago && systemctl restart archipelago`.
+
+**After Step 10 we are blocked on Step 8b** (multi-day manifest ports) before Step 11 (chaos matrix).
+
+---
+
+### Why Step 8 got split (discovered 2026-04-23)
+
+Original plan was one commit "delete bash + edit ISO builder". But on investigation:
+- `first-boot-containers.sh` creates **30+ containers** with per-container logic (wallets, DB init, rpcauth derivations, post-create health waits). The repo only has manifests for 3 (bitcoin-ui, electrs-ui, lnd-ui from Step 7). Deleting bash now = brick first-boot on fresh installs.
+- Script also does non-container setup: secret generation (RPC pw, DB pw, FileBrowser admin pw), UID-mapping chowns for rootless podman subuid, Tor hostnames dir, WireGuard, firewall rules, nostr-relay dir. None of this lives in the Rust orchestrator.
+- `update.rs` (OTA update RPC) invokes `reconcile-containers.sh` at two sites. Deleting the script breaks package updates. Porting those call sites to the orchestrator needs all containers to have manifests.
+- Design doc §505 updated to split 8 → 8a/8b/8c. Only 8a (delete the reconcile systemd unit + timer, BootReconciler covers) is safe to execute before we port manifests.
+
+---
+
+# Archipelago — Current State, Plan, and Releases
+
+Updated: 2026-04-22
+
+This is the "pick this up tomorrow" page. One-stop summary of where we are, what the plan is, and what's shipped. Detailed plan lives in [`bulletproof-containers.md`](./bulletproof-containers.md).
+
+---
+
+## Current state
+
+### Fleet status
+
+All four Gitea mirrors are synced to v1.7.40-alpha:
+
+| Mirror | Host | Status |
+|---|---|---|
+| tx1138 | https://git.tx1138.com | ✅ v1.7.40-alpha live |
+| gitea-local | http://localhost:3000 | ✅ v1.7.40-alpha live |
+| .160 | http://23.182.128.160:3000 | ✅ v1.7.40-alpha live (Gitea recovered via `podman system renumber` — see below) |
+| .168 | http://146.59.87.168:3000 | ✅ v1.7.40-alpha live |
+
+Fleet test nodes:
+
+| Node | Version | State |
+|---|---|---|
+| .103 (dev) | 1.7.40 | running, being developed against |
+| .116 (this box) | 1.7.40 | healed manually via `systemd-run chmod 755 /opt/archipelago/web-ui` after v1.7.38/39 bug |
+| .198 | 1.7.39 → 1.7.40-alpha | healed manually |
+| .228 (primary test) | 1.7.40-alpha | healed manually; bitcoin-core + lnd + electrumx running; UI companions currently missing; bitcoin.conf rpcauth patched live |
+| .249 (ISO test) | unreachable today | |
+| .253 | 1.7.39 → 1.7.40-alpha | healed manually |
+
+### Known open issues (drives the plan below)
+
+1. **UI companion containers disappear** on .228 after daemon restarts — no auto-recreate (fixed by v1.7.45 Quadlet migration)
+2. **bitcoin.conf rpcauth drifts** from canonical secret → ElectrumX "Daemon connection problem" (fixed by v1.7.43 reconcile::derived)
+3. **`host.containers.internal`** resolves to LAN gateway inside containers on some versions (fixed by v1.7.42 containers.conf)
+4. **Podman state DB loss** requires manual recovery (fixed by v1.7.44 startup self-heal)
+5. **LND "Connect Wallet" info** vanishing after crashes — symptom of the same drift class as #2
+6. **ElectrumX not syncing** on .228 — downstream of #2; will resolve when bitcoin.conf is reconciled
+
+### Recent field incident (2026-04-22)
+
+- Shipped v1.7.38 + v1.7.39, both broke nginx fleet-wide because the frontend tarball's root dir was `drwx------` (700). Every node that OTA'd got 500 errors on every page.
+- Root-cause fix shipped in v1.7.40 (`create-release-manifest.sh` chmod + pre-ship assertion that `tar tvzf | head -1` shows `drwxr-xr-x`).
+- .160 Gitea was down all day (502) because its rootless podman's `libpod/bolt_state.db` had vanished. Recovered via clearing `/run/user/$UID/{containers,libpod,podman}` + `podman system renumber`.
+- Full failure-mode audit is in [`bulletproof-containers.md`](./bulletproof-containers.md).
+
+---
+
+## Plan
+
+We're shipping a level-triggered **reconciler + Quadlet** architecture over six incremental releases. Each release closes one failure mode. See [`bulletproof-containers.md`](./bulletproof-containers.md) for the full design, code layout, test harness, chaos matrix, sources.
+
+### Release roadmap
+
+| Release | Closes | What lands | Status |
+|---|---|---|---|
+| **v1.7.41** | FM5 (bad OTA nginx 500) | Post-OTA auto-rollback. New binary probes `https://127.0.0.1/` on boot; if non-200 within 90s, restores `web-ui.bak` + calls `rollback_update()` + restarts | **in flight — deploying to .228 for test** |
+| **v1.7.42** | FM4 (`host.containers.internal` wrong) | `/etc/containers/containers.conf` w/ `host_containers_internal_ip = 10.89.0.1`; every container gets `--add-host=host.archipelago:10.89.0.1` | pending |
+| **v1.7.43** | FM2 (config drift) | `reconcile::derived::render_bitcoin_conf` — pure fn over canonical secret, rewrites on drift. Same for `lnd.conf` | pending |
+| **v1.7.44** | FM6 (podman state loss) | Startup probe detects broken podman state, auto-recovers via `/run/user/$UID/*` clear + `system renumber` | pending |
+| **v1.7.45** | FM1 + FM3 (companion orphans) | `archy-bitcoin-ui` → Quadlet `.container` unit in `/etc/containers/systemd/`. systemd (not archipelago) owns it | pending |
+| **v1.7.46** | — | `archy-lnd-ui` → Quadlet | pending |
+| **v1.7.47** | — | `archy-electrs-ui` → Quadlet | pending |
+| **v1.7.48+** | all (full daemon refactor) | `core/archipelago/src/reconcile/` module replaces imperative `install.rs` container management. Main app containers become Quadlet too | pending |
+
+Test harness (bats + Goss + Chaos Toolkit + vmtest) lands scaffold in v1.7.41, first lifecycle tests blocking v1.7.45, full matrix blocking beta tag.
+
+---
+
+## Release history
+
+### [v1.7.41-alpha](/releases/v1.7.41-alpha/) — IN FLIGHT — 2026-04-22
+**Post-OTA auto-rollback.** After an update lands, the node probes its own web UI through nginx — if the frontend isn't answering cleanly within 90 seconds, the node automatically rolls back to the previous version and restarts. A bad release can no longer leave the fleet stranded on an unreachable node.
+
+Changes:
+- `core/archipelago/src/update.rs`: `PendingVerification` struct, write marker before service restart, `verify_pending_update()` on new binary boot — probes `https://127.0.0.1/`, on fail restores `web-ui.bak` + calls `rollback_update()` + `systemctl restart archipelago`
+- `core/archipelago/src/main.rs`: startup task invokes verifier concurrently with server
+
+### [v1.7.40-alpha](https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.40-alpha/) — 2026-04-22
+**Proper fix for the 500 error.** Fixed the v1.7.38/39 tarball-perms bug at its source — staging dir is now explicitly `chmod 755` before tar; `--mode=u=rwX,go=rX` normalizes archive perms; pre-ship assertion aborts release if `tar tvzf | head -1` isn't `drwxr-xr-x`.
+
+Changes:
+- `scripts/create-release-manifest.sh`: pre-tar chmod + tar --mode flag + post-tar verify
+- Everything from .38 + .39 still in place (onboarding auto-heal, silent logins, app purge, AIUI in tarball)
+
+### [v1.7.39-alpha](https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.39-alpha/) — 2026-04-22
+**Hotfix attempt** for v1.7.38's nginx 500 (didn't fully work — still shipped broken tarball perms). Added startup self-heal chmod in `main.rs` and post-extract chmod in `update.rs` OTA applier.
+
+### [v1.7.38-alpha](https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.38-alpha/) — 2026-04-22
+**Onboarding auto-heal + silent logins + App Store trim.**
+
+Changes:
+- `auth.rs`: `is_onboarding_complete()` auto-heals from `setup_complete` + `password_hash` (prevents clear-cache → onboarding wizard bug)
+- `useOnboarding`: tri-state — backend-unreachable no longer defaults to `/onboarding/intro`
+- Login sounds gated by `isFirstInstallPhase()` — silent after onboarding, typing sounds unaffected
+- Removed FIPS app, Nostr Relay, Nostr VPN, Routstr, Penpot from catalog + Rust + docker + icons
+- Deleted 15 image versions from tx1138, .168, gitea-local registries
+- AIUI baked into release tarball via `demo/aiui/`
+- `prebuild` hook syncs `app-catalog/catalog.json` → `public/catalog.json`
+
+(Shipped with tarball-perms bug; fleet had to be healed before v1.7.40.)
+
+### [v1.7.37-alpha](https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.37-alpha/) — 2026-04-22
+**Bitcoin Core install fixes + dynamic node UI + full-archive default.**
+
+- Bitcoin Core passes explicit `-rpcbind/-rpcallowip/etc.` CLI args so vanilla image exposes RPC
+- Split `bitcoin-core` from `bitcoin-knots` in backend `AppMetadata`
+- bitcoin-ui auto-detects Core vs. Knots from subversion, swaps branding at runtime
+- Storage (Full Archive · X GB / Pruned) indicator on dashboard
+- Node Settings modal shows real values (network, storage, txindex, ZMQ, RPC port)
+- Pull fallback to `docker.io` when no mirror carries the image
+- Removed `prune=550` hardcode — full archive default
+
+---
+
+## Key docs
+
+- [`bulletproof-containers.md`](./bulletproof-containers.md) — full reconcile architecture, code layout, test matrix, chaos scenarios, sources
+- [`BETA-RELEASE-CHECKLIST.md`](./BETA-RELEASE-CHECKLIST.md) — existing beta checklist
+- [`BETA-ISSUES-20260328.md`](./BETA-ISSUES-20260328.md) — prior beta-blocker tracking
+- [`hotfix-process.md`](./hotfix-process.md) — release workflow
+- [`architecture.md`](./architecture.md) — system architecture overview
+
+---
+
+## How to resume
+
+1. Check fleet mirrors are all live: `curl -sS https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/manifest.json | jq .version`
+2. Read [`bulletproof-containers.md`](./bulletproof-containers.md) for the current plan
+3. Check task list (`/list` or via Claude Code) for the in-flight release
+4. Latest in-flight work: v1.7.41 deploying to .228 for test; will ship to all 4 mirrors once verified

docs/STEP-8B-PORT-AUDIT.md

@@ -0,0 +1,179 @@
+# Step 8b Port Audit — container-specs.sh → apps/*/manifest.yml
+
+Last updated: 2026-04-23
+
+This audit is the scope-lock for Step 8b of `docs/rust-orchestrator-migration.md`. Every container currently declared in `scripts/container-specs.sh:ALL_CONTAINER_SPECS` must be port-faithful to `apps/<id>/manifest.yml` before Step 8c can delete the bash scripts.
+
+Findings in short:
+
+- `scripts/container-specs.sh` lists **30 containers** across 5 tiers.
+- `apps/*/manifest.yml` exists for **27 app ids**, but the overlap is partial and most of the overlapping manifests are **aspirational stubs written in the original design phase, never reconciled against production behavior**. The image references, container names, network topology, env, and health checks disagree with what actually runs on `.116` and `.228`.
+- Only the three UI apps (`bitcoin-ui`, `electrs-ui`, `lnd-ui`) plus `aiui` are truly ported (Step 7 scope).
+- The Rust schema (`core/container/src/manifest.rs::AppManifest`) is **missing** several fields needed for a faithful port: `archy-net` network selection, `custom_args`, `entrypoint` override, derived host env (e.g. `HOST_MDNS`), secret-file env injection, and data-dir UID/GID mapping.
+
+---
+
+## Table — every spec, mapped
+
+Legend for **Status**:
+
+- ✅ PORTED — manifest exists and matches reality (Step 7 done).
+- ⚠ STUB — `apps/<id>/manifest.yml` exists but disagrees with `container-specs.sh` (image, name, network, env, or health wrong).
+- ❌ MISSING — no manifest file on disk.
+- — N/A — intentionally out of Step 8b (optional app with no spec, or already managed by a different system).
+
+| Tier | Spec name (container-specs.sh)   | Actual container name | Image source                        | apps/<id>/ matches? | Status | Notes |
+|-----:|----------------------------------|-----------------------|-------------------------------------|---------------------|--------|-------|
+| 0    | archy-mempool-db                 | archy-mempool-db      | `$MARIADB_IMAGE`                    | mempool/            | ⚠      | Existing manifest (if any) targets mempool combined stack, not the DB sidecar. Likely a companion of `apps/mempool`. |
+| 0    | archy-btcpay-db                  | archy-btcpay-db       | `$BTCPAY_POSTGRES_IMAGE`            | btcpay-server/      | ⚠      | Existing manifest describes only the app container. DB is a silent companion in the current model. |
+| 0    | immich_postgres                  | immich_postgres       | `$IMMICH_POSTGRES_IMAGE`            | (none)              | ❌      | Optional. No `apps/immich/` dir. |
+| 0    | immich_redis                     | immich_redis          | `$VALKEY_IMAGE`                     | (none)              | ❌      | Optional. No `apps/immich/` dir. |
+| 1    | bitcoin-knots                    | bitcoin-knots         | `$BITCOIN_KNOTS_IMAGE`              | bitcoin-core/       | ⚠      | `apps/bitcoin-core/manifest.yml` references `bitcoin/bitcoin:28.4`; production runs Bitcoin **Knots** at `$ARCHY_REGISTRY/bitcoin-knots:latest`. App id mismatch: spec is `bitcoin-knots`, manifest is `bitcoin-core`. Decide: rename spec or rename app id. |
+| 1    | electrumx                        | electrumx             | `$ELECTRUMX_IMAGE`                  | (none)              | ❌      | Separate from `electrs-ui`. No `apps/electrumx/` dir. |
+| 2    | lnd                              | lnd                   | `$LND_IMAGE`                        | lnd/                | ⚠      | Manifest exists; needs verification against current env/ports/caps. |
+| 2    | mempool-api                      | mempool-api           | `$MEMPOOL_BACKEND_IMAGE`            | mempool/            | ⚠      | Companion of `apps/mempool`. May need dedicated manifest or stack-form. |
+| 2    | archy-mempool-web                | archy-mempool-web     | `$MEMPOOL_WEB_IMAGE`                | mempool/            | ⚠      | Companion. |
+| 2    | archy-nbxplorer                  | archy-nbxplorer       | `$NBXPLORER_IMAGE`                  | btcpay-server/      | ⚠      | Companion of BTCPay. |
+| 2    | btcpay-server                    | btcpay-server         | `$BTCPAY_IMAGE`                     | btcpay-server/      | ⚠      | Stub; env, ports, deps need reconciliation. |
+| 2    | fedimint                         | fedimint              | `$FEDIMINT_IMAGE`                   | fedimint/           | ⚠      | **This is the bug from yesterday.** Stub references wrong image (`fedimint/fedimintd:v0.10.0` instead of `$ARCHY_REGISTRY/fedimintd:v0.10.0`), wrong RPC target (`bitcoin-core:8332` instead of `bitcoin-knots:8332`), missing `HOST_MDNS` env, missing `archy-net`, missing `FM_BIND_P2P`/`FM_BIND_API`, missing gateway ports etc. |
+| 2    | fedimint-gateway                 | fedimint-gateway      | `$FEDIMINT_GATEWAY_IMAGE`           | (none)              | ❌      | No manifest. Has complex LND-aware entrypoint in `container-specs.sh:load_spec_fedimint-gateway`. |
+| 2    | immich_server                    | immich_server         | `$IMMICH_SERVER_IMAGE`              | (none)              | ❌      | Optional. |
+| 3    | homeassistant                    | homeassistant         | `$HOMEASSISTANT_IMAGE`              | home-assistant/     | ⚠      | id mismatch: `homeassistant` vs `home-assistant`. |
+| 3    | grafana                          | grafana               | `$GRAFANA_IMAGE`                    | grafana/            | ⚠      | Stub. |
+| 3    | uptime-kuma                      | uptime-kuma           | `$UPTIME_KUMA_IMAGE`                | (none)              | ❌      | Optional. |
+| 3    | jellyfin                         | jellyfin              | `$JELLYFIN_IMAGE`                   | (none)              | ❌      | Optional. |
+| 3    | photoprism                       | photoprism            | `$PHOTOPRISM_IMAGE`                 | (none)              | ❌      | Optional. |
+| 3    | vaultwarden                      | vaultwarden           | `$VAULTWARDEN_IMAGE`                | (none)              | ❌      | Optional. Known-bad container on `.228` (see STATUS.md). |
+| 3    | nextcloud                        | nextcloud             | `$NEXTCLOUD_IMAGE`                  | (none)              | ❌      | Optional. |
+| 3    | searxng                          | searxng               | `$SEARXNG_IMAGE`                    | searxng/            | ⚠      | Stub. |
+| 3    | onlyoffice                       | onlyoffice            | `$ONLYOFFICE_IMAGE`                 | onlyoffice/         | ⚠      | Stub. |
+| 3    | filebrowser                      | filebrowser           | `$FILEBROWSER_IMAGE`                | (none)              | ❌      | **Critical** — this is Archipelago baseline (bootstrapped by first-boot), not an optional app. Lost `.filebrowser.json` yesterday. Must have a manifest. |
+| 3    | nginx-proxy-manager              | nginx-proxy-manager   | `$NPM_IMAGE`                        | (none)              | ❌      | Optional. |
+| 3    | portainer                        | portainer             | `$PORTAINER_IMAGE`                  | (none)              | ❌      | Optional. |
+| 3    | ollama                           | ollama                | `$OLLAMA_IMAGE`                     | ollama/             | ⚠      | Stub. |
+| 4    | archy-bitcoin-ui                 | archy-bitcoin-ui      | `localhost/bitcoin-ui:local`        | bitcoin-ui/         | ✅     | Step 7 done. |
+| 4    | archy-lnd-ui                     | archy-lnd-ui          | `localhost/lnd-ui:local`            | lnd-ui/             | ✅     | Step 7 done. |
+| 4    | archy-electrs-ui                 | archy-electrs-ui      | `localhost/electrs-ui:local`        | electrs-ui/         | ✅     | Step 7 done. |
+
+### Non-spec apps that already have manifests (outside `container-specs.sh`)
+
+These are managed entirely by the install RPC today and already have adoption paths in the Rust orchestrator. They are **not** in 8b scope:
+
+- `aiui`, `botfights`, `core-lightning`, `did-wallet`, `endurain`, `gitea`, `indeedhub`, `lightning-stack` (stack), `meshtastic`, `morphos-server`, `nostr-rs-relay`, `router`, `strfry`, `web5-dwn`.
+
+---
+
+## Schema gaps blocking faithful ports
+
+`core/container/src/manifest.rs::AppManifest` currently supports:
+
+- `container.image` OR `container.build` (mutually exclusive, validated).
+- `dependencies: Vec<Dependency>`, `resources: {cpu_limit, memory_limit, disk_limit}`.
+- `security: { capabilities, readonly_root, network_policy: string, apparmor_profile }`.
+- `ports: Vec<{host, container, protocol}>`, `volumes: Vec<{type, source, target, options}>`.
+- `environment: Vec<String>` (each `"KEY=VALUE"`).
+- `health_check: {type, endpoint, path, interval, timeout, retries}`.
+- `devices: Vec<String>`, `extensions: HashMap<String, Value>` (flatten).
+
+What `container-specs.sh` uses that the schema **does not** express first-class:
+
+| Need | Example from bash | Proposed schema addition |
+|---|---|---|
+| Join the named `archy-net` bridge | `SPEC_NETWORK="archy-net"` | `container.network: Option<String>` (Some("archy-net"), or None for `isolated`, or "host"). Existing `security.network_policy` left as-is for policy knobs (e.g. firewall isolation layer); this new field is literally the podman `--network` value. |
+| Extra args / custom flags | `SPEC_CUSTOM_ARGS="-server=1 -prune=550 ..."` | `container.custom_args: Vec<String>`. |
+| Entrypoint override | `SPEC_ENTRYPOINT="gatewayd --data-dir /data ... lnd --lnd-rpc-host lnd:10009"` | `container.entrypoint: Option<Vec<String>>`. |
+| Host-derived env (mDNS hostname, host IP) | `FM_P2P_URL=fedimint://$HOST_MDNS:8173` | `container.derived_env: Vec<{key, template}>` with a small allow-list of `{{HOST_MDNS}}`, `{{HOST_IP}}`, `{{DISK_GB}}` substitutions resolved at apply time. |
+| Secret-file env (read from `/var/lib/archipelago/secrets/<name>`) | `FM_BITCOIND_PASSWORD=$BITCOIN_RPC_PASS` (from secret file in bash) | `container.secret_env: Vec<{key, secret_file}>`, secret_file relative to `$SECRETS_DIR`. Never logged. |
+| Data dir UID/GID (for rootless mapped chown) | `SPEC_DATA_UID="100070:100070"` | `container.data_uid: Option<String>` (e.g. `"100070:100070"`). Applied as `chown -R` before container create. |
+| Exec health check | `SPEC_HEALTH_CMD="bitcoin-cli ..."` | Extend `HealthCheck` so `type: exec` + `command: Vec<String>` works end-to-end; confirm the runtime honors it. |
+| Optional/skip-when-not-installed semantics | `SPEC_OPTIONAL="true"` | Already covered: `BootReconciler` only installs if an `AppManifest` is registered. For baseline-on-first-boot containers (filebrowser), we use the same install path. No schema change. |
+| Local-image flag (don't pull) | `SPEC_LOCAL_IMAGE="true"` | Already covered: `container.build` vs `container.image`. |
+
+Everything else (tier ordering, dependency tree, readonly_root, tmpfs mounts) is either already in the schema or folded into `custom_args` cleanly.
+
+### tmpfs
+
+`SPEC_TMPFS="/tmp:rw,noexec,nosuid,size=256m ..."` used by `grafana`, `searxng`, `ollama`. Currently no first-class field. Proposed: `volumes[].type: tmpfs` with a new `tmpfs_options` field on `Volume`, or a dedicated `container.tmpfs: Vec<{target, options}>`. Either works; the `Volume`-variant keeps all mount declarations in one place.
+
+---
+
+## Proposed commit sequence
+
+Each item is a separate commit. None recreates a container on the fleet.
+
+**8b.0 — schema extensions, no manifest changes, no orchestrator changes**
+
+1. `feat(container/manifest): add network, custom_args, entrypoint, derived_env, secret_env, data_uid, tmpfs fields` — add fields to `ContainerConfig`/`SecurityPolicy`/`Volume`, update `validate()`, add unit tests per new field. Backwards-compat: every existing `apps/*/manifest.yml` must still parse (verify with a `parse_every_real_manifest` test that walks `apps/*/manifest.yml` in the repo).
+
+2. `feat(container/manifest): resolve derived_env against host facts` — add `HostFacts { host_ip, host_mdns, disk_gb }` struct and `resolve_env(facts) -> Vec<String>` method; unit test with a fixed `HostFacts`.
+
+3. `feat(container/manifest): resolve secret_env against a SecretsProvider` — add trait `SecretsProvider { fn read(&self, name: &str) -> Result<String>; }`, stub `FileSecretsProvider` rooted at `/var/lib/archipelago/secrets`, unit test with a tmpdir provider.
+
+**8b.1 — orchestrator honors the new fields**
+
+4. `feat(prod_orchestrator): honor network/custom_args/entrypoint on create` — thread the new `ResolvedContainerConfig` into the runtime's create call. Mock-runtime unit tests for each field.
+5. `feat(prod_orchestrator): chown data dir to data_uid before create` — called from `install_fresh`. Unit test with a tmpdir.
+6. `feat(prod_orchestrator): resolve derived_env + secret_env before create` — wire in `HostFacts` + `SecretsProvider`. Unit test.
+
+**8b.2 — first real backend port: fedimint**
+
+7. `feat(apps/fedimint): port manifest from container-specs.sh with mDNS URLs + archy-net` — rewrites `apps/fedimint/manifest.yml` using the new schema. Includes `container_name: fedimint` (no prefix), `network: archy-net`, `derived_env: [FM_P2P_URL, FM_API_URL]`, `secret_env: [FM_BITCOIND_PASSWORD, ...]`.
+8. `feat(apps/fedimint-gateway): new manifest with LND-aware entrypoint` — creates `apps/fedimint-gateway/manifest.yml`. Dynamic entrypoint is a 2-case template resolved by a derived field `{{LND_AVAILABLE}}` (presence of `/var/lib/archipelago/lnd/tls.cert`). May require a second commit to add that derived fact — scope-judge at write time.
+9. `test(lifecycle): fedimint adoption + fresh-install` — bats scaffold per `docs/bulletproof-containers.md§Test harness`.
+
+**8b.3 — remaining critical backends (one per commit)**
+
+10. `feat(apps/filebrowser): new manifest — baseline Archipelago service` (fixes yesterday's `.filebrowser.json` loss by regenerating via `custom_args: ["--config", "/data/.filebrowser.json"]` + `caps: [..., NET_BIND_SERVICE]`).
+11. `feat(apps/electrumx): new manifest`.
+12. `feat(apps/bitcoin-knots): rename-or-merge with apps/bitcoin-core/manifest.yml` — decide naming once, update everywhere. Recommend: keep `apps/bitcoin-core/` dir (it's the user-visible app name) and use `extensions.container_name: bitcoin-knots` to preserve adoption.
+13. `feat(apps/lnd): reconcile stub against spec`.
+14. `feat(apps/btcpay-server + companions): multi-container stack` — reuse the existing stack path in `api/rpc/package/stacks.rs` OR decide to add `container.companions: Vec<ContainerConfig>`. Defer decision until 10–13 land.
+
+**8b.4 — mempool stack, optional apps**
+
+Continue one-at-a-time until every ⚠ or ❌ row above is ✅.
+
+**8b.5 — port `core/archipelago/src/api/rpc/package/update.rs`**
+
+Replace `reconcile-containers.sh` calls with `ContainerOrchestrator::upgrade(app_id)`. Unblocks 8c.
+
+**8c — delete bash scripts** (per `docs/rust-orchestrator-migration.md`).
+
+---
+
+## Runtime-only drift on `.116` — write it into manifests, not scripts
+
+Per `docs/RESUME.md§Runtime-only fixes on .116`, yesterday's patches are:
+
+1. `~archipelago/.config/containers/containers.conf` (`image_copy_tmp_dir = "storage"`) → lands in `first-boot-setup.sh` (renamed in Step 8c) OR in a Rust startup-side prereq hook. Not a per-manifest concern.
+2. Secrets ownership `archipelago:archipelago` → Rust orchestrator's `ensure_secrets` path (already exists; verify it chowns).
+3. `/var/lib/archipelago/filebrowser-data/.filebrowser.json` → handled by filebrowser's `custom_args: ["--config", "/data/.filebrowser.json"]` plus a pre-start hook (mirrors `bitcoin_ui` precedent) that writes the file if absent. Details in 8b.3 commit 10.
+4. Fedimint data dir chown → handled by `container.data_uid: "100000:100000"` in the fedimint manifest.
+
+All runtime-only fixes end up expressed as manifest fields or Rust-side hooks. None survives as bash.
+
+---
+
+## Open decisions (lock before writing code)
+
+1. **`bitcoin-knots` vs `bitcoin-core` naming.** Recommend: app id stays `bitcoin-core` (user-facing), container name becomes `bitcoin-knots` via `extensions.container_name`, image is Knots. Or rename both to `bitcoin-knots` for honesty. Pick one and apply everywhere.
+2. **`archy-` prefix rule.** Currently `UI_APP_IDS` in `prod_orchestrator.rs` hardcodes `["bitcoin-ui", "electrs-ui", "lnd-ui"]` → `archy-`. Several backends use `archy-` too (`archy-mempool-db`, `archy-mempool-web`, `archy-nbxplorer`, `archy-btcpay-db`). Recommend: drop the hardcoded list, rely on `extensions.container_name` everywhere, audit all existing manifests to set it explicitly so adoption doesn't orphan.
+3. **Companions (mempool-api + mempool-web + mempool-db, btcpay-server + nbxplorer + btcpay-db).** Two options: (a) one manifest per container with explicit deps and an "app group" id; (b) extend `ContainerConfig` with `companions: Vec<…>`. `apps/lightning-stack/manifest.yml` already shipped probably has a precedent — check its shape before deciding.
+4. **Keep `container-specs.sh` as the source of truth until 8b is fully ported?** Yes. `BootReconciler` only acts on what's in `apps/*/manifest.yml`; anything not ported stays on the bash path until its commit lands. Zero-downtime migration.
+
+---
+
+## Where to resume
+
+After user approves this plan: commit 1 in 8b.0 (schema extensions + tests, no orchestrator or manifest changes). Smallest possible diff, highest leverage, and unblocks every subsequent port.
+
+## Validation Snapshot - 2026-04-28
+
+- Runtime cleanup: removed orphan `bold_lichterman` duplicate; retained managed `filebrowser`.
+- Launch policy alignment: local app launches are port-based; iframe-blocked apps (including `gitea`) are forced to new-tab.
+- App icon reliability: image fallback now retries `.svg` when `.png` does not exist.
+- Required stack verification on `.116`:
+  - `tests/lifecycle/bats/required-stack.bats` -> PASS
+  - `ARCHY_ALLOW_DESTRUCTIVE=1 tests/lifecycle/bats/required-stack-destructive.bats` -> PASS
+- Broad host-port probe confirms HTTP 200 responses for user-facing app UIs on mapped ports; non-HTTP ports intentionally excluded from HTTP pass/fail semantics.
+

docs/hotfix-process.md

@@ -28,9 +28,8 @@ For critical bugs discovered after v1.0.0 release.
 ### 3. Release
 - Merge hotfix branch to `main`
 - Tag: `v1.0.1` (increment patch version)
-- Build ISO if needed: `sudo ./image-recipe/build-auto-installer-iso.sh`
-- Update release manifest for OTA updates
-- Copy ISO to FileBrowser Builds folder
+- Update release manifest for OTA updates (`releases/manifest.json`)
+- Push to both Gitea mirrors so nodes can pull via `self-update.sh`
 
 ### 4. Communicate
 - Update RELEASE-NOTES with hotfix details
@@ -45,6 +44,6 @@ For critical bugs discovered after v1.0.0 release.
 ## Rollback
 
 If a hotfix causes regressions:
-1. OTA system supports rollback to previous version
-2. Users can reflash with previous ISO (app data preserved on separate partition)
+1. OTA system supports rollback to previous version (`scripts/self-update.sh --rollback`)
+2. Point `releases/manifest.json` back at the last-known-good version and push to mirrors
 3. Backend binary backup at `/usr/local/bin/archipelago.bak`

docs/rust-orchestrator-migration.md

@@ -0,0 +1,525 @@
+# Rust Orchestrator Migration — Design Doc
+
+Status: **DRAFT — pending user approval**
+Author: OpenCode session, 2026-04-22
+Supersedes planning in `docs/bulletproof-containers.md` v1.7.43 slot
+
+## Problem statement
+
+Today, the archipelago backend has **no production container orchestrator**. Production containers (bitcoin-knots, lnd, electrumx, btcpay, filebrowser, and the three custom UIs archy-bitcoin-ui / archy-electrs-ui / archy-lnd-ui) are installed by **bash scripts** at first boot (`scripts/first-boot-containers.sh`) and optionally reconciled by another bash script (`scripts/reconcile-containers.sh`) that is **not enabled by default**. The existing `DevContainerOrchestrator` (`core/archipelago/src/container/dev_orchestrator.rs`) is hardcoded to append `-dev` suffixes and gated behind `config.dev_mode`, so it has never managed a production container.
+
+This design migrates production container management into Rust, under a single orchestrator that owns install, start, stop, restart, upgrade, uninstall, health, and self-healing for every container. The three custom UI containers are the first-class test fixture: they exercise the "build image from local Dockerfile" path (which today doesn't exist in the manifest schema) and their lifecycle was the original failure class the user asked to fix.
+
+## Non-goals
+
+- Backwards compatibility with `first-boot-containers.sh`: we **delete** it and its systemd unit after verifying Rust parity.
+- Backwards compatibility with the existing `package-install` RPC’s podman shell-outs: those get rewritten to call the orchestrator.
+- Registry signature verification: `image_signature` stays optional. Sigstore/cosign integration is out of scope.
+- Network isolation improvements: existing SecurityPolicy fields stay as-is.
+- Dev mode removal: `DevContainerOrchestrator` keeps existing behavior for local development; prod code path is separate.
+
+## Scope of this migration
+
+In scope:
+1. Extend `ContainerConfig` schema with a `source:` variant supporting `{type: build, context, dockerfile, tag}` alongside `{type: pull, image, pull_policy}`.
+2. Extend `ContainerRuntime` trait + `PodmanRuntime` impl with `build_image(...)` and `image_exists(...)`.
+3. Introduce `ProdContainerOrchestrator` (new type) with identical public surface to `DevContainerOrchestrator` but **no `-dev` suffix**, **no port offset**, **no data-path rewriting**, **no bitcoin_simulator gate**. It is wired into `RpcHandler::orchestrator` in prod (currently `None`).
+4. Add `AdoptionScan` at orchestrator startup: enumerate `podman ps -a`, match by container name against declared manifests, adopt into orchestrator state without recreating.
+5. Add `BootReconciler` task spawned from `main.rs` (replacing the commented-out `run_boot_reconciliation` hook). Walks the manifest set on startup and periodically, ensures each is present-and-running, builds/pulls/creates anything missing, logs failures non-silently.
+6. Ship three manifests in the repo: `apps/bitcoin-ui/manifest.yml`, `apps/electrs-ui/manifest.yml`, `apps/lnd-ui/manifest.yml`. They use the new `source: build` variant pointing at `/opt/archipelago/docker/<name>/`.
+7. Delete `scripts/first-boot-containers.sh`, `scripts/reconcile-containers.sh`, `scripts/container-specs.sh`, `image-recipe/configs/archipelago-first-boot-containers.service`, `image-recipe/configs/archipelago-reconcile.service`. Remove enablement from ISO builder.
+
+Out of scope this migration (tracked separately):
+- Migrating btcpay / mempool / fedimint multi-container stacks to manifests (they currently live in `core/archipelago/src/api/rpc/package/stacks.rs`). They keep working via `package-install` RPC. Phase 2.
+- Rewriting the 26 existing `apps/*/manifest.yml` files to use the new `source:` schema. They stay on `image:` for now; the schema is **additive and backwards-compatible**.
+- Re-enabling signature verification; stays todo.
+
+## Data model changes
+
+### 1. `ContainerConfig` gets a `source` enum
+
+File: `core/container/src/manifest.rs:58`
+
+**Before:**
+```rust
+pub struct ContainerConfig {
+    pub image: String,
+    pub image_signature: Option<String>,
+    pub pull_policy: String,
+}
+```
+
+**After:**
+```rust
+pub struct ContainerConfig {
+    // Legacy shorthand (backwards compatible with all 26 existing manifests):
+    // if `source` is absent, `image` + `pull_policy` are interpreted as
+    // `source: { type: pull, image, pull_policy }`.
+    #[serde(default)]
+    pub image: String,
+    #[serde(default)]
+    pub image_signature: Option<String>,
+    #[serde(default = "default_pull_policy")]
+    pub pull_policy: String,
+
+    // New: explicit source. If present, overrides the legacy shorthand.
+    #[serde(default)]
+    pub source: Option<ContainerSource>,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(tag = "type", rename_all = "lowercase")]
+pub enum ContainerSource {
+    /// Pull an image from a registry.
+    Pull {
+        image: String,
+        #[serde(default)]
+        image_signature: Option<String>,
+        #[serde(default = "default_pull_policy")]
+        pull_policy: String,
+    },
+    /// Build an image from a local Dockerfile.
+    Build {
+        /// Filesystem path to build context, absolute or relative to manifest dir.
+        context: String,
+        /// Dockerfile path relative to context. Defaults to "Dockerfile".
+        #[serde(default = "default_dockerfile")]
+        dockerfile: String,
+        /// Tag to assign to the built image, e.g. "localhost/bitcoin-ui:local".
+        tag: String,
+        /// `--build-arg` key=value pairs.
+        #[serde(default)]
+        build_args: HashMap<String, String>,
+        /// If true, rebuild on every reconcile. If false, only build when tag is missing.
+        #[serde(default)]
+        always_rebuild: bool,
+    },
+}
+```
+
+Validation in `AppManifest::validate`:
+- If `source` is absent AND `image` is empty → error (unchanged rule just rephrased).
+- If `source` is present, legacy `image` field is ignored with a warning.
+- `Build::context` must resolve to an existing directory that contains `dockerfile`.
+
+Tests to add:
+- Parse a legacy manifest → works, produces `ContainerSource::Pull` at resolution time.
+- Parse a `source: { type: build, ... }` manifest → works.
+- Parse a manifest with both legacy `image:` and `source:` → warning logged, `source:` wins.
+- Parse a manifest with neither → rejected.
+
+### 2. `ContainerRuntime` trait gets `build_image` + `image_exists`
+
+File: `core/container/src/runtime.rs:10`
+
+```rust
+#[async_trait]
+pub trait ContainerRuntime: Send + Sync {
+    // existing methods unchanged...
+    async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()>;
+    async fn create_container(...) -> Result<()>;
+    // ...
+
+    // NEW:
+    /// Build an image from a local Dockerfile. Returns Ok(()) if the image now
+    /// exists under the given tag (whether newly built or already present and
+    /// `force=false`). Returns Err if the build failed.
+    async fn build_image(
+        &self,
+        context: &Path,
+        dockerfile: &str,
+        tag: &str,
+        build_args: &HashMap<String, String>,
+        force: bool,
+    ) -> Result<()>;
+
+    /// Check if an image exists in the local image store.
+    async fn image_exists(&self, tag: &str) -> Result<bool>;
+}
+```
+
+`PodmanRuntime::build_image` shells out:
+```
+podman build --tag <tag> \
+    --file <context>/<dockerfile> \
+    --build-arg KEY=VALUE ... \
+    <context>
+```
+
+Force-rebuild semantics: if `force=false`, skip when `image_exists(tag) == true`. If `force=true`, always build (podman's own layer cache handles the fast path).
+
+Tests:
+- `build_image` happy path on a minimal Dockerfile (using a throwaway context in tmpdir).
+- `build_image` failure path (nonsense Dockerfile) → Err.
+- `image_exists` returns false for nonexistent tag.
+- `image_exists` returns true after `build_image`.
+
+### 3. Manifest resolution: `ContainerSource::resolve(manifest_dir) -> ResolvedSource`
+
+New method that turns the raw manifest into something the orchestrator can act on:
+
+```rust
+pub enum ResolvedSource {
+    Pull { image: String, signature: Option<String>, pull_policy: PullPolicy },
+    Build { context: PathBuf, dockerfile: String, tag: String, build_args: HashMap<String,String>, always_rebuild: bool },
+}
+
+impl ContainerConfig {
+    pub fn resolve(&self, manifest_dir: &Path) -> Result<ResolvedSource> {
+        match &self.source {
+            Some(ContainerSource::Pull { image, image_signature, pull_policy }) => Ok(ResolvedSource::Pull { ... }),
+            Some(ContainerSource::Build { context, dockerfile, tag, build_args, always_rebuild }) => {
+                let abs_context = if Path::new(context).is_absolute() {
+                    PathBuf::from(context)
+                } else {
+                    manifest_dir.join(context)
+                };
+                Ok(ResolvedSource::Build { context: abs_context, ... })
+            }
+            None => {
+                // Legacy shorthand
+                if self.image.is_empty() {
+                    return Err(...);
+                }
+                Ok(ResolvedSource::Pull { image: self.image.clone(), ... })
+            }
+        }
+    }
+}
+```
+
+## Runtime architecture
+
+### `ProdContainerOrchestrator`
+
+New file: `core/archipelago/src/container/prod_orchestrator.rs`
+
+```rust
+pub struct ProdContainerOrchestrator {
+    runtime: Arc<dyn ContainerRuntimeTrait>,
+    manifests_dir: PathBuf,   // e.g. /opt/archipelago/apps
+    data_dir: PathBuf,        // e.g. /var/lib/archipelago
+    state: Arc<RwLock<OrchestratorState>>,
+    config: Config,
+}
+
+struct OrchestratorState {
+    /// app_id → known manifest (loaded from disk at startup, refreshed on reconcile)
+    manifests: HashMap<String, AppManifest>,
+    /// app_id → current known state (from adoption scan or our own ops)
+    containers: HashMap<String, ContainerState>,
+    /// app_id → last install/health/build timestamp
+    last_reconciled: HashMap<String, Instant>,
+}
+```
+
+Public surface mirrors `DevContainerOrchestrator` but **container name = `archy-<app_id>` for UI apps, `<app_id>` for backends, matching existing .116 naming**:
+
+```rust
+impl ProdContainerOrchestrator {
+    pub async fn new(config: Config) -> Result<Self> { ... }
+    pub async fn load_manifests(&self) -> Result<()> { /* walks manifests_dir */ }
+    pub async fn adopt_existing(&self) -> Result<AdoptionReport> { /* scans podman ps -a */ }
+    pub async fn reconcile_all(&self) -> Result<ReconcileReport> { /* ensures every manifest has a running container */ }
+    pub async fn install(&self, app_id: &str) -> Result<()> { /* build-or-pull + create + start */ }
+    pub async fn start(&self, app_id: &str) -> Result<()> { ... }
+    pub async fn stop(&self, app_id: &str) -> Result<()> { ... }
+    pub async fn restart(&self, app_id: &str) -> Result<()> { ... }
+    pub async fn remove(&self, app_id: &str, preserve_data: bool) -> Result<()> { ... }
+    pub async fn upgrade(&self, app_id: &str) -> Result<()> { /* re-read manifest, rebuild/pull, recreate */ }
+    pub async fn status(&self, app_id: &str) -> Result<ContainerStatus> { ... }
+    pub async fn list(&self) -> Result<Vec<ContainerStatus>> { ... }
+    pub async fn logs(&self, app_id: &str, lines: u32) -> Result<Vec<String>> { ... }
+    pub async fn health(&self, app_id: &str) -> Result<String> { ... }
+}
+```
+
+**Container naming rule** (matches `.116` existing fixture so adoption works):
+- If the manifest has `extensions["container_name"]` → use that verbatim.
+- Else if the app_id starts with `bitcoin-ui` / `electrs-ui` / `lnd-ui` → `archy-<app_id>`.
+- Else → `<app_id>`.
+
+This is codified and tested; no ad-hoc naming in the codebase.
+
+### `AdoptionScan`
+
+On orchestrator startup, before any reconcile:
+
+```rust
+async fn adopt_existing(&self) -> Result<AdoptionReport> {
+    let all = self.runtime.list_containers().await?;  // podman ps -a
+    let mut report = AdoptionReport::default();
+    for c in all {
+        // For each manifest we have loaded, check if the expected container name matches
+        for (app_id, manifest) in self.state.read().await.manifests.iter() {
+            let expected_name = compute_container_name(manifest);
+            if c.name == expected_name {
+                // This container is ours. Record its state.
+                self.state.write().await.containers.insert(app_id.clone(), c.state.clone());
+                report.adopted.push(app_id.clone());
+            }
+        }
+    }
+    Ok(report)
+}
+```
+
+No recreate. No touching data volumes. Just "we now know this container belongs to app X and its current state is Y".
+
+### `BootReconciler`
+
+New file: `core/archipelago/src/container/boot_reconciler.rs`
+
+```rust
+pub struct BootReconciler {
+    orchestrator: Arc<ProdContainerOrchestrator>,
+    interval: Duration,  // e.g. 5 minutes
+    shutdown: CancellationToken,
+}
+
+impl BootReconciler {
+    pub async fn run_forever(self) {
+        // Initial reconcile immediately (after adoption).
+        let _ = self.orchestrator.reconcile_all().await;
+        loop {
+            tokio::select! {
+                _ = tokio::time::sleep(self.interval) => {
+                    let _ = self.orchestrator.reconcile_all().await;
+                }
+                _ = self.shutdown.cancelled() => break,
+            }
+        }
+    }
+}
+```
+
+`reconcile_all`:
+```rust
+async fn reconcile_all(&self) -> Result<ReconcileReport> {
+    let manifests: Vec<_> = self.state.read().await.manifests.values().cloned().collect();
+    let mut report = ReconcileReport::default();
+    for manifest in manifests {
+        let app_id = &manifest.app.id;
+        match self.ensure_running(&manifest).await {
+            Ok(action) => report.record(app_id, action),
+            Err(e) => {
+                tracing::error!(app_id, error = %e, "Reconcile failed for app");
+                report.failures.push((app_id.clone(), e.to_string()));
+            }
+        }
+    }
+    if !report.failures.is_empty() {
+        // Surface via WebSocket so the UI can show a banner.
+        self.notify_failures(&report).await;
+    }
+    Ok(report)
+}
+
+async fn ensure_running(&self, manifest: &AppManifest) -> Result<ReconcileAction> {
+    let name = compute_container_name(manifest);
+    match self.runtime.get_container_status(&name).await {
+        Ok(status) if matches!(status.state, ContainerState::Running) => Ok(ReconcileAction::NoOp),
+        Ok(status) if matches!(status.state, ContainerState::Exited | ContainerState::Stopped) => {
+            self.runtime.start_container(&name).await?;
+            Ok(ReconcileAction::Started)
+        }
+        Ok(_) => Ok(ReconcileAction::NoOp),  // Created / Paused — leave alone
+        Err(_) => {
+            // Container doesn't exist. Install it.
+            self.install_fresh(manifest).await?;
+            Ok(ReconcileAction::Installed)
+        }
+    }
+}
+
+async fn install_fresh(&self, manifest: &AppManifest) -> Result<()> {
+    let manifest_dir = ...;  // directory of manifest.yml
+    let resolved = manifest.app.container.resolve(manifest_dir)?;
+    match resolved {
+        ResolvedSource::Pull { image, signature, .. } => {
+            self.runtime.pull_image(&image, signature.as_deref()).await?;
+        }
+        ResolvedSource::Build { context, dockerfile, tag, build_args, always_rebuild } => {
+            if always_rebuild || !self.runtime.image_exists(&tag).await? {
+                self.runtime.build_image(&context, &dockerfile, &tag, &build_args, always_rebuild).await?;
+            }
+        }
+    }
+    self.runtime.create_container(manifest, &compute_container_name(manifest), 0).await?;
+    self.runtime.start_container(&compute_container_name(manifest)).await?;
+    Ok(())
+}
+```
+
+### Wire-up in `main.rs`
+
+File: `core/archipelago/src/main.rs`
+
+Replace the commented-out `run_boot_reconciliation` block (`main.rs:107-111`) with:
+
+```rust
+// Load manifests + adopt existing + start reconciler loop.
+let orchestrator = Arc::new(ProdContainerOrchestrator::new(config.clone()).await?);
+orchestrator.load_manifests().await?;
+let adoption = orchestrator.adopt_existing().await?;
+tracing::info!(adopted = adoption.adopted.len(), "Container adoption complete");
+let reconciler = BootReconciler::new(orchestrator.clone(), Duration::from_secs(300), shutdown_token.clone());
+tokio::spawn(reconciler.run_forever());
+```
+
+`RpcHandler` gets the orchestrator regardless of `dev_mode`:
+```rust
+// core/archipelago/src/api/rpc/mod.rs:83
+let orchestrator: Option<Arc<dyn ContainerOrchestrator>> = if config.dev_mode {
+    Some(Arc::new(DevContainerOrchestrator::new(config.clone()).await?))
+} else {
+    Some(Arc::new(prod_orch.clone()))
+};
+```
+
+Where `ContainerOrchestrator` becomes a trait implemented by both `DevContainerOrchestrator` and `ProdContainerOrchestrator`.
+
+### First-boot replacement
+
+There is no separate first-boot code. The reconciler handles it: when the archipelago service starts on a fresh node, `adopt_existing` finds nothing, `reconcile_all` sees no running container for any manifest, and installs each one in dependency order (bitcoin-core first, then everything else). On subsequent boots, adoption finds existing containers and reconcile mostly no-ops.
+
+**Removes completely**:
+- `/var/lib/archipelago/.first-boot-containers-done` marker (no longer needed)
+- `/var/lib/archipelago/.unbundled` handling in first-boot script (becomes a config flag in archipelago.conf if we still need it)
+- `scripts/first-boot-containers.sh` (1392 lines)
+- `scripts/reconcile-containers.sh`
+- `scripts/container-specs.sh`
+- `image-recipe/configs/archipelago-first-boot-containers.service`
+- `image-recipe/configs/archipelago-reconcile.service`
+- Related enable/disable in ISO builder
+
+## The three UI manifests
+
+Example: `apps/bitcoin-ui/manifest.yml`
+
+```yaml
+app:
+  id: bitcoin-ui
+  name: Bitcoin Knots UI
+  version: 1.0.0
+  description: Custom Archipelago UI for Bitcoin Knots
+  container:
+    source:
+      type: build
+      context: /opt/archipelago/docker/bitcoin-ui
+      dockerfile: Dockerfile
+      tag: localhost/bitcoin-ui:local
+      build_args:
+        BITCOIN_RPC_AUTH: ${BITCOIN_RPC_AUTH}  # injected from host-ip.env or secrets
+      always_rebuild: false
+  dependencies:
+    - app_id: bitcoin-core
+  resources:
+    memory_limit: 128Mi
+  security:
+    network_policy: host
+    readonly_root: false
+  ports: []  # host networking
+  volumes: []
+  environment: []
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:8334
+    path: /
+    interval: 30s
+  extensions:
+    container_name: archy-bitcoin-ui
+```
+
+The `extensions.container_name` is how we match the existing running container on .116 for adoption. Same pattern for `electrs-ui` (container_name: `archy-electrs-ui`, port probe 50002) and `lnd-ui` (container_name: `archy-lnd-ui`, port probe 8081).
+
+**BITCOIN_RPC_AUTH injection**: today `first-boot-containers.sh` `sed`s this value into `nginx.conf` (destructively). In the new world, it's a `--build-arg` — the Dockerfile gets `ARG BITCOIN_RPC_AUTH` and templates `nginx.conf` from a template file. Fixes the "sed destroys the source" bug from the mapping.
+
+## Migration path (.116 and .228 specifically)
+
+### .116 (all 3 UIs currently running, adopted from bash install)
+1. Ship the new archipelago binary with the prod orchestrator.
+2. On archipelago restart, `adopt_existing` scans `podman ps -a`, sees `archy-bitcoin-ui`, `archy-electrs-ui`, `archy-lnd-ui` already running.
+3. Matches them against the new manifests by `extensions.container_name`.
+4. Records state. Reconciler sees them Running → NoOp.
+5. Manual test: `podman stop archy-bitcoin-ui` → within 5 minutes, reconciler starts it again. `podman rm -f archy-bitcoin-ui` → reconciler rebuilds from `/opt/archipelago/docker/bitcoin-ui/Dockerfile` and re-creates.
+
+### .228 (no bitcoin-ui, no lnd-ui, has electrs-ui from bash first-boot)
+1. Ship same binary.
+2. Adoption finds only `archy-electrs-ui`.
+3. Reconciler sees `bitcoin-ui` and `lnd-ui` missing → triggers `install_fresh` for each.
+4. For `bitcoin-ui`: `image_exists("localhost/bitcoin-ui:local")` → false. `build_image(/opt/archipelago/docker/bitcoin-ui, Dockerfile, localhost/bitcoin-ui:local, {BITCOIN_RPC_AUTH: ...}, force=false)`. Then create + start.
+5. Same for `lnd-ui`.
+6. Manual test: HTTP probe ports 8334 and 8081 return 200 within ~5 minutes of service restart.
+
+## Test plan
+
+Unit tests (Rust, in-process):
+- `manifest::tests::legacy_image_parses_as_pull_source`
+- `manifest::tests::explicit_pull_source_parses`
+- `manifest::tests::explicit_build_source_parses`
+- `manifest::tests::source_build_requires_tag`
+- `runtime::tests::build_image_happy_path` (uses a minimal Dockerfile in `tempfile::TempDir`)
+- `runtime::tests::build_image_failure`
+- `runtime::tests::image_exists_roundtrip`
+- `prod_orchestrator::tests::install_fresh_pull`
+- `prod_orchestrator::tests::install_fresh_build`
+- `prod_orchestrator::tests::adopt_existing_matches_by_name`
+- `prod_orchestrator::tests::reconcile_starts_exited_container` (with a mock runtime)
+- `prod_orchestrator::tests::reconcile_installs_missing_container`
+- `prod_orchestrator::tests::compute_container_name_ui_apps_prefixed`
+- `prod_orchestrator::tests::compute_container_name_backend_apps_bare`
+
+Integration tests (require real podman, run on archy node):
+- Fresh-install path: wipe containers + images, start archipelago, verify all 3 UIs up within 60s.
+- Adoption path: containers pre-running, start archipelago, verify no recreate (compare container IDs before/after).
+- Reconcile-start path: `podman stop archy-bitcoin-ui`, wait, verify restart.
+- Reconcile-recreate path: `podman rm -f archy-bitcoin-ui`, wait, verify rebuild+recreate.
+- Rebuild-on-Dockerfile-change path: edit Dockerfile, call `upgrade` RPC, verify image rebuilt and container recreated.
+
+Chaos matrix (bash + Playwright, the original goal):
+- For each UI (bitcoin-ui, electrs-ui, lnd-ui) × each event (stop, start, restart, remove+reconcile, SIGKILL, archipelago-service-restart, host-reboot) × each node (.116, .228): assert HTTP 200 + page-title marker returns within 60s of event.
+
+## Risks + mitigations
+
+| Risk | Mitigation |
+|------|------------|
+| Adoption mismatches and re-creates a container we already had, losing its data | Adoption matches by exact name; `install_fresh` only runs when `get_container_status` returns Err (container doesn't exist), not when it returns Stopped/Exited. Unit tested. |
+| Build loop: reconciler rebuilds on every tick | `always_rebuild: false` + `image_exists` check. Only rebuilds when image tag is missing OR `upgrade` RPC is called. |
+| Reconciler runs while user is mid-install via the UI | Orchestrator state has per-app mutex; reconcile waits. Install path takes the same mutex. |
+| Auto-rollback (v1.7.41) fires during testing | `reconcile_all` is spawned AFTER server is healthy and responding; if it fails, archipelago the service still passes verification. Individual container failures are logged, not fatal. |
+| Dependency ordering: bitcoin-ui needs BITCOIN_RPC_AUTH which is generated at first boot | Reconciler handles dependency order by reading `manifest.app.dependencies` and installing in topological order. If the dep doesn't exist yet, skip and retry next tick. |
+| Moving `/opt/archipelago/docker/<name>` content breaks the build context | That path is stable per the ISO builder at `image-recipe/build-auto-installer-iso.sh:1671-1685`. Manifests reference it absolutely. |
+| Dropping bash scripts breaks existing ISOs in the field | Target release cycle is disposable alpha nodes. For existing alpha nodes (.116, .228) we hot-swap the binary and let the reconciler take over, then the next reboot doesn't need the systemd units; we mask them manually. |
+| User wants to downgrade to v1.7.42 | Auto-rollback mechanism already handles that; binary swap is reversible. The removed bash scripts are still in git history. |
+
+## Implementation order
+
+1. **Schema first**: extend `ContainerConfig` + `ContainerSource` + `resolve()` + validation + unit tests. ~100 LOC Rust + ~80 LOC tests.
+2. **Runtime**: `build_image` + `image_exists` in trait, `PodmanRuntime`, `DockerRuntime` (can stub), `AutoRuntime`. ~150 LOC + tests with throwaway tempdir Dockerfile.
+3. **ProdContainerOrchestrator**: new type with `install/start/stop/restart/remove/status/list/logs/health/adopt_existing/reconcile_all/ensure_running/install_fresh`. ~400 LOC + unit tests with mocked runtime.
+4. **ContainerOrchestrator trait**: abstract over Dev and Prod so `RpcHandler` is polymorphic. ~50 LOC refactor.
+5. **BootReconciler**: task spawner with loop + cancellation. ~80 LOC + unit tests.
+6. **main.rs wire-up**: adopt + spawn reconciler. ~20 LOC.
+7. **3 UI manifests + Dockerfile BITCOIN_RPC_AUTH refactor** (use ARG + template file, not sed). ~60 lines of YAML + ~20 lines of Dockerfile.
+8. **Remove bash scripts + services**: split into sub-steps because `first-boot-containers.sh` creates 25+ containers (only 3 ported in Step 7) AND does non-container setup (secret gen, UID-mapping chowns, Tor hostnames, WireGuard, firewall, nostr-relay dir):
+   - **8a** (cheap, safe): delete `image-recipe/configs/archipelago-reconcile.{service,timer}` + their ISO-builder touchpoints (the systemd enablement + `cp` into `$WORK_DIR`). `BootReconciler` fully replaces the timer-driven path — no more periodic bash invocation. **Keep** `scripts/reconcile-containers.sh` + `scripts/container-specs.sh` because `core/archipelago/src/api/rpc/package/update.rs` still shells out to reconcile-containers.sh during OTA updates; porting that call site requires manifests for every container it touches (which is Step 8b's scope). Atomic commit, low risk.
+   - **8b** (large, deferred): port the remaining ~25 container creations from `first-boot-containers.sh` into `apps/<id>/manifest.yml` files. One manifest per commit, validated against current bash behavior (ports, volumes, env, deps, health checks, post-create wallet/db bootstrap). Probably 1-2 days of careful porting. Includes `apps/filebrowser/manifest.yml`. Then port `update.rs`'s two `reconcile-containers.sh` call sites to the `ContainerOrchestrator` trait (`upgrade(app_id)`).
+   - **8c** (final, one-way door): rename `first-boot-containers.sh` → `first-boot-setup.sh`, strip out all `$DOCKER run/pull/exec` calls, keep only secret generation + dir prep + Tor/WG/firewall/nostr setup. Rename `archipelago-first-boot-containers.service` → `archipelago-first-boot-setup.service`. Delete `scripts/reconcile-containers.sh` + `scripts/container-specs.sh` (update.rs no longer needs them). Add ISO builder lines to copy `apps/*/manifest.yml` → `/opt/archipelago/apps/`. Full ISO build test on .116 required before commit.
+9. **Live test on .228**: hot-swap binary, expect 3 UIs to come up within 60s of service restart.
+10. **Live test on .116**: hot-swap binary, expect zero container recreation + adoption-confirmed log lines.
+11. **Chaos matrix** on both nodes.
+
+Each step is a separate commit. Steps 1–6 are independent-enough that they can each have their own test gate.
+
+## Estimated total
+
+~1000 LOC Rust added, ~1500 lines bash deleted, ~50 LOC Rust deleted. 8–12 hours of focused work across multiple sessions. No release pressure per user decision.
+
+## Open questions for user
+
+1. **Container naming**: I propose `archy-<app_id>` for UIs, `<app_id>` for backends (matches current .116 fixture). Alternative: unify on `archy-<app_id>` for everything and migrate existing backends by renaming at adoption. Which?
+2. **BITCOIN_RPC_AUTH injection**: the build-arg approach rebuilds the UI image when the auth value changes. Fine during normal operation (rare). Alternative: mount the nginx.conf at runtime as a volume, never bake auth into the image. Which?
+3. **Reconciler interval**: 5 minutes. Too slow for a dropped container (user sees a broken UI for up to 5 min). Alternative: 30 seconds + more expensive `podman ps` calls. Which?
+4. **Concurrent reconcile + user install**: per-app mutex is the simple answer. Alternative: a single orchestrator-wide mutex (simpler, slower). Which?
+5. **Delete bash scripts in this migration, or keep them around as fallback?** I recommend delete (single source of truth), but deleting `first-boot-containers.sh` is a one-way door in terms of field recovery.

image-recipe/_archived/README.md

@@ -0,0 +1,37 @@
+# Archived ISO build recipes
+
+These scripts built the Archipelago auto-installer ISO (bundled and
+unbundled variants). As of v1.7.43-alpha, ISOs are no longer part of the
+release deliverable. Releases ship as tarballs consumed by
+`scripts/self-update.sh` on existing nodes.
+
+Archived here rather than deleted so they can be resurrected if ISO
+distribution is reintroduced.
+
+## Contents
+
+- `build-auto-installer-iso.sh` — orchestrator, bundles container images into squashfs
+- `build-unbundled-iso.sh` — thin wrapper that sets BUNDLE_IMAGES=0 and delegates
+- `test-iso-qemu.sh` — smoke-tests a built ISO under QEMU
+- `scripts/convert-iso-to-disk.sh` — converts an ISO to a raw disk image
+- `BUILD-ISO-STATUS.md`, `ISO-BUILD-CHECKLIST.md` — contributor guides
+- `branding/isohdpfx.bin` — isolinux MBR hybrid image
+- `.gitea-workflows/build-iso-dev.yml` — CI workflow that ran the build+smoke-test
+
+## To resurrect
+
+1. `git mv image-recipe/_archived/* image-recipe/` (adjust paths back)
+2. Restore `.gitea/workflows/build-iso-dev.yml`
+3. Re-add release-process references (see `scripts/create-release.sh`,
+   `docs/BETA-RELEASE-CHECKLIST.md`, `docs/hotfix-process.md`, `README.md`).
+
+## Why archived
+
+The release flow is simpler and faster as tarball-only:
+- `releases/vX.Y.Z-alpha/archipelago` (backend binary)
+- `releases/vX.Y.Z-alpha/archipelago-frontend-X.Y.Z-alpha.tar.gz` (frontend + AIUI + filebrowser UI assets)
+- `releases/manifest.json` (pointers + changelog)
+
+Nodes pull these via `scripts/self-update.sh` from either Gitea mirror.
+Filebrowser and AIUI remain bundled inside the frontend tarball and deployed
+atomically by `self-update.sh`.