Compare commits

...
37 Commits
Author SHA1 Message Date
archipelago 259c353147 Clear completed candidate deployment instructions 2026-09-30 13:46:50 -04:00
archipelago 1724ea05d1 Show readiness reason first and record live dashboard acceptance
Demo images / Build & push demo images (push) Failing after 45s
2026-09-30 13:46:16 -04:00
archipelago c1e20a71ae Check companion dashboards and omit headless UI waiting messages
Demo images / Build & push demo images (push) Failing after 37s
2026-09-30 13:28:37 -04:00
archipelago bf56956790 Record UI acceptance and remaining normal-startup release gate 2026-09-30 12:57:35 -04:00
archipelago 2f1a3ade07 Promote runtime manifests before starting app reconciliation 2026-09-30 12:50:46 -04:00
archipelago ef8254272c Name waiting apps, align card actions, and update Angor icon
Demo images / Build & push demo images (push) Failing after 39s
2026-09-30 12:43:53 -04:00
archipelago d50be13232 Normalize Mempool frontend aliases in restored app inventory 2026-09-30 12:41:15 -04:00
archipelago 439b55a236 Use manifest names for new services and document release acceptance
Demo images / Build & push demo images (push) Failing after 38s
2026-09-30 12:20:29 -04:00
archipelago 5ab65f7581 Preserve apostrophes in Quadlet commands and record funded acceptance 2026-09-30 12:08:35 -04:00
archipelago 169bf77de6 Add headless Angor services and shared-index install guard
Demo images / Build & push demo images (push) Failing after 43s
2026-09-30 11:52:19 -04:00
archipelago 7c4169867c docs: consolidate release scope and record migration recovery checks 2026-09-30 10:52:41 -04:00
archipelago acf544500f fix(apps): preserve state across runtime repairs and restore Gitea SSH 2026-09-30 10:46:38 -04:00
archipelago 7d767c8cb0 fix(catalog): gate network migration manifests on backup support 2026-09-30 10:08:56 -04:00
archipelago eb3ccfa00b Merge branch 'fix/gitea-portainer-20260930' 2026-09-30 09:57:49 -04:00
archipelago eda28c4cd6 fix(portainer): repair same-node Git routing with recoverable network migration 2026-09-30 09:57:25 -04:00
archipelago d69e845216 Merge remote-tracking branch 'origin/main'
Demo images / Build & push demo images (push) Failing after 1m10s
2026-09-30 09:32:20 -04:00
archipelago dc962c53b0 docs: record live lifecycle acceptance and next release blockers 2026-09-30 09:31:18 -04:00
archipelago 6ac26f637c fix(apps): preserve lifecycle state and wait for usable launch endpoints 2026-09-30 09:10:30 -04:00
archipelago 27d81e956d fix(installer): ship all app build contexts and refresh GitWorkshop dependencies 2026-09-30 09:09:21 -04:00
archipelago eb39391223 fix(ui): keep Bitcoin version choices readable in kiosk 2026-09-30 09:09:21 -04:00
chaum b02ba4100d Merge pull request 'fix(files): save purchased files atomically with rootless ownership' (#162) from fix/filebrowser-purchase-filing into main 2026-09-30 12:58:36 +00:00
chaum 3daea6623b Merge pull request 'fix(ecash): prevent paid-download replay and read failures after charging' (#161) from fix/ecash-paid-download-v2-keyset into main 2026-09-30 12:58:33 +00:00
archipelago d42f448e31 docs: close verified 1.8.21 OTA and ISO publication 2026-09-30 07:46:05 -04:00
archipelago 1566f1bb00 docs: record tested paid-download PRs for next release 2026-09-30 07:34:18 -04:00
archipelago 0677924a64 Merge current main and make purchase filing atomic under concurrent writes 2026-09-30 07:26:51 -04:00
archipelago 971d477795 Merge current main and harden paid-download delivery 2026-09-30 07:25:47 -04:00
archipelago f12042f194 docs: track X250 kiosk Bitcoin version selector regression 2026-09-30 07:01:57 -04:00
archipelago e7cf336665 chore: publish release v1.8.21-alpha
Demo images / Build & push demo images (push) Failing after 37s
2026-09-30 05:55:13 -04:00
archipelago 8ca20de82e release: prepare signed 1.8.21-alpha OTA 2026-09-30 05:51:16 -04:00
archipelago 1fa654cb6a docs: record 1.8.21 artifact and two-node release acceptance 2026-09-30 05:39:26 -04:00
archipelago c993d9dd0d fix(lnd): require observed Bitcoin lifecycle change before dependency restart 2026-09-30 05:16:17 -04:00
archipelago 33d2b3ce60 fix(containers): preserve graceful shutdown through Quadlet and prepare 1.8.21 2026-09-30 04:59:30 -04:00
archipelago c7ce35bd43 chore: publish release v1.8.20-alpha
Demo images / Build & push demo images (push) Failing after 1m31s
2026-09-30 04:32:43 -04:00
ssmithxandClaude Opus 5.5 33477f284b fix(files): file purchased content into FileBrowser folders again
Every paid download logged "filing into filebrowser/Music/... failed
(non-fatal): Permission denied". The purchase played in-app but never
appeared in Files. FileBrowser's folders belong to its rootless container
range (host uid 100000, mode 755). This service is host uid 1000, outside
that range, so it can read them but not create files in them.

New container::filebrowser::save_new_file:
- Writes directly when the folder allows it.
- Otherwise writes through `podman unshare`, where that uid range is
  ours: to a temp file, then chowned to the folder's owner, set to 0644,
  and hard-linked into place. FileBrowser never sees a partial file and an
  existing file is never replaced. A missing folder is created and given
  its parent's owner. No sudo.
- Keeps the "name (2).ext" de-duplication the RPC did inline.

Checked the unshare script on amishparadise in a scratch folder owned
like FileBrowser's: new folder + file OK, owner/mode right, no clobber,
no temp file left, and the service can read the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:36:31 +00:00
ssmithxandClaude Opus 5.5 03e38d1ca3 test: regression tests for the paid-download fixes
- mint_client: a stub mint shows swap() sends the full v2 keyset id when
  given a cashuB short id, and leaves complete v1/v2 ids unchanged.
- fips::dial: the single-delivery decisions are now small functions
  (fips_answer_is_final, fips_retryable). Tests cover them and, against a
  silent local peer, check that a single-delivery request isn't resent
  after a timeout while an ordinary one still is.
- content_server: an unreadable paid file returns Unavailable before the
  payment gate runs, and a readable one still returns 402. Also covers
  ensure_readable's grant/reopen behaviour. The podman grant is replaced
  by a refusal under cfg(test) so results don't depend on the host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:12:16 +00:00
ssmithxandClaude Opus 5.5 e5fc99d66c fix(content): never charge for a file the seller can't serve or replay a spent token
After the keyset-id fix, a Minibits paid download still failed and the
buyer lost the sats. What happened, 2026-09-29, amishparadise:

1. The seller redeemed the token, then failed to read the file. It was a
   FileBrowser upload owned by the container subuid (100999) with mode
   0640. The handler mapped that Err to 404.
2. The buyer's FIPS dial treats 404 as "fall back to Tor" and resent the
   request with the same, now spent, token. The seller answered 402, and
   the buyer showed "seller doesn't accept your Cashu mint".

Fixes:
- serve_content checks the file is readable before the paid gate. If it
  isn't, it grants read with `podman unshare chmod a+r`, which matches
  the other shared files. If that also fails it returns Unavailable (503)
  without taking payment.
- The content handler returns 500 on internal errors and logs them,
  instead of a silent 404.
- New PeerRequest::single_delivery(), used for the paid download: the
  FIPS answer is final, FIPS retries only when it never connected, and
  there's no Tor replay once the request may have been delivered.
- The buyer shows the seller's error text for non-402 failures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:42:20 +00:00
ssmithxandClaude Opus 5.5 8b74803290 fix(ecash): repair short v2 keyset ids on every swap, not just receive
Paid cloud downloads paid with Minibits ecash were always rejected. The
buyer sends a cashuB token, which carries NUT-02 v2 keyset ids in their
8-byte short form. Minibits rotated its active keyset to a v2 id, and the
seller's verify_and_receive_payment called MintClient::swap directly,
skipping the short->full id repair that only receive_token applied. The
mint answered 422 ("ID length invalid"). The buyer then showed the
misleading "seller doesn't accept your Cashu mint" hint.

Move the repair into swap() so every caller is covered: payment verify,
streaming gate, send change, and cross-mint swaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 15:47:23 +00:00
96 changed files with 5697 additions and 1085 deletions
+24
View File
@@ -2,6 +2,30 @@
## Unreleased
- Checked Bitcoin and Electrum companion dashboards instead of backend protocol ports, preserving dashboard access during initial sync.
- Removed web-interface waiting messages from headless services such as Phoenixd and clarified which interface is unavailable for launchable apps.
- Finished runtime app-file promotion before manifest loading, preventing startup catalog refresh from forgetting disk-only apps.
- Named the app in compact readiness messages and kept app-card actions aligned at the bottom.
- Removed duplicate Mempool cards caused by frontend container aliases in restored inventory.
- Kept installed apps visible through restarts and hard refreshes, and delayed app launches until their web interface is ready.
- Made Bitcoin version selection readable and usable in the ThinkPad kiosk, above the pruning settings.
- Restored GitWorkshop build files in installation/update payloads and made slow image-pull progress clearer.
- Fixed same-node Gitea access from Portainer, with persistent runtime migration, state backups and recovery after failed restarts.
- Preserved Gitea configuration and SSH operation during fresh setup and upgrades.
- Improved paid-file delivery, saved-file permissions and repeat-download compatibility; verified Tor-only payment with change, rejection refunds and free repeat downloads.
- Added a headless Angor Indexer service using the existing Mempool/ElectrumX stack, and an optional separate Angor relay.
- Prevented manifest command arguments containing apostrophes from being corrupted in generated services.
## v1.8.21-alpha (2026-09-30)
- Fixed Bitcoin and other containers being forcibly stopped after ten seconds during managed updates and restarts.
- Existing installations now receive the same graceful shutdown allowance as new containers, without restarting apps just to apply this setting.
- Prevented unnecessary Lightning restarts when Bitcoin has stayed running; dependency restarts now require an observed Bitcoin container change.
- Includes the Cashu payment, optional Bitcoin pruning, Lightning readiness, and explorer improvements from 1.8.20.
## v1.8.20-alpha (2026-09-29)
- Fixed Cashu file payments rejected despite a shared mint, and preserved the payment amount when mint fees reduce change.
+29
View File
@@ -644,6 +644,35 @@
"/var/lib/archipelago/vaultwarden:/data"
]
}
},
{
"id": "angor-indexer",
"title": "Angor Indexer",
"version": "1.0.1",
"description": "Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s address as the custom indexer in Angor settings. A relay is optional and installed separately.",
"dockerImage": "source.archipelago-foundation.org/chaum/angor-indexer:1.0.1",
"author": "Angor / Archipelago",
"requires": [
"Mempool API",
"Unpruned Bitcoin"
],
"category": "money",
"tier": "optional",
"icon": "/assets/img/app-icons/angor-green.png",
"repoUrl": "https://github.com/block-core/angor"
},
{
"id": "angor-relay",
"title": "Angor Relay",
"version": "1.1.2",
"description": "Optional dedicated Nostr relay for Angor project metadata. Separate storage and access settings keep the node’s internal relay private. Add this service’s address to Angor’s relay settings; use WSS for browser clients.",
"dockerImage": "source.archipelago-foundation.org/chaum/angor-relay:1.1.2",
"author": "Angor / Archipelago",
"requires": [],
"category": "nostr",
"tier": "optional",
"icon": "/assets/img/app-icons/angor-green.png",
"repoUrl": "https://github.com/hoytech/strfry"
}
]
}
+57
View File
@@ -0,0 +1,57 @@
# Angor Indexer
Headless mainnet API endpoint for Angor. The service reuses this node's Mempool
backend and Electrum index instead of creating a second blockchain database.
An unpruned, fully synced Bitcoin node is required. Installing against a pruned
node must show the existing archival-node requirement; it must never silently
unprune or replace its Bitcoin data.
## Connect Angor
Install **Angor Indexer** in the store. Its API appears under **Services**.
In Angor settings, use `http://<node-address>:8998/` as the custom indexer origin.
The `/health` endpoint reports readiness against Mempool's indexed block height;
it returns 503 while that backend is unavailable. Index building may take time.
Browser clients require a reachable HTTPS origin with a trusted certificate.
Configure your HTTPS reverse proxy to forward to port 8998, then use that HTTPS
origin in Angor. Do not disable browser TLS checks. The API supports both
`/api/v1/` and `/api/` paths, transaction broadcast, and CORS without cookies.
This endpoint intentionally exposes public blockchain queries and transaction
broadcast through the app gate without dashboard-cookie login. It has no Bitcoin
RPC password, wallet keys, or persistent wallet data. The backend stays on the
managed container network; its private port does not become publicly exposed.
You can change network access using the node's normal access controls.
## Relay
A relay is optional. Angor can continue using its configured external relays.
Install **Angor Relay** separately to host project metadata locally, then add
`ws://<node-address>:8091/` in Angor, or a trusted `wss://` proxy origin for browser
clients. Its storage and configuration are separate from the node's internal
relay; installing or uninstalling it does not change the internal relay.
## Packaging
Build the pinned image with:
```
podman build -t source.archipelago-foundation.org/chaum/angor-indexer:1.0.1 apps/angor-indexer/container
```
The image runs as UID 101 with a read-only root filesystem and no capabilities.
Only temporary nginx state is writable. Runtime DNS is read from resolv.conf so
Mempool recreation does not require editing IP addresses or restarting this app.
No app-specific Rust installer is required.
Source documentation: [Angor's official deployment guide](https://github.com/block-core/angor/blob/869dd43cf38332dd7128a284a6bf4c1cac44c1a7/docker/DEPLOY-INDEXER-AND-RELAY.md).
The app icon is based on [Angor’s dark-mode app icon](https://angor.io/images/app-icon-dark-mode.png), retrieved 2026-09-30. At the operator’s request, the outer corners use the same green as the background. The built-in imagegen edit preserved the black mark and filled the square green; the project asset is `neode-ui/public/assets/img/app-icons/angor-green.png`.
Tests and release acceptance are recorded in the next-release checklist. The
health probe establishes backend availability, not a guarantee that every
address query is indexed at the latest Bitcoin tip.
Install Mempool Explorer first. The declarative `install_prerequisites` check
refuses a new adapter installation if its Mempool API component is absent, before
creating an installed-app record. It does not install or resync Bitcoin for you.
+6
View File
@@ -0,0 +1,6 @@
FROM docker.io/library/nginx:1.31.3-alpine@sha256:1d40e3eb3bf4f138de1d67193f2aa5309fcaf343eb5ffadbf5e9439de1eb1ebb
COPY nginx.conf /etc/angor-nginx.conf.template
COPY entrypoint.sh /usr/local/bin/angor-indexer
USER 101:101
EXPOSE 8080
ENTRYPOINT ["/usr/local/bin/angor-indexer"]
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
set -eu
# Resolve through the container runtime's DNS, including after dependency
# recreation. Never bake a container IP into the indexer endpoint.
DNS_RESOLVER=$(awk '/^nameserver[[:space:]]/ {print $2; exit}' /etc/resolv.conf)
case "$DNS_RESOLVER" in
''|*[!0-9a-fA-F.:]*) echo 'Container DNS resolver is unavailable' >&2; exit 1 ;;
esac
case "$DNS_RESOLVER" in *:*) DNS_RESOLVER="[$DNS_RESOLVER]" ;; esac
export DNS_RESOLVER
envsubst '${DNS_RESOLVER}' < /etc/angor-nginx.conf.template > /tmp/nginx.conf
exec nginx -c /tmp/nginx.conf -g 'daemon off;'
+63
View File
@@ -0,0 +1,63 @@
worker_processes 1;
pid /tmp/nginx.pid;
error_log /dev/stderr warn;
events { worker_connections 512; }
http {
access_log off;
server_tokens off;
client_body_temp_path /tmp/client_temp;
proxy_temp_path /tmp/proxy_temp;
fastcgi_temp_path /tmp/fastcgi_temp;
uwsgi_temp_path /tmp/uwsgi_temp;
scgi_temp_path /tmp/scgi_temp;
resolver ${DNS_RESOLVER} valid=10s ipv6=off;
upstream mempool_backend {
zone mempool_backend 64k;
server mempool-api:8999 resolve;
}
server {
listen 8080;
client_max_body_size 4m;
proxy_connect_timeout 5s;
proxy_read_timeout 60s;
proxy_send_timeout 30s;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Connection "";
proxy_set_header Authorization "";
proxy_set_header Cookie "";
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin '*' always;
add_header Access-Control-Allow-Methods 'GET, HEAD, POST, OPTIONS' always;
add_header Access-Control-Allow-Headers 'Content-Type' always;
add_header Cache-Control 'no-store' always;
if ($request_method = OPTIONS) { return 204; }
# Mempool's backend uses /api/v1. Match its frontend's shorter /api
# surface too, without doubling already-versioned Angor URLs.
rewrite ^/api/(?!v1/)(.*)$ /api/v1/$1 last;
location = / {
default_type application/json;
return 200 '{"service":"Angor Indexer","network":"mainnet","api":"/api/v1","health":"/health"}\n';
}
# Readiness checks the indexing backend, not this gateway's process.
location = /health {
limit_except GET { deny all; }
proxy_pass http://mempool_backend/api/v1/blocks/tip/height;
proxy_intercept_errors on;
error_page 500 502 503 504 =503 @waiting;
}
location @waiting {
default_type application/json;
return 503 '{"status":"waiting","message":"Waiting for Bitcoin and Mempool indexing"}\n';
}
location ~ ^/api/(v1/)?tx$ {
limit_except GET POST { deny all; }
proxy_pass http://mempool_backend;
}
location /api/ {
limit_except GET { deny all; }
proxy_pass http://mempool_backend;
}
location / { return 404; }
}
}
+68
View File
@@ -0,0 +1,68 @@
app:
id: angor-indexer
name: Angor Indexer
version: 1.0.1
description: Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool
and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s
address as the custom indexer in Angor settings. A relay is optional and installed
separately.
category: money
install_prerequisites:
- mempool-api
upstream:
kind: github
repo: block-core/angor
container:
image: source.archipelago-foundation.org/chaum/angor-indexer:1.0.1
pull_policy: if-not-present
network: archy-net
dependencies:
- app_id: mempool-api
version: '>=3.0.0'
- bitcoin:archival
resources:
cpu_limit: 1
memory_limit: 128Mi
disk_limit: 128Mi
security:
capabilities: []
readonly_root: true
no_new_privileges: true
user: 101
network_policy: isolated
ports:
- host: 8998
container: 8080
protocol: tcp
bind: 127.0.0.1
auth: open
auth_rationale: Public Bitcoin chain-data API and validated transaction broadcast for Angor clients; no wallet keys or node RPC credentials are exposed. Browser cookie login would break machine clients.
interfaces:
main:
name: Angor Indexer API
description: Use this origin as Angor’s custom mainnet indexer URL. HTTPS is
required for browser clients.
type: api
port: 8998
protocol: http
path: /
health_check:
type: http
endpoint: http://localhost:8080
path: /health
interval: 30s
timeout: 8s
retries: 3
bitcoin_integration:
rpc_access: none
sync_required: true
pruning_support: false
metadata:
icon: /assets/img/app-icons/angor-green.png
tier: optional
repo: https://github.com/block-core/angor
features:
- Angor mainnet API
- Reuses existing Mempool indexing
- No separate blockchain database
- Optional independent relay
+21
View File
@@ -0,0 +1,21 @@
# Angor Relay
Optional standalone strfry relay for Angor's public project metadata. See
[Angor Indexer setup](../angor-indexer/README.md) for client URLs and HTTPS/WSS.
The gate exposes port 8091 for Nostr clients. strfry validates event signatures;
this is a public relay, not a private messaging archive. It mounts only
`/var/lib/archipelago/angor-relay` and its separate configuration directory.
It never opens, reconfigures or shares the node's internal strfry database.
The configuration is seeded only when absent, preserving operator changes.
Stop the service before making a consistent backup of its event database.
Ordinary start/restart/recreation preserves both mounts. Use the standard app
lifecycle; do not manually recreate a systemd-managed container.
## Image provenance
Mirrored from `docker.io/dockurr/strfry:1.1.2`, upstream manifest digest
`sha256:e81d238db13507f6ef24c49d47cd0b0ea58ff207961f10581fa2a7c901054df4`.
The public Angor policy is supplied by this app's own configuration; it does not
reuse the internal relay's event whitelist.
+223
View File
@@ -0,0 +1,223 @@
app:
id: angor-relay
name: Angor Relay
version: 1.1.2
upstream:
kind: github
repo: hoytech/strfry
description: Optional dedicated Nostr relay for Angor project metadata. Separate
storage and access settings keep the node’s internal relay private. Add this service’s
address to Angor’s relay settings; use WSS for browser clients.
container:
image: source.archipelago-foundation.org/chaum/angor-relay:1.1.2
pull_policy: if-not-present
dependencies:
- storage: 5Gi
resources:
cpu_limit: 1
memory_limit: 512Mi
disk_limit: 5Gi
security:
capabilities: []
readonly_root: true
no_new_privileges: true
seccomp_profile: default
network_policy: isolated
apparmor_profile: nostr-relay
ports:
- host: 8091
container: 7777
protocol: tcp
bind: 127.0.0.1
auth: open
auth_rationale: Dedicated public Nostr relay for Angor project metadata; strfry verifies event signatures. It has separate storage from the private node relay and no wallet or node credentials.
volumes:
- type: bind
source: /var/lib/archipelago/angor-relay
target: /app/strfry-db
options:
- rw
- type: bind
source: /var/lib/archipelago/angor-relay-config/angor-relay.conf
target: /etc/strfry.conf
options:
- ro
files:
- path: /var/lib/archipelago/angor-relay-config/angor-relay.conf
overwrite: false
content: |
##
## Default strfry config
##
# Directory that contains the strfry LMDB database (restart required)
db = "./strfry-db/"
dbParams {
# Maximum number of threads/processes that can simultaneously have LMDB transactions open (restart required)
maxreaders = 256
# Size of mmap() to use when loading LMDB (default is 10TB, does *not* correspond to disk-space used) (restart required)
mapsize = 10995116277760
# Disables read-ahead when accessing the LMDB mapping. Reduces IO activity when DB size is larger than RAM. (restart required)
noReadAhead = false
}
events {
# Maximum size of normalised JSON, in bytes
maxEventSize = 65536
# Events newer than this will be rejected
rejectEventsNewerThanSeconds = 900
# Events older than this will be rejected
rejectEventsOlderThanSeconds = 94608000
# Ephemeral events older than this will be rejected
rejectEphemeralEventsOlderThanSeconds = 60
# Ephemeral events will be deleted from the DB when older than this
ephemeralEventsLifetimeSeconds = 300
# Maximum number of tags allowed
maxNumTags = 2000
# Maximum size for tag values, in bytes
maxTagValSize = 1024
}
relay {
# Interface to listen on. Use 0.0.0.0 to listen on all interfaces (restart required)
bind = "0.0.0.0"
# Port to open for the nostr websocket protocol (restart required)
port = 7777
# Set OS-limit on maximum number of open files/sockets (if 0, don't attempt to set) (restart required)
nofiles = 0
# HTTP header that contains the client's real IP, before reverse proxying (ie x-real-ip) (MUST be all lower-case)
realIpHeader = ""
info {
# NIP-11: Name of this server. Short/descriptive (< 30 characters)
name = "Angor Relay"
# NIP-11: Detailed information about relay, free-form
description = "Dedicated public relay for Angor project metadata."
# NIP-11: Administrative nostr pubkey, for contact purposes
pubkey = ""
# NIP-11: Alternative administrative contact (email, website, etc)
contact = ""
# NIP-11: URL pointing to an image to be used as an icon for the relay
icon = ""
# List of supported lists as JSON array, or empty string to use default. Example: "[1,2]"
nips = ""
}
# Maximum accepted incoming websocket frame size (should be larger than max event) (restart required)
maxWebsocketPayloadSize = 131072
# Maximum number of filters allowed in a REQ
maxReqFilterSize = 200
# Websocket-level PING message frequency (should be less than any reverse proxy idle timeouts) (restart required)
autoPingSeconds = 55
# If TCP keep-alive should be enabled (detect dropped connections to upstream reverse proxy)
enableTcpKeepalive = false
# How much uninterrupted CPU time a REQ query should get during its DB scan
queryTimesliceBudgetMicroseconds = 10000
# Maximum records that can be returned per filter
maxFilterLimit = 500
# Maximum number of subscriptions (concurrent REQs) a connection can have open at any time
maxSubsPerConnection = 20
writePolicy {
# If non-empty, path to an executable script that implements the writePolicy plugin logic
plugin = ""
}
compression {
# Use permessage-deflate compression if supported by client. Reduces bandwidth, but slight increase in CPU (restart required)
enabled = true
# Maintain a sliding window buffer for each connection. Improves compression, but uses more memory (restart required)
slidingWindow = true
}
logging {
# Dump all incoming messages
dumpInAll = false
# Dump all incoming EVENT messages
dumpInEvents = false
# Dump all incoming REQ/CLOSE messages
dumpInReqs = false
# Log performance metrics for initial REQ database scans
dbScanPerf = false
# Log reason for invalid event rejection? Can be disabled to silence excessive logging
invalidEvents = true
}
numThreads {
# Ingester threads: route incoming requests, validate events/sigs (restart required)
ingester = 3
# reqWorker threads: Handle initial DB scan for events (restart required)
reqWorker = 3
# reqMonitor threads: Handle filtering of new events (restart required)
reqMonitor = 3
# negentropy threads: Handle negentropy protocol messages (restart required)
negentropy = 2
}
negentropy {
# Support negentropy protocol messages
enabled = true
# Maximum records that sync will process before returning an error
maxSyncEvents = 1000000
}
}
health_check:
type: http
endpoint: http://127.0.0.1:7777
path: /health
interval: 30s
timeout: 5s
retries: 3
nostr_integration:
relay_type: public
monetization_enabled: false
category: nostr
interfaces:
main:
name: Angor Relay
description: Nostr WebSocket endpoint; use ws:// for LAN or wss:// through your
HTTPS domain.
type: api
port: 8091
protocol: http
path: /
metadata:
icon: /assets/img/app-icons/angor-green.png
tier: optional
repo: https://github.com/hoytech/strfry
features:
- Angor project metadata
- Separate from the node relay
- Persistent Nostr event storage
+15 -8
View File
@@ -15,6 +15,9 @@ app:
image: source.archipelago-foundation.org/lfg2025/gitea:1.27.3
pull_policy: if-not-present
# Preserve repositories, database, keys and configuration during runtime repairs.
backup_before_runtime_change: true
dependencies:
# Source history, LFS objects, release artifacts and OCI layers all share
# this persistent store. 500Mi was only suitable for an empty demo node.
@@ -25,7 +28,7 @@ app:
disk_limit: 50Gi
security:
capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_BIND_SERVICE]
capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_BIND_SERVICE, SYS_CHROOT]
readonly_root: false
no_new_privileges: false
network_policy: bridge
@@ -62,6 +65,17 @@ app:
target: /etc/gitea
options: [rw]
# Seed a fresh installation with the same origin advertised by the app gate.
# Existing app.ini (including custom HTTPS/domain settings) is never replaced.
files:
- path: /var/lib/archipelago/gitea/data/gitea/conf/app.ini
overwrite: false
content: |
[server]
DOMAIN = {{HOST_IP}}
SSH_DOMAIN = {{HOST_IP}}
ROOT_URL = http://{{HOST_IP}}:3001/
environment:
- GITEA__database__DB_TYPE=sqlite3
- GITEA__server__SSH_PORT=2222
@@ -106,10 +120,3 @@ app:
- Issue tracking and pull requests
- CI/CD via Gitea Actions
- Lightweight SQLite deployment
nginx_proxy:
listen: 3000
proxy_pass: http://127.0.0.1:3001
extra_headers:
- proxy_hide_header X-Frame-Options
- proxy_hide_header Content-Security-Policy
+8
View File
@@ -14,8 +14,16 @@ app:
container:
image: source.archipelago-foundation.org/lfg2025/portainer:2.45.0
pull_policy: if-not-present
# Portainer fetches Git sources and images from services on this same node.
# Rootless pasta copies the host LAN address into its namespace, so a LAN
# URL points back at Portainer itself. Give it a private address with the
# supported rootless slirp backend; public app URLs still traverse the gate.
network: slirp4netns
data_uid: "1000:1000"
# Snapshot state before an upgrade recreates this app with new networking.
backup_before_runtime_change: true
dependencies:
- storage: 1Gi
+1 -1
View File
@@ -104,7 +104,7 @@ dependencies = [
[[package]]
name = "archipelago"
version = "1.8.20-alpha"
version = "1.8.21-alpha"
dependencies = [
"anyhow",
"archipelago-container",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "archipelago"
version = "1.8.20-alpha"
version = "1.8.21-alpha"
edition = "2021"
license.workspace = true
description = "Archipelago Bitcoin Node OS - Native backend"
+23 -1
View File
@@ -162,11 +162,33 @@ impl ApiHandler {
r#"{"error":"This file is shared with the host's federation peers only. Federate with that node (exchange invites) so it recognizes you, then try again."}"#,
),
)),
Ok(content_server::ServeResult::NotFound) | Err(_) => Ok(build_response(
Ok(content_server::ServeResult::Unavailable) => Ok(build_response(
StatusCode::SERVICE_UNAVAILABLE,
"application/json",
hyper::Body::from(
r#"{"error":"The seller's node can't read this file right now. This request did not redeem an ecash payment."}"#,
),
)),
Ok(content_server::ServeResult::RangeNotSatisfiable(total)) => Ok(Response::builder()
.status(StatusCode::RANGE_NOT_SATISFIABLE)
.header("Content-Range", format!("bytes */{total}"))
.body(hyper::Body::empty())
.unwrap()),
Ok(content_server::ServeResult::NotFound) => Ok(build_response(
StatusCode::NOT_FOUND,
"text/plain",
hyper::Body::from("Content not found"),
)),
// Not a 404: a paid request may already have been charged by the
// time this fails, and "not found" hid the real error entirely.
Err(e) => {
tracing::error!("Serving content {content_id} failed: {e:#}");
Ok(build_response(
StatusCode::INTERNAL_SERVER_ERROR,
"text/plain",
hyper::Body::from("Failed to serve content"),
))
}
}
}
+53 -85
View File
@@ -43,6 +43,25 @@ async fn reclaim_spent_ecash(data_dir: &std::path::Path, token: &str, backend: &
}
}
/// Only pass through the peer's bounded, printable explanation; refund status
/// is always determined locally and must never come from the peer's wording.
fn seller_error_message(status: reqwest::StatusCode, body: &str) -> String {
let reason = serde_json::from_str::<serde_json::Value>(body)
.ok()
.and_then(|v| v.get("error").and_then(|e| e.as_str()).map(str::to_owned));
match reason {
Some(reason) if !reason.trim().is_empty() => {
let clean: String = reason
.chars()
.filter(|c| !c.is_control())
.take(240)
.collect();
format!("Seller response ({status}): {clean}")
}
_ => format!("Peer returned an error ({status})."),
}
}
/// Keep first purchases and cached repeats compatible with both existing clients.
fn paid_content_response(bytes: &[u8], mime: &str, paid_sats: u64) -> serde_json::Value {
use base64::Engine;
@@ -54,13 +73,9 @@ fn paid_content_response(bytes: &[u8], mime: &str, paid_sats: u64) -> serde_json
})
}
/// FileBrowser owns its files through a rootless UID mapping. Use its authenticated
/// API rather than writing host paths with the backend's unrelated UID. Its
/// override=false upload atomically refuses existing names, including races.
/// File purchases through an atomic no-clobber write in Files' own namespace.
async fn file_purchase_in_files(
client: &reqwest::Client,
base_url: &str,
token: &str,
data_dir: &std::path::Path,
filename: &str,
mime: &str,
bytes: &[u8],
@@ -72,59 +87,24 @@ async fn file_purchase_in_files(
} else {
"Documents"
};
let mut folder_url = reqwest::Url::parse(base_url)?;
folder_url
.path_segments_mut()
.map_err(|_| anyhow::anyhow!("Invalid Files URL"))?
.extend(["api", "resources", folder, ""]);
let response = client
.get(folder_url.clone())
.header("X-Auth", token)
.send()
.await?;
if response.status() == reqwest::StatusCode::NOT_FOUND {
let response = client
.post(folder_url.clone())
.header("X-Auth", token)
.send()
.await?;
if response.status() != reqwest::StatusCode::CONFLICT {
response.error_for_status()?;
}
} else {
response.error_for_status()?;
}
let base = std::path::Path::new(filename)
let root = data_dir.join("filebrowser");
anyhow::ensure!(
tokio::fs::metadata(&root).await?.is_dir(),
"Files storage is unavailable"
);
let name = std::path::Path::new(filename)
.file_name()
.and_then(|n| n.to_str())
.filter(|n| !n.is_empty())
.unwrap_or("download");
let (stem, extension) = match base.rsplit_once('.') {
Some((stem, ext)) if !stem.is_empty() => (stem, format!(".{ext}")),
_ => (base, String::new()),
};
for attempt in 1..=100 {
let name = if attempt == 1 {
base.to_string()
} else {
format!("{stem} ({attempt}){extension}")
};
let mut url = folder_url.clone();
url.path_segments_mut().unwrap().pop_if_empty().push(&name);
url.query_pairs_mut().append_pair("override", "false");
let response = client
.post(url)
.header("X-Auth", token)
.body(bytes.to_vec())
.send()
.await?;
if response.status() == reqwest::StatusCode::CONFLICT {
continue;
}
response.error_for_status()?;
return Ok(format!("{folder}/{name}"));
}
anyhow::bail!("Too many existing copies; purchased file remains in the purchase cache")
let path =
crate::container::filebrowser::save_new_file(&root.join(folder), name, bytes).await?;
Ok(format!(
"{folder}/{}",
path.file_name()
.and_then(|n| n.to_str())
.context("Invalid Files name")?
))
}
impl RpcHandler {
@@ -623,13 +603,14 @@ impl RpcHandler {
let path = format!("/content/{}", content_id);
// Surface a real reason instead of the generic sanitized error (#30):
// the dial already tries FIPS/mesh then falls back to Tor, so a failure
// here means the peer is genuinely unreachable on both transports.
// A bearer token must not be replayed after an ambiguous delivery.
// A transport error can mean the seller received it without replying.
let (response, transport) =
match crate::fips::dial::PeerRequest::new(fips_npub.as_deref(), onion, &path)
.service(crate::settings::transport::PeerService::PeerFiles)
.header("X-Federation-DID", local_did)
.header("X-Payment-Token", token_str.clone())
.single_delivery()
.timeout(std::time::Duration::from_secs(900))
.send_get()
.await
@@ -642,7 +623,7 @@ impl RpcHandler {
let refund =
reclaim_spent_ecash(&self.config.data_dir, &token_str, used_backend).await;
return Ok(serde_json::json!({
"error": format!("Could not reach the peer over mesh or Tor. {refund}")
"error": format!("The purchase could not be completed. {refund}")
}));
}
};
@@ -679,7 +660,7 @@ impl RpcHandler {
tracing::warn!("paid download: seller {onion} returned {status}: {body}");
let refund = reclaim_spent_ecash(&self.config.data_dir, &token_str, used_backend).await;
return Ok(serde_json::json!({
"error": format!("Peer returned an error ({status}). {refund}")
"error": format!("{} {refund}", seller_error_message(status, &body))
}));
}
@@ -693,10 +674,17 @@ impl RpcHandler {
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "application/octet-stream".to_string());
let bytes = response
.bytes()
.await
.context("Failed to read response body")?;
let bytes = match response.bytes().await {
Ok(bytes) => bytes,
Err(error) => {
tracing::warn!("paid download: response body failed: {error}");
let refund =
reclaim_spent_ecash(&self.config.data_dir, &token_str, used_backend).await;
return Ok(serde_json::json!({
"error": format!("The file transfer was interrupted after payment was sent. {refund}")
}));
}
};
// Persist the purchase so it "stays unlocked" for this buyer: cache the
// bytes + metadata keyed by (onion, content_id). The gallery then renders
@@ -728,28 +716,8 @@ impl RpcHandler {
// The durable purchased-content cache above is primary. A Files copy
// remains optional: a stopped FileBrowser must not undo a paid download.
let filed = async {
let auth = self.handle_filebrowser_token().await?;
let token = auth
.get("token")
.and_then(|v| v.as_str())
.context("FileBrowser omitted its authentication token")?;
let client = reqwest::Client::builder()
.no_proxy()
.redirect(reqwest::redirect::Policy::none())
.timeout(std::time::Duration::from_secs(30))
.build()?;
file_purchase_in_files(
&client,
"http://127.0.0.1:8083",
token,
&filename,
&mime_type,
&bytes,
)
.await
}
.await;
let filed =
file_purchase_in_files(&self.config.data_dir, &filename, &mime_type, &bytes).await;
match filed {
Ok(path) => tracing::info!("paid download: filed into Files/{path}"),
Err(error) => tracing::warn!(
+41 -132
View File
@@ -1,69 +1,4 @@
use super::*;
use hyper::{
service::{make_service_fn, service_fn},
Body, Response, Server,
};
use std::{
collections::VecDeque,
convert::Infallible,
sync::{Arc, Mutex},
};
struct FilesApi {
url: String,
seen: Arc<Mutex<Vec<(String, String, Vec<u8>)>>>,
task: tokio::task::JoinHandle<()>,
}
impl Drop for FilesApi {
fn drop(&mut self) {
self.task.abort();
}
}
fn files_api(statuses: Vec<u16>) -> FilesApi {
let statuses = Arc::new(Mutex::new(VecDeque::from(statuses)));
let seen = Arc::new(Mutex::new(Vec::new()));
let history = seen.clone();
let server = Server::bind(&([127, 0, 0, 1], 0).into());
let address = server.local_addr();
let service = make_service_fn(move |_| {
let statuses = statuses.clone();
let seen = history.clone();
async move {
Ok::<_, Infallible>(service_fn(move |request: hyper::Request<Body>| {
let statuses = statuses.clone();
let seen = seen.clone();
async move {
assert_eq!(request.headers().get("X-Auth").unwrap(), "test-session");
let method = request.method().to_string();
let uri = request.uri().to_string();
let body = hyper::body::to_bytes(request.into_body())
.await
.unwrap()
.to_vec();
seen.lock().unwrap().push((method, uri, body));
let status = statuses
.lock()
.unwrap()
.pop_front()
.expect("unexpected extra Files request");
Ok::<_, Infallible>(
Response::builder()
.status(status)
.body(Body::empty())
.unwrap(),
)
}
}))
}
});
FilesApi {
url: format!("http://{address}"),
seen,
task: tokio::spawn(async move {
server.serve(service).await.unwrap();
}),
}
}
#[test]
fn first_and_cached_paid_downloads_have_the_same_client_payload_contract() {
@@ -85,80 +20,54 @@ fn first_and_cached_paid_downloads_have_the_same_client_payload_contract() {
}
#[tokio::test]
async fn files_copy_uses_authenticated_api_and_preserves_existing_names() {
let api = files_api(vec![200, 409, 200]);
let client = reqwest::Client::new();
let path = file_purchase_in_files(
&client,
&api.url,
"test-session",
"../my #file?.txt",
"text/plain",
b"paid bytes",
)
.await
.unwrap();
assert_eq!(path, "Documents/my #file? (2).txt");
let seen = api.seen.lock().unwrap();
assert_eq!(seen[0].0, "GET");
assert_eq!(seen[0].1, "/api/resources/Documents/");
assert_eq!(seen.len(), 3);
for (_, uri, body) in &seen[1..] {
assert!(uri.contains("override=false"));
assert!(uri.contains("%23file%3F"));
assert!(!uri.contains("../"));
assert_eq!(body, b"paid bytes");
}
}
#[tokio::test]
async fn files_copy_creates_missing_media_folder() {
async fn files_copy_routes_media_and_sanitizes_the_filename() {
let dir = tempfile::tempdir().unwrap();
tokio::fs::create_dir(dir.path().join("filebrowser"))
.await
.unwrap();
for (mime, folder) in [
("image/png", "Photos"),
("video/mp4", "Photos"),
("audio/ogg", "Music"),
("audio/mpeg", "Music"),
("text/plain", "Documents"),
] {
let api = files_api(vec![404, 200, 200]);
let path = file_purchase_in_files(
&reqwest::Client::new(),
&api.url,
"test-session",
"file",
mime,
b"bytes",
)
.await
.unwrap();
assert_eq!(path, format!("{folder}/file"));
let seen = api.seen.lock().unwrap();
assert_eq!(seen[1].0, "POST");
assert!(seen[1].1.ends_with('/'));
assert!(seen[1].2.is_empty());
assert_eq!(seen[2].2, b"bytes");
let relative = file_purchase_in_files(dir.path(), "../name #?.bin", mime, b"paid")
.await
.unwrap();
assert!(relative.starts_with(&format!("{folder}/name #?")));
assert_eq!(
tokio::fs::read(dir.path().join("filebrowser").join(relative))
.await
.unwrap(),
b"paid"
);
}
}
#[tokio::test]
async fn files_copy_fails_without_overwriting_or_claiming_success_on_errors() {
for statuses in [
vec![401],
vec![503],
vec![404, 500],
vec![200, 507],
vec![200, 403],
] {
let expected = statuses.len();
let api = files_api(statuses);
assert!(file_purchase_in_files(
&reqwest::Client::new(),
&api.url,
"test-session",
"file.txt",
"text/plain",
b"bytes"
)
.await
.is_err());
assert_eq!(api.seen.lock().unwrap().len(), expected);
async fn unavailable_files_storage_is_reported_without_creating_a_fake_installation() {
let dir = tempfile::tempdir().unwrap();
assert!(
file_purchase_in_files(dir.path(), "name", "text/plain", b"bytes")
.await
.is_err()
);
assert!(!dir.path().join("filebrowser").exists());
}
#[test]
fn seller_errors_are_bounded_printable_and_identified_as_peer_text() {
let status = reqwest::StatusCode::SERVICE_UNAVAILABLE;
let message = seller_error_message(status, r#"{"error":"Cannot read file\n\u0000"}"#);
assert!(message.starts_with("Seller response (503"));
assert!(message.ends_with("Cannot read file"));
assert!(!message.contains('\n') && !message.contains('\0'));
let body = serde_json::json!({"error": "é".repeat(1000)}).to_string();
assert!(seller_error_message(status, &body).chars().count() < 300);
for body in ["not JSON", r#"{"error": 7}"#, r#"{"error":" "}"#] {
assert_eq!(
seller_error_message(status, body),
"Peer returned an error (503 Service Unavailable)."
);
}
}
@@ -89,6 +89,15 @@ impl RpcHandler {
match handler.handle_package_install(params).await {
Ok(_) => {
info!("package.install {}: complete", package_id_spawn);
for id in [&package_id_spawn, &format!("archy-{}", package_id_spawn)] {
crate::crash_recovery::clear_user_uninstalled(&handler.config.data_dir, id)
.await;
}
crate::crash_recovery::mark_installed(
&handler.config.data_dir,
&package_id_spawn,
)
.await;
// The install pipeline has verified the container is up
// and healthy (see install.rs post-start exit check).
// Kick the scanner first so the fresh manifest (with
@@ -184,17 +193,20 @@ impl RpcHandler {
// phase is cleared (None) so no stale InstallPhase
// lingers on the card.
let err_msg = format!("Install failed: {:#}", e);
let (mut data, _) = handler.state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(&package_id_spawn) {
entry.state = PackageState::Stopped;
entry.install_progress = Some(crate::data_model::InstallProgress {
size: 0,
downloaded: 0,
phase: None,
message: Some(err_msg),
});
handler.state_manager.update_data(data).await;
}
handler
.state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(&package_id_spawn) {
entry.state = PackageState::Stopped;
entry.install_progress = Some(crate::data_model::InstallProgress {
size: 0,
downloaded: 0,
phase: None,
message: Some(err_msg),
});
}
})
.await;
}
}
});
@@ -252,6 +264,11 @@ impl RpcHandler {
match handler.handle_package_uninstall(params).await {
Ok(_) => {
info!("package.uninstall {}: complete", package_id_spawn);
for id in [&package_id_spawn, &format!("archy-{}", package_id_spawn)] {
crate::crash_recovery::mark_user_uninstalled(&handler.config.data_dir, id)
.await;
crate::crash_recovery::clear_installed(&handler.config.data_dir, id).await;
}
// Inner handler already removed the package entry on
// success. Nothing more to do here.
}
@@ -382,52 +399,56 @@ impl RpcHandler {
/// call, but fires before the spawn so the UI sees it immediately.
async fn flip_to_installing(state_manager: &StateManager, package_id: &str) {
use crate::data_model::{Description, Manifest, PackageDataEntry, StaticFiles};
let (mut data, _) = state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| PackageDataEntry {
state: PackageState::Installing,
health: None,
exit_code: None,
static_files: StaticFiles {
license: String::new(),
instructions: String::new(),
// Leave icon empty during the transient Installing window:
// hardcoding `<id>.png` is wrong for ~half our apps (many use
// `.svg` / `.webp`), producing a broken-image flicker until
// the scanner refreshes the entry. The frontend's `icon`
// computed falls through to `curatedMap.get(id)?.icon` which
// has the correct extensions for known apps.
icon: String::new(),
},
manifest: Manifest {
id: package_id.to_string(),
title: package_id.to_string(),
version: String::new(),
description: Description {
short: "Installing...".to_string(),
long: String::new(),
},
release_notes: String::new(),
license: String::new(),
wrapper_repo: String::new(),
upstream_repo: String::new(),
support_site: String::new(),
marketing_site: String::new(),
donation_url: None,
author: None,
website: None,
interfaces: None,
tier: None,
},
installed: None,
install_progress: None,
uninstall_stage: None,
available_update: None,
});
entry.state = PackageState::Installing;
state_manager.update_data(data).await;
state_manager
.mutate_data(|data| {
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| PackageDataEntry {
ui_ready: None,
state: PackageState::Installing,
health: None,
exit_code: None,
static_files: StaticFiles {
license: String::new(),
instructions: String::new(),
// Leave icon empty during the transient Installing window:
// hardcoding `<id>.png` is wrong for ~half our apps (many use
// `.svg` / `.webp`), producing a broken-image flicker until
// the scanner refreshes the entry. The frontend's `icon`
// computed falls through to `curatedMap.get(id)?.icon` which
// has the correct extensions for known apps.
icon: String::new(),
},
manifest: Manifest {
id: package_id.to_string(),
title: package_id.to_string(),
version: String::new(),
description: Description {
short: "Installing...".to_string(),
long: String::new(),
},
release_notes: String::new(),
license: String::new(),
wrapper_repo: String::new(),
upstream_repo: String::new(),
support_site: String::new(),
marketing_site: String::new(),
donation_url: None,
author: None,
website: None,
interfaces: None,
tier: None,
},
installed: None,
install_progress: None,
uninstall_stage: None,
available_update: None,
});
entry.ui_ready = Some(false);
entry.state = PackageState::Installing;
})
.await;
}
/// True when the failed install still has a real footprint: any container
@@ -485,20 +506,23 @@ async fn remove_entry_with_notification(
id_prefix: &str,
message: &str,
) {
let (mut data, _) = handler.state_manager.get_snapshot().await;
data.package_data.remove(package_id);
data.notifications.push(crate::data_model::Notification {
id: format!("{id_prefix}-{package_id}"),
level: crate::data_model::NotificationLevel::Error,
title: format!("Could not install {package_id}"),
message: message.to_string(),
timestamp: chrono::Utc::now().to_rfc3339(),
app_id: Some(package_id.to_string()),
});
while data.notifications.len() > 20 {
data.notifications.remove(0);
}
handler.state_manager.update_data(data).await;
handler
.state_manager
.mutate_data(|data| {
data.package_data.remove(package_id);
data.notifications.push(crate::data_model::Notification {
id: format!("{id_prefix}-{package_id}"),
level: crate::data_model::NotificationLevel::Error,
title: format!("Could not install {package_id}"),
message: message.to_string(),
timestamp: chrono::Utc::now().to_rfc3339(),
app_id: Some(package_id.to_string()),
});
while data.notifications.len() > 20 {
data.notifications.remove(0);
}
})
.await;
}
/// Flip an existing entry's state and return the pre-flip value (or None if
@@ -508,18 +532,23 @@ async fn flip_package_state(
package_id: &str,
new_state: PackageState,
) -> Option<PackageState> {
let (mut data, _) = state_manager.get_snapshot().await;
let prev = data.package_data.get(package_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.state = new_state;
state_manager.update_data(data).await;
} else {
warn!(
"flip_package_state: no entry for {} — cannot flip",
package_id
);
}
prev
state_manager
.mutate_data(|data| {
let prev = data.package_data.get(package_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(package_id) {
if new_state != PackageState::Running {
entry.ui_ready = Some(false);
}
entry.state = new_state;
} else {
warn!(
"flip_package_state: no entry for {} — cannot flip",
package_id
);
}
prev
})
.await
}
/// Set state unconditionally (no-op if entry no longer exists).
@@ -528,13 +557,18 @@ async fn set_package_state(
package_id: &str,
new_state: PackageState,
) {
let (mut data, _) = state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(package_id) {
if entry.state != new_state {
entry.state = new_state;
state_manager.update_data(data).await;
}
}
state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(package_id) {
if entry.state != new_state {
if new_state != PackageState::Running {
entry.ui_ready = Some(false);
}
entry.state = new_state;
}
}
})
.await
}
/// Set state and clear the uninstall_stage label. Used when an uninstall
@@ -545,12 +579,17 @@ async fn set_package_state_and_clear_uninstall_stage(
package_id: &str,
new_state: PackageState,
) {
let (mut data, _) = state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.state = new_state;
entry.uninstall_stage = None;
state_manager.update_data(data).await;
}
state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(package_id) {
if new_state != PackageState::Running {
entry.ui_ready = Some(false);
}
entry.state = new_state;
entry.uninstall_stage = None;
}
})
.await
}
/// Kick the container scanner to run immediately and wait for it to finish
@@ -22,6 +22,18 @@ const ARCHIVAL_BITCOIN_DEPENDENCY: &str = "bitcoin:archival";
/// hardcoded id list below — a new app just declares the dependency instead
/// of needing a code change here.
fn manifest_declares_archival_bitcoin(package_id: &str) -> bool {
// Registry-only apps need the same guard as OTA-bundled manifests. Honor
// the verified catalog's effective manifest before the disk fallback.
if let Some((_, value)) = crate::container::app_catalog::catalog_manifest_values()
.into_iter()
.find(|(id, _)| id == package_id)
{
if let Some(manifest) =
crate::container::app_catalog::catalog_manifest_overlay(package_id, value)
{
return dependency_list_declares_archival_bitcoin(&manifest.app.dependencies);
}
}
for apps_dir in manifest_apps_dirs() {
let path = apps_dir.join(package_id).join("manifest.yml");
let Ok(contents) = std::fs::read_to_string(&path) else {
@@ -1055,6 +1067,14 @@ mod tests {
// edit to `requires_unpruned_bitcoin`.
assert!(manifest_declares_archival_bitcoin("electrumx"));
assert!(manifest_declares_archival_bitcoin("mempool"));
let angor = archipelago_container::AppManifest::parse(include_str!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../apps/angor-indexer/manifest.yml"
)))
.unwrap();
assert!(dependency_list_declares_archival_bitcoin(
&angor.app.dependencies
));
// An app whose manifest exists but never declares the marker.
assert!(!manifest_declares_archival_bitcoin("bitcoin-knots"));
// An id with no manifest on disk at all.
+14 -132
View File
@@ -545,7 +545,7 @@ impl RpcHandler {
// Keep legacy install flow as default while migration is in progress.
if orchestrator_managed {
let orchestrator_app_id = orchestrator_install_app_id(package_id);
self.set_install_phase(package_id, InstallPhase::CreatingContainer)
self.set_install_phase(package_id, InstallPhase::PreparingApp)
.await;
install_log(&format!(
"INSTALL ORCH: {} — attempting orchestrator install as {}",
@@ -573,6 +573,9 @@ impl RpcHandler {
"message": format!("Package {} installed and started", package_id)
}));
}
Err(e) if e.downcast_ref::<crate::container::prod_orchestrator::InstallPrerequisiteError>().is_some() => {
return Err(super::dependencies::DependencyGateError(e.to_string()).into());
}
Err(e) if is_unknown_app_id_error(&e) => {
info!(
"Install {}: orchestrator has no manifest mapping yet, falling back to legacy installer",
@@ -1699,32 +1702,10 @@ autopilot.active=false\n",
patch_indeedhub_nostr_provider().await;
}
// Gitea: keep it on its native host port (3001). The UI opens Gitea
// in a new tab on that direct port so absolute asset URLs must be
// rooted at the host port rather than Archipelago's /app/gitea/ path.
if package_id == "gitea" {
let _ = tokio::fs::remove_file("/etc/nginx/conf.d/gitea-iframe.conf").await;
// Set ROOT_URL to the direct launch route so links/assets stay
// anchored under the same origin Gitea is launched from.
let host_ip = &self.config.host_ip;
let _ = tokio::process::Command::new("podman")
.args(["exec", "gitea", "sh", "-c",
&format!("grep -q ROOT_URL /data/gitea/conf/app.ini && sed -i 's|ROOT_URL.*|ROOT_URL = http://{}:3001/|' /data/gitea/conf/app.ini || true", host_ip)])
.output()
.await;
// Also ensure X_FRAME_OPTIONS is empty so Gitea doesn't send the header
let _ = tokio::process::Command::new("podman")
.args(["exec", "gitea", "sh", "-c",
"grep -q X_FRAME_OPTIONS /data/gitea/conf/app.ini && sed -i 's|X_FRAME_OPTIONS.*|X_FRAME_OPTIONS =|' /data/gitea/conf/app.ini || sed -i '/^\\[security\\]/a X_FRAME_OPTIONS =' /data/gitea/conf/app.ini"])
.output()
.await;
info!(
"Gitea: ROOT_URL set to http://{}:3001/, X_FRAME_OPTIONS cleared",
host_ip
);
}
// Gitea owns its public URL and security settings in app.ini, including
// values chosen in its first-run setup. Do not rewrite operator values
// or claim success from best-effort grep/sed commands. The app gate
// fronts its declared HTTP port and handles frame headers separately.
if package_id == "nextcloud" {
let host_ip = &self.config.host_ip;
@@ -2053,25 +2034,8 @@ fn parse_setup_token(lines: &[&str]) -> Option<String> {
}
async fn cleanup_stale_package_ports(package_id: &str) {
match package_id {
"grafana" => cleanup_stale_pasta_port("3000").await,
"homeassistant" | "home-assistant" => cleanup_stale_pasta_port("8123").await,
"searxng" => cleanup_stale_pasta_port("8888").await,
"uptime-kuma" => cleanup_stale_pasta_port("3002").await,
"gitea" => {
cleanup_stale_pasta_port("3001").await;
cleanup_stale_pasta_port("2222").await;
cleanup_stale_pasta_port("3000").await;
}
"nginx-proxy-manager" => {
cleanup_stale_pasta_port("8081").await;
cleanup_stale_pasta_port("8084").await;
cleanup_stale_pasta_port("8444").await;
}
"nextcloud" => cleanup_stale_pasta_port("8085").await,
"portainer" => cleanup_stale_pasta_port("9000").await,
_ => {}
}
// Never kill by port: another app or the management gate may own it.
crate::container::ghost_reaper::reap_for_app(package_id).await;
}
fn install_command_tail(
@@ -2196,93 +2160,11 @@ async fn cleanup_start_conflict(package_id: &str, stderr: &str) -> bool {
return true;
}
match package_id {
"grafana"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("3000").await;
true
}
"homeassistant" | "home-assistant"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("8123").await;
true
}
"searxng"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("8888").await;
true
}
"uptime-kuma"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("3002").await;
true
}
"gitea" if stderr.contains("pasta failed") || stderr.contains("address already in use") => {
cleanup_stale_pasta_port("3001").await;
cleanup_stale_pasta_port("2222").await;
cleanup_stale_pasta_port("3000").await;
true
}
"nginx-proxy-manager"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("8081").await;
cleanup_stale_pasta_port("8084").await;
cleanup_stale_pasta_port("8444").await;
true
}
"nextcloud"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("8085").await;
true
}
"portainer"
if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
{
cleanup_stale_pasta_port("9000").await;
true
}
_ => false,
if stderr.contains("pasta failed") || stderr.contains("address already in use") {
crate::container::ghost_reaper::reap_for_app(package_id).await;
return true;
}
}
async fn cleanup_stale_pasta_port(port: &str) {
// NEVER kill our own process. The daemon holds catalog app ports over
// IPv6 (the mesh app-port relay), so a blunt `fuser -k <port>/tcp` would
// terminate archipelago itself mid-install — installs failed and apps
// vanished on a test node 2026-07-27. Kill every listener on the port
// EXCEPT our PID (and our process group), leaving the relay/daemon alive.
let self_pid = std::process::id();
let kill_listener = format!(
"ss -ltnp 'sport = :{port}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | \
while read p; do [ \"$p\" = \"{self_pid}\" ] || kill \"$p\" 2>/dev/null; done || true",
);
let _ = tokio::process::Command::new("sh")
.args(["-c", &kill_listener])
.output()
.await;
// sudo fuser -k, but exclude our own PID: fuser prints the PIDs holding
// the port; kill each except self. (`fuser -k` has no exclusion flag.)
let fuser_kill = format!(
"for p in $(sudo fuser {port}/tcp 2>/dev/null); do [ \"$p\" = \"{self_pid}\" ] || sudo kill \"$p\" 2>/dev/null; done || true",
);
let _ = tokio::process::Command::new("sh")
.args(["-c", &fuser_kill])
.output()
.await;
let pattern = format!("pasta.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
false
}
async fn repair_nextcloud_permissions() {
@@ -14,20 +14,23 @@ impl RpcHandler {
/// the rare case where the pull stream actually parses, but podman
/// almost never emits parseable progress on a piped stderr.
pub(super) async fn set_install_progress(&self, package_id: &str, downloaded: u64, size: u64) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
entry.state = PackageState::Installing;
let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase: existing_phase,
message: None,
});
self.state_manager.update_data(data).await;
self.state_manager
.mutate_data(|data| {
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
entry.ui_ready = Some(false);
entry.state = PackageState::Installing;
let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase: existing_phase,
message: None,
});
})
.await;
}
/// Set the install pipeline phase and broadcast. This is the
@@ -35,76 +38,86 @@ impl RpcHandler {
/// percentage and a user-facing label. Byte counters are retained
/// for the rare case podman emits parseable progress.
pub(super) async fn set_install_phase(&self, package_id: &str, phase: InstallPhase) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
// Preparing / PullingImage / CreatingContainer / StartingContainer /
// WaitingHealthy / PostInstall all map to the Installing state.
// Updates use Updating state — the wrapper has already flipped
// state to Updating, so don't clobber it.
if entry.state != PackageState::Updating {
entry.state = PackageState::Installing;
}
let (size, downloaded) = entry
.install_progress
.as_ref()
.map(|p| (p.size, p.downloaded))
.unwrap_or((0, 0));
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase: Some(phase),
message: None,
});
self.state_manager.update_data(data).await;
self.state_manager
.mutate_data(|data| {
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
// Preparing / PullingImage / CreatingContainer / StartingContainer /
// WaitingHealthy / PostInstall all map to the Installing state.
// Updates use Updating state — the wrapper has already flipped
// state to Updating, so don't clobber it.
if entry.state != PackageState::Updating {
entry.ui_ready = Some(false);
entry.state = PackageState::Installing;
}
let (size, downloaded) = entry
.install_progress
.as_ref()
.map(|p| (p.size, p.downloaded))
.unwrap_or((0, 0));
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase: Some(phase),
message: None,
});
})
.await;
}
/// Set a user-facing install status message (e.g. "Waiting for Bitcoin
/// to start…") without disturbing the current phase/byte counters.
pub(super) async fn set_install_message(&self, package_id: &str, message: &str) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
if entry.state != PackageState::Updating {
entry.state = PackageState::Installing;
}
let (size, downloaded, phase) = entry
.install_progress
.as_ref()
.map(|p| (p.size, p.downloaded, p.phase))
.unwrap_or((0, 0, None));
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase,
message: Some(message.to_string()),
});
self.state_manager.update_data(data).await;
self.state_manager
.mutate_data(|data| {
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
if entry.state != PackageState::Updating {
entry.ui_ready = Some(false);
entry.state = PackageState::Installing;
}
let (size, downloaded, phase) = entry
.install_progress
.as_ref()
.map(|p| (p.size, p.downloaded, p.phase))
.unwrap_or((0, 0, None));
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase,
message: Some(message.to_string()),
});
})
.await;
}
/// Clear install progress after pull completes or fails.
pub(super) async fn clear_install_progress(&self, package_id: &str) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.install_progress = None;
}
self.state_manager.update_data(data).await;
self.state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.install_progress = None;
}
})
.await;
}
/// Set the uninstall stage label so the UI can show what's happening
/// instead of a generic spinner. Each call broadcasts a state change
/// — call sparingly (one per pipeline phase, not per container).
pub(super) async fn set_uninstall_stage(&self, package_id: &str, stage: &str) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.uninstall_stage = Some(stage.to_string());
entry.state = crate::data_model::PackageState::Removing;
}
self.state_manager.update_data(data).await;
self.state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.uninstall_stage = Some(stage.to_string());
entry.state = crate::data_model::PackageState::Removing;
}
})
.await;
}
/// Update install progress (static method for use in async closures).
@@ -114,25 +127,28 @@ impl RpcHandler {
downloaded: u64,
total: u64,
) {
let (mut data, _rev) = state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
entry.install_progress = Some(InstallProgress {
size: total,
downloaded,
phase: existing_phase,
message: None,
});
state_manager.update_data(data).await;
state_manager
.mutate_data(|data| {
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
entry.install_progress = Some(InstallProgress {
size: total,
downloaded,
phase: existing_phase,
message: None,
});
})
.await;
}
}
/// Create a minimal PackageDataEntry for a package being installed.
fn create_installing_entry(package_id: &str) -> PackageDataEntry {
PackageDataEntry {
ui_ready: None,
state: PackageState::Installing,
health: None,
exit_code: None,
+46 -64
View File
@@ -1431,10 +1431,9 @@ async fn repair_before_package_start(container_name: &str) {
// published port and the data-dir file locks, so the replacement either
// fails to bind (`address already in use`) or starts and dies on the
// lock — and `Restart=always` loops it there forever. Ordered before
// the port cleanup below: killing the owner is what actually frees the
// port, and the port sweep alone cannot tell a ghost from a live app.
// starting the replacement. A port sweep cannot distinguish a ghost
// from the dashboard gate or another live app and must never kill it.
crate::container::ghost_reaper::reap_for_app(container_name).await;
cleanup_runtime_host_ports(container_name).await;
}
async fn wait_before_package_start(container_name: &str) {
@@ -1579,7 +1578,6 @@ async fn repair_netbird_network() {
async fn repair_nginx_proxy_manager_container() {
repair_nginx_proxy_manager_dirs().await;
if !nginx_proxy_manager_has_legacy_admin_port().await {
cleanup_nginx_proxy_manager_ports().await;
return;
}
@@ -1588,7 +1586,7 @@ async fn repair_nginx_proxy_manager_container() {
)
.await;
let _ = podman_control(&["rm", "-f", "nginx-proxy-manager"]).await;
cleanup_nginx_proxy_manager_ports().await;
crate::container::ghost_reaper::reap_for_app("nginx-proxy-manager").await;
if let Err(err) = recreate_nginx_proxy_manager_container().await {
tracing::warn!(error = %err, "failed to recreate stale nginx-proxy-manager container");
}
@@ -1812,6 +1810,9 @@ fn manifest_host_ports(container_name: &str) -> Vec<u16> {
pub(super) fn manifest_apps_dirs() -> Vec<std::path::PathBuf> {
let mut dirs = Vec::new();
if let Some(root) = std::env::var_os("ARCHIPELAGO_APPS_DIR") {
dirs.push(root.into());
}
if let Ok(manifest_dir) = std::env::var("CARGO_MANIFEST_DIR") {
dirs.push(Path::new(&manifest_dir).join("../../apps"));
}
@@ -2032,51 +2033,10 @@ async fn cleanup_start_conflict(container_name: &str, stderr: &str) {
return;
}
let ports = runtime_host_ports(container_name);
if !ports.is_empty() {
cleanup_ports(&ports).await;
return;
}
}
async fn cleanup_runtime_host_ports(container_name: &str) {
let ports = runtime_host_ports(container_name);
if !ports.is_empty() {
cleanup_ports(&ports).await;
}
}
async fn cleanup_nginx_proxy_manager_ports() {
cleanup_ports(&[8081, 8084, 8444]).await;
}
async fn cleanup_ports(ports: &[u16]) {
for port in ports {
cleanup_stale_pasta_port(&port.to_string()).await;
}
}
async fn cleanup_stale_pasta_port(port: &str) {
let kill_listener = format!(
"ss -ltnp 'sport = :{}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | xargs -r kill 2>/dev/null || true",
port
);
let _ = tokio::process::Command::new("sh")
.args(["-c", &kill_listener])
.output()
.await;
let pattern = format!("pasta.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
let pattern = format!("rootlessport.*{}", port);
let _ = tokio::process::Command::new("pkill")
.args(["-f", &pattern])
.output()
.await;
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
// Only reap processes proven to belong to an absent container. The app
// gate shares the app's port on other addresses and lives in this daemon;
// killing port owners (or matching argv with pkill) kills the dashboard.
crate::container::ghost_reaper::reap_for_app(container_name).await;
}
pub(super) fn is_missing_companion_ok(name: &str, stderr: &str) -> bool {
@@ -2095,13 +2055,16 @@ async fn flip_package_state(
package_id: &str,
transitional: PackageState,
) -> Option<PackageState> {
let (mut data, _) = state_manager.get_snapshot().await;
let prev = data.package_data.get(package_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.state = transitional;
state_manager.update_data(data).await;
}
prev
state_manager
.mutate_data(|data| {
let prev = data.package_data.get(package_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(package_id) {
entry.ui_ready = Some(false);
entry.state = transitional;
}
prev
})
.await
}
/// Write the package entry's final state. No-op if the entry has since
@@ -2111,13 +2074,18 @@ async fn set_package_state(
package_id: &str,
new_state: PackageState,
) {
let (mut data, _) = state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(package_id) {
if entry.state != new_state {
entry.state = new_state;
state_manager.update_data(data).await;
}
}
state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(package_id) {
if entry.state != new_state {
if new_state != PackageState::Running {
entry.ui_ready = Some(false);
}
entry.state = new_state;
}
}
})
.await
}
pub(super) async fn reconcile_companions_for(package_id: &str) {
@@ -2185,6 +2153,20 @@ pub(super) fn orchestrator_uninstall_app_ids(package_id: &str) -> Vec<String> {
mod tests {
use super::*;
#[tokio::test]
async fn port_conflict_cleanup_preserves_live_host_listener() {
// The previous ss|kill sweep terminated the daemon's app gate on a
// restart. Keep a real listening socket owned by this test process.
let listener = tokio::net::TcpListener::bind("127.0.0.2:2342")
.await
.unwrap();
let addr = listener.local_addr().unwrap();
cleanup_start_conflict("photoprism", "address already in use").await;
let client = tokio::net::TcpStream::connect(addr).await.unwrap();
let _connection = listener.accept().await.unwrap();
drop(client);
}
#[test]
fn missing_container_classifier_covers_podman5_phrasings() {
// Regression (.228 gate 2026-07-08): podman 5.x `inspect` on a missing
+22 -14
View File
@@ -150,23 +150,31 @@ async fn flip_to_transitional(
app_id: &str,
transitional: PackageState,
) -> Option<PackageState> {
let (mut data, _) = state_manager.get_snapshot().await;
let prev = data.package_data.get(app_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(app_id) {
entry.state = transitional;
state_manager.update_data(data).await;
}
prev
state_manager
.mutate_data(|data| {
let prev = data.package_data.get(app_id).map(|e| e.state.clone());
if let Some(entry) = data.package_data.get_mut(app_id) {
entry.ui_ready = Some(false);
entry.state = transitional;
}
prev
})
.await
}
/// Set the entry's state to `new_state`. No-ops if the entry has since been
/// removed (e.g. uninstall ran concurrently).
async fn set_state(state_manager: &StateManager, app_id: &str, new_state: PackageState) {
let (mut data, _) = state_manager.get_snapshot().await;
if let Some(entry) = data.package_data.get_mut(app_id) {
if entry.state != new_state {
entry.state = new_state;
state_manager.update_data(data).await;
}
}
state_manager
.mutate_data(|data| {
if let Some(entry) = data.package_data.get_mut(app_id) {
if entry.state != new_state {
if new_state != PackageState::Running {
entry.ui_ready = Some(false);
}
entry.state = new_state;
}
}
})
.await
}
+3
View File
@@ -114,6 +114,9 @@ impl PortMap {
/// there.
fn apps_dirs() -> Vec<PathBuf> {
let mut dirs = Vec::new();
if let Some(root) = std::env::var_os("ARCHIPELAGO_APPS_DIR") {
dirs.push(root.into());
}
if let Ok(manifest_dir) = std::env::var("CARGO_MANIFEST_DIR") {
dirs.push(PathBuf::from(manifest_dir).join("../../apps"));
}
+48 -3
View File
@@ -144,6 +144,34 @@ pub fn shared_status() -> Arc<RwLock<GateStatus>> {
.clone()
}
static REFRESH_KICK: std::sync::LazyLock<tokio::sync::Notify> =
std::sync::LazyLock::new(tokio::sync::Notify::new);
static REFRESH_REV: std::sync::LazyLock<tokio::sync::watch::Sender<u64>> =
std::sync::LazyLock::new(|| tokio::sync::watch::channel(0).0);
/// Installation must not wait for the minute sweep before becoming reachable.
/// Wait for a completed sweep, bounded if shutdown/startup prevents one.
pub async fn refresh_now() {
let mut completed = REFRESH_REV.subscribe();
REFRESH_KICK.notify_one();
let _ = tokio::time::timeout(std::time::Duration::from_secs(3), completed.changed()).await;
}
pub fn port_claimed(status: &GateStatus, port: u16) -> bool {
let mut external = false;
let mut tor = false;
for (claimed_port, address) in &status.claimed {
if *claimed_port != port {
continue;
}
if let Ok(ip) = address.parse::<IpAddr>() {
tor |= ip == GATE_TOR_UPSTREAM;
external |= !ip.is_loopback();
}
}
external && tor
}
/// Run the gate. Returns only on shutdown.
pub async fn run(
gate: Arc<AppGate>,
@@ -162,11 +190,12 @@ pub async fn run(
loop {
tokio::select! {
_ = interval.tick() => {
sweep(&gate, &status, &mut held, &shutdown_rx).await;
}
_ = interval.tick() => {}
_ = REFRESH_KICK.notified() => {}
_ = shutdown_rx.changed() => return,
}
sweep(&gate, &status, &mut held, &shutdown_rx).await;
REFRESH_REV.send_modify(|revision| *revision = revision.wrapping_add(1));
}
}
@@ -461,3 +490,19 @@ mod tests {
assert!(!status.is_fully_enforced());
}
}
#[cfg(test)]
mod readiness_tests {
use super::*;
#[test]
fn readiness_requires_external_and_tor_claims_for_the_same_port() {
let mut status = GateStatus::default();
assert!(!port_claimed(&status, 3001));
status.claimed.push((3001, "127.0.0.2".into()));
assert!(!port_claimed(&status, 3001));
status.claimed.push((3002, "192.0.2.10".into()));
assert!(!port_claimed(&status, 3001));
status.claimed.push((3001, "192.0.2.10".into()));
assert!(port_claimed(&status, 3001));
}
}
+1
View File
@@ -322,6 +322,7 @@ async fn eval_rpc_handler() -> (Arc<RpcHandler>, tempfile::TempDir) {
fn installed_entry(app_id: &str) -> crate::data_model::PackageDataEntry {
use crate::data_model::{Description, Manifest, PackageDataEntry, PackageState, StaticFiles};
PackageDataEntry {
ui_ready: None,
state: PackageState::Running,
health: None,
exit_code: None,
+1
View File
@@ -1069,6 +1069,7 @@ mod tests {
Description, Manifest, PackageDataEntry, PackageState, StaticFiles,
};
PackageDataEntry {
ui_ready: None,
state: PackageState::Running,
health: None,
exit_code: None,
+18 -12
View File
@@ -138,6 +138,24 @@ const NGINX_FEDIMINT_NEW: &str = " sub_filter_types text/css application/
const NGINX_FEDIMINT_SNIPPET_ANCHOR: &str = "proxy_pass http://127.0.0.1:8175/;";
const NGINX_FEDIMINT_SNIPPET_INSERT: &str = "proxy_pass http://127.0.0.1:8175/;\n proxy_set_header Accept-Encoding \"\";\n sub_filter_types text/css application/javascript application/json;\n sub_filter_once off;\n sub_filter 'href=\"/' 'href=\"/app/fedimint/';\n sub_filter 'src=\"/' 'src=\"/app/fedimint/';\n sub_filter \"href='/\" \"href='/app/fedimint/\";\n sub_filter \"src='/\" \"src='/app/fedimint/\";\n sub_filter 'url(\"/' 'url(\"/app/fedimint/';\n sub_filter \"url('/\" \"url('/app/fedimint/\";\n sub_filter '</head>' '<script src=\"/nostr-provider.js\"></script></head>';";
/// Finish manifest promotion before constructing the orchestrator or starting
/// catalog refresh/reconciliation. Replacing the app tree in the background
/// could let a reload observe its temporary empty state and forget disk-only apps.
pub async fn ensure_runtime_assets_ready() {
match run_runtime_assets().await {
Ok(changed) if changed => info!("Runtime assets synchronized from OTA payload"),
Ok(_) => debug!("No OTA runtime payload to synchronize"),
Err(e) => warn!("Runtime asset bootstrap failed (non-fatal): {:#}", e),
}
match run_apps_dir_repair().await {
Ok(true) => {
info!("Populated /opt/archipelago/apps from installer copy at /etc/archipelago/apps")
}
Ok(false) => debug!("/opt/archipelago/apps already populated (or no installer copy)"),
Err(e) => warn!("Apps dir repair failed (non-fatal): {:#}", e),
}
}
/// Entry point called from main startup. Never returns an error to the caller —
/// failing to bootstrap host artifacts must not prevent the backend from serving.
pub async fn ensure_doctor_installed() {
@@ -146,11 +164,6 @@ pub async fn ensure_doctor_installed() {
Ok(false) => debug!("No stale Archipelago dev-mode service override found"),
Err(e) => warn!("Service override repair failed (non-fatal): {:#}", e),
}
match run_runtime_assets().await {
Ok(changed) if changed => info!("Runtime assets synchronized from OTA payload"),
Ok(_) => debug!("No OTA runtime payload to synchronize"),
Err(e) => warn!("Runtime asset bootstrap failed (non-fatal): {:#}", e),
}
match run().await {
Ok(changed) if changed => info!("Doctor artifacts synchronized with binary"),
Ok(_) => debug!("Doctor artifacts already in sync"),
@@ -168,13 +181,6 @@ pub async fn ensure_doctor_installed() {
Ok(false) => debug!("No stale bitcoin.conf found"),
Err(e) => warn!("Bitcoin RPC repair failed (non-fatal): {:#}", e),
}
match run_apps_dir_repair().await {
Ok(true) => {
info!("Populated /opt/archipelago/apps from installer copy at /etc/archipelago/apps")
}
Ok(false) => debug!("/opt/archipelago/apps already populated (or no installer copy)"),
Err(e) => warn!("Apps dir repair failed (non-fatal): {:#}", e),
}
match run_tor_helper_sync().await {
Ok(true) => info!("tor-helper.sh synchronized with binary"),
Ok(false) => debug!("tor-helper.sh already current"),
+52 -1
View File
@@ -102,6 +102,31 @@ pub struct AppCatalogEntry {
/// `docs/registry-manifest-design.md`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub manifest: Option<serde_json::Value>,
/// Backward-compatible catalog rollout: old daemons ignore these and keep
/// the base manifest. New daemons choose only variants they can safely apply.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub manifest_variants: Vec<CatalogManifestVariant>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CatalogManifestVariant {
pub requires: Vec<String>,
pub manifest: serde_json::Value,
}
fn selected_manifest(entry: AppCatalogEntry) -> Option<serde_json::Value> {
// Never let an unknown future requirement become an unsafe partial match.
for variant in entry.manifest_variants.into_iter().rev() {
if !variant.requires.is_empty()
&& variant
.requires
.iter()
.all(|capability| capability == "runtime-migration-backup-v1")
{
return Some(variant.manifest);
}
}
entry.manifest
}
/// One selectable version in an app's `versions[]` list. The catalog carries a
@@ -234,7 +259,7 @@ pub fn catalog_manifest_values() -> Vec<(String, serde_json::Value)> {
load_catalog()
.apps
.into_iter()
.filter_map(|(id, e)| e.manifest.map(|m| (id, m)))
.filter_map(|(id, e)| selected_manifest(e).map(|m| (id, m)))
.collect()
}
@@ -557,6 +582,32 @@ fn write_cache(data_dir: &Path, body: &str) -> anyhow::Result<bool> {
mod tests {
use super::*;
#[test]
fn catalog_migration_variant_is_compatible_with_old_and_future_daemons() {
let raw = serde_json::json!({
"version": "2.45.0", "manifest": {"app": {"id": "portainer", "container": {}}},
"manifest_variants": [{"requires": ["runtime-migration-backup-v1"],
"manifest": {"app": {"id": "portainer", "container": {"network": "slirp4netns"}, "backup_before_runtime_change": true}}}]
});
#[derive(Deserialize)]
struct OldEntry {
manifest: serde_json::Value,
}
let old: OldEntry = serde_json::from_value(raw.clone()).unwrap();
assert!(old.manifest["app"]["container"].get("network").is_none());
let current: AppCatalogEntry = serde_json::from_value(raw.clone()).unwrap();
let chosen = selected_manifest(current).unwrap();
assert_eq!(chosen["app"]["container"]["network"], "slirp4netns");
assert_eq!(chosen["app"]["backup_before_runtime_change"], true);
let mut future = raw;
future["manifest_variants"][0]["requires"]
.as_array_mut()
.unwrap()
.push(serde_json::json!("unknown-next-capability"));
let chosen = selected_manifest(serde_json::from_value(future).unwrap()).unwrap();
assert!(chosen["app"]["container"].get("network").is_none());
}
#[test]
fn parses_and_ignores_unknown_fields() {
let json = r#"{
+331 -32
View File
@@ -3,8 +3,9 @@
use anyhow::Result;
use archipelago_container::{
ContainerRuntime as ContainerRuntimeTrait, ContainerState, PodmanClient,
ContainerRuntime as ContainerRuntimeTrait, ContainerState, ContainerStatus, PodmanClient,
};
use futures_util::StreamExt;
use std::collections::HashMap;
use std::sync::Arc;
use tracing::{debug, info};
@@ -15,6 +16,16 @@ use crate::data_model::{
PackageDataEntry, PackageState, ServiceStatus, StaticFiles,
};
/// One displayed package for each known container/manifest alias. Keep the
/// stopped-app restoration path in agreement with live-container discovery.
fn canonical_package_id(name: &str) -> &str {
match name.strip_prefix("archy-").unwrap_or(name) {
"immich_server" => "immich",
"mempool-web" | "mempool-frontend" => "mempool",
name => name,
}
}
pub struct DockerPackageScanner {
runtime: Arc<dyn ContainerRuntimeTrait>,
}
@@ -25,8 +36,15 @@ impl DockerPackageScanner {
}
/// Scan Docker containers and convert to package data
pub async fn scan_containers(&self) -> Result<HashMap<String, PackageDataEntry>> {
let containers = self.runtime.list_containers().await?;
pub async fn scan_containers(
&self,
data_dir: &std::path::Path,
cached: &HashMap<String, PackageDataEntry>,
) -> Result<HashMap<String, PackageDataEntry>> {
let mut containers = self.runtime.list_containers().await?;
let installed = crate::crash_recovery::load_installed_apps(data_dir).await;
let uninstalled = crate::crash_recovery::load_user_uninstalled(data_dir).await;
restore_absent_installed(&mut containers, &installed, &uninstalled);
debug!("Found {} containers", containers.len());
@@ -91,24 +109,8 @@ impl DockerPackageScanner {
debug!("Found {} UI containers", ui_containers.len());
for container in containers {
// Extract app ID from container name
// Support both archy-* containers (docker-compose) and plain names (manual)
let app_id = if container.name.starts_with("archy-") {
container
.name
.strip_prefix("archy-")
.unwrap_or(&container.name)
.to_string()
} else {
// Use the container name as-is for manually started containers
container.name.clone()
};
// Normalize multi-container app IDs to their canonical names
let app_id = match app_id.as_str() {
"immich_server" => "immich".to_string(),
_ => app_id,
};
// Use the same alias mapping as stopped-app restoration.
let app_id = canonical_package_id(&container.name).to_owned();
// Skip backend services (databases, APIs, etc.)
if excluded_services.contains(&app_id.as_str()) {
@@ -139,6 +141,18 @@ impl DockerPackageScanner {
continue;
}
if container.id.is_empty() {
if let Some(previous) = cached.get(&app_id) {
let mut held = previous.clone();
held.state = PackageState::Stopped;
held.ui_ready = Some(false);
held.health = None;
held.exit_code = None;
packages.insert(app_id.clone(), held);
continue;
}
}
// Get metadata for this app
let metadata = get_app_metadata(&app_id);
// Manifest-owned metadata (icon) wins over the static table: the
@@ -158,13 +172,11 @@ impl DockerPackageScanner {
} else {
// Prefer the known web UI port over arbitrary first binding
// (for example Gitea exposes SSH on 2222 before web on 3001).
let candidate = if uses_allocated_launch_port(&app_id) {
extract_lan_address(&container.ports)
.or_else(|| PodmanClient::lan_address_for(&app_id))
} else {
PodmanClient::lan_address_for(&app_id)
.or_else(|| extract_lan_address(&container.ports))
};
let candidate = package_launch_candidate(
&app_id,
&container.ports,
PodmanClient::lan_address_for(&app_id),
);
reachable_lan_address(&app_id, candidate).await
};
@@ -179,14 +191,22 @@ impl DockerPackageScanner {
let tor_address = read_tor_address(&app_id).await;
// Extract actual version from container image tag
let running_version = image_versions::extract_version_from_image(&container.image);
let running_version = if container.id.is_empty() {
String::new() // Absence cannot establish the installed image version.
} else {
image_versions::extract_version_from_image(&container.image)
};
// Decoupled from the binary OTA: prefer the remote app catalog,
// falling back to the image-versions.sh pin when uncovered/offline.
let available_update =
crate::container::app_catalog::available_update_for_app(&app_id, &container.image);
let available_update = if container.id.is_empty() {
None
} else {
crate::container::app_catalog::available_update_for_app(&app_id, &container.image)
};
let package = PackageDataEntry {
ui_ready: Some(false),
state: package_state.clone(),
health: container.health.clone(),
exit_code: if package_state == PackageState::Exited {
@@ -283,10 +303,237 @@ impl DockerPackageScanner {
);
}
let probes: Vec<_> = packages
.iter()
.filter_map(|(id, pkg)| {
if pkg.state != PackageState::Running {
return None;
}
let url = pkg
.installed
.as_ref()?
.interface_addresses
.get("main")?
.lan_address
.clone()?;
Some((id.clone(), url))
})
.collect();
let mut results = futures_util::stream::iter(
probes
.into_iter()
.map(|(id, url)| async move { (id, launch_http_ready(&url).await) }),
)
.buffer_unordered(8);
while let Some((id, ready)) = results.next().await {
if let Some(pkg) = packages.get_mut(&id) {
pkg.ui_ready = Some(ready);
}
}
// HTTP on loopback can precede the LAN/Tor listener after install.
let port_map = crate::appgate::identity::build_port_map();
let gated: Vec<_> = packages
.iter()
.filter_map(|(id, pkg)| {
if pkg.ui_ready != Some(true) {
return None;
}
let url = pkg
.installed
.as_ref()?
.interface_addresses
.get("main")?
.lan_address
.as_deref()?;
let port = launch_url_port(url)?;
port_map
.gated(port)
.filter(|gate| gate.declared)
.map(|_| (id.clone(), port))
})
.collect();
if !gated.is_empty() {
use crate::appgate::listener::{port_claimed, refresh_now, shared_status};
let status = shared_status();
let needs_refresh = {
let current = status.read().await;
gated.iter().any(|(_, port)| !port_claimed(&current, *port))
};
if needs_refresh {
refresh_now().await;
}
let current = status.read().await;
for (id, port) in gated {
if !port_claimed(&current, port) {
packages.get_mut(&id).unwrap().ui_ready = Some(false);
}
}
}
Ok(packages)
}
}
/// Quadlet removes containers during ordinary stops/restarts. Rebuild installed
/// entries even on the daemon's first scan; a runtime absence is not uninstall.
fn restore_absent_installed(
containers: &mut Vec<ContainerStatus>,
installed: &std::collections::HashSet<String>,
uninstalled: &std::collections::HashSet<String>,
) {
let mut present: std::collections::HashSet<String> = containers
.iter()
.map(|c| canonical_package_id(&c.name).to_owned())
.collect();
let removed: std::collections::HashSet<_> = uninstalled
.iter()
.map(|id| canonical_package_id(id))
.collect();
for name in installed {
let id = canonical_package_id(name);
if removed.contains(id) || !present.insert(id.to_owned()) {
continue;
}
containers.push(ContainerStatus {
id: String::new(),
name: id.to_owned(),
state: ContainerState::Stopped,
health: None,
exit_code: None,
started_at: None,
image: String::new(),
created: String::new(),
ports: Vec::new(),
lan_address: None,
});
}
}
/// Probe the actual loopback upstream, not the app gate's login page. A bound
/// TCP socket alone can still reset requests or serve a startup 503.
async fn launch_http_ready(candidate: &str) -> bool {
let Ok(mut url) = reqwest::Url::parse(candidate) else {
return false;
};
if !matches!(url.scheme(), "http" | "https") {
return false;
}
if url.set_host(Some("127.0.0.1")).is_err() {
return false;
}
static CLIENT: std::sync::OnceLock<reqwest::Client> = std::sync::OnceLock::new();
let client = CLIENT.get_or_init(|| {
reqwest::Client::builder()
.no_proxy()
.timeout(std::time::Duration::from_secs(2))
.redirect(reqwest::redirect::Policy::none())
// Self-signed local app certificates are normal. This client only
// contacts loopback and never sends credentials or follows redirects.
.danger_accept_invalid_certs(true)
.build()
.expect("local readiness client")
});
match client.get(url).send().await {
Ok(response) => matches!(response.status().as_u16(), 200..=399 | 401 | 403),
Err(_) => false,
}
}
#[cfg(test)]
mod lifecycle_regression_tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
#[test]
fn registry_survives_empty_runtime_and_deduplicates_aliases() {
let installed = ["archy-gitea", "gitea", "immich_server", "archy-removed"]
.into_iter()
.map(str::to_owned)
.collect();
let removed = ["removed".to_owned()].into_iter().collect();
let mut containers = Vec::new();
restore_absent_installed(&mut containers, &installed, &removed);
assert_eq!(containers.len(), 2);
assert!(containers
.iter()
.all(|c| c.state == ContainerState::Stopped));
containers[0].state = ContainerState::Running;
restore_absent_installed(&mut containers, &installed, &removed);
assert_eq!(containers.len(), 2);
assert_eq!(containers[0].state, ContainerState::Running);
}
#[test]
fn mempool_frontend_inventory_alias_does_not_create_a_second_package() {
let installed = ["mempool", "archy-mempool-web", "mempool-web"]
.into_iter()
.map(str::to_owned)
.collect();
let mut containers = Vec::new();
restore_absent_installed(&mut containers, &installed, &Default::default());
assert_eq!(containers.len(), 1);
assert_eq!(containers[0].name, "mempool");
containers[0].id = "live-frontend".into();
containers[0].state = ContainerState::Running;
restore_absent_installed(&mut containers, &installed, &Default::default());
assert_eq!(containers.len(), 1);
assert_eq!(containers[0].id, "live-frontend");
assert_eq!(containers[0].state, ContainerState::Running);
assert_eq!(canonical_package_id("archy-mempool-web"), "mempool");
assert_eq!(canonical_package_id("mempool-api"), "mempool-api");
containers.clear();
restore_absent_installed(
&mut containers,
&installed,
&["mempool".into()].into_iter().collect(),
);
assert!(containers.is_empty());
}
#[tokio::test]
async fn readiness_rejects_startup_errors_and_accepts_auth_and_redirects() {
for (status, expected) in [
(200, true),
(302, true),
(401, true),
(403, true),
(404, false),
(500, false),
(502, false),
(503, false),
] {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let (mut stream, _) = listener.accept().await.unwrap();
let mut buf = [0; 2048];
let n = stream.read(&mut buf).await.unwrap();
assert!(String::from_utf8_lossy(&buf[..n]).starts_with("GET /start HTTP/1.1"));
stream.write_all(format!("HTTP/1.1 {status} Test\r\nContent-Length: 0\r\nConnection: close\r\n\r\n").as_bytes()).await.unwrap();
});
assert_eq!(
launch_http_ready(&format!("http://localhost:{port}/start")).await,
expected,
"status {status}"
);
task.await.unwrap();
}
}
#[tokio::test]
async fn readiness_rejects_tcp_accept_without_http() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
drop(stream);
});
assert!(!launch_http_ready(&format!("http://localhost:{port}/")).await);
task.await.unwrap();
assert!(!launch_http_ready(&format!("http://localhost:{port}/")).await);
assert!(!launch_http_ready("file:///tmp/test").await);
}
}
struct AppMetadata {
title: String,
description: String,
@@ -856,6 +1103,23 @@ fn companion_lan_address(app_id: &str) -> Option<String> {
}
}
/// Companion dashboards remain usable while their backend is syncing. Never
/// probe a Bitcoin RPC or Electrum protocol socket as dashboard readiness.
fn package_launch_candidate(
app_id: &str,
ports: &[String],
known: Option<String>,
) -> Option<String> {
if let Some(companion) = companion_lan_address(app_id) {
return Some(companion);
}
if uses_allocated_launch_port(app_id) {
extract_lan_address(ports).or(known)
} else {
known.or_else(|| extract_lan_address(ports))
}
}
fn uses_allocated_launch_port(app_id: &str) -> bool {
matches!(
app_id,
@@ -940,7 +1204,42 @@ mod tor_service_name_tests {
#[cfg(test)]
mod extract_lan_address_tests {
use super::extract_lan_address;
use super::{extract_lan_address, package_launch_candidate};
#[test]
fn companion_dashboard_wins_over_backend_protocol_ports() {
for id in ["bitcoin", "bitcoin-core", "bitcoin-knots"] {
assert_eq!(
package_launch_candidate(
id,
&["127.0.0.1:8332->8332/tcp".into()],
Some("http://localhost:8332".into())
)
.as_deref(),
Some("http://localhost:8334")
);
}
for id in ["electrumx", "electrs", "mempool-electrs"] {
assert_eq!(
package_launch_candidate(
id,
&["127.0.0.1:50001->50001/tcp".into()],
Some("http://localhost:50001".into())
)
.as_deref(),
Some("http://localhost:50002")
);
}
assert_eq!(
package_launch_candidate(
"filebrowser",
&["127.0.0.1:19080->80/tcp".into()],
Some("http://localhost:8080".into())
)
.as_deref(),
Some("http://localhost:19080")
);
}
#[test]
fn skips_ssh_port_when_web_port_is_published() {
+420 -1
View File
@@ -5,7 +5,7 @@
//! starting the container with `--config /data/.filebrowser.json`.
use anyhow::{Context, Result};
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use tokio::fs;
use crate::update::host_sudo;
@@ -117,6 +117,197 @@ fn shell_quote(s: &str) -> String {
s.replace('\'', "'\\''")
}
/// Save a complete purchase without overwriting any existing directory entry.
/// Both host and rootless-namespace paths publish with a no-clobber hard link.
pub async fn save_new_file(dir: &Path, name: &str, bytes: &[u8]) -> Result<PathBuf> {
save_new_file_with(dir, name, bytes, write_via_userns).await
}
fn validate_filename(name: &str) -> Result<()> {
anyhow::ensure!(
!name.is_empty()
&& name != "."
&& name != ".."
&& !name.contains(['/', '\\', '\0'])
&& name.len() <= 255,
"Invalid purchased filename"
);
Ok(())
}
async fn save_new_file_with<F, Fut>(
dir: &Path,
name: &str,
bytes: &[u8],
fallback: F,
) -> Result<PathBuf>
where
F: FnOnce(PathBuf, String, Vec<u8>) -> Fut,
Fut: std::future::Future<Output = Result<PathBuf>>,
{
validate_filename(name)?;
// Never follow a user-created destination directory symlink.
match fs::symlink_metadata(dir).await {
Ok(meta) => anyhow::ensure!(meta.is_dir(), "Files destination is not a directory"),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
Err(error) => return Err(error.into()),
}
save_after_direct_result(
write_direct(dir, name, bytes).await,
dir,
name,
bytes,
fallback,
)
.await
}
async fn save_after_direct_result<F, Fut>(
result: std::io::Result<PathBuf>,
dir: &Path,
name: &str,
bytes: &[u8],
fallback: F,
) -> Result<PathBuf>
where
F: FnOnce(PathBuf, String, Vec<u8>) -> Fut,
Fut: std::future::Future<Output = Result<PathBuf>>,
{
match result {
Ok(path) => Ok(path),
Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
fallback(dir.to_owned(), name.to_owned(), bytes.to_vec())
.await
.context("Saving purchase in Files user namespace")
}
Err(error) => Err(error).context("Saving purchase in Files"),
}
}
fn numbered_name(name: &str, attempt: usize) -> String {
if attempt == 1 {
return name.to_owned();
}
match name.rsplit_once('.') {
Some((stem, extension)) if !stem.is_empty() => format!("{stem} ({attempt}).{extension}"),
_ => format!("{name} ({attempt})"),
}
}
struct PendingFile(PathBuf);
impl Drop for PendingFile {
fn drop(&mut self) {
let _ = std::fs::remove_file(&self.0);
}
}
async fn write_direct(dir: &Path, name: &str, bytes: &[u8]) -> std::io::Result<PathBuf> {
use std::os::unix::fs::PermissionsExt;
use tokio::io::AsyncWriteExt;
fs::create_dir_all(dir).await?;
let temp_path = dir.join(format!(".archy-saving-{}", uuid::Uuid::new_v4()));
let mut file = fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&temp_path)
.await?;
let temp = PendingFile(temp_path);
file.write_all(bytes).await?;
file.set_permissions(std::fs::Permissions::from_mode(0o644))
.await?;
file.sync_all().await?;
for attempt in 1..=100 {
let target = dir.join(numbered_name(name, attempt));
match fs::hard_link(&temp.0, &target).await {
Ok(()) => return Ok(target),
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
Err(error) => return Err(error),
}
}
Err(std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"Too many existing copies; purchase cache retained",
))
}
// Positional arguments carry all user-controlled text. mktemp prevents temp-name
// collisions; ln -T refuses files, symlinks and directories, including races.
const WRITE_VIA_USERNS: &str = r#"set -eu
dir=$1
name=$2
expected=$3
[ ! -L "$dir" ] || exit 1
if [ ! -d "$dir" ]; then
mkdir -p -- "$dir"
chown --reference="$(dirname -- "$dir")" -- "$dir"
fi
tmp=$(mktemp "$dir/.archy-saving.XXXXXXXXXX")
trap 'rm -f -- "$tmp"' EXIT HUP INT TERM
cat > "$tmp"
[ "$(wc -c < "$tmp")" -eq "$expected" ] || exit 1
chown --reference="$dir" -- "$tmp"
chmod 0644 -- "$tmp"
sync -f -- "$tmp"
stem=$name
ext=
case "$name" in
*.*) prefix=${name%.*}; if [ -n "$prefix" ]; then stem=$prefix; ext=.${name##*.}; fi ;;
esac
n=1
while [ "$n" -le 100 ]; do
candidate=$name
if [ "$n" -gt 1 ]; then candidate="$stem ($n)$ext"; fi
dst="$dir/$candidate"
if ln -T -- "$tmp" "$dst" 2>/dev/null; then
printf '%s' "$candidate"
exit 0
fi
# A conflict may be a dangling symlink; never follow it or overwrite it.
if [ ! -e "$dst" ] && [ ! -L "$dst" ]; then exit 1; fi
n=$((n + 1))
done
exit 1
"#;
async fn write_via_userns(dir: PathBuf, name: String, bytes: Vec<u8>) -> Result<PathBuf> {
use tokio::io::AsyncWriteExt;
let mut child = tokio::process::Command::new("podman")
.args(["unshare", "sh", "-c", WRITE_VIA_USERNS, "sh"])
.arg(&dir)
.arg(&name)
.arg(bytes.len().to_string())
.kill_on_drop(true)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.context("Starting Files namespace writer")?;
let mut stdin = child.stdin.take().context("Files writer stdin missing")?;
let operation = async {
let fed = stdin.write_all(&bytes).await;
drop(stdin);
let output = child.wait_with_output().await?;
anyhow::ensure!(
output.status.success(),
"Files namespace writer failed: {}",
output.status
);
fed.context("Sending purchase bytes to Files")?;
let chosen =
String::from_utf8(output.stdout).context("Files writer returned an invalid name")?;
validate_filename(&chosen)?;
anyhow::ensure!(
(1..=100).any(|n| numbered_name(&name, n) == chosen),
"Files writer returned an unexpected name"
);
Ok(dir.join(chosen))
};
tokio::time::timeout(std::time::Duration::from_secs(120), operation)
.await
.context("Files namespace writer timed out")?
}
#[cfg(test)]
mod tests {
use super::*;
@@ -152,3 +343,231 @@ mod tests {
assert_eq!(second, EnsureOutcome::Unchanged);
}
}
#[cfg(test)]
mod purchase_write_tests {
use super::*;
use std::{
collections::HashSet,
os::unix::fs::{symlink, PermissionsExt},
};
fn no_temps(dir: &Path) {
assert!(std::fs::read_dir(dir).unwrap().all(|e| !e
.unwrap()
.file_name()
.to_string_lossy()
.starts_with(".archy-saving")));
}
#[tokio::test]
async fn direct_write_uses_complete_bytes_and_preserves_originals() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("song.mp3"), b"original")
.await
.unwrap();
let target = save_new_file(dir.path(), "song.mp3", b"new").await.unwrap();
assert_eq!(target.file_name().unwrap(), "song (2).mp3");
assert_eq!(fs::read(target).await.unwrap(), b"new");
assert_eq!(
fs::read(dir.path().join("song.mp3")).await.unwrap(),
b"original"
);
no_temps(dir.path());
}
#[tokio::test]
async fn simultaneous_saves_publish_unique_complete_files() {
let dir = tempfile::tempdir().unwrap();
let mut tasks = Vec::new();
for n in 0..24u8 {
let dir = dir.path().to_owned();
tasks.push(tokio::spawn(async move {
let bytes = vec![n; 32768];
let path = save_new_file(&dir, "same.bin", &bytes).await.unwrap();
assert_eq!(fs::read(&path).await.unwrap(), bytes);
path
}));
}
let mut paths = HashSet::new();
for task in tasks {
assert!(paths.insert(task.await.unwrap()));
}
assert_eq!(paths.len(), 24);
no_temps(dir.path());
}
#[tokio::test]
async fn existing_directories_and_dangling_symlinks_are_conflicts() {
let dir = tempfile::tempdir().unwrap();
fs::create_dir(dir.path().join("name")).await.unwrap();
symlink("missing", dir.path().join("name (2)")).unwrap();
let path = save_new_file(dir.path(), "name", b"new").await.unwrap();
assert_eq!(path.file_name().unwrap(), "name (3)");
assert!(dir.path().join("name").is_dir());
assert!(fs::symlink_metadata(dir.path().join("name (2)"))
.await
.unwrap()
.is_symlink());
no_temps(dir.path());
}
#[tokio::test]
async fn invalid_names_and_symlink_destination_are_refused() {
let dir = tempfile::tempdir().unwrap();
for name in [
"",
".",
"..",
"../escape",
"/absolute",
"a/b",
"a\\b",
"a\0b",
] {
assert!(save_new_file(dir.path(), name, b"bytes").await.is_err());
}
let outside = tempfile::tempdir().unwrap();
symlink(outside.path(), dir.path().join("Music")).unwrap();
assert!(save_new_file(&dir.path().join("Music"), "song", b"bytes")
.await
.is_err());
assert_eq!(std::fs::read_dir(outside.path()).unwrap().count(), 0);
}
#[tokio::test]
async fn collision_limit_preserves_all_files_and_cleans_temporary_data() {
let dir = tempfile::tempdir().unwrap();
for n in 1..=100 {
fs::write(dir.path().join(numbered_name("a.txt", n)), b"keep")
.await
.unwrap();
}
assert!(save_new_file(dir.path(), "a.txt", b"new").await.is_err());
for n in 1..=100 {
assert_eq!(
fs::read(dir.path().join(numbered_name("a.txt", n)))
.await
.unwrap(),
b"keep"
);
}
no_temps(dir.path());
}
#[tokio::test]
async fn permission_fallback_is_exercised_without_skipping_as_root() {
let dir = tempfile::tempdir().unwrap();
let result = save_after_direct_result(
Err(std::io::ErrorKind::PermissionDenied.into()),
dir.path(),
"a",
b"abc",
|dir, name, bytes| async move {
assert_eq!(bytes, b"abc");
Ok(dir.join(name))
},
)
.await
.unwrap();
assert_eq!(result, dir.path().join("a"));
assert!(save_after_direct_result(
Err(std::io::ErrorKind::PermissionDenied.into()),
dir.path(),
"a",
b"abc",
|_, _, _| async { anyhow::bail!("namespace unavailable") }
)
.await
.unwrap_err()
.to_string()
.contains("namespace"));
assert!(save_after_direct_result(
Err(std::io::ErrorKind::StorageFull.into()),
dir.path(),
"a",
b"abc",
|_, _, _| async { panic!("disk full must not trigger permission fallback") }
)
.await
.is_err());
}
async fn run_script(
dir: &Path,
name: &str,
bytes: &[u8],
expected: usize,
) -> std::process::Output {
use tokio::io::AsyncWriteExt;
let mut child = tokio::process::Command::new("sh")
.args(["-c", WRITE_VIA_USERNS, "sh"])
.arg(dir)
.arg(name)
.arg(expected.to_string())
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.unwrap();
let mut input = child.stdin.take().unwrap();
input.write_all(bytes).await.unwrap();
drop(input);
child.wait_with_output().await.unwrap()
}
#[tokio::test]
async fn namespace_script_preserves_names_bytes_modes_and_existing_entries() {
let dir = tempfile::tempdir().unwrap();
let folder = dir.path().join("Music");
let name = "song ' $() ; #.mp3";
for n in 1..=2 {
let output = run_script(&folder, name, b"abc", 3).await;
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let chosen = String::from_utf8(output.stdout).unwrap();
assert_eq!(chosen, numbered_name(name, n));
let path = folder.join(chosen);
assert_eq!(fs::read(&path).await.unwrap(), b"abc");
assert_eq!(
fs::metadata(path).await.unwrap().permissions().mode() & 0o777,
0o644
);
}
no_temps(&folder);
}
#[tokio::test]
async fn namespace_script_refuses_truncated_input_and_cleans_up() {
let dir = tempfile::tempdir().unwrap();
let output = run_script(dir.path(), "never.bin", b"partial", 100).await;
assert!(!output.status.success());
assert!(!dir.path().join("never.bin").exists());
no_temps(dir.path());
}
#[tokio::test]
async fn namespace_script_does_not_link_inside_existing_directory() {
let dir = tempfile::tempdir().unwrap();
fs::create_dir(dir.path().join("name")).await.unwrap();
symlink("missing", dir.path().join("name (2)")).unwrap();
let output = run_script(dir.path(), "name", b"abc", 3).await;
assert!(output.status.success());
assert_eq!(output.stdout, b"name (3)");
assert_eq!(
std::fs::read_dir(dir.path().join("name")).unwrap().count(),
0
);
no_temps(dir.path());
}
#[test]
fn names_keep_extensions_and_dotfiles() {
assert_eq!(numbered_name("a.tar.gz", 2), "a.tar (2).gz");
assert_eq!(numbered_name(".hidden", 2), ".hidden (2)");
assert_eq!(numbered_name("README", 2), "README (2)");
}
}
@@ -0,0 +1,254 @@
//! Consistent, private snapshots for declaratively opted-in runtime migrations.
use anyhow::{bail, Context, Result};
use archipelago_container::AppManifest;
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
pub fn enabled(manifest: &AppManifest) -> Result<bool> {
match manifest.app.extensions.get("backup_before_runtime_change") {
None => Ok(false),
Some(value) => value
.as_bool()
.context("backup_before_runtime_change must be boolean"),
}
}
fn relative_sources(manifest: &AppManifest, data_dir: &Path) -> Result<Vec<PathBuf>> {
let mut sources = Vec::new();
for volume in &manifest.app.volumes {
if volume.options.iter().any(|v| v == "ro") || volume.volume_type == "tmpfs" {
continue;
}
// A runtime socket is a connection, not application state.
if volume.source == "/run/user/1000/podman/podman.sock" {
continue;
}
if volume.volume_type != "bind" {
bail!("runtime migration backup requires bind-mounted persistent state");
}
let path = Path::new(&volume.source);
let relative = path
.strip_prefix(data_dir)
.context("runtime migration state must be inside the node data directory")?;
if relative.starts_with("migration-backups") {
bail!("migration backup cannot include its own archive directory");
}
if relative.as_os_str().is_empty()
|| relative
.components()
.any(|c| !matches!(c, std::path::Component::Normal(_)))
{
bail!("invalid runtime migration state path");
}
sources.push(relative.to_path_buf());
}
sources.sort();
sources.dedup();
let mut roots: Vec<PathBuf> = Vec::new();
for source in sources {
if !roots.iter().any(|root| source.starts_with(root)) {
roots.push(source);
}
}
if roots.is_empty() {
bail!("runtime migration backup has no persistent state mounts");
}
Ok(roots)
}
/// Caller must gracefully stop the app before this function, and resume the old
/// service if it fails. No source files are changed or deleted by this operation.
pub async fn snapshot(
manifest: &AppManifest,
data_dir: &Path,
previous_unit: Option<&[u8]>,
) -> Result<PathBuf> {
let mut command = tokio::process::Command::new("podman");
command.args(["unshare", "tar"]);
snapshot_with_command(manifest, data_dir, previous_unit, command).await
}
async fn snapshot_with_command(
manifest: &AppManifest,
data_dir: &Path,
previous_unit: Option<&[u8]>,
mut command: tokio::process::Command,
) -> Result<PathBuf> {
let sources = relative_sources(manifest, data_dir)?;
let canonical_root = tokio::fs::canonicalize(data_dir).await?;
for source in &sources {
let path = data_dir.join(source);
if tokio::fs::symlink_metadata(&path)
.await?
.file_type()
.is_symlink()
{
bail!("runtime migration state mount is a symlink; explicit backup required");
}
let canonical = tokio::fs::canonicalize(&path).await?;
if !canonical.starts_with(&canonical_root) {
bail!("runtime migration state path resolves outside node data directory");
}
}
let root = data_dir.join("migration-backups");
tokio::fs::create_dir_all(&root).await?;
tokio::fs::set_permissions(&root, std::fs::Permissions::from_mode(0o700)).await?;
let dir = root.join(uuid::Uuid::new_v4().to_string());
tokio::fs::create_dir(&dir).await?;
tokio::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o700)).await?;
if let Some(unit) = previous_unit {
let path = dir.join("previous.container");
tokio::fs::write(&path, unit).await?;
tokio::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).await?;
tokio::fs::File::open(&path).await?.sync_all().await?;
}
let partial = dir.join("state.tar.partial");
let archive = dir.join("state.tar");
let output = command
.args([
"--create",
"--numeric-owner",
"--acls",
"--xattrs",
"--file",
])
.arg(&partial)
.arg("--directory")
.arg(data_dir)
.arg("--")
.args(&sources)
.output()
.await
.context("start rootless migration snapshot")?;
if !output.status.success() {
// No tar stderr in public logs: it can contain private filenames.
let _ = tokio::fs::remove_file(&partial).await;
bail!("persistent-state snapshot failed; original state was left intact");
}
tokio::fs::set_permissions(&partial, std::fs::Permissions::from_mode(0o600)).await?;
tokio::fs::File::open(&partial).await?.sync_all().await?;
tokio::fs::rename(&partial, &archive).await?;
let metadata = serde_json::json!({"app": manifest.app.id, "version": manifest.app.version,
"network": manifest.app.container.network, "capabilities": manifest.app.security.capabilities, "sources": sources});
tokio::fs::write(
dir.join("metadata.json"),
serde_json::to_vec_pretty(&metadata)?,
)
.await?;
tokio::fs::File::open(&dir).await?.sync_all().await?;
Ok(archive)
}
#[cfg(test)]
mod tests {
use super::*;
fn portainer() -> AppManifest {
AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml")).unwrap()
}
#[tokio::test]
async fn stopped_state_archive_round_trips_database_compose_and_old_unit() {
let dir = tempfile::tempdir().unwrap();
let state = dir.path().join("portainer");
tokio::fs::create_dir_all(state.join("compose"))
.await
.unwrap();
tokio::fs::write(state.join("portainer.db"), b"fixture database")
.await
.unwrap();
tokio::fs::write(state.join("compose/stack.yml"), b"services: {}\n")
.await
.unwrap();
let mut m = portainer();
m.app.volumes[0].source = state.display().to_string();
m.app.volumes[1].source = state.join("compose").display().to_string();
let archive = snapshot_with_command(
&m,
dir.path(),
Some(b"old unit"),
tokio::process::Command::new("tar"),
)
.await
.unwrap();
assert_eq!(
std::fs::metadata(&archive).unwrap().permissions().mode() & 0o777,
0o600
);
assert_eq!(
tokio::fs::read(archive.parent().unwrap().join("previous.container"))
.await
.unwrap(),
b"old unit"
);
let restored = tempfile::tempdir().unwrap();
assert!(tokio::process::Command::new("tar")
.arg("-xf")
.arg(archive)
.arg("-C")
.arg(restored.path())
.status()
.await
.unwrap()
.success());
assert_eq!(
tokio::fs::read(restored.path().join("portainer/portainer.db"))
.await
.unwrap(),
b"fixture database"
);
assert_eq!(
tokio::fs::read(restored.path().join("portainer/compose/stack.yml"))
.await
.unwrap(),
b"services: {}\n"
);
assert_eq!(
tokio::fs::read(state.join("portainer.db")).await.unwrap(),
b"fixture database"
);
}
#[tokio::test]
async fn failed_snapshot_never_publishes_archive_or_changes_original_state() {
let dir = tempfile::tempdir().unwrap();
let state = dir.path().join("portainer");
tokio::fs::create_dir_all(state.join("compose"))
.await
.unwrap();
tokio::fs::write(state.join("portainer.db"), b"unchanged")
.await
.unwrap();
let mut m = portainer();
m.app.volumes[0].source = state.display().to_string();
m.app.volumes[1].source = state.join("compose").display().to_string();
assert!(
snapshot_with_command(&m, dir.path(), None, tokio::process::Command::new("false"))
.await
.is_err()
);
assert_eq!(
tokio::fs::read(state.join("portainer.db")).await.unwrap(),
b"unchanged"
);
for entry in std::fs::read_dir(dir.path().join("migration-backups")).unwrap() {
assert!(!entry.unwrap().path().join("state.tar").exists());
}
}
#[test]
fn backup_covers_all_portainer_state_once_and_excludes_runtime_socket() {
let m = portainer();
assert!(enabled(&m).unwrap());
assert_eq!(
relative_sources(&m, Path::new("/var/lib/archipelago")).unwrap(),
vec![PathBuf::from("portainer")]
);
}
#[test]
fn backup_refuses_unknown_state_locations_instead_of_silently_omitting_them() {
let mut m = portainer();
m.app.volumes[0].source = "/other/operator/state".into();
assert!(relative_sources(&m, Path::new("/var/lib/archipelago")).is_err());
m.app.volumes[0].source = "/var/lib/archipelago/../secret".into();
assert!(relative_sources(&m, Path::new("/var/lib/archipelago")).is_err());
}
}
+1
View File
@@ -12,6 +12,7 @@ pub mod hooks;
pub mod image_policy;
pub mod image_versions;
pub mod lnd;
pub mod migration_backup;
pub mod prod_orchestrator;
pub mod quadlet;
pub mod registry;
@@ -36,6 +36,11 @@ use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::sync::{Mutex, RwLock};
/// Refusal before installation has created state or changed any dependency.
#[derive(Debug, thiserror::Error)]
#[error("{0}")]
pub struct InstallPrerequisiteError(pub String);
use crate::config::{Config, ContainerRuntime as ConfigContainerRuntime};
use crate::container::bitcoin_ui;
use crate::container::quadlet;
@@ -91,6 +96,23 @@ fn is_builtin_network_mode(network: &str) -> bool {
)
}
// Only an explicitly selected rootless mode establishes drift. An omitted
// network delegates to Podman and must not recreate unrelated installed apps.
fn rootless_network_mode_drifted(expected: Option<&str>, actual: &str) -> bool {
matches!(expected, Some("slirp4netns" | "pasta"))
&& !actual.trim().is_empty()
&& actual.trim().split(':').next() != expected
}
fn missing_declared_capability(expected: &[String], actual: &[String]) -> bool {
expected.iter().any(|required| {
let required = required.strip_prefix("CAP_").unwrap_or(required);
!actual
.iter()
.any(|cap| cap.strip_prefix("CAP_").unwrap_or(cap) == required)
})
}
fn uses_pasta_network(manifest: &AppManifest) -> bool {
manifest.app.container.network.as_deref() == Some("pasta")
}
@@ -1174,15 +1196,21 @@ impl ReconcileReport {
fn cascade_pairs_for_report<'r>(
report: &'r ReconcileReport,
user_stopped: &std::collections::HashSet<String>,
changed_backends: &HashSet<String>,
) -> Vec<(&'r str, &'static str)> {
let mut pairs = Vec::new();
for (backend, action) in &report.actions {
if !matches!(
action,
ReconcileAction::Installed | ReconcileAction::Started
ReconcileAction::NoOp | ReconcileAction::Started | ReconcileAction::Installed
) {
continue;
}
// A successful systemctl start can be a no-op after a transient
// Podman inspect failure. Require a witnessed lifecycle change.
if !changed_backends.contains(backend) {
continue;
}
for dep in crate::app_ops::address_caching_dependents(backend) {
let dep_untouched = report
.actions
@@ -1196,6 +1224,25 @@ fn cascade_pairs_for_report<'r>(
pairs
}
/// Only positive runtime evidence permits disrupting an address-caching wallet.
/// A known absent/stopped backend becoming running, a new container ID, or a
/// changed start timestamp qualifies. A failed observation never does.
fn backend_instance_changed(before: Option<&ContainerStatus>, after: &ContainerStatus) -> bool {
if after.state != ContainerState::Running || after.id.is_empty() {
return false;
}
let Some(before) = before else {
return true;
};
if before.id.is_empty() {
return false;
}
if before.id != after.id || before.state != ContainerState::Running {
return true;
}
matches!((&before.started_at, &after.started_at), (Some(a), Some(b)) if !a.is_empty() && !b.is_empty() && a != b)
}
#[derive(Debug, Default)]
pub struct AdoptionReport {
pub adopted: Vec<String>,
@@ -1909,12 +1956,33 @@ impl ProdContainerOrchestrator {
_ => 2,
});
// Live container names (any state), for the same recovery check.
let present_containers: std::collections::HashSet<String> = self
.runtime
.list_containers()
.await
.map(|cs| cs.into_iter().map(|c| c.name).collect())
let listed_containers = self.runtime.list_containers().await.ok();
let present_containers: HashSet<String> = listed_containers
.as_ref()
.map(|cs| cs.iter().map(|c| c.name.clone()).collect())
.unwrap_or_default();
// Keep unknown distinct from confirmed absence. Runtime queries can
// fail under load while systemd still has a healthy running backend.
let mut backend_before: HashMap<String, Option<ContainerStatus>> = HashMap::new();
for lm in &manifests {
let id = &lm.manifest.app.id;
if crate::app_ops::address_caching_dependents(id).is_empty() {
continue;
}
let name = compute_container_name(&lm.manifest);
match self.runtime.get_container_status(&name).await {
Ok(status) => {
backend_before.insert(id.clone(), Some(status));
}
Err(_) if listed_containers.is_some() && !present_containers.contains(&name) => {
backend_before.insert(id.clone(), None);
}
Err(err) => {
tracing::warn!(backend = %id, error = %err,
"cannot observe backend before reconcile; will not infer a dependency restart from an action report");
}
}
}
let mut report = ReconcileReport::default();
let disk_gb = self.disk_gb().await;
let bitcoin_pruned = disk_gb < ARCHIVAL_BITCOIN_DISK_GB
@@ -2096,7 +2164,20 @@ impl ProdContainerOrchestrator {
// state recovery, repair recreate, boot InstallMissing) moves the
// address behind a running dependent's back — §C "restart lnd after
// ANY bitcoin recreate".
for (backend, dep) in cascade_pairs_for_report(&report, &user_stopped) {
let mut changed_backends = HashSet::new();
for (backend, before) in &backend_before {
let Some(name) = container_name_by_app_id.get(backend) else {
continue;
};
if let Ok(after) = self.runtime.get_container_status(name).await {
if backend_instance_changed(before.as_ref(), &after) {
changed_backends.insert(backend.clone());
}
}
}
// A user stop during a slow reconcile pass still takes precedence.
let user_stopped = crate::crash_recovery::load_user_stopped(&self.data_dir).await;
for (backend, dep) in cascade_pairs_for_report(&report, &user_stopped, &changed_backends) {
// Same rule as the RPC cascade: hold the dependent's op lock
// across the restart; skip when a worker is mid-sequence.
let lock = crate::app_ops::op_lock(dep);
@@ -2405,6 +2486,8 @@ impl ProdContainerOrchestrator {
.await
{
tracing::info!(app_id = %app_id, container = %name, "container published-port drift detected — recreating");
self.backup_runtime_change(&name, &resolved_manifest)
.await?;
let _ = self.runtime.stop_container(&name).await;
let _ = self.runtime.remove_container(&name).await;
self.install_fresh(lm).await?;
@@ -2440,6 +2523,8 @@ impl ProdContainerOrchestrator {
return Ok(ReconcileAction::NoOp);
}
tracing::info!(app_id = %app_id, container = %name, "container env drift detected — recreating");
self.backup_runtime_change(&name, &resolved_manifest)
.await?;
let _ = self.runtime.stop_container(&name).await;
let _ = self.runtime.remove_container(&name).await;
self.install_fresh(lm).await?;
@@ -2496,6 +2581,8 @@ impl ProdContainerOrchestrator {
.await
{
tracing::info!(app_id = %app_id, container = %name, "stopped container env/port drift detected — recreating");
self.backup_runtime_change(&name, &resolved_manifest)
.await?;
let _ = self.runtime.remove_container(&name).await;
self.install_fresh(lm).await?;
return Ok(ReconcileAction::Installed);
@@ -2552,6 +2639,8 @@ impl ProdContainerOrchestrator {
self.prepare_for_start(&resolved_manifest).await?;
if self.container_env_drifted(&name, &resolved_manifest).await {
tracing::info!(app_id = %app_id, container = %name, "created container env drift detected — recreating");
self.backup_runtime_change(&name, &resolved_manifest)
.await?;
let _ = self.runtime.remove_container(&name).await;
self.install_fresh(lm).await?;
return Ok(ReconcileAction::Installed);
@@ -3021,13 +3110,9 @@ impl ProdContainerOrchestrator {
/// app is a companion (companion.rs owns those units), or when no
/// unit file exists yet (install_via_quadlet handles first-write).
///
/// We DON'T restart the .service when content changes — running
/// containers keep their current config until an operator-initiated
/// restart picks up the new file. That's the right tradeoff: file
/// updates are cheap and non-destructive; service restarts are
/// destructive (the SIGKILL cascade we're trying to eliminate).
/// systemctl --user daemon-reload runs only when content actually
/// changed, so steady-state reconcile ticks pay just one fs read.
/// Ordinary metadata changes wait for an operator restart. Runtime-affecting
/// changes restart the service and retain a durable pending marker until
/// that succeeds, including across daemon restarts and failed reloads.
async fn sync_quadlet_unit(&self, lm: &LoadedManifest, name: &str) -> Result<()> {
// Companions: same reasoning as migrate_to_quadlet_if_needed —
// companion.rs renders these units with a different shape, syncing
@@ -3047,7 +3132,7 @@ impl ProdContainerOrchestrator {
}
let old_body = tokio::fs::read_to_string(&unit_path)
.await
.unwrap_or_default();
.with_context(|| format!("read existing quadlet for {name}"))?;
let restart_required = quadlet::contains_stale_health_gate(&old_body);
let mut resolved = lm.manifest.clone();
@@ -3063,49 +3148,49 @@ impl ProdContainerOrchestrator {
quadlet::network_aliases_changed(&old_body, &new_body);
let restart_for_exec_change = quadlet::exec_changed(&old_body, &new_body);
let restart_for_health_change = quadlet::health_cmd_changed(&old_body, &new_body);
let restart_for_security_change = quadlet::security_changed(&old_body, &new_body);
let needs_restart = restart_required
|| restart_for_port_change
|| restart_for_network_alias_change
|| restart_for_exec_change
|| restart_for_health_change
|| restart_for_security_change;
// Record the obligation BEFORE replacing the unit. A failed reload or
// restart must not become a no-op on the next tick just because the
// generated file already matches the manifest.
let pending = quadlet::RestartObligation::prepare(&unit_path, needs_restart).await?;
if pending.is_pending() {
self.ensure_resolved_source_available(lm).await?;
}
if needs_restart {
self.backup_runtime_change(name, &resolved).await?;
}
let changed = quadlet::write_if_changed(&unit, &unit_dir)
.await
.with_context(|| format!("drift-sync quadlet unit for {name}"))?;
if changed {
if changed || pending.is_pending() {
quadlet::daemon_reload_user()
.await
.context("systemctl --user daemon-reload after drift-syncing quadlet unit")?;
tracing::info!(
app_id = %lm.manifest.app.id,
container = %name,
"Quadlet unit drift-synced — file rewritten, .service NOT restarted (operator restart picks up new config)"
);
}
if changed
&& (restart_required
|| restart_for_port_change
|| restart_for_network_alias_change
|| restart_for_exec_change
|| restart_for_health_change)
{
self.ensure_resolved_source_available(lm).await?;
if pending.is_pending() {
let service = unit.service_name();
let reason = if restart_required {
"stale health gate"
} else if restart_for_port_change {
"port binding drift"
} else if restart_for_network_alias_change {
"network alias drift"
} else if restart_for_health_change {
"health command drift"
} else {
"exec drift"
};
tracing::info!(
app_id = %lm.manifest.app.id,
container = %name,
service = %service,
reason = reason,
"Quadlet unit rewrite requires service restart"
"Applying pending Quadlet runtime change"
);
quadlet::restart_service(&service)
.await
.with_context(|| format!("restart drifted quadlet service {service}"))?;
pending.complete().await?;
} else if changed {
tracing::info!(
app_id = %lm.manifest.app.id,
container = %name,
"Quadlet metadata updated; operator restart will apply it"
);
}
Ok(())
}
@@ -3424,11 +3509,9 @@ impl ProdContainerOrchestrator {
}
async fn cleanup_stale_grafana_port(&self) {
let _ = tokio::process::Command::new("pkill")
.args(["-f", "pasta.*3001"])
.output()
.await;
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
// Port 3001 can belong to Gitea or the daemon's gate. Reap only a
// Grafana container proven absent from Podman's inventory.
crate::container::ghost_reaper::reap_for_app("grafana").await;
}
async fn detect_host_facts(&self) -> HostFacts {
@@ -3809,6 +3892,77 @@ impl ProdContainerOrchestrator {
Ok(())
}
async fn backup_runtime_change(&self, name: &str, manifest: &AppManifest) -> Result<()> {
if !crate::container::migration_backup::enabled(manifest)? {
return Ok(());
}
// A persistent disk/permission failure must not repeatedly stop a
// working old service. Reuse the reconciler's bounded repair budget.
if !self.should_attempt_repair(name).await {
anyhow::bail!("runtime migration retry budget exhausted; original service retained, inspect backup failure before retrying");
}
// Called only before a known runtime change. No app-specific commands;
// opted-in manifests identify their persistent state through bind mounts.
let output = tokio::process::Command::new("podman")
.args(["inspect", name, "--format", "{{.HostConfig.NetworkMode}}"])
.output()
.await
.context("inspect network before migration backup")?;
let present = if output.status.success() {
true
} else {
// A crash after gracefully stopping a --rm Quadlet container can
// leave only its data and old unit. Prove absence before snapshotting
// stopped state; an inspect/Podman failure is not proof of absence.
let exists = tokio::process::Command::new("podman")
.args(["container", "exists", name])
.status()
.await?;
if exists.code() != Some(1) {
anyhow::bail!("cannot verify existing container before runtime migration backup");
}
false
};
let service = format!("{name}.service");
let managed = quadlet::unit_exists(name).await;
let previous_unit = if managed {
Some(
tokio::fs::read(quadlet::unit_dir().await?.join(format!("{name}.container")))
.await?,
)
} else {
None
};
if managed {
quadlet::stop_service(&service).await?;
} else if present {
self.runtime.stop_container(name).await?;
}
match crate::container::migration_backup::snapshot(
manifest,
&self.data_dir,
previous_unit.as_deref(),
)
.await
{
Ok(archive) => {
tracing::info!(container = %name, backup = %archive.display(), "Persistent state saved before runtime migration");
Ok(())
}
Err(error) => {
// The unit has not been rewritten yet. Restore its previous
// service on backup failure and report the migration failure.
let restored = if managed {
quadlet::enable_now(&service).await
} else {
self.runtime.start_container(name).await
};
restored.context("restore original app after failed migration snapshot")?;
Err(error)
}
}
}
async fn container_env_drifted(&self, name: &str, manifest: &AppManifest) -> bool {
if cfg!(test) {
return false;
@@ -3818,6 +3972,52 @@ impl ProdContainerOrchestrator {
return true;
}
// Generated-unit drift handles managed services; preserve deliberate
// systemd drop-in overrides instead of recreating them every tick.
let unmanaged = !quadlet::unit_exists(name).await;
// Podman's effective bounding set, not Docker-compatible CapAdd (which
// can be empty even when Quadlet supplied capabilities).
if unmanaged && !manifest.app.security.capabilities.is_empty() {
if let Ok(output) = tokio::process::Command::new("podman")
.args(["inspect", name, "--format", "{{json .BoundingCaps}}"])
.output()
.await
{
if output.status.success() {
if let Ok(actual) = serde_json::from_slice::<Vec<String>>(&output.stdout) {
if missing_declared_capability(&manifest.app.security.capabilities, &actual)
{
return true;
}
}
}
}
}
// Quadlet handles declarative Network= drift above. Legacy rootless
// Podman containers need the same convergence when no unit owns them.
if unmanaged
&& matches!(
manifest.app.container.network.as_deref(),
Some("slirp4netns" | "pasta")
)
{
if let Ok(output) = tokio::process::Command::new("podman")
.args(["inspect", name, "--format", "{{.HostConfig.NetworkMode}}"])
.output()
.await
{
if output.status.success()
&& rootless_network_mode_drifted(
manifest.app.container.network.as_deref(),
&String::from_utf8_lossy(&output.stdout),
)
{
return true;
}
}
}
let inspect = tokio::process::Command::new("podman")
.args([
"inspect",
@@ -4384,6 +4584,45 @@ impl ContainerOrchestrator for ProdContainerOrchestrator {
}
async fn install(&self, app_id: &str) -> Result<String> {
let lm = self.loaded(app_id).await?;
// Optional shared-service preconditions are checked before recording
// installation or creating anything. A headless adapter must not claim
// successful installation against a missing indexing stack.
if let Some(required) = lm
.manifest
.app
.extensions
.get("install_prerequisites")
.and_then(|value| value.as_sequence())
{
let present = self
.runtime
.list_containers()
.await
.context("check installed prerequisite services")?;
for id in required.iter().filter_map(|value| value.as_str()) {
let dependency = self.loaded(id).await.map_err(|_| InstallPrerequisiteError(
format!("Required app {id} is unavailable. Refresh the app catalog before installing {}.",
lm.manifest.app.name)))?;
let name = compute_container_name(&dependency.manifest);
if !present
.iter()
.any(|container| container.name.trim_start_matches('/') == name)
{
let owner = crate::app_ops::owning_package(id);
let title = self
.loaded(owner)
.await
.map(|app| app.manifest.app.name)
.unwrap_or(dependency.manifest.app.name);
return Err(InstallPrerequisiteError(format!(
"Install {title} first, then install {}.",
lm.manifest.app.name
))
.into());
}
}
}
{
let mut state = self.state.write().await;
state.disabled.remove(app_id);
@@ -4410,7 +4649,6 @@ impl ContainerOrchestrator for ProdContainerOrchestrator {
// health verification (the .228 "running but unreachable" failure
// mode). Routing every install through here means the orchestrator
// is the one source of truth for what "installed" means.
let lm = self.loaded(app_id).await?;
let name = compute_container_name(&lm.manifest);
// ensure_running takes the per-app lock itself; release the install
// path lock first if we hold one (we don't — install is the entry
@@ -4860,6 +5098,78 @@ mod tests {
/// recovered when its siblings have live containers (the stack is
/// installed), and left alone when the whole stack is gone or the app
/// is not a stack member at all.
#[tokio::test]
async fn gitea_fresh_url_seed_preserves_operator_config_and_reports_write_failure() {
let manifest =
AppManifest::parse(include_str!("../../../../apps/gitea/manifest.yml")).unwrap();
let seed = &manifest.app.files[0];
assert!(!seed.overwrite);
let content = seed.content.replace("{{HOST_IP}}", "192.0.2.1");
assert!(content.contains("ROOT_URL = http://192.0.2.1:3001/"));
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("fresh/app.ini");
assert_eq!(
ensure_rendered_file(path.to_str().unwrap(), &content, seed.overwrite)
.await
.unwrap(),
HookOutcome::Rewritten
);
assert!(tokio::fs::read_to_string(&path)
.await
.unwrap()
.contains("ROOT_URL"));
let custom =
"[server]\nROOT_URL = https://git.example.test/\n[database]\nDB_TYPE = postgres\n";
tokio::fs::write(&path, custom).await.unwrap();
assert_eq!(
ensure_rendered_file(path.to_str().unwrap(), &content, seed.overwrite)
.await
.unwrap(),
HookOutcome::Unchanged
);
assert_eq!(tokio::fs::read_to_string(&path).await.unwrap(), custom);
let impossible = path.join("app.ini");
assert!(
ensure_rendered_file(impossible.to_str().unwrap(), &content, seed.overwrite)
.await
.is_err()
);
}
#[test]
fn ssh_sandbox_capability_repair_uses_bounding_set_and_preserves_extra_overrides() {
let required = vec!["CHOWN".into(), "SYS_CHROOT".into()];
assert!(missing_declared_capability(
&required,
&["CAP_CHOWN".into()]
));
assert!(!missing_declared_capability(
&required,
&["CAP_CHOWN".into(), "CAP_SYS_CHROOT".into()]
));
assert!(!missing_declared_capability(
&required,
&["CHOWN".into(), "SYS_CHROOT".into(), "CAP_KILL".into()]
));
}
#[test]
fn explicit_rootless_network_change_converges_without_guessing_defaults() {
assert!(rootless_network_mode_drifted(Some("slirp4netns"), "pasta"));
assert!(rootless_network_mode_drifted(Some("slirp4netns"), "bridge"));
assert!(!rootless_network_mode_drifted(
Some("slirp4netns"),
"slirp4netns"
));
assert!(!rootless_network_mode_drifted(
Some("slirp4netns"),
"slirp4netns:allow_host_loopback=true"
));
assert!(!rootless_network_mode_drifted(None, "pasta"));
assert!(!rootless_network_mode_drifted(Some("slirp4netns"), ""));
assert!(!rootless_network_mode_drifted(Some("archy-net"), "bridge"));
}
#[test]
fn absent_stack_member_recovery_requires_a_live_sibling() {
let present: HashSet<String> = ["indeedhub-redis", "indeedhub-relay", "indeedhub"]
@@ -5541,6 +5851,38 @@ app:
orch
}
#[tokio::test]
async fn missing_install_prerequisite_refuses_without_inventory_or_container_mutation() {
let rt = Arc::new(MockRuntime::default());
let orch = orch_with(rt.clone()).await;
let mut app = pull_manifest("indexer-adapter", "docker.io/library/alpine:3.20");
app.app.extensions.insert(
"install_prerequisites".into(),
serde_yaml::to_value(vec!["shared-index"]).unwrap(),
);
orch.insert_manifest_for_test(app, PathBuf::from("/tmp"))
.await;
orch.insert_manifest_for_test(
pull_manifest("shared-index", "index:1"),
PathBuf::from("/tmp"),
)
.await;
let error = orch.install("indexer-adapter").await.unwrap_err();
assert!(error.downcast_ref::<InstallPrerequisiteError>().is_some());
assert!(!crate::crash_recovery::load_installed_apps(&orch.data_dir)
.await
.contains("indexer-adapter"));
assert_eq!(rt.calls(), vec!["list_containers"]);
// An installed prerequisite satisfies the guard; it is never recreated
// or reconfigured as part of installing this adapter.
rt.set_state("shared-index", ContainerState::Running);
orch.install("indexer-adapter").await.unwrap();
assert!(!rt
.calls()
.iter()
.any(|c| c.starts_with("create_container:shared-index")));
}
fn pull_manifest_with_dynamic_env(id: &str, image: &str) -> AppManifest {
let yaml = format!(
"app:\n id: {id}\n name: {id}\n version: 1.0.0\n container:\n image: {image}\n derived_env:\n - key: FM_API_URL\n template: \"ws://{{{{HOST_MDNS}}}}:8174\"\n secret_env:\n - key: FM_BITCOIND_PASSWORD\n secret_file: bitcoin-rpc-password\n environment:\n - STATIC=1\n"
@@ -6409,6 +6751,67 @@ app:
);
}
#[test]
fn backend_cascade_requires_observed_instance_change() {
let running = ContainerStatus {
id: "container-1".into(),
name: "bitcoin-core".into(),
state: ContainerState::Running,
started_at: Some("start-1".into()),
health: None,
exit_code: None,
image: "bitcoin:1".into(),
created: "created-1".into(),
ports: vec![],
lan_address: None,
};
assert!(!backend_instance_changed(Some(&running), &running));
assert!(backend_instance_changed(None, &running));
let mut before = running.clone();
before.state = ContainerState::Exited;
assert!(backend_instance_changed(Some(&before), &running));
before = running.clone();
before.id = "old-container".into();
assert!(backend_instance_changed(Some(&before), &running));
before = running.clone();
before.started_at = Some("earlier-start".into());
assert!(backend_instance_changed(Some(&before), &running));
before.started_at = None;
assert!(!backend_instance_changed(Some(&before), &running));
before.id.clear();
assert!(!backend_instance_changed(Some(&before), &running));
let mut after = running.clone();
after.state = ContainerState::Exited;
assert!(!backend_instance_changed(None, &after));
after = running.clone();
after.id.clear();
assert!(!backend_instance_changed(None, &after));
}
#[test]
fn cascade_ignores_false_started_report_but_detects_real_exec_drift() {
let none = HashSet::new();
let mut report = ReconcileReport {
actions: vec![
("bitcoin-core".into(), ReconcileAction::Started),
("lnd".into(), ReconcileAction::NoOp),
],
failures: vec![],
};
// systemctl start of an already active unit does not move its address.
assert!(cascade_pairs_for_report(&report, &none, &none).is_empty());
// A unit exec rewrite can restart Bitcoin while the outer reconcile
// action remains NoOp. Runtime evidence still requires LND to reconnect.
let changed = ["bitcoin-core".into()].into();
report.actions[0].1 = ReconcileAction::NoOp;
assert_eq!(
cascade_pairs_for_report(&report, &none, &changed),
vec![("bitcoin-core", "lnd")]
);
report.actions[0].1 = ReconcileAction::Left("lifecycle-op-in-flight".into());
assert!(cascade_pairs_for_report(&report, &none, &changed).is_empty());
}
#[test]
fn cascade_pairs_cover_backend_recreate_with_running_dependent() {
use std::collections::HashSet;
@@ -6420,6 +6823,7 @@ app:
failures: vec![],
};
let none = HashSet::new();
let changed: HashSet<String> = ["bitcoin-core".into(), "bitcoin-knots".into()].into();
// Backend recreated while lnd sat running (NoOp) → cascade.
let r = report(vec![
@@ -6427,7 +6831,7 @@ app:
("lnd", ReconcileAction::NoOp),
]);
assert_eq!(
cascade_pairs_for_report(&r, &none),
cascade_pairs_for_report(&r, &none, &changed),
vec![("bitcoin-knots", "lnd")]
);
@@ -6437,7 +6841,7 @@ app:
("lnd", ReconcileAction::NoOp),
]);
assert_eq!(
cascade_pairs_for_report(&r, &none),
cascade_pairs_for_report(&r, &none, &changed),
vec![("bitcoin-core", "lnd")]
);
@@ -6446,7 +6850,7 @@ app:
("bitcoin-knots", ReconcileAction::NoOp),
("lnd", ReconcileAction::NoOp),
]);
assert!(cascade_pairs_for_report(&r, &none).is_empty());
assert!(cascade_pairs_for_report(&r, &none, &none).is_empty());
// Dependent itself (re)started this pass → it already resolved the
// fresh address; no cascade.
@@ -6454,7 +6858,7 @@ app:
("bitcoin-knots", ReconcileAction::Installed),
("lnd", ReconcileAction::Started),
]);
assert!(cascade_pairs_for_report(&r, &none).is_empty());
assert!(cascade_pairs_for_report(&r, &none, &changed).is_empty());
// User-stopped dependent is never bounced.
let r = report(vec![
@@ -6462,14 +6866,14 @@ app:
("lnd", ReconcileAction::NoOp),
]);
let stopped: HashSet<String> = ["lnd".to_string()].into();
assert!(cascade_pairs_for_report(&r, &stopped).is_empty());
assert!(cascade_pairs_for_report(&r, &stopped, &changed).is_empty());
// Non-backend recreates don't cascade anything.
let r = report(vec![
("grafana", ReconcileAction::Installed),
("lnd", ReconcileAction::NoOp),
]);
assert!(cascade_pairs_for_report(&r, &none).is_empty());
assert!(cascade_pairs_for_report(&r, &none, &changed).is_empty());
}
#[tokio::test]
+310 -7
View File
@@ -184,6 +184,7 @@ pub struct QuadletUnit {
pub no_new_privileges: bool,
pub cpu_quota: Option<u32>,
pub restart_policy: RestartPolicy,
pub stop_grace_secs: Option<u64>,
}
impl QuadletUnit {
@@ -216,6 +217,10 @@ impl QuadletUnit {
let _ = writeln!(s, "[Container]");
let _ = writeln!(s, "ContainerName={}", self.name);
let _ = writeln!(s, "Image={}", self.image);
let grace = self
.stop_grace_secs
.unwrap_or_else(|| archipelago_container::runtime::stop_grace_secs_for(&self.name));
let _ = writeln!(s, "StopTimeout={grace}");
// Pull=never: companions are pre-pulled or built. A missing image
// must surface as a unit start failure, not a silent retry storm.
let _ = writeln!(s, "Pull=never");
@@ -350,6 +355,15 @@ impl QuadletUnit {
// the unit stuck in deactivating. Health/status remains app-level state,
// not a systemd start gate.
let _ = writeln!(s, "TimeoutStartSec=0");
let _ = writeln!(s, "TimeoutStopSec={}", grace.saturating_add(15));
// Stop explicitly before Quadlet's generated `podman rm -f`. The
// existing container may still carry Podman's old 10-second default;
// StopTimeout alone only protects containers created after migration.
let _ = writeln!(s, "ExecStop=");
let _ = writeln!(
s,
"ExecStop=/usr/bin/podman stop --ignore --time={grace} --cidfile=%t/%N.cid"
);
// Restart policy + 10s backoff. RestartSec keeps a crash-loop
// from saturating the journal. Companions: Always. Backends:
// OnFailure (clean stops stay stopped).
@@ -376,7 +390,10 @@ fn shell_join(parts: &[String]) -> String {
.iter()
.map(|p| {
let p = p.replace(['\r', '\n'], " ").replace('%', "%%");
if p.is_empty() || p.chars().any(|c| c.is_whitespace() || "\"\\$`".contains(c)) {
if p.is_empty()
|| p.chars()
.any(|c| c.is_whitespace() || "'\"\\$`".contains(c))
{
let escaped = p
.replace('\\', "\\\\")
.replace('"', "\\\"")
@@ -396,7 +413,7 @@ fn quote_environment(env: &str) -> String {
if env.is_empty()
|| env
.chars()
.any(|c| c.is_whitespace() || "\"\\$`".contains(c))
.any(|c| c.is_whitespace() || "'\"\\$`".contains(c))
{
let escaped = env
.replace('\\', "\\\\")
@@ -525,6 +542,9 @@ impl QuadletUnit {
// Always, not OnFailure: with quadlet's `--rm`, OnFailure left a
// cleanly-exited app deleted and unrestarted. See RestartPolicy.
restart_policy: RestartPolicy::Always,
stop_grace_secs: Some(super::prod_orchestrator::resolve_stop_grace_secs(
manifest, name,
)),
}
}
}
@@ -792,7 +812,11 @@ pub async fn stop_service(service: &str) -> Result<()> {
/// corruption — so the orchestrator passes the per-app grace here. Never waits
/// less than `QUADLET_STOP_TIMEOUT`.
pub async fn stop_service_with_timeout(service: &str, timeout: Duration) -> Result<()> {
let timeout = timeout.max(QUADLET_STOP_TIMEOUT);
let name = service.strip_suffix(".service").unwrap_or(service);
let body = fs::read_to_string(unit_dir().await?.join(format!("{name}.container")))
.await
.unwrap_or_default();
let timeout = timeout.max(stop_wait_timeout(name, &body));
match systemctl_user_status(&["stop", service], timeout).await {
Ok(status) if status.success() => Ok(()),
Ok(status) => Err(anyhow!("systemctl --user stop {service} exited {status}")),
@@ -813,6 +837,20 @@ pub async fn stop_service_with_timeout(service: &str, timeout: Duration) -> Resu
}
}
/// The command waiter must outlive both the container grace and systemd's
/// stop deadline. Restart/repair callers must not kill Bitcoin at 45 seconds.
fn stop_wait_timeout(name: &str, unit_body: &str) -> Duration {
Duration::from_secs(stop_grace_from_unit(name, unit_body).saturating_add(30))
.max(QUADLET_STOP_TIMEOUT)
}
fn stop_grace_from_unit(name: &str, unit_body: &str) -> u64 {
directive_values(unit_body, "StopTimeout=")
.last()
.and_then(|value| value.parse::<u64>().ok())
.unwrap_or_else(|| archipelago_container::runtime::stop_grace_secs_for(name))
}
async fn systemctl_user_status(
args: &[&str],
timeout: Duration,
@@ -903,12 +941,77 @@ pub fn health_cmd_changed(old_body: &str, new_body: &str) -> bool {
!= directive_values(new_body, "HealthRetries=")
}
/// A unit rewrite and a successful systemd restart are separate operations.
/// Keep the restart obligation across errors or a management-daemon restart.
pub struct RestartObligation {
marker: PathBuf,
pending: bool,
}
impl RestartObligation {
pub async fn prepare(unit_path: &Path, newly_required: bool) -> Result<Self> {
let marker = unit_path.with_extension("restart-pending");
if newly_required {
// Contents contain no manifest environment or credentials. sync_all
// makes the obligation durable before the subsequent unit rename.
let file = tokio::fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(false)
.open(&marker)
.await
.context("record pending Quadlet restart")?;
file.sync_all().await?;
if let Some(parent) = marker.parent() {
tokio::fs::File::open(parent).await?.sync_all().await?;
}
}
let pending = tokio::fs::try_exists(&marker).await?;
Ok(Self { marker, pending })
}
pub fn is_pending(&self) -> bool {
self.pending
}
/// Call only after systemd accepted the replacement service successfully.
pub async fn complete(self) -> Result<()> {
if self.pending {
tokio::fs::remove_file(&self.marker)
.await
.context("clear completed Quadlet restart")?;
if let Some(parent) = self.marker.parent() {
tokio::fs::File::open(parent).await?.sync_all().await?;
}
}
Ok(())
}
}
pub fn publish_ports_changed(old_body: &str, new_body: &str) -> bool {
let old_ports = directive_values(old_body, "PublishPort=");
let new_ports = directive_values(new_body, "PublishPort=");
old_ports != new_ports
}
pub fn security_changed(old_body: &str, new_body: &str) -> bool {
[
"AddCapability=",
"DropCapability=",
"NoNewPrivileges=",
"ReadOnly=",
"User=",
]
.iter()
.any(|directive| {
let mut old = directive_values(old_body, directive);
let mut new = directive_values(new_body, directive);
old.sort();
new.sort();
old != new
})
}
pub fn network_aliases_changed(old_body: &str, new_body: &str) -> bool {
let old_network = directive_values(old_body, "Network=");
let new_network = directive_values(new_body, "Network=");
@@ -939,6 +1042,10 @@ fn directive_values(unit_body: &str, prefix: &str) -> Vec<String> {
/// that systemd no longer knows about.
pub async fn disable_remove(unit_name: &str, dir: &Path) -> Result<()> {
let svc = format!("{unit_name}.service");
let path = dir.join(format!("{unit_name}.container"));
let body = fs::read_to_string(&path).await.unwrap_or_default();
let timeout = stop_wait_timeout(unit_name, &body);
let grace = stop_grace_from_unit(unit_name, &body).to_string();
// Stop first; ignore failure (unit may already be down). BOUNDED — on
// rootless podman a generated unit can wedge in "deactivating" while
// `podman rm -f` hangs underneath it, and an unbounded `systemctl stop`
@@ -946,13 +1053,12 @@ pub async fn disable_remove(unit_name: &str, dir: &Path) -> Result<()> {
// the package entry is stranded in `Removing` (a ghost in My Apps that also
// blocks reinstall). If the graceful stop times out, escalate to
// SIGKILL + reset-failed so teardown always proceeds.
if systemctl_user_status(&["stop", &svc], QUADLET_STOP_TIMEOUT)
if systemctl_user_status(&["stop", &svc], timeout)
.await
.is_err()
{
let _ = kill_and_reset_service(&svc).await;
}
let path = dir.join(format!("{unit_name}.container"));
if fs::try_exists(&path).await.unwrap_or(false) {
match fs::remove_file(&path).await {
Ok(()) => {}
@@ -965,9 +1071,9 @@ pub async fn disable_remove(unit_name: &str, dir: &Path) -> Result<()> {
// Bounded so a hung podman store can't re-introduce the stall this function
// exists to avoid.
let _ = tokio::time::timeout(
QUADLET_STOP_TIMEOUT,
timeout,
Command::new("podman")
.args(["rm", "-f", unit_name])
.args(["rm", "-f", "--ignore", "--time", &grace, unit_name])
.status(),
)
.await;
@@ -992,6 +1098,118 @@ mod tests {
use super::*;
use tempfile::tempdir;
#[test]
fn shutdown_grace_covers_container_systemd_and_caller() {
for (name, grace) in [
("bitcoin-core", 600),
("bitcoin-knots", 600),
("lnd", 330),
("electrumx", 300),
("other", 30),
] {
let unit = QuadletUnit {
name: name.into(),
..Default::default()
};
let body = unit.render();
assert!(body.contains(&format!("StopTimeout={grace}\n")));
assert!(body.contains(&format!("TimeoutStopSec={}\n", grace + 15)));
assert!(body.contains(&format!("podman stop --ignore --time={grace} --cidfile=")));
assert_eq!(
stop_wait_timeout(name, &body),
Duration::from_secs(grace + 30)
);
// Legacy units have no StopTimeout directive yet.
assert_eq!(stop_wait_timeout(name, ""), Duration::from_secs(grace + 30));
}
}
#[test]
fn custom_stop_grace_survives_render_and_restart_budget() {
let manifest: AppManifest = serde_yaml::from_str(
r#"
app:
id: custom-db
name: Custom database
version: 1.0.0
stop_grace_secs: 900
container:
image: example/db:1
"#,
)
.unwrap();
let unit = QuadletUnit::from_manifest(&manifest, "custom-db");
assert_eq!(unit.stop_grace_secs, Some(900));
assert_eq!(
stop_wait_timeout("custom-db", &unit.render()),
Duration::from_secs(930)
);
assert_eq!(
stop_wait_timeout("lnd", "StopTimeout=invalid"),
Duration::from_secs(360)
);
}
#[test]
fn stop_grace_migration_does_not_request_an_execution_restart() {
let unit = sample_unit();
let new = unit.render();
let old = new
.lines()
.filter(|line| {
!line.starts_with("StopTimeout=")
&& !line.starts_with("TimeoutStopSec=")
&& !line.starts_with("ExecStop=")
})
.collect::<Vec<_>>()
.join("\n");
assert!(!exec_changed(&old, &new));
assert!(!publish_ports_changed(&old, &new));
assert!(!network_aliases_changed(&old, &new));
assert!(!health_cmd_changed(&old, &new));
}
#[test]
fn actual_quadlet_generator_stops_before_forced_removal() {
let generator = Path::new("/usr/lib/systemd/system-generators/podman-system-generator");
if !generator.exists() {
eprintln!(
"Quadlet generator unavailable; run this regression on the Linux release host"
);
return;
}
let dir = tempdir().unwrap();
let unit = QuadletUnit {
name: "grace-test".into(),
image: "localhost/test:latest".into(),
stop_grace_secs: Some(600),
..Default::default()
};
std::fs::write(dir.path().join("grace-test.container"), unit.render()).unwrap();
let output = std::process::Command::new(generator)
.args(["--user", "--dryrun"])
.env("QUADLET_UNIT_DIRS", dir.path())
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let generated = String::from_utf8_lossy(&output.stdout).to_string()
+ &String::from_utf8_lossy(&output.stderr);
let stop = generated
.find("ExecStop=/usr/bin/podman stop --ignore --time=600")
.unwrap();
let remove = generated.find("ExecStop=/usr/bin/podman rm ").unwrap();
assert!(
stop < remove,
"Legacy container must stop gracefully before removal"
);
assert!(generated.contains("--stop-timeout 600"));
assert!(generated.contains("TimeoutStopSec=615"));
}
#[test]
fn render_emits_secret_env_by_reference_never_value() {
let u = QuadletUnit {
@@ -1179,6 +1397,18 @@ mod tests {
);
}
#[test]
fn apostrophes_survive_quadlet_argument_and_environment_parsing() {
// A whitespace-free Node script reproduced this in a real Quadlet:
// unquoted apostrophes were consumed by the parser, changing JS strings
// into identifiers and preventing the app from starting.
assert_eq!(
shell_join(&["require('http')".into()]),
"\"require('http')\""
);
assert_eq!(quote_environment("NAME=O'Brien"), "\"NAME=O'Brien\"");
}
#[test]
fn quote_environment_quotes_values_with_spaces() {
assert_eq!(
@@ -1391,6 +1621,26 @@ app:
assert!(!s.contains("Network=host"));
}
#[test]
fn portainer_catalog_network_repairs_same_node_routing_without_exposing_backend() {
let manifest = AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml"))
.expect("shipped Portainer manifest must parse");
let new = QuadletUnit::from_manifest(&manifest, "portainer").render();
assert!(new.contains("Network=slirp4netns\n"));
assert!(!new.contains("NetworkAlias="));
assert!(new.contains("PublishPort=127.0.0.1:9000:9000/tcp"));
assert!(!new.contains("PublishPort=0.0.0.0"));
// The upgrade changes networking only: retain both state mounts and the
// existing rootless socket, without an app.ini or repository rewrite.
assert!(new.contains("Volume=/var/lib/archipelago/portainer:/data"));
assert!(new.contains("Volume=/var/lib/archipelago/portainer/compose:/data/compose"));
assert!(new.contains("Volume=/run/user/1000/podman/podman.sock:/var/run/docker.sock"));
let old = new.replace("Network=slirp4netns\n", "");
assert!(network_aliases_changed(&old, &new));
assert!(!network_aliases_changed(&new, &new));
assert!(!publish_ports_changed(&old, &new));
}
#[test]
fn from_manifest_slirp4netns_omits_network_alias() {
let yaml = r#"
@@ -1741,6 +1991,59 @@ app:
assert!(!network_aliases_changed(new, new));
}
#[tokio::test]
async fn failed_runtime_change_remains_pending_when_unit_already_matches() {
let dir = tempfile::tempdir().unwrap();
let unit = dir.path().join("portainer.container");
tokio::fs::write(&unit, "[Container]\n").await.unwrap();
let pending = RestartObligation::prepare(&unit, true).await.unwrap();
assert!(pending.is_pending());
tokio::fs::write(&unit, "[Container]\nNetwork=slirp4netns\n")
.await
.unwrap();
// Simulate systemctl failure or daemon interruption after unit rewrite.
drop(pending);
let retry = RestartObligation::prepare(&unit, false).await.unwrap();
assert!(
retry.is_pending(),
"matching unit must not discard failed restart"
);
retry.complete().await.unwrap();
assert!(!RestartObligation::prepare(&unit, false)
.await
.unwrap()
.is_pending());
}
#[tokio::test]
async fn pending_runtime_change_errors_are_not_reported_as_success() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("missing/app.container");
assert!(RestartObligation::prepare(&missing, true).await.is_err());
let unit = dir.path().join("app.container");
let pending = RestartObligation::prepare(&unit, true).await.unwrap();
tokio::fs::remove_file(unit.with_extension("restart-pending"))
.await
.unwrap();
assert!(pending.complete().await.is_err());
}
#[test]
fn gitea_ssh_sandbox_capability_is_applied_as_a_runtime_change() {
let manifest =
AppManifest::parse(include_str!("../../../../apps/gitea/manifest.yml")).unwrap();
manifest.validate().unwrap();
let new = QuadletUnit::from_manifest(&manifest, "gitea").render();
assert!(new.contains("AddCapability=SYS_CHROOT\n"));
let old = new.replace("AddCapability=SYS_CHROOT\n", "");
assert!(security_changed(&old, &new));
assert!(!security_changed(&new, &new));
assert!(!security_changed(
"AddCapability=CHOWN\nAddCapability=SETUID\n",
"AddCapability=SETUID\nAddCapability=CHOWN\n"
));
}
#[test]
fn network_aliases_changed_detects_network_mode_drift() {
let old = "[Container]\nNetwork=slirp4netns\n";
+471 -40
View File
@@ -238,6 +238,11 @@ pub enum ServeResult {
Forbidden,
/// Content not found.
NotFound,
/// The catalog entry and file exist but this node can't read the file.
/// Returned before any payment is taken.
Unavailable,
/// Requested byte range cannot be served; no payment was taken.
RangeNotSatisfiable(u64),
}
/// Serve a content item by ID with access control and optional range request.
@@ -252,6 +257,39 @@ pub async fn serve_content(
range: Option<ByteRange>,
owner_session: bool,
) -> Result<ServeResult> {
serve_content_with(
data_dir,
id,
payment_token,
invoice_hash,
peer_did,
range,
owner_session,
|path, range, mime| prepare_content(data_dir, path, range, mime),
|token, amount| async move { verify_payment_token(data_dir, &token, amount).await },
)
.await
}
// Inject only the read and payment boundaries, so tests can prove ordering
// without mint access, file-permission assumptions or privileged commands.
async fn serve_content_with<R, RF, V, VF>(
data_dir: &Path,
id: &str,
payment_token: Option<&str>,
invoice_hash: Option<&str>,
peer_did: Option<&str>,
range: Option<ByteRange>,
owner_session: bool,
read: R,
verify: V,
) -> Result<ServeResult>
where
R: FnOnce(PathBuf, Option<ByteRange>, String) -> RF,
RF: std::future::Future<Output = Result<ServeResult>>,
V: FnOnce(String, u64) -> VF,
VF: std::future::Future<Output = bool>,
{
let catalog = load_catalog(data_dir).await?;
let item = match catalog.items.iter().find(|i| i.id == id) {
Some(i) => i,
@@ -314,6 +352,29 @@ pub async fn serve_content(
return Ok(ServeResult::NotFound);
}
// Refuse unauthorized viewers before opening or reading any bytes.
if !owner_session && matches!(item.access, AccessControl::PeersOnly) && !is_known_peer {
return Ok(ServeResult::Forbidden);
}
if !owner_session {
if let AccessControl::Paid { price_sats, .. } = &item.access {
if payment_token.is_none() && invoice_hash.is_none() {
return Ok(ServeResult::PaymentRequired(*price_sats));
}
}
}
// Finish all file I/O before consuming bearer payment. Merely opening then
// reopening after charging still lost payments on read errors or deletion.
let prepared = match read(file_path, range, item.mime_type.clone()).await {
Ok(result @ (ServeResult::Ok(..) | ServeResult::Partial { .. })) => result,
Ok(other) => return Ok(other),
Err(error) => {
warn!(content_id = %id, "Cannot prepare shared content: {error:#}");
return Ok(ServeResult::Unavailable);
}
};
// Check access control
if !owner_session {
match &item.access {
@@ -331,7 +392,7 @@ pub async fn serve_content(
"fedimint"
};
if method_accepted(&item.access, method)
&& verify_payment_token(data_dir, token, *price_sats).await
&& verify(token.to_owned(), *price_sats).await
{
authorized = true;
}
@@ -358,55 +419,127 @@ pub async fn serve_content(
}
}
let metadata = fs::metadata(&file_path)
Ok(prepared)
}
async fn prepare_content(
data_dir: &Path,
path: PathBuf,
range: Option<ByteRange>,
mime: String,
) -> Result<ServeResult> {
use tokio::io::{AsyncReadExt, AsyncSeekExt};
let mut file = match fs::OpenOptions::new()
.read(true)
.custom_flags(libc::O_NONBLOCK)
.open(&path)
.await
.context("Failed to read file metadata")?;
let total_size = metadata.len();
// Handle range request for streaming
if let Some(range) = range {
let start = range.start.min(total_size.saturating_sub(1));
let end = range
.end
.map(|e| e.min(total_size - 1))
.unwrap_or(total_size - 1);
if start > end || start >= total_size {
return Ok(ServeResult::NotFound);
{
Ok(file) => file,
Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
let bytes = read_filebrowser_via_userns(data_dir, &path).await?;
return slice_prepared_content(bytes, range, mime);
}
let len = (end - start + 1) as usize;
use tokio::io::{AsyncReadExt, AsyncSeekExt};
let mut file = tokio::fs::File::open(&file_path)
Err(error) => return Err(error).context("Opening shared content"),
};
let metadata = file.metadata().await?;
anyhow::ensure!(metadata.is_file(), "Shared content is not a regular file");
let total = metadata.len();
if let Some(range) = range {
let Some((start, end)) = checked_range(&range, total) else {
return Ok(ServeResult::RangeNotSatisfiable(total));
};
file.seek(std::io::SeekFrom::Start(start)).await?;
let len = usize::try_from(end - start + 1).context("Content range is too large")?;
let mut bytes = vec![0; len];
file.read_exact(&mut bytes)
.await
.context("Failed to open content file")?;
file.seek(std::io::SeekFrom::Start(start))
.await
.context("Failed to seek")?;
let mut buf = vec![0u8; len];
file.read_exact(&mut buf)
.await
.context("Failed to read range")?;
debug!(
"Serving content '{}' range {}-{}/{} ({} bytes)",
id, start, end, total_size, len
);
.context("Reading shared content range")?;
return Ok(ServeResult::Partial {
bytes: buf,
mime_type: item.mime_type.clone(),
bytes,
mime_type: mime,
start,
end,
total: total_size,
total,
});
}
let bytes = fs::read(&file_path)
let mut bytes = Vec::new();
file.read_to_end(&mut bytes)
.await
.context("Failed to read content file")?;
.context("Reading shared content")?;
Ok(ServeResult::Ok(bytes, mime))
}
debug!("Serving content '{}' ({} bytes)", id, bytes.len());
Ok(ServeResult::Ok(bytes, item.mime_type.clone()))
fn checked_range(range: &ByteRange, total: u64) -> Option<(u64, u64)> {
let last = total.checked_sub(1)?;
let end = range.end.unwrap_or(last).min(last);
(range.start <= end && range.start < total).then_some((range.start, end))
}
fn slice_prepared_content(
bytes: Vec<u8>,
range: Option<ByteRange>,
mime: String,
) -> Result<ServeResult> {
let total = bytes.len() as u64;
match range {
None => Ok(ServeResult::Ok(bytes, mime)),
Some(range) => match checked_range(&range, total) {
Some((start, end)) => Ok(ServeResult::Partial {
bytes: bytes[start as usize..=end as usize].to_vec(),
mime_type: mime,
start,
end,
total,
}),
None => Ok(ServeResult::RangeNotSatisfiable(total)),
},
}
}
/// Read only an explicitly shared, regular file within FileBrowser storage.
/// Do not change its mode or grant world-readable access to paid/private data.
async fn filebrowser_read_path(data_dir: &Path, path: &Path) -> Result<PathBuf> {
let root = fs::canonicalize(data_dir.join("filebrowser")).await?;
let target = fs::canonicalize(path).await?;
anyhow::ensure!(
target.starts_with(&root) && target != root,
"Shared file is outside Files storage"
);
anyhow::ensure!(
fs::metadata(&target).await?.is_file(),
"Shared content is not a regular file"
);
Ok(target)
}
async fn read_filebrowser_via_userns(data_dir: &Path, path: &Path) -> Result<Vec<u8>> {
let path = filebrowser_read_path(data_dir, path).await?;
// Tests exercise the boundary explicitly; they never launch the host Podman.
#[cfg(test)]
{
let _ = path;
anyhow::bail!("Files namespace read disabled in unit tests")
}
#[cfg(not(test))]
{
let output = tokio::time::timeout(
std::time::Duration::from_secs(900),
tokio::process::Command::new("podman")
.args(["unshare", "cat", "--"])
.arg(path)
.kill_on_drop(true)
.output(),
)
.await
.context("Files namespace read timed out")??;
anyhow::ensure!(
output.status.success(),
"Files namespace read failed: {}",
output.status
);
Ok(output.stdout)
}
}
/// Result of attempting to serve a preview.
@@ -729,3 +862,301 @@ mod prune_missing_content_tests {
assert_eq!(reloaded.items[0].id, "present-item");
}
}
#[cfg(test)]
mod paid_read_order_tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
async fn fixture(bytes: &[u8]) -> tempfile::TempDir {
let dir = tempfile::tempdir().unwrap();
fs::create_dir_all(dir.path().join("content/files"))
.await
.unwrap();
fs::write(dir.path().join("content/files/test.bin"), bytes)
.await
.unwrap();
save_catalog(
dir.path(),
&ContentCatalog {
items: vec![ContentItem {
id: "paid".into(),
filename: "test.bin".into(),
mime_type: "application/octet-stream".into(),
size_bytes: bytes.len() as u64,
description: String::new(),
access: AccessControl::Paid {
price_sats: 10,
accepted: vec!["ecash".into()],
},
availability: Availability::AllPeers,
added_at: "2026-09-30".into(),
}],
},
)
.await
.unwrap();
dir
}
#[tokio::test]
async fn all_read_failures_precede_redemption_even_as_root() {
for kind in [
std::io::ErrorKind::PermissionDenied,
std::io::ErrorKind::UnexpectedEof,
std::io::ErrorKind::NotFound,
std::io::ErrorKind::Other,
] {
let dir = fixture(b"abc").await;
let charged = AtomicUsize::new(0);
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
None,
false,
|_, _, _| async move { Err(std::io::Error::from(kind).into()) },
|_, _| async {
charged.fetch_add(1, Ordering::SeqCst);
true
},
)
.await
.unwrap();
assert!(matches!(result, ServeResult::Unavailable));
assert_eq!(charged.load(Ordering::SeqCst), 0);
assert_eq!(load_catalog(dir.path()).await.unwrap().items.len(), 1);
}
}
#[tokio::test]
async fn deletion_during_payment_cannot_lose_prepared_bytes() {
let dir = fixture(b"original").await;
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
None,
false,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, amount| {
assert_eq!(amount, 10);
async {
fs::remove_file(dir.path().join("content/files/test.bin"))
.await
.unwrap();
true
}
},
)
.await
.unwrap();
assert!(matches!(result, ServeResult::Ok(bytes, _) if bytes == b"original"));
}
#[tokio::test]
async fn empty_out_of_bounds_and_reversed_ranges_never_charge() {
for (bytes, start, end) in [
(b"".as_slice(), 0, None),
(b"abc".as_slice(), 3, None),
(b"abc".as_slice(), 2, Some(1)),
] {
let dir = fixture(bytes).await;
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
Some(ByteRange { start, end }),
false,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, _| async { panic!("invalid range reached payment") },
)
.await
.unwrap();
assert!(
matches!(result, ServeResult::RangeNotSatisfiable(n) if n == bytes.len() as u64)
);
}
}
#[tokio::test]
async fn prepared_range_survives_file_change_while_payment_is_verified() {
let dir = fixture(b"abcdef").await;
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
Some(ByteRange {
start: 2,
end: Some(999),
}),
false,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, _| async {
fs::write(dir.path().join("content/files/test.bin"), b"x")
.await
.unwrap();
true
},
)
.await
.unwrap();
assert!(
matches!(result, ServeResult::Partial { bytes, start: 2, end: 5, total: 6, .. } if bytes == b"cdef")
);
}
#[tokio::test]
async fn payment_denial_never_returns_prepared_content() {
let dir = fixture(b"secret").await;
let charged = AtomicUsize::new(0);
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
None,
false,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, _| async {
charged.fetch_add(1, Ordering::SeqCst);
false
},
)
.await
.unwrap();
assert!(matches!(result, ServeResult::PaymentRequired(10)));
assert_eq!(charged.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn missing_payment_and_peer_restrictions_precede_file_reads() {
let dir = fixture(b"secret").await;
let result = serve_content_with(
dir.path(),
"paid",
None,
None,
None,
None,
false,
|_, _, _| async { panic!("unauthorized file read") },
|_, _| async { panic!("unexpected payment") },
)
.await
.unwrap();
assert!(matches!(result, ServeResult::PaymentRequired(10)));
let mut catalog = load_catalog(dir.path()).await.unwrap();
catalog.items[0].access = AccessControl::PeersOnly;
save_catalog(dir.path(), &catalog).await.unwrap();
let result = serve_content_with(
dir.path(),
"paid",
None,
None,
None,
None,
false,
|_, _, _| async { panic!("unauthorized file read") },
|_, _| async { panic!("unexpected payment") },
)
.await
.unwrap();
assert!(matches!(result, ServeResult::Forbidden));
}
#[tokio::test]
async fn owner_reads_paid_content_without_redemption() {
let dir = fixture(b"own file").await;
let result = serve_content_with(
dir.path(),
"paid",
None,
None,
None,
None,
true,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, _| async { panic!("owner charged") },
)
.await
.unwrap();
assert!(matches!(result, ServeResult::Ok(bytes, _) if bytes == b"own file"));
}
#[tokio::test]
async fn directory_in_place_of_file_does_not_charge() {
let dir = fixture(b"abc").await;
let path = dir.path().join("content/files/test.bin");
fs::remove_file(&path).await.unwrap();
fs::create_dir(&path).await.unwrap();
let result = serve_content_with(
dir.path(),
"paid",
Some("cashuBtest"),
None,
None,
None,
false,
|path, range, mime| prepare_content(dir.path(), path, range, mime),
|_, _| async { panic!("directory charged") },
)
.await
.unwrap();
assert!(matches!(result, ServeResult::Unavailable));
}
#[tokio::test]
async fn files_namespace_read_is_scoped_to_regular_files_and_keeps_mode() {
use std::os::unix::fs::{symlink, PermissionsExt};
let dir = fixture(b"outside").await;
let root = dir.path().join("filebrowser");
fs::create_dir(&root).await.unwrap();
let inside = root.join("song");
fs::write(&inside, b"song").await.unwrap();
fs::set_permissions(&inside, std::fs::Permissions::from_mode(0o640))
.await
.unwrap();
assert_eq!(
filebrowser_read_path(dir.path(), &inside).await.unwrap(),
inside
);
assert_eq!(
fs::metadata(&inside).await.unwrap().permissions().mode() & 0o777,
0o640
);
let outside = dir.path().join("content/files/test.bin");
symlink(&outside, root.join("escape")).unwrap();
for path in [outside, root.join("escape"), root.clone()] {
assert!(filebrowser_read_path(dir.path(), &path).await.is_err());
}
}
#[test]
fn user_namespace_bytes_use_the_same_range_rules() {
assert!(matches!(
slice_prepared_content(
vec![],
Some(ByteRange {
start: 0,
end: None
}),
"x".into()
)
.unwrap(),
ServeResult::RangeNotSatisfiable(0)
));
assert!(
matches!(slice_prepared_content(b"abc".to_vec(), Some(ByteRange { start: 1, end: None }), "x".into()).unwrap(), ServeResult::Partial { bytes, start: 1, end: 2, total: 3, .. } if bytes == b"bc")
);
}
}
+38 -1
View File
@@ -194,6 +194,7 @@ pub async fn clear_user_stopped(data_dir: &Path, name: &str) {
// Installation is a decision, not a runtime observation, so it gets a record
// of its own that no amount of downtime erodes.
const INSTALLED_APPS_FILE: &str = "installed-apps.json";
static INSTALLED_APPS_LOCK: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(());
/// Load the durable set of installed app ids / container names.
pub async fn load_installed_apps(data_dir: &Path) -> std::collections::HashSet<String> {
@@ -220,12 +221,23 @@ pub async fn load_installed_apps_if_recorded(
async fn save_installed_apps(data_dir: &Path, installed: &std::collections::HashSet<String>) {
let path = data_dir.join(INSTALLED_APPS_FILE);
if let Ok(json) = serde_json::to_string_pretty(installed) {
let _ = fs::write(&path, json).await;
let tmp = path.with_extension("json.tmp");
let result = async {
fs::write(&tmp, json).await?;
fs::File::open(&tmp).await?.sync_all().await?;
fs::rename(&tmp, &path).await?;
fs::File::open(data_dir).await?.sync_all().await
}
.await;
if let Err(error) = result {
warn!(%error, "could not persist installed apps");
}
}
}
/// Record that an app is installed. Called when an install succeeds.
pub async fn mark_installed(data_dir: &Path, name: &str) {
let _guard = INSTALLED_APPS_LOCK.lock().await;
let mut installed = load_installed_apps(data_dir).await;
if installed.insert(name.to_string()) {
save_installed_apps(data_dir, &installed).await;
@@ -235,6 +247,7 @@ pub async fn mark_installed(data_dir: &Path, name: &str) {
/// Forget an app. Called on uninstall, beside `mark_user_uninstalled` — the
/// two must move together or a reinstall-after-uninstall leaves a stale claim.
pub async fn clear_installed(data_dir: &Path, name: &str) {
let _guard = INSTALLED_APPS_LOCK.lock().await;
let mut installed = load_installed_apps(data_dir).await;
if installed.remove(name) {
save_installed_apps(data_dir, &installed).await;
@@ -252,6 +265,7 @@ pub async fn clear_installed(data_dir: &Path, name: &str) {
/// need it. Runs on every boot, so an app installed before the upgrade is
/// still picked up whenever it is next seen alive.
pub async fn backfill_installed_apps(data_dir: &Path, present_container_names: &[String]) {
let _guard = INSTALLED_APPS_LOCK.lock().await;
if present_container_names.is_empty() {
return;
}
@@ -1497,3 +1511,26 @@ mod tests {
);
}
}
#[cfg(test)]
mod installed_concurrency_tests {
use super::*;
#[tokio::test]
async fn concurrent_install_records_are_not_lost() {
let dir = tempfile::tempdir().unwrap();
let mut tasks = Vec::new();
for i in 0..24 {
let path = dir.path().to_owned();
tasks.push(tokio::spawn(async move {
mark_installed(&path, &format!("app-{i}")).await;
}));
}
for task in tasks {
task.await.unwrap();
}
assert_eq!(load_installed_apps(dir.path()).await.len(), 24);
clear_installed(dir.path(), "app-3").await;
assert_eq!(load_installed_apps(dir.path()).await.len(), 23);
assert!(!dir.path().join("installed-apps.json.tmp").exists());
}
}
+6
View File
@@ -146,6 +146,10 @@ pub enum PackageState {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct PackageDataEntry {
/// Whether the app's HTTP upstream answered this scan (independent of
/// container health and blockchain sync). Missing on older nodes.
#[serde(rename = "ui-ready", default, skip_serializing_if = "Option::is_none")]
pub ui_ready: Option<bool>,
pub state: PackageState,
/// Container health: "healthy", "unhealthy", "starting", or null
#[serde(skip_serializing_if = "Option::is_none")]
@@ -297,6 +301,8 @@ pub enum InstallPhase {
/// `podman pull` in progress (the longest phase — up to several
/// minutes for large images on slow networks).
PullingImage,
/// Orchestrator owns download/build and startup as one operation.
PreparingApp,
/// Creating data directories, writing app-specific configs
/// (bitcoin.conf, lnd.conf, searxng settings.yml, chown).
CreatingContainer,
+40 -3
View File
@@ -5,8 +5,45 @@
//! are reachable over the mesh; ports of apps that aren't installed have
//! no listener, so allowing them is inert.
#[rustfmt::skip]
pub const APP_LAUNCH_PORTS: &[u16] = &[
2283, 2342, 3000, 3001, 3002, 4080, 5180, 7778, 8080, 8081, 8082, 8083, 8084, 8085, 8087, 8090,
8096, 8123, 8175, 8176, 8187, 8240, 8334, 8336, 8337, 8888, 8999, 9000, 9100, 10380, 11434,
18081, 18083, 18091, 23000, 32838, 50002,
2283,
2342,
3000,
3001,
3002,
4080,
5180,
7778,
8080,
8081,
8082,
8083,
8084,
8085,
8087,
8090,
8091,
8096,
8123,
8175,
8176,
8187,
8240,
8334,
8336,
8337,
8888,
8998,
8999,
9000,
9100,
10380,
11434,
18081,
18083,
18091,
23000,
32838,
50002,
];
+268 -7
View File
@@ -46,6 +46,25 @@ fn fips_should_fall_back(status: reqwest::StatusCode) -> bool {
status == reqwest::StatusCode::NOT_FOUND || status.is_server_error()
}
/// Is this FIPS answer the final one, or should the request go again over
/// Tor? A single-delivery request already reached the peer, so any answer
/// is final: a Tor replay would carry the same (possibly spent) payload.
fn fips_answer_is_final(
pref: crate::settings::transport::TransportPref,
single_delivery: bool,
status: reqwest::StatusCode,
) -> bool {
pref == crate::settings::transport::TransportPref::Fips
|| single_delivery
|| !fips_should_fall_back(status)
}
/// May a failed FIPS attempt be sent again? Only a failed connect proves the
/// peer never saw it; a timeout can land after the request was delivered.
fn fips_retryable(single_delivery: bool, e: &reqwest::Error) -> bool {
e.is_connect() || (!single_delivery && e.is_timeout())
}
/// DNS suffix appended to a peer's bech32 npub.
pub const FIPS_DNS_SUFFIX: &str = "fips";
@@ -113,7 +132,21 @@ pub fn client() -> reqwest::Client {
/// before the Tor fallback ever gets a chance. The generous `connect_timeout`
/// is preserved so a cold hole-punched path still gets time to establish.
pub fn client_with_timeout(timeout: Duration) -> reqwest::Client {
client_with_delivery_policy(timeout, false)
}
fn delivery_redirect_policy(single: bool) -> reqwest::redirect::Policy {
if single {
reqwest::redirect::Policy::none()
} else {
reqwest::redirect::Policy::default()
}
}
fn client_with_delivery_policy(timeout: Duration, single: bool) -> reqwest::Client {
reqwest::Client::builder()
.no_proxy()
.redirect(delivery_redirect_policy(single))
.timeout(timeout)
.connect_timeout(Duration::from_secs(8))
.user_agent("archipelago-fips/1")
@@ -130,10 +163,18 @@ pub fn client_with_timeout(timeout: Duration) -> reqwest::Client {
/// robust". Only connect/timeout errors are retried (a real HTTP response,
/// including 4xx/5xx, is returned as-is for the caller to interpret).
async fn send_with_retry(rb: reqwest::RequestBuilder) -> Result<reqwest::Response, reqwest::Error> {
send_with_retry_if(rb, |e| e.is_connect() || e.is_timeout()).await
}
/// [`send_with_retry`], retrying only on errors `retryable` accepts.
async fn send_with_retry_if(
rb: reqwest::RequestBuilder,
retryable: impl Fn(&reqwest::Error) -> bool,
) -> Result<reqwest::Response, reqwest::Error> {
let retry = rb.try_clone();
match rb.send().await {
Ok(resp) => Ok(resp),
Err(e) if (e.is_connect() || e.is_timeout()) && retry.is_some() => {
Err(e) if retryable(&e) && retry.is_some() => {
// Brief pause so the hole-punch packets from the first attempt can
// traverse before we re-dial onto the warmed path.
tokio::time::sleep(Duration::from_millis(600)).await;
@@ -350,6 +391,9 @@ pub struct PeerRequest<'a> {
/// the per-peer FIPS/Tor badge reflects reality. Opt-in because not
/// every caller has a data dir in scope.
pub record_data_dir: Option<std::path::PathBuf>,
/// The request carries something that must reach the peer at most once
/// (a bearer ecash token). See [`PeerRequest::single_delivery`].
pub single_delivery: bool,
}
impl<'a> PeerRequest<'a> {
@@ -363,9 +407,25 @@ impl<'a> PeerRequest<'a> {
fips_timeout: None,
service: None,
record_data_dir: None,
single_delivery: false,
}
}
/// Never send this request twice. A paid download carries a bearer ecash
/// token that the seller redeems on first sight; replaying it over Tor
/// after FIPS already delivered it hands the seller a spent token, so the
/// buyer is charged and gets a 402 instead of the file (2026-09-29: FIPS
/// answered 404 after the seller redeemed, the Tor retry got 402).
///
/// With this set, whatever FIPS answers is final, the FIPS retry fires
/// only when the first attempt never connected, and Tor is used only when
/// FIPS could not have delivered the request. An attempt that may have
/// been delivered but timed out is an error, not a fallback.
pub fn single_delivery(mut self) -> Self {
self.single_delivery = true;
self
}
/// Record the transport that serves this request into federation storage
/// (matched by this request's onion host). Best-effort, off the hot path.
pub fn record_transport(mut self, data_dir: impl Into<std::path::PathBuf>) -> Self {
@@ -442,7 +502,7 @@ impl<'a> PeerRequest<'a> {
// Use the FIPS reply unless it's one a Tor retry could
// fix (404 path-not-served / 5xx) and we're allowed to
// fall back. FIPS-only never falls back.
if pref == TransportPref::Fips || !fips_should_fall_back(resp.status()) {
if fips_answer_is_final(pref, self.single_delivery, resp.status()) {
telemetry::record_fips_ok();
self.spawn_record(crate::transport::TransportKind::Fips);
return Ok((resp, crate::transport::TransportKind::Fips));
@@ -481,7 +541,7 @@ impl<'a> PeerRequest<'a> {
if matches!(pref, TransportPref::Auto | TransportPref::Fips) {
match self.try_fips_get().await? {
Some(resp) => {
if pref == TransportPref::Fips || !fips_should_fall_back(resp.status()) {
if fips_answer_is_final(pref, self.single_delivery, resp.status()) {
telemetry::record_fips_ok();
self.spawn_record(crate::transport::TransportKind::Fips);
return Ok((resp, crate::transport::TransportKind::Fips));
@@ -551,13 +611,21 @@ impl<'a> PeerRequest<'a> {
} else {
budget
};
let c = client_with_timeout(per_attempt);
let c = client_with_delivery_policy(per_attempt, self.single_delivery);
let mut rb = c.post(&url).json(body);
for (k, v) in &self.headers {
rb = rb.header(*k, v);
}
match tokio::time::timeout(budget, send_with_retry(rb)).await {
let single = self.single_delivery;
let attempt = send_with_retry_if(rb, |e| fips_retryable(single, e));
match tokio::time::timeout(budget, attempt).await {
Ok(Ok(r)) => Ok(Some(r)),
Ok(Err(e)) if single && !e.is_connect() => Err(anyhow::anyhow!(
"FIPS POST failed after possible delivery; not replaying: {e}"
)),
Err(_) if single => Err(anyhow::anyhow!(
"FIPS POST exceeded its budget after possible delivery; not replaying"
)),
Ok(Err(e)) => {
telemetry::record_fallback(FallbackReason::ConnectFail);
tracing::info!(
@@ -612,13 +680,28 @@ impl<'a> PeerRequest<'a> {
} else {
budget
};
let c = client_with_timeout(per_attempt);
let c = client_with_delivery_policy(per_attempt, self.single_delivery);
let mut rb = c.get(&url);
for (k, v) in &self.headers {
rb = rb.header(*k, v);
}
match tokio::time::timeout(budget, send_with_retry(rb)).await {
let single = self.single_delivery;
let attempt = send_with_retry_if(rb, |e| fips_retryable(single, e));
match tokio::time::timeout(budget, attempt).await {
Ok(Ok(r)) => Ok(Some(r)),
// Anything but a failed connect may have reached the peer.
Ok(Err(e)) if single && !e.is_connect() => Err(anyhow::anyhow!(
"FIPS GET {} failed after the request may have been delivered \
(not retrying over Tor): {}",
self.path,
e
)),
Err(_) if single => Err(anyhow::anyhow!(
"FIPS GET {} exceeded its {:?} budget after the request may have \
been delivered (not retrying over Tor)",
self.path,
budget
)),
Ok(Err(e)) => {
telemetry::record_fallback(FallbackReason::ConnectFail);
tracing::info!(
@@ -676,6 +759,7 @@ impl<'a> PeerRequest<'a> {
.context("Invalid Tor SOCKS proxy URL")?;
reqwest::Client::builder()
.proxy(proxy)
.redirect(delivery_redirect_policy(self.single_delivery))
.timeout(self.timeout)
.build()
.context("Build Tor HTTP client")
@@ -759,4 +843,181 @@ mod tests {
let err = decode_response(0xAABB, &r, "x").unwrap_err();
assert!(err.to_string().contains("no AAAA"));
}
#[test]
fn a_single_delivery_answer_is_final_whatever_its_status() {
use crate::settings::transport::TransportPref;
use reqwest::StatusCode;
// Regression (2026-09-29): the seller redeemed a paid download's
// token, answered 404, and the Tor fallback replayed the spent token.
for status in [
StatusCode::NOT_FOUND,
StatusCode::INTERNAL_SERVER_ERROR,
StatusCode::SERVICE_UNAVAILABLE,
StatusCode::OK,
] {
assert!(fips_answer_is_final(TransportPref::Auto, true, status));
}
// Everything else keeps the existing fallback rules.
assert!(!fips_answer_is_final(
TransportPref::Auto,
false,
StatusCode::NOT_FOUND
));
assert!(!fips_answer_is_final(
TransportPref::Auto,
false,
StatusCode::BAD_GATEWAY
));
assert!(fips_answer_is_final(
TransportPref::Auto,
false,
StatusCode::PAYMENT_REQUIRED
));
assert!(fips_answer_is_final(
TransportPref::Fips,
false,
StatusCode::NOT_FOUND
));
}
/// A listener that accepts connections and never answers, counting them.
async fn silent_peer() -> (String, std::sync::Arc<std::sync::atomic::AtomicUsize>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let seen = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let counter = seen.clone();
tokio::spawn(async move {
let mut held = Vec::new();
while let Ok((stream, _)) = listener.accept().await {
counter.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
held.push(stream); // keep it open, never reply
}
});
(format!("http://{addr}/content/x"), seen)
}
#[tokio::test]
async fn a_single_delivery_request_is_not_resent_after_a_timeout() {
let (url, seen) = silent_peer().await;
let c = client_with_timeout(Duration::from_millis(300));
let err = send_with_retry_if(c.get(&url), |e| fips_retryable(true, e))
.await
.expect_err("peer never answers");
assert!(err.is_timeout());
assert_eq!(seen.load(std::sync::atomic::Ordering::SeqCst), 1);
}
#[tokio::test]
async fn an_ordinary_request_is_still_retried_once_after_a_timeout() {
let (url, seen) = silent_peer().await;
let c = client_with_timeout(Duration::from_millis(300));
let _ = send_with_retry_if(c.get(&url), |e| fips_retryable(false, e)).await;
assert_eq!(seen.load(std::sync::atomic::Ordering::SeqCst), 2);
}
#[tokio::test]
async fn a_single_delivery_request_still_retries_a_refused_connect() {
// Nothing listening: the peer provably never saw the request.
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
drop(listener);
let c = client_with_timeout(Duration::from_millis(500));
let err = send_with_retry_if(c.get(format!("http://{addr}/")), |e| {
fips_retryable(true, e)
})
.await
.expect_err("nothing listening");
assert!(err.is_connect());
assert!(fips_retryable(true, &err));
}
}
#[cfg(test)]
mod delivery_redirect_tests {
use super::*;
use hyper::{
service::{make_service_fn, service_fn},
Body, Response, Server,
};
use std::{
convert::Infallible,
sync::{
atomic::{AtomicUsize, Ordering},
Arc,
},
};
#[tokio::test]
async fn paid_bearer_request_does_not_follow_redirects_but_normal_get_does() {
let seen = Arc::new(AtomicUsize::new(0));
let counter = seen.clone();
let server = Server::bind(&([127, 0, 0, 1], 0).into());
let address = server.local_addr();
let service = make_service_fn(move |_| {
let counter = counter.clone();
async move {
Ok::<_, Infallible>(service_fn(move |request: hyper::Request<Body>| {
let counter = counter.clone();
async move {
counter.fetch_add(1, Ordering::SeqCst);
let response = if request.uri().path() == "/first" {
Response::builder()
.status(302)
.header("Location", "/replay")
.body(Body::empty())
.unwrap()
} else {
Response::new(Body::from("replayed"))
};
Ok::<_, Infallible>(response)
}
}))
}
});
let task = tokio::spawn(server.serve(service));
let url = format!("http://{address}/first");
let response = client_with_delivery_policy(Duration::from_secs(2), true)
.get(&url)
.header("X-Payment-Token", "dummy-test-token")
.send()
.await
.unwrap();
assert_eq!(response.status(), reqwest::StatusCode::FOUND);
assert_eq!(seen.load(Ordering::SeqCst), 1);
let response = client_with_delivery_policy(Duration::from_secs(2), false)
.get(url)
.send()
.await
.unwrap();
assert_eq!(response.status(), reqwest::StatusCode::OK);
assert_eq!(seen.load(Ordering::SeqCst), 3);
task.abort();
}
#[tokio::test]
async fn paid_request_is_not_resent_when_peer_disconnects_after_reading_it() {
use tokio::io::AsyncReadExt;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let seen = Arc::new(AtomicUsize::new(0));
let counter = seen.clone();
let task = tokio::spawn(async move {
while let Ok((mut stream, _)) = listener.accept().await {
let mut buf = [0; 4096];
let _ = stream.read(&mut buf).await;
counter.fetch_add(1, Ordering::SeqCst);
drop(stream);
}
});
let c = client_with_delivery_policy(Duration::from_secs(2), true);
let error = send_with_retry_if(c.get(format!("http://{address}/")), |e| {
fips_retryable(true, e)
})
.await
.unwrap_err();
assert!(!error.is_connect());
assert_eq!(seen.load(Ordering::SeqCst), 1);
task.abort();
}
}
+4
View File
@@ -256,6 +256,10 @@ async fn main() -> Result<()> {
boot_report.recovered, boot_report.total, boot_report.failed
);
}
// Disk manifests must be stable before the initial load and all later
// catalog reloads. Do not move this into the background doctor bootstrap.
bootstrap::ensure_runtime_assets_ready().await;
// Construct the container orchestrator once. In prod mode we load the
// on-disk app manifests, do an initial adoption pass, and spawn the
// BootReconciler loop (Step 5/6 of the rust-orchestrator migration).
+86 -18
View File
@@ -1765,12 +1765,17 @@ fn merge_preserving_transitional(
};
crate::data_model::PackageDataEntry {
state,
state: state.clone(),
// install_progress and uninstall_stage are also owned by the
// initiating op (same reason as state) — keep them.
install_progress: existing.install_progress.clone(),
uninstall_stage: existing.uninstall_stage.clone(),
// Everything else comes from the fresh scan.
ui_ready: if state == crate::data_model::PackageState::Running {
fresh.ui_ready
} else {
Some(false)
},
health: fresh.health.clone(),
exit_code: fresh.exit_code,
static_files: fresh.static_files.clone(),
@@ -1809,7 +1814,10 @@ async fn scan_and_update_packages(
absence_tracker: &mut HashMap<String, u32>,
transitional_since: &mut HashMap<String, Instant>,
) -> Result<()> {
let mut packages = scanner.scan_containers().await?;
let (before_scan, _) = state.get_snapshot().await;
let mut packages = scanner
.scan_containers(data_dir, &before_scan.package_data)
.await?;
let user_stopped = crate::crash_recovery::load_user_stopped(data_dir).await;
for (id, pkg) in packages.iter_mut() {
if pkg.state == crate::data_model::PackageState::Exited && user_stopped.contains(id) {
@@ -1870,11 +1878,14 @@ async fn scan_and_update_packages(
// once at load ~2). Better to keep saying "scanning…" than to say "empty".
if packages.is_empty() && (!first_scan || !installed_registry.is_empty()) {
if tor_changed || update_changed {
let mut data = current_data;
data.server_info.tor_address = tor_addr.clone();
data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
data.server_info.status_info.updated = update_available;
state.update_data(data).await;
state
.mutate_data(|data| {
data.server_info.tor_address = tor_addr.clone();
data.server_info.node_address =
tor_addr.as_ref().map(|t| identity.node_address(t));
data.server_info.status_info.updated = update_available;
})
.await;
}
return Ok(());
}
@@ -1899,6 +1910,13 @@ async fn scan_and_update_packages(
// died without cleanup and let the scan override it.
let now = Instant::now();
for (id, pkg) in &packages {
if user_uninstalled.contains(id)
|| user_uninstalled.contains(&format!("archy-{id}"))
|| (before_scan.package_data.contains_key(id)
&& !current_data.package_data.contains_key(id))
{
continue;
}
absence_tracker.remove(id);
let existing = merged.get(id);
let overwrite = match existing {
@@ -2054,22 +2072,40 @@ async fn scan_and_update_packages(
}
if changed || tor_changed || first_scan || update_changed {
let mut data = current_data;
data.package_data = merged;
data.server_info.tor_address = tor_addr.clone();
data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
data.server_info.status_info.containers_scanned = true;
data.server_info.status_info.updated = update_available;
state.update_data(data).await;
debug!(
"📦 State changed (packages={}, tor={}, first_scan={}, update={}), broadcasting update",
changed, tor_changed, first_scan, update_changed
);
state
.mutate_data(|data| {
// A lifecycle operation may have started/finished while this scan
// awaited probes or disk I/O. Never overwrite that newer entry or
// resurrect one that an uninstall removed in the meantime.
apply_scanned_packages(&mut data.package_data, &current_data.package_data, &merged);
data.server_info.tor_address = tor_addr.clone();
data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
data.server_info.status_info.containers_scanned = true;
data.server_info.status_info.updated = update_available;
})
.await;
}
Ok(())
}
fn apply_scanned_packages(
latest: &mut HashMap<String, crate::data_model::PackageDataEntry>,
base: &HashMap<String, crate::data_model::PackageDataEntry>,
scanned: &HashMap<String, crate::data_model::PackageDataEntry>,
) {
for (id, fresh) in scanned {
if latest.get(id) == base.get(id) {
latest.insert(id.clone(), fresh.clone());
}
}
for id in base.keys() {
if !scanned.contains_key(id) && latest.get(id) == base.get(id) {
latest.remove(id);
}
}
}
async fn normalize_reachable_package_health(
packages: &mut HashMap<String, crate::data_model::PackageDataEntry>,
) {
@@ -2268,6 +2304,7 @@ mod merge_tests {
fn make_entry(state: PackageState, health: Option<&str>) -> PackageDataEntry {
PackageDataEntry {
ui_ready: None,
state,
health: health.map(|s| s.to_string()),
exit_code: None,
@@ -2280,6 +2317,37 @@ mod merge_tests {
}
}
#[test]
fn stale_scan_cannot_remove_new_installs_or_overwrite_lifecycle_changes() {
let running = make_entry(PackageState::Running, Some("healthy"));
let restarting = make_entry(PackageState::Restarting, None);
let base = [
("restart".into(), running.clone()),
("uninstalled".into(), running.clone()),
]
.into_iter()
.collect();
let mut latest = [
("restart".into(), restarting.clone()),
("new".into(), running.clone()),
]
.into_iter()
.collect();
let scanned = [
("restart".into(), running.clone()),
("uninstalled".into(), running.clone()),
]
.into_iter()
.collect();
apply_scanned_packages(&mut latest, &base, &scanned);
assert_eq!(latest.get("restart"), Some(&restarting));
assert_eq!(latest.get("new"), Some(&running));
assert!(!latest.contains_key("uninstalled"));
apply_scanned_packages(&mut latest, &base, &HashMap::new());
assert_eq!(latest.get("restart"), Some(&restarting));
assert!(latest.contains_key("new"));
}
#[test]
fn peer_path_filter_allows_content_catalog_and_items() {
// Regression: the content *catalog* is exactly "/content" (no trailing
+41
View File
@@ -54,6 +54,21 @@ impl StateManager {
let _ = self.broadcast_tx.send(message);
}
/// Apply a small state change while holding the write lock. A lifecycle
/// task must not replace the entire model from an earlier snapshot.
pub async fn mutate_data<T>(&self, change: impl FnOnce(&mut DataModel) -> T) -> T {
let mut data = self.data.write().await;
let result = change(&mut data);
let mut rev = self.revision.write().await;
*rev += 1;
let _ = self.broadcast_tx.send(WebSocketMessage {
rev: *rev,
data: Some(data.clone()),
patch: None,
});
result
}
/// Get a WebSocket message with the current state
pub async fn get_initial_message(&self) -> WebSocketMessage {
let (data, rev) = self.get_snapshot().await;
@@ -190,3 +205,29 @@ mod tests {
assert_eq!(rev, 1);
}
}
#[cfg(test)]
mod atomic_mutation_tests {
use super::*;
#[tokio::test]
async fn concurrent_updates_preserve_independent_entries() {
let state = Arc::new(StateManager::new());
let mut tasks = Vec::new();
for i in 0..24 {
let state = state.clone();
tasks.push(tokio::spawn(async move {
state
.mutate_data(|data| {
data.peer_health.insert(format!("peer-{i}"), true);
})
.await;
}));
}
for task in tasks {
task.await.unwrap();
}
let (data, revision) = state.get_snapshot().await;
assert_eq!(data.peer_health.len(), 24);
assert_eq!(revision, 24);
}
}
+83 -29
View File
@@ -989,6 +989,24 @@ impl AppManifest {
validate_security(&self.app.security)?;
validate_ports(&self.app.ports)?;
validate_interfaces(&self.app.interfaces)?;
if let Some(value) = self.app.extensions.get("install_prerequisites") {
let items = value.as_sequence().ok_or_else(|| {
ManifestError::Invalid("install_prerequisites must be a list of app ids".into())
})?;
for item in items {
let id = item.as_str().unwrap_or_default();
if id.is_empty()
|| id == self.app.id
|| !id
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
{
return Err(ManifestError::Invalid(
"install_prerequisites must contain valid other app ids".into(),
));
}
}
}
validate_environment(&self.app.environment)?;
validate_devices(&self.app.devices)?;
@@ -1074,6 +1092,14 @@ impl AppManifest {
// `..` copy sources). See docs/manifest-hooks-design.md.
self.app.hooks.validate()?;
if let Some(value) = self.app.extensions.get("backup_before_runtime_change") {
if value.as_bool().is_none() {
return Err(ManifestError::Invalid(
"backup_before_runtime_change must be boolean".into(),
));
}
}
Ok(())
}
}
@@ -1111,6 +1137,7 @@ fn validate_security(policy: &SecurityPolicy) -> Result<(), ManifestError> {
"SETGID",
"SETUID",
"SYS_ADMIN",
"SYS_CHROOT",
];
let mut seen = HashSet::new();
for cap in &policy.capabilities {
@@ -1746,40 +1773,48 @@ app:
}
}
exempt.sort();
// 28 as of 2026-08-23: the 26 below plus cuprate's two exemptions —
// 18183 (Monero p2p gossip, same reasoning as bitcoin's 8333) and
// 18090 (host mapping for Monero's canonical 18089 restricted RPC,
// upstream's own safe-for-public
// subset that wallets connect to directly as a "remote node" over
// plain HTTP JSON-RPC — same reasoning as electrumx's 50001).
// cuprate's unrestricted RPC (full node control) stays loopback-only
// (auth: local), not in this set.
//
// 26 as of 2026-08-16: the 25 below plus phoenixd 9740, a
// loopback-only JSON API whose own generated http password
// authenticates every request (added with the phoenixd onboarding,
// which did not update this count — exactly the drift this test
// exists to catch).
//
// 25 as of the v1.7.123 port-policy round: bitcoin p2p (8333 ×2),
// core-lightning 9736/9835, electrumx 50001, fedimint 8173/8174,
// fedimint-gateway 8176/9737, gitea ssh 2222, lightning-stack
// 8091/9738/10010, lnd 9735/10009/18080, netbird 3478/8086/8087,
// pine TLS 10381 + the three voice ports (10200/10300/10400 — the
// disclosed known gap), router SSDP/mDNS 1900/5353. Every one is a
// deliberate, rationale-carrying exemption; the release-gate test
// stage timed out that cycle, so the count here lagged at 17.
// Reviewed 2026-09-30: lightning-stack's three retired endpoints
// disappeared; Cuprate restricted RPC moved from none to gate-open.
// Compare exact endpoints, not just a count that can hide substitutions.
let expected = [
("bitcoin-core", 8333),
("bitcoin-knots", 8333),
("core-lightning", 9736),
("core-lightning", 9835),
("cuprate", 18183),
("electrumx", 50001),
("fedimint", 8173),
("fedimint", 8174),
("fedimint-gateway", 8176),
("fedimint-gateway", 9737),
("gitea", 2222),
("lnd", 9735),
("lnd", 10009),
("lnd", 18080),
("netbird", 8087),
("netbird-server", 3478),
("netbird-server", 8086),
("phoenixd", 9740),
("pine", 10381),
("pine-openwakeword", 10400),
("pine-piper", 10200),
("pine-whisper", 10300),
("router", 1900),
("router", 5353),
]
.into_iter()
.map(|(id, port)| (id.to_owned(), port))
.collect::<Vec<_>>();
assert_eq!(
exempt.len(),
28,
"unauthenticated port set changed — review before updating this count: {exempt:?}"
exempt, expected,
"unauthenticated endpoint set changed; review each exemption"
);
}
/// `auth: open` ports are served by the gate WITHOUT its login challenge,
/// so they are the second unauthenticated-by-the-gate surface and get the
/// same review guard as `auth: none`. Each one must be an app that
/// enforces a real login of its own.
/// same review guard as `auth: none`. Each must enforce its own login or
/// have an explicitly reviewed public protocol purpose.
#[test]
fn gate_open_ports_are_all_accounted_for() {
let apps = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../apps");
@@ -1801,6 +1836,8 @@ app:
}
}
open.sort();
// Cuprate 18090 is its deliberately public restricted RPC subset;
// unrestricted node-control RPC remains container-loopback-only.
// Gitea 3001 (git clients speak basic-auth, not browser cookies),
// BTCPay 23000 (checkout/invoice/webhook endpoints must be reachable
// by anonymous payers), and — since the v1.8.7 platform round — the
@@ -1808,18 +1845,35 @@ app:
// nginx-proxy-manager 8081 (NPM admin accounts), tailscale 8240
// (tailnet login on the web console). Both enforce their own login,
// and an operator can re-gate either from Settings → Access control.
// Angor's indexer exposes public chain data/transaction broadcast;
// its optional standalone relay accepts signed public Nostr events.
// Neither mounts credentials or the node's internal relay database.
assert_eq!(
open,
vec![
("angor-indexer".to_string(), 8998u16),
("angor-relay".to_string(), 8091u16),
("btcpay-server".to_string(), 23000u16),
("cuprate".to_string(), 18090u16),
("gitea".to_string(), 3001u16),
("nginx-proxy-manager".to_string(), 8081u16),
("tailscale".to_string(), 8240u16),
],
"gate-open port set changed — every entry must be an app with its own login"
"gate-open port set changed — review login or intentional public protocol purpose"
);
}
#[test]
fn invalid_install_prerequisites_are_rejected() {
for value in ["not-a-list", "[demo]", "['../other']", "[false]", "['']"] {
let yaml = format!("app:\n id: demo\n name: Demo\n version: 1.0.0\n container:\n image: docker.io/library/alpine:3.20\n install_prerequisites: {value}\n");
assert!(AppManifest::parse(&yaml)
.unwrap_err()
.to_string()
.contains("install_prerequisites"));
}
}
#[test]
fn an_undeclared_port_classifies_as_session_but_is_not_declared() {
// Two different questions, and conflating them caused both gate
+38 -53
View File
@@ -310,59 +310,7 @@ impl PodmanClient {
);
continue;
}
// Honour the manifest's protocol (default tcp). netbird's STUN port
// is 3478/udp; forcing tcp here would publish the wrong protocol and
// silently break relay discovery.
let protocol = match port.protocol.to_ascii_lowercase().as_str() {
"udp" => "udp",
"sctp" => "sctp",
_ => "tcp",
};
// Effective bind. A gated port with no declared bind would
// publish 0.0.0.0 — the app would own every host address, which
// is both the exposure itself and the reason the daemon's app
// gate cannot bind those addresses to authenticate them. Pin it
// to loopback so the gate can take the external addresses.
//
// Doing it HERE, at container creation, is the point: the pin and
// the gate's takeover then both come from the daemon and cannot
// disagree. The earlier attempt put this decision in manifest
// data instead, and a node whose manifests lagged the binary
// published Bitcoin's loopback-only RPC across the LAN
// (test node, 2026-08-03).
//
// A port that already declares a bind is never overridden — that
// is exactly what keeps `bind: 127.0.0.1` ports host-local and
// leaves `auth: none` protocol ports (LND gRPC/REST, electrum)
// published as they are, so remote wallets keep working.
// NOTE: the daemon deliberately does NOT rewrite this. Pinning a
// published port to loopback is how an app hands its external
// addresses to the gate, but it belongs in the manifest, not in
// daemon-side inference:
//
// * `bind` is already honoured by every publish path (here and
// in package::install), so a manifest edit needs no code.
// * inference here would cover only THIS path — proven on
// a test node, where a recreate went through another one and
// the pin never applied.
// * and inferring from an ABSENT field is what republished
// Bitcoin's loopback RPC across the LAN, and came within one
// container-recreate of pinning LND's gRPC/REST and breaking
// every remote wallet.
//
// So the migration ships as `bind: 127.0.0.1` in the signed
// catalog. Verified 2026-08-03 that a disk-only manifest edit is
// overridden by the catalog, which is precisely why the catalog is
// the right and only place to carry it.
let mut mapping = serde_json::json!({
"container_port": port.container,
"host_port": port.host,
"protocol": protocol,
});
if !port.bind.is_empty() {
mapping["host_ip"] = serde_json::json!(port.bind);
}
port_mappings.push(mapping);
port_mappings.push(podman_publish_mapping(port));
}
let mut mounts = Vec::new();
@@ -751,6 +699,25 @@ pub fn image_uses_insecure_registry(image: &str) -> bool {
.is_some_and(|host| INSECURE_REGISTRY_HOSTS.contains(&host))
}
// Keep the explicitly declared bind and transport identical to Quadlet. The
// app gate owns external listeners; container publication must not bypass it.
fn podman_publish_mapping(port: &crate::manifest::PortMapping) -> serde_json::Value {
let protocol = match port.protocol.to_ascii_lowercase().as_str() {
"udp" => "udp",
"sctp" => "sctp",
_ => "tcp",
};
let mut mapping = serde_json::json!({
"container_port": port.container,
"host_port": port.host,
"protocol": protocol,
});
if !port.bind.is_empty() {
mapping["host_ip"] = serde_json::json!(port.bind);
}
mapping
}
fn podman_network_settings(
network: Option<&str>,
network_policy: &str,
@@ -1110,6 +1077,24 @@ mod tests {
));
}
#[test]
fn portainer_manifest_keeps_private_network_and_loopback_api_publication() {
let m = AppManifest::parse(include_str!("../../../apps/portainer/manifest.yml")).unwrap();
assert_eq!(
podman_network_settings(
m.app.container.network.as_deref(),
&m.app.security.network_policy
),
("slirp4netns", None)
);
assert_eq!(
podman_publish_mapping(&m.app.ports[0]),
serde_json::json!({
"container_port": 9000, "host_port": 9000, "protocol": "tcp", "host_ip": "127.0.0.1"
})
);
}
#[test]
fn podman_network_settings_uses_networks_map_for_custom_networks() {
assert_eq!(
+56 -19
View File
@@ -618,6 +618,46 @@ impl DockerRuntime {
}
}
// Docker is a development fallback. Refuse Podman-only network modes instead
// of silently installing a different topology; still honor binds for other apps.
fn docker_network_and_ports(manifest: &AppManifest, offset: u16) -> Result<Vec<String>> {
let network = manifest
.app
.container
.network
.as_deref()
.filter(|v| !v.is_empty())
.unwrap_or(&manifest.app.security.network_policy);
if matches!(network, "slirp4netns" | "pasta") {
anyhow::bail!("this app requires rootless Podman networking ({network})");
}
let mut args = Vec::new();
if !network.is_empty() && network != "isolated" {
args.extend(["--network".to_owned(), network.to_owned()]);
}
for port in &manifest.app.ports {
let host = port
.host
.checked_add(offset)
.context("published port offset overflow")?;
let bind = if port.bind.is_empty() {
String::new()
} else {
format!("{}:", port.bind)
};
let protocol = if port.protocol.is_empty() {
"tcp"
} else {
&port.protocol
};
args.extend([
"-p".to_owned(),
format!("{bind}{host}:{}/{protocol}", port.container),
]);
}
Ok(args)
}
#[async_trait]
impl ContainerRuntime for DockerRuntime {
async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
@@ -657,25 +697,7 @@ impl ContainerRuntime for DockerRuntime {
cmd.arg("--read-only");
}
match manifest.app.security.network_policy.as_str() {
"host" => {
cmd.arg("--network").arg("host");
}
"isolated" => {
// Docker uses bridge network by default
}
_ => {
cmd.arg("--network")
.arg(&manifest.app.security.network_policy);
}
}
// Port mappings with offset
for port in &manifest.app.ports {
let host_port = port.host + port_offset;
cmd.arg("-p")
.arg(format!("{}:{}", host_port, port.container));
}
cmd.args(docker_network_and_ports(manifest, port_offset)?);
// Volumes
for volume in &manifest.app.volumes {
@@ -1035,6 +1057,21 @@ mod tests {
use super::*;
use std::collections::HashMap;
#[test]
fn docker_fallback_rejects_rootless_only_topology_and_preserves_bind_protocol() {
let mut m =
AppManifest::parse(include_str!("../../../apps/portainer/manifest.yml")).unwrap();
assert!(docker_network_and_ports(&m, 0).is_err());
m.app.container.network = Some("bridge".into());
m.app.ports[0].protocol = "udp".into();
let args = docker_network_and_ports(&m, 1).unwrap();
assert_eq!(
args,
vec!["--network", "bridge", "-p", "127.0.0.1:9001:9000/udp"]
);
assert!(docker_network_and_ports(&m, u16::MAX).is_err());
}
#[test]
fn missing_container_classifier_covers_podman5_phrasings() {
// podman 5.x `inspect` phrasing for a missing container.
+5 -3
View File
@@ -16,9 +16,11 @@ lookup relays from the defaults. It does not replace GitWorkshop's NIP-34,
GRASP, repository browser, issue, pull-request, or review interfaces.
The separate dependency patch refreshes the npm lockfile and moves `fflate` to
0.8.3, `react-router-dom` to 7.18.3, and Vitest to 5.0.0. The resulting clean
install reports zero npm advisories; its type-check, 152 unit tests, and
Archipelago subpath production build pass. Keeping this mechanical security
0.8.3, `react-router-dom` to 7.18.3, and Vitest to 5.0.0. On 2026-09-30 the lockfile was refreshed again for `brace-expansion`
1.1.21/5.0.12, `fast-uri` 3.1.8 and `ip-address` 10.7.2 after fresh node
installs failed the retained dependency audit. The resulting clean install
reports zero npm advisories; its type-check, 152 unit tests, and Archipelago
subpath production build pass. The complete image also builds on the X250. Keeping this mechanical security
update separate makes both the upstream integration and future dependency
refreshes auditable.
@@ -1,5 +1,5 @@
diff --git a/package-lock.json b/package-lock.json
index 20631bb..0933917 100644
index 20631bb..86b6f86 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -63,7 +63,7 @@
@@ -495,9 +495,9 @@ index 20631bb..0933917 100644
- "version": "5.0.7",
- "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz",
- "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==",
+ "version": "5.0.9",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+ "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
+ "version": "5.0.12",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz",
+ "integrity": "sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -678,9 +678,9 @@ index 20631bb..0933917 100644
- "version": "1.1.15",
- "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.15.tgz",
- "integrity": "sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==",
+ "version": "1.1.18",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
+ "integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
+ "version": "1.1.21",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
+ "integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -833,9 +833,9 @@ index 20631bb..0933917 100644
- "version": "3.1.3",
- "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.3.tgz",
- "integrity": "sha512-i70LwGWUduXqzicKXWshooq+sWL1K3WUU5rKZNG/0i3a1OSoX3HqhH5WbWwTmqWfor4urUakGPiRQcleRZTwOg==",
+ "version": "3.1.7",
+ "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.7.tgz",
+ "integrity": "sha512-dOvZVzjdZdz7phd9v6jCbwxrBW3fK6n8Rc0CtdmM4bumzMnxywBYhuph6J819RRw/ku+rLbelwfMunktuzVVHg==",
+ "version": "3.1.8",
+ "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.8.tgz",
+ "integrity": "sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==",
"funding": [
{
"type": "github",
@@ -859,9 +859,9 @@ index 20631bb..0933917 100644
- "version": "5.0.7",
- "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz",
- "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==",
+ "version": "5.0.9",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+ "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
+ "version": "5.0.12",
+ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz",
+ "integrity": "sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -893,9 +893,9 @@ index 20631bb..0933917 100644
- "version": "10.2.0",
- "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
- "integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
+ "version": "10.7.0",
+ "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.7.0.tgz",
+ "integrity": "sha512-BGFsyJd5mpXp3rK6jIdADLNgpJUK1jnjzvYF8lK+VyDab9JAmqN0YOKDdP17HlgKb2+ehPgDc8EtnRLbGCAMhA==",
+ "version": "10.7.2",
+ "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.7.2.tgz",
+ "integrity": "sha512-7H/2gFSIitxc0hG3nOI1glS8QLo/EHBFFLk8vEUjXY/xu0AdL8jZ9U1IzO2PUm0d2D/ofQcAifb0g6OBkt8U7w==",
"license": "MIT",
"engines": {
"node": ">= 12"
@@ -1445,4 +1445,3 @@ index bd7190c..6aa5a6f 100644
import { vi } from "vitest";
// Mock window.matchMedia
+76 -13
View File
@@ -3,29 +3,92 @@
Working backlog of forward-looking items not yet scoped into a dedicated plan
doc. See [`ROADMAP.md`](ROADMAP.md) for the curated, public-facing direction.
## Blocking incident — before unrelated work
## Framework incident — closed with operator acceptance
- **OPEN: Framework LND startup / missing Receive address / false zero balance.**
User requires investigation and a verified fix on the actual node before later
unrelated work. Startup and native balances were verified on the actual node;
final display confirmation is pending. See the incident record for evidence.
- **CLOSED WITH OPERATOR ACCEPTANCE (2026-09-30): Framework LND startup /
missing Receive address / false zero balance.** Startup, native balances,
Cashu address and source integration were verified; the operator accepted the
remaining display check and authorized release. See the incident record for evidence.
See [incident evidence and closure criteria](incident-framework-lnd-startup.md)
and the repository `AGENTS.md` session-start instructions.
## Current repair and release tasks — 2026-09-29
## Next release after 1.8.21 — reported 2026-09-30
Release is blocked until these pass; see [execution record](repair-release-20260929.md).
Release status and acceptance gates: [execution checklist](next-release-20260930.md).
- [ ] Fix Cashu paid-file redemption between dev and Shorty; test keyset IDs,
- [ ] **Release blocker: Gitea → Portainer repository integration.** Diagnose
smart-HTTP reachability from Portainer's actual request namespace, then provide
one declarative topology and idempotent migration for fresh installs and
existing nodes. Preserve gate/auth boundaries, operator configuration,
repository/key/database mounts and Portainer stacks. Cover install order,
lifecycle/reboot/update convergence, clone/push and source-branch/Compose-file
acceptance with a disposable integration setup. Ship in both OTA and ISO;
a healthy Gitea root page is insufficient. Operator supplied a private handover;
deployment addresses and credentials must not be committed.
- [ ] **New X250: GitWorkshop failed at 70%; slow Nginx installation.** Missing
ISO build contexts restored on-node; package staging/smoke checks added.
GitWorkshop dependency audit refreshed and build/HTTP recovery verified;
Nginx was a slow successful image pull. Aggregate progress label corrected.
Include the validated repair in the next OTA/ISO. See lifecycle evidence.
- [ ] **Angor indexer service in the app store**, requested after the other
current repair/review work (2026-09-30). Follow the repository's app-development
and packaging documentation; treat it as a headless service unless upstream
documentation establishes a UI. Verify Bitcoin/Mempool requirements, decide
whether an existing first-class relay meets Angor's requirements or a relay
must be packaged with the indexer, and use the Angor logo from angor.io for its
service icon. Official current deployment documentation located and reviewed: stock Mempool
plus an optional strfry relay. Both headless services and the dependency guard
are implemented; API outage/recovery and five relay lifecycle cycles passed.
Final candidate install/lifecycle checks and signed delivery remain pending.
Install on the development box; Bitcoin must finish syncing for indexed queries.
- [ ] **App lifecycle: keep installed apps visible through restart and hard
refresh; gate embedded/browser launches on actual web and listener readiness.**
Source repair and scoped live acceptance passed; full release gate pending.
Includes durable inventory reconstruction,
concurrent inventory writes, stale scan/lifecycle updates, delayed HTTP startup,
and the app gate's post-install listener delay. See
[app lifecycle repair evidence](app-lifecycle-repair-20260930.md).
- [x] Review and repair open paid-download PRs #161 and #162, refresh both
branches from main, run independent and combined isolated suites, and verify
rootless file permissions in disposable scratch storage. Combined result:
1,585 passed, zero failed, four existing tests ignored. See the
[review evidence and remaining acceptance work](pr-review-20260930.md).
- [x] Integrate the reviewed PR branches into the next release and run funded
candidate acceptance, including Tor-only transport and payments with change.
Operator authorized completing the normal merge/closure workflow on
2026-09-30. Both PRs are now merged and closed through Gitea; integrate
local repair commits and sync git/ngit before release. The combined candidate
is deployed on both test endpoints. Funded Tor-only purchase with change,
confirmed refund, exact Files bytes and zero-cost repeat delivery passed.
The updated source still needs inclusion in signed OTA/ISO artifacts.
- [ ] Design durable recovery for an accepted payment whose response is lost.
Preserve the truthful unconfirmed-refund warning and prevent automatic
duplicate payment while that recovery work is outstanding.
- [x] **ThinkPad X250 kiosk: Bitcoin version choices readable above pruning.**
Replaced the native popup with inline radio choices. Actual Chromium 152 kiosk
assertions and screenshot verify white-on-dark choices, selection changes and
layout above pruning controls. Focused component tests pass. Included in the
next-release source; published 1.8.21 artifacts remain unchanged.
## 1.8.21 repair and release tasks — completed 2026-09-30
See the [execution record](repair-release-20260929.md) for evidence and limits.
- [x] Fix Cashu paid-file redemption between dev and Shorty; test keyset IDs,
mint errors, fees, and refund reporting before live validation.
- [ ] Complete the remaining Framework incident verification and evidence.
- [ ] Replace the unavailable tx1138.com explorer default with mempool.space;
- [x] Record Framework verification and the operator's acceptance of the
remaining display check before release.
- [x] Replace the unavailable tx1138.com explorer default with mempool.space;
migrate the old default with fresh consent and preserve custom/local explorers.
- [ ] Offer pruning in the Bitcoin installation version modal, using the same
- [x] Offer pruning in the Bitcoin installation version modal, using the same
pruning settings as automatic pruning even on large disks.
- [ ] Explain Bitcoin warmup without raw RPC errors; gate LND unlock on Bitcoin
- [x] Explain Bitcoin warmup without raw RPC errors; gate LND unlock on Bitcoin
RPC readiness and show install/start/sync waiting states with automatic recovery.
- [ ] Test the completed changes on this development box, then publish a new
- [x] Test the completed changes on this development box, then publish a new
signed OTA and raw ISO release. Record any remaining verification gaps.
## Dev & build process (priority)
+10
View File
@@ -765,3 +765,13 @@ Every supported app must satisfy the lifecycle contract:
For apps with special dependencies, launch must explain dependency wait states instead of showing a dead iframe. Examples include Bitcoin sync/IBD, Lightning wallet readiness, Nostr signer bridge injection, Tailscale login/auth, and app-specific setup screens.
Runtime changes should be validated with focused tests first, then the release lifecycle harness on the validation host when host access is intentionally resumed.
### Adapters for shared services
A service that reuses an installed stack can declare `install_prerequisites`
with the required component app ids and keep the runtime relationship in
`dependencies`. This refuses an incomplete installation before creating the
adapter instead of reporting a successful installation with no usable backend.
For example, Angor Indexer requires `mempool-api` (shown to users as its owning
Mempool app), shares that index and declares only an `api` interface. API-only
interfaces belong in Services and do not generate browser launch buttons.
+111
View File
@@ -0,0 +1,111 @@
# App lifecycle repair — 2026-09-30
Status: source repairs, optimized build, new-node recovery and scoped live
lifecycle acceptance verified. Full release gate remains pending.
These are next-release changes. Published 1.8.21 artifacts remain unchanged.
## Report
The operator reports that restarting an app can make it disappear, and a hard
refresh offers installation again. Newly installed apps sometimes fail to
connect in both embedded views and browser tabs. The new X250 additionally reproduced GitWorkshop disappearing during install
and Nginx Proxy Manager spending approximately 14 minutes at 70%. A disposable
app on the dev box exposed a separate restart failure.
## Findings and repairs
- Quadlet removes containers during stop/restart. The scanner protected existing
in-memory entries but did not reconstruct an absent app on a fresh daemon.
It now synthesizes stopped entries from the durable installed set, respecting
uninstall records, normalizing container prefixes, and preserving cached
metadata. Absence does not establish an image version or available update.
- Concurrent read/modify/write operations could lose installed-app records;
in-place writes could expose truncated JSON to readers. Serialize writers,
publish by atomic rename, and sync the file and parent directory. Legacy
package install/uninstall success paths update the durable record too.
- Scans and lifecycle/progress operations could replace a newer model from an
older snapshot. Use locked mutations for lifecycle/progress, and merge scan
results only into entries unchanged since the scan's merge snapshot.
- Container running state and TCP accept alone did not establish HTTP readiness.
Add explicit `ui-ready` based on bounded HTTP probes of the loopback upstream;
reject connection failures and server errors, accept normal redirects and
authentication challenges, and do not follow redirects or send credentials.
Self-signed HTTPS apps are probed locally without certificate validation.
- The app gate swept new listeners only every 60 seconds. Wake that sweep
immediately for a ready upstream whose declared gate port is not yet claimed,
and withhold readiness until external and Tor listener claims exist.
- Fixed launch URLs could bypass suppressed runtime URLs. Enforce readiness in
app cards, details, centralized embedded/browser launchers, and session frames.
Starting/restarting clears readiness immediately. A waiting frame does not
load an iframe and resumes when the backend reports readiness.
### New X250 findings
- The published ISO copied only `bitcoin-ui`, `lnd-ui` and `electrs-ui` build
directories. GitWorkshop failed because `/opt/archipelago/docker/archipelago-source`
was missing. Copy the complete docker source tree for bundled and unbundled
ISOs, matching OTA packaging. Validate every manifest build context and
Dockerfile in OTA staging, ISO staging and the mounted ISO smoke test.
- After restoring the omitted contexts, GitWorkshop's retained npm audit rejected
newly reported brace-expansion, fast-uri and ip-address vulnerabilities.
Refresh the existing pinned dependency patch, keeping the audit enabled.
Clean install/audit (zero advisories), type-check, 152 upstream tests and
subpath production build pass. The image builds on the X250 and `/healthz`
returns 200. No wallet or Bitcoin container restart was needed.
- Nginx was receiving data, not frozen: over 1 GB read during the pull. It
completed at 12:40:46 UTC after starting at 12:26:27; its web endpoint returns
200. The orchestrated path previously labelled the entire download/build/start
operation "Creating container" at 70%. Give that aggregate operation its own
truthful label and earlier phase; no byte-level download estimate is claimed.
- Restore install progress immediately from an already-loaded server snapshot,
so a new store created after hard refresh does not wait for another mutation.
- Replace the install modal's native version popup with inline radio choices.
On this actual X250's Chromium 152 kiosk renderer, selection changes work,
options have white text on dark backgrounds, and remain above pruning controls.
Screenshot and browser assertions captured; no install confirmation was clicked.
### Restart safety
The disposable fixture restart at 12:38:05 UTC stopped its container, then
`ss | kill` in runtime port cleanup sent SIGTERM to the management daemon at
12:38:35. The daemon owned the gate listener on the same port at other addresses.
Systemd restarted management; Bitcoin and LND container IDs/start times were
unchanged. Remove port-owner kills and broad `pkill` patterns from restart,
install recovery and Grafana preparation. Recovery now uses the existing
container-ID-aware ghost reaper: absent container ownership must be established
before a process is terminated. A real listening-socket regression checks that
conflict cleanup preserves the host listener. App-gate manifest lookup now honors
`ARCHIPELAGO_APPS_DIR`, matching the orchestrator's configured manifest root.
## Validation
- Full frontend suite: 139 files, 1,126 tests passed; final focused kiosk/store
checks: nine passed. Production frontend build passed.
- Final isolated backend suite: 1,567 passed, zero failed, four existing ignored
tests. Optimized backend build passed and was deployed to the development node.
- Tests cover empty runtime inventory, alias deduplication, uninstall exclusion,
concurrent durable writes, concurrent state changes, stale scan publication,
TCP-without-HTTP, HTTP statuses including 502/503, and gate listener claims.
- Live disposable Node fixture delayed HTTP startup by 25 seconds. Desktop and
mobile retained the waiting screen through hard refresh without mounting an
iframe, then opened the exact fixture page automatically when ready.
- Restart retained the app in both state APIs throughout and returned to ready;
the management PID did not change. Stopping removed the Quadlet container;
restarting management reconstructed its installed/stopped entry without a
false update offer. Starting it again succeeded. Desktop and mobile continued
to show the installed app after hard refresh.
- LAN access required node authentication and returned exact fixture bytes after
authentication. The fixture was uninstalled through the package lifecycle API;
its temporary manifest root and service override were removed.
- Bitcoin and LND container IDs and start times stayed unchanged through all
scoped checks and management restarts. No wallet data was used by the fixture.
- X250 kiosk checks also opened the repaired GitWorkshop and Nginx Proxy Manager
pages successfully, with no failed local resource loads.
## Limits
This prevents the identified lifecycle/readiness failures; it cannot guarantee
that an app or network never fails after a successful readiness check. Actual
application failures must remain visible rather than being labelled successful.
The full lifecycle/reboot release gate and funded acceptance of the reviewed
paid-download PRs remain pending. The X250 kiosk fix has live rendering evidence.
+32
View File
@@ -290,3 +290,35 @@ app:
Validate with `scripts/validate-app-manifest.sh` and regenerate the catalog
with `scripts/generate-app-catalog.py` (drift-checked in CI by
`scripts/check-app-catalog-drift.py`).
### Persistent-state backup for runtime repairs
`app.backup_before_runtime_change: true` opts an app into a stopped-state snapshot
before reconciliation changes a service’s network, ports, security settings,
command or health configuration. Image-upgrade backup policy remains separate. The orchestrator
archives writable persistent bind mounts under the node data directory, collapses
nested mounts, excludes the runtime Podman socket, and preserves the previous
Quadlet definition for rollback. Named volumes, outside-data-root state and
symlinked mount roots fail closed rather than silently producing an incomplete
backup. A failed snapshot resumes the original service and leaves migration
pending. Private archives are retained under `migration-backups/`; fresh installs
and unchanged runtime configurations do not create migration snapshots.
Catalog generation preserves the previously published base manifest for older
daemons and puts opted-in network changes in a signed `manifest_variants` entry
requiring `runtime-migration-backup-v1`. New runtimes select only variants whose
complete requirement list they support. Supply `BASE_CATALOG` when generating
against a different reviewed pre-migration catalog. This keeps catalog refresh
from applying a migration before the matching OTA code is installed.
### Existing shared-service prerequisites
`app.install_prerequisites` is an optional list of existing app ids, for example
`[mempool-api]` for a headless indexer adapter. The runtime checks their manifest
container names before recording installation or changing any dependency. If one
is missing, installation refuses with its owning app's title and removes the
optimistic install tile. Runtime observation errors fail closed. This does not
automatically install dependencies, alter Bitcoin pruning, or require a synced
backend merely to recognize an already-installed service. Declare ongoing
relationships separately in `dependencies`; use the app health check for actual
API readiness. Self-dependencies and malformed ids are invalid.
+167
View File
@@ -0,0 +1,167 @@
# Same-node Gitea sources in Portainer
Status: root cause reproduced and network repair verified in disposable and actual
production Portainer instances; final migration integration and release acceptance remain in progress.
This change belongs to the next signed catalog, OTA and ISO. It does not modify
published 1.8.21 artifacts.
## Confirmed cause
On the affected X250, Gitea 1.27.3 and Portainer 2.45.0 run in rootless Podman
5.4.2, managed by user Quadlet services. Gitea publishes HTTP on loopback and the
Archipelago app gate serves its public port. Gitea's public ROOT_URL already
matches that gate URL.
Portainer had no explicit network selection and Podman selected pasta. Its
network namespace contained the host's LAN address. A Git request to that same
LAN address therefore reached Portainer's namespace rather than the host gate:
connection refused before authentication. The exact smart-HTTP request from the
host returned 200 with `application/x-git-upload-pack-advertisement`. From
Portainer's actual namespace the LAN request was refused, while its host mapping
returned a Git advertisement and the expected branch tip. Direct container-IP
requests timed out. Container health and host-only HTTP checks missed the defect.
A disposable Portainer using `slirp4netns` successfully created a Source through
Portainer's own API, using the original LAN clone URL. Returning that fixture to
pasta reproduced the refusal; recreating with slirp repaired it while preserving
its account and saved Source. Restart also passed. The requested branch tip and
Compose file were read from that actual Portainer network namespace. No user
stack was deployed. Deployment addresses and repository details are kept outside
this public record.
## Source changes
- Declare Portainer's rootless `slirp4netns` mode in its manifest. No shared static
container IP, host networking, all-interface backend publication or auth bypass.
- Keep Gitea's loopback HTTP backend and gate port; machine Git uses Gitea's
authentication. Remove obsolete port-3000 nginx metadata/template and the old
best-effort installer commands which silently rewrote app.ini and falsely
claimed success. Gitea owns first-run setup and operator configuration.
- Gitea SSH also failed before authentication: OpenSSH logged a denied
`chroot("/var/empty")` because the manifest dropped `SYS_CHROOT`. Add that
specific sandbox capability and reconcile security-directive changes. A
disposable fixture then passed SSH clone/push with host-key checking enabled.
- Existing Quadlet reconciliation applies Network= drift. Record a durable
pending restart before updating the unit and clear it only after a successful
restart, so failed reloads/restarts and management interruptions retry.
- Detect explicit rootless network-mode drift in the older Podman runtime too.
Unspecified networks do not trigger inferred changes to unrelated apps.
- Portainer and Gitea opt into `backup_before_runtime_change`. Before recreation, gracefully
stop the app and archive its writable persistent bind mounts, including nested
Compose state, once each. Runtime sockets are excluded. Save the previous
Quadlet definition, where present. Archives live under the node data directory's
private `migration-backups/<id>/` directory; state is never deleted. Backup
failures resume the original service and fail the migration visibly.
- Keep Podman API and Quadlet bind/network behavior covered by actual-manifest
tests. Docker remains a development fallback: it now preserves bind/protocol
declarations and rejects Podman-only networking instead of silently changing it.
## Operator use and diagnostics
Use Gitea's advertised HTTP(S) clone URL in Portainer Sources, with the Gitea
username and token in the credential fields. On first-run Gitea setup, the public
base URL must match the origin opened through Archipelago (including its port).
Keep a deliberately configured HTTPS/domain origin when one exists. Do not use a
container IP or put a token into the URL. A private repository requires repository
read permission. A successful Source check fetches Git refs; it does not deploy
a stack or establish that a Compose build uses a desired application revision.
`scripts/check-portainer-git-source.py` calls Portainer's own read-only Source
connection test. Supply a private mode-600 JSON credential file containing
`api_key` or `jwt`, and optionally `git: {username, password}`. Pass
`--portainer-url`, `--repository-url` and `--credentials-file`. It does not create
Sources or stacks and prints no credentials or raw server errors. It distinguishes
Portainer login/API failures from Git connection refusal, timeout, DNS/TLS
failure, HTML/login interception and repository authentication failure. TLS
verification stays enabled and API redirects are refused.
## Upgrade and rollback
The signed catalog embeds manifests and overrides installed disk copies.
Capability-gated manifest variants keep the previous Portainer manifest as the
base for older daemons; only daemons supporting `runtime-migration-backup-v1`
select the network repair. This prevents catalog refresh from triggering an
unbacked recreation before the OTA is installed. A disk
edit alone cannot deliver this fix. Publish the matching catalog with the tested
runtime, then verify the generated unit, actual network mode and Source API.
Expect a Portainer interruption while the snapshot and recreation run; duration
depends on its saved state size.
The Portainer routing repair does not require a Gitea configuration change.
The separate SSH capability repair does recreate Gitea, preserving and snapshotting
both data/config mounts first. Supported systemd drop-in overrides remain intact.
Keep the previous trusted catalog/runtime for rollback. Restore that catalog
before restoring the saved `previous.container`, reloading user systemd and
starting Portainer; otherwise reconciliation will correctly reapply the new
manifest. The archive is a stopped-state emergency backup, not an instruction to
roll back a live database automatically. Restore it only with Portainer stopped
and after preserving any newer state. Do not replace Gitea data/config, keys,
repositories or the production Portainer database with disposable test data.
## Validation and remaining gates
- Disposable X250 Portainer Source API: old mode refuses; repaired mode succeeds;
saved account/Source survive recreation; restart succeeds.
- Invalid Git credentials produce a repository-authentication error, distinct
from TCP refusal. Requested branch and Compose file read from Portainer context.
- Combined backend suite including the reviewed paid-download PRs and catalog
rollout guard: 1,605 passed, zero failed, four existing ignored
tests, including stopped-state archive round trips and failure preservation. Container runtime suite: 78 passed.
Five diagnostic regression tests passed; catalog regeneration is idempotent
and the generated catalog has zero manifest metadata drift.
- Fresh managed Gitea and Portainer fixtures: authenticated private Source
creation, invalid-token rejection, workstation clone/push and exact branch
lookup from Portainer namespace passed. LFS batch/upload/download and OCI
registry authentication/blob/manifest round trips passed. Desktop and mobile
login/private-repository/assets/hard-refresh checks passed.
- Still required before release: live automatic migration with the new runtime,
snapshot/rollback verification and reversed install-order acceptance,
lifecycle/reboot convergence, and signed-catalog delivery to the existing app.
Record LFS/registry/SSH/browser checks and actual hardware/runtime coverage.
### Affected X250: production routing repair verified
Applied the tested rootless network setting to the actual installed Portainer
through a persistent Quadlet drop-in, after gracefully stopping it and creating a
private archive of its database and Compose directory. Compared the archive
against the stopped original before changing configuration; retained the original
unit and a rollback path. A verification helper initially compared mount list
order rather than mount identity and safely rolled back; the corrected check
compares sorted source/destination/write-mode tuples and passed.
The actual production Portainer namespace reproduced connection refusal before
repair. After repair it received a Git smart-HTTP advertisement, fetched the
requested branch at its current tip and read its Compose file. Repeating these
checks after restarting the managed Portainer service passed. All original data
and socket mounts and the loopback-only HTTP binding are retained. Gitea,
Bitcoin and the wallet container IDs and start times were unchanged. No stack
was deployed and no repository credential was changed.
This establishes the routing repair on the affected hardware. A logged-in
production Portainer Source UI/API acceptance has not yet been recorded; the
corresponding API checks passed on disposable instances as documented above.
The installed-node drop-in persists through service restart/reboot but is not the
fleet delivery mechanism. Automatic migration and signed catalog/OTA/ISO release
validation remain pending; the source manifest declares the same network mode.
Private deployment addresses, branch details and state archives are not committed.
### Managed automatic migration and archive restore
The new runtime candidate migrated an existing managed fixture from pasta to
slirp without a manual unit edit. It preserved the account, saved Source and
mount set, saved a private stopped-state archive plus the previous unit, restored
Source API access, and cleared the pending restart marker. A management-service
restart preserved the new container identity/start time and did not create
another archive. The archive extracted into an isolated scratch directory and
compared cleanly, including the database and Compose directory. Rootless archive
ownership required scratch cleanup inside `podman unshare`; no production data
was overwritten. Native Bitcoin and LND IDs/start times remained unchanged.
This optimized candidate predates the final bounded backup-retry guard; that
latest source passed the isolated 1,605-test suite and must also be exercised in
the final release build. A fixture-only systemd start failure was then injected during a security
directive migration. The failure retained the durable restart marker. After
removing the injected failure, the reconciler restarted the service without a
manual container start, restored Source API access and cleared the marker.
Reverse install order, final-build retry-budget coverage, full reboot and
signed delivery remain open.
+227
View File
@@ -0,0 +1,227 @@
# Next OTA and raw ISO after 1.8.21
**Status: implementation and acceptance in progress; NOT ready to release.**
This is the consolidated execution checklist for the operator's chat requests.
A targeted node repair is not completion of the release. Finish the remaining
acceptance gates, preserve live wallets and app data, and publish both artifacts
through git and ngit. No universal absence of future failures is claimed.
## Changes already shipped in 1.8.21 or earlier
Keep these fixes in the next build and include relevant regressions:
- Mempool image/catalog version agreement and update-button behavior.
- Minibits integration; Framework automatic LND startup and safe unavailable
balances. Framework incident closed with operator acceptance.
- Shorter, single-column ecash backup messaging.
- AIUI transparent background on desktop/mobile.
- Cashu paid-file keyset/mint/error/refund corrections, with live purchases.
- mempool.space explorer fallback, preserving local/custom explorer settings.
- Bitcoin install pruning choice and matching automatic-pruning behavior.
- Friendly Bitcoin warmup and LND install/start/sync waiting states.
- Raw ISO publishing and upload support.
The Primal automatic LNURL comment problem was traced to sender behavior and
Minibits metadata. The user accepted clearing the sender's automatic comment;
no unsupported local metadata rewrite or wallet-identity replacement is planned.
See the Framework incident and 1.8.21 execution records for evidence/limits.
## New release scope and gates
| Task | Implemented/verified | Remaining before release |
| --- | --- | --- |
| X250 Bitcoin picker | Inline choices; actual Chromium kiosk selection, readability and pruning layout passed | Include in final UI/build checks |
| App disappearance/readiness | Durable inventory and safe lifecycle repair; delayed HTTP and desktop/mobile hard-refresh checks passed | Final lifecycle/reboot gate on candidate |
| X250 GitWorkshop/Nginx | Missing build contexts restored, dependency/build checks and live UI passed; Nginx slow pull diagnosed; truthful progress label | Verify both artifact payloads contain all build contexts |
| PRs 161/162 | Reviewed, repaired, merged/closed normally; combined regression suite passed | Funded Tor-only candidate purchase, retained change, refund, Files bytes and cached repeat passed; include in signed artifacts |
| Gitea/Portainer | Root cause confirmed; source network/backup/retry/catalog changes; real X250 routing repair and restart verified; private Git, SSH, LFS, registry and browser fixture checks passed | Automatic migration, scratch restore, failed-start recovery and reverse installation order passed. Operator confirms production site works through Portainer; production host reboot also preserved network/Git/Compose access; final candidate delivery and release checks remain |
| Angor headless store service | Implemented standard Mempool adapter and separate optional relay, official logo, headless store entries and declarative dependency guard. API security/outage/DNS tests and five relay lifecycle cycles passed | Final candidate prerequisite/install acceptance, management restart/reboot checks and signed catalog delivery; real indexing on dev waits for Bitcoin sync |
Durable payment receipts after a lost seller response remain a separately
recorded design follow-up. Preserve the truthful unconfirmed-refund warning and
prevent duplicate automatic payment; do not describe an unconfirmed refund as
completed. See PR review for the accepted scope and coverage limits.
## Final release checklist
- [ ] Finish all new-scope implementation and specific acceptance above.
- [x] Remove disposable fixtures and temporary test overrides; verify native
Bitcoin/LND identity and start-state baselines remain protected.
- [x] Commit and push completed source changes to git and ngit.
- [ ] Run final backend/UI/regression/release gates on the final source; inspect
skipped tests and report actual hardware/runtime coverage.
- [ ] Prepare compatible signed app catalog; old runtimes must not apply a
migration before they have backup/recovery support.
- [ ] Version/changelog and OTA payload prepared, validated and signed by user.
- [ ] Raw ISO built; payload hashes/content verified; installer boot tested.
- [ ] User signs ISO checksums; publish OTA and ISO plus verification files on
git and ngit; independently read back hashes and update discovery.
- [ ] Provide LAN scp command for the new raw ISO.
Latest backend source verification: 1,609 passed, zero failed, four existing
ignored tests. This is one layer of evidence, not a substitute for live gates.
## Angor verification — 2026-09-30
- Isolated backend suite: 1,606 passed, four existing ignored; container suite:
79 passed. Frontend: 140 files / 1,130 tests passed; production build passed.
- Disposable rootless API gateway: versioned and legacy API paths, query/body
forwarding, transaction-only POST, method/body limits, CORS, removal of
dashboard credentials, read-only non-root operation, truthful backend outage
and DNS recovery after backend recreation passed. No real transaction broadcast.
- Dedicated relay: NIP-11, signed event publish/read, invalid signature rejection
and event/config persistence across five managed stop/start/restart cycles
passed. Internal relay identity and start time stayed unchanged. Follow-up
acknowledgement samples were 2–9 ms through both backend and app gate.
- Published adapter 1.0.1 and relay 1.1.2 to the authenticated maintainer namespace.
Anonymous registry readback succeeded. Adapter digest:
`sha256:997be611700b55c521ad801fa92daaca2ae6951ac71407434c85eb9603f77c38`;
relay mirror digest:
`sha256:80444ad1304a0e504948b48ea1550c091b18b9f10757f07ce9a68fc261b8f6c1`.
- Delivery target is the development box, as clarified by the operator. Do not
install Angor on the separate Portainer node. Full indexer availability still
requires the dev box's Bitcoin sync and Mempool/Electrum indexing to finish.
- Funded PR acceptance passed after the operator funded the dev Cashu wallet
with 16 sats. Exact net payment was 1 sat; underpayment refunded in full;
repeat delivery cost zero. Both endpoints ran the combined candidate.
No spent proofs were reactivated and no native Bitcoin/LND funds were moved.
## Development candidate and cleanup
The combined optimized backend and production UI are deployed on the development
box with a private rollback copy. Native Bitcoin/LND containers were unchanged
during deployment. The operator separately uninstalled/reinstalled Bitcoin Core
to select an unpruned node; RPC confirmed `pruned=false`, and a separate baseline
was recorded after that operator action. Do not compare subsequent checks with
the pre-reinstall container start times.
Completed Gitea setup/private-repository and Portainer integration fixtures were
uninstalled through the supported lifecycle and removed from installed inventory.
Their private evidence/data were retained outside the active manifests. The old
Cuprate UI review container was also removed. Active Angor acceptance fixtures
must be removed on completion; the requested Angor services remain installed.
Funded acceptance used Tor-only peer-file transport, verified exact delivery
bytes and compatibility response fields, and read the result back through
FileBrowser. The original transport preference was restored, and temporary
seller catalog entries/files and the exact buyer test document were removed.
Financial receipt history was retained.
The final managed-install fixture exposed a separate Quadlet quoting defect:
whitespace-free command arguments containing apostrophes lost those characters
in the generated service. The renderer now quotes these arguments and
environment values; the updated isolated backend suite passed (1,607 passed, four opt-in tests
ignored), and the final candidate rebuild is in progress. Do not tag a release before this live regression is verified.
The production Portainer host subsequently rebooted after the routing repair.
A post-boot probe from the actual Portainer namespace again verified the Git
smart-HTTP response type, current branch ref and Compose contents. Its
slirp4netns route and all production app containers survived. The temporary
Portainer fixture was absent. This verifies the repaired production route
across reboot; it does not substitute for final new-runtime delivery checks.
## Follow-up acceptance: app cards and Angor icon
- Mempool duplicate traced to `archy-mempool-web` durable inventory alias being
restored beside the real `mempool` frontend. Shared scanner canonicalization
fixes live and absent-container paths without deleting installed markers.
Frontend suppresses aliases only while a canonical tile exists.
- Readiness text names the app and condition: “Web UI not ready: Gitea”. It shares the status row,
with full text available through its title; card actions use bottom alignment.
- Angor uses the operator-supplied dark-mode icon with green outer corners.
Built-in imagegen prompt: fill transparent/white corners with the existing
flat green, preserve the black symbol, square opaque PNG, no added details.
- Backend alias suite: 1608 passed, 4 ignored. Focused readiness/frame UI tests:
30 passed. Production UI build passed and is live on dev. Browser checks at
1440 and 1024 pixels verified named waiting text, bottom-aligned actions,
equal row heights, no overflow and one Mempool card after hard refresh.
The 390-pixel mobile icon layout also passed hard refresh. Final-source
isolated Mempool alias regression passed after the scanner simplification.
- Managed Angor adapter acceptance: five stop/start/restart cycles, missing
prerequisite refusal, management restart and cleanup all passed. Both temporary
fixtures and their network were removed. Actual dev API verification remains
pending after removing an incomplete legacy-created adapter.
## Startup manifest reload race
Live Angor acceptance exposed a separate startup race: runtime asset bootstrap
cleared and copied `/opt/archipelago/apps` in the background while the startup
catalog refresh reloaded it. The daemon logged 62 loaded manifests followed by
54 and then rejected the new disk-only app as unknown. A stable manifest snapshot
confirmed the diagnosis: supported uninstall/reinstall produced the correct
rootless Quadlet service with its declared port and network.
Runtime promotion and the legacy installer-directory repair now finish before
orchestrator construction. The background doctor no longer changes that tree.
The final source backend suite passed 1,608 tests (four existing opt-in tests
ignored). Optimized build and normal-path live startup/restart verification have now passed (see final follow-up below).
Actual dev Angor acceptance passed managed service identity, no capabilities,
UID 101:101, archy-net, public block height, CORS and both fee URL forms. During
Bitcoin initial sync, the real Mempool fee API returns 503; the adapter faithfully
returns the same status and body. Full-sync fee availability remains unverified;
ready-backend API and failure/recovery behavior passed the isolated live fixture.
The temporary `/run/archy-candidate-manifests` snapshot override and snapshot
are now removed; normal startup/reload verification passed.
The latest complete UI suite passed 140 files / 1,132 tests. Release preflight
passed all static, manifest, catalog, type and UI gates. The requested named
waiting message, compact card layout and green Angor icon are deployed to dev;
desktop 1440/1024 and mobile 390 browser checks passed after hard refresh.
The startup-order optimized build and normal-path startup checks are complete.
The later dashboard-address candidate is now deployed with rollback; see the
final live follow-up below. Do not rerun the earlier deployment helper: its
temporary override has already been removed. No new release version/tag, OTA
or ISO has been created.
## Mempool and dashboard follow-up
- Deployed the Mempool alias and runtime-promotion-order backend to dev. Real
server state contains one healthy `mempool`; the stale `mempool-web` record
is gone. Real-data browser checks at 1440/390 pixels found exactly one tile
before and after hard refresh. Bitcoin/LND identities/start times unchanged.
- Removed the candidate manifest override. Normal management startup passed
two full cycles with 62 manifests retained through both initial catalog
refreshes. The next cycle hit a single readiness assertion; a subsequent
read-only check found Angor healthy and the manifest count intact. Remaining
repeat coverage should use bounded polling to distinguish transient request
failures from loss of app definitions; do not report five cycles passed yet.
- Bitcoin Core's dashboard was serving HTTP 200 on 8334 while readiness checked
RPC 8332. Companion URL selection now takes priority over protocol sockets
for Core/Knots and Electrum aliases, with a regression preserving allocated
UI ports for other apps. Backend suite: 1,609 passed, four opt-in ignored.
Optimized build is `/tmp/archy-dashboard-address-build.log`; deployment and
live IBD verification helper: `/tmp/archy-dashboard-address-deploy.py`.
- Phoenixd has no browser UI. Headless services now omit web-readiness messages;
actual browser apps name their web interface rather than waiting for
themselves. Focused 23 UI tests and production build passed; deployed to dev.
## Final live follow-up: all three reported readiness/display defects fixed
- Final optimized backend is deployed on dev. Bitcoin Core's launch address is
`http://localhost:8334` and `ui-ready` is true during initial block download.
Live verification recorded height 293,855 with `initialblockdownload=true`.
Chromium at 1440 and 390 pixels opened the embedded dashboard, read a numeric
current block height, and repeated that check after hard refresh.
- One healthy Mempool remains in server state and in desktop/mobile My Apps
after hard refresh. Its durable install markers were preserved.
- Phoenixd remains a running headless service without a web launcher or false
web-readiness message. Desktop/mobile browser checks passed. For actual web
apps, the compact copy is “Web UI not ready: [app]”; the reason comes first so
narrower cards do not truncate it into a misleading self-dependency.
- Five normal management startup and managed Angor restart cycles passed
across the two acceptance logs. The retry harness uses bounded readiness
polling; it does not accept a running container alone as API readiness.
Disk + catalog manifest count remained 62 across startup refreshes, replacing
the previous 62-to-54 failure. Temporary override and snapshot are removed.
- Final backend tests: 1,609 passed, zero failed, four existing opt-in ignored.
Final UI tests: 140 files / 1,133 passed. Production UI build passed and is live.
Bitcoin/LND container identities and start timestamps stayed unchanged.
- Evidence: `/tmp/archy-dashboard-address-deploy.log`,
`/tmp/archy-bitcoin-ibd-browser.log`, `/tmp/archy-mempool-live-browser.log`,
`/tmp/archy-service-readiness-browser.log`,
`/tmp/archy-runtime-order-remaining-cycles.log`, and
`/tmp/archy-readiness-final-ui-tests.log`.
- These are live development fixes. The new signed catalog, versioned OTA and
raw ISO still need preparation, artifact verification, signing and publication.
+126
View File
@@ -0,0 +1,126 @@
# Paid-download PR review — 2026-09-30
## Scope and result
Reviewed both open PRs from the repository pull-request list: [#161](https://source.archipelago-foundation.org/lfg2025/archy/pulls/161)
and [#162](https://source.archipelago-foundation.org/lfg2025/archy/pulls/162).
Both branches were updated from main, repaired and tested independently and
together. Their existing remote branches were advanced without rewriting the
contributors' history. Both were subsequently merged and closed and are now
integrated on main. Candidate live-wallet acceptance remains pending.
The signed 1.8.21 artifacts are unchanged.
| Candidate | Tested commit | Isolated backend result |
| --- | --- | --- |
| PR #161 | `971d4777` | 1,576 passed, 0 failed, 4 existing tests ignored |
| PR #162 | `0677924a` | 1,568 passed, 0 failed, 4 existing tests ignored |
| Both together | `4bf4bf1a` | 1,585 passed, 0 failed, 4 existing tests ignored |
Both individual branches also passed production `cargo check`, with the
repository's existing 16 warnings. The combined merge required no conflict
resolution. Backend tests ran through `scripts/test-backend-isolated.sh` so they
could not access host wallets, native services or production container storage.
## Findings and repairs
### #161 — payment delivery and file readability
- The branch conflicted with newer mint-fee, keyset-ID and truthful refund
reporting fixes. Preserve those implementations from main; do not reintroduce
its older unconditional “refunded” messages or duplicate keyset resolution.
- Opening a file before charging, then reopening/reading it afterward, still
permits a read failure after payment. Prepare the complete requested bytes
before redemption, including ranged reads. Tests delete or alter the backing
file during payment verification and still receive the prepared original data.
- Empty/out-of-bounds/reversed ranges could fail after redemption, and empty
files could underflow the range calculation. Validate ranges before charging
and return HTTP 416 when unsatisfiable.
- `chmod a+r` unnecessarily changed the permissions of shared paid/private
files. Read restricted FileBrowser files through the rootless namespace while
retaining their mode. Scope the fallback to regular files canonically inside
FileBrowser storage, and reject unauthorized peers before reading.
- A single-delivery flag must also prevent redirects and ambiguous transport
retries. Payment-bearing GET and POST requests now retain the first HTTP
response and do not retry after timeouts or disconnects that might follow
delivery. Refused connections and normal nonpayment browsing retain the
appropriate retry behavior.
- Interrupted response bodies now report the outcome using the actual local
refund result. Seller explanations are bounded, stripped of control
characters and explicitly identified as peer text.
- Original permission tests silently returned when run as root. Replacement
tests inject read/payment boundary failures, exercise them under the isolated
runner, and assert that read failures never invoke redemption.
### #162 — saving purchases in Files
- Its host-permission repair overlapped 1.8.21's authenticated Files API path.
Review of [FileBrowser v2.63.23's resource handler](https://github.com/filebrowser/filebrowser/blob/v2.63.23/http/resource.go)
showed that `override=false` checks for existence separately from opening
with truncation. It does not guarantee no overwrites under concurrent saves.
- The proposed direct path exposed the final filename before the write
completed. Both direct and namespace paths now finish a private temporary
file and publish it through a no-clobber hard link, retrying numbered names.
- Plain `ln` could place a temporary file inside an existing directory instead
of treating the destination as a collision. Use `ln -T`; existing directories
and dangling symlinks are conflicts, never replacement targets.
- Add filename and destination checks, unique temporary names, bounded name
retries, synchronization before publication, and exact input-length checks.
Truncated pipe input cannot become a completed purchased file.
- Files storage remains optional. An unavailable copy destination does not
undo the purchase or create a fake FileBrowser installation; the durable
purchased-content cache remains primary.
## Additional verification on the development node
Used disposable scratch directories only, then removed them:
- Reproduced a FileBrowser-style rootless-owned 0640 upload. The host backend
UID could not read it. `podman unshare cat` returned identical bytes without
changing its 0640 mode.
- Ran the exact namespace writer script with four concurrent writers against
a directory owned by the container UID range. Every file had unique naming,
exact bytes, the expected owner and mode, and no remaining temporary file.
- Sent truncated input to the namespace writer and verified refusal, no final
file and temporary-file cleanup.
The isolated tests additionally exercised 24 simultaneous direct writes,
existing-file preservation, symlink/directory conflicts, collision exhaustion,
root-independent permission failures, read-before-redemption ordering,
authorization, redirects and peer disconnects.
Logs on the development box:
`/tmp/archy-pr161-tests.log`, `/tmp/archy-pr162-tests.log`,
`/tmp/archy-pr-integration-tests.log`, `/tmp/archy-pr161-check.log`,
`/tmp/archy-pr162-check.log`, `/tmp/archy-pr-userns-scratch-test.log`.
## Next-release acceptance and limits
- Both reviewed branches are integrated on main alongside the lifecycle fixes.
Repeat release gates against the final release commit after remaining changes.
- Perform funded peer-to-peer acceptance on the candidate build, including a
Tor-only purchase and a purchase requiring change, before the next release.
The new review branches were not deployed to funded live wallets here.
- These PRs do not implement durable payment receipts. If a seller redeems a
payment and the connection subsequently loses the response, the buyer may
receive an unconfirmed-refund warning. Do not represent that warning as proof
of a refund or automatically charge the buyer again. Receipt-based recovery
remains separate follow-up work.
- Abrupt process termination can leave a hidden namespace temporary file;
ordinary write failures and truncated input are tested to clean up. The final
filename is published only after complete input, and existing files remain
protected.
- The separately reported X250 kiosk selector is fixed and verified on the
actual kiosk; see the lifecycle evidence and consolidated release checklist.
## Authorized merge — 2026-09-30
The operator explicitly requested normal merged/closed PR status after review.
Re-read both PRs and verified their heads still exactly matched the reviewed
commits. Changes from the integration-test base to main were documentation only.
Gitea normal merges completed and read-back confirmed `merged=true`, `state=closed`:
- #161: `3daea6623be3e2c7222101b8e6ac411423c7e16c`.
- #162: `b02ba4100d922dd1b75c6a78121ef446c2159a54`.
Local next-release lifecycle work was integrated with main at `d69e8452`. Funded release acceptance and the documented delivery-receipt
limitation remain as recorded above; merging does not claim a new release.
+144 -3
View File
@@ -1,6 +1,9 @@
# Repair and release execution — 2026-09-29
**Status: IN PROGRESS. Do not publish an OTA or ISO until the release gates pass.**
**Status: 1.8.21 PUBLISHED — see the completion record at the end.**
The next release is tracked in [the current execution checklist](next-release-20260930.md).
The dated entries below preserve the investigation history.
User requires all tasks completed and tested on the development box before the
next OTA and raw ISO. Passing unit tests alone does not establish live correctness.
@@ -52,8 +55,8 @@ next OTA and raw ISO. Passing unit tests alone does not establish live correctne
- [x] Live waiting/UI verified on dev; recovery covered by deterministic tests.
- [x] Framework operator acceptance and authorization to release recorded.
- [x] Release version/changelog, catalog/image implications, signing prepared.
- [ ] Signed OTA built, tested, published to git and ngit.
- [ ] Raw ISO built, boot-tested, signed and published; download command supplied.
- [x] Signed OTA built, tested, published to git and ngit.
- [x] Raw ISO built, boot-tested, signed and published; download command supplied.
Tests must not wipe/recreate wallets, prune the operator's existing full chain,
or claim that arbitrary failures can never happen. Record material gaps before
@@ -257,3 +260,141 @@ Both catalog and OTA signatures verify against the pinned release root. Staged
artifact hash/size checks and catalog drift/trust checks pass. User accepted the
remaining Framework display check and explicitly authorized release. Publication
and ISO build may proceed; do not regenerate the signed manifest or artifacts.
### Published OTA; ISO withheld after live shutdown defect — 2026-09-30
Signed 1.8.20 OTA/catalog published to git and ngit, with public asset hashes
verified. Catalog rollout triggered a Bitcoin command update at 08:34 UTC.
Although the orchestrator allowed a long stop, Quadlet's generated Podman removal
still used its ten-second default and killed Bitcoin. Core replayed its block
index; LND later lost its connection to the previous Bitcoin container IP.
Stopped the ISO build and queued boot check; any partial 1.8.20 ISO is invalid
and must not be published. Preparing 1.8.21 to supersede the immutable signed OTA.
Installed explicit graceful-stop systemd overrides on dev and Shorty without
restarting native services. Candidate Quadlet fix adds per-app container, systemd,
and command-wait budgets, including existing containers and uninstall fallback.
Focused 43 tests pass, including actual Quadlet generator stop-before-remove order.
Full tests, disposable slow-stop verification, build and deployment remain pending.
Disposable live regression passed: started an Alpine container with its legacy
ten-second stop setting, rewrote and reloaded its Quadlet with explicit twenty-
second graceful stop, verified the same container ID and old internal timeout
remained running, then stopped it. Its twelve-second shutdown handler completed
in 12.6 seconds, emitted the completion marker, and exited without SIGKILL/137.
Fixture had no network or wallet mounts and was removed afterward.
Core finished index loading and resumed unpruned initial sync. LND automatically
unlocked at 08:47 UTC. The existing backend-address cascade then performed a
graceful LND restart at 08:57 UTC after Bitcoin reconciliation completed; LND
automatically unlocked again and reached chain-sync waiting. No manual wallet
unlock or restart was used for this recovery.
### False dependency restart exposed during monitoring — 09:08 UTC
The initial 1.8.21 candidate passed all 1,557 isolated backend tests and 1,120
frontend tests. Monitoring nevertheless found another managed LND restart at
09:08:32 while Bitcoin's container/start timestamp remained unchanged. Management
logs explicitly attribute it to the backend-address cascade. This also makes
the earlier 08:57 cascade suspect; it must not be described as a proven necessary
restart. These service restarts preceded the isolated test executable, whose
namespace boundaries remain intact.
The cascade trusted Started/Installed action reports. A failed runtime inspection
followed by successful systemctl start of an already active unit can produce
Started without changing Bitcoin. Dependency restarts now require observed
container-ID, running-state, or start-time changes. Failed observations remain
unknown, not absence; a known absent backend becoming running still qualifies.
Actual exec-drift restarts are recognized even when their outer report is NoOp.
Stopped/lifecycle-in-flight dependents remain excluded, and user stop markers
are re-read after the potentially slow pass. Added runtime-observation and
false-action/real-exec-drift regression cases; full isolated rerun pending.
Stopped the first optimized build and preparing new artifacts from this correction.
### Final 1.8.21 artifacts and live verification — 2026-09-30
Source and frontend/AIUI attribution: c993d9dd. Full isolated backend suite:
1,559 passed, zero failed, four existing hardware/live tests ignored. Frontend
suite: 1,120 passed; final production type-check/build passed after the last
release-note-only edit. Optimized backend built in 13m22s.
Staged unsigned 1.8.21 OTA manifest and artifacts:
- Backend: 64,748,176 bytes; SHA256
ff602e85f340aff7e43d9d94f7f84f11f713735c964c0d8ba150e23b065c30eb.
- Frontend archive: 97,152,546 bytes; SHA256
6c0842ec83a440269a353808a4cf154174f5232c9989b4a5448bc6486e1d0620.
Artifact validator passed. Actual archive has flat paths, readable root index,
and exact fresh AIUI, doctor and LND UI payload bytes. Exact files deployed to
dev at 09:32 UTC and Shorty at 09:35 UTC; rollback binaries and dashboards under
root-only /var/lib/archipelago/support/release-1821 on each node. Only management
services restarted. Existing Bitcoin/Core-or-Knots and native LND container IDs
and start times were preserved. Correct generated graceful-stop commands are
present before forced removal on both nodes; temporary grace overrides removed.
Dev systemd deadlines are 615 seconds for Bitcoin and 345 seconds for LND.
Desktop/mobile Lightning UI checks passed again: waiting for Bitcoin sync,
unknown balance, no unavailable native RPC requests. Served dashboard and AIUI
attribution bytes match the release. Native LND states: dev RPC_ACTIVE while
Bitcoin syncs; Shorty SERVER_ACTIVE. Dev completed full reconciliation passes
at 09:34:21 and 09:36:40 with Bitcoin/LND NoOp, and no dependency restart.
Shorty's first full pass completed 09:36:55 with Knots/LND NoOp.
Final paid-file check on these exact binaries passed: fresh one-sat dev-to-Shorty
purchase, exact one-sat refund on underpayment, identical response aliases,
correct Files copy, and zero-charge cached repeat. Removed temporary seller
entries/files and Files copy; retained purchase audit and owned cache. Total net
transfer across all three live payment rounds in this repair session: three sats.
Remaining: finish Shorty observation and remove temporary diagnostic logging;
user-local 1.8.21 OTA signature (existing catalog signature remains valid),
publish git/ngit, build/boot-test/sign and publish the raw 1.8.21 ISO.
No 1.8.21 release tag or public OTA yet. Do not publish the quarantined partial
1.8.20 ISO. The existing 1.8.20 git/ngit release notes now explain the withheld ISO
and pending hotfix; signed 1.8.20 assets remain immutable.
Shorty's second clean full pass completed at 09:38:06 UTC. Removed temporary
diagnostic logging on both nodes and restarted only management again; native
Bitcoin and LND IDs/start times remained unchanged, with generated stop settings
still verified. No temporary graceful-stop overrides remain. Catalog signature
verifies against the pinned release root; final 1.8.21 artifact validator passes.
The candidate is ready for the user's local OTA signing ceremony.
### 1.8.21 publication completed — 2026-09-30
The operator signed the OTA manifest and subsequently the ISO checksum JSON.
Both signatures verified against the pinned release root. The signed OTA was
published on git/ngit, and the operator confirmed that Framework could see the
update. Source main and the annotated `v1.8.21-alpha` tag were published.
Raw ISO:
`archipelago-installer-1.8.21-alpha-unbundled-x86_64_RC1.iso`
- Size: 2,682,419,200 bytes.
- SHA256: `8667b5522c476a40e29abba19df4180086191527a194a88765aa70ed527f9406`.
- Build and ISO smoke checks passed. The mounted backend matched the staged
OTA backend hash, and the full dashboard/AIUI tree matched the fresh build.
- An isolated UEFI QEMU guest, with no network or host disks attached, booted to
the installer prompt. The VM was stopped and the ISO unmounted afterward.
This was an installer boot check, not a full installation onto hardware.
- Uploaded the raw ISO, plain SHA256 sidecar and signed checksum JSON to the
[1.8.21 release](https://source.archipelago-foundation.org/lfg2025/archy/releases/tag/v1.8.21-alpha).
The stored server file hashes matched, the public ISO headers and first/last
byte samples matched, and both public checksum files matched byte-for-byte.
- The ngit downloader's full-ISO acquisition exceeded its fixed 30-minute
deadline on the available connection. Published Nostr asset records using
the already verified hashes, sizes and public URLs with the existing ngit
signer; both repository relays acknowledged them. Ngit then accepted those
records and final readback resolved all five release assets with the expected
hashes and sizes. No new release-root signing was performed by the assistant.
Final publication evidence: `/tmp/archy-1821-finish-events.log`,
`/tmp/archy-ngit-1821-complete-view.json`, and
`/tmp/archy-1821-verified-asset-events.log` on the development box.
Subsequent review of PRs #161/#162 found additional delivery and concurrent
file-save edge cases. Their repaired, tested branches are recorded in
[the next-release review](pr-review-20260930.md); those changes are not in the
signed 1.8.21 artifacts. The X250 kiosk selector report is also tracked for the
next release. No claim of exhaustive hardware or network-failure coverage is
made for this release.
@@ -2607,21 +2607,14 @@ if [ -f "$SCRIPT_DIR/../../scripts/image-versions.sh" ]; then
echo " ✅ Bundled image-versions.sh"
fi
# Bundle docker UI source files for building custom UIs on first boot
# Always bundle — these are tiny HTML/CSS files, not container images
if true; then
DOCKER_UI_DIR="$SCRIPT_DIR/../../docker"
if [ -d "$DOCKER_UI_DIR" ]; then
echo " Bundling docker UI source files..."
mkdir -p "$ARCH_DIR/docker"
for ui_dir in bitcoin-ui lnd-ui electrs-ui; do
if [ -d "$DOCKER_UI_DIR/$ui_dir" ]; then
cp -r "$DOCKER_UI_DIR/$ui_dir" "$ARCH_DIR/docker/"
echo " ✅ Bundled $ui_dir source"
fi
done
fi
fi
# Build-source apps need their complete contexts even on unbundled ISOs.
# Keep this identical to the OTA runtime payload; a per-app allowlist silently
# omitted GitWorkshop, FIPS and Cuprate and made fresh installs fail at 70%.
DOCKER_UI_DIR="$SCRIPT_DIR/../../docker"
[ -d "$DOCKER_UI_DIR" ] || { echo "Missing docker build sources" >&2; exit 1; }
mkdir -p "$ARCH_DIR/docker"
cp -a "$DOCKER_UI_DIR/." "$ARCH_DIR/docker/"
python3 "$SCRIPT_DIR/../../scripts/check-app-build-contexts.py" "$ARCH_DIR"
if [ "$UNBUNDLED" = "1" ]; then
echo " ✅ Unbundled build ready (Tor setup included, no container images)"
@@ -1,21 +0,0 @@
# Gitea iframe proxy — strips X-Frame-Options so Gitea works in Archipelago iframe.
# Gitea container binds to port 3001, this proxy listens on port 3000 (the public port).
# Deployed to /etc/nginx/conf.d/gitea-iframe.conf
server {
listen 3000;
server_name _;
client_max_body_size 1G;
location / {
proxy_pass http://127.0.0.1:3001;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_hide_header X-Frame-Options;
proxy_hide_header Content-Security-Policy;
}
}
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "neode-ui",
"version": "1.8.19-alpha",
"version": "1.8.21-alpha",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "neode-ui",
"version": "1.8.19-alpha",
"version": "1.8.21-alpha",
"dependencies": {
"@scure/bip39": "^2.2.0",
"@types/dompurify": "^3.0.5",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "neode-ui",
"private": true,
"version": "1.8.20-alpha",
"version": "1.8.21-alpha",
"type": "module",
"scripts": {
"start": "./start-dev.sh",
Binary file not shown.

After

Width:  |  Height:  |  Size: 948 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 24 KiB

+29
View File
@@ -644,6 +644,35 @@
"/var/lib/archipelago/vaultwarden:/data"
]
}
},
{
"id": "angor-indexer",
"title": "Angor Indexer",
"version": "1.0.1",
"description": "Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s address as the custom indexer in Angor settings. A relay is optional and installed separately.",
"dockerImage": "source.archipelago-foundation.org/chaum/angor-indexer:1.0.1",
"author": "Angor / Archipelago",
"requires": [
"Mempool API",
"Unpruned Bitcoin"
],
"category": "money",
"tier": "optional",
"icon": "/assets/img/app-icons/angor-green.png",
"repoUrl": "https://github.com/block-core/angor"
},
{
"id": "angor-relay",
"title": "Angor Relay",
"version": "1.1.2",
"description": "Optional dedicated Nostr relay for Angor project metadata. Separate storage and access settings keep the node’s internal relay private. Add this service’s address to Angor’s relay settings; use WSS for browser clients.",
"dockerImage": "source.archipelago-foundation.org/chaum/angor-relay:1.1.2",
"author": "Angor / Archipelago",
"requires": [],
"category": "nostr",
"tier": "optional",
"icon": "/assets/img/app-icons/angor-green.png",
"repoUrl": "https://github.com/hoytech/strfry"
}
]
}
@@ -25,13 +25,22 @@
<div v-if="loading" class="py-6 text-center text-white/60 text-sm">{{ t('common.loading') }}</div>
<div v-else class="space-y-2">
<label class="block text-white/60 text-sm">{{ t('appDetails.selectVersion') }}</label>
<select
v-model="selected"
class="w-full rounded-lg bg-white/[0.06] border border-white/10 text-white pl-3 pr-9 py-2 text-sm focus:outline-none focus:border-blue-400/60"
>
<option v-for="v in versions" :key="v.version" :value="v.version">{{ optionLabel(v) }}</option>
</select>
<fieldset class="space-y-2">
<legend class="text-white/60 text-sm mb-2">{{ t('appDetails.selectVersion') }}</legend>
<!-- Inline options avoid native popup rendering in the kiosk WebView.
They stay in document flow above the pruning explanation. -->
<div class="max-h-40 overflow-y-auto space-y-2 rounded-lg">
<label
v-for="v in versions"
:key="v.version"
class="flex items-center gap-3 rounded-lg border px-3 py-2.5 text-sm text-white cursor-pointer"
:class="selected === v.version ? 'border-blue-400/60 bg-slate-800' : 'border-white/10 bg-slate-900'"
>
<input v-model="selected" type="radio" :name="`install-version-${appId}`" :value="v.version" class="shrink-0 accent-blue-400" />
<span>{{ optionLabel(v) }}</span>
</label>
</div>
</fieldset>
<p class="text-white/40 text-xs">{{ t('marketplace.installModalHint') }}</p>
</div>
@@ -113,6 +122,7 @@ async function load() {
} catch (err) {
if (import.meta.env.DEV) console.warn('[InstallVersionModal] getPackageVersions failed:', err)
// Fall back to the floating "latest" so the install can still proceed.
versions.value = [{ version: 'latest' } as CatalogVersionInfo]
selected.value = 'latest'
} finally {
loading.value = false
@@ -16,12 +16,14 @@ describe('Bitcoin install storage choice', () => {
versions.mockResolvedValue({ bitcoinPrune: false, default: 'latest', versions: [{ version: 'latest' }, { version: '28.4' }] })
const wrapper = modal(id)
await flushPromises()
await wrapper.get('select').setValue('28.4')
await wrapper.get('input[type=radio][value="28.4"]').setValue(true)
await wrapper.get('input[type=checkbox]').setValue(true)
await wrapper.get('button').trigger('click')
expect(wrapper.emitted('confirm')).toEqual([['28.4', true]])
expect(wrapper.text()).toContain('automatic pruning')
expect(wrapper.text()).toContain('Mempool')
expect(wrapper.find('select').exists()).toBe(false)
expect(wrapper.findAll('input[type=radio]')).toHaveLength(2)
})
it('keeps automatic disk selection by default and resets on reopening', async () => {
versions.mockResolvedValue({ bitcoinPrune: false, versions: [{ version: 'latest' }] })
@@ -29,17 +31,17 @@ describe('Bitcoin install storage choice', () => {
await flushPromises()
await wrapper.get('button').trigger('click')
expect(wrapper.emitted('confirm')).toEqual([['latest', false]])
await wrapper.get('input').setValue(true)
await wrapper.get('input[type=checkbox]').setValue(true)
await wrapper.setProps({ show: false })
await wrapper.setProps({ show: true })
await flushPromises()
expect((wrapper.get('input').element as HTMLInputElement).checked).toBe(false)
expect((wrapper.get('input[type=checkbox]').element as HTMLInputElement).checked).toBe(false)
})
it('still allows choosing pruning when version lookup fails', async () => {
versions.mockRejectedValue(new Error('offline'))
const wrapper = modal()
await flushPromises()
await wrapper.get('input').setValue(true)
await wrapper.get('input[type=checkbox]').setValue(true)
await wrapper.get('button').trigger('click')
expect(wrapper.emitted('confirm')).toEqual([['latest', true]])
})
@@ -47,7 +49,7 @@ describe('Bitcoin install storage choice', () => {
versions.mockResolvedValue({ bitcoinPrune: true, versions: [{ version: 'latest' }] })
const wrapper = modal('bitcoin-knots')
await flushPromises()
expect((wrapper.get('input').element as HTMLInputElement).checked).toBe(true)
expect((wrapper.get('input[type=checkbox]').element as HTMLInputElement).checked).toBe(true)
await wrapper.get('button').trigger('click')
expect(wrapper.emitted('confirm')).toEqual([['latest', true]])
})
@@ -62,6 +64,6 @@ describe('Bitcoin install storage choice', () => {
versions.mockResolvedValue({ bitcoinPrune: false, versions: [{ version: 'latest' }] })
const wrapper = modal('other')
await flushPromises()
expect(wrapper.find('input').exists()).toBe(false)
expect(wrapper.find('input[type=checkbox]').exists()).toBe(false)
})
})
@@ -34,6 +34,7 @@ vi.mock('@/api/rpc-client', () => ({
vi.stubGlobal('open', mockWindowOpen)
import { useAppLauncherStore, senderMatchesApp } from '../appLauncher'
import { useAppStore } from '../app'
describe('useAppLauncherStore', () => {
beforeEach(() => {
@@ -54,6 +55,25 @@ describe('useAppLauncherStore', () => {
})
})
it('blocks both browser and embedded launch while HTTP is unready', () => {
const app = useAppStore()
app.data = { 'package-data': { gitea: { state: 'running', 'ui-ready': false, health: 'healthy', manifest: { id: 'gitea', title: 'Gitea' } } } } as never
const launcher = useAppLauncherStore()
launcher.openSession('gitea')
expect(launcher.panelAppId).toBeNull()
launcher.open({ url: 'http://192.0.2.10:3001/', title: 'Gitea', openInNewTab: true })
expect(mockWindowOpen).not.toHaveBeenCalled()
expect(launcher.isOpen).toBe(false)
})
it('also gates a dynamic app resolved through its runtime URL', () => {
useAppStore().data = { 'package-data': { custom: { state: 'running', 'ui-ready': false, manifest: { id: 'custom', title: 'Custom' }, installed: { 'interface-addresses': { main: { 'lan-address': 'http://localhost:18993/' } } } } } } as never
const launcher = useAppLauncherStore()
launcher.open({ url: 'http://192.0.2.10:18993/', title: 'Custom', openInNewTab: true })
expect(mockWindowOpen).not.toHaveBeenCalled()
expect(launcher.isOpen).toBe(false)
})
it('starts closed with empty state', () => {
const store = useAppLauncherStore()
expect(store.isOpen).toBe(false)
@@ -0,0 +1,42 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { createPinia, setActivePinia } from 'pinia'
import { reactive, nextTick } from 'vue'
const fake = reactive<{ packages: Record<string, unknown> }>({ packages: {} })
vi.mock('../sync', () => ({ useSyncStore: () => fake }))
vi.mock('../../api/rpc-client', () => ({ rpcClient: {} }))
import { useServerStore } from '../server'
function installing(phase = 'preparing-app') {
return { state: 'installing', manifest: { title: 'Git Workshop' }, 'install-progress': { phase, size: 0, downloaded: 0 } }
}
describe('installation state after hard refresh', () => {
beforeEach(() => {
setActivePinia(createPinia())
fake.packages = {}
})
it('restores an in-flight install from an already-loaded server snapshot', () => {
fake.packages = { 'archipelago-source': installing() }
const store = useServerStore()
expect(store.isInstalling('archipelago-source')).toBe(true)
expect(store.installingApps.get('archipelago-source')).toMatchObject({
progress: 20,
message: 'Downloading, building and starting app…',
})
})
it('keeps a long download visible and clears it on terminal success', async () => {
const store = useServerStore()
fake.packages = { 'nginx-proxy-manager': installing() }
await nextTick()
expect(store.isInstalling('nginx-proxy-manager')).toBe(true)
fake.packages = { 'nginx-proxy-manager': installing() }
await nextTick()
expect(store.installingApps.get('nginx-proxy-manager')?.progress).toBe(20)
fake.packages = { 'nginx-proxy-manager': { state: 'running' } }
await nextTick()
expect(store.isInstalling('nginx-proxy-manager')).toBe(false)
})
})
+23 -1
View File
@@ -239,6 +239,11 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
const panelPath = ref<string | null>(null)
function openSessionNow(appId: string, opts: LaunchOptions = {}) {
const pkg = useAppStore().data?.['package-data']?.[appId]
if (pkg?.['ui-ready'] === false) {
useToast().info(`${pkg.manifest?.title || appId} is not ready to open yet`)
return
}
recordAppLaunch(appId)
const mobile = isMobileViewport()
@@ -295,7 +300,7 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
// Apply the same readiness gate here so a container that has just entered
// `running` cannot race nginx and show a transient 502 to the user.
const pkg = useAppStore().data?.['package-data']?.[appId]
if (pkg && pkg.state === 'running' && !isAppReadyForLaunch(pkg)) {
if (pkg && (pkg['ui-ready'] === false || (pkg.state === 'running' && !isAppReadyForLaunch(pkg)))) {
useToast().info(`${pkg.manifest?.title || appId} is still starting — try again in a moment`)
return
}
@@ -394,6 +399,12 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
let launchUrl = normalizeLaunchUrl(payload.url, titleHintId)
const resolvedId = resolveAppIdFromUrl(launchUrl) || titleHintId
const pkg = resolvedId ? useAppStore().data?.['package-data']?.[resolvedId] : undefined
if (pkg?.['ui-ready'] === false) {
useToast().info(`${pkg.manifest?.title || resolvedId} is not ready to open yet`)
return
}
// Scheme discipline for everything launched on this host. Ports fronted
// by the node's app gate (manifest auth gated/open) serve TLS on the same
// port — on an HTTPS connection those must open over https. Ports that
@@ -472,6 +483,17 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
// Check /app/{id}/ path-style routes first (HTTPS proxy mode)
const m = u.pathname.match(/^\/app\/([a-z0-9._-]+)(?:\/|$)/i)
if (m?.[1]) return m[1].toLowerCase()
// Dynamic/sideloaded apps have no entry in the static port map.
if (u.hostname === window.location.hostname && u.port) {
for (const [id, pkg] of Object.entries(useAppStore().data?.['package-data'] || {})) {
const address = pkg.installed?.['interface-addresses']?.main?.['lan-address']
if (!address) continue
try {
const runtime = new URL(address)
if (runtime.port === u.port && ['localhost', '127.0.0.1', window.location.hostname].includes(runtime.hostname)) return id
} catch { /* malformed runtime address is not a launch target */ }
}
}
// Check port-based apps
const appId = PORT_TO_APP_ID[u.port]
if (appId) return appId
+2 -1
View File
@@ -25,6 +25,7 @@ import type { InstallPhase } from '../types/api'
const PHASE_INFO: Record<InstallPhase, { progress: number; message: string; status: InstallProgress['status'] }> = {
'preparing': { progress: 5, message: 'Preparing…', status: 'downloading' },
'pulling-image': { progress: 20, message: 'Downloading image…', status: 'downloading' },
'preparing-app': { progress: 20, message: 'Downloading, building and starting app…', status: 'downloading' },
'creating-container': { progress: 70, message: 'Creating container…', status: 'installing' },
'starting-container': { progress: 80, message: 'Starting container…', status: 'starting' },
'waiting-healthy': { progress: 88, message: 'Finalizing first start…', status: 'starting' },
@@ -149,7 +150,7 @@ export const useServerStore = defineStore('server', () => {
uninstallingApps.value.delete(appId)
}
}
}, { deep: true })
}, { deep: true, immediate: true })
function setInstallProgress(appId: string, progress: Partial<InstallProgress> & { id: string; title: string }) {
const existing = installingApps.value.get(appId)
+2
View File
@@ -88,6 +88,7 @@ export const PackageState = {
export type PackageState = typeof PackageState[keyof typeof PackageState]
export interface PackageDataEntry {
'ui-ready'?: boolean // HTTP upstream readiness, separate from container health
state: PackageState
health?: string | null // "healthy", "unhealthy", "starting", or null
'exit-code'?: number | null // container exit code: 0 = clean stop, non-zero = crash
@@ -180,6 +181,7 @@ export type ServiceStatus = typeof ServiceStatus[keyof typeof ServiceStatus]
export type InstallPhase =
| 'preparing'
| 'pulling-image'
| 'preparing-app'
| 'creating-container'
| 'starting-container'
| 'waiting-healthy'
+1 -1
View File
@@ -259,7 +259,7 @@ const canLaunch = computed(() => {
const hasRuntimeAddress = !!pkg.value.installed?.['interface-addresses']?.main?.['lan-address']
const hasKnownLaunchUrl = typeof window !== 'undefined' && !!resolveAppUrl(pkg.value.manifest.id)
const hasUI = !!(pkg.value.manifest.interfaces?.main?.ui || hasRuntimeAddress || hasKnownLaunchUrl)
return hasUI && pkg.value.state === 'running' && pkg.value.health !== 'starting' && pkg.value.health !== 'unhealthy'
return hasUI && pkg.value['ui-ready'] !== false && pkg.value.state === 'running' && pkg.value.health !== 'starting' && pkg.value.health !== 'unhealthy'
})
const features = computed(() => {
+18
View File
@@ -39,6 +39,7 @@
:must-open-new-tab="mustOpenNewTab"
:auto-retry-count="autoRetryCount"
:refresh-key="refreshKey"
:ui-ready-blocked="packageEntry?.['ui-ready'] === false"
:blocked-reason="blockedReason"
:blocked-title="blockedTitle"
:warming-up="warmingUp"
@@ -375,6 +376,20 @@ const panelClasses = computed(() => {
return `${base} app-session-overlay`
})
// A cold/restarting upstream is held outside the iframe. Start one fresh
// load when the scanner observes HTTP readiness; no manual refresh required.
watch(() => packageEntry.value?.['ui-ready'], (ready, previous) => {
if (ready === false) {
if (loadTimeoutId) clearTimeout(loadTimeoutId)
if (autoRetryId) clearTimeout(autoRetryId)
if (iframeCheckId) clearTimeout(iframeCheckId)
loading.value = false
} else if (previous === false && ready === true) {
autoRetryCount.value = 0
refresh()
}
})
// --- Lifecycle handlers ---
function onLoad() {
@@ -433,6 +448,7 @@ function refresh() {
function startLoadTimeout() {
if (loadTimeoutId) clearTimeout(loadTimeoutId)
if (packageEntry.value?.['ui-ready'] === false) return
loadTimeoutId = setTimeout(() => {
if (loading.value) {
loading.value = false
@@ -442,11 +458,13 @@ function startLoadTimeout() {
}
function openNewTabAndBack() {
if (packageEntry.value?.['ui-ready'] === false) return
if (appUrl.value) openExternalUrl(appUrl.value)
closeSession()
}
function openNewTab() {
if (packageEntry.value?.['ui-ready'] === false) return
if (appUrl.value) openExternalUrl(appUrl.value)
}
@@ -6,7 +6,7 @@
first, then sync status arrives), and the sync screen is strictly
more informative, so it takes precedence instead of the two
rendering on top of each other. -->
<AppLoadingScreen v-if="loading && !(electrsSync && !electrsSync.stale)" :icon="appIcon" :title="appTitle" :progress="loadProgress" />
<AppLoadingScreen v-if="loading && !uiReadyBlocked && !(electrsSync && !electrsSync.stale)" :icon="appIcon" :title="appTitle" :progress="loadProgress" />
</Transition>
<!-- ElectrumX sync screen — shown before the real UI while the on-chain
@@ -43,7 +43,7 @@
</Transition>
<div
v-if="appUrl && !iframeBlocked && (!electrsSync || electrsSync.stale)"
v-if="appUrl && !iframeBlocked && !uiReadyBlocked && (!electrsSync || electrsSync.stale)"
class="absolute inset-0 app-session-frame-scroll-host"
tabindex="-1"
@pointerdown="focusIframe"
@@ -66,7 +66,7 @@
reachable yet, so the "App not reachable / retry" overlay would just
paint over the sync progress and read as a hard error. -->
<Transition name="content-fade">
<div v-if="iframeBlocked && !electrsSync" class="absolute inset-0 z-10 flex flex-col items-center justify-center">
<div v-if="(iframeBlocked || uiReadyBlocked) && !electrsSync" class="absolute inset-0 z-10 flex flex-col items-center justify-center">
<div class="text-center px-8">
<!-- Warm-up uses the app's own icon, pulsing, rather than the padlock:
the padlock reads as "blocked/denied" and this state is neither. -->
@@ -78,7 +78,8 @@
</div>
<h3 class="text-lg font-semibold text-white mb-2">{{ warmingUp ? `${appTitle} is starting…` : blockedReason ? blockedTitle : (mustOpenNewTab ? 'This app opens in a new tab' : 'App not reachable') }}</h3>
<p class="text-white/50 text-sm mb-6">
<template v-if="mustOpenNewTab">{{ appTitle }} sets security headers that prevent iframe embedding.<br>Open it in a new browser tab instead.</template>
<template v-if="uiReadyBlocked">{{ blockedReason }} This screen opens automatically when it is ready.</template>
<template v-else-if="mustOpenNewTab">{{ appTitle }} sets security headers that prevent iframe embedding.<br>Open it in a new browser tab instead.</template>
<template v-else-if="warmingUp">The container is running but hasn't finished warming up yet.<br>This screen opens on its own as soon as it answers.<span v-if="autoRetryCount > 0" class="block text-yellow-400/70">Checking again automatically ({{ autoRetryCount }})...</span></template>
<template v-else-if="blockedReason">{{ blockedReason }}<br><span v-if="autoRetryCount > 0" class="text-yellow-400/70">Checking again automatically ({{ autoRetryCount }})...</span></template>
<template v-else>{{ appTitle }} may still be starting up or the container is stopped.<br><span v-if="autoRetryCount > 0" class="text-yellow-400/70">Retrying automatically ({{ autoRetryCount }})...</span></template>
@@ -95,6 +96,7 @@
Retry now
</button>
<button
v-if="!uiReadyBlocked"
@click="$emit('openNewTabAndBack')"
class="glass-button px-6 py-3 rounded-lg text-sm font-semibold inline-flex items-center gap-2"
>
@@ -108,7 +110,7 @@
</div>
</Transition>
<div v-if="!appUrl" class="absolute inset-0 flex items-center justify-center">
<div v-if="!appUrl && !uiReadyBlocked" class="absolute inset-0 flex items-center justify-center">
<div class="text-center px-8">
<h3 class="text-lg font-semibold text-white mb-2">App not configured</h3>
<p class="text-white/50 text-sm">No URL found for {{ appId }}</p>
@@ -133,6 +135,7 @@ const props = defineProps<{
mustOpenNewTab: boolean
autoRetryCount: number
refreshKey: number
uiReadyBlocked?: boolean
blockedReason?: string
blockedTitle?: string
// True while the container is up but its probe hasn't answered yet and the
@@ -66,3 +66,19 @@ describe('AppSessionFrame warm-up state', () => {
expect(text).toContain('This app opens in a new tab')
})
})
describe('HTTP readiness gate', () => {
it('does not show a missing-configuration error during initial installation', () => {
const frame = mountFrame({ appUrl: '', uiReadyBlocked: true, blockedReason: 'Waiting for the app to be ready…' })
expect(frame.text()).not.toContain('App not configured')
expect(frame.find('iframe').exists()).toBe(false)
})
it('does not mount an iframe before readiness, then opens automatically', async () => {
const frame = mountFrame({ iframeBlocked: false, uiReadyBlocked: true, blockedReason: 'Waiting for the app to be ready…', blockedTitle: 'App not ready' })
expect(frame.find('iframe').exists()).toBe(false)
expect(frame.text()).toContain('opens automatically')
expect(frame.text()).not.toContain('Open in new tab')
await frame.setProps({ uiReadyBlocked: false, blockedReason: '' })
expect(frame.find('iframe').exists()).toBe(true)
})
})
@@ -42,6 +42,8 @@ export const GENERATED_APP_PORTS: Record<string, number> = {
export const GENERATED_APP_TITLES: Record<string, string> = {
"aiui": "AI Assistant",
"alby-hub": "Alby Hub",
"angor-indexer": "Angor Indexer",
"angor-relay": "Angor Relay",
"archipelago-source": "GitWorkshop",
"archy-btcpay-db": "BTCPay Postgres",
"archy-mempool-db": "Mempool MariaDB",
+11 -14
View File
@@ -4,7 +4,7 @@
:data-controller-launch="canLaunch(pkg) ? '' : undefined"
tabindex="0"
role="link"
class="glass-card p-6 transition-all hover:-translate-y-1 cursor-pointer relative min-w-0 overflow-hidden"
class="glass-card flex flex-col h-full p-6 transition-all hover:-translate-y-1 cursor-pointer relative min-w-0 overflow-hidden"
:class="{ 'card-stagger': showStagger }"
:style="{ '--stagger-index': index }"
@click="$emit('goToApp', id)"
@@ -56,9 +56,9 @@
{{ description }}
</p>
<div v-if="!isInstalling && !isUninstalling && pkg.state !== 'installing'" class="flex items-center gap-2">
<div v-if="!isInstalling && !isUninstalling && pkg.state !== 'installing'" class="flex items-center gap-2 min-w-0">
<span
class="inline-flex items-center gap-1.5 px-2 py-1 rounded text-xs font-medium"
class="shrink-0 inline-flex items-center gap-1.5 px-2 py-1 rounded text-xs font-medium"
:class="getStatusClass(pkg.state, pkg.health, pkg['exit-code'])"
>
<svg
@@ -74,14 +74,14 @@
<span v-if="pkg.state === 'running' && pkg.health === 'unhealthy'" class="w-1.5 h-1.5 rounded-full bg-orange-400 animate-pulse"></span>
{{ getStatusLabel(pkg.state, pkg.health, pkg['exit-code']) }}
</span>
<p v-if="blockedReason" :title="blockedReason" class="min-w-0 truncate text-xs leading-snug text-yellow-200/80">
{{ blockedReason }}
</p>
</div>
<p v-if="blockedReason" class="mt-2 text-xs leading-snug text-yellow-200/80">
{{ blockedReason }}
</p>
<!-- Quick Actions — icon buttons in uniform dark containers -->
<!-- Installing progress — replaces action buttons -->
<div v-if="isInstalling || pkg.state === 'installing'" class="mt-4">
<div v-if="isInstalling || pkg.state === 'installing'" class="mt-auto pt-4">
<div class="flex items-center justify-between mb-1.5">
<span class="text-xs text-white/70 flex items-center gap-1.5">
<svg class="animate-spin h-3 w-3" fill="none" viewBox="0 0 24 24">
@@ -101,7 +101,7 @@
</div>
<!-- Uninstalling progress — truthful stage-driven bar (mirrors install) -->
<div v-else-if="isUninstalling" class="mt-4">
<div v-else-if="isUninstalling" class="mt-auto pt-4">
<div class="flex items-center justify-between mb-1.5">
<span class="text-xs text-white/70 flex items-center gap-1.5">
<svg class="animate-spin h-3 w-3" fill="none" viewBox="0 0 24 24">
@@ -120,7 +120,7 @@
</div>
</div>
<div v-else class="mt-4 flex gap-2">
<div v-else class="mt-auto pt-4 flex gap-2">
<!-- Update available -->
<button
v-if="pkg['available-update'] && pkg.state !== 'updating'"
@@ -214,7 +214,7 @@ import { computed } from 'vue'
import { useI18n } from 'vue-i18n'
import type { PackageDataEntry } from '@/types/api'
import {
isWebOnlyApp, opensInTab, canLaunch, launchBlockedReason, resolveAppIcon,
isWebOnlyApp, opensInTab, canLaunch, launchBlockedReason, resolveAppIcon, resolveAppTitle,
getStatusClass, getStatusLabel, handleImageError,
} from './appsConfig'
import { getCuratedAppList } from '../discover/curatedApps'
@@ -255,10 +255,7 @@ const isWebOnly = computed(() => isWebOnlyApp(props.id))
// Enrich from marketplace when backend data is sparse (e.g. during install)
const curated = computed(() => curatedMap.get(props.id))
const title = computed(() => {
const t = props.pkg.manifest?.title
return (t && t !== props.id) ? t : (curated.value?.title || t || props.id)
})
const title = computed(() => resolveAppTitle(props.id, props.pkg, curated.value?.title))
const description = computed(() => {
const d = props.pkg.manifest?.description?.short
return (d && d !== 'Installing...') ? d : (curated.value?.description || d || '')
+2 -4
View File
@@ -132,7 +132,7 @@ import type { AppCredential, AppCredentialsResponse, PackageDataEntry } from '@/
import { rpcClient } from '@/api/rpc-client'
import { resolveAppUrl } from '@/views/appSession/appSessionConfig'
import { resolveAppCredentials } from './appCredentials'
import { canLaunch, handleImageError, isWebsitePackage, opensInTab, resolveAppIcon, resolveRuntimeLaunchUrl, WEB_ONLY_APP_URLS } from './appsConfig'
import { canLaunch, handleImageError, isWebsitePackage, opensInTab, resolveAppIcon, resolveAppTitle, resolveRuntimeLaunchUrl, WEB_ONLY_APP_URLS } from './appsConfig'
import { getCuratedAppList } from '../discover/curatedApps'
const ITEMS_PER_PAGE = 16 // 4 columns x 4 rows
@@ -194,9 +194,7 @@ const pages = computed(() => {
})
function getTitle(id: string, pkg: PackageDataEntry): string {
const t = pkg.manifest?.title
if (t && t !== id) return t
return curatedMap.get(id)?.title || t || id
return resolveAppTitle(id, pkg, curatedMap.get(id)?.title)
}
function getIcon(id: string, pkg: PackageDataEntry): string {
@@ -1,7 +1,7 @@
import { describe, expect, it } from 'vitest'
import { ref } from 'vue'
import { PackageState, type PackageDataEntry } from '@/types/api'
import { APP_CATEGORY_MAP, canLaunch, filterEntriesForTab, hasFrontendUi, isServiceContainer, isServicePackage, isWebsitePackage, isAppReadyForLaunch, launchBlockedReason, resolveAppIcon, useCategoriesWithApps, DEFAULT_APP_ICON } from '../appsConfig'
import { APP_CATEGORY_MAP, canLaunch, filterEntriesForTab, hasFrontendUi, isServiceContainer, isServicePackage, isWebsitePackage, isAppReadyForLaunch, launchBlockedReason, resolveAppIcon, resolveAppTitle, useCategoriesWithApps, DEFAULT_APP_ICON } from '../appsConfig'
function makePkg(id: string, title: string, category: string): PackageDataEntry {
return {
@@ -25,6 +25,15 @@ function makePkg(id: string, title: string, category: string): PackageDataEntry
}
describe('appsConfig service filtering', () => {
it('keeps standalone Angor APIs in Services without a launch button', () => {
for (const id of ['angor-indexer', 'angor-relay']) {
const pkg = makePkg(id, id, 'money')
expect(filterEntriesForTab([[id, pkg]], 'services', 'all')).toHaveLength(1)
expect(filterEntriesForTab([[id, pkg]], 'apps', 'all')).toHaveLength(0)
expect(canLaunch(pkg)).toBe(false)
}
})
it('treats bitcoin stack UI sidecars as services', () => {
expect(isServiceContainer('bitcoin-ui')).toBe(true)
expect(isServiceContainer('lnd-ui')).toBe(true)
@@ -158,7 +167,7 @@ describe('appsConfig service filtering', () => {
pkg.health = null
expect(isAppReadyForLaunch(pkg)).toBe(false)
expect(canLaunch(pkg)).toBe(false)
expect(launchBlockedReason(pkg.manifest.id, pkg)).toContain('Starting up')
expect(launchBlockedReason(pkg.manifest.id, pkg)).toBe('Web UI not ready: GitWorkshop')
pkg.health = 'healthy'
expect(canLaunch(pkg)).toBe(true)
})
@@ -184,3 +193,56 @@ describe('appsConfig service filtering', () => {
expect(canLaunch(pkg)).toBe(true)
})
})
describe('headless service readiness messages', () => {
it('does not treat an API-only service as waiting for its own nonexistent UI', () => {
for (const id of ['phoenixd', 'angor-indexer', 'angor-relay', 'custom-api']) {
const pkg = makePkg(id, id, 'other')
pkg['ui-ready'] = false
pkg.health = 'healthy'
expect(canLaunch(pkg)).toBe(false)
expect(launchBlockedReason(id, pkg)).toBe('')
}
})
})
describe('HTTP readiness independent of container health', () => {
it('blocks fixed launch URLs while the HTTP upstream is unavailable', () => {
for (const id of ['gitea', 'filebrowser', 'fedimint', 'lnd']) {
const pkg = makePkg(id, id, 'other')
pkg['ui-ready'] = false
pkg.health = 'healthy'
expect(canLaunch(pkg)).toBe(false)
expect(isAppReadyForLaunch(pkg)).toBe(false)
expect(launchBlockedReason(id, pkg)).toBe(`Web UI not ready: ${resolveAppTitle(id, pkg)}`)
pkg['ui-ready'] = true
expect(isAppReadyForLaunch(pkg)).toBe(true)
}
})
it('allows a ready companion while its backend is syncing', () => {
const pkg = makePkg('lnd', 'Lightning', 'bitcoin')
pkg.health = 'starting'
pkg['ui-ready'] = true
expect(isAppReadyForLaunch(pkg)).toBe(true)
})
})
describe('manifest-generated service names', () => {
it('gives a new headless service its declared name without a hardcoded curated entry', () => {
const pkg = makePkg('angor-relay', 'angor-relay', 'money')
expect(resolveAppTitle('angor-relay', pkg)).toBe('Angor Relay')
pkg.manifest.title = 'My project relay'
expect(resolveAppTitle('angor-relay', pkg)).toBe('My project relay')
expect(resolveAppTitle('unknown-service', makePkg('unknown-service', 'unknown-service', ''))).toBe('unknown-service')
})
})
it('does not display a stale Mempool frontend alias beside its live package', () => {
const main = makePkg('mempool', 'Mempool', 'money')
const alias = makePkg('mempool-web', 'Mempool', 'money')
alias.state = PackageState.Restarting
const entries: Array<[string, PackageDataEntry]> = [['mempool', main], ['mempool-web', alias]]
const shown = [...filterEntriesForTab(entries, 'apps', 'all'), ...filterEntriesForTab(entries, 'services', 'all')]
expect(shown.map(([id]) => id)).toEqual(['mempool'])
expect(filterEntriesForTab([['mempool-web', alias]], 'apps', 'all').map(([id]) => id)).toEqual(['mempool-web'])
})
+26 -1
View File
@@ -4,6 +4,7 @@ import type { Ref } from 'vue'
import { computed } from 'vue'
import { PackageState, type PackageDataEntry } from '@/types/api'
import { matchPageScheme, resolveAppUrl } from '../appSession/appSessionConfig'
import { GENERATED_APP_TITLES } from '../appSession/generatedAppSessionConfig'
import { portIsGateFronted } from '../discover/curatedApps'
import { isAutoTabApp } from '@/utils/autoTabApps'
import {
@@ -13,6 +14,14 @@ import {
export type AppsTab = 'apps' | 'websites' | 'services'
/** Use manifest-generated names when a new service has only its ID in runtime state. */
export function resolveAppTitle(id: string, pkg: PackageDataEntry, curatedTitle?: string): string {
const title = pkg.manifest?.title
if (title && title !== id) return title
return curatedTitle || GENERATED_APP_TITLES[id] || title || id
}
// Re-exported for every existing caller — the canon moved to serviceNames.ts
// so the App Store's catalog merge can share it without a circular import.
export const SERVICE_NAMES = SHARED_SERVICE_NAMES
@@ -102,7 +111,12 @@ export function filterEntriesForTab(
activeTab: AppsTab,
selectedCategory: string,
): Array<[string, PackageDataEntry]> {
const hasMempool = entries.some(([id]) => id === 'mempool')
return entries.filter(([id, pkg]) => {
// Older daemons can retain the frontend manifest alias during a restart.
// Keep one tile while the updated scanner converges; a legacy-only node
// must still be able to see and operate its sole Mempool entry.
if (hasMempool && ['mempool-web', 'mempool-frontend', 'archy-mempool-web'].includes(id)) return false
if (isInternalToolingPackage(id, pkg)) return false
const wantsWebsites = activeTab === 'websites' || activeTab === 'services'
const isWebsite = isWebsitePackage(id, pkg)
@@ -222,6 +236,7 @@ function serviceParentIcon(id: string): string | undefined {
export const DEFAULT_APP_ICON = '/assets/icon/favico-black-v2.svg'
export function resolveAppIcon(id: string, pkg: PackageDataEntry, curatedIcon?: string): string {
if (id === 'angor-indexer' || id === 'angor-relay') return '/assets/img/app-icons/angor-green.png'
const rawIcon = (pkg["static-files"]?.icon || "").trim()
const icon = rawIcon === '/assets/img/favico.png' ? '' : rawIcon
if (
@@ -244,6 +259,7 @@ export function resolveAppIcon(id: string, pkg: PackageDataEntry, curatedIcon?:
export function canLaunch(pkg: PackageDataEntry): boolean {
if (isWebOnlyApp(pkg.manifest.id)) return true
if (pkg['ui-ready'] === false) return false
// Headless backends never get a Launch button, even with a published port.
if (isServicePackage(pkg.manifest.id, pkg)) return false
const hasRuntimeAddress = !!pkg.installed?.['interface-addresses']?.main?.['lan-address']
@@ -277,6 +293,7 @@ export function canLaunch(pkg: PackageDataEntry): boolean {
* health check retain the legacy state/port behaviour.
*/
export function isAppReadyForLaunch(pkg: PackageDataEntry): boolean {
if (pkg['ui-ready'] !== undefined) return pkg['ui-ready']
const manifest = pkg.manifest as unknown as Record<string, unknown>
const hasHealthCheck = Boolean(manifest.health_check || manifest['health-check'])
if (!hasHealthCheck) return pkg.health !== 'unhealthy'
@@ -285,6 +302,14 @@ export function isAppReadyForLaunch(pkg: PackageDataEntry): boolean {
export function launchBlockedReason(id: string, pkg?: PackageDataEntry | null): string {
const appId = pkg?.manifest?.id || id
const title = pkg ? resolveAppTitle(appId, pkg) : id
// API-only services have no web interface to wait for. Their health badge
// describes service availability; a failed HTML probe is not a dependency.
if (pkg && isWebsitePackage(appId, pkg)) return ''
if (pkg?.['ui-ready'] === false && !isServicePackage(appId, pkg)) {
if (pkg.state === PackageState.Stopped || pkg.state === PackageState.Exited) return 'App is stopped. Start it to open it.'
return `Web UI not ready: ${title}`
}
if (
(appId === 'fedimint' || appId === 'fedimintd') &&
(pkg?.state === PackageState.Starting || (pkg?.state === PackageState.Running && pkg?.health === 'starting'))
@@ -292,7 +317,7 @@ export function launchBlockedReason(id: string, pkg?: PackageDataEntry | null):
return 'Guardian opens a wait page until Bitcoin finishes initial sync.'
}
if (pkg && pkg.state === PackageState.Running && !isAppReadyForLaunch(pkg)) {
return 'Starting up — Launch will appear when the app is ready.'
return `Web UI not ready: ${title}`
}
return ''
}
@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it } from 'vitest'
import { __setSignedCatalogForTests, portAuth, portIsGateFronted, type SignedAppCatalog } from '../curatedApps'
import { __setSignedCatalogForTests, signedCatalogToApps, portAuth, portIsGateFronted, type SignedAppCatalog } from '../curatedApps'
/** Catalog fragments mirroring the live signed catalog's port declarations
* (releases/app-catalog.json, 2026-09-01). */
@@ -77,3 +77,12 @@ describe('portAuth', () => {
expect(portAuth('mempool-web', 4080)).toBeNull()
})
})
describe('standalone headless services', () => {
it('lists Angor services while keeping shared Mempool and node relay internals hidden', () => {
const apps = signedCatalogToApps(catalog(Object.fromEntries(
['angor-indexer', 'angor-relay', 'mempool-api', 'strfry'].map(id => [id, { version: '1' }]),
)))
expect(apps.map(app => app.id)).toEqual(['angor-indexer', 'angor-relay'])
})
})
@@ -362,6 +362,19 @@ init()
</button>
</div>
<div class="overflow-y-auto flex-1 min-h-0 space-y-6 pr-1">
<!-- v1.8.21-alpha -->
<div>
<div class="flex items-center gap-2 mb-3">
<span class="text-xs font-mono px-2 py-0.5 rounded bg-orange-500/20 text-orange-300">v1.8.21-alpha</span>
<span class="text-xs text-white/40">September 30, 2026</span>
</div>
<div class="space-y-3 text-sm text-white/80 pl-3 border-l border-white/10">
<p>Fixed Bitcoin and other containers being forcibly stopped after ten seconds during managed updates and restarts.</p>
<p>Existing installations now receive the same graceful shutdown allowance as new containers, without restarting apps just to apply this setting.</p>
<p>Prevented unnecessary Lightning restarts when Bitcoin has stayed running; dependency restarts now require an observed Bitcoin container change.</p>
<p>Includes the Cashu payment, optional Bitcoin pruning, Lightning readiness, and explorer improvements from 1.8.20.</p>
</div>
</div>
<!-- v1.8.20-alpha -->
<div>
<div class="flex items-center gap-2 mb-3">
+18 -17
View File
@@ -1,29 +1,30 @@
{
"changelog": [
"Fixed the embedded AIUI chat page painting a second background and dark scrim over Archy’s dashboard background.",
"Embedded AIUI now stays transparent so the dashboard background appears once.",
"AIUI background fixes are now included reliably in OTA updates and fresh installations."
"Fixed Bitcoin and other containers being forcibly stopped after ten seconds during managed updates and restarts.",
"Existing installations now receive the same graceful shutdown allowance as new containers, without restarting apps just to apply this setting.",
"Prevented unnecessary Lightning restarts when Bitcoin has stayed running; dependency restarts now require an observed Bitcoin container change.",
"Includes the Cashu payment, optional Bitcoin pruning, Lightning readiness, and explorer improvements from 1.8.20."
],
"components": [
{
"current_version": "1.8.19-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.19-alpha/archipelago",
"current_version": "1.8.21-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.21-alpha/archipelago",
"name": "archipelago",
"new_version": "1.8.19-alpha",
"sha256": "bb500d02567ad16179179d43138cc6fdafee680835262026eeaa7d1bc8cdd307",
"size_bytes": 64495784
"new_version": "1.8.21-alpha",
"sha256": "ff602e85f340aff7e43d9d94f7f84f11f713735c964c0d8ba150e23b065c30eb",
"size_bytes": 64748176
},
{
"current_version": "1.8.19-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.19-alpha/archipelago-frontend-1.8.19-alpha.tar.gz",
"name": "archipelago-frontend-1.8.19-alpha.tar.gz",
"new_version": "1.8.19-alpha",
"sha256": "fbbaa237e2ea4e9e576dda9b15fdcf3c59c40bc55bfee4ebbea303b0172fc49b",
"size_bytes": 97142031
"current_version": "1.8.21-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.21-alpha/archipelago-frontend-1.8.21-alpha.tar.gz",
"name": "archipelago-frontend-1.8.21-alpha.tar.gz",
"new_version": "1.8.21-alpha",
"sha256": "6c0842ec83a440269a353808a4cf154174f5232c9989b4a5448bc6486e1d0620",
"size_bytes": 97152546
}
],
"release_date": "2026-09-28",
"signature": "4da9bf766d94c2de6641619a36663106791c7a1d342b729c666662cdde1feabdf11c51994c1cbbe2a0e2b270c316d77763435e976ce22730855b3822bc9d390a",
"release_date": "2026-09-30",
"signature": "2ba21dde08284a13f511f11f0b925f09a56c1b36e40424558601b9ab6beea17edfa316e0baa51474bf084fd4da25429ec35a77d9a831554b309845c8226d4f0d",
"signed_by": "did:key:z6Mkfu5LT8d4DjETtrkATvHh9Dvcbnr7zBCUwfau8Sw7DLWT",
"version": "1.8.19-alpha"
"version": "1.8.21-alpha"
}
+18 -17
View File
@@ -1,29 +1,30 @@
{
"changelog": [
"Fixed the embedded AIUI chat page painting a second background and dark scrim over Archy’s dashboard background.",
"Embedded AIUI now stays transparent so the dashboard background appears once.",
"AIUI background fixes are now included reliably in OTA updates and fresh installations."
"Fixed Bitcoin and other containers being forcibly stopped after ten seconds during managed updates and restarts.",
"Existing installations now receive the same graceful shutdown allowance as new containers, without restarting apps just to apply this setting.",
"Prevented unnecessary Lightning restarts when Bitcoin has stayed running; dependency restarts now require an observed Bitcoin container change.",
"Includes the Cashu payment, optional Bitcoin pruning, Lightning readiness, and explorer improvements from 1.8.20."
],
"components": [
{
"current_version": "1.8.19-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.19-alpha/archipelago",
"current_version": "1.8.21-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.21-alpha/archipelago",
"name": "archipelago",
"new_version": "1.8.19-alpha",
"sha256": "bb500d02567ad16179179d43138cc6fdafee680835262026eeaa7d1bc8cdd307",
"size_bytes": 64495784
"new_version": "1.8.21-alpha",
"sha256": "ff602e85f340aff7e43d9d94f7f84f11f713735c964c0d8ba150e23b065c30eb",
"size_bytes": 64748176
},
{
"current_version": "1.8.19-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.19-alpha/archipelago-frontend-1.8.19-alpha.tar.gz",
"name": "archipelago-frontend-1.8.19-alpha.tar.gz",
"new_version": "1.8.19-alpha",
"sha256": "fbbaa237e2ea4e9e576dda9b15fdcf3c59c40bc55bfee4ebbea303b0172fc49b",
"size_bytes": 97142031
"current_version": "1.8.21-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.21-alpha/archipelago-frontend-1.8.21-alpha.tar.gz",
"name": "archipelago-frontend-1.8.21-alpha.tar.gz",
"new_version": "1.8.21-alpha",
"sha256": "6c0842ec83a440269a353808a4cf154174f5232c9989b4a5448bc6486e1d0620",
"size_bytes": 97152546
}
],
"release_date": "2026-09-28",
"signature": "4da9bf766d94c2de6641619a36663106791c7a1d342b729c666662cdde1feabdf11c51994c1cbbe2a0e2b270c316d77763435e976ce22730855b3822bc9d390a",
"release_date": "2026-09-30",
"signature": "2ba21dde08284a13f511f11f0b925f09a56c1b36e40424558601b9ab6beea17edfa316e0baa51474bf084fd4da25429ec35a77d9a831554b309845c8226d4f0d",
"signed_by": "did:key:z6Mkfu5LT8d4DjETtrkATvHh9Dvcbnr7zBCUwfau8Sw7DLWT",
"version": "1.8.19-alpha"
"version": "1.8.21-alpha"
}
@@ -1,34 +0,0 @@
{
"changelog": [
"Fixed Cashu file payments rejected despite a shared mint, and preserved the payment amount when mint fees reduce change.",
"Payment failures now report whether a refund actually succeeded; missing files and unsupported payment methods are rejected before charging.",
"Improved saving paid files into Files and reopening purchases without paying again.",
"Bitcoin Core and Knots installation offers optional pruning on larger disks, using the same settings as automatic pruning.",
"Fixed false missing-port checks that unnecessarily restarted Bitcoin and LND; recovery now respects managed shutdown timeouts.",
"LND explains when it is waiting for Bitcoin installation, startup, or sync, without treating normal synchronization as a restart-worthy failure.",
"Bitcoin startup messages explain block-index loading without exposing raw RPC errors, and Lightning keeps known balances clearly marked during outages.",
"Changed the public transaction-explorer default to mempool.space while preserving local explorers and custom choices."
],
"components": [
{
"current_version": "1.8.20-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.20-alpha/archipelago",
"name": "archipelago",
"new_version": "1.8.20-alpha",
"sha256": "16a173129672cbb40c250446ec52ba4a9bd1974cbb3a4988f90c6f3187b7a1f7",
"size_bytes": 64716656
},
{
"current_version": "1.8.20-alpha",
"download_url": "https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.20-alpha/archipelago-frontend-1.8.20-alpha.tar.gz",
"name": "archipelago-frontend-1.8.20-alpha.tar.gz",
"new_version": "1.8.20-alpha",
"sha256": "658b78fce0dfa20a627c987dd153b24cbac15adbde905cc6518744c637e12802",
"size_bytes": 97152297
}
],
"release_date": "2026-09-29",
"signature": "326cab454902abcb4f8037751af67aaae2860ecf00300177ec377a1f223a6a85b6e92d49297e7bc29a73220d501e6f4c83ee23477e2b688e33e19d093c6d210b",
"signed_by": "did:key:z6Mkfu5LT8d4DjETtrkATvHh9Dvcbnr7zBCUwfau8Sw7DLWT",
"version": "1.8.20-alpha"
}
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""Validate build-source apps against an OTA/ISO runtime payload before shipping."""
import sys
from pathlib import Path
import yaml
def check(root: Path) -> int:
root = root.resolve()
manifests = sorted((root / 'apps').glob('*/manifest.y*ml'))
if not manifests:
raise ValueError(f'No app manifests in {root / "apps"}')
count = 0
for manifest in manifests:
app = yaml.safe_load(manifest.read_text())['app']
build = app.get('container', {}).get('build')
if not build:
continue
context = Path(build['context'])
if context.is_absolute():
context = root / context.relative_to('/opt/archipelago')
else:
context = manifest.parent / context
context = context.resolve()
if not context.is_relative_to(root) or not context.is_dir():
raise ValueError(f'{app["id"]}: missing or out-of-payload build context: {context}')
dockerfile = (context / build.get('dockerfile', 'Dockerfile')).resolve()
if not dockerfile.is_relative_to(context) or not dockerfile.is_file():
raise ValueError(f'{app["id"]}: missing or out-of-context Dockerfile: {dockerfile}')
count += 1
return count
if __name__ == '__main__':
try:
count = check(Path(sys.argv[1] if len(sys.argv) > 1 else '.'))
except (ValueError, KeyError, OSError, yaml.YAMLError) as error:
sys.exit(f'Invalid app build payload: {error}')
print(f'Validated {count} app build contexts and Dockerfiles.')
+5
View File
@@ -81,6 +81,11 @@ def load_catalog(path: Path) -> dict[str, dict[str, Any]]:
if not isinstance(entry, dict):
continue
manifest = entry.get("manifest")
for variant in reversed(entry.get("manifest_variants", [])):
requires = variant.get("requires", [])
if requires and all(cap == "runtime-migration-backup-v1" for cap in requires):
manifest = variant.get("manifest")
break
if isinstance(manifest, dict) and isinstance(manifest.get("app"), dict):
# Embedded manifest: compare against the same fields the disk
# manifests expose, plus the entry's own version.
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""Test Git from Portainer's server context without creating a Source or stack."""
import argparse
import json
import pathlib
import socket
import stat
import urllib.error
import urllib.parse
import urllib.request
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
def classify(error):
text = error.lower()
for category, patterns in (
('connection-refused', ('connection refused',)),
('dns-failure', ('no such host', 'name resolution', 'server misbehaving')),
('timeout', ('timeout', 'timed out', 'deadline exceeded')),
('tls-failure', ('x509:', 'certificate', 'tls handshake')),
('proxy-or-login-interception', ('text/html', '<html', '<!doctype', 'unexpected content-type', 'invalid pkt-len')),
('repository-authentication', ('authentication required', 'authentication failed', 'authorization failed', '401', '403')),
('repository-not-found-or-private', ('repository not found', '404')),
):
if any(pattern in text for pattern in patterns):
return category
return 'git-error'
def safe_url(value):
parsed = urllib.parse.urlsplit(value)
if parsed.scheme not in ('http', 'https') or not parsed.hostname:
raise ValueError('Use an HTTP(S) URL')
if parsed.username is not None or parsed.password is not None or parsed.query or parsed.fragment:
raise ValueError('URLs must not contain credentials, query parameters or fragments')
return value.rstrip('/')
def private_json(path):
path = pathlib.Path(path)
if stat.S_IMODE(path.stat().st_mode) & 0o077:
raise ValueError('Credential file must be private (chmod 600)')
return json.loads(path.read_text())
def check(base, repository, credentials, opener=None):
base, repository = safe_url(base), safe_url(repository)
# JWT/API keys and Git credentials travel in headers/body, never URLs or logs.
headers = {'Content-Type': 'application/json'}
if credentials.get('api_key'):
headers['X-API-Key'] = credentials['api_key']
elif credentials.get('jwt'):
headers['Authorization'] = 'Bearer ' + credentials['jwt']
else:
raise ValueError('Credential file needs api_key or jwt')
payload = {'url': repository, 'tlsSkipVerify': False, 'interval': '5m'}
if credentials.get('git'):
payload['authentication'] = credentials['git']
request = urllib.request.Request(base + '/api/gitops/sources/test',
data=json.dumps(payload).encode(), headers=headers)
opener = opener or urllib.request.build_opener(NoRedirect())
try:
with opener.open(request, timeout=45) as response:
result = json.load(response)
except urllib.error.HTTPError as error:
return {'success': False, 'category': 'portainer-authentication' if error.code in (401, 403) else 'portainer-api-error', 'http_status': error.code}
except (urllib.error.URLError, TimeoutError, socket.timeout) as error:
return {'success': False, 'category': 'portainer-api-' + classify(str(error))}
except (ValueError, UnicodeError):
return {'success': False, 'category': 'portainer-api-invalid-response'}
if not isinstance(result, dict) or not isinstance(result.get('success'), bool):
return {'success': False, 'category': 'portainer-api-invalid-response'}
return {'success': result['success'], 'category': 'git-refs-readable' if result['success'] else classify(str(result.get('error', '')))}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--portainer-url', required=True, help='Reachable Portainer origin, without /api')
parser.add_argument('--repository-url', required=True, help='The same clone URL entered in Portainer')
parser.add_argument('--credentials-file', required=True, help='Mode 600 JSON: api_key or jwt; optional git: {username,password}')
args = parser.parse_args()
try:
result = check(args.portainer_url, args.repository_url, private_json(args.credentials_file))
except (OSError, ValueError):
parser.exit(2, 'Invalid URL or private credential file; no credentials were printed.\n')
print(json.dumps(result))
return 0 if result['success'] else 1
if __name__ == '__main__':
raise SystemExit(main())
+1
View File
@@ -101,6 +101,7 @@ if [ -z "$FRONTEND_ARCHIVE" ]; then
cp -r "$PROJECT_ROOT/$runtime_path" "$RUNTIME_DIR/$runtime_path"
fi
done
python3 "$PROJECT_ROOT/scripts/check-app-build-contexts.py" "$RUNTIME_DIR"
# KEEP IN SYNC with the `for unit in [...]` array in
# core/archipelago/src/bootstrap.rs (run_runtime_assets). A unit that
# bootstrap installs but this list does not ship simply never reaches a
+21 -1
View File
@@ -72,6 +72,10 @@ def manifest_launch_port(app: dict[str, Any]) -> int | None:
return port
if isinstance(port, str) and port.isdigit():
return int(port)
# An explicitly headless API/metrics declaration must not gain a
# browser launch button just because it has an HTTP health check.
if interfaces:
return None
health_check = app.get("health_check")
if not isinstance(health_check, dict) or str(health_check.get("type", "")).lower() != "http":
@@ -95,6 +99,20 @@ def manifest_launch_port(app: dict[str, Any]) -> int | None:
return None
def manifest_service_ports(app: dict[str, Any]) -> list[int]:
"""Declared API endpoints served by the gate also need mesh reachability."""
interfaces = app.get("interfaces") or {}
declared = {
int(i["port"]) for i in interfaces.values()
if isinstance(i, dict) and i.get("type") in ("api", "metrics")
and str(i.get("port", "")).isdigit()
}
return [int(p["host"]) for p in app.get("ports", [])
if str(p.get("host", "")).isdigit() and int(p["host"]) in declared
and p.get("auth") in ("open", "gated", "session")
and p.get("protocol", "tcp") == "tcp"]
def manifest_opens_in_new_tab(app: dict[str, Any]) -> bool:
"""Return whether manifest launch metadata opts the app out of iframe launch."""
launch = metadata(app).get("launch")
@@ -190,6 +208,7 @@ def render_rust_ports(ports: dict[str, int], extra_ports: list[int]) -> str:
"//! are reachable over the mesh; ports of apps that aren\'t installed have",
"//! no listener, so allowing them is inert.",
"",
"#[rustfmt::skip]",
"pub const APP_LAUNCH_PORTS: &[u16] = &[",
]
lines.extend(f" {port}," for port in distinct)
@@ -274,7 +293,8 @@ def main() -> int:
if (port := manifest_launch_port(app))
}
rust_path = Path(args.rust_app_ports)
rust_content = render_rust_ports(ports, RUST_EXTRA_PORTS)
service_ports = [p for app in manifests.values() for p in manifest_service_ports(app)]
rust_content = render_rust_ports(ports, RUST_EXTRA_PORTS + service_ports)
rust_old = rust_path.read_text(encoding="utf-8") if rust_path.exists() else ""
if rust_old != rust_content:
rust_path.write_text(rust_content, encoding="utf-8")
+15 -1
View File
@@ -36,11 +36,15 @@ source "$ROOT/scripts/image-versions.sh"
set +a
UPDATED="$(date -u +%Y-%m-%d)" OUT="$OUT" APPS_DIR="$ROOT/apps" \
BASE_CATALOG="${BASE_CATALOG:-$ROOT/releases/app-catalog.json}" \
PUBLIC_CATALOG="$ROOT/app-catalog/catalog.json" \
EMBED_MANIFESTS="${EMBED_MANIFESTS:-1}" python3 - <<'PY'
import glob
import json, os
with open(os.environ["BASE_CATALOG"], encoding="utf-8") as baseline_file:
baseline_entries = json.load(baseline_file).get("apps", {})
try:
import yaml
except ImportError:
@@ -182,7 +186,17 @@ if os.environ.get("EMBED_MANIFESTS") and apps_dir:
continue
entry = apps.setdefault(str(app_id), {})
entry.setdefault("version", str(app.get("version", "")) or "0")
entry["manifest"] = _retarget_registry(data)
rendered = _retarget_registry(data)
if data["app"].get("backup_before_runtime_change"):
baseline = baseline_entries.get(app_id, {}).get("manifest")
if not baseline or baseline.get("app", {}).get("backup_before_runtime_change"):
raise SystemExit(f"{app_id}: a pre-migration BASE_CATALOG manifest is required for old-node compatibility")
entry["manifest"] = baseline
entry["manifest_variants"] = [{
"requires": ["runtime-migration-backup-v1"], "manifest": rendered,
}]
else:
entry["manifest"] = rendered
embedded += 1
# Multi-version support (docs/bitcoin-multi-version-design.md §3 Phase 1):
+7
View File
@@ -64,6 +64,13 @@ for f in live/vmlinuz live/initrd.img live/filesystem.squashfs \
fi
done
# Verify the mounted artifact carries every manifest-declared build source.
if python3 "$REPO/scripts/check-app-build-contexts.py" "$MNT/archipelago"; then
ok "app build contexts and Dockerfiles"
else
bad "incomplete app build payload"
fi
# ── GRUB must boot the live system ───────────────────────────────────
if grep -q "boot=live" "$MNT/boot/grub/grub.cfg" 2>/dev/null; then
ok "grub.cfg has boot=live"
+6 -1
View File
@@ -9,7 +9,12 @@ command -v setpriv >/dev/null
sudo -n true || { echo 'Isolated backend tests require noninteractive sudo for systemd namespaces.' >&2; exit 1; }
metadata=$(mktemp)
trap 'rm -f "$metadata"' EXIT
if ! cargo test --manifest-path "$REPO/core/Cargo.toml" -p archipelago --bin archipelago \
case "${ARCHY_TEST_PACKAGE:-archipelago}" in
archipelago) test_target=(-p archipelago --bin archipelago) ;;
archipelago-container) test_target=(-p archipelago-container --lib) ;;
*) echo 'Unsupported isolated test package' >&2; exit 2 ;;
esac
if ! cargo test --manifest-path "$REPO/core/Cargo.toml" "${test_target[@]}" \
--locked --no-run --message-format=json --config 'profile.test.package.archipelago.opt-level=0' > "$metadata"; then
python3 - "$metadata" <<'PYDIAG'
import json,sys
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Opt-in disposable rootless Angor gateway integration checks. No native app changes."""
import subprocess,pathlib,json,urllib.request,urllib.error,time,tempfile,os,uuid
if os.environ.get('ARCHY_ALLOW_DISPOSABLE_CONTAINERS') != '1':
raise SystemExit('Set ARCHY_ALLOW_DISPOSABLE_CONTAINERS=1 to run isolated test containers')
run_id=uuid.uuid4().hex[:12]
net='archy-angor-test-'+run_id;backend='angor-test-backend-'+run_id;gateway='angor-test-gateway-'+run_id
def run(*a):
r=subprocess.run(a,capture_output=True,text=True)
if r.returncode:raise RuntimeError(r.stderr)
return r.stdout.strip()
def req(path,data=None,method=None,headers={}):
r=urllib.request.Request('http://127.0.0.1:19098'+path,data=data,method=method,headers=headers)
try:
with urllib.request.urlopen(r,timeout=10) as f:return f.status,f.headers,f.read()
except urllib.error.HTTPError as e:return e.code,e.headers,e.read()
script="""require('http').createServer((q,r)=>{let b='';q.on('data',x=>b+=x);q.on('end',()=>{r.setHeader('Access-Control-Allow-Origin','https://wrong.example');if(q.url==='/api/v1/blocks/tip/height'){r.end('900000');return}r.setHeader('Content-Type','application/json');r.end(JSON.stringify({url:q.url,method:q.method,body:b,cookie:q.headers.cookie||null,auth:q.headers.authorization||null}))})}).listen(8999,'0.0.0.0')"""
def start_backend():run('podman','run','-d','--name',backend,'--network',net,'--network-alias','mempool-api','--cap-drop=all','--security-opt=no-new-privileges','docker.io/library/node:24-alpine','node','-e',script)
def ready(seconds=40):
end=time.monotonic()+seconds
while time.monotonic()<end:
try:
if req('/health')[0]==200:return
except OSError:pass
time.sleep(1)
raise RuntimeError('Gateway readiness did not recover')
assert subprocess.run(['podman','network','exists',net]).returncode==1
run('podman','network','create',net)
try:
start_backend()
run('podman','run','-d','--name',gateway,'--network',net,'--read-only','--cap-drop=all','--security-opt=no-new-privileges','--memory','128m','-p','127.0.0.1:19098:8080','source.archipelago-foundation.org/chaum/angor-indexer:1.0.1')
ready()
for path in ['/api/v1/address/bc1fixture/txs?after_txid=abc','/api/v1/fees/recommended','/api/tx/fixture/hex']:
status,headers,body=req(path,headers={'Cookie':'node-secret=do-not-forward','Authorization':'Bearer do-not-forward'})
result=json.loads(body);assert status==200 and result['url']==(path if path.startswith('/api/v1/') else path.replace('/api/','/api/v1/',1)) and result['cookie'] is None and result['auth'] is None
assert headers.get_all('Access-Control-Allow-Origin')==['*']
assert req('/api/v1/tx',b'deadbeef')[0]==200
assert json.loads(req('/api/v1/tx',b'deadbeef')[2])['body']=='deadbeef'
assert req('/api/v1/fees/recommended',b'bad')[0]==403
assert req('/api/v1/tx',b'bad',method='DELETE')[0]==403
assert req('/api/v1/tx',method='OPTIONS')[0]==204
assert req('/api/v1/tx',b'x'*(4*1024*1024+1))[0]==413
assert req('/unknown')[0]==404
d=json.loads(run('podman','inspect',gateway))[0];assert d['Config']['User']=='101:101' and not d['BoundingCaps']
print('PASS API paths/query/body, transaction-only POST, method/size limits, CORS, credential stripping and unprivileged read-only image',flush=True)
run('podman','stop',backend)
status,headers,body=req('/health');assert status==503 and json.loads(body)['status']=='waiting'
run('podman','rm',backend);start_backend();ready()
print('PASS backend outage returns truthful 503; backend recreation recovers through runtime DNS without gateway restart',flush=True)
except BaseException:
subprocess.run(['podman','logs','--tail','15',gateway],check=False)
raise
finally:
for name in [gateway,backend]:subprocess.run(['podman','rm','-f','--time','3',name],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
subprocess.run(['podman','network','rm',net],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
@@ -0,0 +1,42 @@
#!/usr/bin/env python3
"""Headless store apps must be discoverable without acquiring a UI launcher."""
import importlib.util
import pathlib
import unittest
import yaml
ROOT = pathlib.Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location('catalog_generator', ROOT / 'scripts/generate-app-catalog.py')
generator = importlib.util.module_from_spec(spec)
spec.loader.exec_module(generator)
class ServiceMetadata(unittest.TestCase):
def test_headless_services_have_mesh_ports_but_no_browser_launcher(self):
for name, port in [('angor-indexer', 8998), ('angor-relay', 8091)]:
app = yaml.safe_load((ROOT / 'apps' / name / 'manifest.yml').read_text())['app']
self.assertIsNone(generator.manifest_launch_port(app))
self.assertEqual(generator.manifest_service_ports(app), [port])
self.assertEqual(app['ports'][0]['bind'], '127.0.0.1')
self.assertEqual(app['security']['capabilities'], [])
def test_host_local_api_never_opens_mesh_port(self):
app = {'interfaces': {'main': {'type': 'api', 'port': 8999}},
'ports': [{'host': 8999, 'auth': 'local'}],
'health_check': {'type': 'http'}}
self.assertIsNone(generator.manifest_launch_port(app))
self.assertEqual(generator.manifest_service_ports(app), [])
def test_legacy_ui_fallback_retained(self):
self.assertEqual(generator.manifest_launch_port({'ports': [{'host': 8080}],
'health_check': {'type': 'http'}}), 8080)
def test_relay_storage_cannot_share_node_identity_or_database(self):
node = yaml.safe_load((ROOT / 'apps/strfry/manifest.yml').read_text())['app']
angor = yaml.safe_load((ROOT / 'apps/angor-relay/manifest.yml').read_text())['app']
node_paths = {v['source'] for v in node['volumes']}
self.assertTrue(node_paths.isdisjoint(v['source'] for v in angor['volumes']))
self.assertTrue(all(not f['overwrite'] for f in angor['files']))
self.assertEqual(angor['interfaces']['main']['type'], 'api')
if __name__ == '__main__':
unittest.main()
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""Exercise release payload checks with complete, incomplete and escaping contexts."""
import importlib.util
import shutil
import tempfile
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location('contexts', REPO / 'scripts/check-app-build-contexts.py')
contexts = importlib.util.module_from_spec(spec)
spec.loader.exec_module(contexts)
class BuildPayloadTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
shutil.copytree(REPO / 'apps', self.root / 'apps')
shutil.copytree(REPO / 'docker', self.root / 'docker')
def test_complete_payload(self):
self.assertGreaterEqual(contexts.check(self.root), 6)
def test_iso_old_allowlist_rejected(self):
shutil.rmtree(self.root / 'docker/archipelago-source')
with self.assertRaisesRegex(ValueError, 'archipelago-source.*missing'):
contexts.check(self.root)
def test_missing_dockerfile_rejected(self):
(self.root / 'docker/archipelago-source/Dockerfile').unlink()
with self.assertRaisesRegex(ValueError, 'archipelago-source.*Dockerfile'):
contexts.check(self.root)
def test_context_symlink_cannot_escape_payload(self):
target = self.root / 'docker/archipelago-source'
shutil.rmtree(target)
target.symlink_to(REPO / 'docker/archipelago-source', target_is_directory=True)
with self.assertRaisesRegex(ValueError, 'out-of-payload'):
contexts.check(self.root)
def test_empty_payload_rejected(self):
shutil.rmtree(self.root / 'apps')
with self.assertRaisesRegex(ValueError, 'No app manifests'):
contexts.check(self.root)
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
import importlib.util
import io
import json
import pathlib
import unittest
import urllib.error
ROOT = pathlib.Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location('diagnostic', ROOT / 'scripts/check-portainer-git-source.py')
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
class Response(io.BytesIO):
pass
class FakeAPI:
def __init__(self, result=None, error=None):
self.result, self.error, self.request = result, error, None
def open(self, request, timeout):
self.request = request
if self.error:
raise self.error
return Response(json.dumps(self.result).encode())
class Diagnostics(unittest.TestCase):
def test_server_context_credentials_not_in_url_and_tls_stays_enabled(self):
api = FakeAPI({'success': True})
result = m.check('http://localhost:9000', 'http://node:3001/user/repo',
{'jwt': 'test-jwt', 'git': {'username': 'test-user', 'password': 'test-secret'}}, api)
self.assertTrue(result['success'])
self.assertEqual(api.request.full_url, 'http://localhost:9000/api/gitops/sources/test')
payload = json.loads(api.request.data)
self.assertFalse(payload['tlsSkipVerify'])
self.assertEqual(payload['authentication']['password'], 'test-secret')
self.assertNotIn('test-secret', json.dumps(result))
def test_failure_categories_from_source_api(self):
cases = [('dial tcp: connection refused', 'connection-refused'),
('lookup node: no such host', 'dns-failure'),
('context deadline exceeded', 'timeout'),
('unexpected content-type text/html', 'proxy-or-login-interception'),
('authentication required', 'repository-authentication'),
('x509: certificate signed by unknown authority', 'tls-failure'),
('repository not found', 'repository-not-found-or-private')]
for error, expected in cases:
with self.subTest(error=error):
result = m.check('http://localhost:9000', 'http://node/repo', {'jwt': 'test'}, FakeAPI({'success': False, 'error': error}))
self.assertEqual(result['category'], expected)
self.assertFalse(result['success'])
def test_portainer_auth_is_distinct_from_repository_auth(self):
api = FakeAPI(error=urllib.error.HTTPError('http://localhost', 401, 'Unauthorized', {}, None))
self.assertEqual(m.check('http://localhost', 'http://node/repo', {'jwt': 'bad'}, api)['category'], 'portainer-authentication')
def test_html_or_malformed_api_response_never_proves_git_success(self):
for value in ({'status': 1}, {'success': 'true'}, [], '<html>login</html>'):
self.assertFalse(m.check('http://localhost', 'http://node/repo', {'jwt': 'test'}, FakeAPI(value))['success'])
def test_credential_urls_rejected_before_request(self):
for value in ('http://user:secret@node/repo', 'http://node/repo?token=secret', 'file:///data/repo'):
with self.assertRaises(ValueError):
m.check('http://localhost', value, {'jwt': 'test'}, FakeAPI())
if __name__ == '__main__':
unittest.main()
+1
View File
@@ -71,6 +71,7 @@ summary() {
# ── Stage 1: static ──────────────────────────────────────────────────
stage "git-diff-check" git diff --check
stage "cargo-fmt" timeout 240 cargo fmt --manifest-path core/Cargo.toml --all --check
stage "app-build-contexts" python3 tests/regression/app-build-contexts.py
stage "manifest-shell" python3 scripts/check-manifest-shell.py
stage "doctor-ports" bash tests/regression/container-doctor-ports.sh
stage "bitcoin-pruning" python3 tests/regression/bitcoin-prune-entrypoint.py