The gate removed `Authorization` unconditionally before proxying, so every
credential an app owns was destroyed one hop before the app saw it.
IndeeHub's Nostr login is the reported case: it signs a NIP-98 event and
sends `Authorization: Nostr <event>` to its own /api/auth/nostr/session.
The header arrived stripped and its backend answered "Authorization header
is missing" — a 401 that no signer could ever satisfy. That is why a NIP-07
browser extension in a tab, the parent frame's NIP-07 bridge
(nostr-provider.js) and AIUI all broke at once while the signing itself was
never at fault. Proven on the node: the same POST returns a real NIP-98
validation error on loopback and the gate's login page through the gate.
The gate accepts exactly one header credential — `Authorization: Bearer
<app-scoped device token>` — so only that one is ours to withhold. authorize()
now reports which credential allowed the request, and the header is dropped
only when it WAS the gate's token, mirroring the surgical cookie strip
directly above it. Any other scheme (Nostr, Basic, an app's own bearer)
rides through untouched.
Credential-less allowlist paths still drop the header: nothing there needs
auth, so an unverified token is not handed to the app.
Tests: an app's Authorization is not classified as the gate's, and a real
proxy hop against a local server shows the app's credential arriving intact
while a gate device token does not.
IndeeHub worked all year and broke when the gate rolled out. Cause, verified on
the node: GET /manifest.json returns 401 + the gate's login HTML. A browser
fetches <link rel="manifest"> in no-credentials mode unless the tag opts in
with crossorigin="use-credentials", so the session cookie is NEVER offered and
the gate challenges a fully authenticated user. The app's service worker then
serves its cached shell, whose every network call fails — which reads as "the
app is broken" rather than "the gate refused it". Any gated app with a PWA
manifest has the same failure.
Passed through unauthenticated on purpose, and deliberately as small as the
problem: an EXACT-match allowlist of /manifest.json, /site.webmanifest and
/favicon.ico. Static, non-user-specific, and no more revealing than the gate's
own login page, which already shows the app's name and icon.
Exact match, never a prefix — a prefix would let /manifest.json/../api/secrets
ride through. A test pins that: 8 near-miss paths (traversal, query-string
traversal, /api/manifest.json, /manifest.jsonx, case variants, /admin,
/api/auth/nostr/session) must all still be challenged.
19/19 appgate tests pass. This does NOT address the app's own auth endpoints
being intercepted — that needs a session-aware decision and is recorded
separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An app port must answer whatever the browser asks for: an HTTP dashboard
embeds http://host:PORT, an HTTPS one embeds https://host:PORT, and an HTTPS
page cannot embed an HTTP frame at all. So the choice is per-node, not
per-fleet, and a second port number would mean every manifest changes and
torrc doubles.
Instead the gate peeks the first byte. A TLS ClientHello is 0x16; no HTTP
method starts with it. peek() leaves the bytes in the socket buffer, so the
acceptor still sees a complete, untouched ClientHello. TLS and plain share one
generic serve_http(), so authentication, proxying and upgrade handling cannot
drift apart by scheme.
EXISTING NODES ARE UNAFFECTED BY CONSTRUCTION. Anything that is not a TLS
handshake takes the identical path as before, and a node with no certificate
serves plain HTTP exactly as today — TLS is strictly additive.
rustls does NOT verify that a private key matches its certificate. Established
by test, not assumed: with_single_cert accepted a pair from two different keys
and would only have failed mid-handshake in a user's browser — a security
control that reports success and does nothing, the exact shape this module's
own docs warn about. So the pairing is now proven explicitly (sign a fixed
message with the key, verify against the certificate's public key) and a
mismatch refuses to serve.
Also: cert and key mtimes are stamped as a PAIR, because reissuing writes them
separately and keying on one would serve a certificate that no longer matches
its key; a 15s first-byte timeout closes the slowloris window one step earlier
than the existing header-read timeout; PKCS#8 and PKCS#1 keys are both
accepted so a hand-made key does not silently downgrade a working node.
Deps pinned to the rustls 0.21 line reqwest already resolves — no new vendor,
no second rustls major. Test fixtures are throwaway (localhost SANs only), not
any node's identity.
38/38 appgate tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The port map deferred to DISK manifests for any app with a build source
— which is exactly the four companion UIs (lnd-ui, bitcoin-ui,
electrs-ui, fips-ui). Their disk manifests reach nodes only via the
frontend runtime payload or a per-node repo checkout, and in the
v1.7.125 rollout both proved stale or entirely absent: one node had no
checkout at all, others restored an older payload over apps/ at every
boot. Result: session_passthrough never reached the gate, so the node's
own screens 401'd on every data call, and on nodes whose UI rebuilt
from a stale context the app held its port UNGATED.
Classification now uses a ports-only overlay that accepts build-source
manifests (install/orchestration still defers to disk — unchanged). The
signed catalog is the freshest, operator-signed source, and the gate's
address binds fail safely against a container publishing differently
(logged CANNOT PROTECT), so this can only tighten policy, never expose.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both predate this session's changes and were masked by the release
gate's cargo-test-weekly compile timeout:
- login_page_sources_its_art_from_the_gate still asserted the retired
wordmark (logo-archipelago.svg); the login page ships the sidebar A
mark (favico-black-v2.svg) since the 2026-08-05 rework.
- unauthenticated_ports_are_all_accounted_for lagged at 17; the
v1.7.123 port-policy round grew the rationale-carrying exempt set
to 25 (reviewed and enumerated in the test comment).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two daemon bugs, one debugging arc (2026-08-05, operator-reported):
1. The app gate removed the ENTIRE Cookie header before proxying. That
broke the data plane of every first-party companion UI behind the gate
(lnd-ui/bitcoin-ui/electrs-ui/fips-ui render their shell, then every
/proxy/* and /lnd-connect-info call 401s — observed as "LND UI
unreachable"), and silently logged users out of every gated app with
its own cookie login (vaultwarden, nextcloud, gitea) on each request.
The gate now strips only its own cookie pairs (session, csrf_token);
a new per-port manifest opt-in `session_passthrough: true` forwards
the node session to first-party UIs whose nginx proxies the daemon's
authenticated endpoints. Undeclared ports never get passthrough.
2. podman_client::create_container sent named volumes to the libpod API
as bind mounts with the bare volume name as source, so creating any
manifest app with a `type: volume` mount failed. On .38 the reconciler
removed indeedhub-postgres/-minio for env drift and then could never
create their replacements, leaving the stack half-missing forever.
Named volumes now ride the spec's `volumes` field ({Name, Dest,
Options}). Also: the reconcile-failure log now prints the full anyhow
chain — `%e` showed only "create_container X" and hid the real error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unbreaks Bitcoin on every node running the 1.7.124 catalog: the embedded
start script had a shell syntax error, so bitcoind never launched and the
app vanished. Delivered by catalog rather than a release because manifests
reach nodes through the signed catalog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Portainer: nodes have been running :latest — which is 2.39.1 — while the
manifest pinned 2.19.4 from two years ago. The port migration recreated the
container onto that old pin and Portainer refused to start: it migrates a
database forward, never backward, so an existing install died with 'schema
version does not align' and My Apps showed 'app is not responding'
(100.82.34.38). 2.39.1 published as an immutable tag and pinned forward, so
existing databases keep working and older ones migrate up.
Bitcoin: complements PR #131. That removes the code which kept writing a
datadir bitcoin.conf; -allowignoredconf=1 additionally makes an existing
one non-fatal, so a node already carrying the file recovers on restart
instead of crash-looping until something reinstalls it.
App gate login: rebuilt against the dashboard's own design — rotating
intro backgrounds served from the gate, the glass panel, the Archipelago
mark in its gradient ring, the app's icon as a My Apps tile, and the glass
button. Crucially it no longer sends X-Frame-Options: DENY, which made
every gated app render as unreachable inside My Apps' embedded frame;
frame-ancestors expresses 'only this node may frame me', which
X-Frame-Options cannot.
OTA origin: primary mirror is now source.archipelago-foundation.org over
TLS instead of a bare IP on plaintext. The IP stays as an automatic
fallback for nodes whose DNS or clock is broken — both break TLS, and the
signature, not the transport, is what establishes trust.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Caught verifying the gate fixes on archi-dev-box: [fips0-ULA]:32838
answered HTTP 200 straight from nbxplorer with no credential. The
catalog declares that port auth: local — host-local by intent, pinned to
loopback, the gate deliberately keeps its hands off — but the mesh relay
bridges a STATIC port list to 127.0.0.1, so it republished it to the
whole mesh. Same bug class as the Tor onion gap: a transport that
converges on the app loopback without consulting the declaration.
PortMap now records declared-local ports and the relay withholds them
(tearing down an existing bridge if a catalog refresh newly declares
one), alongside the declared-gated withhold. Undeclared ports keep
todays behaviour — silence is not an instruction in either direction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dev-box verification of the Tor/FIPS fixes caught a pre-existing split
brain: the orchestrator publishes containers from the signed catalog's
embedded manifests (origin-wins), but the gate classified ports from the
stale disk manifests — so it externally bound nbxplorer 32838, a port
the catalog declares auth: local and pins to loopback. Reachable behind
a login, but reachable where it deliberately was not.
- build_port_map now consults the catalog overlay first, via the same
parse/validate/image-only filter the orchestrator uses (moved to
app_catalog::catalog_manifest_overlay so the two cannot diverge again).
- GatedPort carries . The gated set still includes undeclared
Session-default ports for challenge/audit, but every action that
REDIRECTS traffic — the torrc 127.0.0.2 repoint, the FIPS relay
stand-down, the Tor-upstream bind — now keys on the declaration.
- The sweep releases held claims whose port left the gated set, so a
catalog refresh that withdraws a port (gated → local/none) takes
effect without a daemon restart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tor carries no session cookie, so HiddenServicePort → 127.0.0.1:<port>
reached the app around the gate — the last transport the gate did not
cover. The gate now binds 127.0.0.2 (its own loopback, distinct from the
app's 127.0.0.1, so no app needs a second port), and regenerate_torrc
forwards declared-gated ports there. Undeclared ports keep today's
target: absence of the field is not an instruction.
The 127.0.0.2 claim deliberately does not count toward the unprotected
audit — a port whose only claim is the Tor loopback is still wide open
on the LAN and must keep warning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two live incidents on archi-dev-box today, one bug. Both times a safety
decision read an ABSENT manifest field as if it were a value, and a
node's installed manifests always lag the binary — so "absent" is the
state of essentially every port on every node.
1. Gating any `session` port regardless of `bind` published Bitcoin's
loopback-only RPC 8332 on the LAN, Tailscale and IPv6 within seconds
of deploy.
2. The `bind`-keyed replacement looked safe because it protected
`bind: 127.0.0.1` ports — but LND's gRPC 10009 and REST 18080 carry
an EMPTY bind, so they fell through. One container recreate from
pinning them to loopback and breaking Zeus and every remote wallet.
`auth` is now `Option<PortAuth>`, separating two questions that were
conflated:
* `auth_policy()` — what to CLASSIFY the port as. Undeclared reports as
Session, i.e. shows in the audit as something that should be behind
the gate. Reporting is always safe.
* `auth_is_declared()` — whether the daemon may ACT. Only an explicit
declaration authorises changing how a port is published.
Also reverts the daemon-side publish rewriting entirely. The node proved
it wrong twice over: the recreate path that actually ran was in
package::install, not podman_client, so the pin never fired; and even
`bind: 127.0.0.1` written directly into the node's manifest was
overridden by the signed catalog. Publishes are built in several places
and all of them already honour `bind`, so the migration belongs in the
catalog as data — not in daemon-side inference that can only ever cover
one path and guess wrong on the rest.
Tests: 75/75 container, incl. the LND wallet-port shape (`host: 10009`,
empty bind, no auth) asserted to be non-actionable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both found while setting up the on-node test, and both fail silently in
the same direction — the gate reports success while protecting nothing,
which is the exact failure the module was written to prevent.
1. Loopback-pinned ports were skipped entirely.
`identity.rs` dropped any port whose manifest sets `bind: 127.0.0.1`,
reasoning that a loopback publish is not externally reachable. But
`listener.rs` requires loopback-pinning as the PRECONDITION for gating —
while an app holds 0.0.0.0:<port> the kernel will not let the gate bind
that port at all. So the two contradicted each other: pinning an app, the
one action that lets the gate take over, was also what removed it from
the gated set. Completing the entire migration would have gated nothing,
and GateStatus would have reported zero unprotected ports while doing it.
`bind` cannot carry this decision, because two unrelated intentions
produce an identical loopback publish: Bitcoin's RPC 8332 is pinned so
the LAN CANNOT reach it (fronting it would newly expose it on every host
address, behind a login but exposed where it deliberately was not),
whereas a migrated app is pinned precisely so the gate CAN. Inferring
from `bind` breaks one or the other, so the intent is now declared:
`PortAuth::Local` means the first case. The three ports that are
host-local by intent (bitcoin-core/knots 8332, aiui 5180 — all already
`bind: 127.0.0.1`) say so, and a loopback publish with `auth: session`
stays gated. A test pins that property.
2. The port map was never refreshed.
`AppGate::refresh()` existed, was documented as making catalog changes
apply without a restart, and was called by nothing. The map was built
once in `new()`, so an app installed while the daemon runs would never be
gated — and would never appear in `unprotected` either, so the node would
report itself fully enforced while serving a brand-new app to anyone who
asked. The sweep now refreshes before classifying.
Tests: 22/22 appgate, 73/73 archipelago-container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reproduced again on this node today: with no session cookie, six app
ports answered HTTP 200 with their real UIs (18083 LND, 8334, 8175
Fedimint Guardian, 8336 FIPS Mesh, 8090, 7777), all bound 0.0.0.0 and so
served on every host address. Same bug class as the /lnd-connect-info
and /bitcoin-rpc/ leaks closed in v1.7.120, but across every app.
LAN, Tailscale, Tor and the FIPS mesh all converge on 127.0.0.1:<port>,
so this is one gate rather than four. It lives in the daemon rather than
a per-app sidecar (umbrel's app_proxy model): rootless, no extra
container per app, and it can reuse machinery that already exists.
It invents no authentication policy. verify_password, TOTP secret
decryption, verify_code with used-step replay protection, the session
store, and — importantly — the SAME LoginRateLimiter instance as the
JSON-RPC path, so an attacker cannot get a fresh budget of password
guesses by moving to an app port. Only the transport differs, an HTML
form instead of JSON-RPC, because a browser being sent to an app cannot
speak JSON-RPC.
2FA comes for free: a session still pending its TOTP step fails
validate(), so the gate rejects it without knowing what a second factor
is.
Details worth keeping:
- 401, not a redirect. A redirect to a login page is indistinguishable
from the app itself redirecting, and machine clients would follow it
and parse HTML as their API response.
- Cookie and Authorization are stripped before proxying. The app has no
use for the node session and must never be able to log or forward it.
- The challenge page names and pictures the app being opened, so the
visitor can confirm what they are authenticating to.
- device_tokens grew `apps: Option<Vec<String>>` and verify_for_app for
machine clients. None = node-wide, which every existing companion
token is; migrating them by guessing a scope would silently revoke
access nobody asked to revoke. An empty list is rejected rather than
minted, since it reads as unrestricted while authorising nothing.
The rollout is necessarily per-app and the gate is built to say so. A
container publishing 0.0.0.0:<port> claims every host address, so the
gate cannot bind that port until the app is pinned to bind: 127.0.0.1
and recreated — gate-first is impossible, and all-at-once would recreate
every container on a node simultaneously. Every port it cannot claim is
logged at warn each sweep and recorded in GateStatus::unprotected,
surfaced by security.app-gate-status. The failure mode being designed
against is a gate that binds nothing, logs at debug, and reports success
while every app stays exactly as open as before — worse than no gate,
because it stops anyone looking. Same reasoning that ruled out an
nft drop-in, whose absence is a silent no-op.
Not yet done: pinning the 39 gated ports to loopback, repointing
HiddenServicePort at the gate, and on-node verification.
Tests: 21/21 appgate, workspace builds clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
archipelago