security: untrack operations docs; scrub infra identifiers from public docs

Operations docs move out of git entirely rather than being sanitized. They
stay on disk for local use and are gitignored, so the Phase 6 export (which
takes HEAD) can never carry them. 15 files: the fleet runbook, hotfix
process, node inventories, internal trackers, session handoffs, the key
rotation/signing-posture records, and the open-source plan itself.

For the docs that remain public, infra identifiers are replaced with things
that are better documentation rather than placeholders: curl examples now
use `archipelago.local`, the product's own mDNS name, so a reader can run
them as-is instead of substituting an address that was never theirs.

Deliberately NOT scrubbed, both verified as functional rather than leaked:
- `tx1138.com` is the shipped default block explorer (DEFAULT_TX_EXPLORER in
  useTxExplorer.ts, surfaced in WalletSettingsModal). Product behavior.
- `git.tx1138.com` in core/container/{image_policy,registry}.rs is a retired-
  registry constant the code matches on to strip stale entries from legacy
  node configs. Removing it would break migration for older nodes.
- `192.168.1.254` in bulletproof-containers.md is the LAN gateway in a podman
  bug description, and `192.168.1.x` in user-walkthrough.md is already generic.

Whether a personal domain should be the shipped explorer default in a public
product is a separate product question, not a security one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-07 10:08:18 -04:00
co-authored by Claude Opus 5
parent 39eb6b0553
commit fda7feda60
24 changed files with 49 additions and 5085 deletions
-244
View File
@@ -1,244 +0,0 @@
# KEY-02 — fleet host-secret detection and rotation (F-03, deployed half)
Phase 10 plan 10-04. Companion to `docs/security/KEY-02-ROOTFS-EVIDENCE.md`, which covers the
build half (10-03).
10-03 stopped the exposure growing: the ISO no longer bakes SSH host keys or a TLS keypair into
the shared rootfs, and first-boot regeneration now fails closed instead of setting its completion
marker on a failed run. That does **nothing** for nodes already in the field, which is exactly
where the exposure sits — a node that hit the old fail-open path is running the SSH host key and
TLS private key that every downloader of that ISO also holds, and it will never try again.
This document records the two human decisions that govern the deployed half.
---
## D-06 rotation trigger
**Chosen option: `detect-report-then-apply`** — recorded 2026-08-02.
Verbatim option id as written in `10-04-PLAN.md`: **`detect-report-then-apply`**
("Detect and report on boot; rotate only when an operator runs the script with an explicit apply
flag").
### Why
Rotating an SSH host key is one-way. Every `known_hosts` entry for that node breaks, on every
machine that has ever connected to it, and the old private key is destroyed by the swap. The
fleet is reached over Tailscale for day-to-day work and several nodes are remote — `.228` is at
a remote site and is in real use (CLAUDE.md). `auto-on-boot` would fire that rotation on many
nodes simultaneously during an OTA rollout, with no advance notice and no operator holding the
new fingerprints. A node whose only access path is SSH and whose tooling pins the host key
becomes unreachable until someone clears the entry; a rotation that fails partway on a remote
node needs physical console access to recover, which for `.228` means a site visit.
Against that, the cost of `detect-report-then-apply` is that exposure persists on any node whose
operator does not act. That cost is bounded by making the verdict **visible**: detection runs at
boot on every node and the verdict reaches `system.stats`, so an exposed node shows up in the
dashboard without shell access. The exposure becomes measured rather than assumed, and the list
of nodes still to rotate is a fact on a screen rather than a guess.
This also matches the project's standing policy that changes are verified on the dev pair
(archi-dev-box + x250-dev) before they reach the fleet (CLAUDE.md, `feedback_dev_pair_before_ota`).
A rotation that fires unattended on first boot after an OTA cannot be dev-paired — by the time it
has been observed on the dev pair it has already run everywhere.
### What this decision binds
- `scripts/security/host-secrets-audit.sh` defaults to `--detect`, which is read-only.
- `--apply` **without** `--yes` prints its plan and exits 0 having touched nothing, so a mistyped
invocation is inert.
- `image-recipe/configs/archipelago-host-secrets-audit.service` ships in **detect-only** mode.
It contains no apply path. Making the boot unit rotate would require editing the unit, which is
a deliberate act, not a default.
- `--apply --yes` refuses to do anything unless the detect pass returned `shared`. A node whose
verdict is `per-node` cannot have its keys rotated by this script even by explicit command —
the guard against "operator runs it on the wrong node" is structural, not procedural.
### Consequence recorded honestly
Any node whose verdict comes back `shared` and which is never revisited stays exposed
indefinitely. The mitigation is the visibility, not the automation. The list under
"Nodes with a `shared` verdict, deliberately not rotated" below exists so that no such node is
quietly forgotten, and it is part of this plan's acceptance criteria that the list is kept.
---
## How a node decides
Four on-disk signals, evaluated in this precedence order by
`scripts/security/host-secrets-audit.sh --detect`. Every verdict carries the evidence strings
that produced it, and each evidence string names the file it was read from.
| # | Signal | Source |
|---|---|---|
| 1 | mtime of each host key / the TLS key against the first-boot anchor | `/var/lib/archipelago/.secrets-regenerated`, falling back to `/root/.luks-archipelago.key` then `/etc/machine-id` |
| 2 | The fail-open fingerprint: marker present **and** a `WARNING:` line in the first-boot log | `/var/log/archipelago-first-boot-secrets.log` |
| 3 | 10-03's durable failure record | `/var/lib/archipelago/first-boot-secrets.failed` |
| 4 | Rootfs provenance | `/opt/archipelago/rootfs-identity-stripped` |
Verdicts: `per-node`, `shared`, `fail-closed-missing`, `unknown`.
**`per-node` is never reported on the strength of an absent signal.** With no anchor at all the
verdict is `unknown`, and while a durable failure record stands the verdict is `unknown` rather
than `per-node` — the node's own generator most recently reported failure, so a clean-looking
mtime is not evidence of success.
Signal 4 changes the meaning of missing material rather than adding to the shared/per-node
question: on a node flashed from a 10-03-or-later ISO the rootfs shipped identity-free, so an
absent host key is a **fail-closed** state (generation never succeeded), not a shared one.
---
## C-3 — per-node host key and TLS uniqueness
Audit checklist item C-3 (`docs/security/ENTROPY-SEED-AUDIT-2026-07-31.md` §855), described
there as "the highest-value check here".
### Status: **FAILED — with finding.** Recorded 2026-08-02.
> **This section names live fleet nodes that are still running shared key material.
> Review it before this repository is made public** (`docs/OPEN-SOURCE-READINESS-PLAN.md`).
> Digests below are truncated; the fingerprints of public keys are public data — every SSH
> handshake offers them — but there is no reason to make a target list convenient.
**Three distinct live fleet nodes share all three of their SSH host keys. Two of those three
also share their TLS certificate, and therefore their TLS private key.** This is not a
theoretical exposure: it is F-03 in production, today.
#### Method
Gathered **remotely and read-only** — no node was logged into, nothing was written to any node,
nothing was rotated. Host keys came from `ssh-keyscan`, which is what every SSH client does
before it decides whether to trust a host, and certificates from an anonymous TLS handshake:
```bash
ssh-keyscan -T 6 <node> | ssh-keygen -lf -
openssl s_client -connect <node>:443 </dev/null 2>/dev/null \
| openssl x509 -noout -fingerprint -sha256 -subject
```
This is a deliberately weaker instrument than the checklist's on-node commands, and it was chosen
because it needs no access and can therefore cover the whole reachable fleet rather than two
nodes. What it can prove is exactly the FAIL condition: *any fingerprint appearing on two nodes*.
#### Result
| Node label | SSH host keys (ECDSA/ED25519/RSA, truncated) | TLS cert sha256 (truncated) | Cert CN |
|---|---|---|---|
| `archipelago-1` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `62:F6:A6:02…` | `archipelago` |
| `archy-x250-beta` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `62:F6:A6:02…` | `archipelago` |
| `archipelago` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `7C:6B:CD:98…` | `austin-sapien` |
| `archipelago-5` | `/bmgd6jS…` / `SpaNfLLf…` / `hhVFABi3…` | `95:FE:EB:C7…` | `archipelago.local` |
| `archi-dev-box` | `8hFU7QGM…` / `GAxNAcgX…` / `Tv7AfaVp…` | (no :443 listener) | — |
| `archy-dev-pa` | `JtD/RM0a…` / `XD2A5OVL…` / `esIBpbWk…` | not probed | — |
| `framework-pt` | `oicpsj3Y…` / `zxA1/kRU…` / `oxi+tMli…` | `88:85:CE:CC…` | `framework-pt` |
| `shorty-s` (`.228`) | `YVsgrv8M…` / `D/5n851i…` / `YMFLUerk…` | `4D:98:D4:9B…` | `shorty-s` |
Unreachable at scan time, so **UNVERIFIED**: `archy-x250-dev`, `archy-x250-pa`, `archy-x250-r2`,
`quantumterminal`.
#### That the three are genuinely different machines, not one host seen three times
The obvious alternative explanation for identical host keys is a single machine registered on the
tailnet more than once. Ruled out:
- All three answered a live TCP connection on port 22 within the same minute. One `tailscaled`
instance serves one tailnet identity, so three simultaneously-live addresses are three hosts.
- `tailscale ping` resolves them to **different physical endpoints**: `archy-x250-beta` answers
from `178.38.147.13` (and over the Frankfurt DERP), while `archipelago-1` and `archipelago`
answer from `45.20.199.86` on different source ports — a different continent for the first,
and two distinct machines behind one NAT for the other two.
- They are owned by different tailnet accounts.
#### Why `archipelago` has a different TLS cert but the same SSH keys
Its cert CN is `austin-sapien`, not the image default `archipelago`. That is the signature of a
node that was **renamed** through `server.set-name`, which re-mints the TLS cert via
`regenerate_tls_cert()` so the SAN matches the new hostname — and touches nothing else.
This is worth stating plainly because it is a trap: **TLS uniqueness alone is not evidence that
a node's key material is per-node.** Any renamed node gets a unique certificate for free while
its SSH host keys stay exactly as the image shipped them. Had C-3 been checked on TLS
fingerprints only, `archipelago` would have looked clean. The SSH host key is the reliable
signal, and this is why the audit script treats the two classes separately and reports which one
is shared rather than issuing a single node-level verdict.
#### What this does NOT establish — UNVERIFIED
| Claim | Status | Evidence still needed |
|---|---|---|
| The three nodes were flashed from the **same ISO** | UNVERIFIED | Not required for the FAIL — shared host keys are the exposure however they got there — but the ISO build id would tell us how many other downloads carry the same keys. Needs on-node `/opt/archipelago/` provenance. |
| The audit script's verdict on those three nodes | UNVERIFIED | `sudo /opt/archipelago/scripts/security/host-secrets-audit.sh --detect` on each. Requires the OTA carrying this plan's runtime payload to land, or the script to be hand-staged. Predicted `shared`; predicted is not observed. |
| A rotation preserves the operator's own session | UNVERIFIED **on hardware** | Checkpoint steps 46: run `--apply --yes` on one disposable node from a session you are willing to lose, confirm that session survives, confirm a second connection shows the expected mismatch. The harness proves the script's ordering and its abort path; it cannot prove that `systemctl reload ssh` keeps a real forked session alive. |
| `host_secrets` reaches `system.stats` on a real node | UNVERIFIED | Needs a build carrying this plan deployed to the dev pair, then a `system.stats` call. Proven in unit tests against the file contract only. |
| The four unreachable nodes | UNVERIFIED | Re-run the scan when they come back online. |
#### Consequence
`archipelago-1`, `archy-x250-beta` and `archipelago` are a **confirmed live F-03 instance**.
Anyone holding a copy of the ISO these nodes were flashed from holds their SSH host private keys,
and for the first two, their TLS private key as well — enough for undetectable SSH host
impersonation and transparent MITM of the web UI.
None of them was rotated as part of this verification, and that is deliberate: this checkpoint
verifies, it does not remediate, and remediating a node inside a verification task is how a
verification task takes a node offline. They are recorded below.
---
## Nodes with a `shared` verdict, deliberately not rotated
Any node that reports `shared` and is not rotated in the same session MUST be added here with the
date and the reason, so that the standing consequence of `detect-report-then-apply` is a visible
list rather than an assumption.
| Node label | Date detected | Why not rotated | Next step |
|---|---|---|---|
| `archipelago-1` | 2026-08-02 | Detected by remote fingerprint comparison during C-3, not by an operator running the script. In real use; rotating it inside a verification task is exactly what the task forbids. | Stage the script, run `--detect`, then rotate from a session the operator is willing to lose. |
| `archy-x250-beta` | 2026-08-02 | Same. Also shares its **TLS private key** with `archipelago-1`, so it is the more urgent of the two. Reached over a DERP relay from another continent — the least recoverable node in the set if a rotation goes wrong. | Rotate from physical or console access if available; otherwise rotate TLS first, confirm, then SSH. |
| `archipelago` | 2026-08-02 | Same. TLS is already unique (the node was renamed, which re-mints the cert); only its SSH host keys are shared. | `--apply --yes` will rotate SSH only — the detect pass flags the classes separately, so this node's already-unique TLS pair is left alone. |
**Nobody has been told their `known_hosts` is about to break.** Three nodes here are in real use;
the rotation is one-way and every existing entry for them dies with it. Sequencing that is an
operator decision, which is the whole content of D-06.
---
## Operator runbook — rotating one node
Run this from a session you are willing to lose, on **one node at a time**. Never on `.228` or
any node in real use without arranging access recovery first.
```bash
# 1. Detect. Read-only; safe on any node, including production.
sudo /opt/archipelago/scripts/security/host-secrets-audit.sh --detect
cat /var/lib/archipelago/host-secrets-audit.json
# 2. Dry run. Prints the plan, touches nothing, exits 0.
sudo /opt/archipelago/scripts/security/host-secrets-audit.sh --apply
# 3. Rotate. Only proceeds if the verdict is `shared`.
sudo /opt/archipelago/scripts/security/host-secrets-audit.sh --apply --yes
# 4. WITHOUT closing that session, prove it survived:
echo still-here
# 5. From a second terminal, expect a host-key mismatch warning. That is the
# correct outcome. Update known_hosts against the fingerprints printed by
# step 3 (also in /var/lib/archipelago/host-key-rotation.json), never by
# blindly accepting whatever is offered.
ssh-keygen -R <node>
ssh <node>
# 6. The web UI will present a new self-signed cert. A fresh browser trust
# prompt is expected and is the correct outcome.
```
The script reloads sshd rather than restarting it. A reload re-execs the listener while
already-forked session children keep running, which is why the operator's own SSH session
survives its own rotation. `restart` would kill it, and on a remote node with no console that is
unrecoverable.
Old fingerprints are written to `/var/lib/archipelago/host-key-rotation.json` **before** the
swap, so an operator who loses access anyway can still identify what changed.
-448
View File
@@ -1,448 +0,0 @@
# KEY-03 — Signing posture after the Bitcoin Core wallet deletion
> **What this document is.** The evidence-backed record of how Archipelago's Bitcoin signing
> posture stands after Phase 10 KEY-03. It supersedes, for the Bitcoin Core wallet specifically,
> the target state described in `docs/security/PSBT-SIGNING-ARCHITECTURE.md` §8 Phase 1 — that
> phase planned to *convert* Core's wallet to watch-only; **D-07b deleted the path instead.**
>
> **Governing decisions:** `.planning/phases/10-key-material-hardening/10-CONTEXT.md`
> **D-07b** (final KEY-03 scope — delete, do not migrate) and **D-07c** (the deferred BDK cold
> vault, recorded so it is not lost with the code). D-07b supersedes D-07 and D-07a's conditional
> migration.
>
> **Audit finding closed:** F-13 (High) —
> `docs/security/ENTROPY-SEED-AUDIT-2026-07-31.md:604`, remediation register R-04.
---
## Bitcoin Core wallet path — deleted (D-07b)
### What was deleted
| Symbol | Kind | Location before deletion |
|---|---|---|
| `handle_bitcoin_init_wallet_from_seed` | `async fn` | `core/archipelago/src/api/rpc/bitcoin.rs:161-295` |
| `"bitcoin.init-wallet-from-seed"` | JSON-RPC dispatch arm | `core/archipelago/src/api/rpc/dispatcher.rs:122-124` |
### The defect (F-13)
The handler loaded the encrypted seed, derived the **BIP-84 account extended private key**
(`crate::seed::derive_bitcoin_xprv`, `bitcoin.rs:188`), stringified it (`:189`), and imported
`wpkh(xprv/0/*)` and `wpkh(xprv/1/*)` (`:230-231`) into a Bitcoin Core descriptor wallet created
with `disable_private_keys = false` (`:203`) and an **empty** wallet passphrase (`:205`).
The result was a **second copy of the node's spending key**, persisted in Core's `wallet.dat`
inside the Bitcoin container's data volume, with no Argon2 passphrase — while the first copy sits
in the daemon's Argon2 + ChaCha20-Poly1305 envelope written `0600`
(`core/archipelago/src/seed.rs:238-269`, `:318-324`). That duplication, into weaker protection,
was the entire finding.
### Evidence that deletion was the right close (re-established for this task, not inherited)
The four D-07a evidence points, verified again against the tree before anything was removed:
**1. No caller anywhere.** Repo-wide search across `core/`, `neode-ui/src`, `scripts/`, `web/`,
`apps/`, `tests/` and `docs/`, excluding `core/target`, `node_modules` and `.git`:
```
$ grep -rn 'bitcoin\.init-wallet-from-seed' core/ neode-ui/src scripts/ web/ apps/ tests/ docs/
core/archipelago/src/api/rpc/dispatcher.rs:122: "bitcoin.init-wallet-from-seed" => {
$ grep -rn 'handle_bitcoin_init_wallet_from_seed' core/ neode-ui/src scripts/ web/ apps/ tests/ docs/
core/archipelago/src/api/rpc/bitcoin.rs:161: pub(super) async fn handle_bitcoin_init_wallet_from_seed(
core/archipelago/src/api/rpc/dispatcher.rs:123: self.handle_bitcoin_init_wallet_from_seed(params).await
docs/UNIFIED-TASK-TRACKER.md:208: §8 Phase 1). `handle_bitcoin_init_wallet_from_seed` passes
docs/security/ENTROPY-SEED-AUDIT-2026-07-31.md:607:(`handle_bitcoin_init_wallet_from_seed`):
docs/security/PSBT-SIGNING-ARCHITECTURE.md:147: `handle_bitcoin_init_wallet_from_seed`, `core/archipelago/src/api/rpc/bitcoin.rs:161-294`).
```
Exactly one occurrence of the method name (its own dispatcher registration) and two of the symbol
in code (its definition and the dispatcher call). The three remaining symbol hits are prose in
documentation — the audit, the task tracker, and the PSBT architecture spec — not callers. No
frontend, script, test or other Rust module invoked it.
**2. LND is the wallet the product actually drives.** Across all of `neode-ui/src`, every
`bitcoin.*` RPC call is read-only status: `bitcoin.getinfo` (14 call sites),
`bitcoin.prune-status` (3), `bitcoin.onion` (1). There are **no** `bitcoin.*` wallet operations.
The wallet UI (`Web5Wallet.vue`, `SendBitcoinModal.vue`) sends via `lnd.sendcoins`, estimates via
`lnd.estimatefee`, and reads balance via `lnd.getinfo`.
**3. The wallet it creates never existed on the reference node.** Verified live on
**archi-dev-box, 2026-08-02**, against the running `bitcoin-knots` container (read-only RPCs
only — see the census section for the exact commands and the standing ban on
`listdescriptors true`):
```
listwalletdir → { "wallets": [ "gatewayd-02004b91…", "gatewayd-03443c0c…", "" ] }
listwallets → [ "" ]
```
**There is no wallet named `archipelago`** — the handler's default `wallet_name`
(`bitcoin.rs:170-173`). It has never run on this node. `getwalletinfo` on the one loaded wallet
(the unnamed default) reports:
```
walletname: "" blank: true keypoolsize: 0
txcount: 0 balance: 0.00000000
descriptors: true private_keys_enabled: true
```
`blank: true` with `keypoolsize: 0` and `txcount: 0` is Bitcoin Core's own statement that **no
key was ever imported into it and no transaction ever touched it**. The two `gatewayd-*` entries
are Fedimint gateway wallets, unrelated to the BIP-84 path. The `wallet.dat` at the datadir root
is Core's own legacy default-wallet location, not this handler's output.
**This is one node.** The same check was subsequently run across the reachable fleet — see the
census below: **4 nodes examined and clear, 6 unreachable and therefore unknown.**
**Supporting history evidence:** `git log -S "init-wallet-from-seed"` scoped to
`core/archipelago/src/api/rpc/dispatcher.rs` and `neode-ui/src` returns exactly one commit —
`19dcfd4f feat: BIP-39 master seed for unified key derivation`, the commit that **added** it. No
frontend wrapper was ever written: it was built and never wired up.
**4. It was never remotely reachable.** The endpoint is absent from `UNAUTHENTICATED_METHODS`
(`core/archipelago/src/api/rpc/middleware.rs:5-40`) — so it required an authenticated session —
**and** it additionally re-verified the user's password before touching the seed
(`self.auth_manager.verify_password(password)`, `bitcoin.rs:176-179`). **F-13 was therefore
key-at-rest duplication, not an exposed endpoint.** That is why it was rated High rather than
Critical, and why deleting it is a hardening measure rather than an incident response.
### What was *not* wrong with it
Worth stating so the record is fair, and so the next reader does not mistake the lesson. The
in-memory handling of the xprv string was **careful**: it was zeroized on the error path
(`bitcoin.rs:222`) and on the success path (`:284`), matching the standard set elsewhere in
`seed.rs`. The wallet type was also correct — `createwallet` already passed `descriptors = true`
(`:207`), which is the right foundation.
**The defect was which key went into the wallet, not how the key was held in memory or what kind
of wallet it was.** A watch-only rewrite (xpub + `[fingerprint/derivation]` key origin) would
have been a legitimate fix. Deletion was chosen over rewrite because the endpoint had no caller,
no consumer, and no product role: rewriting it would have produced a correct implementation of
something nothing uses, and left a wallet-creating code path to be maintained and re-audited
forever.
### How F-13 is closed
**By removal, not by conversion to watch-only.** After this change there is no code path in the
daemon that writes the BIP-84 account private key into Bitcoin Core. The only on-node copy of
that key is the daemon's Argon2 + ChaCha20-Poly1305 envelope.
**No migration was performed and none is planned.** D-07's parity-proof migration and its
one-way checkpoint are **withdrawn** (D-07b) — there is no wallet to migrate. If a fleet node is
ever found holding a descriptor wallet this handler created, that is a **finding to surface and
stop on**, not a trigger to auto-migrate: it would mean the endpoint was invoked by hand and that
node's spending key is duplicated in Core, which deserves a human decision rather than an
automated rewrite of a wallet that may hold funds.
### This deletion removes code, not wallets
Stated explicitly so nobody reading the change later has to wonder whether it was destructive:
> **Nothing on disk is touched.** No `wallet.dat` is modified, unloaded or removed. No funds
> move. No LND state, secret, descriptor or seed is altered. The change removes a Rust function
> and a `match` arm — the *path* by which a private key could be imported into Bitcoin Core —
> and nothing else.
This holds even on a hypothetical node where the endpoint had been invoked by hand: deleting the
handler destroys nothing there either. It closes the door; it does not clean the room. Cleaning
up such a wallet, if one is ever found, is a separate human decision (see the census below), and
CLAUDE.md's **"migrations never destroy data"** invariant is not engaged by this change because
there is no migration.
### What deletion does to D-08 and D-09
Neither decision lapses; both are satisfied by a different mechanism.
- **D-08** asked that the spending key exist in exactly one place, with an opt-in air-gapped
path. Deleting the Core import achieves the first half outright. The opt-in path is LND's
existing PSBT round trip, not a Core watch-only wallet — see the next section, including the
recorded verdict on how far that actually goes today.
- **D-09** required a `[fingerprint/derivation]` key origin on emitted descriptors so a hardware
signer can locate its key. With Core's descriptors deleted there are **no Archipelago-emitted
descriptors left to annotate**, so D-09's actual protection moves to the PSBT itself. That is
why `lnd.create-psbt` now inspects and reports the key-origin data its PSBT carries
(`psbt_key_origin_report`, `core/archipelago/src/api/rpc/lnd/wallet.rs`).
### `derive_bitcoin_xprv` is retained deliberately (D-07c)
`crate::seed::derive_bitcoin_xprv` (`core/archipelago/src/seed.rs:231`) lost its only non-test
caller and was **kept**, marked `#[allow(dead_code)]` with the reason in its doc comment. It is
covered by existing tests (`seed.rs:601-602`, `:856`) and it is the derivation **D-07c's deferred
BDK cold vault** — a descriptor wallet in the daemon using the node's own ElectrumX app
(`apps/electrumx`, `electrs_status.rs`) as chain source — will need.
D-07c was considered and deliberately deferred out of Phase 10 (it needs its own phase: a new
dependency and a new UI surface). It is recorded here, and in the function's doc comment, so the
option is not quietly lost along with the code that was deleted. The alternative shape — LND
watch-only via `importaccount` plus remote signing — was considered and rejected for coupling
cold storage to LND's upgrade path.
---
## LND PSBT round trip — what is covered
With Core's wallet deleted, LND is the only wallet Archipelago has, and its PSBT round trip is
the only external-signer path that exists. This section records what that path actually consists
of, what is tested, and — the question that decides whether any of it is an air gap — whether an
externally-held signer can sign a default node's PSBT at all.
### Per-step coverage map
Round trip: **fund → export → sign offline → import → finalize → broadcast.**
| # | Step | Where it lives | `file:line` | Automated test coverage |
|---|---|---|---|---|
| 1 | **Fund** — build a funded PSBT via LND WalletKit `/v2/wallet/psbt/fund` | `lnd.create-psbt` handler | `core/archipelago/src/api/rpc/lnd/wallet.rs:605`; dispatch arm `api/rpc/dispatcher.rs:136` | **Untested.** No LND mock exists; the handler's request/response handling is exercised only by hand. |
| 1a | **Inspect** — report BIP-32 key origin on the funded PSBT | `psbt_key_origin_report` + wiring | `lnd/wallet.rs:1186` (fn), `:1169` (struct), `:705` (call site), `:737` (response field) | **Tested.** 3 unit tests, below. |
| 2 | **Export** — hand the base64 PSBT to the user | UI renders `psbt_base64` for copy | `neode-ui/src/api/rpc-client.ts:407-423`; `neode-ui/src/views/web5/Web5SendReceiveModals.vue:308` | **Partial.** `neode-ui/src/api/__tests__/rpc-client.test.ts:319-323` asserts only that the client calls the method `lnd.create-psbt`; it does not test the payload or the rendering. |
| 3 | **Sign offline** — external signer produces a signed PSBT | **Not in this repo.** No first-party signer ships today. | — | N/A |
| 4 | **Import** — user pastes the signed PSBT back | textarea → `signedPsbtInput` | `Web5SendReceiveModals.vue:102`, `:419-424` | **Untested.** |
| 5 | **Finalize**`/v2/wallet/psbt/finalize` | `lnd.finalize-psbt` handler | `lnd/wallet.rs:743`; dispatch arm `dispatcher.rs:137` | **Untested.** |
| 6 | **Broadcast**`/v2/wallet/tx`, in the same handler | `handle_lnd_finalize_psbt` tail | `lnd/wallet.rs:795` | **Untested.** |
| — | **Rate limiting** — both endpoints at 5 calls / 300s | `RateLimiter` defaults | `core/archipelago/src/rate_limit.rs:68-69` | **Untested for these two methods specifically.** |
**Stated plainly, because an untested path must not be described as verified:** of the six steps,
**one** (the key-origin inspection added by this plan) has automated coverage in the Rust
crate. Steps 1, 4, 5 and 6 have **none** — no test exercises the LND REST calls, the finalize
handler, or the broadcast. Step 2's only test asserts a method name. **No end-to-end test of the
round trip exists**, and none of it has been verified against a real hardware signer.
There is also **no air-gap transport**: no animated QR encode/decode, no `.psbt` file
download/upload. Export and import are copy-paste of base64 in a textarea. The BC-UR v2 / BBQr
design in `PSBT-SIGNING-ARCHITECTURE.md` §4 is unimplemented.
### New tests added by this plan
In `core/archipelago/src/api/rpc/lnd/wallet.rs`'s `mod tests`, with fixtures built
programmatically from the `bitcoin` crate rather than pasted as opaque base64:
| Test | Asserts |
|---|---|
| `psbt_without_derivations_reports_no_key_origin` | A one-input unsigned PSBT with no `bip32_derivation` reports `inputs_with_key_origin: 0` and `all_inputs_have_key_origin: false`. |
| `psbt_with_derivations_reports_key_origin` | The same PSBT with a `(Fingerprint, DerivationPath)` inserted on input 0 reports `1/1` and `true`. |
| `malformed_psbt_is_an_error_not_a_panic` | Non-base64, truncated-PSBT and empty inputs all return `Err`, never panic. |
```
running 3 tests
test api::rpc::lnd::wallet::tests::psbt_with_derivations_reports_key_origin ... ok
test api::rpc::lnd::wallet::tests::psbt_without_derivations_reports_no_key_origin ... ok
test api::rpc::lnd::wallet::tests::malformed_psbt_is_an_error_not_a_panic ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 1014 filtered out
```
`lnd.create-psbt` now returns an additive `key_origin` field:
```json
"key_origin": { "input_count": 1, "inputs_with_key_origin": 0, "all_inputs_have_key_origin": false }
```
It is computed **best-effort**: a decode failure degrades to `null` and logs a warning, never to
an error — a user's send must not fail because an inspection helper could not parse something.
When `all_inputs_have_key_origin` is false the handler emits a `tracing::warn!` with the counts,
because that is the exact condition under which a hardware signer refuses the PSBT. Existing
response fields are unchanged; `handle_lnd_finalize_psbt` and `handle_lnd_create_raw_tx` (the
sibling that deliberately auto-signs with LND's hot keys) were not touched.
### Can an external signer actually sign a default node's PSBT? — **No, not today**
This is the question that separates "we have PSBT plumbing" from "we have air-gapped custody",
and the two must not be allowed to blur.
**Verdict: on a default Archipelago node, an externally-held signer cannot meaningfully sign a
PSBT produced by `lnd.create-psbt`.** The evidence:
1. **The PSBT is funded from LND's own wallet.** `lnd.create-psbt` POSTs to LND's WalletKit
`/v2/wallet/psbt/fund` (`lnd/wallet.rs:672`), which selects UTXOs belonging to **LND's**
wallet. The keys for those inputs are the keys LND holds.
2. **LND's wallet on every node is a full key-holding wallet, created locally.**
`container::lnd::ensure_wallet_initialized` (`core/archipelago/src/container/lnd.rs:86`) calls
`init_wallet_via_rest`, which POSTs `/v1/initwallet` with a `cipher_seed_mnemonic`
(`container/lnd.rs:504-516`) and persists the aezeed backup (`:523-525`). That is a normal
wallet with private keys, not a watch-only one.
3. **No node's `lnd.conf` carries a remote-signing block.** The config Archipelago generates
(`container/lnd.rs:64-79`) contains `bitcoin.node=bitcoind` and the bitcoind RPC settings, and
**no `remotesigner.*` keys at all**.
4. **Nothing in the repo provisions watch-only LND.** A search of `apps/`, `scripts/`,
`core/archipelago/src` and `image-recipe/` for `remotesigner`, `createwatchonly` and
`nochainbackend` returns **zero matches**. There is no code path, script or manifest that sets
any node up this way.
An external signer could only sign these inputs if LND were first provisioned **watch-only
against that signer** — `remotesigner.*` on the node plus `lncli createwatchonly` from the
signer's exported accounts, with the level-3 accounts and the p2tr import step described in
`PSBT-SIGNING-ARCHITECTURE.md` §5.1-5.2. **No fleet node is so provisioned.**
**What therefore ships today is the PSBT *transport*, not air-gapped custody.** The round trip is
real and rate-limited, and it is genuinely useful for signing a PSBT whose inputs belong to some
*other* wallet — but on a default node the signer that holds the input keys is LND itself, so
routing the PSBT out to an external device and back adds a step without moving custody anywhere.
The gap between here and D-08's opt-in air-gapped path is **provisioning, not plumbing**, and
that provisioning is out of scope for Phase 10 (it is `PSBT-SIGNING-ARCHITECTURE.md` §8 Phase 6).
Nothing in the UI currently claims otherwise, and nothing added by this plan does either. If
copy is ever written for this flow, it must not describe it as cold storage on the strength of
the PSBT round trip alone.
### Lightning channel, revocation and HTLC keys are not air-gappable — at all
This is a standing constraint, not a caveat, and it survives every change in this document.
> **A Lightning node's channel, revocation and HTLC keys must sign in real time to answer
> counterparty commitments. They cannot be air-gapped.** A routing node cannot tolerate a
> human-in-the-loop signing step: a delayed response to a commitment update risks a force-close,
> and a missing revocation risks loss. LND remote signing **relocates** these keys to a hardened
> host — it does **not** cool them. There is no configuration, present or future, in which a
> live Lightning node's channel keys are cold.
This is the same limit stated in `PSBT-SIGNING-ARCHITECTURE.md` §5.1 ("Air-gap channel /
revocation / HTLC keys — **No**") and §5.4, whose honesty table remains correct and unmodified.
The consequence for user-facing copy, quoted from §5.4 and repeated here so it cannot be lost:
> *A Lightning routing node's channel keys are necessarily hot. Remote signing moves them to a
> hardened machine; it does not make them cold. Only your on-chain balance can be genuinely
> protected by an offline signer.*
**No wording in this document, or in any document this phase touches, may imply that Lightning
funds can be held cold.** A user who believes their Lightning balance is cold will keep more in
it than they otherwise would, which is exactly the miscalibration that turns an incident into a
loss.
---
## Fleet census — Core descriptor wallets
**Status: run 2026-08-02 — 4 nodes examined and CLEAR, 6 nodes UNCHECKED. No escalation.**
This section answers one question per node: *does this node hold a Bitcoin Core descriptor wallet
that the deleted wallet-init handler created, and does it hold private keys?* It is recorded per
node rather than assumed, because deletion closes the door but does not tell us whether anyone
walked through it before.
The nodes that could **not** be examined are listed with their reasons, not omitted. A census
that quietly drops its failures is worthless — an auditor must be able to see exactly which
machines were looked at and which were not.
### Hard constraint on every command in this census
> **Never run `listdescriptors true`.** The `true` argument makes Bitcoin Core return the
> descriptors **including private keys**, which would print an xprv to a terminal and into a
> transcript — creating the exact exposure this census exists to measure.
> `listwalletdir`, `listwallets`, `getwalletinfo` and `listdescriptors` **with no second
> argument** answer the question completely.
>
> If any output unexpectedly contains a string beginning `xprv`, **stop immediately, do not
> paste it**, and report only that it occurred.
### Commands (re-runnable by an auditor)
Per node, against the Bitcoin Core / Knots container:
```bash
# 0. Does the handler's wallets directory exist at all? An absent directory is
# itself a complete answer for that node — paste the output as-is.
ls -la /var/lib/archipelago/bitcoin/wallets/ 2>&1
# bitcoin-cli is NOT on $PATH inside the container. On archi-dev-box (Knots
# 29.3) it lives at:
# /opt/bitcoin-29.3.knots20260210/bin/bitcoin-cli
# The RPC user is `archipelago`; the password is read from
# /var/lib/archipelago/secrets/bitcoin-rpc-password
# — reference that path, never the value, and prefer -stdinrpcpass so the
# password never appears in a process list or shell history.
# 1. Every wallet on disk, loaded or not.
bitcoin-cli -rpcuser=archipelago -stdinrpcpass listwalletdir
# 2. Currently loaded wallets.
bitcoin-cli -rpcuser=archipelago -stdinrpcpass listwallets
# 3. Per wallet returned: record walletname, private_keys_enabled, descriptors,
# blank, keypoolsize, txcount, balance.
bitcoin-cli -rpcuser=archipelago -stdinrpcpass -rpcwallet=<name> getwalletinfo
# 4. ONLY for a wallet with private_keys_enabled: true — NOTE: no second argument.
# Record descriptor prefixes (`wpkh(...`) only, never a full key string.
bitcoin-cli -rpcuser=archipelago -stdinrpcpass -rpcwallet=<name> listdescriptors
# 5. Which Bitcoin app and version.
bitcoin-cli -rpcuser=archipelago -stdinrpcpass getnetworkinfo | head
```
### Results — examined, 2026-08-02 (4 nodes, all CLEAR)
Run by the operator over Tailscale, read-only RPCs only.
| Node | Tailscale IP | Container | `listwalletdir` | `listwallets` | `archipelago` wallet? | Default wallet state | Verdict |
|---|---|---|---|---|---|---|---|
| **archi-dev-box** | `100.69.68.39` | `bitcoin-knots` | 2× `gatewayd-*`, `""` | `[ "" ]` | **No** | `blank: true`, `keypoolsize: 0`, `txcount: 0`, `balance: 0.00000000`, `descriptors: true` | **CLEAR** |
| **shorty-s** (`.228`) | `100.64.204.114` | `bitcoin-knots` | 1× `gatewayd-*`, `""` | `[ "" ]` | **No** | same | **CLEAR** |
| **archy-x250-beta** | `100.72.136.5` | `bitcoin-core` | 1× `gatewayd-*`, `""` | `[ "" ]` | **No** | same | **CLEAR** |
| **archy-x250-pa** | `100.89.209.89` | `bitcoin-core` | 1× `gatewayd-*`, `""` | `[ "" ]` | **No** | same | **CLEAR** |
On every examined node there is **no wallet named `archipelago`** — the deleted handler's default
`wallet_name`. The only named wallets are Fedimint `gatewayd-*`, unrelated to the BIP-84 path.
The one loaded wallet on each node is Core's unnamed default. It does report
`private_keys_enabled: true`, but also `blank: true` with `keypoolsize: 0`, `txcount: 0` and
`balance: 0.00000000` — **Bitcoin Core's own statement that no key was ever imported into it and
no transaction ever touched it.** It is not the deleted handler's output, and it holds nothing.
**The result holds across two container vintages**`bitcoin-knots` on two nodes and
`bitcoin-core` on two others. That matters: it is not four copies of one image behaving
identically, so the finding is a property of the fleet rather than an artefact of a single build.
**No key material appeared in any output, and `listdescriptors true` was never run.**
### Not examined, 2026-08-02 (6 nodes, with reasons)
| Node | Tailscale IP | Why not checked |
|---|---|---|
| framework-pt | `100.65.115.109` | `Permission denied (publickey,password)` — SSH password rotated, not held |
| archipelago-1 | `100.82.34.38` | `Permission denied (publickey,password)` |
| archipelago | `100.70.96.88` | `Permission denied (publickey,password)` |
| archy-dev-pa | `100.64.83.15` | `Permission denied (publickey,password)` |
| archipelago-5 | `100.114.134.21` | Timed out during SSH banner exchange |
| archy-x250-dev | `100.113.100.55` | Offline — Tailscale reports last seen 2 days prior |
**Password authentication was deliberately not attempted on any of these.** Several fleet nodes
lock PAM quickly on a wrong password, and locking an in-use production node out is a worse
outcome than an incomplete census. These are recorded as UNCHECKED, **not** as clear.
### Conclusion, at the strength the evidence supports
> **No examined node holds a wallet created by the deleted handler, and no examined node holds
> any wallet with keys or funds.** Four nodes, across two container vintages, on 2026-08-02.
**This is deliberately not a claim that "the fleet is clear."** Six nodes were not examined, and
an unexamined node is unknown, not safe. F-13 is closed **by deletion** — the code that could
create such a wallet is gone from every future build, which is true regardless of the census —
and the census adds that no such wallet was found where anyone could look.
### Standing item — finish the census
The six unchecked nodes remain open. **Homed in `docs/UNIFIED-TASK-TRACKER.md`** (the project's
canonical "what's open" list) as *"Finish the Core-wallet fleet census — 6 nodes unchecked"*,
rather than only here, so it is visible to someone who is not already reading a security
document. It is flagged there as a natural fold-in for **KEY-04's on-node work**, which needs
node access anyway — but it is tracked independently so it does not vanish if KEY-04 is
re-scoped.
Re-run the read-only procedure above when credentials or connectivity allow.
### Standing rule if a wallet is found
If any node reports a wallet named `archipelago` (or any descriptor wallet with
`private_keys_enabled: true` that this handler plausibly created), that is a **finding**:
1. **Stop.** Record it here with the node label and wallet name.
2. **Raise it as a blocker.** KEY-03 does not close until a human decides what to do about it.
3. **Do not migrate, unload, rescan or modify it.** D-07b withdrew the migration deliberately.
Rewriting a wallet that might hold funds is exactly the kind of decision that belongs to a
human, and CLAUDE.md's "migrations never destroy data" invariant applies the moment anyone
touches it.
Such a wallet would mean the endpoint was invoked manually before this plan deleted it, and that
node's spending key is duplicated in Core outside the Argon2 envelope.