docs(10-04): C-3 FAILED — three live nodes share their SSH host keys
Audit checklist item C-3 ("the highest-value check here") is no longer
UNVERIFIED. It failed, and the failure is a live F-03 instance rather than a
theoretical one.
Three distinct fleet nodes — archipelago-1, archy-x250-beta and archipelago —
present byte-identical ECDSA, ED25519 and RSA host key fingerprints. Two of
them (archipelago-1, archy-x250-beta) also present the same TLS certificate,
so they share the TLS private key as well.
Gathered read-only and remotely: ssh-keyscan plus an anonymous TLS handshake.
No node was logged into, nothing was written, nothing was rotated. A weaker
instrument than the checklist's on-node commands, chosen because it needs no
access and therefore covers the reachable fleet rather than two nodes — and it
is sufficient for the FAIL condition, which is any fingerprint appearing twice.
Ruled out the obvious alternative (one machine registered three times on the
tailnet): all three answered live TCP within the same minute, and tailscale
ping resolves them to different physical endpoints on different continents
under different tailnet accounts.
One finding worth more than the count: `archipelago` has a UNIQUE TLS cert
(CN=austin-sapien) and SHARED SSH host keys, because it was renamed and
server.set-name re-mints the cert via regenerate_tls_cert() while touching
nothing else. So TLS uniqueness is not evidence that a node's key material is
per-node — any renamed node gets a unique certificate for free. Checked on TLS
alone, that node would have looked clean. Recorded because it justifies the
audit script reporting the two key classes separately instead of issuing one
node-level verdict.
All three are listed under "shared verdict, deliberately not rotated" with the
reason and the next step. A verification task that remediates is a
verification task that takes a node offline.
Also records what this does NOT establish, each with the evidence still
needed: same-ISO provenance, the script's own verdict on those nodes, that a
rotation preserves the operator's live session on real hardware, that
host_secrets reaches system.stats on a real node, and the four nodes that were
unreachable at scan time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
co-authored by
Claude Opus 5
Claude Opus 5
parent
373c3bb302
commit
a806a658a6
@@ -95,23 +95,113 @@ absent host key is a **fail-closed** state (generation never succeeded), not a s
|
||||
Audit checklist item C-3 (`docs/security/ENTROPY-SEED-AUDIT-2026-07-31.md` §855), described
|
||||
there as "the highest-value check here".
|
||||
|
||||
### Status: PARTIAL — see the table below
|
||||
### Status: **FAILED — with finding.** Recorded 2026-08-02.
|
||||
|
||||
<!-- gsd:c3-evidence -->
|
||||
> **This section names live fleet nodes that are still running shared key material.
|
||||
> Review it before this repository is made public** (`docs/OPEN-SOURCE-READINESS-PLAN.md`).
|
||||
> Digests below are truncated; the fingerprints of public keys are public data — every SSH
|
||||
> handshake offers them — but there is no reason to make a target list convenient.
|
||||
|
||||
**Three distinct live fleet nodes share all three of their SSH host keys. Two of those three
|
||||
also share their TLS certificate, and therefore their TLS private key.** This is not a
|
||||
theoretical exposure: it is F-03 in production, today.
|
||||
|
||||
#### Method
|
||||
|
||||
Gathered **remotely and read-only** — no node was logged into, nothing was written to any node,
|
||||
nothing was rotated. Host keys came from `ssh-keyscan`, which is what every SSH client does
|
||||
before it decides whether to trust a host, and certificates from an anonymous TLS handshake:
|
||||
|
||||
```bash
|
||||
ssh-keyscan -T 6 <node> | ssh-keygen -lf -
|
||||
openssl s_client -connect <node>:443 </dev/null 2>/dev/null \
|
||||
| openssl x509 -noout -fingerprint -sha256 -subject
|
||||
```
|
||||
|
||||
This is a deliberately weaker instrument than the checklist's on-node commands, and it was chosen
|
||||
because it needs no access and can therefore cover the whole reachable fleet rather than two
|
||||
nodes. What it can prove is exactly the FAIL condition: *any fingerprint appearing on two nodes*.
|
||||
|
||||
#### Result
|
||||
|
||||
| Node label | SSH host keys (ECDSA/ED25519/RSA, truncated) | TLS cert sha256 (truncated) | Cert CN |
|
||||
|---|---|---|---|
|
||||
| `archipelago-1` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `62:F6:A6:02…` | `archipelago` |
|
||||
| `archy-x250-beta` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `62:F6:A6:02…` | `archipelago` |
|
||||
| `archipelago` | `8WJplzKW…` / `lQgRXZ1n…` / `ym+gMOio…` | `7C:6B:CD:98…` | `austin-sapien` |
|
||||
| `archipelago-5` | `/bmgd6jS…` / `SpaNfLLf…` / `hhVFABi3…` | `95:FE:EB:C7…` | `archipelago.local` |
|
||||
| `archi-dev-box` | `8hFU7QGM…` / `GAxNAcgX…` / `Tv7AfaVp…` | (no :443 listener) | — |
|
||||
| `archy-dev-pa` | `JtD/RM0a…` / `XD2A5OVL…` / `esIBpbWk…` | not probed | — |
|
||||
| `framework-pt` | `oicpsj3Y…` / `zxA1/kRU…` / `oxi+tMli…` | `88:85:CE:CC…` | `framework-pt` |
|
||||
| `shorty-s` (`.228`) | `YVsgrv8M…` / `D/5n851i…` / `YMFLUerk…` | `4D:98:D4:9B…` | `shorty-s` |
|
||||
|
||||
Unreachable at scan time, so **UNVERIFIED**: `archy-x250-dev`, `archy-x250-pa`, `archy-x250-r2`,
|
||||
`quantumterminal`.
|
||||
|
||||
#### That the three are genuinely different machines, not one host seen three times
|
||||
|
||||
The obvious alternative explanation for identical host keys is a single machine registered on the
|
||||
tailnet more than once. Ruled out:
|
||||
|
||||
- All three answered a live TCP connection on port 22 within the same minute. One `tailscaled`
|
||||
instance serves one tailnet identity, so three simultaneously-live addresses are three hosts.
|
||||
- `tailscale ping` resolves them to **different physical endpoints**: `archy-x250-beta` answers
|
||||
from `178.38.147.13` (and over the Frankfurt DERP), while `archipelago-1` and `archipelago`
|
||||
answer from `45.20.199.86` on different source ports — a different continent for the first,
|
||||
and two distinct machines behind one NAT for the other two.
|
||||
- They are owned by different tailnet accounts.
|
||||
|
||||
#### Why `archipelago` has a different TLS cert but the same SSH keys
|
||||
|
||||
Its cert CN is `austin-sapien`, not the image default `archipelago`. That is the signature of a
|
||||
node that was **renamed** through `server.set-name`, which re-mints the TLS cert via
|
||||
`regenerate_tls_cert()` so the SAN matches the new hostname — and touches nothing else.
|
||||
|
||||
This is worth stating plainly because it is a trap: **TLS uniqueness alone is not evidence that
|
||||
a node's key material is per-node.** Any renamed node gets a unique certificate for free while
|
||||
its SSH host keys stay exactly as the image shipped them. Had C-3 been checked on TLS
|
||||
fingerprints only, `archipelago` would have looked clean. The SSH host key is the reliable
|
||||
signal, and this is why the audit script treats the two classes separately and reports which one
|
||||
is shared rather than issuing a single node-level verdict.
|
||||
|
||||
#### What this does NOT establish — UNVERIFIED
|
||||
|
||||
| Claim | Status | Evidence still needed |
|
||||
|---|---|---|
|
||||
| The three nodes were flashed from the **same ISO** | UNVERIFIED | Not required for the FAIL — shared host keys are the exposure however they got there — but the ISO build id would tell us how many other downloads carry the same keys. Needs on-node `/opt/archipelago/` provenance. |
|
||||
| The audit script's verdict on those three nodes | UNVERIFIED | `sudo /opt/archipelago/scripts/security/host-secrets-audit.sh --detect` on each. Requires the OTA carrying this plan's runtime payload to land, or the script to be hand-staged. Predicted `shared`; predicted is not observed. |
|
||||
| A rotation preserves the operator's own session | UNVERIFIED **on hardware** | Checkpoint steps 4–6: run `--apply --yes` on one disposable node from a session you are willing to lose, confirm that session survives, confirm a second connection shows the expected mismatch. The harness proves the script's ordering and its abort path; it cannot prove that `systemctl reload ssh` keeps a real forked session alive. |
|
||||
| `host_secrets` reaches `system.stats` on a real node | UNVERIFIED | Needs a build carrying this plan deployed to the dev pair, then a `system.stats` call. Proven in unit tests against the file contract only. |
|
||||
| The four unreachable nodes | UNVERIFIED | Re-run the scan when they come back online. |
|
||||
|
||||
#### Consequence
|
||||
|
||||
`archipelago-1`, `archy-x250-beta` and `archipelago` are a **confirmed live F-03 instance**.
|
||||
Anyone holding a copy of the ISO these nodes were flashed from holds their SSH host private keys,
|
||||
and for the first two, their TLS private key as well — enough for undetectable SSH host
|
||||
impersonation and transparent MITM of the web UI.
|
||||
|
||||
None of them was rotated as part of this verification, and that is deliberate: this checkpoint
|
||||
verifies, it does not remediate, and remediating a node inside a verification task is how a
|
||||
verification task takes a node offline. They are recorded below.
|
||||
|
||||
---
|
||||
|
||||
## Nodes with a `shared` verdict, deliberately not rotated
|
||||
|
||||
None recorded yet.
|
||||
|
||||
Any node that reports `shared` and is not rotated in the same session MUST be added here with the
|
||||
date and the reason, so that the standing consequence of `detect-report-then-apply` is a visible
|
||||
list rather than an assumption.
|
||||
|
||||
| Node label | Date detected | Why not rotated | Owner |
|
||||
| Node label | Date detected | Why not rotated | Next step |
|
||||
|---|---|---|---|
|
||||
| — | — | — | — |
|
||||
| `archipelago-1` | 2026-08-02 | Detected by remote fingerprint comparison during C-3, not by an operator running the script. In real use; rotating it inside a verification task is exactly what the task forbids. | Stage the script, run `--detect`, then rotate from a session the operator is willing to lose. |
|
||||
| `archy-x250-beta` | 2026-08-02 | Same. Also shares its **TLS private key** with `archipelago-1`, so it is the more urgent of the two. Reached over a DERP relay from another continent — the least recoverable node in the set if a rotation goes wrong. | Rotate from physical or console access if available; otherwise rotate TLS first, confirm, then SSH. |
|
||||
| `archipelago` | 2026-08-02 | Same. TLS is already unique (the node was renamed, which re-mints the cert); only its SSH host keys are shared. | `--apply --yes` will rotate SSH only — the detect pass flags the classes separately, so this node's already-unique TLS pair is left alone. |
|
||||
|
||||
**Nobody has been told their `known_hosts` is about to break.** Three nodes here are in real use;
|
||||
the rotation is one-way and every existing entry for them dies with it. Sequencing that is an
|
||||
operator decision, which is the whole content of D-06.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user