integration: preserve deployed 1.8.0 OTA work

This commit is contained in:
archipelago
2026-06-30 05:08:17 -04:00
parent f4f45c1a09
commit df9d3a55be
39 changed files with 1066 additions and 203 deletions
+82
View File
@@ -304,6 +304,88 @@ phases 26 (`dual-ecash-design.md`).
## 8b. SESSION STATE + RESUME (updated 2026-06-26) — READ §8b "CURRENT STATE + RESUME" FIRST
### ▶ SESSION i (2026-06-30) — CURRENT HANDOFF / 1.8.0 OTA RESUME
**Branch/worktree:** currently on `bitcoin-version-bulletproof`, not `main`. Worktree is dirty.
Do **not** discard mesh changes: they include E2E/transport indicator plumbing and the Meshtastic
receive-path fixes below. Separate recovery note: `docs/SESSION-1.8.0-OTA-PROGRESS.md`.
**What was done this session:**
1.**Local Rust release gate fixed and green.** `cargo test -p archipelago --bin archipelago` is
green: **849/849** after fixing stale tests and the invalid `fedimint-clientd` manifest
(`cpu_limit` was `0.25`, invalid for the current schema; now integer). `cargo check -p archipelago`
also green after mesh edits.
2.**Catalog/release static gates green.** `python3 scripts/check-app-catalog-drift.py --release
--strict` is green. `scripts/check-release-manifest.sh` is green for the currently staged
`1.7.99-alpha` manifest/artifacts. `npm run build` and `npm run type-check` are green.
3. ✅ **Frontend unit gate fixed.** `npx vitest run --silent` now green: **81 files / 668 tests**. Fixes
were test-only: add `router.onError` to the login test router mock and update the `AppIconGrid`
mobile unresolved-new-tab expectation to match current app-launcher behavior.
4. ✅ **Workstream F harness gap closed.** `tests/lifecycle/bats/cascade-uninstall.bats` now asserts
uninstall progress truthfulness via backend `uninstall-stage`: stage must be parseable, monotonic,
below 100 before terminal absence, and present before the app disappears. Non-destructive skip-mode
parse check is green: `ARCHY_PASSWORD=dummy bats tests/lifecycle/bats/cascade-uninstall.bats` → 7 skip-ok.
5. ✅ **3ccc → .116 Meshtastic receive bug taken over and partially live-validated.** Context: `3ccc`
is the stock/non-Archy Meshtastic peer. The bug was LoRa text from `3ccc` not surfacing in
`.116` `mesh.messages`. Root causes/fixes:
- The prior attempted fix dropped any packet older than 10 minutes by `rx_time`; live `.116` logs
showed `FromRadio.packet` from `!433e3ccc` being dropped as stale (`rx_time` about an hour old).
The window is now **24h**, so recent radio FIFO/store-forward backlog surfaces instead of vanishing.
- Radios with unset clocks can report tiny nonzero epoch values; those are now treated as unknown,
not stale.
- Serial prevalidation was rejecting valid `FromRadio.queueStatus` frames (`field 11`, live bytes like
`5a04100e1810`) as corrupt payloads; field 11 and other modern non-message `FromRadio` variants
are now accepted/ignored instead of poisoning the stream.
- Focused Meshtastic tests green: **8/8**, including `packet_to_inbound_frame_accepts_recent_meshtastic_backlog`
and `packet_to_inbound_frame_accepts_stock_peer_with_unset_clock`.
- Deployed patched binary to **.116**: sha256
`028ec6ff9a60ca8970c081987457d78ed1c517cd81f7089f51b9a01745b5c3c4` at `/usr/local/bin/archipelago`.
Service active. Post-deploy checked window showed `FromRadio field=11` accepted and no new
`Dropping stale ... !433e3ccc` entries.
- There are stale other-agent `RXDIAG` shell watcher processes on `.116`; leave them unless they
actively interfere.
6. ✅ **Phase-3 Quadlet read-only check on .116 skip-clean.** Copied lifecycle tests to `.116` and ran
`bats bats/use-quadlet-backends-install.bats`: **6/6 skip-clean** because no backend `.container`
units exist. This confirms `use_quadlet_backends` is not active on `.116`; Phase-3 remains a rollout gate.
**Commands/results worth trusting:**
- `cargo test -p archipelago --bin archipelago` → 849/849 green.
- `npx vitest run --silent` from `neode-ui/` → 81 files / 668 tests green.
- `npm run build` from `neode-ui/` → green, bundle `index-CYaDgfX3.js`.
- `python3 scripts/check-app-catalog-drift.py --release --strict` → green.
- `scripts/check-release-manifest.sh` → green for **v1.7.99-alpha** staged artifacts.
- `tests/release/run.sh --manifest` was rerun after `cargo fmt`; it previously reached frontend tests,
which are now fixed. Re-run it from scratch as the next static gate.
**Remaining blockers / decisions before 1.8.0 OTA:**
1. **Release version metadata is not 1.8.0 yet.** `releases/manifest.json`, Cargo, and npm still say
`1.7.99-alpha`; `CHANGELOG.md` top says `v1.8.00-alpha` (note double zero). Do not silently publish
until the release version naming is decided (`1.8.0-alpha` vs `1.8.00-alpha` vs `1.8.0`).
2. **Workstream B signing is blocked on the offline release-root mnemonic.** `docs/workstream-b-signing-runbook.md`
says catalog distribution/embedded manifests are live, but authenticity requires the publisher to pin
`RELEASE_ROOT_PUBKEY_HEX` and sign `releases/app-catalog.json` with `RELEASE_MASTER_MNEMONIC`.
This cannot be automated by an agent without the offline mnemonic.
3. **Phase-3 `use_quadlet_backends` is implemented but default-off.** Completing this requires explicit
node/fleet flag rollout plus backend reinstall/migration verification. `.116` currently skip-clean only.
4. **Bitcoin multi-version coordinated rollout is still separately owned/blocked by its runbook.** See
`docs/bitcoin-version-bulletproof-rollout.md`; do not repoint `bitcoin-knots:latest` before fixed binary
is fleet-wide.
5. **True RF validation of 3ccc requires either a live 3ccc send or waiting for another FIFO/backlog packet.**
Parser/unit coverage and `.116` logs strongly validate the drop-path fix, but no human was available to
send a fresh 3ccc message during this session.
**Immediate next steps for the next agent:**
1. Run `tests/release/run.sh --manifest` from repo root again; frontend unit failures are fixed, so expect
it to pass or continue from the next failing stage.
2. If `.116` is still the canary, monitor logs after any 3ccc activity:
`journalctl -u archipelago --since "<time>" | grep -Ei "!433e3ccc|3ccc|Dropping stale|Meshtastic received text|FromRadio field field=2"`.
3. Decide/reconcile version naming for the actual 1.8.0 OTA, then use the release scripts intentionally
(do not run `create-release.sh` casually: it commits/tags and requires `main` + clean tree).
4. If pursuing Workstream B completion, get the offline release mnemonic from the publisher and follow
`docs/workstream-b-signing-runbook.md` exactly.
5. If pursuing Phase-3 Quadlet, enable `ARCHY_USE_QUADLET_BACKENDS=1` only on a canary first and run the
Quadlet/lifecycle gates before considering fleet rollout.
### ▶ SESSION h (2026-06-26) — LATEST, RESUME FROM HERE
**Canonical resume detail: memory `project_session_resume_2026_06_23b` (▶️ top of MEMORY.md).**
+45
View File
@@ -0,0 +1,45 @@
# 1.8.0 OTA Session Progress
Updated: 2026-06-29
Current scope:
- Preserve existing mesh work: E2E indicators, FIPS/Tor transport indicators, typed-message paths, Meshtastic region/channel provisioning, and dirty Meshtastic receive-attempt changes.
- Take over the `3ccc` stock Meshtastic peer bug: LoRa text from `3ccc` to Archipelago `.116` does not surface in `mesh.messages`.
- Keep release-gate fixes already made in this session.
Local gate status so far:
- `cargo test -p archipelago --bin archipelago`: green, 849/849 after Meshtastic fixes.
- `python3 scripts/check-app-catalog-drift.py --release --strict`: green.
- `npm run type-check`: green.
Key changes made so far:
- Added cascade uninstall progress truthfulness assertion to `tests/lifecycle/bats/cascade-uninstall.bats`.
- Fixed release catalog drift filters and regenerated catalog metadata.
- Fixed invalid `apps/fedimint-clientd/manifest.yml` `cpu_limit` schema value.
- Updated stale/tight Rust tests without changing production behavior.
Remaining non-automatable / operational gates:
- Workstream B signing is blocked on the offline `RELEASE_MASTER_MNEMONIC`; code + runbook exist, but the publisher must pin/sign the release-root catalog.
- Phase-3 Quadlet backend rollout is implemented behind `use_quadlet_backends` and default-off. The gate skip-passes until explicitly enabled on a node; flipping it fleet-wide requires a coordinated flag rollout plus backend reinstall/migration verification.
- `.116` read-only `use-quadlet-backends-install.bats`: 6/6 skip-clean; no backend `.container` units, so Phase-3 is not active on that node.
- Release metadata still says `1.7.99-alpha` in `releases/manifest.json`; changelog top is `v1.8.00-alpha`. Cutting an actual 1.8.0 OTA requires an explicit version/manifest update.
Do not discard:
- `core/archipelago/src/mesh/listener/decode.rs`
- `core/archipelago/src/mesh/listener/session.rs`
- `core/archipelago/src/mesh/meshtastic.rs`
3ccc bug current hypothesis:
- The prior attempted Meshtastic fix added a hard stale-packet filter using `rx_time`.
- Stock Meshtastic radios without GPS/RTC can report tiny nonzero epoch values until time sync.
- That would make live `3ccc` packets look older than 10 minutes and get dropped before `mesh.messages`.
- Current patch treats implausibly early `rx_time` values as unknown rather than stale.
.116 live validation:
- `.116` reachable by SSH; `archipelago` active; `/dev/mesh-radio -> ttyUSB0` attached.
- Recent logs show repeated `FromRadio.queueStatus` frames (`field 11`, bytes like `5a04100e1810`) being rejected by the serial frame prevalidator as invalid payloads.
- Current patch accepts `FromRadio.queueStatus` as a valid ignored frame so non-message status frames no longer look like corrupt serial data.
- Focused Meshtastic tests: green, 7/7.
- Updated patch deployed to `.116` as binary sha `028ec6ff9a60ca8970c081987457d78ed1c517cd81f7089f51b9a01745b5c3c4`.
- After redeploy, logs show `FromRadio field=11` accepted and no new `Dropping stale ... !433e3ccc` entries in the checked post-deploy window.
- There are stale other-agent shell watcher processes on `.116` referencing `RXDIAG`; leave alone unless they interfere.
+171
View File
@@ -0,0 +1,171 @@
# Archipelago Hardware Signer — Design Notes (PSBT + Nostr)
> Status: **exploratory / spec stub** (2026-06-24). No code yet. This captures the
> hardware-selection reasoning and architecture for a small, air-gapped, super-secure
> signing device built around the Tropic Square **TROPIC01** secure element, intended
> to integrate with Archipelago as an external signer.
## 1. Goal
A small, super-secure, air-gapped handheld device that:
- Signs **Bitcoin PSBTs** for the Archipelago wallet.
- (Stretch / dual-function) Signs **Nostr events** for the node's sovereign identity.
- Communicates **only via QR** (camera in, screen out) — no USB data path, no radio in
use. Pure air-gap, same threat model as SeedSigner but with a real audited secure element.
- Anchors key-at-rest security and RNG in the **TROPIC01** open-source secure element.
## 2. The critical curve caveat
**TROPIC01's signing engine supports P-256 (ECDSA) and Ed25519 (EdDSA) — NOT secp256k1.**
Bitcoin and Nostr both require secp256k1. Therefore:
- The secure element is the **vault + RNG + attestation**, not the signer.
- The seed lives encrypted inside TROPIC01 (tamper mesh, pairing, secure channel).
- The host MCU does the actual **secp256k1 ECDSA (Bitcoin)** and **Schnorr / BIP-340
(Taproot + Nostr)** signing in software.
- TODO before committing: re-check whether a firmware revision adds secp256k1 — it's
open RISC-V silicon and has been a community ask. If/when it lands, this design gets
materially stronger (signing in-silicon).
## 3. Architecture (two chips)
```
[ QR in ] --> Camera (OV2640)
|
Host MCU (ESP32-S3) <--SPI--> TROPIC01 (Mini Board)
| (seed vault, RNG,
Touch screen secure channel, attest)
|
[ QR out ] <-- Display (signed PSBT / signed event)
```
- **Host MCU** drives camera, touch screen, QR parse/render, PSBT + Nostr logic, and
the secp256k1/Schnorr signing.
- **TROPIC01** protects the seed at rest and supplies the TRNG + secure boot/attestation
over an authenticated+encrypted SPI channel.
## 4. Hardware selection
### 4.1 MCU — the camera-ease vs radio-purity fork
| | **ESP32-S3** (recommended) | **RP2350** |
|---|---|---|
| Camera | Native DVP interface; huge QR-scan code ecosystem | No camera peripheral — bit-bang over PIO (harder) |
| Radios on die | WiFi + BLE present (con for air-gap purists) | **None** |
| Security | Secure boot, flash encryption | Cortex-M33 + TrustZone, signed boot, OTP |
| secp256k1 in SW | Fine (240 MHz dual-core) | Fine (150 MHz dual-core M33) |
| Price (chip / board) | ~$3 / ~$6 | ~$1.20 / ~$5 |
**Pick: ESP32-S3 (N16R8 — 16MB flash / 8MB PSRAM).** The camera is the hard part of the
build and the S3 is the only cheap MCU with a native camera interface. PSRAM matters for
holding camera frames during QR decode. The on-die radio is the one downside — acceptable
because trust is anchored in the TROPIC01, not the MCU. If radio-on-die is a hard no,
switch to RP2350 and accept harder camera bring-up. (SeedSigner deliberately chose a
no-WiFi Pi Zero 1.3 for exactly this reason — the concern is legitimate.)
### 4.2 Camera
- **OV2640** 2MP module — standard ESP32-cam sensor, code everywhere. ~$24.
### 4.3 Thin touch screen
Pick by review legibility (the whole security value is the human verifying address +
amount before tap-to-approve):
- **2.0" IPS ST7789 capacitive, 240×320 — recommended.** Easiest to read a full Bitcoin
address/amount. ~$812.
- 1.69" rounded-rect IPS ST7789 + CST816 cap touch — best size/compactness balance.
~$710.
- 1.28" round (GC9A01 + CST816) — smallest/thinnest but **too cramped** for address
verification; skip for a signer.
**Do not go below ~1.69".** Use capacitive (not resistive) touch for a thin glass-front
tap-to-confirm feel.
### 4.4 TROPIC01 board (from the Tropic Square order form)
All options speak SPI (wires to the S3 the same way). Two-board plan:
- **Development: TROPIC01 USB DevKit (€50)** — STM32 + USB-to-SPI stick. Bring up the
secure-element stack (pairing, key gen, secure channel) on a PC first, independent of
the camera/screen work.
- **Final device: TROPIC01 Mini Board (€9.50)** — small easy-to-solder module exposing
SPI; solder straight to the S3's SPI bus inside the enclosure.
- Skip: Standalone Sample (€5, bare QFN — needs hot-air), Raspberry Pi / Arduino Shields
(wrong host form factor), MIKROE Click (€20, only if you have a mikroBUS rig).
### 4.5 Rough BOM
| Item | ~Cost |
|---|---|
| ESP32-S3 N16R8 board | $68 |
| OV2640 camera | $24 |
| 2.0" cap-touch IPS | $812 |
| TROPIC01 Mini Board | €9.50 |
| (Dev only) TROPIC01 USB DevKit | €50 |
**Core device BOM ≈ $2030** + TROPIC01 Mini Board, before enclosure/battery.
## 5. Dual-function: Nostr signer
Genuinely viable and a natural fit — **Nostr signs with Schnorr/BIP-340 over secp256k1,
the same scheme as Bitcoin Taproot.** So Nostr signing reuses the secp256k1+Schnorr code
already needed for Bitcoin — near-zero marginal firmware cost.
### 5.1 One seed → two separated keys
From the single seed in the TROPIC01:
- **Bitcoin:** BIP-32/39/84 HD derivation.
- **Nostr:** **NIP-06** deterministic derivation (`m/44'/1237'/…`) → `nsec`/`npub`.
One backup, two independent identities, no cross-contamination.
### 5.2 Cold vs hot tension
| | Bitcoin | Nostr |
|---|---|---|
| Frequency | Rare, high-value | Frequent, often interactive |
| Natural transport | QR / PSBT — air-gap perfect | Apps want real-time signing |
| Air-gap comfort | Excellent | Fine for occasional events, painful for chat |
Two possible modes:
1. **Air-gapped QR Nostr signer (recommended):** app shows unsigned-event QR → camera
scan → touch approve → signed-event QR back. Great for high-value/infrequent events
(root identity, profile/metadata, key rotation, announcements). Keeps 100% air-gap.
2. **Connected NIP-46 "bunker" over USB/serial:** enables interactive real-time signing
but **breaks the air-gap** and reintroduces the USB/radio attack surface. Not
recommended for this device.
### 5.3 Recommendation
Keep it **cold for both roles.** The device guards the Bitcoin spending key *and* the
high-value Nostr **identity** key — neither ever touches a network. Day-to-day Nostr
chatter uses a separate hot software key; the hardware device protects only the
identity-defining key you can't afford to leak. Avoids putting a hot key next to cold
Bitcoin funds.
## 6. Archipelago integration
- Slots in as an **external signer** path alongside the existing wallet flow — does not
touch the orchestrator. Archipelago builds PSBT → renders QR (animated QR for large
txs) → device scans → touch review → returns signed-PSBT QR → Archipelago broadcasts.
- Especially apt given Archipelago's Nostr/Blossom catalog + node-identity direction
(see `dht-distribution-design.md`): the device becomes the **hardware root of trust**
for both halves of a node's identity — its `npub`/DID and its Bitcoin keys — aligning
with the sovereign/secure/rootless north star.
## 7. Open items / next steps
- [ ] **Pin budget:** confirm the S3 GPIO/SPI budget fits camera DVP + display SPI +
TROPIC01 SPI simultaneously. (Biggest unknown before buying.)
- [ ] Confirm current TROPIC01 firmware secp256k1 status (could remove the §2 caveat).
- [ ] Define QR payload formats for both roles (PSBT vs unsigned Nostr-event JSON) so a
single scan→approve→return firmware loop handles either transparently.
- [ ] Animated/multi-part QR strategy for large PSBTs.
- [ ] Seed provisioning ceremony into the TROPIC01 (gen on-device via its TRNG; never
import in clear).
- [ ] Enclosure + power (battery vs USB-power-only-while-airgapped).
- [ ] Decide: ESP32-S3 (radio present) vs RP2350 (no radio, harder camera) — final call.
+74
View File
@@ -0,0 +1,74 @@
# Workstream B — Signed app-catalog: completion runbook
**Status (2026-06-28):** The registry-distributed manifest pipeline is live — nodes fetch
`releases/app-catalog.json` from the OTA mirror and embed manifests (origin-wins, disk
fallback). What remains for Workstream B is **authenticity**: pin the release-root anchor and
ship a *signed* catalog so nodes can cryptographically verify the publisher.
Today the catalog is **accepted unsigned** ("migration window") and the anchor is **unpinned**
(`core/archipelago/src/trust/anchor.rs:21``RELEASE_ROOT_PUBKEY_HEX = None`). Completing B is
a coordinated ceremony that **only the publisher can run** — it needs the offline
`RELEASE_MASTER_MNEMONIC`, which is not (and must not be) stored on any node or build host.
## Why this is gated on you (not automatable)
- The signing key is an **offline mnemonic** you hold (`archipelago ceremony gen` output, backed
up offline / via `seed.reveal`). It is intentionally absent from the repo and all hosts.
- Order matters: once a binary **pins** the anchor, a catalog carrying a signature from the
*wrong* key is **hard-rejected fleet-wide** (`trust/signed_doc.rs:79`). Unsigned and
correctly-signed catalogs are both accepted; only a *mismatched* signature breaks nodes.
- So the pinned pubkey and the signature MUST come from the same key, shipped consistently.
## The ceremony (run from `core/`, with your mnemonic)
```bash
# 0. (only if you don't already have a release-root key) generate one and back the
# mnemonic up OFFLINE. Prints the pubkey hex + signer did:key.
cargo run --release -p archipelago -- ceremony gen
# 1. Print the release-root pubkey hex for the anchor (idempotent; same mnemonic → same key)
RELEASE_MASTER_MNEMONIC="word1 word2 …" cargo run --release -p archipelago -- ceremony pubkey
# → copy the 64-char hex.
# 2. Pin it in code:
# core/archipelago/src/trust/anchor.rs:21
# - pub const RELEASE_ROOT_PUBKEY_HEX: Option<&str> = None;
# + pub const RELEASE_ROOT_PUBKEY_HEX: Option<&str> = Some("<64-char-hex-from-step-1>");
# 3. Sign the published catalog in place (inserts `signature` + `signed_by` over the
# canonical JSON — re-run after ANY catalog regen, since signing covers the exact bytes):
RELEASE_MASTER_MNEMONIC="word1 word2 …" \
cargo run --release -p archipelago -- ceremony sign releases/app-catalog.json
# 4. Verify locally before shipping (optional sanity): a node build with the pinned anchor
# should log "app-catalog: release-root signature verified (<did>)" rather than
# "self-consistent but anchor not pinned".
```
## Ship order (backward-compatible)
1. Commit the **signed** `releases/app-catalog.json` + the `anchor.rs` change together.
2. Push the signed catalog to the OTA mirror (gitea-vps2 `main`) — old binaries (no pinned
anchor) still accept it (verified-but-unconfirmed); nothing breaks.
3. Build + OTA the binary with the pinned anchor. New nodes now **verify** the catalog against
the anchor. (This is the normal release path — gate the tag per the ship-ritual.)
4. **Later / optional hardening:** once the whole fleet is on the pinned-anchor binary, flip
the policy from "accept unsigned (migration window)" to "reject unsigned" in
`container/app_catalog.rs` (the `SignatureStatus::Unsigned` arm). Do this LAST — while any
node still runs an unsigned catalog it must keep being accepted.
## Env-override escape hatch (no rebuild)
For staging/canary you can pin the anchor without editing code via
`ARCHY_RELEASE_ROOT_PUBKEY=<hex>` (`trust/anchor.rs:23`) on a single node, then sign the catalog
and confirm that node verifies it before baking the constant in.
## What's already done (so this is the only remaining step)
- Catalog distribution + manifest embedding: live (this session's `169ff2e2` published the
corrected catalog to the mirror).
- `ceremony gen|pubkey|sign` tooling: shipped (`core/archipelago/src/ceremony.rs`).
- Verify path: `trust::verify_detached` accepts unsigned, verifies signed against the anchor,
hard-rejects mismatches (`trust/signed_doc.rs`).
- Detached-signature schema fields (`signature`/`signed_by`) already part of the signed
preimage (`container/app_catalog.rs`).