archy/docs/combined-test-plan-2026-07-22.md

# Combined test session — 2026-07-22 batch (one sitting)

Staged on **framework-pt** (`100.65.115.109`) AND **archi thinkpad** (this
machine's node) so everything can be tested in one pass. Items marked ✅ were
already verified by the agent on a node; ❑ items need a human.

**What's in this batch:** mesh message/DM persistence across restarts ·
first-message + DM announce fix · 15s announce poll · radio hot-swap modal
(probe / keep-as-is / apply-settings) · whisper beam-1 (release-gated, see §D) ·
real-time wallet push (0-conf tx shows in seconds) · calm Lightning
"still starting" notice · external tx-explorer fallback with consent modal +
wallet-settings On-chain tab · apps open ABOVE modals with the launch
animation · mempool installs no longer blocked by a resyncing ElectrumX ·
[pending: other agents' two push sets — section F fills in when their code
lands].

## H. Wallet & explorer (new — test on the thinkpad node, it's pruned)

1. ❑ **Real-time tx display:** send a small on-chain amount to this node's
   wallet → the balance and the yellow "unconfirmed" transaction appear
   within a few seconds of broadcast, no refresh, no wallet action.
2. ❑ **External explorer consent:** with no local Mempool app running, tap a
   transaction → amber consent modal explains it opens on another node's
   mempool (default tx1138.com, placeholder mempool.guide, editable) →
   Open Explorer opens `<explorer>/tx/<hash>` in a new tab. Tick "don't ask
   again" and confirm the next tap opens directly.
3. ❑ **Wallet Settings → On-chain tab:** explorer URL editable, warning shown,
   "don't warn" toggle; tabs now read Channels / Cashu / Fedi / Ark / On-chain
   and fit on one row (check mobile too).
4. ❑ **Modal → app animation:** on a node WITH Mempool running, open
   Transactions and tap a tx → the Mempool app animates in ABOVE the modal
   (previously loaded invisibly underneath); closing it returns to the modal.
5. ❑ **Lightning "still starting":** right after a node restart, try opening a
   channel → either it just works (silent retry) or a calm amber ⏳ notice
   appears — never the red "Failed to connect to peer" error.

---

## A. Staged state (agent-verified before you start)

- ✅ Dev binary (persistence + announce seeder + hot-swap) on
  `/usr/local/bin/archipelago`, service healthy, no crash-loop.
- ✅ Frontend bundle with the new device modal at `/opt/archipelago/web-ui`.
- ✅ Seeder re-ran: `automations.yaml` upgraded v1→v2 (first-message announce),
  `configuration.yaml` rest block at `scan_interval: 15`, HA restarted clean.
- ✅ `mesh-messages.json` persisting + restored across a service restart.
- ✅ `mesh.probe-device` returns real firmware details for the plugged stick.

## B. Mesh history survives restarts (the "messages go missing" fix)

1. ❑ Open Mesh chat — your existing DM/channel history from today is visible.
2. ❑ Send one channel message and one DM (either direction).
3. ❑ Reboot the whole node (not just the service). After it's back: history
   still there, including the two new messages, correct timestamps/senders.
4. ❑ Send a NEW message to another node right after the reboot and confirm the
   other side receives it (this exercises the send-seq fix — before it, the
   first post-reboot sends were silently dropped by peers as replays).

## C. Speaker announcements

1. ❑ Have another node send a **public channel** message → speaker announces
   sender + text within ~15s (was ~30s).
2. ❑ Have another node send you a **DM** → speaker announces it the same way.
3. ❑ Restart Home Assistant (or the node) → the last old message is NOT
   re-announced (no announce storm).
4. ❑ (First-message case — the original bug — only reproducible on a node with
   an empty history: optional, covered by agent verification of the guard.)

## D. Voice (regression + speed)

1. ❑ "Hey Jarvis, what's the block height" and one fuzzy phrasing — same
   correct answers as before (no behavior change is the pass condition).
2. ⓘ The ~45% faster speech-to-text (whisper beam-1) ships via the **signed
   catalog in the release** — it is NOT on the node during this test session.
   Benchmarked on this exact hardware: identical transcripts, 0.94s → 0.51s.

## E. Radio hot-swap modal (your Reticulum stick is already plugged in)

1. ❑ Open the web UI anywhere — within ~30s a "Mesh Radio Detected" modal
   appears showing the stick on `/dev/ttyACM0`, with a card of what's on it
   (firmware badge: Reticulum RNode / MeshCore / Meshtastic + current
   name/region/channels where the firmware exposes them).
2. ❑ Press **Keep As Is** → mesh connects using the radio exactly as flashed
   (check Mesh → Device tab: connected, firmware type correct; nothing on the
   radio changed).
3. ❑ Unplug the stick, plug the old MeshCore one → the modal appears AGAIN
   (every plug re-triggers, same or different /dev path).
4. ❑ This time press **Set Up with Archipelago Settings** → second screen
   shows channel `archipelago`, your region, and the node's RF params (the
   validated Portugal preset on this fleet) BEFORE anything is written;
   confirm → radio provisions and joins the mesh.
5. ❑ Swap sticks once more with no UI interaction except "Keep As Is" — chat
   still works end-to-end afterwards (hot-swap without ceremony).

## F. Companion pairing + mobile onboarding (other agent — push set #1, MERGED)

1. ❑ Companion app: pair with the node via the new named QR (device tokens) —
   pairing completes instantly, device appears in the paired-devices list.
2. ❑ Remote access now rides the embedded FIPS mesh (WireGuard replaced):
   with the phone OFF the node's WiFi, the companion still reaches the node.
3. ❑ The reworked mobile onboarding/intro overlay screens flow correctly on
   first launch of the new APK (in-tarball APK is the 27MB build).
4. ❑ (Push set #2 from the other agents is still pending — the release waits
   for it; this staged build does NOT include it yet.)

## G. Quick regressions

1. ❑ Pine launcher page (:10380) still shows the live node card; "Connect
   Pine to WiFi" button loads without JS errors.
2. ❑ Mobile Home: wallet card directly under My Apps (G5 from the voice epic).
3. ❑ Mesh RF settings panel (Mesh → Device) still loads and saves.

## H. LoRa radio firmware flashing (Heltec V3/V4, new — extends Section E)

Full v1 scope is 3 firmware families × 2 boards (6 cells); mark each cell
tested on real hardware vs. code-reviewed only as this is run.

1. ❑ From the hot-swap modal's step 1 (device already probed), press
   **Flash Firmware…** → new step shows firmware-family + board pickers and
   the erase-confirmation checkbox; "Erase & Flash Now" stays disabled until
   family, board, AND the checkbox are all set.
2. ❑ Confirm what's currently on the test stick via the existing probe
   BEFORE flashing it — don't flash the only known-good device without a
   fallback board on hand.
3. ❑ Prefer a spare Heltec V3/V4 for the first destructive erase+flash run;
   only exercise a primary/in-use stick once the flow is proven safe.
4. ❑ MeshCore → Heltec V3: erase + write completes, progress bar and log
   tail update live, ends at "Flash complete".
5. ❑ Meshtastic → Heltec V3: same, using the extracted `*.factory.bin` from
   the esp32s3 release zip.
6. ❑ Reticulum/RNode → Heltec V3: `archy-rnodeconf --autoinstall` path
   completes (no raw esptool erase/write step for this family — see
   `mesh/flash.rs` doc comment).
7. ❑ Repeat 4-6 against a Heltec V4. Confirmed 2026-07-23 on real hardware:
   V4 uses the ESP32-S3's native-USB JTAG/serial peripheral (vid:pid
   303a:1001, generic to every native-USB ESP32-S3 board, not V4-specific)
   — so unlike V3's CP2102 bridge chip, V4 is permanently NOT auto-matchable
   by vid:pid. Board auto-detect should fail closed for it every time
   (manual board selection required, "couldn't confirm automatically"
   warning shown) — this is expected steady-state behavior, not a gap to
   close later.
8. ❑ After a successful flash, the modal automatically re-probes and shows
   the NEW firmware's badge/details — same as unplugging and replugging
   (Section E item 3), but without physically touching the cable.
9. ❑ Deliberately test a failure path once (disconnect the board mid-write,
   or point at a bad cached asset) — confirm the error surfaces in the
   progress log AND that `docs/troubleshooting.md`'s "LoRa radio firmware
   flash failed" recovery steps (BOOT+RST bootloader entry, manual esptool/
   rnodeconf command) actually get the board back to a flashable state.
10. ❑ Cancel button only appears (and only works) while still in the
    "Downloading firmware…" stage — once erasing/writing starts, no cancel
    affordance is offered.
11. ❑ **Boot-loop regression (2026-07-23 incident)**: after a *failed* flash
    (e.g. kill network access mid-download to force a failure), confirm the
    mesh listener does NOT auto-resume — `journalctl -u archipelago` should
    show a single `Leaving mesh listener stopped after failed flash` line
    and then go quiet for that device, not a repeating `mesh::serial:
    Opened serial port... Starting Meshcore handshake` cycle every few
    seconds. Reconnect manually via the hot-swap modal afterward and confirm
    it connects normally (the board itself should be untouched — the
    download fails before esptool/rnodeconf ever runs).
12. ❑ Separately, force a device to flap connected/disconnected a few times
    in under 20s each (e.g. a marginal USB connection) and confirm
    `reconnect_delay` in the logs actually escalates (5s → 10s → 20s → ...)
    rather than resetting to 5s on every attempt — see
    `STABLE_SESSION_THRESHOLD` in `mesh/listener/mod.rs`.

---

After this passes: fold the batch + other agent's work into the next release
(OTA binary + frontend tarball + catalog regen/sign/publish for pine-whisper
3.4.2), then re-run `tests/lifecycle/run-gate.sh` on .228 (back online as
Tailscale `shorty-s`).