fix(mesh): stop the flash-triggered device boot-loop, fix stuck/wedged jobs

Three real incidents from live-testing the flash feature on archy-x250-dev,
each traced through logs on the node and fixed at the root:

- A failed flash left the mesh listener auto-resuming into a reconnect loop
  whose backoff reset to its 5s minimum on any momentary connection, even
  mid-boot-loop — every retry's open() toggles DTR/RTS, which resets many
  ESP32 boards, so the retries were themselves sustaining the loop. Backoff
  now only resets after a session runs stably for 20s+, and the flash job
  no longer auto-resumes the listener after a failure (only on success,
  after a settle delay).

- The HTTP client's blanket 30s request timeout covered entire downloads
  (Meshtastic's zip is ~170MB), killing large transfers mid-stream; fixed
  with a per-chunk stall timeout instead of a fixed total-transfer cap. But
  the download's initial request had no timeout at all, so a slow-to-start
  server hung it forever — wedging the single-flash-job guard permanently
  ("already in progress" on every subsequent attempt). Fixed with a bounded
  wait for the response to start, plus an absolute ceiling around the whole
  job as a last-resort safety net.

- archy-rnodeconf's PyInstaller freeze broke its own internal esptool
  invocation (sys.executable pointed at the frozen binary itself instead of
  a real interpreter) — fixed via a new runtime hook. esptool's own
  auto-reset-into-bootloader handshake is separately known to be flaky on
  some CP2102/CH340 boards; it now retries once at a conservative baud
  rate instead of failing outright, and failure logs show the full error
  chain instead of just the outer context message.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-23 02:18:55 +00:00
co-authored by Claude Sonnet 5
parent 7d31ca5d65
commit beff5dd577
6 changed files with 340 additions and 40 deletions
+34
View File
@@ -489,6 +489,40 @@ which esptool; ls -la /usr/local/bin/archy-rnodeconf
for the next detection poll) — the hot-swap modal re-probes automatically
and shows whatever firmware is actually on the board now.
**Known incident (2026-07-23) — reconnect storm / device boot-loop after a
failed flash**: a real Heltec V3 got stuck cycling "connect → partial
handshake → drop" every 5-15s for 5+ minutes after a `mesh.flash-device`
attempt failed with `Reading firmware download stream`. Root cause was two
compounding issues, both now fixed:
1. `spawn_mesh_listener`'s reconnect backoff (`core/archipelago/src/mesh/listener/mod.rs`)
reset to its 5s minimum any time the prior session had been `device_connected`
at all, even for under a second — so a device that connects-then-drops
repeatedly never actually backed off. Every retry's `open()` toggles
DTR/RTS, which resets many ESP32 boards' MCU (native-USB *and*
CP2102/CH340 auto-reset-circuit boards), so the aggressive retries were
themselves *causing* the boot loop, not just observing one. Fixed by only
resetting backoff when a session ran for at least `STABLE_SESSION_THRESHOLD`
(20s) — see that constant's doc comment.
2. `mesh::flash::start_flash_job`'s post-completion handler auto-resumed the
listener unconditionally, even after a *failed* flash, immediately
re-entering the reconnect loop above with no cooldown. Fixed: on failure
the listener is now deliberately left stopped (reconnect manually via the
UI once the board is confirmed alive); on success there's a 5s settle
delay before resuming, so the board finishes booting from the flash
tool's own reset before Archipelago starts probing it again.
3. Separately, the download itself was failing because `mesh::flash`'s HTTP
client had a blanket 30s request timeout that covered the *entire*
download (including streaming a 170MB Meshtastic zip), not just
connection setup — fixed with a per-chunk stall timeout instead of a
fixed total-transfer cap.
If this symptom recurs (rapid repeating `mesh::serial: Opened serial
port... Starting Meshcore handshake` lines in `journalctl -u archipelago`
without a `LoRa firmware flash` job in progress), it's a NEW instance of the
same class of bug, not the one above — check whether backoff is actually
escalating (`Mesh session error: ... (retry in Xs)` — X should grow past 5s
within a few cycles) before assuming it's flashing-related.
---
## General Maintenance