A failed federation sync existed only as a `debug!` line on the node, so a
peer that had not synced in days looked identical in the UI to one that
synced a minute ago. Now the failure is persisted per peer and rendered.
- `FederatedNode.last_sync_error` / `.last_sync_error_at` — the failure-side
mirror of the existing `last_transport` / `last_transport_at` pair.
- `federation::record_sync_result(data_dir, did, outcome)` — records the
message on `Err`, CLEARS both fields on `Ok` so the badge disappears when
the peer recovers. Runs under FEDERATION_STORE_LOCK via the `*_inner`
load/save convention established by plan 01-01. An unknown DID is a silent
Ok that writes nothing, so a peer removed mid-pass is never resurrected by
an in-flight sync's error write. Skips the save entirely when nothing
changed, keeping the steady state read-only rather than rewriting
nodes.json (and contending for the lock) every 90s.
- Message truncated to MAX_SYNC_ERROR_CHARS (256), counted in chars not
bytes so truncation cannot split a UTF-8 sequence (T-01-18).
- The 90s auto-sync loop calls it on both arms; the existing `debug!` line
is kept — persisting is additive, not a replacement for logs.
- `federation.list-nodes` emits both fields when set, omits them when unset.
- NodeList renders a red SYNC badge beside the transport badge on both the
trusted-node and peer rows, message + age in the `title` so the row stays
single-line.
Tests (written first, confirmed failing — 16 compile errors, E0425 on
`record_sync_result` and E0609 on `last_sync_error`):
- persists_error / success_clears_error / missing_did_is_noop /
on_empty_store_is_noop / truncates_long_error
- NodeList: badge present when set, ABSENT when unset (the guard against a
badge that always renders), and present on an observer peer row.
cargo test -p archipelago federation — 42 passed, 0 failed.
vitest NodeList.test.ts — 4 passed. npm run build — green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With Bitcoin Core's wallet deleted, LND's PSBT round trip is the only
external-signer path Archipelago has, and D-09's key-origin protection moves
from Core descriptors (of which none remain) to the PSBT itself.
Adds `psbt_key_origin_report(&str) -> Result<PsbtKeyOriginReport>` to
lnd/wallet.rs, reporting `input_count`, `inputs_with_key_origin` and
`all_inputs_have_key_origin`. An input counts as carrying key origin when
either its `bip32_derivation` or `tap_key_origins` map is non-empty. A PSBT
with zero inputs reports false rather than vacuous truth. Parsed with the
already-present `bitcoin` and `base64` crates; no dependency added.
`lnd.create-psbt` gains an additive `key_origin` object on its response and a
`tracing::warn!` with the counts when key origin is missing, because that is
the exact condition under which a hardware signer refuses the PSBT. Computed
best-effort: a decode failure degrades to `null`, never to an error, so a
user's send cannot fail because an inspection helper could not parse
something. `handle_lnd_finalize_psbt` and `handle_lnd_create_raw_tx` (which
deliberately auto-signs with LND's hot keys) are untouched.
Three tests, with fixtures built programmatically from the `bitcoin` crate
rather than pasted as opaque base64: with-derivations, without-derivations,
and malformed-is-an-error-not-a-panic.
KEY-03-SIGNING-POSTURE.md gains an honest per-step coverage map of the
fund -> export -> sign offline -> import -> finalize -> broadcast round trip.
Of six steps, only the new inspection has automated coverage; steps 1, 4, 5
and 6 have none, and there is no air-gap transport (no animated QR, no .psbt
file exchange) — export/import is copy-paste of base64. Untested paths are
named as untested.
Records the verdict that decides whether any of this is an air gap: on a
default node an external signer CANNOT meaningfully sign a PSBT from
`lnd.create-psbt`, because LND holds the keys for every input it selects.
Evidence: the PSBT is funded from LND's own wallet; `ensure_wallet_initialized`
creates a full key-holding wallet via /v1/initwallet; the generated lnd.conf
carries no `remotesigner.*` block; and a search of apps/, scripts/,
core/archipelago/src and image-recipe/ for remotesigner/createwatchonly/
nochainbackend returns zero matches. No fleet node is provisioned watch-only.
What ships is PSBT transport, not air-gapped custody — the gap is
provisioning, not plumbing.
Adds the standing honesty statement in its own subsection: Lightning channel,
revocation and HTLC keys are NOT air-gappable at all. They must sign in real
time to answer counterparty commitments; remote signing relocates them to a
hardened host, it does not cool them.
Also adds a status banner to PSBT-SIGNING-ARCHITECTURE.md recording that its
Phase 1 was superseded by deletion rather than delivered, so §0's "single
highest-value change" and §2.1's invariant now read against a code path that
no longer exists. Banner only; §5.4's honesty table is byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`handle_bitcoin_init_wallet_from_seed` derived the BIP-84 account extended
*private* key, stringified it, and imported `wpkh(xprv/0/*)` / `wpkh(xprv/1/*)`
into a Bitcoin Core descriptor wallet created with `disable_private_keys=false`
and an empty passphrase. That put a second copy of the node's spending key in
Core's `wallet.dat`, outside the daemon's Argon2 + ChaCha20-Poly1305 envelope.
That duplication into weaker protection was audit finding F-13 (High).
Deleted rather than rewritten watch-only (D-07b supersedes D-07/D-07a):
- No caller anywhere. Repo-wide search leaves exactly one occurrence of the
method name (its own dispatcher registration) and two of the symbol in code
(definition + dispatch call); every other hit is prose in docs.
- LND is the wallet the product drives. Across neode-ui/src every `bitcoin.*`
call is read-only status (getinfo/prune-status/onion); the wallet UI sends
via `lnd.sendcoins`.
- It never ran on archi-dev-box: no wallet named `archipelago` exists there,
and the one loaded wallet reports blank=true, keypoolsize=0, txcount=0.
- It was authenticated AND password-gated, so F-13 was key-at-rest
duplication, not an exposed endpoint.
No migration is performed and none is planned. This removes code, not wallets:
nothing on disk is touched, no funds move, no wallet.dat is modified. If a node
is ever found holding a wallet this handler created, that is a finding to
surface and stop on, not a trigger to auto-migrate.
`seed::derive_bitcoin_xprv` loses its only non-test caller and is retained
deliberately with `#[allow(dead_code)]` and a stated reason: it keeps its
existing test coverage and it is the derivation D-07c's deferred BDK cold vault
will need.
Records the evidence, the D-08/D-09 consequences and the D-07c deferral in
docs/security/KEY-03-SIGNING-POSTURE.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The checkpoint on archi-dev-box proved rotation alone doesn't close FED-07:
the credential file went unique while the RUNNING container kept serving the
compromised one, because the Quadlet path rewrites a unit without restarting
it and fedimint-gateway is classified restart-sensitive, so drift was detected
and deliberately ignored on every tick.
Rotation now records the app id, and the drift check consumes that flag to
recreate even a restart-sensitive app, with a WARN naming the reason. This
mirrors the published-port carve-out a few lines above, which already makes
the same trade for the same reason: a container that is already broken (there)
or already compromised (here) is not protected by leaving it running.
Restart-sensitivity protects working services. A gateway answering to a
credential published in this repository is not working, it is compromised, and
gateway admin can drain Lightning liquidity — indefinite exposure loses to a
few seconds of restart. Rotating-but-only-alerting was rejected: the
monitoring system fires on metric thresholds only, so it would have needed new
event-alert plumbing to deliver something strictly weaker.
Re-verified on the same node, same scenario: rotation at 06:39:23, recreate at
06:39:27, PID 3923125 -> 148426, running credential now matches the file,
container healthy with the same name and ports, gatewayd.db intact at 18 files
with IDENTITY present, 32 containers untouched, no repeat rotation.
3 new tests. Also lands the missing 01-19 and 01-20 SUMMARYs: both had code
committed 2026-07-31 but no summary and no roadmap tick, so they read as
unstarted. Phase 1 is 11/20. FED-09 carries 15h of Tor uptime and 0
permission-fixes across 542 doctor runs on archi-dev-box.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
01-11 stopped new installs from ever taking a shipped credential, but did
nothing for the nodes that already did — those gateways still answer to a
password published in this repository.
rotate_compromised_gateway_credential() detects an EXACT match against the
denylist and replaces the pair; absent, unique, or merely unrecognised values
are left alone and return false. That distinction is the point: an operator
who deliberately set their own credential also has an "unrecognised" one, and
rotating it would be the same class of harm as leaving the default in place.
It hangs off resolve_dynamic_env beside ensure_generated_secrets, gated on the
gateway's app id, so an affected node heals on its next reconcile tick. There
is deliberately no teardown here: the new hash changes the resolved secret env,
which changes secret_env_hash, which the drift check reads as a container-label
mismatch — so the platform's own recreate path rebuilds the gateway around its
unchanged data directory, ports, volumes and name.
Rotation is self-terminating (the value written is not on the denylist, so the
next tick is a no-op) and errors propagate rather than being swallowed, because
the atomic write leaves the previous credential intact on failure.
Bcrypt generation was factored out of ensure_one into write_bcrypt_pair, which
both generation and rotation call — 01-11's SUMMARY claimed such a helper
existed but the arm was still inline, and rotation cannot reuse
ensure_gateway_credential because its idempotent fast path returns early
exactly when the file is present, which is the case rotation acts on.
Also fixes cargo fmt drift left by 42652547 in install.rs.
Verified: 6 new tests, secrets suite 16/16; full suite 1008 passed with one
known wall-clock flake (green 4/4 in isolation). NOT verified on a node —
Task 2's blocking checkpoint has not been run, so FED-07 stays open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bip39::Mnemonic::generate(24) resolves through Mnemonic::generate_in to
&mut rand::thread_rng() INSIDE the bip39 crate (bip39-2.1.0/src/lib.rs:
311-313 -> :296-298 -> :267-283), so the entropy source behind Archipelago's
entire key hierarchy -- node Ed25519 did:key, node Nostr key, FIPS mesh key,
per-identity keys, the BIP-84 wallet, the LND aezeed entropy, and the fleet
release-root SIGNING key -- was chosen by a dependency default rather than
stated at the call site.
Not a vulnerability today: rand 0.8.5's thread_rng is a fork-protected
ChaCha12 CSPRNG seeded from getrandom(2). But it is precisely the structural
shape of the 2026-07-30 COLDCARD entropy defect (T1), where a refactor
silently rebound seed generation to a non-cryptographic PRNG with no compile
error and no test failure.
- New private helper generate_mnemonic_with<R: CryptoRng + RngCore> calls
bip39's injectable generate_in_with; MasterSeed::generate passes OsRng
explicitly, with the rationale pinned in a doc comment
- mnemonic_generation_uses_injected_rng: drives generation from a
deterministic test RNG and asserts the result equals from_entropy(exactly
the bytes that RNG emitted) -- direct proof the INJECTED rng is consumed --
plus a known-answer pin and a determinism check. This test cannot be
written against the previous code: there was no seam to inject through
- mnemonic_generation_is_256_bit: the OsRng path yields 24 words and two
successive productions differ
No change to derivation paths, word count, the empty-BIP-39-passphrase
decision, or the at-rest encryption envelope.
Verified: CARGO_INCREMENTAL=0 cargo test -p archipelago seed:: -> 25 passed,
0 failed.
Full analysis: docs/security/ENTROPY-SEED-AUDIT-2026-07-31.md (F-02, §4, §7).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six code paths configured the Lightning gateway with a bcrypt hash committed
to this repository — and one deploy path with a plaintext password literal —
whenever the per-install secret was missing. Anyone holding a copy of the repo
held the admin credential for every gateway that ever took a fallback.
container::secrets now owns the credential end to end: ensure_gateway_credential
(idempotent, delegates to ensure_one's bcrypt arm) and gateway_bcrypt_hash,
which returns Err when the secret is missing/empty and when the stored value is
on the KNOWN_DEFAULT_GATEWAY_HASHES denylist — so this codebase cannot hand
back the compromised value even to a node already carrying it.
get_app_config was widened to Result so a credential-less install cannot reach
podman run at all; configure_fedimint_lnd takes the resolved hash instead of
re-reading with its own fallback. The four shell paths stop generating
credentials entirely (dropping the htpasswd host dependency) and skip container
creation with a printed reason rather than substituting anything.
Naming converges on the manifest's fedimint-gateway-hash/.pw, with legacy
fedimint-gateway-password values copied forward rather than regenerated so no
node loses a working unique credential. Plan 01-16 owns rotation of installs
already carrying the default.
Verified: cargo build clean; cargo test -p archipelago 999 passed (2
boot_reconciler timing tests failed under concurrent load, green in isolation,
untouched by this diff); bash -n clean on all five scripts; the compromised
literal now appears exactly once in the tree, as the denylist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 2 of 01-01-PLAN.md, closing the gap left after Task 1's initial commit
(2f99db5e):
- record_peer_transport and update_node now hold FEDERATION_STORE_LOCK for
their whole load-mutate-save cycle via the *_inner variants, instead of
calling the public (separately-locked) load_nodes/save_nodes — closing
the same class of race the lock was introduced to fix, just for the two
mutators Task 1 didn't reach.
- Add test_remove_errors_when_tombstone_write_fails: pre-creates the
removed-nodes path as a directory so the tombstone write fails, then
asserts remove_node returns Err AND load_nodes still contains the node —
proving a failed removal never half-applies.
cargo test -p archipelago federation::storage: 14/14 green (was 11, +3 across
Task 1/2). cargo build -p archipelago: no new warnings, no dead-code warnings
on any *_inner fn. Public signatures unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
create-release-manifest.sh's changelog extraction pulls every non-blank
line between the version header and the next "## ", not just "- "
bullets — so the previous commit's "### Known gap" markdown heading and
its paragraph leaked into releases/manifest.json (and, via
sync-whats-new.py, the Settings "What's New" modal) as a malformed,
truncated entry (the closing clarification sentence was cut by the
extractor's 10-line cap).
Rewritten as a single "- " bullet, matching every other CHANGELOG entry,
so it renders cleanly and completely in both the OTA manifest and the
in-app modal instead of showing raw "### " syntax to node operators.
Also folds in core/Cargo.lock's version bump, which create-release.sh's
own commit step omits from its `git add` list.
Same binary/frontend artifacts as the prior commit (identical sha256/
size in the regenerated manifest) — only the changelog text changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Whitespace-only reflow in storage.rs/seed.rs/update.rs (rustfmt line-
wrapping rules) and app_ports.rs (array literal reflow after the port
list grew). No logic change. tests/release/run.sh's cargo-fmt --check
stage was failing on this before v1.7.119-alpha could be cut.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
handle_lnd_createinvoice posted to LND's /v1/invoices with only value/memo,
so LND defaulted private to false and returned invoices with empty
route_hints. Any node whose only channels are private/unannounced was
unpayable through the wallet UI's Receive flow. Diagnosed on
archy-x250-mad2, whose only channel (to Olympus by ZEUS) is private with
~40.8k sats usable inbound; three wallet-UI invoices never received an
HTLC.
Audited every other invoice-creation call site and found a second one with
the identical omission: create_invoice, the seller-side/peer-file paid-
content flow (content.rs -> handler for paid downloads). Same bug, same
fix, wider blast radius than the one-node report suggested -- paid-file
sales were unreceivable on private-channel nodes too.
Extracted both call sites' invoice_body construction into one shared
build_invoice_request_body() that sets private: true unconditionally, and
added a unit test pinning that field so neither site can silently drift
back to false. private:true is harmless on nodes with public channels --
LND still prefers a direct public route and the hint is just an unused
alternate path.
1.2.x's auth-hardening work added Hono secureHeaders() with a default
X-Frame-Options: SAMEORIGIN, which unconditionally blocked the Archipelago
node dashboard's iframe (different origin by port) — a real regression
versus 1.1.0, which never sent this header. Fixed upstream in the botfight
repo (commit 8eb27ed): X-Frame-Options is now conditional on ARCHY_EMBEDDED,
disabled only for the first-party node-embedded instance.
apps/botfights/manifest.yml: image/version -> 1.2.2, adds
ARCHY_EMBEDDED=1 to environment, drops the interim
metadata.launch.open_in_new_tab workaround (no longer needed — the app can
now be framed). app-catalog/catalog.json, scripts/image-versions.sh,
neode-ui/public/catalog.json bumped in lockstep via
scripts/generate-app-catalog.py. core/archipelago/src/fips/app_ports.rs
regenerated (formatting only, same port set).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seed entropy comes from bip39 -> rand::thread_rng -> getrandom(2), which
blocks until the kernel pool is initialized -- but that ordering was
invisible in logs on first-boot ISO flows where the seed is generated
early. MasterSeed::generate() now probes getrandom(GRND_NONBLOCK) and
logs whether the pool was already seeded (warn if it would block).
Also adds a regression test that 64 generated mnemonics are all unique
with sane word diversity, guarding against a fixed/seeded RNG ever
being wired into seed generation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Add FEDERATION_STORE_LOCK (tokio::sync::Mutex<()>) guarding every
load-mutate-save cycle in federation/storage.rs, closing the race where
the 90s auto-sync loop's stale pre-removal snapshot could silently
re-save a peer the operator just removed (no error logged anywhere).
- Split load_nodes/save_nodes/tombstone_did/untombstone_did into thin
locked outer wrappers + lock-free *_inner bodies so remove_node and
add_node can hold the guard across their whole tombstone+save critical
section without self-deadlocking (Mutex is not re-entrant).
- Route load_nodes, save_nodes, remove_node, add_node, tombstone_did,
untombstone_did, set_trust_level, and update_node_state through the
lock (set_trust_level pulled forward from Task 2's scope — required for
test_concurrent_writes_do_not_lose_updates, part of Task 1's own
required-green test suite, to pass; documented in SUMMARY).
- Convert save_nodes_inner to an atomic write: serialize to a sibling
nodes.json.tmp in the same directory, then tokio::fs::rename onto the
real path, so a crash mid-write never leaves a partially-written
nodes.json for a concurrent reader.
- Add 3 new regression tests, two of which are fail-first proven: heavy
tokio::spawn-based concurrency (not just tokio::join!, since
remove_node's extra tombstone I/O hop structurally biased a simple
2-task race toward the safe ordering) reliably reproduced both the lost
concurrent write and the removed-node-reappears bug pre-fix; both are
green post-fix along with the existing suite (13/13).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Framework-pt report: a paid invoice stalled the UI with no success
shown, and lightning/total balances disappeared until it settled.
Three compounding causes, three fixes:
- Backend payinvoice's synchronous wait drops 120s → 8s. Fast payments
(the majority) still settle in one round trip; slow multi-hop routes
return pending + payment_hash quickly and the caller's 3s poll takes
over — instead of the modal freezing for up to two minutes.
- payLightningInvoice gains an onPending hook: SendBitcoinModal and the
scan modal now flip to a visible "Settling…" success pane the moment
the payment goes pending (safe to close), and the ongoing poll
upgrades it to Paid — or replaces it with LND's real failure.
- One slow lnd.getinfo poll (5s budget) flipped the Home wallet card to
"disconnected", hiding balances the user already knew. Three
consecutive failures are now required (~30s) before the card gives up;
last-known balances keep rendering throughout.
rpc-client tests 75/75.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Another update operation is already running" surfaced as a scary
failure while the update was in fact applying fine (OptiPlex, v1.7.118
rollout). The apply path now joins the in-flight install — same
overlay, same wait-for-new-version polling — and a concurrent download
attempt shows a calm in-progress note (EN+ES strings added). The
backend's tarball extractions run under ionice -c3 nice -n10 so a
200MB update can't starve podman/status calls into multi-minute
timeouts on small disks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v1.7.117 broke Reticulum mesh on OTA-only fleet nodes two ways: the
update swaps only the backend binary and frontend tarball, so nodes
kept a stale archy-reticulum-daemon whose argparse exits on the new
--enable-transport flag (mesh session died on every spawn — confirmed
on framework-pt), and they never had archy-rnodeconf at all, so the
in-app Flash LoRa flow failed with a bare "No such file or directory".
Four-part fix:
- The Rust supervisor probes `daemon --help` and only passes
--enable-transport when the daemon advertises it; unsupported daemons
run edge-only exactly as pre-1.7.117 (tested against stub daemons
both ways + missing-binary fail-safe).
- Both PyInstaller tools ride the frontend tarball's runtime payload
(radio-tools/) and bootstrap.rs promotes them to /usr/local/bin on
startup when bytes differ — the first OTA path that ever updates
them. create-release.sh now rebuilds them every release and the
manifest script hard-fails if they're missing.
- The ISO bundles archy-rnodeconf alongside the daemon (it never did).
- Flash LoRa reports "tool not installed — update the node" instead of
the bare spawn error when rnodeconf is absent everywhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Reticulum daemon gains --enable-transport, which writes
enable_transport = yes into the RNS config it regenerates on every
start, and the Rust supervisor always passes it. Archy nodes now relay
RNS traffic and rebroadcast announces, so archy nodes (and Sideband/
NomadNet peers) beyond direct RF range discover and reach each other
through any archy node in between — edge-only operation left every
node limited to its own radio horizon. RNS's per-interface airtime
caps bound the extra announce overhead on LoRa.
Verified: config generation with the flag on/off, daemon --selftest
green with transport enabled, mesh test module 116/116.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A published port no longer implies a web UI. The package scanner used to
synthesize interfaces.main.ui="true" for any container with a port or
onion address, so headless backends — including self-deployed compose
stacks like podsteadr — showed up as launchable apps. New ui_detection
module decides instead: a manifest interfaces declaration (catalog
overlay first, disk second) is definitive; undeclared apps get a short
HTTP probe of the launch port (HTML page, redirect, or browser auth
wall = UI; JSON APIs, raw TCP, dead ports = service), with cached
verdicts and probes gated on running containers. Frontend canLaunch
now refuses curated services outright and only treats a bare runtime
address as launchable for curated known apps.
Works identically for manifest apps and containers deployed by hand
outside the orchestrator. ui_detection tests 6/6, frontend suite
696/696.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-merge regression from combining two individually-correct changes:
main's 2026-07-23 fix makes open_preferred_path bail WITHOUT touching the
port when no device_kind is pinned, while the hw-config branch's skip_path
dedup excludes the preferred path from the auto-detect fallback on the
assumption it was already probed this cycle. Together, on a single-radio
node with no pin (the common fleet state), the only candidate was never
probed at all and the mesh never came up — hit live on archi-dev-box
right after deploying merged main.
Fix: when device_kind is None, skip open_preferred_path entirely and go
straight to auto_detect_and_open with skip_path=None. The pinned path
keeps the existing probe-then-skip fallback.
Verified live on archi-dev-box: radio auto-detected, Reticulum daemon
ready, 5 persisted peers loaded. Mesh tests 116 passed / 0 failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Image quality modal: 'Send via' pills (LoRa / FIPS / Tor) when the
peer is federation-reachable — mesh.transport-advice now returns
has_fips + last_transport alongside has_tor. Picking FIPS/Tor routes
the image over the content-ref path instead of the radio.
- Attachment modals (transport chooser, image quality, new hop modal)
Teleport to body so the backdrop dims the FULL viewport — rendered
in-place they sat inside a transformed glass panel that trapped
position:fixed to the right chat panel.
- Click a message's transport pill → route modal: radio hops + live
SNR/RSSI quality for LoRa transports, overlay/circuit shape for
FIPS/Tor, delivery + E2E state.
- Reactions move behind a compact 'React ▾' dropdown with a larger
12-emoji palette.
- Unread badges now clear like a normal chat app: opening a contact
clears ALL twins of the merged conversation (badge sums every
contact_id — clearing just the clicked one left it stuck), and only
once the chat has scrolled to the latest messages; scrolled up into
history, new arrivals accumulate until you scroll back down.
- Refresh button shows only the spinner while refreshing (text+spinner
overflowed the fixed button width).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
E2E-verified both directions dev-box<->x250 over real RF (5KB image ~60s):
1. Sender daemon never called link.identify() — receiver's
get_remote_identity() was None, so every arrived transfer carried an
empty source_hash and the Rust side dropped it (now also warns
instead of silently vanishing it).
2. Receiver treated resource.data as bytes, but RNS hands a concluded
Resource's data as a file-like BufferedReader — b64encode raised
TypeError and the transfer was lost even when attributed.
3. Radio twins of merged contacts carry the peer's Archipelago ed25519
key as pubkey_hex, not an RNS hash — prefix lookup could never match
('Unknown Reticulum prefix', observed live). resolve_dest_hash now
falls back to matching the announce-bound arch_pubkey_hex.
4. The daemon RPC socket kept asyncio's default 64KiB line limit; any
attachment >~48KB overflowed it and tore down the whole daemon
connection ('reticulum-daemon is gone'). Raised to 16MiB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both sides independently fixed the serial-alias dedup and the ESP32
boot-reset races; kept the branch's defer-to-auto-detect for unpinned
preferred paths (single probe pass per cycle) on top of main's
advert-name threading, Reticulum name propagation and radio-first
routing. Modal keeps main's 'Set Recommended' naming + probe progress
bar alongside the branch's in-app firmware flasher step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two halves of the same twin-resolution gap, found live while testing
images between archi-dev-box and archy-x250-dev:
- peer_dest_prefix resolved the given contact row's own pubkey. For the
UI's merged conversation (the federation-synthetic id) that's the
Archipelago ed25519 identity key, NOT a radio routing key — so every
Reticulum resource send (images/files over LoRa) failed with 'Unknown
Reticulum prefix' while the UI showed the message as sent. It now
resolves through the radio twin (same arch identity, radio-range id).
- send_typed_wire sent EVERY federation-synthetic contact over the
federation path (FIPS→Tor), even with the same node one LoRa hop away.
Policy per operator: LoRa first when the payload fits and the radio
twin is reachable, then FIPS, then Tor. Verified live: text to the
merged contact now logs 'Radio-first routing' and lands with
transport=reticulum on the peer.
Also restyles the mesh-chat attachment download controls: the pre-fetch
button was a bare .btn that squished to text width in the narrow mobile
bubble; now a full-width glass pill with a download icon and fetch
spinner, and the on-image overlay swaps the emoji glyph for a crisp SVG
in a properly-sized glass circle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root causes found and fixed after live debugging on archi-dev-box (all
verified on real RNode hardware, archi-dev-box <-> archy-x250-dev E2E):
- probe_rnode raced the board's own boot: opening the port pulses DTR/RTS
through USB-UART bridges (CP2102/Heltec V3), the ESP32 power-cycles and
spends ~2.5-3s in boot ROM, and the KISS DETECT written 300ms after open
landed in the void — so an RNode could NEVER connect on these boards.
Now: immediate probe (fast path), then drain-until-quiet boot settle and
a second DETECT with a fresh response window.
- MeshService::configure() only restarted the listener on enable/disable —
device_kind/device_path/advert_name/RF-param changes were silent no-ops
until a full process restart (the setup modal's apply/keep-as-is did
nothing). Material config changes now bounce the listener; the open
sequence races the shutdown signal so stop() no longer burns the full
15s timeout mid-probe; mesh.configure applies in the background instead
of stalling every status poll behind the service write-lock.
- The mesh name was write-only: config.advert_name had no reader,
server.set-name never reached the mesh service, and Reticulum's
set_advert_name was a no-op (daemon display name fixed at spawn, and the
ARCHY:2 announce blob REPLACED the LXMF display name — every archy node
was anonymous on RNS). Now: advert_name > server name precedence feeds
the session, renames restart it live, the daemon gets --display-name at
spawn plus a set_name RPC verb, and announces carry the LXMF-standard
msgpack name with the identity blob appended as an extra list element
stock clients (Sideband/NomadNet) ignore.
- Dead reticulum-daemon was invisible for up to 30min (RX-stall watchdog):
child exit / RPC-EOF now fails try_recv_frame so the session reconnects.
- Setup modal re-trigger loop: plugged_at used the tty node's mtime, which
bumps on every open — each probe invalidated the dismissal key. Use
btime/ctime (only change on real plugs).
- ARCHY:2 identity adverts (re-emitted every 60s over Reticulum) stomped
the federation twin's real name with a synthetic Archy-… placeholder and
nulled its position; blob-only announces no longer assert a name, blob
strings can never become display names, and stale blob names are healed
at peers.json load.
- mesh.broadcast on Meshtastic sent heartbeat+time only (no identity);
SendAdvert now also fires a want_response NodeInfo broadcast.
- New mesh.refresh RPC: actively re-queries the radio contact table (the
UI Refresh button previously only re-read server caches).
- Reticulum peers now track last_advert (announce time) and mark existing
peers reachable on inbound traffic.
- Boot auto-enable no longer force-enables mesh when an operator
explicitly disabled it (only fires when no config file exists).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HTTP accept loop parked on acquire_owned() when the connection
budget drained, freezing accept() for every client (the .228
session-flapping / CLOSE-WAIT signature). Permits drained because
half-open clients and hung upstreams held them indefinitely.
- try_acquire_owned + immediate 503-and-close when the budget is
exhausted; the accept loop itself never blocks
- 30s http1_header_read_timeout drops slowloris/half-open clients
- 900s watchdog bounds non-upgraded connections; websocket upgrades
are exempt (legitimately long-lived)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the .198 load spiral (2026-07-28): needs_repair() called
ensure_image_present() every 30s tick to render the expected unit, so
under IO pressure the image-existence check timed out, read as "image
missing", and a 900s podman build ran inside the PROBE while the
companion was up — each build pegging the disk that made probes fail.
- needs_repair() is now build/pull-free: unit file present → service
is-active (10s cap; a hung systemctl reads as "assume active", never
as dead) → unit matches one of the three image refs install_one could
have written → context-newer-than-image staleness only when the unit
uses the auto-built :latest.
- Per-companion 10-min repair cooldown after a failed install_one, so a
failing build retries at most every REPAIR_COOLDOWN instead of every
reconcile tick.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase A3 of docs/FIPS-UPTIME-AND-UI-STATE-PLAN.md (RC5), measured against
the A2 dial_stats baseline:
- 25s connectivity watcher in the fips supervisor: re-applies seed anchors
immediately on an anchor-link drop, on startup-disconnected, AND on
silent data-path death (connect_fails growing with zero fips_ok — the
live .198 failure where the daemon reported "connected" while every
dial blackholed and the 300s tick never healed it). Bounded to one
re-apply per 60s.
- anchors::apply is now concurrent with a 15s per-connect cap — the old
serial loop waited unbounded on each `sudo fipsctl connect`, so one
hung subprocess stalled the whole periodic tick.
- rebindable peer listener: the accept loop returns after persistent
accept errors (was: continue forever = inbound-dead until restart) and
peer_late_bind_loop rebinds — also on fips0 ULA change.
- is_service_active gets a 10s TTL cache (was up to 2 systemctl spawns
per dial attempt and per warm-tick peer).
- the warm tick now warms the union of federation peers + configured
seed anchors (direct anchor links used to go cold between 300s ticks),
skipping the redundant per-peer service check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase A2 of docs/FIPS-UPTIME-AND-UI-STATE-PLAN.md (RC6). Fallbacks to Tor
were debug!-only and uncounted, so "FIPS uptime" was unfalsifiable and
paths that were 100% Tor by construction went unnoticed for months.
- fips::telemetry: process-lifetime counters for FIPS successes and the
six fallback reasons (no_npub, service_inactive, dns_fail, connect_fail,
http_404, http_5xx), exposed as `dial_stats` in fips.status
- dial.rs: every fallback branch now counts + logs at info! with a
`reason` field (resolve/connect/status branches)
- PeerRequest::record_transport(data_dir): opt-in hook that writes the
transport actually used to federation storage off the hot path — wired
into the dial sites that never recorded (DWN sync ×3, mesh blob fetch,
federation deploy notify, onion-rotation notify, node messages via a
new send_to_peer data-dir param)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase A1 of docs/FIPS-UPTIME-AND-UI-STATE-PLAN.md — the five changes that
made FIPS fall back to Tor even when a FIPS path existed:
- RC0: the fips.d drop-in now opens PEER_PORT 5679 (was 80+8443 only, so
every hardened node firewalled peers' FIPS dials; 28k drops on .198)
- RC4: /blob/ and /dwn/ added to the peer-path allowlist — mesh file
sharing and DWN sync were 404 → 100% Tor by construction
- RC2-G2: lan_fips_anchors dials PUBLISHED_UDP_PORT (2121) instead of the
dead 8668, with a drift-guard test against the rendered daemon config
- RC2-G1: direct LAN peering actually runs now — mDNS TXT advertises the
FIPS npub, discovery calls set_fips_npub, and the anchor tick hydrates
npubs from federation storage for peers on older builds
- RC3: FIPS attempt budget is a hard cap (retry no longer doubles it) and
the 12 hot call sites get explicit fips_timeout fast-fail so Tor keeps
its full budget (browse-peer, preview, /blob, DWN, node-message,
rotation notifies)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Slow multi-hop payments (>15s routing) surfaced as "Payment failed"
while LND settled them in the background: the shared LND REST client's
15s total timeout aborted the synchronous /v1/channels/transactions
wait, and every UI path treated that abort as a definitive failure. The
payment then succeeded anyway and only appeared in history on the next
background poll.
Backend: lnd.payinvoice now decodes the invoice up front for its payment
hash, pays on a dedicated 120s client, and answers status:"pending" with
the hash (never an error) when the wait elapses after the payment was
handed to LND — only a pre-connect failure is still a hard error. New
lnd.paymentstatus RPC reports succeeded/failed/in_flight (with humanized
failure reasons) from /v1/payments.
Frontend: new rpcClient.payLightningInvoice() pays then polls
lnd.paymentstatus to a real terminal state (3s interval, up to 2 min);
all five call sites (send modal, scan modal, web5 unified send, peer-file
purchase, app-launcher payments) migrated. Failure is only shown when LND
itself declares FAILED; a still-settling payment shows an in-flight state
and success fires the transaction refresh.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bundles all post-115 fixes into an OTA release: boot READY-before-recovery
+ Restart=always (no more 'server starting up'), and the app-install
daemon-kill fix (v6 relay only bridges live app ports; port-cleanup
excludes our own PID).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The v6 app-port relay preemptively bound [::]:<port> for ALL catalog
ports, even apps not installed. Installing such an app (grafana:3000,
photoprism, uptime-kuma, jellyfin — framework-pt 2026-07-27) then hit
'address already in use' from the relay, which triggered
cleanup_stale_pasta_port -> -> killed archipelago
itself (it held the port) mid-install. The daemon crash-looped and the
half-created apps were rolled back and vanished.
Two fixes:
- relay only bridges a port that a running app already answers on over
IPv4 (probe 127.0.0.1:port first) — an uninstalled app's port is never
held, so its install sees a free port and never triggers the cleanup.
- cleanup_stale_pasta_port excludes our own PID from both the ss-based
kill and the fuser kill, so freeing a port can never terminate the
daemon even when the relay legitimately holds it (reinstall case).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of 'server starting up' forever / crash-on-install
(framework-pt, v1.7.114->115, 2026-07-26): on a node with many stacks,
the synchronous boot recovery (recover + start_stopped_containers) runs
BEFORE sd_notify(Ready), so the unit sits in 'activating' for minutes.
Anything touching the service in that window — a superseding
start/restart, an install-time reconcile churn — killed a half-started
instance; it exits 0 on SIGTERM and Restart=on-failure then never
restarts it. Node dead behind 'server starting up'.
Fixes:
- signal READY (+ start the watchdog keepalive) BEFORE boot recovery, so
the unit reaches 'active' in seconds; recovery/reconcile/listener
continue after. No more minutes-long activating window.
- Restart=always (was on-failure): a clean-exit SIGTERM must still bring
the daemon back. Manual is still honored.
- OTA restart via a PID1-owned transient timer (systemd-run --on-active=2)
instead of a tokio-sleep child of the process being stopped, whose
start-half was being lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rootless podman's wildcard publish claims [::] but BLACK-HOLES inbound
v6 (accepts, forwards nothing — vaultwarden 'empty response' from the
phone, 2026-07-26), while most apps got no v6 listener at all. Two
halves: quadlets now publish unbound ports on 0.0.0.0 explicitly
(frees the v6 side; the one-time drift/recreate at upgrade is the
deploy vehicle), and the daemon runs a self-selecting V6ONLY relay on
every catalog launch port forwarding raw TCP to the v4 loopback
listener. Rescans every 60s so new installs bridge without a restart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The companion opens catalog apps by direct port over the mesh; the
fips0 default-deny baseline blocked every one of them (apps 'stuck'
from the phone, 2026-07-26). Core now writes a second fips.d drop-in
from the generated catalog launch-port list on every install/upgrade.
Service/RPC ports (bitcoind 8332, LND 10009, Tor) stay closed.
generate-app-catalog.py emits the Rust port list alongside the TS one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fips::config::install() now writes /etc/fips/fips.d/80-web-ui.nft
(tcp 80/8443 accept) and reloads the baseline on every install/upgrade —
the hardening firewall default-denies inbound on fips0 and the UI was
unreachable over the mesh without it (root-caused live 2026-07-26).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mesh is IPv6-only; the main web listener bound 0.0.0.0 only, so a
phone reaching a node over its fips0 ULA got RST on :80
(ERR_CONNECTION_ABORTED) — UI over mesh was structurally impossible.
Confirmed 2026-07-26 on framework-pt: v4:80 = 200, v6:80 = refused.
Mirror an IPv4-any main listener with a V6ONLY [::] socket on the same
port; V6ONLY so it coexists with the v4 listener regardless of
net.ipv6.bindv6only.
Also: fips.yaml generator carries the fast-reconnect profile (validated
live on framework-pt since 2026-07-24; snapshot tests updated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Observed live on .116 after the dedup/backoff fixes: the kiosk browser's
hot-swap auto-probe (mesh.probe-device) still interleaved its handshakes
with the listener's open sequence on the same tty every backoff window —
Linux double-opens ttys silently, both handshakes corrupted each other,
and every collision's open() DTR/RTS-reset the board again. The retry
heuristic from 5f01ec31 narrowed but couldn't close the race.
A static PORT_OPEN_LOCK now covers the listener's whole device-open
sequence and each probe attempt, so exactly one prober touches the port
at a time. With this + the settle/backoff/dedup fixes, the .116 radio
completed its first MeshCore handshake in days (node-fa161601, 8
contacts) and the session has been stable since.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
