Merge main into archy-hwconfig — reconcile probe/dedup/name work
Both sides independently fixed the serial-alias dedup and the ESP32 boot-reset races; kept the branch's defer-to-auto-detect for unpinned preferred paths (single probe pass per cycle) on top of main's advert-name threading, Reticulum name propagation and radio-first routing. Modal keeps main's 'Set Recommended' naming + probe progress bar alongside the branch's in-app firmware flasher step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,300 @@
|
||||
# FIPS near-100% uptime + optimistic UI state — implementation plan
|
||||
|
||||
**Date:** 2026-07-27. **Status:** researched + root-caused live on the fleet; ready to
|
||||
implement for the next release. Two workstreams: (A) make node↔node FIPS transport
|
||||
succeed whenever a FIPS path physically exists, (B) stop the UI reloading everything
|
||||
on every navigation (optimistic/cached cards, stale-while-revalidate) while keeping
|
||||
data fresh.
|
||||
|
||||
**Honesty note on "100%":** if a node's network blackholes every anchor (the .116
|
||||
WiFi case, `docs/HANDOFF-2026-07-20-fips-peer-files.md:117-133`), Tor fallback is
|
||||
*correct*. The achievable target is: **FIPS wins whenever a FIPS path exists, and
|
||||
fallback frequency is measured in-product so regressions are visible.** Today several
|
||||
paths are 0% FIPS *by construction* regardless of network health — that's the bug.
|
||||
|
||||
---
|
||||
|
||||
## Part A — why Cloud/FIPS "commonly falls back to Tor": ranked root causes
|
||||
|
||||
All verified live on 2026-07-27 (.116 local, .198, .228, Framework PT, x250s) plus a
|
||||
full code audit of `core/archipelago/src/{fips,transport,federation,server.rs}`.
|
||||
|
||||
### RC0 — 🔥 The hardening firewall drops the peer-API port on every hardened node (PROVEN)
|
||||
|
||||
The fips0 default-deny baseline (`/etc/fips/fips.nft`) is opened by archipelago's
|
||||
drop-in `80-web-ui.nft` (`fips/config.rs:236-255`) for **80 + 8443 + app ports only**.
|
||||
The peer-API listener — which carries *all* federation sync, cloud browse/download,
|
||||
mesh envelopes, DWN, invoices — is **`PEER_PORT = 5679`** (`fips/dial.rs:35`).
|
||||
**5679 is not in the allowlist.** The drop-in's own comment claims "web UI + peer
|
||||
API" but the peer API port was never added.
|
||||
|
||||
Live proof (2026-07-27):
|
||||
- .116 nft chain: 5,965 dropped packets; .198: **28,670 dropped packets** — that's
|
||||
peers' FIPS dials dying at the firewall.
|
||||
- .198 → .116 `GET :5679/health`: **timeout (6s)** before; **HTTP 200 in 0.35s**
|
||||
after `nft insert rule inet fips inbound iifname fips0 tcp dport 5679 accept`.
|
||||
Same result in reverse direction (200 in 0.64s).
|
||||
- Explains the exact fleet split in `federation/nodes.json`: hardened-baseline nodes
|
||||
(Framework PT, .198, .228, x250-dev, x250-mad2) = `last_transport: tor`;
|
||||
non-hardened nodes (Austin Sapien, X250-Beta, X250-PA) answer :5679 (404 from the
|
||||
path allowlist = listener reachable) = `last_transport: fips`.
|
||||
- Every dial to a hardened peer pays the 8s FIPS connect timeout
|
||||
(`dial.rs:114`) ×2 (retry, `dial.rs:128-140`) → then Tor. That's the "Cloud takes
|
||||
forever / shows Tor" experience.
|
||||
|
||||
**Fix (one line + reload):** add `tcp dport 5679 accept` to the drop-in in
|
||||
`fips/config.rs` (use a constant shared with `dial.rs::PEER_PORT`, not a literal).
|
||||
The drop-in reinstalls on every daemon config install, so it heals fleet-wide on OTA.
|
||||
⚠️ Transient manual rules were inserted on .116 and .198 during diagnosis (2026-07-27)
|
||||
— they vanish on the next `nft -f /etc/fips/fips.nft` reload or reboot; the code fix
|
||||
makes them permanent.
|
||||
|
||||
### RC1 — .228 (Shorty's) runs fips 0.3.0-dev; the 0.4.1 fleet can't reach it
|
||||
|
||||
.228's daemon: `0.3.0-dev (rev 34e00b9f6e)`, both anchor links "connected", but its
|
||||
ULA is 100% unreachable from 0.4.1 nodes (ping loss 100%). FIPS wire format is not
|
||||
stable across revs (`docs/HANDOFF-2026-07-23-companion-apk-deploy.md:78`). Everything
|
||||
to/from .228 rides Tor no matter what else we fix.
|
||||
|
||||
**Fix:** fleet fips-version audit + upgrade to v0.4.1 everywhere (in-product updater
|
||||
exists: `fips/update.rs`; .deb path per `reference_vps2_fips_anchor`). Add a version
|
||||
check to `fips.status` and surface a "peer daemon outdated" warning.
|
||||
|
||||
### RC2 — Direct LAN/endpoint peering is dead code + wrong port + stale seed anchors
|
||||
|
||||
Without direct links, all peer traffic hairpins through the vps2 anchor spanning
|
||||
tree (observed: .116→.198 cold RTT 1.5–3.5s on the same LAN; also the wedged-anchor
|
||||
latency-rot incident, `HANDOFF-2026-07-23:141-160`).
|
||||
|
||||
- **G1 — `lan_fips_anchors()` has never run.** It needs `PeerRecord.fips_npub`, but
|
||||
`PeerRegistry::set_fips_npub` (`transport/mod.rs:302`) has **zero callers** — mDNS
|
||||
TXT records only carry `did`/`pubkey`/`version` (`transport/lan.rs:50-54`). So the
|
||||
"co-located peers form a direct link" feature (`anchors.rs:294-305`,
|
||||
`server.rs:761-766`) is a fleet-wide no-op.
|
||||
- **G2 — wrong UDP port.** `anchors.rs:293` dials `8668`, but the generated
|
||||
fips.yaml binds UDP **2121** (`fips/config.rs:187`, `fips/mod.rs:130`). Even if G1
|
||||
ran, it would dial a dead port. `.116`'s live `seed-anchors.json` still carries
|
||||
`.198@192.168.1.198:8668` — **stale IP (LAN renumbered to 192.168.63.x) AND dead
|
||||
port**; both manual entries are useless today.
|
||||
- No Tailscale/alternate endpoint fallback when LAN is unreachable (the .116↔.198
|
||||
fix of 2026-07-20 was hand-applied per-node config, never productized).
|
||||
|
||||
**Fix:** (a) `FIPS_UDP_PORT` → `crate::fips::PUBLISHED_UDP_PORT` + drift-guard test;
|
||||
(b) hydrate `fips_npub` into the registry from federation storage (did-keyed join) so
|
||||
`lan_fips_anchors` goes live with no wire change; (c) advertise the npub in the mDNS
|
||||
TXT + `set_fips_npub` on resolve as the proper fix; (d) teach the LAN-anchor tick to
|
||||
also try a peer's Tailscale/last-known-good endpoint when LAN fails (reviewed change
|
||||
— this area got handoffs wrong twice, per memory).
|
||||
|
||||
### RC3 — No fast-fail on the hottest call sites; retry silently doubles every budget
|
||||
|
||||
- `content.browse-peer` — **the Cloud page** — has NO `fips_timeout`
|
||||
(`api/rpc/content.rs:363-366`): a cold FIPS path burns up to ~16.6s (8s connect +
|
||||
600ms + 8s retry) before Tor even starts, against a UI deadline of 30s
|
||||
(`Cloud.vue:720`) — and the frontend then retries ×3. Users see errors, not
|
||||
fallback. 12 call sites total lack `fips_timeout` (browse/download/preview-peer,
|
||||
`/blob`, DWN, node_message, rotation notifies).
|
||||
- `dial.rs:128-140` runs 2 full-budget attempts, so `fips_timeout(6s)` really means
|
||||
~12.6s everywhere.
|
||||
|
||||
**Fix:** wrap `send_with_retry` in a single `tokio::time::timeout(fips_attempt_timeout())`
|
||||
(call sites `dial.rs:455`, `dial.rs:488`; halve per-attempt client timeout), then add
|
||||
`.fips_timeout(...)`: `content.rs:366` (6s), `content.rs:281` (8s), `content.rs:1139`
|
||||
(6s), `typed_messages.rs:822` (8s), `dwn_sync.rs:188/213/272` (6s),
|
||||
`node_message.rs:376` (8s), `node_message.rs:412` (4s), `tor/mod.rs:501` (6s),
|
||||
`federation/handlers.rs:869` (6s). **Skip the three 900s streaming downloads**
|
||||
(`content.rs:552/870/1061`, `proxy.rs:236`) — `dial.rs:311-319` documents why; the
|
||||
retry-budget wrap covers their connect phase.
|
||||
|
||||
### RC4 — Two features are 100% Tor by construction (allowlist 404)
|
||||
|
||||
The peer listener path allowlist (`server.rs:1219-1239`) omits `/blob/<cid>` (mesh
|
||||
file sharing, `typed_messages.rs:813-822`) and `/dwn/health` (step 1 of DWN sync,
|
||||
`dwn_sync.rs:186`) → deterministic 404 over FIPS (`dial.rs:44-46` treats 404 as
|
||||
fall-back) → deterministic Tor, after paying the full FIPS cost. Both endpoints are
|
||||
already cryptographically gated, so they meet the allowlist's stated criterion.
|
||||
|
||||
**Fix:** add `|| path.starts_with("/blob/") || path.starts_with("/dwn/")`; extend the
|
||||
existing test block at `server.rs:1935-1945` (assert `/blob/abc` + `/dwn/health`
|
||||
allowed, `/blobber` + `/dwnx` denied).
|
||||
|
||||
### RC5 — Inbound listener can't heal; anchor flap = 5-minute Tor window; probe overhead
|
||||
|
||||
- `peer_late_bind_loop` returns after first successful bind (`server.rs:1203`) and
|
||||
`accept_loop` `continue`s on errors forever (`server.rs:1249-1258`): a fips0
|
||||
teardown/re-key leaves the node inbound-dead until process restart → **every peer**
|
||||
falls back to Tor against it.
|
||||
- Nothing reacts to anchor-link drops: anchors re-apply only on the 300s tick
|
||||
(`server.rs:731`); worst-case 5min Tor-only after a flap (the historic "link dead
|
||||
timeout 30s" flapping made this chronic).
|
||||
- `is_service_active()` spawns up to 2 `systemctl` per FIPS attempt *and* per peer
|
||||
per 25s warm tick (`dial.rs:284-294`); `warm_path` skips peers without
|
||||
`fips_npub` in federation storage (`fips/mod.rs:88-95`); `anchors::apply` is
|
||||
serial with unbounded subprocess waits (`anchors.rs:234-283`).
|
||||
|
||||
**Fix:** rebindable listener; a ~25s connectivity watcher (reuse
|
||||
`service::peer_connectivity_summary`, `fips/service.rs:178-207`) that re-applies
|
||||
anchors immediately on a connected→disconnected edge with bounded backoff; 10s TTL
|
||||
cache for `is_service_active` (mirror `transport/fips.rs:24-107`); warm the union of
|
||||
federation+registry peers; make `apply()` concurrent with per-connect timeouts.
|
||||
|
||||
### RC6 — Zero observability: fallbacks are invisible, so "uptime" is unfalsifiable
|
||||
|
||||
Fallbacks log at `debug!` only (`dial.rs:458,491`); no counters; `last_transport` is
|
||||
written by only 7 of ~20 call sites and **never read** to influence anything
|
||||
(`storage.rs:120-147`). The parallel `TransportRouter` system can't even see FIPS
|
||||
(`FipsTransport` is never constructed — `server.rs:422-442` registers Tor/Mesh/LAN
|
||||
only).
|
||||
|
||||
**Fix:** per-reason fallback counters (F1 no-npub / F2 service-inactive / F3
|
||||
DNS-fail / F4 connect-fail / F5 404 / F6 5xx) surfaced in `fips.status` + `info!`
|
||||
logs with a `reason` field; call `record_peer_transport` from all peer-dial sites;
|
||||
UI: per-peer transport badge on Cloud (the response already carries `transport` —
|
||||
`content.rs:392-400` — Cloud.vue currently throws it away at `:716-721`).
|
||||
|
||||
---
|
||||
|
||||
## Part A — execution phases
|
||||
|
||||
### Phase A0 — fleet triage (no release needed; do first, validates everything)
|
||||
1. Fleet audit: `fipsctl --version` + `nft list table inet fips` + `ss -tlnp | grep 5679`
|
||||
on every node (roster: `reference_test_deploy_roster`).
|
||||
2. Transient `nft insert rule inet fips inbound iifname fips0 tcp dport 5679 accept`
|
||||
on hardened nodes (already done on .116 + .198, 2026-07-27) — instant fleet-wide
|
||||
FIPS recovery while the code fix rides the OTA.
|
||||
3. Upgrade .228 (and any other 0.3.x) fips daemon to v0.4.1.
|
||||
4. Regenerate/clean stale `seed-anchors.json` on .116 (dead 192.168.1.x + :8668 entries).
|
||||
5. Baseline measurement: for each node pair, `content.browse-peer` time + transport.
|
||||
|
||||
### Phase A1 — P0 code (one commit, mechanical, offline-testable)
|
||||
1. **nft drop-in: open 5679** — `fips/config.rs` (share the constant with
|
||||
`dial.rs::PEER_PORT`). ← RC0
|
||||
2. **Allowlist `/blob/`, `/dwn/`** — `server.rs:1219-1239` + tests. ← RC4
|
||||
3. **`FIPS_UDP_PORT` = `PUBLISHED_UDP_PORT` (2121)** — `anchors.rs:293` + drift-guard
|
||||
test against `render_config_yaml()`. ← RC2-G2
|
||||
4. **Un-deaden `lan_fips_anchors`** — hydrate `fips_npub` from federation storage in
|
||||
`server.rs:761-766`; then mDNS TXT `fips` key + `set_fips_npub`
|
||||
(`transport/lan.rs:50-54`, `lan.rs:96-108`, `LanTransport::new` 4th arg via
|
||||
`crate::identity::fips_npub(&data_dir.join("identity"))`). ← RC2-G1
|
||||
5. **Retry-budget wrap + `fips_timeout` on 12 call sites** (list in RC3). ← RC3
|
||||
Verify: `cd core && cargo test -p archipelago` — watch `test_rendered_yaml_exact_snapshot`
|
||||
(`config.rs:419`) + `test_render_is_deterministic` (`config.rs:476`); item 3 must
|
||||
not change rendered output.
|
||||
|
||||
### Phase A2 — telemetry BEFORE tuning (second commit)
|
||||
6. Fallback counters by reason + `fips.status` exposure + `info!` reason logs;
|
||||
`record_peer_transport` from all sites. ← RC6 (gives the baseline that makes A3
|
||||
measurable and "100%" falsifiable)
|
||||
|
||||
### Phase A3 — resilience (third commit, measured against A2 baseline)
|
||||
7. `is_service_active` 10s TTL cache; warm-path union + `warm_path_unchecked`.
|
||||
8. Link-state watcher → immediate anchor re-apply on drop (replaces waiting for the
|
||||
300s tick); concurrent `apply()` with subprocess timeouts.
|
||||
9. Rebindable peer listener (`server.rs:1203`, `1249-1258`).
|
||||
10. (Reviewed, separate PR) endpoint-fallback for direct peering: LAN → Tailscale →
|
||||
last-known-good, npub-keyed. Mesh-routing area — needs careful review per memory.
|
||||
|
||||
### Phase A4 — verification gate (on nodes, before tag)
|
||||
- On .116/.198/framework-pt/.228: `content.browse-peer` to every peer must return
|
||||
`transport: "fips"` with sub-second latency (LAN pairs) / <3s (WAN), 20/20 calls.
|
||||
- Kill the fips daemon on one node → calls fall back to Tor gracefully within the
|
||||
fast-fail budget (<8s), UI shows partial results, no errors.
|
||||
- Restart daemon → FIPS recovers within one watcher tick (~25s), verified in
|
||||
`fips.status` counters.
|
||||
- Flap the anchor link (drop vps2 route) → direct LAN pairs keep FIPS via their
|
||||
direct link (G1 fix proof).
|
||||
- Add these as `tests/multinode/` cases per `docs/multinode-testing-plan.md`; also
|
||||
fix the known `node_rpc()` missing `--max-time` (tracker item).
|
||||
|
||||
---
|
||||
|
||||
## Part B — optimistic loading + state management (frontend)
|
||||
|
||||
Full audit: Pinia exists but pages fetch-on-mount with `loading=true` spinners;
|
||||
`Dashboard.vue:89` keys the router-view by `route.path`, so **every navigation
|
||||
unmounts and refetches everything**; no KeepAlive/onActivated anywhere; no dedup,
|
||||
no abort, no SWR layer. Four hand-rolled cache implementations already exist and
|
||||
prove the pattern (`useFleetData.ts:198-231` sessionStorage hydrate;
|
||||
`homeStatus.ts` sticky-ready loadState; `Home.vue:591-621` wallet localStorage
|
||||
snapshot; `curatedApps.ts:21-77` TTL cache). `SkeletonCard.vue` exists, imported by
|
||||
zero files.
|
||||
|
||||
### B1 — one shared primitive: `useCachedResource` composable + `resources` Pinia store
|
||||
Semantics (generalize `homeStatus.ts` + `useFleetData.ts`):
|
||||
- Keyed resource: `{ data, loadState: idle|loading|ready|error|refreshing, fetchedAt, error }`.
|
||||
- **Hydrate synchronously** from memory (Pinia, survives navigation) → sessionStorage
|
||||
snapshot (survives reload) → then revalidate in background.
|
||||
- Sticky-ready: once `ready`, never regress to `loading`
|
||||
(`loadState = loadState==='ready' ? 'ready' : 'loading'` — the `homeStatus.ts:80` idiom);
|
||||
keep-last-known-value on error with a stale badge (age from `fetchedAt`).
|
||||
- TTL per resource; `revalidateOnFocus` + on WS push (debounced, the
|
||||
`Home.vue:539-542` pattern); explicit `invalidate(key)` for mutations.
|
||||
- Optimistic mutation helper: apply → RPC → rollback on error (generalize
|
||||
`TransportPrefsCard.vue:112-127`).
|
||||
|
||||
### B2 — rpc-client upgrades (`src/api/rpc-client.ts`)
|
||||
- `AbortSignal` in `RPCOptions` (today the AbortController at `:87` is timeout-only)
|
||||
→ abort-on-unmount for fan-outs.
|
||||
- In-flight dedup keyed `method+JSON(params)` — collapses duplicate concurrent calls.
|
||||
- Per-call `maxRetries` override; set `maxRetries: 1` for `content.browse-peer` /
|
||||
`preview-peer` (retry×3 on a 30s timeout is why one slow peer = 90s spinner).
|
||||
|
||||
### B3 — Cloud page conversion (worst offender, the marquee win)
|
||||
- Move `sectionCounts`, `peerNodes`, `myFiles`, `peerFiles`, `paidItems` out of
|
||||
`Cloud.vue` component state (`:403,:476,:582,:689,:427`) into the cached store —
|
||||
instant render on revisit, background refresh.
|
||||
- **Incremental per-peer fan-in**: render each peer's card as its
|
||||
`content.browse-peer` resolves (today `Promise.allSettled` at `:708-747` blocks on
|
||||
the slowest peer). Per-peer states: cached/fresh/loading/unreachable.
|
||||
- **Surface `transport` per peer** (already in the response, discarded at `:716-721`):
|
||||
FIPS/Tor badge + latency — this is also the fleet-wide FIPS-uptime dashboard the
|
||||
user asked for, for free.
|
||||
- Skeleton cards (revive `SkeletonCard.vue`, copy `FileGrid.vue:3-19` shimmer) instead
|
||||
of spinners for counts/folders/peer grids.
|
||||
- Stop `CloudFolder.vue:307-319` calling `cloudStore.reset()` on every folder entry —
|
||||
cache per-path listings, navigate renders cache + revalidates.
|
||||
- `PeerFiles.vue`: persist catalog + preview cache in the store; cap the
|
||||
`preview-peer` fan-out (`:832-841`, currently unbounded) with a small concurrency
|
||||
queue + abort-on-unmount.
|
||||
|
||||
### B4 — roll out to remaining offenders (in audit order)
|
||||
PeerFiles → Web5 wallet/ecash/LND slices → Monitoring → Lightning channels
|
||||
(`LightningChannelsPanel.vue:650`) → Federation (already has `{showLoader:false}` —
|
||||
just adopt the store) → Server → Credentials/OpenWrtGateway/ContainerApps.
|
||||
`Apps.vue`/`Marketplace.vue`/`Fleet.vue` are already good; don't touch.
|
||||
|
||||
### B5 — freshness via the existing push channel
|
||||
`/ws/db` firehose + `sync.ts` JSON-patch already exist. Wire `useCachedResource`
|
||||
revalidation to relevant WS pushes (debounced 800ms), keep the 30s staleness
|
||||
reconciliation as backstop. No new backend needed for v1; a per-topic subscribe can
|
||||
come later.
|
||||
|
||||
### Part B verification (on nodes)
|
||||
- Navigate Cloud → Apps → Cloud: peer files render instantly from cache (0 spinner),
|
||||
refresh indicator while revalidating, updated data lands without layout jump.
|
||||
- One unreachable peer: its card shows stale/unreachable state; other peers render
|
||||
immediately (no 30s all-or-nothing).
|
||||
- Kill backend mid-view: stale data stays visible with age badge; recovery
|
||||
revalidates automatically.
|
||||
- Hard reload: sessionStorage hydrate paints before first RPC completes.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing for the next release
|
||||
|
||||
1. **A0 now** (fleet triage + transient nft rules + .228 daemon upgrade + baseline).
|
||||
2. **A1 + A2** land together (P0 fixes + telemetry) → deploy to .116/.198 →
|
||||
Phase A4 checks on the pair → framework-pt → full fleet.
|
||||
3. **B1 + B2 + B3** (composable + rpc-client + Cloud) in parallel with A-testing —
|
||||
frontend-only, verifiable against .116 dev (`reference_neode_ui_dev_testing`).
|
||||
4. **A3** after telemetry baseline exists; **B4/B5** ride the same or next OTA.
|
||||
5. Gate: Phase A4 checklist green + Part B verification on-device + existing
|
||||
single-node gate stays green → tag/OTA per ship ritual.
|
||||
|
||||
## Success criteria
|
||||
- `content.browse-peer` transport = fips for ≥99% of calls between healthy 0.4.1
|
||||
nodes over 24h (measured by the new counters), Tor reserved for genuinely
|
||||
FIPS-unreachable peers (.116-WiFi-class networks).
|
||||
- Cloud revisit paints in <100ms from cache; fresh data within one revalidate.
|
||||
- Fallback counters visible in `fips.status` so regressions are caught on the
|
||||
dashboard, not by users.
|
||||
@@ -0,0 +1,108 @@
|
||||
# License Compliance Audit — Open-Source Release
|
||||
|
||||
Audit date: 2026-07-22. Scope: entire repo (core Rust workspace, neode-ui, apps/*, Android companion, image-recipe ISO, docker/, app-catalog, reticulum-daemon, demo/) plus the external FIPS source and registry-mirrored images.
|
||||
|
||||
**Verdict:** the dependency graph is almost entirely permissive (MIT/Apache/BSD) and compatible with a free open-source release. But the repo is not releasable as-is: it has **no license of its own**, one **LGPL Rust dependency**, several **non-redistributable committed assets** (proprietary fonts, unknown-rights media), and **missing attribution machinery**. Everything below is ordered by severity.
|
||||
|
||||
---
|
||||
|
||||
## STATUS UPDATE — 2026-07-23
|
||||
|
||||
**DONE:**
|
||||
- MIT adopted. Root `LICENSE` + `NOTICE` added; `license = "MIT"` in all 5 workspace crates (archy-fips-core already had it); `"license": "MIT"` (+ `"private": true`) in all 4 package.json files.
|
||||
- Deleted: `Courier_New/`, `Benton_Sans/`, `Redacted/` fonts; `wireguard.apk`; `atob.s9pk`; obsolete `test-install.sh` (all git-rm'd; also removed from `web/dist`).
|
||||
- Media provenance resolved: all demo music/photos/posters, UI sfx, backgrounds, and intro video are the author's original work — recorded in `demo/content/README.md` and `NOTICE`.
|
||||
- Meshtastic device artwork attributed (`mesh-devices/ATTRIBUTION.md` + NOTICE); icon attribution added (`assets/icon/ATTRIBUTION.md`: game-icons.net CC BY 3.0, pixelarticons MIT).
|
||||
- Reticulum decision: include + disclose (NOTICE states the Reticulum License restrictions and that it applies only to the optional daemon).
|
||||
- indeedhub: deferred — partnership in place; license the submodule before/at public release.
|
||||
- License inventories generated: `core/THIRD-PARTY-LICENSES.md` (649 crates) and `neode-ui/THIRD-PARTY-LICENSES.md` (runtime deps + fonts + vendored).
|
||||
|
||||
**REMAINING (code changes, awaiting review — see sections below for detail):**
|
||||
1. Replace `zbase32` (LGPL-3.0+) with `z32` or original impl — §2.
|
||||
2. Swap `redis:7.4.8` → Valkey in `scripts/image-versions.sh` and deploys — §3.
|
||||
3. Delete dead StartOS-derived crates `core/{js-engine,container-init,models,helpers}` — §4.
|
||||
4. Attribution build integration: cargo-about in CI → ship full license texts in ISO; vite/rollup license plugin (or UI licenses page) for the web bundle; Android OSS-licenses screen — §5.
|
||||
5. Release-checklist items: per-release Debian source pointer (snapshot.debian.org), catalog `license`/`sourceUrl` fields, restrict ISO image bundling to the audited list — §6.
|
||||
6. Before repo goes public: purge deleted fonts/APKs from git history (`git filter-repo`), and verify game-icons author credit.
|
||||
|
||||
---
|
||||
|
||||
## 1. BLOCKER — the project has no license
|
||||
|
||||
There is no `LICENSE`/`COPYING` file anywhere in the repo. No crate in `core/` declares a `license` field; none of the four `package.json` files do either (and the three `apps/*` packages aren't even `private: true`). Until fixed, the code is "all rights reserved" — publicly visible, but legally not open source and not usable by anyone.
|
||||
|
||||
**Do:**
|
||||
- [ ] Choose a license. **Recommendation: MIT** — the Bitcoin-ecosystem norm (Bitcoin Core, LND are MIT), maximally compatible with everything found in the graph. (Alternatives: Apache-2.0 adds a patent grant; GPLv3 if copyleft is desired — nothing in the deps prevents any of these.)
|
||||
- [ ] Add `LICENSE` at repo root with the year and copyright holder.
|
||||
- [ ] Add `license = "MIT"` to all five workspace member `Cargo.toml`s (archipelago, container, openwrt, performance, security) and `Android/rust/archy-fips-core` (declares MIT but ships no license file — add one).
|
||||
- [ ] Add `"license": "MIT"` to `neode-ui/package.json` and `apps/{morphos-server,router,did-wallet}/package.json`.
|
||||
|
||||
## 2. BLOCKER — copyleft dependency that must be replaced
|
||||
|
||||
- [ ] **`zbase32 0.1.2` — LGPL-3.0+** — the only hard copyleft blocker in all 649 resolved Rust crates. Direct dep of `archipelago`, used in `core/archipelago/src/network/did_dht.rs` for did:dht z-base-32 encoding. LGPL statically linked into a Rust binary requires shipping relinkable objects/source — impractical. **Replace with the MIT `z32` crate** or a ~30-line original alphabet-substitution implementation.
|
||||
|
||||
No GPL, AGPL, SSPL, or unlicensed crates exist anywhere else in the Rust graph. (`r-efi` and `self_cell` list LGPL/GPL only as options in OR-expressions — elect MIT/Apache, no action.)
|
||||
|
||||
## 3. BLOCKER — committed files we may not redistribute
|
||||
|
||||
Remove from git (and **purge from history** before the repo goes public — they're in past commits):
|
||||
|
||||
- [ ] `neode-ui/public/assets/fonts/Courier_New/` — Monotype proprietary font, no license, **unused in CSS**. Delete.
|
||||
- [ ] `neode-ui/public/assets/fonts/Benton_Sans/BentonSans-Regular.otf` — commercial Font Bureau typeface, no license, unused. Delete.
|
||||
- [ ] `neode-ui/public/packages/wireguard.apk` (17 MB) — official WireGuard Android APK containing GPL-2.0 `libwg` components; redistribution triggers GPL source-offer. **Unreferenced since the FIPS migration** — delete.
|
||||
- [ ] `neode-ui/public/packages/atob.s9pk` (24 MB) — Start9 service package, unknown license, referenced only by a test script. Delete.
|
||||
- [ ] `demo/content/music/` (18 full tracks, ~150 MB) and `demo/peer-media/` (17 photos/book covers/film posters) — no recorded rights. If they're your own/AI-generated work, document that in a `demo/content/README`; otherwise remove.
|
||||
- [ ] `neode-ui/public/assets/video/video-intro.mp4`, `Kratter.MP3`, photographic `bg-*.jpg` backgrounds, UI/arcade sound effects in `assets/audio/` — same: document provenance (user-made per project convention) or replace. `welcome-noderunner.mp3` is ElevenLabs TTS — their commercial-use terms allow this on paid plans; note it.
|
||||
- [ ] **Registry: `redis:7.4.8`** (`scripts/image-versions.sh` `REDIS_IMAGE`) — Redis ≥ 7.4 is RSALv2/SSPLv1, **not open source**; re-hosting it on your registry is redistribution under a restricted license. **Switch to Valkey** (BSD-3, already mirrored) everywhere.
|
||||
|
||||
## 4. VERIFY — unknown/third-party provenance
|
||||
|
||||
- [ ] **`neode-ui/public/assets/img/mesh-devices/` (36 SVGs)** — almost certainly Meshtastic project device artwork (meshtastic/web is GPL-3.0). Confirm source; either replace with original art or comply with the upstream license + attribution.
|
||||
- [ ] **`neode-ui/public/assets/icon/`** — `barbarian.svg`, `batteries.svg` match game-icons.net (**CC BY 3.0 — visible attribution required**); pixel-style icons match pixelarticons (MIT). Confirm and add attribution, or replace.
|
||||
- [ ] `Redacted/redacted.regular.ttf` — upstream is SIL OFL 1.1 but no license file is shipped. Add `OFL.txt` or delete (unused).
|
||||
- [ ] **indeedhub** — submodule (private gitea) not checked out; no known license, yet `indeedhub{,-api,-ffmpeg}:1.0.0` images are distributed via registry/ISO. `indeedhub-ffmpeg` implies a bundled FFmpeg (LGPL/GPL → source-offer obligations). Must license the project and audit the ffmpeg build before public release.
|
||||
- [ ] `minmoto/fmcd` v0.8.0 and `ark-bitcoin/bark` (barkd) — binaries redistributed in your images; verify upstream licenses (bark claims Apache-2.0/MIT dual) and include their notices.
|
||||
- [ ] **Start9/StartOS heritage** — `core/{js-engine,container-init,models,helpers}` are StartOS-derived (embassy paths, s9pk handling). start-os is MIT → attribution required if kept. **Better: delete these four crates** — they are not workspace members, cannot compile (broken `../../patch-db` path dep), and carry an unpinned `yajrc = "*"` git dep on a moving branch. Deleting removes both the attribution question and dead code.
|
||||
- [ ] **Reticulum (RNS 1.3.5 + LXMF)** — verified: custom "Reticulum License" — MIT-style **plus field-of-use restrictions** (no systems designed to harm humans; no AI/ML training-dataset use). Redistribution is permitted, so shipping the PyInstaller `archy-reticulum-daemon` binary is fine **if** the license text is included with it — but the OS cannot claim to be 100 % OSI-open-source while bundling it. Options: include + disclose (recommended, matches "plan for decentralization" honesty), or make the daemon an optional download.
|
||||
|
||||
## 5. REQUIRED — attribution / notice machinery (currently absent)
|
||||
|
||||
Nearly every permissive license (MIT/BSD/ISC/Apache) requires reproducing copyright + license text **in distributed binaries** — and right now every distribution channel strips them:
|
||||
|
||||
- [ ] **Rust binaries** (649 crates, ~85 % MIT/Apache dual): generate `THIRD-PARTY-LICENSES` with `cargo-about` (or `cargo-license`) in CI; ship it in the ISO at e.g. `/usr/share/doc/archipelago/`. Include **ring's three license files** (LICENSE, LICENSE-BoringSSL, LICENSE-other-bits) and note the system OpenSSL (Apache-2.0) linked via `ssh2`.
|
||||
- [ ] **Web bundle**: Vite/esbuild strips all `@license` comments from `web/dist`. Add `rollup-plugin-license`/`vite-plugin-license` to emit a third-party attribution file, or add an "Open-source licenses" page in the UI. Runtime deps needing notices: vue/vue-router/pinia/vue-i18n (MIT), d3 (ISC), leaflet (BSD-2), dompurify (elect Apache-2.0 of its MPL/Apache dual), fuse.js (Apache-2.0), qrcode/qr-scanner/qrloop/buffer/fast-json-patch (MIT).
|
||||
- [ ] **Android APK**: `packaging.excludes` strips `META-INF` license texts and there is no licenses screen. Add an OSS-licenses screen or bundled `licenses.txt` covering AndroidX/Compose/OkHttp/ZXing (Apache-2.0), **fips © 2026 Johnathan Corgan (MIT — the core of the VPN feature)**, tokio/tracing (MIT), subtle (BSD-3), tun (WTFPL — permissive, just list it), secp256k1 family (CC0). Generate the Rust side from the committed `Cargo.lock` with cargo-about.
|
||||
- [ ] **AIUI demo bundle** (`demo/aiui/` — committed minified build): bundles Mermaid, Cytoscape, KaTeX, D3, Lodash, Workbox (all MIT/BSD). Add a `THIRD-PARTY-LICENSES` file next to it (or rebuild with a license plugin).
|
||||
- [ ] Keep the intact MIT headers in the two vendored `qrcode.js` copies (docker/lnd-ui, docker/electrs-ui) — already compliant, don't minify them.
|
||||
- [ ] Fonts kept: Montserrat (OFL.txt present ✓), Open Sans (Apache LICENSE.txt present ✓) — keep license files adjacent to the font files in dist.
|
||||
|
||||
## 6. REQUIRED — distribution-level obligations (ISO & registry)
|
||||
|
||||
The ISO redistributes a full Debian (trixie) system plus ~29 container image tarballs; the private registry re-hosts upstream images. Re-hosting = redistribution, same obligations as bundling.
|
||||
|
||||
- [ ] **GPL source offer for the ISO** — kernel, GRUB, busybox/live-boot, coreutils, nftables, cryptsetup, wireguard-tools, SYSLINUX `isohdpfx.bin`, etc. Easiest compliance: keep `/usr/share/doc/*/copyright` (the build already does ✓) **and** publish, per release, either a mirror of the exact Debian source packages (`apt-get source` snapshot / snapshot.debian.org pointer) or a written offer in the docs. Add this to the release checklist.
|
||||
- [ ] **AGPLv3 images redistributed** (mempool, Grafana, Vaultwarden, SearXNG, PhotoPrism, Nextcloud, Immich, CryptPad, MinIO): AGPL compliance = make corresponding source available. You ship a **modified** mempool-frontend (`docker/mempool-frontend` entrypoint patch) — the patch is in-repo, so compliance is met once the repo is public; state this in docs. For unmodified images, link upstream sources in the app catalog.
|
||||
- [ ] **GPLv2/GPLv3 images** (MariaDB, Jellyfin, AdGuard Home, strfry): unmodified redistribution → provide license text + upstream source links (a `license` + `sourceUrl` field per `app-catalog/catalog.json` entry solves this catalog-wide).
|
||||
- [ ] **Non-free firmware** (firmware-realtek/iwlwifi/misc/linux-nonfree, intel/amd microcode): redistributable but proprietary — disclose in docs ("includes non-free firmware for hardware support"), like Debian's own non-free-firmware ISOs do.
|
||||
- [ ] The ISO build's live-server image capture (`podman save` of whatever matches on the dev server) is a compliance hazard — bundle only from the audited image list.
|
||||
- [ ] FIPS daemon (jmcorgan/fips v0.4.1, MIT ✓) and nostr-rs-relay binary (MIT ✓): include their license texts in the notices bundle.
|
||||
|
||||
## 7. Housekeeping (supports compliance)
|
||||
|
||||
- [ ] Add lockfiles + pinned versions in `apps/*` (currently floating `^` ranges, violating the project's own pinning rule) — reproducibility is also what makes license audits stay true.
|
||||
- [ ] `Android` fips dep is pinned to a personal fork rev (`9qeklajc/fips-native@46494a74`) — mirror or vendor it so outside contributors can build.
|
||||
- [ ] Move `@types/dompurify` to devDeps; refresh stale `neode-ui/node_modules`.
|
||||
- [ ] Add a `NOTICE` file at root naming: fips (Johnathan Corgan, MIT), Start9 start-os (if any derived code remains), Kazuhiko Arase qrcode.js, font licenses, icon attributions.
|
||||
- [ ] Consider CI license gating: `cargo-deny` (Rust) + `license-checker` (npm) with an allowlist, so new copyleft deps are caught at PR time.
|
||||
|
||||
---
|
||||
|
||||
## Quick reference: what's already clean
|
||||
|
||||
- All 649 Rust crates except `zbase32`: permissive or dual-licensed.
|
||||
- All 833 npm packages in neode-ui: no GPL/AGPL anywhere; only dev-tool LGPL (sharp's libvips, never distributed).
|
||||
- Android Gradle deps: 100 % Apache-2.0, all pinned, no Play Services/telemetry.
|
||||
- FIPS mesh: MIT (© 2026 Johnathan Corgan) — keep notice.
|
||||
- js-engine binds deno_core (MIT) as a crate, nothing vendored — moot if dead crates are deleted.
|
||||
- reticulum-daemon Python is original code; obligations attach only to the PyInstaller binary (see §4).
|
||||
- Bitcoin Core/Knots, LND, BTCPay, Electrs, Fedimint, core-lightning, Gitea, Home Assistant, Tailscale, Portainer, Uptime-Kuma, filebrowser, ollama, penpot: MIT/Apache/BSD/Zlib/MPL — link + notice is enough.
|
||||
@@ -0,0 +1,288 @@
|
||||
# Open-Source Readiness Plan — Archipelago public launch
|
||||
|
||||
> Working plan, 2026-07-27. Source of truth for the pre-open-source cleanup.
|
||||
> A second agent is working the same goal concurrently — before executing any phase,
|
||||
> diff against `git log` since `7e8d3314` and skip/merge what's already done.
|
||||
> (Session plan file: `~/.claude/plans/resilient-moseying-reef.md`.)
|
||||
|
||||
## Context
|
||||
|
||||
The repo goes public in a few days, targeting bitcoin/bitcoin-level polish. Three deep
|
||||
exploration passes (docs/structure, code health, secrets sweep) found the repo is
|
||||
fundamentally strong — README, `apps/` manifest examples, ADRs, the bats lifecycle gate,
|
||||
1,104 Rust tests — but has hard blockers: **two live Anthropic API keys committed in
|
||||
tracked files**, node passwords in 7 tracked files, no LICENSE (README links a 404),
|
||||
5.5 GB `.git` (re-committed 27 MB APKs), ~290 hardcoded references to the private Gitea
|
||||
registry `146.59.87.168:3000` that make every app image unpullable for outsiders, and
|
||||
~28 internal AI-session/tracker docs mixed into `docs/`.
|
||||
|
||||
**Decisions made by the user:**
|
||||
1. **Fresh-history publish** — new public repo with a clean initial commit; private repo keeps full history.
|
||||
2. **Registry: domain + parameterize** — real domain in front of the existing registry; host configurable everywhere.
|
||||
3. **Deep code cleanup** — orphan crates, dead_code lifts, clippy trims, legacy fallback deletion (sequenced, cut-line-friendly).
|
||||
4. **Internal docs: sanitize and keep public** — scrub creds/IPs/hostnames but publish plans/trackers for transparency.
|
||||
|
||||
**Invariant throughout:** the single-node production gate (`tests/lifecycle/run-gate.sh`)
|
||||
is GREEN and must stay green. Re-run after any orchestrator/lifecycle change (Phase E
|
||||
especially). All cargo verification uses `--all-features` to match CI. Stage by explicit
|
||||
path, never `git add -A` (shared tree).
|
||||
|
||||
## Current local pass status
|
||||
|
||||
This branch is replayed on top of `origin/main` as `public-prelaunch`.
|
||||
|
||||
Completed locally in this pass:
|
||||
|
||||
- Redacted the two tracked Anthropic API key literals from
|
||||
`scripts/setup-aiui-server.sh` and
|
||||
`image-recipe/_archived/build-auto-installer-iso.sh`.
|
||||
- Removed `Android/app/debug.keystore` and `core/.env.production` from the
|
||||
source tree; copies were preserved in
|
||||
`~/Desktop/archipelago-sensitive-backup-2026-07-27/`.
|
||||
- Reworked `scripts/audit-secrets.sh` to scan tracked source more aggressively
|
||||
and to catch non-example env files and credential file patterns.
|
||||
- Reworked `scripts/validate-app-manifest.sh` so the current `app:` manifest
|
||||
schema can be audited without a Python `PyYAML` dependency.
|
||||
- Updated root/community docs, CI, PR template, app developer notes, and
|
||||
container/deployment docs toward public contributor expectations.
|
||||
- Fixed native FIPS activation fallback: nodes that have the packaged
|
||||
`fips.service` but not `archipelago-fips.service` now start the available
|
||||
unit instead of repeatedly failing activation against a missing unit. This
|
||||
now covers startup, supervisor self-heal, manual dashboard start/reconnect,
|
||||
and post-onboarding activation. The UI now labels the action as `Start`
|
||||
instead of making native FIPS look like an installable app.
|
||||
- Fixed the FIPS app-port relay design so it binds relays to the node's FIPS
|
||||
ULA instead of wildcard `[::]`, avoiding collisions with Podman-published app
|
||||
ports such as FileBrowser `8083` and Botfights `9100`.
|
||||
- Added `docs/nostr-git-source-hosting.md`, a NIP-34/ngit/GRASP source hosting
|
||||
plan using a Bitcoin Core-style maintainer model: public review and easy
|
||||
forks, with canonical merge rights held by a small signed maintainer set.
|
||||
|
||||
Verified locally:
|
||||
|
||||
- `./scripts/audit-secrets.sh` passes.
|
||||
- Full `apps/*/manifest.yml` repository audit passes with warnings only.
|
||||
- `bash -n` passes for the edited shell scripts.
|
||||
- Targeted FIPS dashboard vitest passes.
|
||||
- Targeted Rust tests for FIPS service unit detection and FIPS app relay
|
||||
address selection pass.
|
||||
|
||||
Verified on a Linux Archipelago verification node:
|
||||
|
||||
- Native FIPS was restored by starting the already-installed packaged
|
||||
`fips.service`; the daemon became active and joined the FIPS tree.
|
||||
- Correct local lifecycle API endpoint is HTTP, not HTTPS
|
||||
(`ARCHY_HOST=127.0.0.1 ARCHY_SCHEME=http`).
|
||||
- Read-only lifecycle run progressed past login and confirmed required
|
||||
containers, Bitcoin RPC, ElectrumX TCP, and manifest port-drift checks, but
|
||||
did not complete cleanly: `botfights` and `filebrowser` remained in
|
||||
`restarting` longer than the matrix window, and the LND `lncli getinfo`
|
||||
probe hung. Do not run the destructive gate until those live-node issues are
|
||||
understood.
|
||||
- After the node updated to `1.7.116-alpha`, `botfights`, `filebrowser`, and
|
||||
`lnd` were active/running and ports `8083`/`9100` were held by Podman's
|
||||
`rootlessport` as expected. The packaged `fips.service` remained installed
|
||||
and enabled but inactive, so the native FIPS service fallback should still
|
||||
ship before the public launch.
|
||||
|
||||
Still required before public publish:
|
||||
|
||||
- Rotate/revoke compromised credentials listed in Phase 0.
|
||||
- Finish Phase 1 password/node/token sanitization beyond the two API keys.
|
||||
- Publish from fresh history after the sanitized tree is final.
|
||||
- Run full Rust, frontend, Android, and lifecycle gate verification.
|
||||
- Resolve the live-node lifecycle blockers above, then rerun the read-only
|
||||
suite followed by the destructive gate only on an approved verification node.
|
||||
- Decide the canonical Archipelago maintainer npub and merge-maintainer npub
|
||||
list before publishing the Nostr Git source-hosting workflow.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Credential rotation (immediate, independent of the repo)
|
||||
|
||||
Treat all of these as already compromised; rotate even though we're doing fresh-history:
|
||||
|
||||
- **Anthropic API key #1**: `image-recipe/_archived/build-auto-installer-iso.sh:2837` (the "intentional alpha" ISO key). Revoke + reissue; move the live key OUT of source into a build-time secret/env (`ISO_ANTHROPIC_API_KEY`), keep the alpha-baking behavior if desired but never the literal in git.
|
||||
- **Anthropic API key #2**: `scripts/setup-aiui-server.sh:28` — a *different* live key, not covered by the documented alpha exception. Revoke; parameterize the script.
|
||||
- **The shared node SSH/sudo/UI password** (two variants) — in 7 tracked files + 24+ commits. Rotate fleet-wide (user task).
|
||||
- **Gitea `ai` account password + 2 Gitea tokens** — embedded in `.git/config` remote URLs (not tracked, but leaks in any directory copy/tarball). Rotate; switch remotes to credential-helper storage instead of URL-embedded creds.
|
||||
|
||||
## Phase 1 — Secrets & sanitization of tracked files
|
||||
|
||||
1. Strip the password/credential lines from the 7 files:
|
||||
`docs/PRODUCTION-MASTER-PLAN.md` (lines ~428–429, 454–457, 483, 521–528, 886 — the fleet cred table),
|
||||
`docs/archive/SESSION-1.8.0-OTA-PROGRESS.md`, `docs/archive/HANDOVER-2026-07-02-iso-feedback.md`,
|
||||
`docs/bitcoin-version-bulletproof-rollout.md`, `tests/production-quality/TRACKER.md`,
|
||||
`tests/multinode/meshtastic.sh:26`, `neode-ui/test-openwrt.mjs:4` (→ env var).
|
||||
2. `.gitea/workflows/post-install-tests.yml` — remove `sshpass -p '…'` + default target IP; use secrets/vars.
|
||||
3. Sanitize infra identifiers repo-wide (in the *sanitize-and-keep* docs and scripts):
|
||||
replace Tailscale IPs (17 unique, 14 files), LAN IPs (`192.168.1.x`, 93 files), hostnames
|
||||
(`tx1138`, `shorty-s`, `archy-x250`, `archy-dev-pa`) with placeholders like `<node-a>` /
|
||||
`NODE_IP`. Key script targets: `scripts/deploy-config-defaults.sh`, `scripts/deploy-tailscale.sh`,
|
||||
`docs/operations-runbook.md` (opens with real node IPs), `docs/developer-guide.md`, `docs/api-reference.md`, `docs/hotfix-process.md`.
|
||||
4. Fix the audit tool that let this happen: `scripts/audit-secrets.sh:28` — remove `\.md$` and
|
||||
bare `test` from ALLOW_PATTERNS; add `sk-ant-` and password-table patterns; scan all
|
||||
tracked files not just `*.env`. Run it clean as a Phase-1 exit check.
|
||||
5. `.gitignore` additions: `.claude/`, `*.key`, `*.pem`, `id_rsa*`, `*.sqlite`, `*.db`
|
||||
(`.claude/settings.local.json` with creds is currently only ignored by a machine-global rule).
|
||||
6. Product-security note to raise (not fix now): `password123` is a shipped default (auth.rs, en.json, user-walkthrough) — file a public issue for forced first-run password change if not already enforced.
|
||||
|
||||
## Phase 2 — Repo restructure: deletions, binaries, layout
|
||||
|
||||
Delete (each its own commit):
|
||||
- `loop/` (AI overnight harness w/ node SSH lines), `.agents/`, `.codex`, `.githooks/pre-push`
|
||||
(the hook that re-commits the 27 MB APK — root cause of the 5.5 GB history).
|
||||
- `indeedhub/` submodule + `.gitmodules` entry (points at private HTTP Gitea, breaks `--recursive`
|
||||
clones); `indeedhub-demo/` (single Dockerfile — merge or drop).
|
||||
- `RELEASE-NOTES-v1.0.0.md` (superseded by CHANGELOG), `neode-ui/docs/GAMEPAD-NAV-MAP.md` (duplicate of `docs/GAMEPAD-NAV.md`).
|
||||
- Stray generated HTML: `docs/container-architecture.html` (311 KB), `docs/archive/architecture-review.html`, `docs/archive/lora-functionality.html`.
|
||||
- `Android/local.properties` from tracking (local absolute path); remove `Android/app/debug.keystore` (standard practice).
|
||||
|
||||
Move out of git (→ release assets on the Releases page, referenced by URL):
|
||||
- `neode-ui/public/packages/archipelago-companion.apk` (27 MB), `wireguard.apk` (17 MB), `atob.s9pk` (23 MB).
|
||||
- `Android/archipelago-0.3.0-debug.apk.zip` (16 MB, stale).
|
||||
- `demo/content/music/*` + heavy `demo/aiui/assets` (~261 MB, third-party/unclear-licence media — MUST not ship publicly regardless of size).
|
||||
- `neode-ui/dev-dist/` (generated Workbox output) → gitignore.
|
||||
|
||||
Rename/fix the naming lie: `image-recipe/_archived/` contains the *production* ISO builder
|
||||
(`build-auto-installer-iso.sh`, referenced by `.gitea/workflows/build-iso.yml`). Move live
|
||||
files up into `image-recipe/`, delete the genuinely archived rest.
|
||||
|
||||
## Phase 3 — Registry domain + parameterization (functional blocker)
|
||||
|
||||
Infra (user assists: DNS + TLS):
|
||||
- Put a domain (e.g. `registry.archipelago-os.org` / `git.archipelago-os.org`) with HTTPS in
|
||||
front of the existing Gitea on vps2. OTA download URLs move from plain HTTP to HTTPS.
|
||||
|
||||
Repo changes:
|
||||
- Introduce a single source of truth for the registry host (e.g. `REGISTRY_HOST` in
|
||||
`scripts/lib/` + a default in the orchestrator config). Replace `146.59.87.168:3000` in:
|
||||
all 56 `apps/*/manifest.yml`, `app-catalog/catalog.json`, `releases/manifest.json`,
|
||||
`release-manifest.json`, the 11 scripts (`self-update.sh`, `create-release.sh`,
|
||||
`generate-app-catalog.sh`, `validate-app-manifest.sh`, `first-boot-containers.sh`, …),
|
||||
both `demo-images.yml` workflows, `demo-deploy/.env.example`, and the Android sources
|
||||
(`FipsPreferences.kt`, `PartyScreen.kt`).
|
||||
- Because the catalog is signed: regenerate + re-sign + republish the app catalog after the
|
||||
manifest host change (catalog-overlay supremacy — disk edits don't apply otherwise).
|
||||
Signing needs the user's mnemonic → schedule one ceremony after manifests are final.
|
||||
- Verify: fresh machine with no LAN/tailnet access can `podman pull` one app image via the
|
||||
domain and the gate node still installs apps after the re-signed catalog lands.
|
||||
|
||||
## Phase 4 — Documentation overhaul
|
||||
|
||||
### 4a. Community/legal files (missing today)
|
||||
- `LICENSE` — MIT (matches existing README badge). Add `[workspace.package] license` +
|
||||
`license.workspace = true` in the 5 member Cargo.tomls (also see Phase A4).
|
||||
- `SECURITY.md` — disclosure address, PGP key, supported-versions; cite the March 2026 audit (`docs/archive/security-code-audit-2026-03.md`).
|
||||
- `CODE_OF_CONDUCT.md` — Contributor Covenant (CONTRIBUTING.md already links to it, 404 today).
|
||||
- `CONTRIBUTING.md` edits: Gitea→GitHub fork flow, remove private deploy instructions, absorb
|
||||
the public-worthy CLAUDE.md invariants (rootless podman, manifest-driven, secrets model,
|
||||
non-destructive migrations), versioning policy note for the `-alpha` scheme.
|
||||
- `CLAUDE.md` — rewrite: keep invariants/build-verify (public-worthy), remove status banner,
|
||||
node numbers, `gitea-ai` push mechanics, MEMORY references (those move to private notes).
|
||||
|
||||
### 4b. New developer docs (the three real gaps for app developers)
|
||||
1. **`docs/quadlet-compilation.md`** — how a manifest becomes a Quadlet/systemd unit: naming,
|
||||
`systemctl --user` lifecycle, where units land, how to inspect/debug one. (Source:
|
||||
`core/archipelago/src/container/quadlet*.rs`, prod_orchestrator.)
|
||||
2. **`docs/container-lifecycle.md`** — the 30 s level-triggered reconciler, install/adopt/
|
||||
restart/uninstall state machine, health checks, crash recovery. (Replaces the plan-shaped
|
||||
`docs/bulletproof-containers.md` as the current description; salvage its content.)
|
||||
3. **`docs/secrets.md`** — `generated_secrets` declaration → materialisation by
|
||||
`container::secrets` (0600, rootless) → injection; what developers must never do.
|
||||
- Also: make every example in `docs/app-developer-guide.md` + `apps/*/manifest.yml` copy-paste
|
||||
work against the new public registry host; add an end-to-end "write your first app" walkthrough
|
||||
that a stranger can follow with only the public repo + an Archipelago node.
|
||||
|
||||
### 4c. Sanitize-and-keep internal docs (user's transparency choice)
|
||||
- Keep, after Phase-1 scrubbing: `docs/PRODUCTION-MASTER-PLAN.md`, `docs/UNIFIED-TASK-TRACKER.md`,
|
||||
`docs/1.8.0-RELEASE-HARDENING-PLAN.md`, `docs/RETICULUM-TRANSPORT-PROGRESS.md`, HANDOFF-*, test
|
||||
plans, `docs/archive/*` — but **move all session/handoff/tracker material under
|
||||
`docs/history/`** (extending the existing honest `docs/archive/README.md` pattern) so the
|
||||
top-level `docs/` reads as current reference only. Add a banner to each: "historical working
|
||||
document, sanitized; not maintained."
|
||||
- Remove dangling agent-memory references in tracked docs (`docs/bulletproof-containers.md`,
|
||||
`docs/RETICULUM-TRANSPORT-PROGRESS.md`, `docs/registry-manifest-design.md`,
|
||||
`docs/bitcoin-multi-version-design.md` progress block).
|
||||
- De-status the 14 design docs (strip "Status/RESUME POINT" headers into a one-line status
|
||||
field; e.g. `docs/APP-PACKAGING-MIGRATION-PLAN.md` → public app-platform design doc).
|
||||
- Extract North-Star narrative from PRODUCTION-MASTER-PLAN into `docs/ROADMAP.md`; extract
|
||||
the "run the gate ON the node" philosophy from `docs/multinode-testing-plan.md` into
|
||||
`tests/lifecycle/TESTING.md`.
|
||||
- Add `docs/README.md` index (bitcoin/bitcoin `doc/` style): Getting started / Architecture /
|
||||
App development / Operations / Design docs (ADRs) / History.
|
||||
- README fixes: LICENSE link becomes real, Documentation table repointed at the reorganized
|
||||
docs, remove "Deploy to a Test Node" private-LAN section, point Contributing at
|
||||
CONTRIBUTING.md only.
|
||||
|
||||
## Phase 5 — Deep code cleanup (ordered zero-risk → highest-risk; cut-line after any commit)
|
||||
|
||||
### A. Zero-risk deletions & metadata (S each, own commits)
|
||||
- **A1** Delete orphan non-compiling StartOS crates: `core/models`, `core/helpers`,
|
||||
`core/js-engine` (incl. 2 committed `JS_SNAPSHOT.*.bin`), `core/container-init` (~4,100 LOC,
|
||||
zero references). Verify: `cargo build --workspace && cargo test --all-features`.
|
||||
- **A2** Delete unreferenced Vue components: `neode-ui/src/components/{AppSwitcher,EmptyState,SkeletonCard}.vue`. Verify: `npm run type-check && npm run build`.
|
||||
- **A3** Fix `.gitignore` lockfile lines (7: `Cargo.lock`, 15: `package-lock.json`) — lockfiles are intentionally tracked; the rules are misleading and swallow future lockfiles.
|
||||
- **A4** LICENSE + Cargo license fields (see 4a). Verify with `cargo metadata`.
|
||||
- **A5** `core/rust-toolchain.toml` pinning `1.95.0`; align `.github/workflows/ci.yml` (remove explicit `toolchain: stable` input so the file wins). Upgrades become deliberate PRs.
|
||||
- **A6** `core/rustfmt.toml` codifying **defaults only** (`edition = "2021"` + comment) — do NOT add style options days before launch (whole-tree reformat churn). Verify `cargo fmt --all -- --check` yields no diff.
|
||||
|
||||
### B. CI guards (zero runtime risk)
|
||||
- **B1** Enable vitest in CI: run `cd neode-ui && npm run test` locally; fix trivial failures, `.skip`+issue flaky ones; add step to the frontend job. Playwright → tracked issue only (needs browsers + mock backend orchestration).
|
||||
- **B2** Raw podman/systemctl **ratchet, not migration**: the 132 raw `Command::new("podman"/"systemctl")` sites use subcommands the `core/container/src/podman_client.rs` wrapper doesn't expose (network/inspect/ps/port), 43 sites are in gate-critical `install.rs`, and the prod path intentionally uses Quadlet+systemctl. Add `scripts/ci/raw-podman-ratchet.sh` (count vs committed baseline, fail on increase) as a CI step + tracked issue for wrapper API design.
|
||||
|
||||
### C. Clippy suppression trim (`core/archipelago/src/main.rs:8-18`, per-lint commits)
|
||||
- Remove cheaply: `assertions_on_constants`, `drop_non_drop`, `wildcard_in_or_patterns`, `doc_lazy_continuation`, `enum_variant_names` (targeted allows on serde enums — never rename wire variants).
|
||||
- Own careful commit: `unused_io_amount` — a **correctness** lint; fix sites with `read_exact`/`write_all` or documented targeted allows (`mesh/serial.rs:456,496` has raw partial reads; serial framing may be intentional). Full test suite + gate after.
|
||||
- Keep crate-wide with justifying comment: `too_many_arguments`, `type_complexity`; attempt `ptr_arg` (`&Vec<T>`→`&[T]`, mechanical) if time allows — first to cut.
|
||||
- Verify each: `cargo clippy --all-targets --all-features -- -D warnings && cargo test --all-features`.
|
||||
|
||||
### D. dead_code lift — Tiers 1–2 pre-launch, Tier 3 → commented allows + issues
|
||||
Per-module procedure (one file per commit): remove `#![allow(dead_code)]` → `cargo check
|
||||
--all-targets --all-features` → triage each warning: (a) genuinely dead → delete;
|
||||
(b) future-feature/protocol-mandated → targeted `#[allow(dead_code)] // TODO(#NNN): …`;
|
||||
(c) missing wiring → keep + targeted allow + issue (don't fix wiring in this workstream) →
|
||||
clippy `-D warnings` + tests → commit.
|
||||
- **Tier 1 (small/leaf, S each):** `swarm/seed_advert.rs`, `transport/{mesh_transport,lan,chunking,delta}.rs`, `mesh/{crypto,alerts,types,outbox}.rs`, `streaming/mod.rs`, `wallet/mod.rs`.
|
||||
- **Tier 2 (M each):** `fips/{mod,iface,dial}.rs` (41 external refs → little residual deadness), `mesh/{x3dh,ratchet,steganography,message_types}.rs` — for crypto files bias to (b) with roadmap comments (unused crypto attracts auditor noise; every kept item needs its why).
|
||||
- **Tier 3 (defer, riskiest):** `mesh/{mod,reticulum,protocol,serial,bitcoin_relay}.rs`, `transport/mod.rs` — change each blanket allow to `#![allow(dead_code)] // Hardware-mesh surface partially wired; triage tracked in #NNN`.
|
||||
- Optional S/M win: move `prod_orchestrator.rs`'s 5,034-line `#[cfg(test)]` module to a sibling file via `#[path]` (pure move, halves the 6,291-line file).
|
||||
|
||||
### E. stacks.rs legacy fallbacks (highest risk — LAST, evidence-gated)
|
||||
Legacy installers for immich/btcpay/mempool/indeedhub (`core/archipelago/src/api/rpc/package/stacks.rs:838/1047/1267/1498`, ~1,000 LOC with hardcoded registry IPs) fire only on "unknown app_id, zero members installed", logging `INSTALL ORCH SKIP` (stacks.rs:673). Netbird already uses the hard-error replacement (stacks.rs:1898-1920).
|
||||
1. Run the full gate on the node; grep install logs for `INSTALL ORCH SKIP`.
|
||||
2. Zero SKIPs → replace each legacy body with the netbird-style hard error (keep orchestrator call + `adopt_stack_if_exists`; satisfies migrations-never-destroy-data). Re-run gate; any red → revert + issue.
|
||||
3. Any SKIP → don't delete; issue: "deploy manifests fleet-wide, then delete legacy installers".
|
||||
|
||||
### Explicitly deferred → public tracked issues at launch
|
||||
PodmanClient API extension + call-site migration; god-module splits (`install.rs`, `update.rs`, `mesh/mod.rs`); Playwright in CI; Tier-3 dead_code triage; `password123` default hardening.
|
||||
|
||||
## Phase 6 — Fresh-history publish
|
||||
|
||||
1. Freeze: all phases merged on internal `main`, gate green, catalog re-signed.
|
||||
2. Build the public tree: `git archive`-style export of HEAD (never copy `.git/` — it holds
|
||||
credentialed remotes) → new repo, single initial commit ("Initial public release, vX.Y.Z"),
|
||||
optionally preserving CHANGELOG.md as the human-readable history.
|
||||
3. Pre-publish gate on the export: `scripts/audit-secrets.sh` (fixed version) clean; grep-zero for
|
||||
`sk-ant-`, rotated-password strings, `146.59.87.168`, tailnet `100.` IPs, `192.168.1.`,
|
||||
internal hostnames; `du -sh .git` sanity (< ~100 MB); fresh `git clone` + `cd core && cargo build`
|
||||
+ `cd neode-ui && npm ci && npm run build` on a clean machine/container; one app image pull
|
||||
from the public domain.
|
||||
4. Publish to GitHub; enable issue templates (already present in `.github/`); file the deferred-work
|
||||
issues (from Phase 5's issue list) as the initial public issue set — honest and gives contributors entry points.
|
||||
5. Internal repo remains the private full-history remote; decide sync direction post-launch
|
||||
(recommend: public repo becomes canonical, private keeps only ops/infra notes).
|
||||
|
||||
## Verification (end-to-end)
|
||||
|
||||
- `tests/lifecycle/run-gate.sh` green on the node after Phases 3 + 5E (and after any lifecycle-touching commit).
|
||||
- CI green on every phase commit: `cargo fmt --check`, `clippy -D warnings`, `cargo test --all-features`, frontend type-check + build + (new) vitest.
|
||||
- Phase-6 clean-machine clone/build/pull test is the final acceptance test — it simulates the first outside developer.
|
||||
- Docs acceptance: a reader following `docs/app-developer-guide.md` + the new quadlet/lifecycle/secrets docs can build and install an app manifest without any private infra.
|
||||
|
||||
## Sequencing / cut-line
|
||||
|
||||
Order: 0 → 1 → 2 → (3 ∥ 4) → 5 (A→E) → 6. Phases 0–2 are non-negotiable security; Phase 3 is the
|
||||
functional blocker; Phase 4 is the developer-experience payload; Phase 5 can be cut after any
|
||||
commit (minimum viable: A1–A6, B1–B2, unused_io_amount fix); Phase 6 last. If the timeline
|
||||
compresses, Tier-2 dead_code and Phase E move to public issues — everything else holds.
|
||||
@@ -12,6 +12,33 @@ Full plan: `.claude/plans/enchanted-strolling-rocket.md`. Memory pointer:
|
||||
avoiding `mesh/listener/session.rs` transport plumbing + `mesh/mod.rs` routing, which this work
|
||||
owns. Stay out of `meshtastic.rs`/`protocol.rs` to avoid collisions.
|
||||
|
||||
## Checkpoint 2026-07-28 — RNode connect + names FIXED, live-verified E2E (read this first)
|
||||
|
||||
The fleet reflash back to RNode firmware exposed a stack of bugs that made Reticulum
|
||||
unusable on CP2102-bridged boards (Heltec V3 etc.) and left every archy node nameless on
|
||||
RNS. All fixed in `a8c4694c` (backend) + `3f76b496` (UI), live-verified on archi-dev-box
|
||||
and archy-x250-dev with a real RNode-to-RNode LXMF message (`transport: "reticulum"` in
|
||||
mesh-messages) plus a cross-transport reply:
|
||||
|
||||
1. **probe_rnode boot race** — serial open pulses DTR/RTS via the USB-UART bridge → ESP32
|
||||
power-cycles → KISS DETECT written 300ms later is eaten during ~2.5-3s of boot. Fix:
|
||||
immediate probe (fast path) + drain-until-quiet boot settle + second DETECT window.
|
||||
2. **configure() was a no-op on a running listener** (only enable/disable restarted it) —
|
||||
the setup modal's apply/keep-as-is and every rename did nothing until process restart.
|
||||
3. **Name propagation** — `config.advert_name` had no reader; `server.set-name` never
|
||||
reached mesh; daemon display name fixed at spawn to the "Archy" default; the ARCHY:2
|
||||
announce blob REPLACED the LXMF name. Now: announces carry msgpack
|
||||
`[name, stamp_cost, sf, ARCHY-blob]` (Sideband-compatible, blob invisible to stock
|
||||
clients), daemon has a `set_name` verb, renames bounce the session live.
|
||||
4. **Daemon-death detection** (was invisible up to the 30-min RX-stall watchdog),
|
||||
**modal re-trigger loop** (plugged_at used tty mtime → bumps on every open; now
|
||||
btime/ctime), **ARCHY:2 federation-name clobber**, **mesh.refresh RPC** (Refresh button
|
||||
now actually re-queries the radio), **Meshtastic mesh.broadcast now sends NodeInfo**.
|
||||
|
||||
Still open here: legacy-format peers (old fleet builds) show as `Reticulum <hex4>` until
|
||||
they OTA; RNode RF params still daemon-hardcoded (EU-868 869.525/125k/SF8/CR5); Phase 4
|
||||
multi-radio; duty-cycle guard.
|
||||
|
||||
## Status at a glance
|
||||
|
||||
| Phase | What | Status |
|
||||
|
||||
@@ -15,6 +15,26 @@ those are marked ✅ below with the commit that did it, so we stop re-litigating
|
||||
|
||||
## Tier 0 — Quick / mechanical, no blockers
|
||||
|
||||
- [ ] **Ship the lightning payment false-failure fix in the next release** (fixed
|
||||
on main 2026-07-27, needs OTA). Slow multi-hop payments (>15s) surfaced as
|
||||
"Payment failed" while LND settled them in the background — the shared LND
|
||||
REST client's 15s timeout aborted the synchronous `/v1/channels/transactions`
|
||||
wait. Now: payinvoice decodes the invoice first for its payment hash, waits
|
||||
up to 120s on a dedicated client, returns `status: "pending"` (never a
|
||||
failure) on timeout, and the new `lnd.paymentstatus` RPC + frontend
|
||||
`payLightningInvoice()` helper poll to a real terminal state (all 5 UI call
|
||||
sites migrated). Verify on Framework PT with a real multi-hop payment.
|
||||
- [ ] **Show the app version on the companion mobile-app banner in the app store
|
||||
and on its install/pairing modal** (user request 2026-07-27) — so it's
|
||||
obvious at a glance whether the node is serving the latest APK build.
|
||||
- [ ] **Optimise the companion QR scan — quicker + better** (user request
|
||||
2026-07-27; deferred to a later session on purpose). The pairing/scan QR
|
||||
flow works (user-verified on-device 2026-07-27) but should get faster and
|
||||
smoother: quicker camera start + decode (scan resolution/framerate,
|
||||
continuous autofocus), more forgiving in low light / at an angle, and
|
||||
snappier feedback once the code locks. Touch the native-scan path from
|
||||
PR #104 and the in-app scan modal together so both benefit.
|
||||
|
||||
- [ ] **Update `tests/lifecycle/TESTING.md`'s stale Release Gates checklist** (lines
|
||||
289–296) — several boxes are unchecked but actually true now:
|
||||
- #1 bitcoin-stops: covered by `tests/lifecycle/bats/bitcoin-knots.bats` stop/restart
|
||||
|
||||
@@ -162,7 +162,8 @@ All endpoints use JSON-RPC over HTTP POST to `/rpc/v1`.
|
||||
| `lnd.openchannel` | `{ pubkey: string, amount: number }` | `{ funding_txid: string }` | Yes |
|
||||
| `lnd.closechannel` | `{ channel_point: string }` | `{ closing_txid: string }` | Yes |
|
||||
| `lnd.newaddress` | — | `{ address: string }` | Yes |
|
||||
| `lnd.sendcoins` | `{ addr: string, amount: number }` | `{ txid: string }` | Yes |
|
||||
| `lnd.sendcoins` | `{ addr: string, amount?: number, send_all?: bool, target_conf?: number, sat_per_vbyte?: number }` | `{ txid: string }` | Yes |
|
||||
| `lnd.estimatefee` | `{ addr: string, amount: number, target_conf?: number }` | `{ fee_sat: number, sat_per_vbyte: number }` | Yes |
|
||||
| `lnd.createinvoice` | `{ amount: number, memo?: string }` | `{ payment_request: string }` | Yes |
|
||||
| `lnd.payinvoice` | `{ payment_request: string }` | `{ preimage: string }` | Yes |
|
||||
| `lnd.create-psbt` | `{ outputs: object, ... }` | `{ psbt: string }` | Yes |
|
||||
|
||||
@@ -146,7 +146,7 @@ app:
|
||||
path: /health
|
||||
interfaces:
|
||||
main:
|
||||
type: web
|
||||
type: ui
|
||||
port: 8090
|
||||
```
|
||||
|
||||
|
||||
+16
-12
@@ -74,9 +74,10 @@ archy/
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **macOS** (development machine): Node.js 20+, npm
|
||||
- **Linux server** (`192.168.1.228`): Rust toolchain, Podman, Nginx, Debian 13
|
||||
- SSH key: `~/.ssh/archipelago-deploy`
|
||||
- Node.js 20+ and npm for frontend development.
|
||||
- Rust stable for backend development.
|
||||
- Linux with Podman, systemd, and Nginx for host integration work.
|
||||
- Debian 13 is the target runtime for release validation.
|
||||
|
||||
### Local Frontend Development
|
||||
|
||||
@@ -86,17 +87,18 @@ npm install
|
||||
npm start # Vite dev server on :8100, mock backend on :5959
|
||||
```
|
||||
|
||||
The dev server at `http://localhost:8100` uses a mock backend. Login with `password123`.
|
||||
The dev server at `http://localhost:8100` uses a mock backend.
|
||||
|
||||
### Deploying Changes
|
||||
|
||||
**Never build Rust on macOS.** The deploy script rsyncs source to the Linux server and builds there.
|
||||
Release and host-integration builds should run on Linux. The deploy script rsyncs
|
||||
source to a configured Linux target and builds there.
|
||||
|
||||
```bash
|
||||
# Deploy to live server (builds backend + frontend, restarts services)
|
||||
# Deploy to the configured primary target (builds backend + frontend, restarts services)
|
||||
./scripts/deploy-to-target.sh --live
|
||||
|
||||
# Deploy to both servers
|
||||
# Deploy to both configured targets
|
||||
./scripts/deploy-to-target.sh --both
|
||||
```
|
||||
|
||||
@@ -114,14 +116,16 @@ The deploy script:
|
||||
# Frontend tests
|
||||
cd neode-ui && npm test
|
||||
|
||||
# Backend tests (on dev server via SSH)
|
||||
ssh -i ~/.ssh/archipelago-deploy archipelago@192.168.1.228 \
|
||||
"cd ~/archy/core && cargo test --all-features"
|
||||
# Backend tests
|
||||
cd core && cargo test --all-features
|
||||
|
||||
# Both
|
||||
./scripts/run-tests.sh
|
||||
```
|
||||
|
||||
`scripts/run-tests.sh` can run backend tests on a Linux target when
|
||||
`ARCHIPELAGO_SSH_HOST` and `ARCHIPELAGO_SSH_KEY` are set.
|
||||
|
||||
## Adding a New RPC Endpoint
|
||||
|
||||
### 1. Create the Handler
|
||||
@@ -200,7 +204,7 @@ async myAction(params: { name: string }): Promise<{ ok: boolean; result: string
|
||||
|
||||
```bash
|
||||
./scripts/deploy-to-target.sh --live
|
||||
curl -X POST http://192.168.1.228/rpc/v1 \
|
||||
curl -X POST http://<node-host>/rpc/v1 \
|
||||
-H "Content-Type: application/json" \
|
||||
-b "archipelago_session=YOUR_SESSION" \
|
||||
-d '{"method":"mymodule.action","params":{"name":"test"}}'
|
||||
@@ -309,5 +313,5 @@ mod tests {
|
||||
2. Make changes following the standards above
|
||||
3. Test locally: `cd neode-ui && npm test`
|
||||
4. Deploy to dev server: `./scripts/deploy-to-target.sh --live`
|
||||
5. Verify at `http://192.168.1.228`
|
||||
5. Verify on your configured development target
|
||||
6. Commit with conventional format: `feat: add my feature`
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
# Nostr Git Source Hosting Plan
|
||||
|
||||
This plan describes how Archipelago can publish and accept contributions to its
|
||||
source code through `ngit`, NIP-34, and GRASP while keeping the developer
|
||||
experience inside Archipelago.
|
||||
|
||||
## Goals
|
||||
|
||||
- Publish Archipelago source from a sanitized, fresh-history repository.
|
||||
- Make the in-app registry the primary onboarding path for contributors.
|
||||
- Let contributors clone, branch, push PR branches, open PRs, and discuss issues
|
||||
with a Nostr identity from their Archipelago node.
|
||||
- Follow the Bitcoin Core development model: broad public review and easy forks,
|
||||
with canonical merge authority held by a small maintainer set.
|
||||
- Give contributors full read, fork, and proposal rights, but no direct merge
|
||||
rights on the canonical repository.
|
||||
- Keep the official maintainer identity and merge authority separate from user
|
||||
node identities.
|
||||
|
||||
## Current Building Blocks
|
||||
|
||||
Archipelago already has most of the primitives needed for this:
|
||||
|
||||
- App manifests and the app registry already install developer tooling as
|
||||
rootless Podman apps.
|
||||
- The `gitea` app provides a conventional fallback Git UI and package registry.
|
||||
- The app launcher already exposes a consent-gated NIP-07 bridge for launched
|
||||
apps using `getPublicKey`, `signEvent`, NIP-04, and NIP-44 requests.
|
||||
- The backend exposes node and identity Nostr signing RPC methods.
|
||||
- FIPS gives nodes a stable mesh identity and private transport path, but repo
|
||||
announcements and PRs should remain NIP-34 compatible on normal Nostr relays.
|
||||
- DWN protocol registration exists and can be used later for local contribution
|
||||
metadata/cache, but should not be required for the first public workflow.
|
||||
|
||||
## Protocol Basis
|
||||
|
||||
Use existing Nostr Git conventions rather than inventing an Archipelago-only
|
||||
protocol:
|
||||
|
||||
- NIP-34 repository announcement events identify repositories with kind `30617`.
|
||||
- NIP-34 repository state events publish branch/tag refs with kind `30618`.
|
||||
- NIP-34 patches, pull requests, PR updates, issues, and status events use kinds
|
||||
`1617`, `1618`, `1619`, `1621`, and `1630`-`1633`.
|
||||
- `ngit` provides the `git-remote-nostr` helper for `nostr://` clone URLs and PR
|
||||
branches.
|
||||
- GRASP servers provide Git Smart HTTP storage while Nostr events remain the
|
||||
authority for repository identity, refs, PRs, issues, and maintainer state.
|
||||
|
||||
Primary references:
|
||||
|
||||
- https://nips.nostr.com/34
|
||||
- https://docs.rs/crate/ngit/latest/source/README.md
|
||||
- https://ngit.dev/grasp/
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
### Apps
|
||||
|
||||
Create two first-party apps:
|
||||
|
||||
- `ngit`: CLI/runtime package containing `ngit` and `git-remote-nostr`.
|
||||
- `archipelago-source`: web UI for cloning Archipelago source, viewing NIP-34
|
||||
issues/PRs, opening branches, and submitting PR events.
|
||||
|
||||
The `archipelago-source` app should depend on `ngit`. It can also recommend
|
||||
Gitea for users who want a conventional local web Git UI, but Gitea should not
|
||||
be the source of truth for public contribution permissions.
|
||||
|
||||
### Contributor Onboarding
|
||||
|
||||
When the user installs `archipelago-source` from the registry:
|
||||
|
||||
1. Show a modal before first launch: "Contribute to Archipelago".
|
||||
2. Explain that the app will use their Archipelago Nostr identity to clone and
|
||||
sign contribution events.
|
||||
3. Display the maintainer repository announcement, clone URL, maintainer npub,
|
||||
and relay/GRASP endpoints.
|
||||
4. Ask for consent to:
|
||||
- fetch repository metadata from configured relays,
|
||||
- clone source through `nostr://`,
|
||||
- create local branches,
|
||||
- sign NIP-34 issue/PR/comment events,
|
||||
- push PR branches to approved GRASP servers.
|
||||
5. Store approval per app origin, identity id, repository id, and relay set.
|
||||
|
||||
This should build on the existing NIP-07 app-launcher bridge, but use a more
|
||||
specific permission scope than the generic sign-event approval.
|
||||
|
||||
### Identity And Permissions
|
||||
|
||||
Use four identity classes:
|
||||
|
||||
- `archipelago-maintainer`: an offline or tightly controlled Nostr key that
|
||||
signs the canonical kind `30617` repo announcement and status/merge events.
|
||||
- `archipelago-merge-maintainer`: one of the small set of maintainer npubs
|
||||
allowed to advance canonical refs and publish valid merged/applied status.
|
||||
- `archipelago-build`: release automation key for signed release artifacts and
|
||||
CI status events. It must not have merge authority.
|
||||
- `contributor`: user node or app-specific identity used for PRs, issues, and
|
||||
comments.
|
||||
|
||||
Contributor rights:
|
||||
|
||||
- Clone the repository.
|
||||
- Open issues.
|
||||
- Push proposal branches using `pr/<npub>/<short-topic>` or `pr/<event-id>`.
|
||||
- Publish NIP-34 PR/update/comment events.
|
||||
- Rebase and update their own PR branch.
|
||||
- Run local validation and attach status evidence.
|
||||
|
||||
Contributor restrictions:
|
||||
|
||||
- Cannot update `refs/heads/main` or release branches in canonical state.
|
||||
- Cannot publish maintainer-valid merge/applied status.
|
||||
- Cannot alter the canonical repository announcement.
|
||||
- Cannot publish release catalog signatures.
|
||||
|
||||
Maintainer rights:
|
||||
|
||||
- Publish/update the canonical repo announcement.
|
||||
- Publish canonical `refs/heads/main` state.
|
||||
- Mark PRs merged/closed/draft via NIP-34 status events.
|
||||
- Sign release tags and catalog updates.
|
||||
|
||||
Fork rights:
|
||||
|
||||
- Any contributor can create their own NIP-34 kind `30617` repository
|
||||
announcement for a fork.
|
||||
- Fork announcements should use the NIP-34 `u` tag to point back to the
|
||||
canonical `archy` repository.
|
||||
- The source app should make forking a first-class path: "Fork on Nostr", clone
|
||||
the fork locally, push branches to the contributor's GRASP list, and open PRs
|
||||
back to canonical Archipelago when they want review.
|
||||
- Forks can have their own maintainer npubs, relays, policies, and release
|
||||
cadence, but the app should clearly label them as forks unless signed by the
|
||||
canonical maintainer set.
|
||||
|
||||
The GRASP server policy should enforce this by accepting pushes to maintainer
|
||||
refs only when backed by signed maintainer state, while allowing contributor PR
|
||||
refs from their own npubs.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
Canonical repo announcement:
|
||||
|
||||
- repo id: `archy`
|
||||
- display name: `Archipelago`
|
||||
- clone URLs:
|
||||
- `nostr://<maintainer-npub>/<relay-hint>/archy`
|
||||
- `https://<grasp-host>/<maintainer-npub>/archy.git`
|
||||
- relays:
|
||||
- Archipelago-operated relay
|
||||
- at least two public Nostr relays that support the event load
|
||||
- GRASP servers:
|
||||
- Archipelago-operated GRASP instance
|
||||
- one public GRASP-compatible mirror
|
||||
|
||||
Keep the existing HTTP Git remote as a mirror during launch. The docs can
|
||||
present `nostr://` as the preferred contribution path once the workflow is
|
||||
proven.
|
||||
|
||||
## UI Requirements
|
||||
|
||||
The source app should provide:
|
||||
|
||||
- A first-run contribution modal with a real Archipelago source graphic, not a
|
||||
generic text-only dialog.
|
||||
- Current clone status and local path.
|
||||
- Branch list, changed files, commit form, and push/open-PR flow.
|
||||
- PR inbox, issue list, maintainer status, and relay health.
|
||||
- Explicit identity indicator showing which npub will sign events.
|
||||
- A merge rights indicator that clearly says contributors can propose changes
|
||||
but cannot merge them.
|
||||
- A fork flow that creates a user-owned NIP-34 repo announcement and remote,
|
||||
then offers "Open PR to Archipelago" from any fork branch.
|
||||
- Maintainer badges based only on pinned canonical maintainer npubs, not relay
|
||||
metadata or server-side account names.
|
||||
- Links to container docs, deployment docs, manifest spec, and open-source
|
||||
readiness tasks.
|
||||
|
||||
## Backend Work
|
||||
|
||||
Add an RPC module for source contribution workflow:
|
||||
|
||||
- `source.repo-info`: returns canonical announcement, clone URL, relay set,
|
||||
maintainer npubs, and local clone state.
|
||||
- `source.ensure-ngit`: verifies the `ngit` app/runtime is installed.
|
||||
- `source.clone`: clones or updates the local source checkout.
|
||||
- `source.status`: returns branch, dirty files, ahead/behind, and PR state.
|
||||
- `source.commit`: creates a local commit from selected files.
|
||||
- `source.fork`: creates a contributor-owned NIP-34 fork announcement and local
|
||||
remote.
|
||||
- `source.open-pr`: pushes a PR branch and publishes a kind `1618` event.
|
||||
- `source.update-pr`: updates the branch and publishes kind `1619`.
|
||||
- `source.issue`: publishes a kind `1621` event.
|
||||
|
||||
Backend must shell out through a narrow command wrapper, never arbitrary user
|
||||
commands. The wrapper should set an isolated working tree under
|
||||
`/var/lib/archipelago/source/archy`, run as the Archipelago service user, and
|
||||
deny operations outside that path.
|
||||
|
||||
## Security Model
|
||||
|
||||
- Never expose maintainer private keys to an Archipelago node.
|
||||
- Prefer app-specific contributor identities over the node's default identity.
|
||||
- Require per-action consent for first PR push, issue creation, and signing any
|
||||
event that tags the canonical repository.
|
||||
- Pin the canonical maintainer npub in the app manifest and backend config.
|
||||
- Keep the canonical merge-maintainer allow list signed by the
|
||||
`archipelago-maintainer` key; never infer merge rights from GRASP server
|
||||
accounts.
|
||||
- Verify the canonical kind `30617` event signature before displaying clone
|
||||
instructions.
|
||||
- Treat GRASP servers as untrusted storage; verify Git refs against signed
|
||||
Nostr state.
|
||||
- Do not use destructive git operations from the UI without an explicit modal.
|
||||
- Store local clones and generated patches outside app container writable roots
|
||||
unless the user exports them.
|
||||
|
||||
## MVP
|
||||
|
||||
1. Package `ngit` as a first-party app.
|
||||
2. Stand up one Archipelago-operated GRASP server and one Nostr relay.
|
||||
3. Publish sanitized fresh-history `archy` through `ngit init`.
|
||||
4. Add a simple `archipelago-source` app that clones source and links out to the
|
||||
preferred Nostr Git browser.
|
||||
5. Add app-launcher consent scopes for repository-specific NIP-34 signing.
|
||||
6. Allow issues and PR branch submission from contributor npubs.
|
||||
7. Add a one-click fork flow that publishes a contributor-owned fork
|
||||
announcement referencing canonical Archipelago.
|
||||
8. Keep maintainer merge/status publication manual.
|
||||
|
||||
## Later
|
||||
|
||||
- Native PR review UI with file diffs and inline comments.
|
||||
- CI status events signed by the build identity.
|
||||
- FIPS-first source sync between trusted Archipelago nodes.
|
||||
- Private prerelease repositories using NIP-42 allow lists and/or protected
|
||||
events if the ecosystem support is mature enough.
|
||||
- Multi-maintainer policy with threshold signatures or explicit maintainer-list
|
||||
rotation events.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Which maintainer npub should become canonical for `archy`?
|
||||
- Should contributor identities be node-default or app-specific by default?
|
||||
- Which GRASP implementation should be deployed first: `ngit-grasp` or another
|
||||
NIP-34/GRASP-compatible relay?
|
||||
- Should the source app include a full web Git UI in v1, or launch Gitea/ngit
|
||||
browser links for review while keeping signing/submission native?
|
||||
- What exact license and contribution certificate should contributors accept
|
||||
before submitting PR events?
|
||||
Reference in New Issue
Block a user