diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 4000d369..8b61fc73 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -10,6 +10,19 @@ declared exit criteria (multinode pass + workstreams B/C/F), `.planning/codebase ## v1 Requirements +### Federation & Mesh Hardening (FED) + +- [ ] **FED-01**: Removing a federation node sticks — it disappears from every UI surface, tombstones propagate, it never reappears via later sync cycles, and a failed removal surfaces an error (never a silent no-op) +- [ ] **FED-02**: Federation sync converges and is observable — after sync settles, fleet nodes agree on the node list with fresh status; stale entries, duplicates, and silent sync failures are eliminated and sync errors are operator-visible +- [ ] **FED-03**: A structured code review of the federation/fleet area (`core/archipelago/src/federation`, node sync, FIPS/transport dial layer) and mesh area (`core/archipelago/src/mesh`, mesh RPC surface) is completed, with every finding fixed or explicitly deferred with a reason +- [ ] **FED-04**: Mesh messaging parity — attachment send (and the rest of the mesh chat surface) behaves identically on the demo and on real nodes: the demo backend implements the same RPC surface the UI calls, transport decisions mirror the real size-based tier logic, and no demo-only modals exist + +### UI Performance (PERF) + +- [ ] **PERF-01**: The slowest tab switches and secondary-screen opens are profiled with causes named (remount storms, serial RPC waterfalls, uncached fetches) — fixes are targeted, not guessed +- [ ] **PERF-02**: Main-tab switches render immediately from cached state with background refresh — no blank screens or long spinners on tabs already visited this session +- [ ] **PERF-03**: Secondary screens (screens reached from a tab's main page) open without a blocking full reload and are instant on repeat visits — verified on real node hardware, not just the dev box + ### Multinode Verification (MNODE) - [ ] **MNODE-01**: The 5× destructive lifecycle gate passes on a second fleet node (archy-x250-beta) with 0 failures, run on-node per gate policy @@ -79,32 +92,39 @@ Which phases cover which requirements. Updated during roadmap creation. | Requirement | Phase | Status | |-------------|-------|--------| -| MNODE-01 | Phase 1 | Pending | -| MNODE-02 | Phase 1 | Pending | -| MNODE-03 | Phase 1 | Pending | -| LIFE-01 | Phase 2 | Pending | -| LIFE-02 | Phase 2 | Pending | -| LIFE-03 | Phase 2 | Pending | -| LIFE-04 | Phase 2 | Pending | -| LIFE-05 | Phase 2 | Pending | -| REG-01 | Phase 3 | Pending | -| REG-02 | Phase 3 | Pending | -| SEC-01 | Phase 4 | Pending | -| SEC-02 | Phase 4 | Pending | -| DEV-01 | Phase 5 | Pending | -| DEV-02 | Phase 5 | Pending | -| DEV-03 | Phase 5 | Pending | -| DEV-04 | Phase 5 | Pending | -| MKT-01 | Phase 6 | Pending | -| MKT-02 | Phase 6 | Pending | -| MKT-03 | Phase 6 | Pending | -| MKT-04 | Phase 6 | Pending | +| FED-01 | Phase 1 | Pending | +| FED-02 | Phase 1 | Pending | +| FED-03 | Phase 1 | Pending | +| FED-04 | Phase 1 | Pending | +| PERF-01 | Phase 2 | Pending | +| PERF-02 | Phase 2 | Pending | +| PERF-03 | Phase 2 | Pending | +| MNODE-01 | Phase 3 | Pending | +| MNODE-02 | Phase 3 | Pending | +| MNODE-03 | Phase 3 | Pending | +| LIFE-01 | Phase 4 | Pending | +| LIFE-02 | Phase 4 | Pending | +| LIFE-03 | Phase 4 | Pending | +| LIFE-04 | Phase 4 | Pending | +| LIFE-05 | Phase 4 | Pending | +| REG-01 | Phase 5 | Pending | +| REG-02 | Phase 5 | Pending | +| SEC-01 | Phase 6 | Pending | +| SEC-02 | Phase 6 | Pending | +| DEV-01 | Phase 7 | Pending | +| DEV-02 | Phase 7 | Pending | +| DEV-03 | Phase 7 | Pending | +| DEV-04 | Phase 7 | Pending | +| MKT-01 | Phase 8 | Pending | +| MKT-02 | Phase 8 | Pending | +| MKT-03 | Phase 8 | Pending | +| MKT-04 | Phase 8 | Pending | **Coverage:** -- v1 requirements: 20 total -- Mapped to phases: 20 +- v1 requirements: 27 total +- Mapped to phases: 27 - Unmapped: 0 --- *Requirements defined: 2026-07-29* -*Last updated: 2026-07-29 after roadmap creation* +*Last updated: 2026-07-29 — added FED (federation/mesh hardening) and PERF (UI performance) requirement groups; phases renumbered after inserting them as Phases 1–2* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index e5fed4c0..8e0d334b 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -3,12 +3,14 @@ ## Overview Brownfield milestone starting from a green single-node production gate (5/5 on .228, -2026-06-23). The journey: prove the platform across the fleet (multinode pass), make the -container lifecycle bulletproof (Quadlet default, self-healing, honest progress, no ghosts), -flip manifest distribution from OTA disk files to the signed registry, harden manifest -security enforcement to the full ADR-009 bar, ship the `archy app` developer CLI, and land -the decentralized Nostr marketplace — ending at the north star: a third-party developer -publishes an app via the signed/decentralized registry and a user installs it on their node. +2026-06-23). The journey: make federation and mesh rock-solid (node removal, sync, +messaging parity), fix the UI slowness users feel on every tab switch, prove the platform +across the fleet (multinode pass), make the container lifecycle bulletproof (Quadlet +default, self-healing, honest progress, no ghosts), flip manifest distribution from OTA +disk files to the signed registry, harden manifest security enforcement to the full +ADR-009 bar, ship the `archy app` developer CLI, and land the decentralized Nostr +marketplace — ending at the north star: a third-party developer publishes an app via the +signed/decentralized registry and a user installs it on their node. ## Phases @@ -16,18 +18,43 @@ publishes an app via the signed/decentralized registry and a user installs it on - Integer phases (1, 2, 3): Planned milestone work - Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) -- [ ] **Phase 1: Multinode Verification Pass** - Lifecycle gate green on a second node; cross-node federation/mesh/transport suites pass; federation removal sticks -- [ ] **Phase 2: Lifecycle Perfection & Quadlet Default** - Quadlet backends default, failed-unit self-healing, flap observability, cascade gate, truthful progress -- [ ] **Phase 3: Registry-Distributed Manifests** - Signed catalog carries full manifests; fleet flipped off OTA disk-file distribution -- [ ] **Phase 4: Manifest Security Enforcement** - Validation matches ADR-009 mandates; generated security profiles actually applied -- [ ] **Phase 5: Developer Tooling CLI** - `archy app validate/render/local-install/lifecycle-test` + developer guide -- [ ] **Phase 6: Decentralized Marketplace** - DID-signed publish to Nostr relays, trust-tier discovery, verified third-party install end-to-end +- [ ] **Phase 1: Federation & Mesh Hardening** - Deep review of federation/fleet + mesh code; node removal sticks, sync converges, mesh messaging behaves identically on demo and real nodes +- [ ] **Phase 2: UI Performance** - Tab switches and secondary screens render fast; worst transitions measured and fixed +- [ ] **Phase 3: Multinode Verification Pass** - Lifecycle gate green on a second node; cross-node federation/mesh/transport suites pass; federation removal sticks +- [ ] **Phase 4: Lifecycle Perfection & Quadlet Default** - Quadlet backends default, failed-unit self-healing, flap observability, cascade gate, truthful progress +- [ ] **Phase 5: Registry-Distributed Manifests** - Signed catalog carries full manifests; fleet flipped off OTA disk-file distribution +- [ ] **Phase 6: Manifest Security Enforcement** - Validation matches ADR-009 mandates; generated security profiles actually applied +- [ ] **Phase 7: Developer Tooling CLI** - `archy app validate/render/local-install/lifecycle-test` + developer guide +- [ ] **Phase 8: Decentralized Marketplace** - DID-signed publish to Nostr relays, trust-tier discovery, verified third-party install end-to-end ## Phase Details -### Phase 1: Multinode Verification Pass -**Goal**: The platform's lifecycle and federation guarantees are proven across the fleet, not just on .228 — the declared next exit criterion +### Phase 1: Federation & Mesh Hardening +**Goal**: Federation and mesh are tight — a structured review of the fleet/federation and mesh code feeds fixes so node removal sticks, sync converges, and mesh messaging (including attachments) behaves identically everywhere it runs **Depends on**: Nothing (first phase) +**Requirements**: FED-01, FED-02, FED-03, FED-04 +**Success Criteria** (what must be TRUE): + 1. A structured code review of the federation/fleet area (`core/archipelago/src/federation`, node sync, FIPS/transport dial layer) and the mesh area (`core/archipelago/src/mesh`, mesh RPC surface) produces a findings list, and every finding is fixed or explicitly deferred with a reason + 2. Removing a federation node removes it everywhere — it disappears from all UI surfaces, tombstones propagate, and it never reappears after later sync cycles; a failed removal surfaces an error instead of silently no-opping + 3. Federation sync converges: after sync settles, fleet nodes agree on the node list and node status is fresh — stale entries, duplicates, and silent sync failures are gone, and sync errors are visible to the operator + 4. Mesh attachment send works identically on the demo and on real nodes — same modals, same transport decisions, same success — with the demo backend implementing the same RPC surface the UI calls (no "Method not found", no demo-only chooser modal) +**Plans**: TBD + +### Phase 2: UI Performance +**Goal**: The UI feels fast — switching tabs and opening secondary screens (screens reached from a tab's main page) renders promptly instead of stalling on refetches and remounts +**Depends on**: Nothing (frontend-focused; parallelizable with Phase 1) +**Requirements**: PERF-01, PERF-02, PERF-03 +**Success Criteria** (what must be TRUE): + 1. The slowest tab switches and secondary-screen opens are profiled and the causes named (remount storms, serial RPC waterfalls, uncached fetches) before fixes land + 2. Switching between main tabs renders the target view immediately from cached state, refreshing data in the background — no blank screens or long spinners on tabs already visited this session + 3. Secondary screens open without a blocking full reload; repeat visits are instant + 4. The fixes are verified on real node hardware (not just the dev box) — the sluggishness the user reported is gone on-device +**Plans**: TBD +**UI hint**: yes + +### Phase 3: Multinode Verification Pass +**Goal**: The platform's lifecycle and federation guarantees are proven across the fleet, not just on .228 — the declared next exit criterion +**Depends on**: Phase 1 (proves the federation/mesh fixes hold fleet-wide) **Requirements**: MNODE-01, MNODE-02, MNODE-03 **Success Criteria** (what must be TRUE): 1. The 5× destructive lifecycle gate reports 0 failures on a second fleet node (archy-x250-beta), run on-node @@ -36,9 +63,9 @@ publishes an app via the signed/decentralized registry and a user installs it on 4. The on-air mesh suite passes between two radio-equipped nodes over real RF **Plans**: TBD -### Phase 2: Lifecycle Perfection & Quadlet Default +### Phase 4: Lifecycle Perfection & Quadlet Default **Goal**: An insanely-reliable container environment — every app installs, runs, restarts, uninstalls, and reinstalls cleanly with honest progress, no ghosts, and automatic recovery -**Depends on**: Phase 1 (Quadlet default-flip is gated on the second-node gate reporting clean) +**Depends on**: Phase 3 (Quadlet default-flip is gated on the second-node gate reporting clean) **Requirements**: LIFE-01, LIFE-02, LIFE-03, LIFE-04, LIFE-05 **Success Criteria** (what must be TRUE): 1. Restarting `archipelago.service` on a fleet node leaves every app container running — no SIGKILL-the-world, no multi-minute reconciler rebuild @@ -49,9 +76,9 @@ publishes an app via the signed/decentralized registry and a user installs it on **Plans**: TBD **UI hint**: yes -### Phase 3: Registry-Distributed Manifests +### Phase 5: Registry-Distributed Manifests **Goal**: Manifests ship via the signed registry, not OTA disk files — bumping or adding an app becomes a signed catalog change -**Depends on**: Phase 2 (fleet lifecycle stable under Quadlet default before changing the distribution channel) +**Depends on**: Phase 4 (fleet lifecycle stable under Quadlet default before changing the distribution channel) **Requirements**: REG-01, REG-02 **Success Criteria** (what must be TRUE): 1. A fleet node installs and updates an image-only app from the full manifest embedded in the signed catalog, verified against the pinned release-root key, with no corresponding OTA disk file present (disk remains the fallback for build-source apps) @@ -59,9 +86,9 @@ publishes an app via the signed/decentralized registry and a user installs it on 3. Bumping an app version fleet-wide requires only regenerating, re-signing, and publishing the catalog — no binary OTA, no disk rsync — proven live on the fleet **Plans**: TBD -### Phase 4: Manifest Security Enforcement +### Phase 6: Manifest Security Enforcement **Goal**: A third-party manifest cannot weaken node security — declared security policy is fully validated and actually enforced at runtime -**Depends on**: Phase 3 (enforcement guards the registry channel third-party manifests will arrive through) +**Depends on**: Phase 5 (enforcement guards the registry channel third-party manifests will arrive through) **Requirements**: SEC-01, SEC-02 **Success Criteria** (what must be TRUE): 1. A manifest violating ADR-009 mandates (root user, unpinned `latest` tag, capability outside the allow-list, disabled seccomp) is rejected at validation with a clear error naming the violation @@ -70,9 +97,9 @@ publishes an app via the signed/decentralized registry and a user installs it on 4. The single-node lifecycle gate stays green with enforcement on — existing catalog apps all pass the strengthened validation (or carry documented overrides) **Plans**: TBD -### Phase 5: Developer Tooling CLI +### Phase 7: Developer Tooling CLI **Goal**: A third-party developer can build, validate, and test an Archipelago app locally without reading platform internals -**Depends on**: Phase 4 (CLI validation must mirror the final enforced rule set) +**Depends on**: Phase 6 (CLI validation must mirror the final enforced rule set) **Requirements**: DEV-01, DEV-02, DEV-03, DEV-04 **Success Criteria** (what must be TRUE): 1. A developer runs `archy app validate` on a manifest directory and gets the same pass/fail verdict — including security rules — that a node would enforce at install @@ -81,9 +108,9 @@ publishes an app via the signed/decentralized registry and a user installs it on 4. A new developer following only the developer guide goes from an empty directory to a running app on a node — no tribal knowledge required **Plans**: TBD -### Phase 6: Decentralized Marketplace +### Phase 8: Decentralized Marketplace **Goal**: The north star — third-party developers publish apps via the decentralized registry and users install them on their nodes -**Depends on**: Phase 5 (publish rides the CLI; installs ride registry distribution from Phase 3 and enforcement from Phase 4) +**Depends on**: Phase 7 (publish rides the CLI; installs ride registry distribution from Phase 5 and enforcement from Phase 6) **Requirements**: MKT-01, MKT-02, MKT-03, MKT-04 **Success Criteria** (what must be TRUE): 1. A third-party developer publishes a DID-signed app manifest to public Nostr relays (NIP-78, kind 30078) using the tooling @@ -96,13 +123,16 @@ publishes an app via the signed/decentralized registry and a user installs it on ## Progress **Execution Order:** -Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 +Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 +(Phases 1 and 2 are independent and may be worked in parallel.) | Phase | Plans Complete | Status | Completed | |-------|----------------|--------|-----------| -| 1. Multinode Verification Pass | 0/TBD | Not started | - | -| 2. Lifecycle Perfection & Quadlet Default | 0/TBD | Not started | - | -| 3. Registry-Distributed Manifests | 0/TBD | Not started | - | -| 4. Manifest Security Enforcement | 0/TBD | Not started | - | -| 5. Developer Tooling CLI | 0/TBD | Not started | - | -| 6. Decentralized Marketplace | 0/TBD | Not started | - | +| 1. Federation & Mesh Hardening | 0/TBD | Not started | - | +| 2. UI Performance | 0/TBD | Not started | - | +| 3. Multinode Verification Pass | 0/TBD | Not started | - | +| 4. Lifecycle Perfection & Quadlet Default | 0/TBD | Not started | - | +| 5. Registry-Distributed Manifests | 0/TBD | Not started | - | +| 6. Manifest Security Enforcement | 0/TBD | Not started | - | +| 7. Developer Tooling CLI | 0/TBD | Not started | - | +| 8. Decentralized Marketplace | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 0ac7d1f8..f1705a0c 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,7 +2,7 @@ gsd_state_version: '1.0' status: planning progress: - total_phases: 6 + total_phases: 8 completed_phases: 0 total_plans: 0 completed_plans: 0 @@ -16,14 +16,14 @@ progress: See: .planning/PROJECT.md (updated 2026-07-29) **Core value:** A third-party developer can publish an app via the signed/decentralized registry and a user can install it on their node — manifest-driven, rootless, secure, robust. -**Current focus:** Phase 1 — Multinode Verification Pass +**Current focus:** Phase 1 — Federation & Mesh Hardening ## Current Position -Phase: 1 of 6 (Multinode Verification Pass) +Phase: 1 of 8 (Federation & Mesh Hardening) Plan: 0 of TBD in current phase Status: Ready to plan -Last activity: 2026-07-29 — Project initialized from doc ingest (10 ADRs + 1 SPEC) + codebase map; roadmap created (6 phases, 20 requirements, 100% coverage) +Last activity: 2026-07-29 — Inserted Phase 1 (Federation & Mesh Hardening) and Phase 2 (UI Performance) per user priority; prior phases renumbered 3–8 Progress: [░░░░░░░░░░] 0% @@ -42,6 +42,11 @@ Progress: [░░░░░░░░░░] 0% ## Accumulated Context +### Roadmap Evolution + +- Phase 1 added (2026-07-29): Federation & Mesh Hardening — user-directed top priority (node removal/sync issues, mesh attachment parity incl. demo); prior phases shifted down +- Phase 2 added (2026-07-29): UI Performance — slow tab switches and secondary screens; prior phases shifted down + ### Decisions Decisions are logged in PROJECT.md (10 locked ADRs in the `` block + milestone decisions table). Recent decisions affecting current work: @@ -57,10 +62,10 @@ None yet. ### Blockers/Concerns -- [Phase 1] Multinode gate on archy-x250-beta was launched 2026-07-01 (log on-node); verify outcome before re-running - [Phase 1] Federation tombstone fix touches trust code — fix carefully, re-verify with `tests/multinode/smoke.sh`, don't patch blind -- [Phase 3] Fleet registry flip awaits explicit user authorization + timing call -- [Phase 4] Strengthened ADR-009 validation may reject existing catalog apps — audit manifests before enforcement lands +- [Phase 3] Multinode gate on archy-x250-beta was launched 2026-07-01 (log on-node); verify outcome before re-running +- [Phase 5] Fleet registry flip awaits explicit user authorization + timing call +- [Phase 6] Strengthened ADR-009 validation may reject existing catalog apps — audit manifests before enforcement lands - [Global] Live OTA fleet: deploy to the dev pair before any OTA; gate re-runs required after orchestrator changes; some verification is user/hardware-gated (radios, on-device tests) ## Deferred Items