From fcd5a065dccc16a2accc662809c8c2bf3a064b78 Mon Sep 17 00:00:00 2001 From: archipelago Date: Wed, 8 Jul 2026 17:07:57 -0400 Subject: [PATCH] docs: rewrite README against code ground truth + add public ROADMAP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README fixes: version badge 1.3.1→1.8.0-alpha, real LOC (117k Rust/69k TS-Vue vs the stale 49k/47k), 51 apps not 29, tri-protocol mesh (Meshtastic/MeshCore/Reticulum) not Meshcore-only, Nostr NIP-33/44/04 (NIP-07 was never implemented), Ed25519 signed catalog + pinned release-root anchor, OTA host de-hardcoded (tx1138 retired), dead docs/MASTER_PLAN.md link removed, crate map corrected (openwrt added, phantom crates dropped), new Philosophy and Roadmap sections reflecting the manifest-driven/signed-registry north star. docs/ROADMAP.md is the public-facing shipped/in-progress/planned summary, synthesized from the master plan, unified tracker, and 1.8.0 hardening plan — including the completed signing ceremony and the 1.8.0-alpha version decision. Co-Authored-By: Claude Fable 5 --- README.md | 151 ++++++++++++++++++++++++++++++------------------ docs/ROADMAP.md | 97 +++++++++++++++++++++++++++++++ 2 files changed, 192 insertions(+), 56 deletions(-) create mode 100644 docs/ROADMAP.md diff --git a/README.md b/README.md index cafea0d2..6ce15da1 100644 --- a/README.md +++ b/README.md @@ -2,71 +2,99 @@ > Self-Sovereign Bitcoin Node OS -**Archipelago** is a bootable personal server OS. Flash it to a USB drive, install on any x86_64 or ARM64 machine, and manage Bitcoin infrastructure, self-hosted apps, and decentralized identity through a glassmorphism web UI. +**Archipelago** is a bootable personal server OS. Flash it to a USB drive, install on any x86_64 or ARM64 machine, and manage Bitcoin infrastructure, self-hosted apps, mesh communication, and decentralized identity through a glassmorphism web UI. [![Debian 13](https://img.shields.io/badge/Debian-13%20Trixie-a80030)](https://www.debian.org/) [![License](https://img.shields.io/badge/license-MIT-green)](LICENSE) [![Rust](https://img.shields.io/badge/rust-stable-orange)](https://www.rust-lang.org/) [![Vue.js](https://img.shields.io/badge/vue.js-3.5-brightgreen)](https://vuejs.org/) -[![Version](https://img.shields.io/badge/version-1.3.1--beta-blue)]() +[![Version](https://img.shields.io/badge/version-1.8.0--alpha-blue)]() + +## Philosophy + +Archipelago is being built as a **developer-ready app platform**, not a fixed appliance: + +- **Manifest-driven apps.** Every app is declared in a single `manifest.yml` — image, ports, volumes, secrets, health checks, security policy. The orchestrator owns the entire lifecycle; there is no per-app installer code and no host-level provisioning. +- **Signed distribution.** App manifests ship inside an Ed25519-signed catalog verified against a pinned release-root key, not as loose files on disk. OTA release manifests are signed the same way. +- **Decentralized marketplace.** Third-party developers publish apps via Nostr-based discovery (NIP-78) with DID-signed manifests and federation-weighted trust scoring — no gatekept central store. +- **Rootless and secure by default.** Rootless Podman only. Read-only root, no-new-privileges, capability allow-list, secrets materialised 0600 and never logged. Never rootful, never a Docker socket mount. +- **100%-uptime-capable.** Every container is a systemd Quadlet unit under `user.slice` that survives backend restarts; a level-triggered reconciler self-heals drift every 30 seconds; migrations never destroy data. ## Features ### Bitcoin Infrastructure -- **Bitcoin Knots** full node with pruning support -- **LND** Lightning Network daemon with channel management +- **Bitcoin Core and Bitcoin Knots** full nodes with per-app version pinning and bulletproof version switching, automatic prune/full mode based on disk size +- **LND** and **Core Lightning** with channel management - **ElectrumX** Electrum server for wallet connectivity - **BTCPay Server** for accepting Bitcoin payments - **Mempool** block explorer and fee estimator -- **Fedimint** federation guardian and gateway +- **Fedimint** federation guardian, gateway, and client — plus Cashu ecash wallet support -### Self-Hosted Apps (29) -Bitcoin, Storage (FileBrowser, Immich, Nextcloud), Productivity (Penpot, Vaultwarden), Media (Jellyfin, PhotoPrism), Search (SearXNG), AI (Ollama), Network (Tailscale, Nginx Proxy Manager), Home (Home Assistant), Nostr (nostr-rs-relay, Nostrudel), Dev (Grafana, Portainer), and more. +### Self-Hosted Apps (50+) +Storage (FileBrowser, Immich, Nextcloud), Productivity (Vaultwarden), Media (Jellyfin, PhotoPrism, IndeeHub), Search (SearXNG), Network (NetBird, Tailscale), Home (Home Assistant), Nostr (nostr-rs-relay, strfry), Dev/Ops (Gitea, Grafana, Portainer, Uptime Kuma), and more — 27 curated in the store UI, 50+ packaged as manifests. + +### Mesh Networking (tri-protocol) +- **Meshtastic**, **MeshCore**, and **Reticulum (RNS/LXMF)** LoRa transports behind one mesh chat UI +- End-to-end encryption with X3DH key agreement + double-ratchet +- RNode radio support with an OS-level `archy-rnodeconf` tool; interop verified against Sideband +- Image/voice attachments, mesh AI assistant (`!ai`), Bitcoin balance relay over mesh ### Decentralized Identity - Ed25519 node identity with DID Documents (did:key) - Multi-identity management (Personal/Business/Anonymous) - W3C Verifiable Credentials issuance and verification -- Decentralized Web Node (DWN) with bidirectional sync over Tor -- Nostr relay integration and NIP-07 signing for iframe apps +- Nostr integration: NIP-33 node discovery, NIP-44/NIP-04 encryption, relay hosting +- Decentralized Web Node (DWN) record sync between federated nodes over Tor ### Multi-Node Federation - Invite-based node joining over Tor hidden services - Trust levels (Trusted/Verified/Untrusted) with DID-based auth -- Bidirectional DWN state sync between federated nodes -- File sharing with access controls (free/peers-only/paid) - -### Mesh Networking -- LoRa radio communication via Meshcore protocol -- Device discovery and mesh routing -- Off-grid Bitcoin balance checks (planned) +- State sync and app deployment across federated nodes +- File sharing with access controls (free/peers-only/paid via Lightning, on-chain, or ecash) ### System Updates -- OTA updates from self-hosted Gitea (git.tx1138.com) with SHA256 verification -- Three update modes: Manual, Daily Check, Auto Apply (3 AM window) -- Rollback support with automatic backup before applying -- Full UI for update management in Settings +- OTA updates from a self-hosted Gitea release server, Ed25519-signature-verified against a pinned release-root key +- Resumable downloads, automatic pre-update backup, rollback with a post-update self-verify window +- Manual, scheduled-check, and auto-apply modes (auto-apply refuses unsigned manifests) ### Security -- ChaCha20-Poly1305 encrypted secrets at rest, Argon2id password hashing -- Rootless Podman: read-only root, cap-drop ALL, non-root user, no-new-privileges -- TOTP two-factor authentication -- Per-endpoint rate limiting, CSRF protection, input validation -- AppArmor profiles for container confinement -- Tor hidden services for all inter-node communication -- All crypto and container dependencies pinned to exact versions -- Full penetration test completed (33 findings, all remediated) +- Argon2id password hashing (transparent upgrade from legacy hashes), ChaCha20-Poly1305 encrypted secrets at rest +- Rootless Podman: read-only root, cap-drop ALL with a reviewed allow-list, no-new-privileges +- Signed release manifests and signed app catalog (Ed25519, pinned trust anchor) +- TOTP two-factor authentication, per-endpoint rate limiting, CSRF protection +- AppArmor profiles for container confinement; Tor hidden services for inter-node traffic +- Independent security audit of an early version archived in [`docs/archive/`](docs/archive/security-code-audit-2026-03.md); top findings since remediated + +## Roadmap + +**Done** +- Single-node production gate **green** — install / stop / start / restart / reinstall / reboot-survive / uninstall, 5 consecutive full runs with zero failures on real hardware +- Quadlet migration validated (all backends as `user.slice` services on the canary node) +- Release signing ceremony completed — release-root key pinned, catalog and OTA manifests signed +- Reticulum third mesh transport (real-RF LoRa gates passed), Bitcoin Core/Knots multi-version switching, decentralized marketplace backend, public demo + +**In progress** +- Multinode pass: the same production gate across the whole test fleet ([`docs/multinode-testing-plan.md`](docs/multinode-testing-plan.md)) +- Quadlet default flip fleet-wide + container-flapping elimination +- 1.8.0 release hardening tail ([`docs/1.8.0-RELEASE-HARDENING-PLAN.md`](docs/1.8.0-RELEASE-HARDENING-PLAN.md)): OTA upgrade soak on real hardware, ISO/image hardening (per-device keys, no default creds, signed ISO) + +**Planned** +- Developer CLI (`archy app validate/render/install/test`) to open third-party app publishing +- External marketplace trust UX + publishing tooling ([`docs/marketplace-protocol.md`](docs/marketplace-protocol.md)) +- DHT/P2P distribution of releases and app images ([`docs/dht-distribution-design.md`](docs/dht-distribution-design.md)) +- P2P encrypted voice/video over Tor, dual-ecash (Fedimint + Cashu) phases, paid streaming, hardware signer support + +The live, priority-ordered task list is [`docs/UNIFIED-TASK-TRACKER.md`](docs/UNIFIED-TASK-TRACKER.md); the full narrative plan is [`docs/PRODUCTION-MASTER-PLAN.md`](docs/PRODUCTION-MASTER-PLAN.md). ## Quick Start ### Install from ISO -1. Download the ISO for your architecture (x86_64 or ARM64) +1. Build or download the ISO for your architecture (x86_64 or ARM64) — see [`image-recipe/`](image-recipe/) 2. Flash to USB drive with Balena Etcher or `dd` -3. Boot from USB on target hardware -4. Follow the automated installer -5. Access the web UI at `http://` -6. Set your password and start the onboarding wizard +3. Boot from USB on target hardware and follow the automated installer +4. Access the web UI at `http://` +5. Set your password and complete the onboarding wizard (seed backup, DID identity) ### Supported Hardware @@ -75,7 +103,7 @@ Bitcoin, Storage (FileBrowser, Immich, Nextcloud), Productivity (Penpot, Vaultwa | **x86_64** | Intel NUC, mini PCs, any 64-bit PC | 4GB RAM, 32GB storage | | **ARM64** | Raspberry Pi 5, ARM64 SBCs | 4GB RAM, 32GB storage | -**Recommended**: 8GB+ RAM, 1TB+ NVMe SSD (for full Bitcoin node) +**Recommended**: 8GB+ RAM, 1TB+ NVMe SSD (for a full Bitcoin node). Optional: an RNode-compatible LoRa radio for mesh networking. ## Development @@ -94,7 +122,15 @@ npm run type-check # TypeScript validation npm run build # Production build → web/dist/neode-ui/ ``` -### Deploy to Server +### Backend Development + +```bash +cd core # Rust workspace root (no Cargo.toml at repo root) +cargo build +cargo test +``` + +### Deploy to a Test Node ```bash ./scripts/deploy-to-target.sh --live # Deploy to primary dev server @@ -104,54 +140,57 @@ npm run build # Production build → web/dist/neode-ui/ ### Release (tarball-only) Releases ship as a backend binary and a frontend tarball referenced by -`releases/manifest.json`. Nodes OTA-update via `scripts/self-update.sh`. +`releases/manifest.json`, published to the self-hosted Gitea release server. ```bash ./scripts/create-release.sh 1.2.3 -git push gitea-local main --tags -git push gitea-vps2 main --tags +git push origin main --tags ``` -ISO builds are archived under `image-recipe/_archived/` and not part of the -release deliverable. - ## Architecture ``` Debian 13 (Trixie) - ├── Rootless Podman (30 containers, archy-net DNS) + ├── Rootless Podman — every app a systemd Quadlet unit under user.slice ├── Nginx (reverse proxy, security headers, rate limiting) - ├── Rust Backend (JSON-RPC API on 127.0.0.1:5678) - │ ├── core/archipelago/ — RPC endpoints, auth, identity, federation, mesh - │ ├── core/container/ — PodmanClient (REST API socket), manifests, health - │ ├── core/security/ — AppArmor, secrets, Cosign image verification - │ └── 6 more crates — models, helpers, js-engine, performance, etc. - ├── Vue 3 Frontend (Composition API + TypeScript strict + Pinia + Tailwind) + ├── Rust Backend (JSON-RPC API on 127.0.0.1:5678, ~380 RPC methods) + │ ├── core/archipelago/ — API, orchestrator + reconciler, mesh, identity, + │ │ federation, wallet, updates, marketplace + │ ├── core/container/ — Podman client, manifest schema, Quadlet compiler, + │ │ health monitor, signed app catalog + │ ├── core/security/ — AppArmor/seccomp policy, secrets manager + │ ├── core/openwrt/ — TollGate gateway provisioning (SSH/UCI) + │ └── core/performance/ — resource limits + ├── Vue 3 Frontend (Composition API + TypeScript strict + Pinia + Tailwind, PWA) + │ └── Three UI modes (Pro/Easy/Chat) + gamepad navigation + i18n + ├── Reticulum daemon (supervised Python/PyInstaller, one per LoRa radio) └── System Tor (hidden services, SOCKS5 proxy) ``` -~49,000 lines of Rust | ~47,000 lines of TypeScript/Vue | 78 shell scripts | 30 container apps +~117,000 lines of Rust | ~69,000 lines of TypeScript/Vue | 51 packaged apps | Android companion app ## Documentation | Doc | Purpose | |-----|---------| -| [Architecture](docs/architecture.md) | System design, codebase stats, data paths | -| [Architecture Review (HTML)](docs/architecture-review.html) | Interactive guide with diagrams and learning path | +| [Architecture](docs/architecture.md) | System design, crate map, data paths | | [Developer Guide](docs/developer-guide.md) | Dev setup, workflow, code conventions | -| [API Reference](docs/api-reference.md) | Complete RPC endpoint reference | +| [API Reference](docs/api-reference.md) | RPC endpoint reference | | [App Developer Guide](docs/app-developer-guide.md) | Building and publishing apps | +| [App Manifest Spec](docs/app-manifest-spec.md) | The `manifest.yml` schema | | [User Walkthrough](docs/user-walkthrough.md) | End-user installation and usage guide | | [Troubleshooting](docs/troubleshooting.md) | Diagnostic scenarios and solutions | | [Operations Runbook](docs/operations-runbook.md) | Ops commands and emergency recovery | -| [Security Audit](docs/security-code-audit-2026-03.md) | Penetration test findings | -| [Master Plan](docs/MASTER_PLAN.md) | Phased roadmap and task tracking | +| [Production Master Plan](docs/PRODUCTION-MASTER-PLAN.md) | North star and workstream narrative | +| [Unified Task Tracker](docs/UNIFIED-TASK-TRACKER.md) | Live, priority-ordered open items | +| [Test Gate](tests/lifecycle/TESTING.md) | Production lifecycle test gate (definition of done) | +| [Archive](docs/archive/) | Historical audits, session logs, shipped designs | ## Contributing 1. Fork the repository 2. Create a feature branch (`feature/description`) -3. Follow the coding standards in [CLAUDE.md](CLAUDE.md) +3. Follow the coding standards in [CONTRIBUTING.md](CONTRIBUTING.md) and [CLAUDE.md](CLAUDE.md) 4. Submit a pull request ## License @@ -160,4 +199,4 @@ Debian 13 (Trixie) ## Acknowledgments -Built with: [Rust](https://www.rust-lang.org/), [Vue.js](https://vuejs.org/), [Podman](https://podman.io/), [Bitcoin Core](https://bitcoin.org/), [LND](https://lightning.engineering/), [Debian](https://www.debian.org/) +Built with: [Rust](https://www.rust-lang.org/), [Vue.js](https://vuejs.org/), [Podman](https://podman.io/), [Bitcoin Core](https://bitcoin.org/), [LND](https://lightning.engineering/), [Reticulum](https://reticulum.network/), [Debian](https://www.debian.org/) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 00000000..fe202df2 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,97 @@ +# Archipelago Roadmap + +_Last updated: 2026-07-08. This is the public-facing summary. The live, +priority-ordered engineering list is [`UNIFIED-TASK-TRACKER.md`](UNIFIED-TASK-TRACKER.md); +the narrative plan behind it is [`PRODUCTION-MASTER-PLAN.md`](PRODUCTION-MASTER-PLAN.md)._ + +## North star + +A world-class, **developer-ready app platform**: every app manifest-driven, +manifests distributed via a **signed registry**, and third-party developers +publishing through an **external/decentralized marketplace** — all rootless, +secure, robust, and 100%-uptime-capable. + +Five pillars every app must satisfy: Quadlet-everywhere · level-triggered +reconciler · lifecycle-bulletproof (full test matrix, repeatedly green) · +data-driven (no host changes, no per-app binary code) · rootless + +security-first. + +## ✅ Shipped + +- **Single-node production gate GREEN** (2026-06-23) — the full destructive + lifecycle matrix (install / UI / stop / start / restart / reinstall / + reboot-survive / backend-restart-survive / uninstall) passed 5 consecutive + times with zero failures on real hardware. This was the first exit criterion. +- **Manifest-driven app platform** — 50+ apps as declarative manifests; all + multi-container stacks (BTCPay, Mempool, Immich, NetBird, IndeeHub) install + through the orchestrator; generated-secrets system replaces per-app secret code. +- **Rust orchestrator + level-triggered reconciler** — the bash-script era is + retired; drift self-heals on a 30-second loop. +- **Registry-distributed manifests** — the signed catalog embeds full manifests + per app; nodes verify against the pinned release-root key and overlay + catalog manifests over disk files (catalog wins). +- **Release signing ceremony** (2026-07-02) — release-root Ed25519 key pinned + in the binary; OTA release manifests and the app catalog are signed; + auto-apply refuses unsigned manifests. +- **1.8.0 hardening batches** — supply-chain signature verification, deepened + post-OTA health checks, async-executor and secret-handling fixes, browser + origin checks, dist shrink. +- **Reticulum third mesh transport** — RNS/LXMF over real LoRa hardware + (RNode), interop verified against Sideband; joins Meshtastic and MeshCore + behind one chat UI with X3DH + double-ratchet E2E, attachments, and mesh AI. +- **Bitcoin multi-version** — Core and Knots with per-app version pinning and + safe switching (fleet rollout pending below). +- **Decentralized marketplace backend** — Nostr NIP-78 discovery, DID-signed + manifests, federation-weighted trust scoring, Lightning purchase invoices. +- **Quadlet migration validated** — all backends run as `user.slice` Quadlet + services on the canary node (default flip pending below). +- **Public demo** — multi-visitor sandbox deployed. + +## 🔄 In progress + +- **Multinode pass** — run the same production gate across the whole test + fleet, plus cross-node federation/mesh suites + ([`multinode-testing-plan.md`](multinode-testing-plan.md)). This is the + current exit criterion. +- **Quadlet default flip** — flip the validated Quadlet path from opt-in to + default fleet-wide; eliminates the last container-flapping root cause. +- **Container-flapping elimination** — reconciler churn and failed-unit + self-healing gaps observed on live nodes. +- **Reticulum tail** — ship the daemon binary inside the release tarball; + final fleet redeploys. + +## ⏳ Release-blocking for 1.8.0 + +Tracked in detail in [`1.8.0-RELEASE-HARDENING-PLAN.md`](1.8.0-RELEASE-HARDENING-PLAN.md): + +- **OTA upgrade-from-previous-version soak** on real hardware — the top + untested release risk. +- **ISO/image hardening** — per-device TLS/SSH keys on first boot, remove + default credentials and SSH password auth, signed + checksummed ISO, + registries over HTTPS, unattended-upgrades and firewall defaults. +- **Bitcoin multi-version fleet OTA** — code done; rollout timing is a + deliberate hold. +- Version bump to `1.8.0-alpha` + tag once the pre-tag items close. + +## 🔭 Planned (post-1.8.0) + +- **Developer CLI** — `archy app validate / render / install / test`; the gate + for opening third-party app publishing. +- **External marketplace maturation** — publishing tooling, trust UX, and + reputation surfaces on top of the shipped backend + ([`marketplace-protocol.md`](marketplace-protocol.md)). +- **DHT/P2P distribution** — releases and app images over iroh-based swarm + ([`dht-distribution-design.md`](dht-distribution-design.md); feature-gated + skeleton exists). +- **P2P encrypted voice/video** over Tor between federated nodes. +- **Dual ecash** — Fedimint + Cashu phases 2–6, networking-sats + ([`dual-ecash-design.md`](dual-ecash-design.md)). +- **Paid streaming** — streaming ecash for content + ([`phase4-streaming-ecash-plan.md`](phase4-streaming-ecash-plan.md)). +- **Hardware signer** support ([`hardware-signer-design.md`](hardware-signer-design.md)). +- **Code health** — split god-modules, remove dead crates, route all + podman/systemctl calls through the wrapper. + +## Release pipeline + +Feature Testing (internal) → User Testing (controlled hardware) → Beta Live (public).