archy/docs/OPEN-SOURCE-READINESS-PLAN.md

# Open-Source Readiness Plan — Archipelago public launch

> Working plan, 2026-07-27. Source of truth for the pre-open-source cleanup.
> A second agent is working the same goal concurrently — before executing any phase,
> diff against `git log` since `7e8d3314` and skip/merge what's already done.
> (Session plan file: `~/.claude/plans/resilient-moseying-reef.md`.)

## Context

The repo goes public in a few days, targeting bitcoin/bitcoin-level polish. Three deep
exploration passes (docs/structure, code health, secrets sweep) found the repo is
fundamentally strong — README, `apps/` manifest examples, ADRs, the bats lifecycle gate,
1,104 Rust tests — but has hard blockers: **two live Anthropic API keys committed in
tracked files**, node passwords in 7 tracked files, no LICENSE (README links a 404),
5.5 GB `.git` (re-committed 27 MB APKs), ~290 hardcoded references to the private Gitea
registry `146.59.87.168:3000` that make every app image unpullable for outsiders, and
~28 internal AI-session/tracker docs mixed into `docs/`.

**Decisions made by the user:**
1. **Fresh-history publish** — new public repo with a clean initial commit; private repo keeps full history.
2. **Registry: domain + parameterize** — real domain in front of the existing registry; host configurable everywhere.
3. **Deep code cleanup** — orphan crates, dead_code lifts, clippy trims, legacy fallback deletion (sequenced, cut-line-friendly).
4. **Internal docs: sanitize and keep public** — scrub creds/IPs/hostnames but publish plans/trackers for transparency.

**Invariant throughout:** the single-node production gate (`tests/lifecycle/run-gate.sh`)
is GREEN and must stay green. Re-run after any orchestrator/lifecycle change (Phase E
especially). All cargo verification uses `--all-features` to match CI. Stage by explicit
path, never `git add -A` (shared tree).

## Current local pass status

This branch is replayed on top of `origin/main` as `public-prelaunch`.

Completed locally in this pass:

- Redacted the two tracked Anthropic API key literals from
  `scripts/setup-aiui-server.sh` and
  `image-recipe/_archived/build-auto-installer-iso.sh`.
- Removed `Android/app/debug.keystore` and `core/.env.production` from the
  source tree; copies were preserved in
  `~/Desktop/archipelago-sensitive-backup-2026-07-27/`.
- Reworked `scripts/audit-secrets.sh` to scan tracked source more aggressively
  and to catch non-example env files and credential file patterns.
- Reworked `scripts/validate-app-manifest.sh` so the current `app:` manifest
  schema can be audited without a Python `PyYAML` dependency.
- Updated root/community docs, CI, PR template, app developer notes, and
  container/deployment docs toward public contributor expectations.
- Fixed native FIPS activation fallback: nodes that have the packaged
  `fips.service` but not `archipelago-fips.service` now start the available
  unit instead of repeatedly failing activation against a missing unit. This
  now covers startup, supervisor self-heal, manual dashboard start/reconnect,
  and post-onboarding activation. The UI now labels the action as `Start`
  instead of making native FIPS look like an installable app.
- Fixed the FIPS app-port relay design so it binds relays to the node's FIPS
  ULA instead of wildcard `[::]`, avoiding collisions with Podman-published app
  ports such as FileBrowser `8083` and Botfights `9100`.
- Added `docs/nostr-git-source-hosting.md`, a NIP-34/ngit/GRASP source hosting
  plan using a Bitcoin Core-style maintainer model: public review and easy
  forks, with canonical merge rights held by a small signed maintainer set.

Verified locally:

- `./scripts/audit-secrets.sh` passes.
- Full `apps/*/manifest.yml` repository audit passes with warnings only.
- `bash -n` passes for the edited shell scripts.
- Targeted FIPS dashboard vitest passes.
- Targeted Rust tests for FIPS service unit detection and FIPS app relay
  address selection pass.

Verified on a Linux Archipelago verification node:

- Native FIPS was restored by starting the already-installed packaged
  `fips.service`; the daemon became active and joined the FIPS tree.
- Correct local lifecycle API endpoint is HTTP, not HTTPS
  (`ARCHY_HOST=127.0.0.1 ARCHY_SCHEME=http`).
- Read-only lifecycle run progressed past login and confirmed required
  containers, Bitcoin RPC, ElectrumX TCP, and manifest port-drift checks, but
  did not complete cleanly: `botfights` and `filebrowser` remained in
  `restarting` longer than the matrix window, and the LND `lncli getinfo`
  probe hung. Do not run the destructive gate until those live-node issues are
  understood.
- After the node updated to `1.7.116-alpha`, `botfights`, `filebrowser`, and
  `lnd` were active/running and ports `8083`/`9100` were held by Podman's
  `rootlessport` as expected. The packaged `fips.service` remained installed
  and enabled but inactive, so the native FIPS service fallback should still
  ship before the public launch.

Still required before public publish:

- Rotate/revoke compromised credentials listed in Phase 0.
- Finish Phase 1 password/node/token sanitization beyond the two API keys.
- Publish from fresh history after the sanitized tree is final.
- Run full Rust, frontend, Android, and lifecycle gate verification.
- Resolve the live-node lifecycle blockers above, then rerun the read-only
  suite followed by the destructive gate only on an approved verification node.
- Decide the canonical Archipelago maintainer npub and merge-maintainer npub
  list before publishing the Nostr Git source-hosting workflow.

---

## Phase 0 — Credential rotation (immediate, independent of the repo)

Treat all of these as already compromised; rotate even though we're doing fresh-history:

- **Anthropic API key #1**: `image-recipe/_archived/build-auto-installer-iso.sh:2837` (the "intentional alpha" ISO key). Revoke + reissue; move the live key OUT of source into a build-time secret/env (`ISO_ANTHROPIC_API_KEY`), keep the alpha-baking behavior if desired but never the literal in git.
- **Anthropic API key #2**: `scripts/setup-aiui-server.sh:28` — a *different* live key, not covered by the documented alpha exception. Revoke; parameterize the script.
- **The shared node SSH/sudo/UI password** (two variants) — in 7 tracked files + 24+ commits. Rotate fleet-wide (user task).
- **Gitea `ai` account password + 2 Gitea tokens** — embedded in `.git/config` remote URLs (not tracked, but leaks in any directory copy/tarball). Rotate; switch remotes to credential-helper storage instead of URL-embedded creds.

## Phase 1 — Secrets & sanitization of tracked files

1. Strip the password/credential lines from the 7 files:
   `docs/PRODUCTION-MASTER-PLAN.md` (lines ~428–429, 454–457, 483, 521–528, 886 — the fleet cred table),
   `docs/archive/SESSION-1.8.0-OTA-PROGRESS.md`, `docs/archive/HANDOVER-2026-07-02-iso-feedback.md`,
   `docs/bitcoin-version-bulletproof-rollout.md`, `tests/production-quality/TRACKER.md`,
   `tests/multinode/meshtastic.sh:26`, `neode-ui/test-openwrt.mjs:4` (→ env var).
2. `.gitea/workflows/post-install-tests.yml` — remove `sshpass -p '…'` + default target IP; use secrets/vars.
3. Sanitize infra identifiers repo-wide (in the *sanitize-and-keep* docs and scripts):
   replace Tailscale IPs (17 unique, 14 files), LAN IPs (`192.168.1.x`, 93 files), hostnames
   (`tx1138`, `shorty-s`, `archy-x250`, `archy-dev-pa`) with placeholders like `<node-a>` /
   `NODE_IP`. Key script targets: `scripts/deploy-config-defaults.sh`, `scripts/deploy-tailscale.sh`,
   `docs/operations-runbook.md` (opens with real node IPs), `docs/developer-guide.md`, `docs/api-reference.md`, `docs/hotfix-process.md`.
4. Fix the audit tool that let this happen: `scripts/audit-secrets.sh:28` — remove `\.md$` and
   bare `test` from ALLOW_PATTERNS; add `sk-ant-` and password-table patterns; scan all
   tracked files not just `*.env`. Run it clean as a Phase-1 exit check.
5. `.gitignore` additions: `.claude/`, `*.key`, `*.pem`, `id_rsa*`, `*.sqlite`, `*.db`
   (`.claude/settings.local.json` with creds is currently only ignored by a machine-global rule).
6. Product-security note to raise (not fix now): `password123` is a shipped default (auth.rs, en.json, user-walkthrough) — file a public issue for forced first-run password change if not already enforced.

## Phase 2 — Repo restructure: deletions, binaries, layout

Delete (each its own commit):
- `loop/` (AI overnight harness w/ node SSH lines), `.agents/`, `.codex`, `.githooks/pre-push`
  (the hook that re-commits the 27 MB APK — root cause of the 5.5 GB history).
- `indeedhub/` submodule + `.gitmodules` entry (points at private HTTP Gitea, breaks `--recursive`
  clones); `indeedhub-demo/` (single Dockerfile — merge or drop).
- `RELEASE-NOTES-v1.0.0.md` (superseded by CHANGELOG), `neode-ui/docs/GAMEPAD-NAV-MAP.md` (duplicate of `docs/GAMEPAD-NAV.md`).
- Stray generated HTML: `docs/container-architecture.html` (311 KB), `docs/archive/architecture-review.html`, `docs/archive/lora-functionality.html`.
- `Android/local.properties` from tracking (local absolute path); remove `Android/app/debug.keystore` (standard practice).

Move out of git (→ release assets on the Releases page, referenced by URL):
- `neode-ui/public/packages/archipelago-companion.apk` (27 MB), `wireguard.apk` (17 MB), `atob.s9pk` (23 MB).
- `Android/archipelago-0.3.0-debug.apk.zip` (16 MB, stale).
- `demo/content/music/*` + heavy `demo/aiui/assets` (~261 MB, third-party/unclear-licence media — MUST not ship publicly regardless of size).
- `neode-ui/dev-dist/` (generated Workbox output) → gitignore.

Rename/fix the naming lie: `image-recipe/_archived/` contains the *production* ISO builder
(`build-auto-installer-iso.sh`, referenced by `.gitea/workflows/build-iso.yml`). Move live
files up into `image-recipe/`, delete the genuinely archived rest.

## Phase 3 — Registry domain + parameterization (functional blocker)

Infra (user assists: DNS + TLS):
- Put a domain (e.g. `registry.archipelago-os.org` / `git.archipelago-os.org`) with HTTPS in
  front of the existing Gitea on vps2. OTA download URLs move from plain HTTP to HTTPS.

Repo changes:
- Introduce a single source of truth for the registry host (e.g. `REGISTRY_HOST` in
  `scripts/lib/` + a default in the orchestrator config). Replace `146.59.87.168:3000` in:
  all 56 `apps/*/manifest.yml`, `app-catalog/catalog.json`, `releases/manifest.json`,
  `release-manifest.json`, the 11 scripts (`self-update.sh`, `create-release.sh`,
  `generate-app-catalog.sh`, `validate-app-manifest.sh`, `first-boot-containers.sh`, …),
  both `demo-images.yml` workflows, `demo-deploy/.env.example`, and the Android sources
  (`FipsPreferences.kt`, `PartyScreen.kt`).
- Because the catalog is signed: regenerate + re-sign + republish the app catalog after the
  manifest host change (catalog-overlay supremacy — disk edits don't apply otherwise).
  Signing needs the user's mnemonic → schedule one ceremony after manifests are final.
- Verify: fresh machine with no LAN/tailnet access can `podman pull` one app image via the
  domain and the gate node still installs apps after the re-signed catalog lands.

## Phase 4 — Documentation overhaul

### 4a. Community/legal files (missing today)
- `LICENSE` — MIT (matches existing README badge). Add `[workspace.package] license` +
  `license.workspace = true` in the 5 member Cargo.tomls (also see Phase A4).
- `SECURITY.md` — disclosure address, PGP key, supported-versions; cite the March 2026 audit (`docs/archive/security-code-audit-2026-03.md`).
- `CODE_OF_CONDUCT.md` — Contributor Covenant (CONTRIBUTING.md already links to it, 404 today).
- `CONTRIBUTING.md` edits: Gitea→GitHub fork flow, remove private deploy instructions, absorb
  the public-worthy CLAUDE.md invariants (rootless podman, manifest-driven, secrets model,
  non-destructive migrations), versioning policy note for the `-alpha` scheme.
- `CLAUDE.md` — rewrite: keep invariants/build-verify (public-worthy), remove status banner,
  node numbers, `gitea-ai` push mechanics, MEMORY references (those move to private notes).

### 4b. New developer docs (the three real gaps for app developers)
1. **`docs/quadlet-compilation.md`** — how a manifest becomes a Quadlet/systemd unit: naming,
   `systemctl --user` lifecycle, where units land, how to inspect/debug one. (Source:
   `core/archipelago/src/container/quadlet*.rs`, prod_orchestrator.)
2. **`docs/container-lifecycle.md`** — the 30 s level-triggered reconciler, install/adopt/
   restart/uninstall state machine, health checks, crash recovery. (Replaces the plan-shaped
   `docs/bulletproof-containers.md` as the current description; salvage its content.)
3. **`docs/secrets.md`** — `generated_secrets` declaration → materialisation by
   `container::secrets` (0600, rootless) → injection; what developers must never do.
- Also: make every example in `docs/app-developer-guide.md` + `apps/*/manifest.yml` copy-paste
  work against the new public registry host; add an end-to-end "write your first app" walkthrough
  that a stranger can follow with only the public repo + an Archipelago node.

### 4c. Sanitize-and-keep internal docs (user's transparency choice)
- Keep, after Phase-1 scrubbing: `docs/PRODUCTION-MASTER-PLAN.md`, `docs/UNIFIED-TASK-TRACKER.md`,
  `docs/1.8.0-RELEASE-HARDENING-PLAN.md`, `docs/RETICULUM-TRANSPORT-PROGRESS.md`, HANDOFF-*, test
  plans, `docs/archive/*` — but **move all session/handoff/tracker material under
  `docs/history/`** (extending the existing honest `docs/archive/README.md` pattern) so the
  top-level `docs/` reads as current reference only. Add a banner to each: "historical working
  document, sanitized; not maintained."
- Remove dangling agent-memory references in tracked docs (`docs/bulletproof-containers.md`,
  `docs/RETICULUM-TRANSPORT-PROGRESS.md`, `docs/registry-manifest-design.md`,
  `docs/bitcoin-multi-version-design.md` progress block).
- De-status the 14 design docs (strip "Status/RESUME POINT" headers into a one-line status
  field; e.g. `docs/APP-PACKAGING-MIGRATION-PLAN.md` → public app-platform design doc).
- Extract North-Star narrative from PRODUCTION-MASTER-PLAN into `docs/ROADMAP.md`; extract
  the "run the gate ON the node" philosophy from `docs/multinode-testing-plan.md` into
  `tests/lifecycle/TESTING.md`.
- Add `docs/README.md` index (bitcoin/bitcoin `doc/` style): Getting started / Architecture /
  App development / Operations / Design docs (ADRs) / History.
- README fixes: LICENSE link becomes real, Documentation table repointed at the reorganized
  docs, remove "Deploy to a Test Node" private-LAN section, point Contributing at
  CONTRIBUTING.md only.

## Phase 5 — Deep code cleanup (ordered zero-risk → highest-risk; cut-line after any commit)

### A. Zero-risk deletions & metadata (S each, own commits)
- **A1** Delete orphan non-compiling StartOS crates: `core/models`, `core/helpers`,
  `core/js-engine` (incl. 2 committed `JS_SNAPSHOT.*.bin`), `core/container-init` (~4,100 LOC,
  zero references). Verify: `cargo build --workspace && cargo test --all-features`.
- **A2** Delete unreferenced Vue components: `neode-ui/src/components/{AppSwitcher,EmptyState,SkeletonCard}.vue`. Verify: `npm run type-check && npm run build`.
- **A3** Fix `.gitignore` lockfile lines (7: `Cargo.lock`, 15: `package-lock.json`) — lockfiles are intentionally tracked; the rules are misleading and swallow future lockfiles.
- **A4** LICENSE + Cargo license fields (see 4a). Verify with `cargo metadata`.
- **A5** `core/rust-toolchain.toml` pinning `1.95.0`; align `.github/workflows/ci.yml` (remove explicit `toolchain: stable` input so the file wins). Upgrades become deliberate PRs.
- **A6** `core/rustfmt.toml` codifying **defaults only** (`edition = "2021"` + comment) — do NOT add style options days before launch (whole-tree reformat churn). Verify `cargo fmt --all -- --check` yields no diff.

### B. CI guards (zero runtime risk)
- **B1** Enable vitest in CI: run `cd neode-ui && npm run test` locally; fix trivial failures, `.skip`+issue flaky ones; add step to the frontend job. Playwright → tracked issue only (needs browsers + mock backend orchestration).
- **B2** Raw podman/systemctl **ratchet, not migration**: the 132 raw `Command::new("podman"/"systemctl")` sites use subcommands the `core/container/src/podman_client.rs` wrapper doesn't expose (network/inspect/ps/port), 43 sites are in gate-critical `install.rs`, and the prod path intentionally uses Quadlet+systemctl. Add `scripts/ci/raw-podman-ratchet.sh` (count vs committed baseline, fail on increase) as a CI step + tracked issue for wrapper API design.

### C. Clippy suppression trim (`core/archipelago/src/main.rs:8-18`, per-lint commits)
- Remove cheaply: `assertions_on_constants`, `drop_non_drop`, `wildcard_in_or_patterns`, `doc_lazy_continuation`, `enum_variant_names` (targeted allows on serde enums — never rename wire variants).
- Own careful commit: `unused_io_amount` — a **correctness** lint; fix sites with `read_exact`/`write_all` or documented targeted allows (`mesh/serial.rs:456,496` has raw partial reads; serial framing may be intentional). Full test suite + gate after.
- Keep crate-wide with justifying comment: `too_many_arguments`, `type_complexity`; attempt `ptr_arg` (`&Vec<T>`→`&[T]`, mechanical) if time allows — first to cut.
- Verify each: `cargo clippy --all-targets --all-features -- -D warnings && cargo test --all-features`.

### D. dead_code lift — Tiers 1–2 pre-launch, Tier 3 → commented allows + issues
Per-module procedure (one file per commit): remove `#![allow(dead_code)]` → `cargo check
--all-targets --all-features` → triage each warning: (a) genuinely dead → delete;
(b) future-feature/protocol-mandated → targeted `#[allow(dead_code)] // TODO(#NNN): …`;
(c) missing wiring → keep + targeted allow + issue (don't fix wiring in this workstream) →
clippy `-D warnings` + tests → commit.
- **Tier 1 (small/leaf, S each):** `swarm/seed_advert.rs`, `transport/{mesh_transport,lan,chunking,delta}.rs`, `mesh/{crypto,alerts,types,outbox}.rs`, `streaming/mod.rs`, `wallet/mod.rs`.
- **Tier 2 (M each):** `fips/{mod,iface,dial}.rs` (41 external refs → little residual deadness), `mesh/{x3dh,ratchet,steganography,message_types}.rs` — for crypto files bias to (b) with roadmap comments (unused crypto attracts auditor noise; every kept item needs its why).
- **Tier 3 (defer, riskiest):** `mesh/{mod,reticulum,protocol,serial,bitcoin_relay}.rs`, `transport/mod.rs` — change each blanket allow to `#![allow(dead_code)] // Hardware-mesh surface partially wired; triage tracked in #NNN`.
- Optional S/M win: move `prod_orchestrator.rs`'s 5,034-line `#[cfg(test)]` module to a sibling file via `#[path]` (pure move, halves the 6,291-line file).

### E. stacks.rs legacy fallbacks (highest risk — LAST, evidence-gated)
Legacy installers for immich/btcpay/mempool/indeedhub (`core/archipelago/src/api/rpc/package/stacks.rs:838/1047/1267/1498`, ~1,000 LOC with hardcoded registry IPs) fire only on "unknown app_id, zero members installed", logging `INSTALL ORCH SKIP` (stacks.rs:673). Netbird already uses the hard-error replacement (stacks.rs:1898-1920).
1. Run the full gate on the node; grep install logs for `INSTALL ORCH SKIP`.
2. Zero SKIPs → replace each legacy body with the netbird-style hard error (keep orchestrator call + `adopt_stack_if_exists`; satisfies migrations-never-destroy-data). Re-run gate; any red → revert + issue.
3. Any SKIP → don't delete; issue: "deploy manifests fleet-wide, then delete legacy installers".

### Explicitly deferred → public tracked issues at launch
PodmanClient API extension + call-site migration; god-module splits (`install.rs`, `update.rs`, `mesh/mod.rs`); Playwright in CI; Tier-3 dead_code triage; `password123` default hardening.

## Phase 6 — Fresh-history publish

1. Freeze: all phases merged on internal `main`, gate green, catalog re-signed.
2. Build the public tree: `git archive`-style export of HEAD (never copy `.git/` — it holds
   credentialed remotes) → new repo, single initial commit ("Initial public release, vX.Y.Z"),
   optionally preserving CHANGELOG.md as the human-readable history.
3. Pre-publish gate on the export: `scripts/audit-secrets.sh` (fixed version) clean; grep-zero for
   `sk-ant-`, rotated-password strings, `146.59.87.168`, tailnet `100.` IPs, `192.168.1.`,
   internal hostnames; `du -sh .git` sanity (< ~100 MB); fresh `git clone` + `cd core && cargo build`
   + `cd neode-ui && npm ci && npm run build` on a clean machine/container; one app image pull
   from the public domain.
4. Publish to GitHub; enable issue templates (already present in `.github/`); file the deferred-work
   issues (from Phase 5's issue list) as the initial public issue set — honest and gives contributors entry points.
5. Internal repo remains the private full-history remote; decide sync direction post-launch
   (recommend: public repo becomes canonical, private keeps only ops/infra notes).

## Verification (end-to-end)

- `tests/lifecycle/run-gate.sh` green on the node after Phases 3 + 5E (and after any lifecycle-touching commit).
- CI green on every phase commit: `cargo fmt --check`, `clippy -D warnings`, `cargo test --all-features`, frontend type-check + build + (new) vitest.
- Phase-6 clean-machine clone/build/pull test is the final acceptance test — it simulates the first outside developer.
- Docs acceptance: a reader following `docs/app-developer-guide.md` + the new quadlet/lifecycle/secrets docs can build and install an app manifest without any private infra.

## Sequencing / cut-line

Order: 0 → 1 → 2 → (3 ∥ 4) → 5 (A→E) → 6. Phases 0–2 are non-negotiable security; Phase 3 is the
functional blocker; Phase 4 is the developer-experience payload; Phase 5 can be cut after any
commit (minimum viable: A1–A6, B1–B2, unused_io_amount fix); Phase 6 last. If the timeline
compresses, Tier-2 dead_code and Phase E move to public issues — everything else holds.