archy/.planning/codebase/STRUCTURE.md
2026-07-29 10:25:54 -04:00

435 lines
24 KiB
Markdown

# Codebase Structure
**Analysis Date:** 2026-07-29
## Directory Layout
```
archipelago-repo/
├── core/ # Rust workspace root (Cargo.toml at workspace level)
│ ├── archipelago/ # Main daemon binary (backend)
│ │ ├── src/
│ │ │ ├── main.rs # Entry point, startup, background tasks
│ │ │ ├── server.rs # HTTP server (Hyper), listener, connection multiplexing
│ │ │ ├── state.rs # StateManager + data_model.rs (central state)
│ │ │ ├── auth.rs # User auth, password hashing, session management
│ │ │ ├── identity.rs # Node identity, Ed25519 keys, Tor address
│ │ │ ├── config.rs # Config loading, data directory setup
│ │ │ ├── api/ # HTTP API layer
│ │ │ │ ├── handler/ # HTTP request dispatch, WebSocket, content proxy
│ │ │ │ └── rpc/ # JSON-RPC 2.0 methods (~40 domain modules)
│ │ │ │ ├── auth.rs # auth.login, auth.setup, etc.
│ │ │ │ ├── container.rs # container.install, .list, .start, etc.
│ │ │ │ ├── bitcoin.rs # bitcoin.status, bitcoin.send, etc.
│ │ │ │ ├── mesh.rs # mesh.* (peer discovery, LoRa, federation)
│ │ │ │ ├── wallet.rs # wallet.* (lightning, Bitcoin)
│ │ │ │ └── [20+ other domains]
│ │ │ ├── container/ # Container orchestration (Podman)
│ │ │ │ ├── prod_orchestrator.rs # Main Podman lifecycle (>250KB)
│ │ │ │ ├── boot_reconciler.rs # Periodic manifest sync loop
│ │ │ │ ├── docker_packages.rs # Image registry, image verification
│ │ │ │ ├── quadlet.rs # Systemd Quadlet generation
│ │ │ │ ├── secrets.rs # Secret injection (0600 files)
│ │ │ │ ├── lnd.rs # Lightning Network Daemon container setup
│ │ │ │ ├── app_catalog.rs # Signed app catalog, manifest overlay
│ │ │ │ └── [data managers, registry, image policy]
│ │ │ ├── bootstrap.rs # Post-startup tasks (systemd units, audio stack, gamepad)
│ │ │ ├── crash_recovery.rs # Container recovery after crash, PID marker
│ │ │ ├── health_monitor.rs # Periodic app health checks, restart
│ │ │ ├── mesh.rs # Mesh P2P listener, sender, LoRa radio control
│ │ │ ├── federation.rs # Federation (DNS-SD, HTTP API)
│ │ │ ├── fips/ # FIPS anchor (Tor bridge to peer)
│ │ │ ├── bitcoin_rpc.rs # Bitcoin Core RPC client calls
│ │ │ ├── bitcoin_status.rs # Bitcoin sync status polling
│ │ │ ├── content_server.rs # Content-addressed blob server (CAP tokens)
│ │ │ ├── blobs.rs # BlobStore (hash→file mapping, encryption)
│ │ │ ├── wallet.rs # Lightning + Bitcoin wallet logic
│ │ │ ├── identity_manager.rs # Seed derivation, key rotation
│ │ │ ├── marketplace.rs # Marketplace transaction logic
│ │ │ ├── transport.rs # Transport selection (Mesh/FIPS/Tor routing)
│ │ │ ├── update.rs # OTA update apply, verification, rollback
│ │ │ ├── session.rs # Session store (SQLite or in-memory)
│ │ │ ├── rate_limit.rs # Rate limiter (per-IP, per-endpoint, per-user)
│ │ │ ├── monitoring.rs # Metrics collection (app count, memory, etc.)
│ │ │ ├── data_model.rs # State struct tree (serde Serialize/Deserialize)
│ │ │ ├── constants.rs # Global constants (version, defaults)
│ │ │ ├── peer*.rs, webhook*.rs, nostr*.rs, vpn.rs, etc.
│ │ │ └── seed.rs # Seed storage, backup QR generation
│ │ ├── Cargo.toml # Dependencies
│ │ └── tests/ # Unit/integration tests
│ │
│ ├── container/ # Container management library (OCI types, Podman)
│ │ ├── src/
│ │ │ ├── manifest.rs # Manifest struct (YAML parsing)
│ │ │ ├── runtime.rs # Podman CLI calls (create, start, stop, logs)
│ │ │ ├── podman_client.rs # Podman socket API client
│ │ │ ├── image_verify.rs # Image signature verification (Cosign)
│ │ │ └── port_manager.rs # Port allocation, conflict detection
│ │ └── Cargo.toml
│ │
│ ├── security/ # Secrets management library
│ │ ├── src/
│ │ │ ├── secrets_manager.rs # Secret encryption/decryption (ChaCha20)
│ │ │ └── vault.rs # Vault storage, rotation
│ │ └── Cargo.toml
│ │
│ ├── performance/ # Performance monitoring library
│ ├── openwrt/ # OpenWrt device integration
│ ├── Cargo.toml # Workspace manifest (members: archipelago, container, security, etc.)
│ └── Cargo.lock # Locked dependency versions
├── neode-ui/ # Frontend (Vue 3, TypeScript)
│ ├── src/
│ │ ├── main.ts # Vue app entry point, Router setup, WebSocket init
│ │ ├── App.vue # Root component (layout, nav)
│ │ ├── router/
│ │ │ └── index.ts # Vue Router config (routes, guards)
│ │ ├── views/ # Page-level components (one per route)
│ │ │ ├── Home.vue # Dashboard
│ │ │ ├── Apps.vue # App browser + installer
│ │ │ ├── AppDetails.vue # Single app detail + logs
│ │ │ ├── AppSession.vue # Iframe container for app content
│ │ │ ├── Cloud.vue # File browser (WebDAV/DWN)
│ │ │ ├── Mesh.vue # Mesh map, contacts, messages
│ │ │ ├── Wallet.vue # Lightning + Bitcoin addresses/sends
│ │ │ ├── Marketplace.vue # Paid apps, content marketplace
│ │ │ ├── Server.vue # Node status, settings, restart
│ │ │ ├── Federation.vue # Federation peers, federation apps
│ │ │ ├── Onboarding*/ # Multi-step setup flow
│ │ │ └── [15+ other pages]
│ │ ├── components/ # Reusable UI components
│ │ │ ├── AppCard.vue # App listing card
│ │ │ ├── AppInstaller.vue # Install modal
│ │ │ ├── Modal.vue # Generic modal (teleported)
│ │ │ ├── Toast.vue # Notification toast
│ │ │ ├── MeshGraph.vue # Mesh topology graph (D3)
│ │ │ ├── MapView.vue # Mesh map (Leaflet)
│ │ │ ├── QrScanner.vue # QR code input
│ │ │ └── [30+ other components]
│ │ ├── composables/ # Logic hooks (Vue composition API)
│ │ │ ├── useAuth.ts # Login/logout logic
│ │ │ ├── useAppStore.ts # Access app Pinia store
│ │ │ ├── useRpc.ts # Make RPC calls
│ │ │ ├── useWebSocket.ts # WebSocket connection management
│ │ │ ├── useOnboarding.ts # Onboarding flow state
│ │ │ ├── useControllerNav.ts # Gamepad controller navigation
│ │ │ └── [20+ other composables]
│ │ ├── stores/ # Pinia state management (reactive stores)
│ │ │ ├── appStore.ts # Apps list, install state
│ │ │ ├── walletStore.ts # Lightning/Bitcoin addresses, balance
│ │ │ ├── meshStore.ts # Mesh peers, messages
│ │ │ ├── settingsStore.ts # User settings, theme, language
│ │ │ ├── userStore.ts # Current user identity
│ │ │ └── [other stores]
│ │ ├── api/
│ │ │ └── rpc-client.ts # RPC client library (request, WebSocket, reconnect)
│ │ ├── services/ # Business logic (not Vue-dependent)
│ │ │ ├── qrScanner.ts # QR scanner initialization
│ │ │ └── [other services]
│ │ ├── utils/ # Utility functions
│ │ │ ├── format.ts # Date, number, currency formatting
│ │ │ ├── validate.ts # Input validation (emails, addresses)
│ │ │ ├── crypto.ts # Client-side crypto (BIP39, etc.)
│ │ │ └── [helpers]
│ │ ├── types/ # TypeScript type definitions
│ │ │ ├── index.ts # Export all types
│ │ │ └── [domain-specific types]
│ │ ├── i18n.ts # Internationalization config
│ │ ├── locales/ # Translation files
│ │ │ ├── en.json # English
│ │ │ ├── es.json # Spanish
│ │ │ └── [other languages]
│ │ ├── assets/ # Static assets
│ │ │ └── icon/ # App icons, favicons
│ │ ├── style.css # Global CSS (Tailwind + custom)
│ │ ├── data/ # Static data (country lists, etc.)
│ │ └── e2e/ # Playwright E2E tests
│ │ ├── intro-experience.spec.ts
│ │ └── app-launch.spec.ts
│ │
│ ├── public/ # Static web root
│ │ ├── index.html # HTML entry point
│ │ ├── favicon.ico # Browser tab icon
│ │ ├── manifest.json # PWA manifest
│ │ └── catalog.json # App catalog (copied from app-catalog/catalog.json)
│ │
│ ├── package.json # Frontend dependencies + build scripts
│ ├── tsconfig.json # TypeScript config
│ ├── vite.config.ts # Vite build config
│ ├── vitest.config.ts # Vitest test runner config
│ ├── tailwind.config.js # Tailwind CSS config
│ ├── mock-backend.js # Dev mock server (for `npm run dev:mock`)
│ └── [other build configs]
├── apps/ # Containerized applications (app manifests + build scripts)
│ ├── bitcoin-core/ # Bitcoin Core container
│ │ ├── manifest.yml # Archipelago manifest (interface, ports, secrets, health)
│ │ ├── Dockerfile # OCI image definition
│ │ ├── bitcoin.conf.template # Config template (secrets injected at runtime)
│ │ └── start.sh # Container entrypoint
│ │
│ ├── lightning-stack/ # Lightning Network stack (LND)
│ ├── lnd/ # LND daemon
│ ├── immich/ # Photo backup app
│ ├── nextcloud/ # Cloud storage
│ ├── electrumx/ # Bitcoin block explorer index
│ ├── router/ # Mesh router app
│ ├── pine/ # Voice assistant (whisper + piper + nginx)
│ ├── vaultwarden/ # Password manager
│ ├── [40+ other apps]
│ ├── QUICKSTART.md # App development guide
│ ├── PORTS.md # Port allocation reference
│ └── build.sh # App build script (all apps)
├── web/ # Built frontend output
│ └── dist/
│ └── neode-ui/ # `npm run build` output (served by nginx)
│ ├── index.html
│ ├── [JS bundles]
│ └── [static assets]
├── tests/ # Test suite
│ ├── lifecycle/
│ │ ├── run-gate.sh # Single-node production gate (5 iterations)
│ │ └── TESTING.md # Test plan documentation
│ ├── e2e/ # End-to-end tests (Playwright)
│ └── [other test directories]
├── docs/ # Documentation (user & developer guides)
│ ├── PRODUCTION-MASTER-PLAN.md # North star: manifest-driven, registry-based, decentralized
│ ├── UNIFIED-TASK-TRACKER.md # Open tasks (fastest-first)
│ ├── APP-PACKAGING-MIGRATION-PLAN.md
│ ├── registry-manifest-design.md
│ ├── multinode-testing-plan.md
│ ├── release-workflow.md # OTA + ISO release process
│ ├── app-development.md # Guide for app developers
│ ├── api-rpc-reference.md # JSON-RPC 2.0 method documentation
│ └── [20+ other docs]
├── image-recipe/ # ISO/image build scripts
│ ├── build-debian-iso.sh # Builds bootable Debian ISO
│ ├── include/ # Root filesystem overlays
│ │ └── opt/archipelago/ # Pre-baked config, scripts, systemd units
│ └── [other image components]
├── scripts/ # Utility scripts
│ ├── deploy-to-target.sh # Sideload binary to test node (Tailscale SSH + rsync)
│ ├── resilience/ # Resilience test scripts
│ └── [other scripts]
├── demo/ # Demo deployment (pre-configured node)
│ ├── demo-deploy.yml # Docker Compose for vps2 demo
│ └── [demo-specific scripts]
├── docker/ # Docker/Podman config
│ └── [docker-compose fragments, Dockerfiles]
├── .planning/ # Codebase analysis documents (this is you)
│ └── codebase/
│ ├── ARCHITECTURE.md
│ ├── STRUCTURE.md
│ ├── CONVENTIONS.md
│ ├── TESTING.md
│ ├── STACK.md
│ ├── INTEGRATIONS.md
│ └── CONCERNS.md
├── .claude/ # Claude Code configuration
│ └── skills/ # GSD skills (if any project-specific)
├── .github/ # GitHub Actions CI/CD
├── .gitea/ # Gitea CI/CD (local Gitea runner)
├── .git/ # Git repository
├── .gitignore # Git ignore patterns
├── CLAUDE.md # Project instructions (commit rules, invariants, testing)
├── Cargo.lock # Locked Rust dependency versions
├── CHANGELOG.md # Release notes
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
└── README.md # Project overview
```
## Directory Purposes
**core/**
- Purpose: Rust backend source and dependencies
- Contains: Main daemon binary, container orchestration, RPC handlers, business logic
- Key files: `archipelago/src/main.rs` (entry point), `archipelago/Cargo.toml` (deps)
**neode-ui/**
- Purpose: Vue 3 frontend SPA source
- Contains: Views, components, stores, translations, tests, build config
- Key files: `src/main.ts` (entry point), `vite.config.ts` (build), `package.json` (deps)
**apps/**
- Purpose: Containerized application definitions
- Contains: Manifests (YAML), Dockerfiles, configs for ~50 apps (Bitcoin, LND, Immich, etc.)
- Key files: `*/manifest.yml` (app definition), `*/Dockerfile` (image build)
**web/dist/neode-ui/**
- Purpose: Production frontend output (built by `npm run build`)
- Contains: Bundled JS, HTML, static assets
- Key files: `index.html` (entry), JS chunks (hashed filenames)
- Note: Served by nginx at `/` on node
**tests/**
- Purpose: Test suite (unit, integration, E2E)
- Contains: Lifecycle gate (production exit criterion), E2E specs, test utilities
- Key files: `lifecycle/run-gate.sh` (the production gate), `e2e/*.spec.ts` (user flows)
**docs/**
- Purpose: User and developer documentation
- Contains: Architecture, design decisions, release process, task tracking
- Key files: `PRODUCTION-MASTER-PLAN.md` (north star), `UNIFIED-TASK-TRACKER.md` (open tasks)
**image-recipe/**
- Purpose: ISO/image build scripts
- Contains: Debian ISO recipe, root filesystem overlays, bootloader config
- Key files: `build-debian-iso.sh` (main build), `include/opt/archipelago/` (pre-baked system config)
**scripts/**
- Purpose: Utility and deployment scripts
- Contains: Sideload/deploy to test nodes, resilience testing, CI glue
- Key files: `deploy-to-target.sh` (ship binary to remote node)
## Key File Locations
**Entry Points:**
- Backend daemon: `core/archipelago/src/main.rs` (spawns HTTP server, tasks, orchestrator)
- Frontend app: `neode-ui/src/main.ts` (Vue app, router, WebSocket init)
- App manifest: `apps/*/manifest.yml` (Archipelago-specific; controls container, secrets, UI)
- Production gate: `tests/lifecycle/run-gate.sh` (exit criterion for releases)
**Configuration:**
- Backend config: `core/archipelago/src/config.rs` (data dir, bind port, dev mode flag)
- Frontend config: `neode-ui/vite.config.ts` (build settings, env vars)
- App registries: `/var/lib/archipelago/registries.json` (user-configured Gitea mirrors)
- Secrets location: `/var/lib/archipelago/secrets/` (encrypted ChaCha20 files)
**Core Logic:**
- Container orchestration: `core/archipelago/src/container/prod_orchestrator.rs` (300KB+ main logic)
- RPC dispatch: `core/archipelago/src/api/rpc/mod.rs` (method routing, ~200 RPCs)
- State management: `core/archipelago/src/state.rs` (StateManager) + `core/archipelago/src/data_model.rs` (struct def)
- Frontend stores: `neode-ui/src/stores/` (Pinia stores, reactive state)
**Testing:**
- Rust unit tests: Inline in `core/` modules (use `#[test]` and `#[tokio::test]`)
- Vitest unit tests: `neode-ui/src/**/__tests__/*.test.ts`
- E2E tests: `neode-ui/e2e/*.spec.ts` (Playwright)
- Integration tests: `tests/` (shell scripts, node command testing)
## Naming Conventions
**Files:**
- Rust modules: `snake_case.rs` (e.g., `health_monitor.rs`, `crash_recovery.rs`)
- Vue components: `PascalCase.vue` (e.g., `AppCard.vue`, `MeshGraph.vue`)
- Composables: `use[Name].ts` (e.g., `useAuth.ts`, `useRpc.ts`)
- Stores: `[domain]Store.ts` (e.g., `appStore.ts`, `walletStore.ts`)
- Tests: `[name].test.ts` or `[name].spec.ts` (e.g., `rpc-client.test.ts`, `app-launch.spec.ts`)
- Scripts: lowercase with hyphens (e.g., `deploy-to-target.sh`, `run-gate.sh`)
**Directories:**
- Rust workspace members: lowercase (e.g., `archipelago`, `container`, `security`)
- Feature directories: PascalCase or lowercase depending on context
- `neode-ui/src/views/` — page components (mostly PascalCase)
- `neode-ui/src/composables/` — logic hooks (lowercase files with `use` prefix)
- `core/archipelago/src/api/rpc/` — RPC modules by domain (lowercase: `bitcoin.rs`, `mesh.rs`)
**Functions/Methods:**
- Async functions: `async fn method_name() -> Result<T>` (no special suffix)
- Event handlers: `on[Event]` in Vue (e.g., `@click="onInstall"` calls `onInstall()`)
- Computed properties: `computed(() => ...)` (no special name)
- Public RPC methods: `pub async fn [domain]_[action](...)` (e.g., `container_install`, `bitcoin_send`)
**Variables & Constants:**
- Constants: `UPPER_SNAKE_CASE` (e.g., `RECONCILER_DEFAULT_INTERVAL`, `MAX_FILE_SIZE`)
- State variables: `camelCase` (e.g., `appList`, `isLoading`)
- Type aliases: `PascalCase` (e.g., `AppId`, `MeshPeer`)
**Exports & Modules:**
- Re-exports barrel files: `mod.rs` exporting `pub use child::*;`
- Private internals: `mod private;` (not `pub mod`)
- Path aliases (neode-ui): `@/` = `src/`, `@components/` = `src/components/`
## Where to Add New Code
**New RPC Method (Backend):**
1. Determine domain (auth, container, bitcoin, mesh, wallet, etc.)
2. Add async fn to `core/archipelago/src/api/rpc/[domain].rs`
3. Function signature: `pub async fn [action](handler: &RpcHandler, params: [ParamType]) -> Result<[ResponseType]>`
4. Register in `core/archipelago/src/api/rpc/mod.rs` dispatcher (line ~350+, search for `match method_name`)
5. Test: Unit test in same file with `#[tokio::test]`, or E2E in `tests/`
**New Frontend View (Page):**
1. Create `neode-ui/src/views/[ViewName].vue` (PascalCase)
2. Import Router in `neode-ui/src/router/index.ts`, add route
3. Add navigation link in `neode-ui/src/components/Nav.vue` (if public-facing)
4. State: Use or create Pinia store in `neode-ui/src/stores/`
5. Test: Add E2E test in `neode-ui/e2e/` if user-facing flow
**New Container App:**
1. Create directory `apps/[app-name]/`
2. Write `manifest.yml` (copy structure from existing app; define interfaces.main.ui, health check, secrets)
3. Write `Dockerfile` (base image, deps, entrypoint)
4. Add to `app-catalog/catalog.json` with entry (id, version, url to manifest)
5. Test: `archipelago container.install { manifest_url: "..." }` on dev node
**New Component (Frontend):**
1. Create `neode-ui/src/components/[ComponentName].vue`
2. If reusable logic, extract to `neode-ui/src/composables/use[Logic].ts`
3. If shared state, use Pinia store (don't create component-local state)
4. Example: `components/AppCard.vue` displays one app (reused in Apps.vue listing)
**New Utility Function:**
- Backend service logic: Add to `core/archipelago/src/[domain].rs` or new file if it's cross-cutting
- Frontend helper: Add to `neode-ui/src/utils/` (e.g., `format.ts`, `validate.ts`)
- Example: Lightning address validation → `neode-ui/src/utils/validate.ts:validateLightningAddress()`
**New Test:**
- Rust unit test: Inline in source file (`#[test]` or `#[tokio::test]`)
- Frontend unit test: `neode-ui/src/composables/__tests__/use[Logic].test.ts`
- E2E test: `neode-ui/e2e/[feature].spec.ts` (Playwright)
- Integration test: `tests/[feature].sh` (shell script running node commands)
## Special Directories
**core/target/**
- Purpose: Rust build artifacts (generated)
- Generated: Yes (by `cargo build`)
- Committed: No (in .gitignore)
**neode-ui/node_modules/**
- Purpose: npm dependencies
- Generated: Yes (by `npm install` or `pnpm install`)
- Committed: No (in .gitignore)
**web/dist/**
- Purpose: Built frontend output (generated)
- Generated: Yes (by `npm run build`)
- Committed: No (in .gitignore) — distributed via OTA/ISO
**/var/lib/archipelago/** (at runtime on node)
- Purpose: Data directory (user data, settings, secrets, backups)
- Generated: Yes (created by daemon on first boot)
- Committed: No (runtime data; contains user secrets)
- Subdirs:
- `identity/` — Node Ed25519 keys
- `secrets/` — Encrypted secret vaults (ChaCha20)
- `apps/` — App manifests (disk copies or registry overlays)
- `backups/` — Encrypted backup archives
- `registries.json` — User-configured Gitea mirrors
- etc.
**/opt/archipelago/** (at runtime on node)
- Purpose: System-level Archipelago files (read-only on ISO, writable post-install)
- Contains:
- `web-ui/` — Built frontend (nginx root)
- `bin/archipelago` — Daemon binary
- `docker/` — Docker Compose or Quadlet files (orchestration)
- `scripts/` — System maintenance scripts
- Note: OTA updates overwrite `web-ui/` + `bin/` atomically
---
*Structure analysis: 2026-07-29*