lfg2025/archy
1
0
Fork 0
Code Issues 25 Pull Requests 4 Actions Packages Projects Releases 74 Wiki Activity

refactor: update dependencies and remove unused code

Browse Source 
- Added new dependencies: `adler2`, `crc32fast`, `flate2`, `miniz_oxide`, and `libredox`.
- Updated existing dependencies: `tokio-rustls` to version 0.26.4 and `filetime` to version 0.2.27.
- Removed the `backup.rs` file as it is no longer needed.
- Introduced tests for configuration and credential management.
- Enhanced the `identity` module to generate W3C compliant DID documents.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Dorian
2026-03-12 00:19:30 +00:00
co-authored by Claude Opus 4.6
parent 2a867b32a8
commit 6fee6befed
347 changed files with 18703 additions and 46785 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/BETA-RELEASE-CHECKLIST.md
+253 -171
View File
@@ -1,188 +1,270 @@
# CRITICAL CHANGES FOR BETA ISO BUILD
# Beta Release Checklist (v0.5.0-beta)
## ⚠️ MUST-HAVE CHANGES - Without these, the beta will NOT work!
## Pre-Build Verification
### 1. Backend: Podman Detection Fix
**File:** `core/container/src/podman_client.rs`
### Source Code
```rust
fn podman_async(&self) -> TokioCommand {
// Always use sudo podman to access system-wide containers
let mut cmd = TokioCommand::new("sudo");
cmd.arg("podman");
cmd
}
```
- [ ] All changes committed and pushed to `main`
- [ ] `cargo clippy --all-targets --all-features` passes (zero warnings)
- [ ] `cargo fmt --all` applied
- [ ] `cd neode-ui && npm run type-check` passes (zero errors)
- [ ] `cd neode-ui && npm test` passes (all tests green)
- [ ] `cargo test --all-features` passes on dev server
**System Config:** `/etc/sudoers.d/archipelago-podman`
```
archipelago ALL=(ALL) NOPASSWD: /usr/bin/podman
```
### Critical Files
### 2. Backend: Bitcoin UI Container Mapping
**File:** `core/archipelago/src/container/docker_packages.rs`
Add special case for bitcoin-knots (line ~95):
```rust
} else if app_id == "bitcoin-knots" {
// Check if bitcoin-ui exists (maps to "bitcoin" but serves bitcoin-knots)
if let Some(ui_address) = ui_containers.get("bitcoin") {
debug!("Using bitcoin-ui for bitcoin-knots: {}", ui_address);
Some(ui_address.clone())
} else {
extract_lan_address(&container.ports)
}
```
### 3. Backend: IndeedHub Metadata
**File:** `core/archipelago/src/container/docker_packages.rs`
Add to `get_app_metadata()` function (line ~327):
```rust
"indeedhub" => AppMetadata {
title: "IndeedHub".to_string(),
description: "Decentralized media streaming platform".to_string(),
icon: "/assets/img/app-icons/indeedhub.png".to_string(),
repo: "https://github.com/indeedhub/indeedhub".to_string(),
},
```
### 4. Frontend: Marketplace Bitcoin Knots
**File:** `neode-ui/src/views/Marketplace.vue`
```javascript
{
id: 'bitcoin-knots', // CHANGED from 'bitcoin'
title: 'Bitcoin Knots',
version: '28.1.0', // UPDATED version
dockerImage: 'docker.io/bitcoinknots/bitcoin:latest', // CHANGED image
// ... rest of config
}
```
### 5. Auto-Installer: Profile Script Fix
**File:** `image-recipe/build-auto-installer-iso.sh` (line ~850)
Remove `|| [ ! -t 0 ]` check:
```bash
# CORRECT:
if [ -n "$INSTALLER_STARTED" ]; then
return 0
fi
# WRONG (will break auto-login):
# if [ -n "$INSTALLER_STARTED" ] || [ ! -t 0 ]; then
```
### 6. Nginx Configuration
**File:** Captured from `/etc/nginx/sites-available/default`
MUST include:
- HTTPS on port 443
- HTTP redirect to HTTPS
- Backend proxy: `/rpc/`, `/ws/`, `/health`
- Root: `/opt/archipelago/web-ui`
- SSL certificates in `/etc/nginx/ssl/`
### 7. Bitcoin UI Files
**Files:** `docker/bitcoin-ui/index.html` and `Dockerfile`
MUST be included in ISO or downloadable, so users can deploy the web UI container.
- [ ] `core/container/src/podman_client.rs` — sudo podman
- [ ] `core/archipelago/src/container/docker_packages.rs` — app metadata + UI mapping
- [ ] `core/archipelago/src/api/rpc/package.rs` — app configs, capabilities, dependencies
- [ ] `core/archipelago/src/session.rs` — session security hardening
- [ ] `core/security/src/secrets_manager.rs` — encryption + rotation
- [ ] `neode-ui/src/views/Marketplace.vue` — all app entries with pinned image versions
- [ ] `neode-ui/src/api/websocket.ts` — heartbeat + reconnection
- [ ] `image-recipe/build-auto-installer-iso.sh` — all container images captured
- [ ] `image-recipe/configs/nginx-archipelago.conf` — all app proxies + path traversal blocks
- [ ] All app icons present in `neode-ui/public/assets/img/app-icons/`
---
## Build Verification Before Beta Release
## App Integration Matrix
Run these checks:
Every app must be tested for install, launch, and uninstall on a fresh system.
### Core Bitcoin Stack
| App | Image | Version | Install | Launch | UI Loads | Uninstall |
|-----|-------|---------|---------|--------|----------|-----------|
| Bitcoin Knots | `bitcoinknots/bitcoin` | `v28.1` | [ ] | [ ] | [ ] | [ ] |
| Electrs | `mempool/electrs` | `v0.4.1` | [ ] | [ ] | [ ] | [ ] |
| LND | `lightninglabs/lnd` | `v0.18.4` | [ ] | [ ] | [ ] | [ ] |
| BTCPay Server | `btcpayserver/btcpayserver` | `2.0.6` | [ ] | [ ] | [ ] | [ ] |
| Mempool | `mempool/frontend` | `v3.0.0` | [ ] | [ ] | [ ] | [ ] |
| Fedimint | `fedimintui/fedimint` | `0.5.0` | [ ] | [ ] | [ ] | [ ] |
| Fedimint Gateway | `fedimintui/gateway-ui` | `0.5.0` | [ ] | [ ] | [ ] | [ ] |
### Storage & Media
| App | Image | Version | Install | Launch | UI Loads | Uninstall |
|-----|-------|---------|---------|--------|----------|-----------|
| File Browser | `filebrowser/filebrowser` | `v2` | [ ] | [ ] | [ ] | [ ] |
| Immich | `ghcr.io/immich-app/immich-server` | `v1.121.0` | [ ] | [ ] | [ ] | [ ] |
| PhotoPrism | `photoprism/photoprism` | `240915` | [ ] | [ ] | [ ] | [ ] |
### Productivity & Privacy
| App | Image | Version | Install | Launch | UI Loads | Uninstall |
|-----|-------|---------|---------|--------|----------|-----------|
| Penpot | `penpotapp/frontend` | `2.4` | [ ] | [ ] | [ ] | [ ] |
| SearXNG | `searxng/searxng` | `2024.11.17-e2554de75` | [ ] | [ ] | [ ] | [ ] |
| Ollama | `ollama/ollama` | `0.5.4` | [ ] | [ ] | [ ] | [ ] |
### Network & Infrastructure
| App | Image | Version | Install | Launch | UI Loads | Uninstall |
|-----|-------|---------|---------|--------|----------|-----------|
| Nostr Relay | `scsiblade/nostr-rs-relay` | `0.9.0` | [ ] | [ ] | [ ] | [ ] |
| Nginx Proxy Manager | `jc21/nginx-proxy-manager` | `2.12.1` | [ ] | [ ] | [ ] | [ ] |
| Tailscale | `tailscale/tailscale` | pinned | [ ] | [ ] | [ ] | [ ] |
| Home Assistant | `homeassistant/home-assistant` | pinned | [ ] | [ ] | [ ] | [ ] |
### Virtual Apps (No Container)
| App | Behavior | Works |
|-----|----------|-------|
| IndeedHub | Opens external URL | [ ] |
---
## Dependency Chain Tests
These must be tested in order on a fresh install:
- [ ] Install Bitcoin Knots → starts and begins syncing
- [ ] Install Electrs while Bitcoin running → connects to Bitcoin automatically
- [ ] Install LND while Bitcoin running → connects to Bitcoin automatically
- [ ] Install BTCPay while Bitcoin running → connects; Lightning available if LND present
- [ ] Install Mempool while Bitcoin + Electrs running → shows blockchain data
- [ ] Try installing Electrs without Bitcoin → shows clear error message
- [ ] Try installing LND without Bitcoin → shows clear error message
- [ ] Try installing Mempool without Bitcoin + Electrs → shows missing deps error
- [ ] Fedimint Gateway auto-detects LND credentials when available
---
## Security Hardening Verification
### Session Security
- [ ] Sessions expire after 24 hours of inactivity
- [ ] Password change invalidates all other sessions
- [ ] Maximum 5 concurrent sessions (oldest evicted when exceeded)
- [ ] Session tokens are SHA-256 hashed in memory (never stored as plaintext)
- [ ] Login rate limiting: 5 failures per 60 seconds per IP
### Container Security
- [ ] All container images use pinned versions (no `:latest`)
- [ ] Read-only root filesystem enabled for compatible apps
- [ ] `--cap-drop=ALL` applied to all containers
- [ ] `--security-opt=no-new-privileges:true` applied to all containers
- [ ] Required capabilities added explicitly per app (e.g., CHOWN for File Browser)
### Secrets Management
- [ ] Secrets encrypted with AES-256-GCM on disk
- [ ] Secret metadata tracked (creation date, rotation count)
- [ ] Secret rotation generates new random values and re-encrypts
- [ ] `security.list-expiring` RPC returns secrets older than threshold
### Path Traversal Prevention
- [ ] Nginx blocks `..` in filebrowser API paths (403 response)
- [ ] Frontend `sanitizePath()` strips `..` and resolves paths
- [ ] File Browser token not exposed in URLs
### Authentication
- [ ] TOTP 2FA enrollment and verification works
- [ ] TOTP backup codes work for recovery
- [ ] Maximum 5 TOTP attempts before session invalidation
- [ ] Pending TOTP sessions expire after 5 minutes
- [ ] Cookie-based auth (no tokens in query strings)
---
## WebSocket & Connectivity
- [ ] WebSocket connects on login and receives initial data dump
- [ ] WebSocket reconnects after network interruption (exponential backoff, max 30s)
- [ ] Server sends ping every 30s; client responds with pong
- [ ] Client sends JSON ping every 30s; server responds with JSON pong
- [ ] Server closes inactive connections after 5 minutes
- [ ] Connection state shown in UI (connected/reconnecting/disconnected)
- [ ] Install progress updates delivered in real-time via WebSocket
---
## Fresh Install Testing Matrix
### ISO Build
- [ ] ISO builds successfully on dev server
- [ ] ISO size is reasonable (< 10 GB)
- [ ] All container images captured in ISO
### Installation
- [ ] Boot from USB on x86_64 hardware
- [ ] Auto-installer partitions disk correctly
- [ ] Debian 12 installs without errors
- [ ] Archipelago services start on first boot
- [ ] Web UI accessible at server IP within 3 minutes of first boot
### Onboarding Flow
- [ ] Welcome screen displays with intro video
- [ ] Password creation enforces minimum requirements
- [ ] Path selection shows all 6 options
- [ ] DID generation completes within 60 seconds
- [ ] Identity naming is optional and skippable
- [ ] Backup download produces valid JSON file
- [ ] Onboarding completes and reaches Dashboard
### Post-Onboarding
- [ ] Dashboard shows all overview cards
- [ ] App Store loads with all curated apps
- [ ] Settings shows server name, version, DID, Tor address
- [ ] Logout and re-login works
- [ ] Password change works and invalidates other sessions
---
## Performance Targets
- [ ] Backend startup: < 3 seconds
- [ ] Frontend initial load: < 500 KB gzipped
- [ ] WebSocket initial data: < 1 second after connection
- [ ] App install progress visible in UI within 5 seconds of starting
---
## Nginx Proxy Verification
All app proxies must work in both HTTP and HTTPS blocks:
- [ ] `/rpc/` → backend:5678
- [ ] `/ws/` → backend:5678 (WebSocket upgrade)
- [ ] `/health` → backend:5678
- [ ] `/app/filebrowser/` → filebrowser:80
- [ ] `/app/searxng/` → searxng:8080
- [ ] `/app/immich/` → immich:2283
- [ ] `/app/penpot/` → penpot-frontend:80
- [ ] `/app/ollama/` → ollama:11434
- [ ] `/app/photoprism/` → photoprism:2342
- [ ] `/app/nginx-proxy-manager/` → npm:81
- [ ] `/app/tailscale/` → tailscale:8240
- [ ] BTCPay (port 23000) opens in new tab
- [ ] Home Assistant (port 8123) opens in new tab
- [ ] Tor hidden services resolve for all configured apps
---
## Rollback Procedures
### If Backend Fails to Start
```bash
# 1. Verify all source changes are committed
cd /Users/dorian/Projects/archy
git status # Should show all critical files committed
# Check logs
sudo journalctl -u archipelago -n 50 --no-pager
# 2. Build ISO from live server
cd image-recipe
DEV_SERVER=archipelago@192.168.1.228 ./build-auto-installer-iso.sh
# 3. Test ISO on clean VM
# - Boot ISO
# - Verify auto-installer runs
# - System should boot to login
# - Access http://SERVER-IP
# - Complete onboarding
# - Install Bitcoin Knots from App Store
# - Verify "Already Installed" shows after install
# - Verify "Launch" button works
# - Verify web UI loads on port 8334
# 4. Test all critical features
# - Bitcoin node syncing
# - RPC accessible
# - Web UI functional
# - Backend detects container
# - App Store shows proper status
# Restore previous binary
sudo cp /usr/local/bin/archipelago.bak /usr/local/bin/archipelago
sudo systemctl restart archipelago
```
---
## Critical Files Checklist
Before building beta ISO, ensure these files have the latest changes:
- [ ] `core/container/src/podman_client.rs` - sudo podman
- [ ] `core/archipelago/src/container/docker_packages.rs` - app metadata + UI mapping
- [ ] `neode-ui/src/views/Marketplace.vue` - bitcoin-knots ID
- [ ] `neode-ui/src/utils/dummyApps.ts` - IndeedHub data
- [ ] `image-recipe/build-auto-installer-iso.sh` - auto-start fix
- [ ] `docker/bitcoin-ui/` - UI files present
- [ ] `scripts/deploy-bitcoin-knots.sh` - deployment script
- [ ] All assets: `neode-ui/public/assets/img/app-icons/*.png`
---
## Testing Matrix
| Feature | Expected | Status |
|---------|----------|--------|
| Bitcoin Knots container runs | Running | ✅ |
| Bitcoin UI container runs | Running | ✅ |
| Backend detects bitcoin-knots | Detected | ✅ |
| Backend maps bitcoin-ui → bitcoin-knots | Port 8334 | ✅ |
| App shows in My Apps | Listed | ✅ |
| App Store shows "Already Installed" | Badge shown | ✅ (after ID fix) |
| Launch button visible | Clickable | ✅ |
| Launch opens web UI | Port 8334 | ✅ |
| RPC accessible | Port 8332 | ✅ |
| Blockchain syncing | Active | ✅ |
---
## Roll-Back Plan
If beta ISO fails:
1. Check `/var/log/archipelago.log` on installed system
2. Verify containers with `sudo podman ps -a`
3. Check nginx status: `sudo systemctl status nginx`
4. Test backend: `curl http://localhost:5678/health`
5. Rebuild ISO with `BUILD_FROM_SOURCE=1` if server state is corrupt
---
## Support Commands for Users
### If Frontend is Broken
```bash
# Check Bitcoin status
sudo podman logs -f bitcoin-knots
# Check blockchain sync progress
curl --user archipelago:archipelago123 \
--data-binary '{"jsonrpc":"1.0","id":"test","method":"getblockchaininfo","params":[]}' \
-H 'content-type: text/plain;' http://localhost:8332/ | grep blocks
# Restart if needed
sudo podman restart bitcoin-knots bitcoin-ui
# View Archipelago backend logs
sudo journalctl -u archipelago -f
# Restore previous frontend build
sudo cp -r /opt/archipelago/web-ui.bak/* /opt/archipelago/web-ui/
sudo systemctl reload nginx
```
### If Container Won't Start
```bash
# Check container logs
sudo podman logs <container-name>
# Remove and recreate
sudo podman rm -f <container-name>
# Reinstall from App Store
```
### If ISO Install Fails
1. Boot into rescue mode from USB
2. Check `/var/log/installer.log` on target disk
3. Verify disk partitioning with `lsblk`
4. Re-run installer with `INSTALLER_STARTED= /opt/installer.sh`
### Full System Rollback
If the beta is unusable:
1. Re-flash the ISO from the last known good build
2. Restore user data from `/var/lib/archipelago/` backup
3. Re-import DID from backup JSON file
---
## Sign-Off
| Reviewer | Area | Date | Pass/Fail |
|----------|------|------|-----------|
| | Backend | | |
| | Frontend | | |
| | Security | | |
| | ISO Build | | |
| | Fresh Install | | |
| | App Integrations | | |