changes for build for sxsw
This commit is contained in:
Dorian
@@ -0,0 +1,84 @@
|
||||
# ISO Build Session — 2026-03-10
|
||||
|
||||
## Status: Changes ready, NOT yet deployed or built
|
||||
|
||||
All changes are local. Servers were unreachable at end of session (network issue, not crash).
|
||||
Need to: deploy to .228 → build new ISO → copy to File Browser Builds folder.
|
||||
|
||||
## Changes Made (Local, Uncommitted)
|
||||
|
||||
### 1. ISO Login Fix (`image-recipe/build-auto-installer-iso.sh`)
|
||||
- **Problem**: `chpasswd` fails silently in chroot (PAM not available), leaving password locked
|
||||
- **Fix**: Direct `/etc/shadow` manipulation with `sed` using SHA-512 hash from `openssl passwd -6`
|
||||
- Pre-computed hash as fallback if openssl unavailable
|
||||
- Verification check + chpasswd fallback
|
||||
- Also added `root:archipelago` password in Dockerfile
|
||||
- **Credentials**: `archipelago` / `archipelago` (TTY/SSH), `password123` (Web UI)
|
||||
|
||||
### 2. Onboarding "Server Starting Up" UX (4 Vue files)
|
||||
- **Problem**: On fresh install, backend takes 2-5 min to start. Onboarding shows scary error messages.
|
||||
- **OnboardingDid.vue**: Replaced 3-attempt retry with persistent auto-retry every 4s. Shows "Server starting up" with elapsed timer (e.g. `1:23`) to the right. Keeps trying until backend responds.
|
||||
- **OnboardingIdentity.vue**: Detects 502/503, shows orange "Server is still starting up" instead of red error.
|
||||
- **OnboardingBackup.vue**: Same friendly server-starting message.
|
||||
- **OnboardingVerify.vue**: Same friendly server-starting message.
|
||||
|
||||
### 3. First-Boot Container Fixes (`scripts/first-boot-containers.sh`)
|
||||
- **Problem**: Race conditions — services start before dependencies are ready
|
||||
- Added `wait_for_container()` function with configurable timeout and logging
|
||||
- **Bitcoin Knots**: Added RPC health check wait (up to 60s) before LND/NBXplorer/mempool start
|
||||
- **BTCPay PostgreSQL**: Replaced `sleep 3` with `pg_isready` health check (up to 30s)
|
||||
- **Mempool MariaDB**: Replaced `sleep 3` with connection check (up to 30s)
|
||||
- **File Browser**: Removed `--read-only` and `--cap-drop ALL` (was preventing database creation). Added separate `/database` volume mount.
|
||||
|
||||
### 4. Build Skill Updated (`.claude/skills/build-iso/SKILL.md`)
|
||||
- Added "Post-build: Publish to File Browser" step
|
||||
- ISO gets copied to `/var/lib/archipelago/filebrowser/Builds/` after every build
|
||||
|
||||
## Fresh Install Issues Found on .198
|
||||
- Login was broken (fixed in #1)
|
||||
- Onboarding showed 502 errors at every step (fixed in #2)
|
||||
- Containers not launching: Bitcoin Knots, BTCPay, File Browser, Grafana, LND (fixed in #3)
|
||||
- File Browser specifically: `--read-only` prevented database creation (fixed in #3)
|
||||
- Could not fully diagnose .198 — went offline before SSH diagnostic completed
|
||||
|
||||
## Deploy Steps When Servers Are Back
|
||||
```bash
|
||||
# 1. Deploy to live server
|
||||
./scripts/deploy-to-target.sh --live
|
||||
|
||||
# 2. Sync build script
|
||||
rsync -avz -e "ssh -i ~/.ssh/archipelago-deploy" \
|
||||
image-recipe/build-auto-installer-iso.sh \
|
||||
archipelago@192.168.1.228:~/archy/image-recipe/
|
||||
|
||||
# 3. Sync first-boot script
|
||||
rsync -avz -e "ssh -i ~/.ssh/archipelago-deploy" \
|
||||
scripts/first-boot-containers.sh \
|
||||
archipelago@192.168.1.228:~/archy/scripts/
|
||||
|
||||
# 4. Build ISO on server
|
||||
ssh -i ~/.ssh/archipelago-deploy archipelago@192.168.1.228 \
|
||||
'cd ~/archy/image-recipe && sudo ./build-auto-installer-iso.sh'
|
||||
|
||||
# 5. Copy to File Browser
|
||||
ssh -i ~/.ssh/archipelago-deploy archipelago@192.168.1.228 \
|
||||
'sudo mkdir -p /var/lib/archipelago/filebrowser/Builds && \
|
||||
sudo cp ~/archy/image-recipe/results/archipelago-installer-x86_64.iso \
|
||||
/var/lib/archipelago/filebrowser/Builds/'
|
||||
|
||||
# 6. Download to Mac
|
||||
scp -i ~/.ssh/archipelago-deploy \
|
||||
archipelago@192.168.1.228:~/archy/image-recipe/results/archipelago-installer-x86_64.iso \
|
||||
~/Downloads/
|
||||
```
|
||||
|
||||
## Files Modified (git diff summary)
|
||||
- `image-recipe/build-auto-installer-iso.sh` — password fix + Dockerfile root password
|
||||
- `scripts/first-boot-containers.sh` — health checks + filebrowser fix
|
||||
- `scripts/deploy-to-target.sh` — Tor permission fixes (from earlier)
|
||||
- `neode-ui/src/views/OnboardingDid.vue` — auto-retry with timer
|
||||
- `neode-ui/src/views/OnboardingIdentity.vue` — server-starting detection
|
||||
- `neode-ui/src/views/OnboardingBackup.vue` — server-starting detection
|
||||
- `neode-ui/src/views/OnboardingVerify.vue` — server-starting detection
|
||||
- `.claude/skills/build-iso/SKILL.md` — added File Browser publish step
|
||||
- Frontend already built: `web/dist/neode-ui/` is up to date
|
||||
@@ -0,0 +1,292 @@
|
||||
# Archipelago 3-Year Project Plan
|
||||
|
||||
**Version**: 1.0
|
||||
**Period**: March 2026 -- March 2029
|
||||
**Goal**: Production-ready Bitcoin Node OS with zero issues for end users
|
||||
**Visual constraint**: NEVER change animations, user experience, or visuals -- only neater layouts where highlighted
|
||||
|
||||
## Current Status: Year 1, Q1, Sprint 1 (Starting)
|
||||
|
||||
---
|
||||
|
||||
## Year 1: Foundation & Core Functionality (March 2026 -- February 2027)
|
||||
|
||||
### Q1 2026 (March -- May): Fix Broken UI, Testing Infrastructure, Networking
|
||||
|
||||
#### Sprint 1: Test Infrastructure (Week 1-2)
|
||||
- [ ] Install Vitest and configure frontend test runner
|
||||
- [ ] Create first frontend unit tests: RPC client (8+ test cases)
|
||||
- [ ] Create frontend unit tests: app store (6+ test cases)
|
||||
- [ ] Create frontend unit tests: container store (5+ test cases)
|
||||
- [ ] Create frontend unit tests: router guards (6+ test cases)
|
||||
- [ ] Create backend integration test scaffolding
|
||||
- [ ] Create backend unit tests: auth module (6+ test cases)
|
||||
- [ ] Create backend unit tests: identity module (5+ test cases)
|
||||
- [ ] Add CI-compatible test runner script (scripts/run-tests.sh)
|
||||
|
||||
#### Sprint 2: Fix Broken UI (Week 3-4)
|
||||
- [ ] Fix Settings.vue: replace .path-option-card with .glass-card
|
||||
- [ ] Fix Web5.vue top bar: verify glass sub-card consistency with Server.vue
|
||||
- [ ] Remove duplicate network diagnostics from Settings.vue
|
||||
- [ ] Server.vue: wire real RPC data to Local Network card
|
||||
- [ ] Server.vue: wire real RPC data to Web3 card (show "Coming Soon")
|
||||
|
||||
#### Sprint 3: Backend Robustness (Week 5-6)
|
||||
- [ ] Add system monitoring RPC endpoints (system.stats, system.processes, system.temperature)
|
||||
- [ ] Add system monitoring to frontend Dashboard (CPU/RAM/Disk gauges)
|
||||
- [ ] Add WiFi/Ethernet configuration RPC endpoints
|
||||
- [ ] Add WiFi/Ethernet UI to Server.vue
|
||||
- [ ] Implement CSRF protection on RPC layer
|
||||
- [ ] Fix CORS policy: restrict to same-origin
|
||||
- [ ] Add Nginx security headers
|
||||
|
||||
#### Sprint 4: Quality Baseline (Week 7-8)
|
||||
- [ ] Run full sweep and record baseline in docs/quality-baseline.md
|
||||
- [ ] Fix all silent catch blocks
|
||||
- [ ] Remove all console.log in production paths
|
||||
- [ ] Eliminate any-type usage in frontend
|
||||
- [ ] Health-gated deploy: add pre-deploy health check
|
||||
- [ ] Run canary deploy to secondary server
|
||||
|
||||
### Q2 2026 (June -- August): DWN, Backup/Restore, Kiosk Mode, StartOS Independence
|
||||
|
||||
#### Sprint 5: DWN Protocol Implementation (Week 1-3)
|
||||
- [ ] Implement DWN message store (dwn_store.rs)
|
||||
- [ ] Implement DWN HTTP API (POST /dwn)
|
||||
- [ ] Implement DWN peer sync protocol
|
||||
- [ ] Add DWN management UI (DwnManager.vue)
|
||||
- [ ] Add DWN RPC endpoints for protocol management
|
||||
|
||||
#### Sprint 6: Full Backup/Restore System (Week 4-5)
|
||||
- [ ] Extend backup module for full system backup
|
||||
- [ ] Add backup/restore RPC endpoints
|
||||
- [ ] Add backup/restore UI to Settings
|
||||
- [ ] Add backup to USB drive support
|
||||
|
||||
#### Sprint 7: Kiosk Mode Hardening (Week 6-7)
|
||||
- [ ] Add kiosk mode crash recovery
|
||||
- [ ] Add kiosk failsafe route (/recovery)
|
||||
- [ ] Add kiosk-specific keyboard shortcuts
|
||||
- [ ] Create kiosk systemd service
|
||||
|
||||
#### Sprint 8: StartOS Independence (Week 8-10)
|
||||
- [ ] Audit StartOS code usage → docs/startos-dependency-audit.md
|
||||
- [ ] Migrate essential StartOS utilities to archipelago
|
||||
- [ ] Remove core/startos from workspace
|
||||
- [ ] Run full regression test after removal
|
||||
|
||||
### Q3 2026 (September -- November): App Integration, Auto-Updates, ARM64
|
||||
|
||||
#### Sprint 9: App Integration Testing (Week 1-3)
|
||||
- [ ] Create app integration test suite (scripts/test-all-apps.sh)
|
||||
- [ ] Fix all app integration failures
|
||||
- [ ] Test dependency chains
|
||||
- [ ] Test fresh install end-to-end
|
||||
|
||||
#### Sprint 10: Auto-Update System (Week 4-6)
|
||||
- [ ] Implement update download and apply
|
||||
- [ ] Add update notification to frontend
|
||||
- [ ] Implement automatic update scheduling
|
||||
- [ ] Create release manifest infrastructure
|
||||
|
||||
#### Sprint 11: ARM64 Support (Week 7-9)
|
||||
- [ ] Set up ARM64 cross-compilation
|
||||
- [ ] Test ARM64 container images
|
||||
- [ ] Build ARM64 ISO
|
||||
- [ ] Test ARM64 on Raspberry Pi 5
|
||||
|
||||
#### Sprint 12: Quality Hardening (Week 10-12)
|
||||
- [ ] Achieve 50% frontend test coverage
|
||||
- [ ] Achieve 50% backend test coverage
|
||||
- [ ] Run overnight chaos test
|
||||
- [ ] Run full quality sweep vs baseline
|
||||
|
||||
### Q4 2026 (December -- February 2027): Security, Performance, Beta
|
||||
|
||||
#### Sprint 13: Security Hardening (Week 1-3)
|
||||
- [ ] Implement session expiry and rotation
|
||||
- [ ] Harden container security profiles
|
||||
- [ ] Add secrets rotation mechanism
|
||||
- [ ] Sanitize FileBrowser path traversal
|
||||
- [ ] Remove FileBrowser token from URLs
|
||||
- [ ] Run automated security scan
|
||||
|
||||
#### Sprint 14: Performance Optimization (Week 4-6)
|
||||
- [ ] Profile and optimize backend startup (<3s)
|
||||
- [ ] Optimize frontend bundle size (<500KB gzipped)
|
||||
- [ ] Add WebSocket connection pooling and heartbeat
|
||||
- [ ] Optimize container image pull performance
|
||||
|
||||
#### Sprint 15: Beta Release Prep (Week 7-10)
|
||||
- [ ] Create comprehensive user documentation
|
||||
- [ ] Create beta testing checklist
|
||||
- [ ] Build and test beta ISO
|
||||
- [ ] Publish v0.5.0-beta release
|
||||
- [ ] Run 72-hour stability test
|
||||
|
||||
---
|
||||
|
||||
## Year 2: Feature Completeness & Reliability (March 2027 -- February 2028)
|
||||
|
||||
### Q1 2027 (March -- May): W3C DIDs, JSON-LD VCs, Hardware Wallet
|
||||
|
||||
#### Sprint 16: W3C-Compliant DIDs (Week 1-3)
|
||||
- [ ] Implement W3C DID Document format
|
||||
- [ ] Implement DID Document verification
|
||||
- [ ] Update DID display in Web5.vue
|
||||
- [ ] Add DID resolution across peers
|
||||
|
||||
#### Sprint 17: JSON-LD Verifiable Credentials (Week 4-6)
|
||||
- [ ] Implement JSON-LD credential format
|
||||
- [ ] Add credential presentation protocol
|
||||
- [ ] Add credential management UI
|
||||
|
||||
#### Sprint 18: Hardware Wallet Integration (Week 7-10)
|
||||
- [ ] Research and document hardware wallet integration
|
||||
- [ ] Implement PSBT signing flow in LND RPC
|
||||
- [ ] Add hardware wallet UI flow
|
||||
- [ ] Add USB hardware wallet detection
|
||||
|
||||
### Q2 2027 (June -- August): Multi-Node, VPN, Community Marketplace
|
||||
|
||||
#### Sprint 19: Multi-Node Orchestration (Week 1-4)
|
||||
- [ ] Design multi-node architecture
|
||||
- [ ] Implement node federation protocol
|
||||
- [ ] Add multi-node dashboard
|
||||
- [ ] Implement federated app deployment
|
||||
|
||||
#### Sprint 20: VPN and Mesh Networking (Week 5-8)
|
||||
- [ ] Add Tailscale/WireGuard VPN integration
|
||||
- [ ] Add VPN status to Server.vue
|
||||
- [ ] Implement mesh networking discovery
|
||||
- [ ] Add DNS-over-HTTPS configuration
|
||||
|
||||
#### Sprint 21: Community App Marketplace (Week 9-12)
|
||||
- [ ] Design decentralized marketplace protocol
|
||||
- [ ] Implement marketplace manifest discovery
|
||||
- [ ] Implement app manifest publishing
|
||||
- [ ] Add community marketplace tab to frontend
|
||||
|
||||
### Q3 2027 (September -- November): Documentation, Reliability, Pre-Release
|
||||
|
||||
#### Sprint 22: Comprehensive Documentation (Week 1-3)
|
||||
- [ ] Write developer documentation
|
||||
- [ ] Write API documentation
|
||||
- [ ] Write app developer SDK documentation
|
||||
- [ ] Create Architecture Decision Records
|
||||
|
||||
#### Sprint 23: Reliability Engineering (Week 4-8)
|
||||
- [ ] Implement graceful shutdown
|
||||
- [ ] Add crash recovery
|
||||
- [ ] Implement disk space management
|
||||
- [ ] Add container health monitoring and auto-recovery
|
||||
- [ ] Run 1-week continuous uptime test
|
||||
|
||||
#### Sprint 24: Pre-Release Quality (Week 9-12)
|
||||
- [ ] Achieve 70% frontend test coverage
|
||||
- [ ] Achieve 70% backend test coverage
|
||||
- [ ] Run full regression screenshot comparison
|
||||
- [ ] Publish v0.8.0-rc1 release candidate
|
||||
|
||||
### Q4 2027 (December -- February 2028): Polish, Community, v0.9.0
|
||||
|
||||
#### Sprint 25: User Experience Polish (Week 1-4)
|
||||
- [ ] Run complete UX audit
|
||||
- [ ] Fix all UX audit findings
|
||||
- [ ] Polish error handling across entire frontend
|
||||
- [ ] Polish all forms
|
||||
|
||||
#### Sprint 26: Community Infrastructure (Week 5-8)
|
||||
- [ ] Set up update server infrastructure
|
||||
- [ ] Create community contribution guidelines
|
||||
- [ ] Set up issue tracker and roadmap
|
||||
- [ ] Publish v0.9.0 release
|
||||
|
||||
---
|
||||
|
||||
## Year 3: Production Polish & Scale (March 2028 -- March 2029)
|
||||
|
||||
### Q1 2028 (March -- May): Monitoring, Remote Management, Accessibility
|
||||
|
||||
#### Sprint 27: Advanced Monitoring (Week 1-4)
|
||||
- [ ] Implement real-time metrics collection
|
||||
- [ ] Add monitoring dashboard page
|
||||
- [ ] Implement alerting system
|
||||
- [ ] Add historical data export
|
||||
|
||||
#### Sprint 28: Remote Management (Week 5-8)
|
||||
- [ ] Implement Tailscale-based remote access
|
||||
- [ ] Add mobile-optimized remote management
|
||||
- [ ] Implement remote notification system
|
||||
|
||||
#### Sprint 29: Accessibility and i18n (Week 9-12)
|
||||
- [ ] Add ARIA labels and roles
|
||||
- [ ] Add keyboard navigation testing
|
||||
- [ ] Set up i18n infrastructure
|
||||
|
||||
### Q2 2028 (June -- August): Pen Testing, Final QA
|
||||
|
||||
#### Sprint 30: Security Penetration Testing (Week 1-4)
|
||||
- [ ] Run automated penetration test suite
|
||||
- [ ] Manual security review of all RPC endpoints
|
||||
- [ ] Harden Podman container isolation
|
||||
- [ ] Add rate limiting to all sensitive endpoints
|
||||
|
||||
#### Sprint 31: End-to-End QA (Week 5-8)
|
||||
- [ ] Create golden path test suite
|
||||
- [ ] Run regression test across all hardware
|
||||
- [ ] Achieve 80% test coverage
|
||||
- [ ] Run 30-day soak test
|
||||
|
||||
#### Sprint 32: Documentation and Community (Week 9-12)
|
||||
- [ ] Write troubleshooting guide
|
||||
- [ ] Create walkthrough documentation
|
||||
- [ ] Finalize all ADRs
|
||||
- [ ] Publish v0.95.0-rc2
|
||||
|
||||
### Q3 2028 (September -- November): v1.0 Release
|
||||
|
||||
#### Sprint 33: Final Polish (Week 1-4)
|
||||
- [ ] Final UX audit
|
||||
- [ ] Final security audit
|
||||
- [ ] Final sweep
|
||||
- [ ] Performance benchmark and optimize
|
||||
|
||||
#### Sprint 34: Release Engineering (Week 5-8)
|
||||
- [ ] Create release automation
|
||||
- [ ] Set up download/update infrastructure
|
||||
- [ ] Write v1.0 release notes
|
||||
- [ ] Build v1.0.0 release ISOs
|
||||
|
||||
#### Sprint 35: Launch (Week 9-12)
|
||||
- [ ] Tag and publish v1.0.0
|
||||
- [ ] Run 7-day post-release monitoring
|
||||
- [ ] Create v1.1 roadmap
|
||||
|
||||
### Q4 2028 (December -- February 2029): Maintenance
|
||||
|
||||
#### Sprint 36-39: Ongoing
|
||||
- [ ] Monthly dependency update cycle
|
||||
- [ ] Monthly security scan
|
||||
- [ ] Quarterly quality sweep
|
||||
- [ ] Community app reviews
|
||||
- [ ] Plan v2.0 features
|
||||
|
||||
---
|
||||
|
||||
## Milestone Summary
|
||||
|
||||
| Date | Milestone | Key Deliverables |
|
||||
|------|-----------|-----------------|
|
||||
| May 2026 | Q1 Complete | Tests, UI fixes, security, quality baseline |
|
||||
| Aug 2026 | Q2 Complete | DWN, backup/restore, kiosk, StartOS independence |
|
||||
| Nov 2026 | Q3 Complete | App testing, auto-updates, ARM64 |
|
||||
| Feb 2027 | **v0.5.0-beta** | First public beta |
|
||||
| Nov 2027 | **v0.8.0-rc1** | Release candidate |
|
||||
| Feb 2028 | **v0.9.0** | Pre-release |
|
||||
| Nov 2028 | **v1.0.0** | Production release |
|
||||
|
||||
## Execution Method
|
||||
- Execute via `/overnight` skill — each session picks up next uncompleted tasks
|
||||
- Full detailed acceptance criteria in the original plan conversation
|
||||
- Track progress by checking off items in this file as [x]
|
||||
@@ -0,0 +1,30 @@
|
||||
# Unbundled ISO Build (In Progress)
|
||||
|
||||
## Status: NOT YET BUILT
|
||||
- Server was unreachable (SSH timeout) when we tried to build — user rebooting
|
||||
- Changes are in working tree only, NOT YET COMMITTED
|
||||
|
||||
## What Was Done
|
||||
- Created `image-recipe/build-unbundled-iso.sh` — thin wrapper that sets `UNBUNDLED=1` and delegates to main script
|
||||
- Modified `image-recipe/build-auto-installer-iso.sh` to support `UNBUNDLED=1` env var
|
||||
|
||||
## Changes to build-auto-installer-iso.sh
|
||||
1. Added `UNBUNDLED="${UNBUNDLED:-0}"` config variable
|
||||
2. Step 3b: Skips container image capture from server AND registry pull (~20 tars)
|
||||
3. Skips `first-boot-containers.sh` bundling (no images to create containers from)
|
||||
4. Skips docker UI source bundling (bitcoin-ui, lnd-ui, electrs-ui)
|
||||
5. Different ISO filename: `archipelago-installer-unbundled-x86_64.iso`
|
||||
6. Updated installer completion message (tells user to install from Marketplace)
|
||||
7. Updated build summary output
|
||||
|
||||
## What Still Works in Unbundled
|
||||
- Full rootfs (Debian 12 + Podman + nginx + SSH)
|
||||
- Backend binary + web UI captured from server
|
||||
- Tor setup on first boot
|
||||
- Image loader service (harmlessly handles empty dir)
|
||||
- `package.install` already does `podman pull` — Marketplace works out of the box
|
||||
|
||||
## Next Steps
|
||||
1. Rsync updated scripts to dev server (192.168.1.228)
|
||||
2. Run: `sudo ./build-unbundled-iso.sh`
|
||||
3. Result appears in: `image-recipe/results/archipelago-installer-unbundled-x86_64.iso`
|
||||
Reference in New Issue
Block a user