feat: bitcoin-ui CSS fix, HTTPS proxy support, deploy script improvements
Bitcoin UI: - Replace cdn.tailwindcss.com with locally bundled tailwind.css (CSP blocks external scripts) - Make all asset paths relative for nginx proxy compatibility - Add bitcoin-ui build/deploy to deploy-to-target.sh (was missing entirely) - Use --network host (bitcoin-ui proxies Bitcoin RPC at 127.0.0.1:8332) HTTPS mixed content fix: - Add HTTPS_PROXY_PATHS in AppSession.vue — when parent page is HTTPS, iframe loads through nginx proxy instead of direct HTTP port - Prevents browser blocking HTTP iframes inside HTTPS pages - All Tailscale servers use HTTPS, this was breaking all app iframes Deploy & first-boot improvements: - first-boot-containers.sh auto-detects disk size for pruning vs txindex - first-boot-containers.sh checks fallback source path for UI containers - Added mempool-electrs to APP_PORTS mapping - ElectrumX container creation in first-boot - Podman doctor/fix/uptime skills added Also includes: session persistence, identity management, LND transactions, ElectrumX status UI, nostr-provider improvements, Web5 enhancements Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dorian
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
07e46dce56
commit
30164fd12a
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: podman-doctor
|
||||
description: >
|
||||
Comprehensive Podman container diagnostic for Archipelago. Audits all running containers,
|
||||
port mappings, network connectivity, health status, restart policies, and config consistency
|
||||
across all 4 layers (backend Rust, Podman runtime, Nginx proxy, frontend routing).
|
||||
Use when asked to "diagnose containers", "check podman", "why is app not working",
|
||||
"container health check", "port not reachable", "audit containers", "podman status",
|
||||
or when any container/app is misbehaving.
|
||||
allowed-tools: Bash Read Glob Grep
|
||||
---
|
||||
|
||||
# Podman Doctor — Container Infrastructure Diagnostics
|
||||
|
||||
Systematic diagnostic for Archipelago's Podman container stack. Catches port conflicts, network misconfigurations, health failures, missing restart policies, and config drift across all layers.
|
||||
|
||||
**SSH command**: `ssh -i ~/.ssh/archipelago-deploy archipelago@192.168.1.228`
|
||||
|
||||
If $ARGUMENTS is provided, focus diagnosis on that specific app/container. Otherwise run full audit.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Gather Runtime State
|
||||
|
||||
Run these on the server:
|
||||
|
||||
```bash
|
||||
# All containers with status, ports, networks
|
||||
sudo podman ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}\t{{.Networks}}"
|
||||
|
||||
# Check for port conflicts on known ports
|
||||
sudo ss -tlnp | grep -E ":(80|443|3000|4080|5678|8080|8081|8082|8083|8085|8096|8123|8173|8174|8175|8240|8332|8333|8334|8888|9735|10009|11434|23000|50001)\b"
|
||||
```
|
||||
|
||||
### Step 2: Check Restart Policies
|
||||
|
||||
Every container MUST have `--restart unless-stopped`. This is the #1 cause of downtime after reboots.
|
||||
|
||||
```bash
|
||||
for c in $(sudo podman ps -a --format "{{.Names}}"); do
|
||||
echo -n "$c: "
|
||||
sudo podman inspect "$c" --format "{{.HostConfig.RestartPolicy.Name}}"
|
||||
done
|
||||
```
|
||||
|
||||
**Red flag**: `no` or empty = container won't survive reboot.
|
||||
|
||||
### Step 3: Verify Port Mapping Consistency
|
||||
|
||||
Cross-reference these 4 layers — mismatches between ANY two cause "app not loading" bugs:
|
||||
|
||||
**Layer 1 — Backend Config (Rust)**: Read `core/archipelago/src/api/rpc/package.rs`, look at `get_app_config()` port mappings.
|
||||
|
||||
**Layer 2 — Podman Runtime**: `sudo podman ps --format "{{.Names}}: {{.Ports}}"`
|
||||
|
||||
**Layer 3 — Nginx Proxy**: Read these for `/app/{id}/` location blocks:
|
||||
- `image-recipe/configs/nginx-archipelago.conf` (HTTP)
|
||||
- `image-recipe/configs/snippets/archipelago-https-app-proxies.conf` (HTTPS)
|
||||
|
||||
**Layer 4 — Frontend Routing**: Read `neode-ui/src/stores/appLauncher.ts` — `PORT_TO_APP_ID` map.
|
||||
|
||||
| Symptom | Root Cause |
|
||||
|---------|-----------|
|
||||
| App iframe shows 502/504 | Nginx proxies to wrong port, or container not running |
|
||||
| App loads wrong content | Port collision — two containers on same host port |
|
||||
| Works on port but not /app/ path | Missing nginx location block |
|
||||
| Frontend can't find app | PORT_TO_APP_ID missing in appLauncher.ts |
|
||||
|
||||
### Step 4: Network Connectivity Audit
|
||||
|
||||
```bash
|
||||
# Networks and their containers
|
||||
sudo podman network ls
|
||||
sudo podman network inspect archy-net 2>/dev/null || echo "WARNING: archy-net missing!"
|
||||
```
|
||||
|
||||
**Must be on archy-net**: bitcoin-knots, lnd, electrs, mempool, btcpay-server, nbxplorer, fedimint, fedimint-gateway, nostr-rs-relay, indeedhub, ollama, open-webui
|
||||
|
||||
**Must NOT be on archy-net**: grafana, nextcloud, filebrowser, vaultwarden, bitcoin-ui, lnd-ui, tailscale (host network)
|
||||
|
||||
### Step 5: Health Check Status
|
||||
|
||||
```bash
|
||||
# Containers with health checks — are they passing?
|
||||
for c in $(sudo podman ps --format "{{.Names}}"); do
|
||||
health=$(sudo podman inspect "$c" --format "{{.State.Health.Status}}" 2>/dev/null)
|
||||
if [ -n "$health" ] && [ "$health" != "<no value>" ]; then
|
||||
echo "$c: $health"
|
||||
fi
|
||||
done
|
||||
|
||||
# Containers WITHOUT health checks (gap in monitoring)
|
||||
for c in $(sudo podman ps --format "{{.Names}}"); do
|
||||
hc=$(sudo podman inspect "$c" --format "{{.Config.Healthcheck}}" 2>/dev/null)
|
||||
if [ "$hc" = "<nil>" ] || [ -z "$hc" ]; then
|
||||
echo "NO HEALTHCHECK: $c"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
### Step 6: Resource & Failure Analysis
|
||||
|
||||
```bash
|
||||
# Resource usage
|
||||
sudo podman stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"
|
||||
|
||||
# Recent deaths (last 24h)
|
||||
sudo podman events --filter event=died --since 24h 2>/dev/null | tail -20
|
||||
|
||||
# OOM kills
|
||||
sudo podman ps -a --format "{{.Names}}" | while read c; do
|
||||
oom=$(sudo podman inspect "$c" --format "{{.State.OOMKilled}}" 2>/dev/null)
|
||||
[ "$oom" = "true" ] && echo "OOM KILLED: $c"
|
||||
done
|
||||
|
||||
# Non-zero exits
|
||||
sudo podman ps -a --filter status=exited --format "{{.Names}}\t{{.Status}}"
|
||||
```
|
||||
|
||||
### Step 7: Systemd Integration
|
||||
|
||||
```bash
|
||||
systemctl is-active archipelago nginx
|
||||
systemctl list-units --type=service | grep -i podman
|
||||
systemctl list-timers --all | grep -i -E "podman|container|archipelago"
|
||||
```
|
||||
|
||||
### Step 8: Generate Report
|
||||
|
||||
Produce a structured report:
|
||||
|
||||
```
|
||||
## Container Diagnostic Report
|
||||
|
||||
### Summary
|
||||
- Total containers: X running, Y stopped, Z unhealthy
|
||||
- Port conflicts: [list or "none"]
|
||||
- Missing restart policies: [list or "none"]
|
||||
- Network issues: [list or "none"]
|
||||
- Health check gaps: [list]
|
||||
|
||||
### Critical Issues (fix immediately)
|
||||
1. ...
|
||||
|
||||
### Warnings (fix soon)
|
||||
1. ...
|
||||
|
||||
### Recommended Actions
|
||||
1. ...
|
||||
```
|
||||
|
||||
After diagnosis, suggest running `/podman-fix` for any issues found.
|
||||
|
||||
## Port Reference
|
||||
|
||||
See `references/port-map.md` for the canonical port assignment table across all 4 layers.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Common Podman Failure Patterns
|
||||
|
||||
## Container Won't Start
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `exec format error` | Binary built on wrong arch | Rebuild on the Linux server |
|
||||
| `address already in use` | Port conflict | `ss -tlnp \| grep :PORT` to find offender |
|
||||
| `permission denied` | Missing capability or read-only root | Check `get_app_capabilities()`, add tmpfs |
|
||||
| `OCI runtime error` | Corrupt container state | `podman rm -f NAME && recreate` |
|
||||
| `image not known` | Image not pulled | `podman pull IMAGE:TAG` |
|
||||
| `no such network` | Network missing | `podman network create archy-net` |
|
||||
|
||||
## Container Starts But App Unreachable
|
||||
|
||||
| Symptom | Check Layer | Fix |
|
||||
|---------|------------|-----|
|
||||
| Direct port works, /app/ doesn't | Nginx config | Add `/app/{id}/` location block |
|
||||
| Neither works | Podman ports | `podman port NAME` — verify mapping exists |
|
||||
| Port mapped but refused | Container logs | App crashing internally — check logs |
|
||||
| Works sometimes | Resources | Check OOM kills, CPU, disk space |
|
||||
| 502 Bad Gateway | Nginx→Container | Wrong port in proxy_pass or container restarted |
|
||||
|
||||
## Container Keeps Dying
|
||||
|
||||
| Pattern | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Exits immediately (code 1) | Config error | Check `podman logs NAME` |
|
||||
| Dies after minutes | OOM killed | Increase `--memory` limit |
|
||||
| Dies when dep restarts | No restart policy | Add `--restart unless-stopped` |
|
||||
| Crash loop | Repeated crash | Fix root cause, don't just restart |
|
||||
|
||||
## Network Issues
|
||||
|
||||
| Problem | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Can't resolve container names | Not on archy-net | Recreate with `--network=archy-net` |
|
||||
| Can't reach internet | DNS missing | Add `--dns 1.1.1.1` |
|
||||
| Container-to-container timeout | Different networks | Put both on same network |
|
||||
|
||||
## Capability Reference
|
||||
|
||||
| Capability | Apps That Need It | Failure Mode |
|
||||
|-----------|------------------|-------------|
|
||||
| CHOWN | nextcloud, homeassistant, btcpay, jellyfin, portainer | Can't chown during setup |
|
||||
| SETUID/SETGID | nextcloud, homeassistant, btcpay, jellyfin | Can't switch to service user |
|
||||
| DAC_OVERRIDE | nextcloud, homeassistant, btcpay | Can't access cross-UID files |
|
||||
| FOWNER | bitcoin-knots, lnd, fedimint | Can't modify data dir perms |
|
||||
| NET_BIND_SERVICE | nginx-proxy-manager, vaultwarden | Can't bind ports <1024 |
|
||||
|
||||
## Read-Only Safe Apps
|
||||
|
||||
Only these 8 apps can run with `--read-only`: searxng, grafana, filebrowser, electrs, nostr-rs-relay, ollama, indeedhub
|
||||
|
||||
All others need writable root or will fail silently.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Archipelago Canonical Port Map
|
||||
|
||||
All port assignments across the 4 configuration layers. When adding or debugging an app, every row must be consistent across all columns.
|
||||
|
||||
## Bitcoin Stack
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| bitcoin-knots | 8332, 8333 | 8332, 8333 | archy-net | /app/bitcoin-knots/ | 8332→bitcoin-knots |
|
||||
| bitcoin-ui | 8334 | 80 | bridge | /app/bitcoin-ui/ | 8334→bitcoin-knots |
|
||||
| electrs | 50001 | 50001 | archy-net | /app/electrs/ | 50001→electrs |
|
||||
| lnd | 9735, 10009, 8080 | 9735, 10009, 8080 | archy-net | /app/lnd/ | 10009→lnd |
|
||||
| lnd-ui (RTL) | 8081 | 80 | bridge | /app/lnd-ui/ | 8081→lnd |
|
||||
|
||||
## Lightning & Payment
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| btcpay-server | 23000 | 49392 | archy-net | /app/btcpay/ | 23000→btcpay-server |
|
||||
| nbxplorer | 24444 | 32838 | archy-net | N/A (internal) | N/A |
|
||||
| fedimint | 8173, 8174, 8175 | 8173, 8174, 8175 | archy-net | /app/fedimint/ | 8174→fedimint |
|
||||
| fedimint-gateway | 8175 | 8175 | archy-net | /app/fedimint-gateway/ | 8175→fedimint-gateway |
|
||||
|
||||
## Explorer & Monitoring
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| mempool | 4080 | 8080 | archy-net | /app/mempool/ | 4080→mempool |
|
||||
| grafana | 3000 | 3000 | bridge | /app/grafana/ | 3000→grafana (new tab) |
|
||||
|
||||
## Self-Hosted Apps
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| nextcloud | 8085 | 80 | bridge | /app/nextcloud/ | 8085→nextcloud |
|
||||
| vaultwarden | 8082 | 80 | bridge | /app/vaultwarden/ | 8082→vaultwarden (new tab) |
|
||||
| filebrowser | 8083 | 80 | bridge | /app/filebrowser/ | 8083→filebrowser |
|
||||
| searxng | 8888 | 8080 | bridge | /app/searxng/ | 8888→searxng |
|
||||
| photoprism | 2342 | 2342 | bridge | /app/photoprism/ | 2342→photoprism (new tab) |
|
||||
| jellyfin | 8096 | 8096 | bridge | /app/jellyfin/ | 8096→jellyfin |
|
||||
| homeassistant | 8123 | 8123 | bridge | /app/homeassistant/ | 8123→homeassistant (new tab) |
|
||||
| ollama | 11434 | 11434 | archy-net | /app/ollama/ | 11434→ollama |
|
||||
| open-webui | 3080 | 8080 | archy-net | /app/open-webui/ | 3080→open-webui |
|
||||
|
||||
## Nostr & Social
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| nostr-rs-relay | 7000 | 8080 | archy-net | /app/nostr-rs-relay/ | 7000→nostr-rs-relay |
|
||||
| indeedhub | 3001 | 3000 | archy-net | /app/indeedhub/ | 3001→indeedhub |
|
||||
|
||||
## System
|
||||
|
||||
| App | Host Port(s) | Container Port(s) | Network | Nginx Path | Frontend Map |
|
||||
|-----|-------------|-------------------|---------|------------|-------------|
|
||||
| tailscale | 8240 | 8240 | host | /app/tailscale/ | N/A |
|
||||
| nginx-proxy-manager | 81, 8443 | 81, 443 | bridge | N/A | 81→nginx-proxy-manager |
|
||||
|
||||
## Multi-Container Stacks
|
||||
|
||||
**Immich**: immich-server (2283), immich-postgres (internal 5432), immich-redis (internal 6379) — all on immich-net
|
||||
**Penpot**: penpot-frontend (9001→80), penpot-backend, penpot-exporter, penpot-postgres, penpot-mailcatch — all on penpot-net
|
||||
**Mempool**: mempool (4080→8080), mempool-db (internal 3306) — on archy-net
|
||||
**BTCPay**: btcpay-server (23000→49392), nbxplorer (24444→32838), btcpay-postgres (internal 5432) — on archy-net
|
||||
|
||||
## Key Notes
|
||||
|
||||
- **archy-net apps** resolve each other by container name (e.g., `bitcoin-knots:8332`)
|
||||
- **bridge apps** are standalone — access services via host IP/port
|
||||
- **host network** (tailscale only) — shares host namespace, no port mapping
|
||||
- **New tab apps**: btcpay (23000), grafana (3000), vaultwarden (8082), photoprism (2342), homeassistant (8123) — X-Frame-Options blocks iframe
|
||||
Reference in New Issue
Block a user