fix: rootless UID mapping corrections + credential injection
- Correct off-by-one in UID mapping: container UID N → host UID (100000 + N - 1), not (100000 + N) - Deploy script auto-fixes UID ownership on every deploy - Bitcoin UI nginx uses __BITCOIN_RPC_AUTH__ placeholder injected from secrets at deploy time - container rules updated for rootless podman architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dorian
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
93c2c3ee67
commit
3682855668
@@ -4,6 +4,7 @@ description: >
|
||||
Comprehensive Podman container diagnostic for Archipelago. Audits all running containers,
|
||||
port mappings, network connectivity, health status, restart policies, and config consistency
|
||||
across all 4 layers (backend Rust, Podman runtime, Nginx proxy, frontend routing).
|
||||
Handles rootless Podman (user: archipelago, UID 1000, subuid 100000:65536).
|
||||
Use when asked to "diagnose containers", "check podman", "why is app not working",
|
||||
"container health check", "port not reachable", "audit containers", "podman status",
|
||||
or when any container/app is misbehaving.
|
||||
@@ -12,46 +13,123 @@ allowed-tools: Bash Read Glob Grep
|
||||
|
||||
# Podman Doctor — Container Infrastructure Diagnostics
|
||||
|
||||
Systematic diagnostic for Archipelago's Podman container stack. Catches port conflicts, network misconfigurations, health failures, missing restart policies, and config drift across all layers.
|
||||
Systematic diagnostic for Archipelago's **rootless Podman** container stack. Catches port conflicts, network misconfigurations, health failures, missing restart policies, UID mapping issues, and config drift across all layers.
|
||||
|
||||
**SSH command**: `ssh -i ~/.ssh/archipelago-deploy archipelago@192.168.1.228`
|
||||
|
||||
> **ROOTLESS PODMAN**: Archipelago runs Podman as the `archipelago` user (UID 1000), NOT root.
|
||||
> Never use `sudo podman` — use plain `podman` after SSH'ing in as the `archipelago` user.
|
||||
> Container UIDs are mapped via subuid: container UID N → host UID (100000 + N).
|
||||
|
||||
If $ARGUMENTS is provided, focus diagnosis on that specific app/container. Otherwise run full audit.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Gather Runtime State
|
||||
|
||||
Run these on the server:
|
||||
Run these on the server (as `archipelago` user — NO sudo):
|
||||
|
||||
```bash
|
||||
# All containers with status, ports, networks
|
||||
sudo podman ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}\t{{.Networks}}"
|
||||
podman ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}\t{{.Networks}}"
|
||||
|
||||
# Check for port conflicts on known ports
|
||||
sudo ss -tlnp | grep -E ":(80|443|3000|4080|5678|8080|8081|8082|8083|8085|8096|8123|8173|8174|8175|8240|8332|8333|8334|8888|9735|10009|11434|23000|50001)\b"
|
||||
ss -tlnp | grep -E ":(80|443|3000|4080|5678|8080|8081|8082|8083|8085|8096|8123|8173|8174|8175|8240|8332|8333|8334|8888|9735|10009|11434|23000|50001)\b"
|
||||
```
|
||||
|
||||
### Step 2: Check Restart Policies
|
||||
### Step 2: Rootless Podman Health Check
|
||||
|
||||
Rootless Podman has specific requirements that must be verified:
|
||||
|
||||
```bash
|
||||
# Verify running as archipelago user (NOT root)
|
||||
whoami # Must be "archipelago"
|
||||
id # Must show uid=1000(archipelago)
|
||||
|
||||
# Check XDG_RUNTIME_DIR is set (required for rootless podman socket)
|
||||
echo "XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR" # Must be /run/user/1000
|
||||
|
||||
# Verify subuid/subgid mapping exists
|
||||
grep archipelago /etc/subuid # Must show: archipelago:100000:65536
|
||||
grep archipelago /etc/subgid # Must show: archipelago:100000:65536
|
||||
|
||||
# Verify user lingering is enabled (keeps user services after logout)
|
||||
ls /var/lib/systemd/linger/ | grep archipelago # Must exist
|
||||
|
||||
# Check podman storage is accessible
|
||||
podman info --format "{{.Store.GraphRoot}}" # ~/.local/share/containers/storage
|
||||
ls -la ~/.local/share/containers/storage/ 2>/dev/null || echo "ERROR: Storage not accessible"
|
||||
|
||||
# Check podman socket
|
||||
ls -la /run/user/1000/podman/ 2>/dev/null || echo "WARNING: No podman socket directory"
|
||||
```
|
||||
|
||||
### Step 3: Check Restart Policies
|
||||
|
||||
Every container MUST have `--restart unless-stopped`. This is the #1 cause of downtime after reboots.
|
||||
|
||||
```bash
|
||||
for c in $(sudo podman ps -a --format "{{.Names}}"); do
|
||||
for c in $(podman ps -a --format "{{.Names}}"); do
|
||||
echo -n "$c: "
|
||||
sudo podman inspect "$c" --format "{{.HostConfig.RestartPolicy.Name}}"
|
||||
podman inspect "$c" --format "{{.HostConfig.RestartPolicy.Name}}"
|
||||
done
|
||||
```
|
||||
|
||||
**Red flag**: `no` or empty = container won't survive reboot.
|
||||
|
||||
### Step 3: Verify Port Mapping Consistency
|
||||
### Step 4: Volume Ownership Audit (Rootless UID Mapping)
|
||||
|
||||
Rootless Podman maps container UIDs via subuid. Volume directories must be owned by the MAPPED UID, not the container UID. Formula: `host_uid = 100000 + container_uid`
|
||||
|
||||
```bash
|
||||
echo "=== Volume Ownership Check ==="
|
||||
|
||||
# Default containers (run as root inside = UID 0 → host UID 100000)
|
||||
for dir in lnd fedimint homeassistant jellyfin vaultwarden photoprism ollama filebrowser electrumx btcpay immich; do
|
||||
if [ -d "/var/lib/archipelago/$dir" ]; then
|
||||
owner=$(stat -c '%u:%g' "/var/lib/archipelago/$dir" 2>/dev/null)
|
||||
if [ "$owner" != "100000:100000" ]; then
|
||||
echo "WRONG: /var/lib/archipelago/$dir owned by $owner (should be 100000:100000)"
|
||||
else
|
||||
echo " OK: $dir → $owner"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Bitcoin Knots (container UID 101 → host UID 100101)
|
||||
if [ -d "/var/lib/archipelago/bitcoin" ]; then
|
||||
owner=$(stat -c '%u:%g' "/var/lib/archipelago/bitcoin")
|
||||
[ "$owner" != "100101:100101" ] && echo "WRONG: bitcoin owned by $owner (should be 100101:100101)" || echo " OK: bitcoin → $owner"
|
||||
fi
|
||||
|
||||
# PostgreSQL (container UID 70 → host UID 100070)
|
||||
for dir in /var/lib/archipelago/*-db /var/lib/archipelago/postgres-*; do
|
||||
if [ -d "$dir" ]; then
|
||||
owner=$(stat -c '%u:%g' "$dir")
|
||||
[ "$owner" != "100070:100070" ] && echo "WRONG: $dir owned by $owner (should be 100070:100070)" || echo " OK: $(basename $dir) → $owner"
|
||||
fi
|
||||
done
|
||||
|
||||
# Grafana (container UID 472 → host UID 100472)
|
||||
if [ -d "/var/lib/archipelago/grafana" ]; then
|
||||
owner=$(stat -c '%u:%g' "/var/lib/archipelago/grafana")
|
||||
[ "$owner" != "100472:100472" ] && echo "WRONG: grafana owned by $owner (should be 100472:100472)" || echo " OK: grafana → $owner"
|
||||
fi
|
||||
|
||||
# MariaDB/MySQL (container UID 999 → host UID 100999)
|
||||
if [ -d "/var/lib/archipelago/mysql-mempool" ]; then
|
||||
owner=$(stat -c '%u:%g' "/var/lib/archipelago/mysql-mempool")
|
||||
[ "$owner" != "100999:100999" ] && echo "WRONG: mysql-mempool owned by $owner (should be 100999:100999)" || echo " OK: mysql-mempool → $owner"
|
||||
fi
|
||||
```
|
||||
|
||||
### Step 5: Verify Port Mapping Consistency
|
||||
|
||||
Cross-reference these 4 layers — mismatches between ANY two cause "app not loading" bugs:
|
||||
|
||||
**Layer 1 — Backend Config (Rust)**: Read `core/archipelago/src/api/rpc/package.rs`, look at `get_app_config()` port mappings.
|
||||
|
||||
**Layer 2 — Podman Runtime**: `sudo podman ps --format "{{.Names}}: {{.Ports}}"`
|
||||
**Layer 2 — Podman Runtime**: `podman ps --format "{{.Names}}: {{.Ports}}"`
|
||||
|
||||
**Layer 3 — Nginx Proxy**: Read these for `/app/{id}/` location blocks:
|
||||
- `image-recipe/configs/nginx-archipelago.conf` (HTTP)
|
||||
@@ -66,77 +144,114 @@ Cross-reference these 4 layers — mismatches between ANY two cause "app not loa
|
||||
| Works on port but not /app/ path | Missing nginx location block |
|
||||
| Frontend can't find app | PORT_TO_APP_ID missing in appLauncher.ts |
|
||||
|
||||
### Step 4: Network Connectivity Audit
|
||||
### Step 6: Network Connectivity Audit
|
||||
|
||||
```bash
|
||||
# Networks and their containers
|
||||
sudo podman network ls
|
||||
sudo podman network inspect archy-net 2>/dev/null || echo "WARNING: archy-net missing!"
|
||||
podman network ls
|
||||
podman network inspect archy-net 2>/dev/null || echo "WARNING: archy-net missing!"
|
||||
|
||||
# Check container subnet (rootless uses 10.89.x.x, NOT 10.88.x.x)
|
||||
podman network inspect archy-net --format "{{range .Subnets}}{{.Subnet}}{{end}}" 2>/dev/null
|
||||
```
|
||||
|
||||
**Must be on archy-net**: bitcoin-knots, lnd, electrs, mempool, btcpay-server, nbxplorer, fedimint, fedimint-gateway, nostr-rs-relay, indeedhub, ollama, open-webui
|
||||
**Must be on archy-net**: bitcoin-knots, lnd, electrs/electrumx, mempool, btcpay-server, nbxplorer, fedimint, fedimint-gateway, nostr-rs-relay, indeedhub, ollama, open-webui
|
||||
|
||||
**Must NOT be on archy-net**: grafana, nextcloud, filebrowser, vaultwarden, bitcoin-ui, lnd-ui, tailscale (host network)
|
||||
|
||||
### Step 5: Health Check Status
|
||||
### Step 7: UFW Forward Policy Check
|
||||
|
||||
Rootless Podman requires `DEFAULT_FORWARD_POLICY="ACCEPT"` in UFW, otherwise container ports are unreachable from LAN.
|
||||
|
||||
```bash
|
||||
grep DEFAULT_FORWARD_POLICY /etc/default/ufw
|
||||
# Must be "ACCEPT", NOT "DROP"
|
||||
# If DROP: containers work locally but NOT from other machines on the network
|
||||
```
|
||||
|
||||
### Step 8: Systemd Service Sandbox Check
|
||||
|
||||
The `archipelago.service` must have specific settings relaxed for rootless Podman:
|
||||
|
||||
```bash
|
||||
# Check critical settings
|
||||
systemctl cat archipelago.service | grep -E "ProtectHome|PrivateTmp|RestrictNamespaces|ReadWritePaths|XDG_RUNTIME_DIR"
|
||||
```
|
||||
|
||||
**Required settings for rootless Podman**:
|
||||
- `ProtectHome=no` — podman stores images in `~/.local/share/containers/`
|
||||
- `PrivateTmp=no` or disabled — podman runtime uses `/tmp/podman-run-1000/`
|
||||
- `RestrictNamespaces=` must NOT be set — rootless podman needs user namespaces
|
||||
- `ReadWritePaths=` must include `/var/lib/archipelago /run/user /tmp`
|
||||
- `Environment=XDG_RUNTIME_DIR=/run/user/1000`
|
||||
|
||||
### Step 9: Health Check Status
|
||||
|
||||
```bash
|
||||
# Containers with health checks — are they passing?
|
||||
for c in $(sudo podman ps --format "{{.Names}}"); do
|
||||
health=$(sudo podman inspect "$c" --format "{{.State.Health.Status}}" 2>/dev/null)
|
||||
for c in $(podman ps --format "{{.Names}}"); do
|
||||
health=$(podman inspect "$c" --format "{{.State.Health.Status}}" 2>/dev/null)
|
||||
if [ -n "$health" ] && [ "$health" != "<no value>" ]; then
|
||||
echo "$c: $health"
|
||||
fi
|
||||
done
|
||||
|
||||
# Containers WITHOUT health checks (gap in monitoring)
|
||||
for c in $(sudo podman ps --format "{{.Names}}"); do
|
||||
hc=$(sudo podman inspect "$c" --format "{{.Config.Healthcheck}}" 2>/dev/null)
|
||||
for c in $(podman ps --format "{{.Names}}"); do
|
||||
hc=$(podman inspect "$c" --format "{{.Config.Healthcheck}}" 2>/dev/null)
|
||||
if [ "$hc" = "<nil>" ] || [ -z "$hc" ]; then
|
||||
echo "NO HEALTHCHECK: $c"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
### Step 6: Resource & Failure Analysis
|
||||
### Step 10: Resource & Failure Analysis
|
||||
|
||||
```bash
|
||||
# Resource usage
|
||||
sudo podman stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"
|
||||
podman stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"
|
||||
|
||||
# Recent deaths (last 24h)
|
||||
sudo podman events --filter event=died --since 24h 2>/dev/null | tail -20
|
||||
podman events --filter event=died --since 24h 2>/dev/null | tail -20
|
||||
|
||||
# OOM kills
|
||||
sudo podman ps -a --format "{{.Names}}" | while read c; do
|
||||
oom=$(sudo podman inspect "$c" --format "{{.State.OOMKilled}}" 2>/dev/null)
|
||||
podman ps -a --format "{{.Names}}" | while read c; do
|
||||
oom=$(podman inspect "$c" --format "{{.State.OOMKilled}}" 2>/dev/null)
|
||||
[ "$oom" = "true" ] && echo "OOM KILLED: $c"
|
||||
done
|
||||
|
||||
# Non-zero exits
|
||||
sudo podman ps -a --filter status=exited --format "{{.Names}}\t{{.Status}}"
|
||||
podman ps -a --filter status=exited --format "{{.Names}}\t{{.Status}}"
|
||||
```
|
||||
|
||||
### Step 7: Systemd Integration
|
||||
### Step 11: Systemd Integration
|
||||
|
||||
```bash
|
||||
systemctl is-active archipelago nginx
|
||||
systemctl list-units --type=service | grep -i podman
|
||||
systemctl --user list-units --type=service 2>/dev/null | grep -i podman
|
||||
systemctl list-timers --all | grep -i -E "podman|container|archipelago"
|
||||
```
|
||||
|
||||
### Step 8: Generate Report
|
||||
### Step 12: Generate Report
|
||||
|
||||
Produce a structured report:
|
||||
|
||||
```
|
||||
## Container Diagnostic Report
|
||||
|
||||
### Rootless Podman Status
|
||||
- User: archipelago (UID 1000)
|
||||
- Subuid mapping: [OK/MISSING]
|
||||
- XDG_RUNTIME_DIR: [OK/MISSING]
|
||||
- User linger: [enabled/disabled]
|
||||
- UFW forward policy: [ACCEPT/DROP]
|
||||
|
||||
### Summary
|
||||
- Total containers: X running, Y stopped, Z unhealthy
|
||||
- Port conflicts: [list or "none"]
|
||||
- Missing restart policies: [list or "none"]
|
||||
- Network issues: [list or "none"]
|
||||
- UID mapping issues: [list or "none"]
|
||||
- Health check gaps: [list]
|
||||
|
||||
### Critical Issues (fix immediately)
|
||||
@@ -154,3 +269,7 @@ After diagnosis, suggest running `/podman-fix` for any issues found.
|
||||
## Port Reference
|
||||
|
||||
See `references/port-map.md` for the canonical port assignment table across all 4 layers.
|
||||
|
||||
## UID Mapping Reference
|
||||
|
||||
See `references/uid-mapping.md` for the complete rootless UID mapping table.
|
||||
|
||||
@@ -1,15 +1,31 @@
|
||||
# Common Podman Failure Patterns
|
||||
|
||||
## Rootless Podman Specific Failures
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `ERRO[0000] cannot find UID/GID for user` | subuid/subgid not configured | Add `archipelago:100000:65536` to `/etc/subuid` and `/etc/subgid` |
|
||||
| `Error: unshare: operation not permitted` | Systemd `RestrictNamespaces` blocks user namespaces | Remove `RestrictNamespaces=` from `archipelago.service` |
|
||||
| `Error: could not get runtime: creating runtime` | XDG_RUNTIME_DIR not set or /run/user/1000 missing | Set `Environment=XDG_RUNTIME_DIR=/run/user/1000` in service, ensure `loginctl enable-linger archipelago` |
|
||||
| `permission denied` on volume mount | Wrong UID ownership — must use mapped UIDs | `sudo chown -R 100000:100000 /var/lib/archipelago/APP` (see UID mapping table) |
|
||||
| `ERRO[0000] rootless containers not supported` | Podman not configured for rootless | Run `podman system migrate`, check `/etc/subuid` |
|
||||
| `Error: creating container storage: layer not known` | Corrupted rootless storage | `podman system reset` (destroys all containers — last resort) |
|
||||
| `Error: stat /tmp/podman-run-1000/...: no such file` | PrivateTmp=yes in systemd isolates /tmp | Set `PrivateTmp=no` in `archipelago.service` |
|
||||
| Container ports unreachable from LAN | UFW DEFAULT_FORWARD_POLICY="DROP" | Change to "ACCEPT" in `/etc/default/ufw`, then `sudo ufw reload` |
|
||||
| `Error: error creating network namespace` | Systemd `SystemCallFilter` blocks clone/unshare | Remove `SystemCallFilter=` from `archipelago.service` |
|
||||
| Containers lose network after service restart | podman runtime dir in /tmp cleaned | Ensure `PrivateTmp=no` so /tmp/podman-run-1000/ persists |
|
||||
|
||||
## Container Won't Start
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `exec format error` | Binary built on wrong arch | Rebuild on the Linux server |
|
||||
| `address already in use` | Port conflict | `ss -tlnp \| grep :PORT` to find offender |
|
||||
| `permission denied` | Missing capability or read-only root | Check `get_app_capabilities()`, add tmpfs |
|
||||
| `permission denied` | Missing capability, wrong UID ownership, or read-only root | Check capabilities, check volume ownership with mapped UID, add tmpfs |
|
||||
| `OCI runtime error` | Corrupt container state | `podman rm -f NAME && recreate` |
|
||||
| `image not known` | Image not pulled | `podman pull IMAGE:TAG` |
|
||||
| `no such network` | Network missing | `podman network create archy-net` |
|
||||
| `Error: netavark: ...subnet overlap` | Network CIDR conflict | `podman network rm archy-net && podman network create archy-net` |
|
||||
|
||||
## Container Starts But App Unreachable
|
||||
|
||||
@@ -20,6 +36,7 @@
|
||||
| Port mapped but refused | Container logs | App crashing internally — check logs |
|
||||
| Works sometimes | Resources | Check OOM kills, CPU, disk space |
|
||||
| 502 Bad Gateway | Nginx→Container | Wrong port in proxy_pass or container restarted |
|
||||
| Works locally but not from LAN | UFW forward policy | Set `DEFAULT_FORWARD_POLICY="ACCEPT"` in `/etc/default/ufw` |
|
||||
|
||||
## Container Keeps Dying
|
||||
|
||||
@@ -29,6 +46,8 @@
|
||||
| Dies after minutes | OOM killed | Increase `--memory` limit |
|
||||
| Dies when dep restarts | No restart policy | Add `--restart unless-stopped` |
|
||||
| Crash loop | Repeated crash | Fix root cause, don't just restart |
|
||||
| Exit code 127 | Missing binary in container | Wrong image tag or corrupted image — re-pull |
|
||||
| Exit code 137 | Killed by OOM or signal | Check `dmesg` for OOM kill, check `podman inspect` for OOMKilled |
|
||||
|
||||
## Network Issues
|
||||
|
||||
@@ -37,6 +56,20 @@
|
||||
| Can't resolve container names | Not on archy-net | Recreate with `--network=archy-net` |
|
||||
| Can't reach internet | DNS missing | Add `--dns 1.1.1.1` |
|
||||
| Container-to-container timeout | Different networks | Put both on same network |
|
||||
| Bitcoin RPC refused from container | rpcallowip wrong subnet | Use `rpcallowip=0.0.0.0/0` (safe: port mapped, not exposed) |
|
||||
| Old containers can't find new network | Subnet changed (rootful→rootless) | Recreate containers on new archy-net (rootless uses 10.89.x.x) |
|
||||
|
||||
## Volume Permission Patterns (Rootless UID Mapping)
|
||||
|
||||
Formula: **host_uid = 100000 + container_uid**
|
||||
|
||||
| Container UID | Host UID | Apps | Data Directory |
|
||||
|---|---|---|---|
|
||||
| 0 (root) | 100000 | lnd, fedimint, homeassistant, jellyfin, vaultwarden, photoprism, ollama, filebrowser, electrumx, btcpay, immich | `/var/lib/archipelago/{app}` |
|
||||
| 70 | 100070 | postgres (btcpay-db, immich-db, penpot-postgres) | `/var/lib/archipelago/postgres-*` |
|
||||
| 101 | 100101 | bitcoin-knots | `/var/lib/archipelago/bitcoin` |
|
||||
| 472 | 100472 | grafana | `/var/lib/archipelago/grafana` |
|
||||
| 999 | 100999 | MariaDB (mysql-mempool) | `/var/lib/archipelago/mysql-mempool` |
|
||||
|
||||
## Capability Reference
|
||||
|
||||
@@ -47,9 +80,23 @@
|
||||
| DAC_OVERRIDE | nextcloud, homeassistant, btcpay | Can't access cross-UID files |
|
||||
| FOWNER | bitcoin-knots, lnd, fedimint | Can't modify data dir perms |
|
||||
| NET_BIND_SERVICE | nginx-proxy-manager, vaultwarden | Can't bind ports <1024 |
|
||||
| NET_ADMIN + NET_RAW | tailscale | Can't create TUN device or manage routes |
|
||||
|
||||
## Read-Only Safe Apps
|
||||
|
||||
Only these 8 apps can run with `--read-only`: searxng, grafana, filebrowser, electrs, nostr-rs-relay, ollama, indeedhub
|
||||
Only these apps can run with `--read-only` + tmpfs: searxng, grafana, filebrowser, electrumx, mempool-electrs, electrs, nostr-rs-relay, ollama, indeedhub
|
||||
|
||||
All others need writable root or will fail silently.
|
||||
|
||||
## Systemd Sandbox Requirements for Rootless Podman
|
||||
|
||||
These systemd service settings MUST be configured for rootless Podman to work:
|
||||
|
||||
| Setting | Required Value | Why |
|
||||
|---------|---------------|-----|
|
||||
| `ProtectHome=` | `no` | Podman stores images in `~/.local/share/containers/` |
|
||||
| `PrivateTmp=` | `no` | Podman runtime lives in `/tmp/podman-run-1000/` |
|
||||
| `RestrictNamespaces=` | NOT SET | Rootless podman creates user namespaces |
|
||||
| `SystemCallFilter=` | NOT SET | Rootless podman needs clone/unshare syscalls |
|
||||
| `ReadWritePaths=` | Include `/var/lib/archipelago /run/user /tmp /etc/containers /var/lib/containers /run/containers` | Volume data + podman runtime paths |
|
||||
| `Environment=` | `XDG_RUNTIME_DIR=/run/user/1000` | Podman socket location |
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
# Rootless Podman UID Mapping Reference
|
||||
|
||||
## How Rootless UID Mapping Works
|
||||
|
||||
When Podman runs as the `archipelago` user (UID 1000), container processes don't run as their "apparent" UID on the host. Instead, Linux user namespaces remap UIDs.
|
||||
|
||||
**Mapping formula**: `host_uid = 100000 + container_uid`
|
||||
|
||||
This is configured in `/etc/subuid` and `/etc/subgid`:
|
||||
```
|
||||
archipelago:100000:65536
|
||||
```
|
||||
|
||||
This means:
|
||||
- Container UID 0 (root inside container) → Host UID 100000 (unprivileged on host)
|
||||
- Container UID 70 (postgres) → Host UID 100070
|
||||
- Container UID 101 (bitcoin) → Host UID 100101
|
||||
- etc.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Volume directories (bind mounts) on the host must be owned by the **mapped** UID, not the container UID. If Bitcoin runs as UID 101 inside its container, the host directory must be owned by UID 100101.
|
||||
|
||||
If ownership is wrong, the container gets `permission denied` when trying to read/write its data.
|
||||
|
||||
## Complete UID Mapping Table
|
||||
|
||||
| Container UID | Host UID | Containers | Fix Command |
|
||||
|---|---|---|---|
|
||||
| 0 (root) | 100000 | lnd, fedimint, fedimint-gateway, homeassistant, jellyfin, vaultwarden, photoprism, ollama, filebrowser, electrumx, btcpay-server, nbxplorer, immich, nostr-rs-relay, strfry, nextcloud, searxng, onlyoffice, tailscale, uptime-kuma | `sudo chown -R 100000:100000 /var/lib/archipelago/{app}` |
|
||||
| 70 | 100070 | postgres (btcpay-db, immich-db, penpot-postgres) | `sudo chown -R 100070:100070 /var/lib/archipelago/postgres-*` |
|
||||
| 101 | 100101 | bitcoin-knots, bitcoin-core | `sudo chown -R 100101:100101 /var/lib/archipelago/bitcoin` |
|
||||
| 472 | 100472 | grafana | `sudo chown -R 100472:100472 /var/lib/archipelago/grafana` |
|
||||
| 999 | 100999 | MariaDB (mysql-mempool) | `sudo chown -R 100999:100999 /var/lib/archipelago/mysql-mempool` |
|
||||
|
||||
## How to Find a Container's UID
|
||||
|
||||
If you encounter a new container with permission issues:
|
||||
|
||||
```bash
|
||||
# Check what user the container runs as
|
||||
podman inspect CONTAINER_NAME --format "{{.Config.User}}"
|
||||
|
||||
# If empty, it runs as root (UID 0) → host UID 100000
|
||||
|
||||
# If it shows a username, find the UID inside the image
|
||||
podman run --rm IMAGE_NAME id
|
||||
|
||||
# Then calculate: host_uid = 100000 + container_uid
|
||||
```
|
||||
|
||||
## Fix Script
|
||||
|
||||
Run this after any fresh install, migration, or when containers have permission errors:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Fix all rootless podman volume ownership
|
||||
|
||||
# UID 0 → 100000 (most containers)
|
||||
for dir in lnd fedimint fedimint-gateway homeassistant jellyfin vaultwarden photoprism \
|
||||
ollama filebrowser electrumx btcpay nbxplorer immich nostr-rs-relay nextcloud \
|
||||
searxng onlyoffice uptime-kuma; do
|
||||
[ -d "/var/lib/archipelago/$dir" ] && sudo chown -R 100000:100000 "/var/lib/archipelago/$dir"
|
||||
done
|
||||
|
||||
# UID 101 → 100101 (Bitcoin)
|
||||
[ -d "/var/lib/archipelago/bitcoin" ] && sudo chown -R 100101:100101 /var/lib/archipelago/bitcoin
|
||||
|
||||
# UID 70 → 100070 (PostgreSQL)
|
||||
for dir in /var/lib/archipelago/postgres-* /var/lib/archipelago/btcpay-db /var/lib/archipelago/immich-db; do
|
||||
[ -d "$dir" ] && sudo chown -R 100070:100070 "$dir"
|
||||
done
|
||||
|
||||
# UID 999 → 100999 (MariaDB)
|
||||
[ -d "/var/lib/archipelago/mysql-mempool" ] && sudo chown -R 100999:100999 /var/lib/archipelago/mysql-mempool
|
||||
|
||||
# UID 472 → 100472 (Grafana)
|
||||
[ -d "/var/lib/archipelago/grafana" ] && sudo chown -R 100472:100472 /var/lib/archipelago/grafana
|
||||
```
|
||||
|
||||
## Rootful vs Rootless Comparison
|
||||
|
||||
| Aspect | Rootful (old) | Rootless (current) |
|
||||
|--------|---------------|-------------------|
|
||||
| Podman command | `sudo podman` | `podman` (as archipelago user) |
|
||||
| Container storage | `/var/lib/containers/storage` | `~/.local/share/containers/storage` |
|
||||
| Container subnet | `10.88.0.0/16` | `10.89.0.0/16` |
|
||||
| Volume ownership | Container UID directly | Mapped UID (100000 + container_uid) |
|
||||
| Requires root? | Yes | No (except fixing volume ownership) |
|
||||
| XDG_RUNTIME_DIR | Not needed | Required: `/run/user/1000` |
|
||||
| User lingering | Not needed | Required: `loginctl enable-linger` |
|
||||
| Systemd restrictions | All can be enabled | Must disable: RestrictNamespaces, SystemCallFilter |
|
||||
Reference in New Issue
Block a user