lfg2025/archy
1
0
Fork 0
Code Issues 24 Pull Requests 4 Actions Packages Projects Releases 72 Wiki Activity

docs(troubleshooting): fix advice that doesn't match the node

Browse Source 
Narrative pass over troubleshooting.md against the code. Seven claims were
wrong, several of them actively misleading:

- **Tor is not a container.** §15/§16 told operators to run
  `podman ps --filter name=tor` / `podman restart tor` and to read
  `/var/lib/archipelago/tor/hidden_service/hostname`. Tor is the host's Debian
  package running as `debian-tor`; Archipelago drives it by staging a torrc and
  poking `archipelago-tor-helper` (`scripts/tor-helper.sh`, which does
  `systemctl restart tor`). The hidden-service dir is
  `hidden_service_archipelago` (suffixed), it's root-owned 0700, and the file a
  normal user can actually read is the synced copy at
  `/var/lib/archipelago/tor-hostnames/<service>`.
- **The USB installer has no "Repair" mode.** Cited three times as the recovery
  path. The boot menu has exactly three entries: Install, Install (verbose),
  Boot from local disk. Replaced with what those entries can actually do, plus
  the fact that the installer prompts for a disk and requires typing `yes`, so
  booting it isn't itself destructive.
- **`bitcoin-cli -datadir=/data`** — the container's datadir is
  `/home/bitcoin/.bitcoin` and RPC creds are in a generated `/tmp/rpc.conf`;
  the documented command could not have authenticated.
- **"edit bitcoin.conf to add addnode="** — the entrypoint passes an explicit
  `-conf` and logs "ignoring legacy datadir bitcoin.conf". Flags come from the
  manifest (and the signed catalog entry that overrides it).
- **"Bitcoin requires 600GB+"** — only above the manifest's 1000 GB threshold;
  below it the node runs pruned at `-prune=550`.
- **`sudo systemctl restart podman`** — apps run under rootless Podman as the
  `archipelago` user, so that restarts an unrelated root socket.
- **"Settings > Network"** — DNS config and disk cleanup are both on the Server
  page (`/server`), not Settings.

Also: header claimed "the 20 most common issues" over 21 sections, and §16
presented Tor as required for peering when it's the last fallback after
mesh → LAN → FIPS.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-07 20:26:19 -04:00
co-authored by Claude Opus 5
parent e7d8dfb633
commit 2ed6c71e0c
1 changed files with 67 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/troubleshooting.md
+67 -27
View File
@@ -1,6 +1,6 @@
# Archipelago Troubleshooting Guide
This guide covers the 20 most common issues you may encounter with Archipelago, along with diagnostic commands and solutions.
This guide covers the most common issues you may encounter with Archipelago, along with diagnostic commands and solutions.
## Connection & Access
@@ -111,9 +111,12 @@ df -h /var/lib/archipelago
**Solutions**:
- If the image is missing: reinstall the app from the Marketplace
- If disk is full: run disk cleanup from Settings, or manually `podman system prune`
- If disk is full: run disk cleanup from the **Server** page (`/server`), or manually `podman system prune`
- If the container exits immediately: check logs for the root cause (usually missing config or permissions)
- Restart podman: `sudo systemctl restart podman`
- Restart the Podman socket. Archipelago runs **rootless** Podman as the
`archipelago` user, so this is a `--user` unit — `sudo systemctl restart podman`
would restart the unrelated root socket:
`systemctl --user restart podman.socket`
### 6. App shows "unhealthy" status
@@ -146,20 +149,34 @@ podman logs <app-id> --tail 100 | grep -i error
# Check Bitcoin logs
podman logs bitcoin-knots --tail 50
# Check if Bitcoin is connected to peers
podman exec bitcoin-knots bitcoin-cli -datadir=/data getpeerinfo | grep -c '"addr"'
# Check if Bitcoin is connected to peers.
# The datadir inside the container is /home/bitcoin/.bitcoin, and the RPC
# credentials live in the generated /tmp/rpc.conf (the manifest's entrypoint
# writes it from the BITCOIN_RPC_USER/BITCOIN_RPC_PASS secrets) — bitcoin-cli
# needs both flags or it can't authenticate.
podman exec bitcoin-knots bitcoin-cli \
-datadir=/home/bitcoin/.bitcoin -conf=/tmp/rpc.conf \
getpeerinfo | grep -c '"addr"'
# Check sync progress
podman exec bitcoin-knots bitcoin-cli -datadir=/data getblockchaininfo | grep -E "blocks|headers|verificationprogress"
podman exec bitcoin-knots bitcoin-cli \
-datadir=/home/bitcoin/.bitcoin -conf=/tmp/rpc.conf \
getblockchaininfo | grep -E "blocks|headers|verificationprogress"
```
**Solutions**:
- Initial sync takes 1-7 days depending on hardware — be patient
- Ensure the server has a stable internet connection
- Check disk space: Bitcoin requires 600GB+ for full chain
- Check disk space. The manifest picks the mode from the disk it's given: under
1000 GB it runs **pruned** (`-prune=550`, a few GB); at 1000 GB or more it runs
a full `-txindex=1` archival node, which needs 600 GB+ and growing
- If stuck: restart the container `podman restart bitcoin-knots`
- If peers = 0: check firewall allows port 8333 outbound
- Add manual peers: edit bitcoin.conf to add `addnode=` entries
- Editing `bitcoin.conf` in the data directory has **no effect** — the
entrypoint runs bitcoind with an explicit `-conf=/tmp/rpc.conf` and logs
"ignoring legacy datadir bitcoin.conf". Flags come from the app manifest, so
persistent changes belong there (and, for catalog-covered apps, in the signed
catalog entry that overrides the on-disk manifest)
### 8. LND won't connect to Bitcoin
@@ -254,7 +271,9 @@ df -h /
- Ensure at least 2GB free disk space
- If update hangs: wait 10 minutes, then restart the backend
- Do NOT power off during an update — this can corrupt the system
- If system is in a bad state after failed update: boot from the USB installer and select "Repair"
- If the system is in a bad state after a failed update, recover over SSH — the
USB installer has no repair mode (its boot menu offers only "Install
Archipelago", "Install Archipelago (verbose)" and "Boot from local disk")
### 12. Server won't boot after update
@@ -263,8 +282,11 @@ df -h /
**Solutions**:
- Wait 5 minutes — the first boot after update may take longer
- If still unresponsive: connect a monitor/keyboard to check boot messages
- Try the recovery mode: boot from USB installer and select "Repair"
- As a last resort: reflash the USB and restore from backup
- If it's a bootloader problem rather than a disk problem, boot the USB and pick
"Boot from local disk" to chainload the installed system
- As a last resort: reinstall from USB and restore from backup. The installer is
interactive — it asks for the target disk and requires typing `yes` — so
booting it does not by itself destroy the existing install
---
@@ -312,21 +334,35 @@ xrandr --query 2>/dev/null || echo "No display server"
**Symptoms**: Settings shows "Tor: Not configured" or the .onion address is missing
Tor is **not** a container — it's the host's Debian `tor` package, running as
`debian-tor`. Archipelago never touches it directly: it stages a torrc and asks
`archipelago-tor-helper` (a `.path` unit watching
`/var/lib/archipelago/tor-config/tor-action`) to install it and restart Tor.
**Diagnosis**:
```bash
# Check Tor container
podman ps --filter "name=tor"
podman logs tor --tail 20
# Check the host Tor service and the helper that drives it
sudo systemctl status tor
sudo journalctl -u archipelago-tor-helper --since "10 minutes ago"
# Check if Tor hostname file exists
cat /var/lib/archipelago/tor/hidden_service/hostname 2>/dev/null
# Is the SOCKS port up? (this is the liveness check the backend itself uses)
nc -z 127.0.0.1 9050 && echo "Tor SOCKS OK"
# The readable hostname copy the backend actually reads
cat /var/lib/archipelago/tor-hostnames/archipelago
# The hidden-service dir itself (root-owned 0700 — needs sudo)
sudo cat /var/lib/tor/hidden_service_archipelago/hostname 2>/dev/null \
|| sudo cat /var/lib/archipelago/tor/hidden_service_archipelago/hostname
```
**Solutions**:
- Tor takes 30-60 seconds to bootstrap — wait and refresh
- If Tor container is stopped: start it from the Apps page
- Check that the Tor data directory exists and has correct permissions
- Restart Tor: `podman restart tor`
- If `/var/lib/archipelago/tor-hostnames/archipelago` is missing but the
hidden-service dir has a `hostname`, the readable copy didn't sync — the
helper's `sync-hostnames` action rewrites it
- Check that the Tor data directory exists and is owned by `debian-tor`
- Restart Tor: `sudo systemctl restart tor`
### 16. Peers can't reach my node
@@ -334,11 +370,11 @@ cat /var/lib/archipelago/tor/hidden_service/hostname 2>/dev/null
**Diagnosis**:
```bash
# Check if Tor is running (needed for peer connectivity)
podman ps --filter "name=tor"
# Check if Tor is running (the fallback transport for peer connectivity)
sudo systemctl status tor
# Check your Tor address
cat /var/lib/archipelago/tor/hidden_service/hostname
cat /var/lib/archipelago/tor-hostnames/archipelago
# Test connectivity from the server side
curl -s http://localhost:5678/rpc/v1 \
@@ -347,10 +383,13 @@ curl -s http://localhost:5678/rpc/v1 \
```
**Solutions**:
- Ensure Tor is running (required for peer-to-peer communication)
- Tor is the last-resort transport, not the only one: peering prefers mesh
radio, then LAN, then FIPS, and only falls back to Tor. A peer stuck on
"unreachable" with Tor healthy usually means the higher transports are all
down too — check the FIPS anchor first
- Tor circuits can be slow — connections may take 30+ seconds
- Share your correct .onion address with peers
- Both nodes must have Tor running and be on the same federation
- Both nodes must be on the same federation
### 17. DNS resolution issues
@@ -370,7 +409,7 @@ podman exec bitcoin-knots nslookup seed.bitcoin.sipa.be
```
**Solutions**:
- Configure DNS from Settings > Network: try Cloudflare (1.1.1.1) or Google (8.8.8.8)
- Configure DNS from the **Server** page (`/server`): try Cloudflare (1.1.1.1) or Google (8.8.8.8)
- If using custom DNS, verify the server addresses are correct
- Restart networking: `sudo systemctl restart systemd-resolved`
@@ -418,7 +457,7 @@ podman system df
```
**Solutions**:
- Run disk cleanup from Settings
- Run disk cleanup from the **Server** page (`/server`)
- Remove unused app data: `podman system prune -a` (WARNING: removes all stopped containers and unused images)
- Move Bitcoin data to external drive if chain data is too large
- Check for large log files: `du -sh /var/log/*/ | sort -rh`
@@ -560,7 +599,8 @@ If the system is completely unresponsive:
1. **Power cycle**: Hold power button for 10 seconds, then turn back on
2. **Wait 5 minutes**: Services take time to start, especially if containers need to recover
3. **SSH in**: If web UI is down but SSH works, restart services manually
4. **USB recovery**: Boot from the Archipelago USB installer and select "Repair"
4. **Chainload the installed system**: Boot the Archipelago USB and pick "Boot
from local disk" — this rules out a broken bootloader
5. **Clean install + restore**: As last resort, do a fresh install and restore from backup
### Collecting Diagnostic Information