fix(tor): make the helper's restart actually recover Tor, and stop lying about it

The dashboard's "Restart Tor" button dispatched to this helper and always
got {"ok":true} back. Two real defects, in order of importance — and one
disproved theory, recorded so nobody re-chases it:

1. reset-failed was missing. Once tor@default fails enough times systemd
   latches "Start request repeated too quickly" and refuses to start it at
   all; a plain restart is then a no-op no matter which unit you name.
   All three fleet nodes found dead on 2026-08-09 were in exactly that
   state, which is why the button appeared to do nothing.
2. The result was unconditional. The write-torrc branch waited up to 30s
   for SOCKS and then ignored the outcome; the restart branch slept 3s and
   claimed success. The UI reported "restarted" over a dead daemon.

Disproved: this was NOT wrong-unit targeting. `systemctl restart tor`
does propagate to tor@default — measured on austin-sapien, MainPID
changed. tor@default is still addressed explicitly because it is the
unit carrying the failed state worth resetting.

restart_tor_daemon() now: reset-failed, restart tor@default (fall back to
tor on single-instance installs), wait up to 30s for SOCKS on 9050, and
return {"ok":false,"error":...} pointing at journalctl when it never
comes up. Callers may no longer report success without a live SOCKS port.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-09 06:49:36 -04:00
co-authored by Claude Fable 5
parent 2090cef3bf
commit 24e378c421
+66 -18
View File
@@ -53,6 +53,54 @@ rm -f "$ACTION_FILE"
ACTION_TYPE=$(echo "$ACTION" | python3 -c "import sys,json; print(json.load(sys.stdin).get('action',''))" 2>/dev/null || echo "")
# Restart the daemon that actually serves, and report whether it really came up.
#
# Why the dashboard's "Restart Tor" button did nothing on 2026-08-09, in order
# of importance — note it was NOT wrong-unit targeting: `systemctl restart tor`
# does propagate to tor@default (measured on austin-sapien, MainPID changed).
#
# 1. RESET-FAILED WAS MISSING. Once tor@default has failed enough times
# systemd latches "Start request repeated too quickly" and refuses to start
# it at all; a plain restart is then a no-op no matter which unit you name.
# All three broken nodes were in exactly that state.
# 2. THE RESULT WAS ALWAYS {"ok":true}. This branch waited up to 30s for SOCKS
# and then ignored the outcome; the `restart` branch slept 3s and claimed
# success. So the UI reported "restarted" over a dead daemon.
# 3. The underlying torrc was unbindable, so every restart failed anyway —
# fixed separately in the torrc generator.
#
# tor@default is targeted explicitly because it is the unit that carries the
# state worth resetting; single-instance installs have only `tor`, hence the
# detection and fallback.
#
# Returns 0 only when SOCKS answers. Callers must not report success without it.
restart_tor_daemon() {
local unit=tor
if systemctl list-unit-files 'tor@*.service' 2>/dev/null | grep -q 'tor@'; then
unit=tor@default
fi
systemctl reset-failed "$unit" 2>/dev/null || true
if ! systemctl restart "$unit" 2>/dev/null; then
if [ "$unit" != tor ]; then
log "restart of $unit failed — falling back to tor.service"
systemctl reset-failed tor 2>/dev/null || true
systemctl restart tor 2>/dev/null || true
fi
fi
log "Restarted $unit"
local i
for i in $(seq 1 30); do
if timeout 1 bash -c 'echo > /dev/tcp/127.0.0.1/9050' 2>/dev/null; then
log "Tor SOCKS answering after ${i}s"
return 0
fi
sleep 1
done
log "ERROR: Tor SOCKS not answering 30s after restarting $unit"
return 1
}
case "$ACTION_TYPE" in
write-torrc-and-restart)
if [ ! -f "$TORRC_STAGED" ]; then
@@ -64,27 +112,27 @@ case "$ACTION_TYPE" in
chown debian-tor:debian-tor /etc/tor/torrc 2>/dev/null || true
log "torrc updated from staged file"
systemctl restart tor
log "Tor restarted"
# Wait for SOCKS port
for i in $(seq 1 30); do
if timeout 1 bash -c 'echo > /dev/tcp/127.0.0.1/9050' 2>/dev/null; then
break
fi
sleep 1
done
sync_hostnames
write_result '{"ok":true}'
if restart_tor_daemon; then
sync_hostnames
write_result '{"ok":true}'
else
# Never claim success on a dead daemon: the caller surfaces this straight
# to the operator, and a false "restarted" is how an outage stays hidden.
sync_hostnames
write_result '{"ok":false,"error":"Tor restarted but SOCKS never came up on 127.0.0.1:9050 — check journalctl -u tor@default"}'
exit 1
fi
;;
restart)
systemctl restart tor
log "Tor restarted"
sleep 3
sync_hostnames
write_result '{"ok":true}'
if restart_tor_daemon; then
sync_hostnames
write_result '{"ok":true}'
else
sync_hostnames
write_result '{"ok":false,"error":"Tor restarted but SOCKS never came up on 127.0.0.1:9050 — check journalctl -u tor@default"}'
exit 1
fi
;;
delete-service)