feat(lnd): rotate Lightning macaroons from the dashboard, and stop stranding BTCPay
Demo images / Build & push demo images (push) Successful in 3m34s
Demo images / Build & push demo images (push) Successful in 3m34s
Rotating LND's macaroons was an SSH-only script, which in practice meant it did not happen — while a macaroon is a bearer token with no revocation and no expiry, so anything that ever read one keeps the ability to spend until they are replaced. Settings → Lightning credentials now does it behind the node password, shows a step checklist, and refuses to report success unless it has confirmed the node identity and channel census are unchanged. Three findings from performing a real rotation on a dev node, each fixed here: 1. BTCPay was left holding a dead credential, silently. Its connection string carries the macaroon INLINE (LND's datadir is owned by its container subuid, so btcpay cannot bind-mount the file), and the daemon only regenerates that secret when LND's TLS cert thumbprint changes — which macaroon rotation does not touch. Result: btcpay up, LND up, both healthy, every Lightning payment failing, nothing anywhere saying why. 2. Rewriting the secret is not enough to fix it. `secret_env_hash` makes the change visible as env drift, but the reconcile loop runs `ExistingOnly` at boot AND periodically, and there it deliberately leaves running restart-sensitive apps untouched — observed once per tick for half an hour on the dev node. So this reuses FED-07's `credential_rotated` carve-out via a new default-no-op `ContainerOrchestrator::mark_credential_rotated`, on the same reasoning: restart sensitivity protects apps that are working, and this one is working only in appearance. The shell script cannot reach an in-process flag, so it removes the container and lets desired-state recovery rebuild it. 3. LND stayed locked forever on a loaded node. The unlocker is only served after channel.db/graph.db/wallet.db open, measured at 2m38s on a box running 30 containers; the unlock helper gave up at ~60s. That is not a harmless retry — reconcile records the post-start hook as failed, restarts LND, and the slow open begins again, so the wallet never opens and every LND-dependent app stays broken. The not-ready budget is now ~10 minutes; a genuinely wrong password still exits on the first pass via `all_rejected`. Safety properties worth not regressing: - No macaroon content in any response, error, log line or the polled progress feed — digests and byte counts only. - Rotation unlocks via a new `unlock_existing_wallet_no_wipe`, so there is no code path from "rotate my credentials" to `recreate_wallet_destructively`. A wallet whose password this node lacks fails the rotation with the wallet intact. - Channels are compared as active+inactive totals, not `num_active_channels`, which legitimately dips after any restart while peers reconnect. - Backup verified by file count before anything is deleted. Verified: cargo check + fmt clean, 6 new unit tests and the 6 existing container::lnd tests pass, vue-tsc clean, and the built bundle contains the three new RPC method names (the frontend build can silently no-op). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
b7e57ca9cf
commit
d15cd58d7f
@@ -268,6 +268,57 @@ if [ -n "$FAIL" ]; then
|
||||
die "rotation verification FAILED:$FAIL — old material is in $BACKUP"
|
||||
fi
|
||||
|
||||
# ── BTCPay's inline copy ──────────────────────────────────────────────
|
||||
# BTCPay reaches the internal LND node with a connection string that carries
|
||||
# the macaroon INLINE as hex, not as a file path: LND's datadir is owned by its
|
||||
# container's mapped uid, so btcpay cannot bind-mount the file. That copy is
|
||||
# therefore now a dead credential, and nothing else will notice — the daemon
|
||||
# only regenerates this secret when LND's TLS *cert* thumbprint changes, which
|
||||
# macaroon rotation does not touch. The node keeps looking healthy (btcpay up,
|
||||
# LND up) while every Lightning invoice BTCPay tries to create fails.
|
||||
#
|
||||
# Deleting the secret file gets the daemon to regenerate it from the new
|
||||
# macaroon on its next reconcile tick. That is necessary but NOT sufficient, and
|
||||
# the difference matters: the RUNNING container still holds the dead value, and
|
||||
# the periodic reconciler only ever runs in `ExistingOnly` mode, where env drift
|
||||
# on a restart-sensitive app (btcpay-server is one) is detected and then
|
||||
# deliberately skipped — "leaving running restart-sensitive app untouched". So
|
||||
# the container has to be recreated on purpose. The dashboard path
|
||||
# (Settings → Lightning credentials) does this itself by flagging the app as
|
||||
# credential-rotated; a shell script cannot reach that in-process flag, so it
|
||||
# removes the container instead and lets the orchestrator's own desired-state
|
||||
# recovery rebuild it around unchanged data, ports and volumes.
|
||||
#
|
||||
# Nothing is printed but a path — never the value.
|
||||
BTCPAY_SECRET="/var/lib/archipelago/secrets/btcpay-lnd-connection"
|
||||
BTCPAY_NOTE=no
|
||||
if sudo test -f "$BTCPAY_SECRET"; then
|
||||
if sudo rm -f "$BTCPAY_SECRET"; then
|
||||
say
|
||||
say "btcpay : removed its stale connection string ($BTCPAY_SECRET)."
|
||||
say " The daemon regenerates it from the new macaroon within a minute."
|
||||
BTCPAY_NOTE=yes
|
||||
if podman container exists btcpay-server 2>/dev/null; then
|
||||
say " Recreating btcpay-server so it stops using the dead one."
|
||||
podman stop btcpay-server >/dev/null 2>&1 || true
|
||||
if podman rm -f btcpay-server >/dev/null 2>&1; then
|
||||
say " Removed; the orchestrator rebuilds it around its existing"
|
||||
say " data (it was running, so desired-state recovery restores it)."
|
||||
else
|
||||
say " ⚠ could not remove btcpay-server. Its Lightning payments will"
|
||||
say " fail until it is recreated."
|
||||
BTCPAY_NOTE=warn
|
||||
fi
|
||||
fi
|
||||
else
|
||||
say
|
||||
say "btcpay : ⚠ could not remove $BTCPAY_SECRET. BTCPay is still holding"
|
||||
say " the OLD macaroon, so its Lightning payments will fail until"
|
||||
say " that file is deleted and btcpay-server is recreated."
|
||||
BTCPAY_NOTE=warn
|
||||
fi
|
||||
fi
|
||||
|
||||
say
|
||||
say "✅ Rotated. Every macaroon issued before now no longer verifies."
|
||||
say
|
||||
@@ -278,6 +329,13 @@ say " and scan the new pairing QR; it serves the new macaroon."
|
||||
say
|
||||
say " Your funds and channels are untouched: the node kept its identity and"
|
||||
say " no channel was closed."
|
||||
if [ "${BTCPAY_NOTE:-no}" != no ]; then
|
||||
say
|
||||
say " CONFIRM BTCPAY CAME BACK. A silent failure here looks identical to success:"
|
||||
say " btcpay stays up and healthy while every Lightning payment it tries fails."
|
||||
say " podman inspect btcpay-server --format '{{.Created}}' # should be just now"
|
||||
say " sudo test -f $BTCPAY_SECRET && echo regenerated"
|
||||
fi
|
||||
say
|
||||
say " Once every client is re-paired, delete the backup — it holds the OLD"
|
||||
say " root key, which is still sensitive:"
|
||||
|
||||
Reference in New Issue
Block a user