fix(stop): honour per-app graceful-stop grace in orchestrator stop path

package.stop left slow-to-SIGTERM apps (fedimint/electrumx/bitcoin/btcpay/immich)
running: the orchestrator path hardcoded podman API ?t=10 / CLI -t 30 and the CLI
wrapper deadline (30s) equalled the -t grace, so the await fired exactly as podman
SIGKILLed -> stop reported failed -> state reverted to running. Reproduced live on
clean .198 (fedimint).

- container/runtime.rs: add ContainerRuntime::stop_container_with_grace (defaulted
  so mock/dev impls are unchanged); PodmanRuntime honours grace for API + CLI with
  deadline = grace + 15s buffer; AutoRuntime delegates. New canonical per-app table
  stop_grace_secs_for() + DEFAULT_STOP_GRACE_SECS / STOP_GRACE_DEADLINE_BUFFER_SECS.
- podman_client.rs: stop_container_with_grace uses ?t=<grace> + longer HTTP deadline.
- prod_orchestrator::stop: resolve grace = manifest stop_grace_secs (north-star) else
  the table; pass to quadlet::stop_service_with_timeout AND stop_container_with_grace.
- quadlet.rs: stop_service_with_timeout so slow apps aren't SIGKILLed at 45s.
- rpc/package/runtime.rs: doc-note its &str stop_timeout_secs mirrors the canonical table.
- tests: resolve_stop_grace_secs (manifest field wins / table fallback / default 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-06-22 06:59:40 -04:00
co-authored by Claude Opus 4.8
parent 470e3c649a
commit 2dad64b2ee
5 changed files with 152 additions and 8 deletions
+13 -2
View File
@@ -422,11 +422,22 @@ impl PodmanClient {
}
pub async fn stop_container(&self, name: &str) -> Result<()> {
self.stop_container_with_grace(name, 10).await
}
/// Stop via libpod honouring a per-app grace (seconds). The HTTP deadline is
/// kept above the grace so the post-grace SIGKILL lands before we give up —
/// otherwise slow-to-SIGTERM apps (fedimint, bitcoin-core, electrumx…) time
/// out at exactly the grace boundary and the stop is reported as failed.
pub async fn stop_container_with_grace(&self, name: &str, grace_secs: u64) -> Result<()> {
let deadline = std::time::Duration::from_secs(
grace_secs + crate::runtime::STOP_GRACE_DEADLINE_BUFFER_SECS,
);
self.api_request(
"POST",
&format!("libpod/containers/{}/stop?t=10", name),
&format!("libpod/containers/{}/stop?t={}", name, grace_secs),
None,
DEFAULT_TIMEOUT,
deadline,
)
.await
.map(|_| ())