test(gate): make 5× the canonical gate, drop 20x naming
Rename run-20x.sh → run-gate.sh, default ARCHY_ITERATIONS 20→5, and scrub 20× references across CLAUDE.md, the master plan, TESTING.md, app-registry status, the orchestrator/config doc-comments, and the bats suites. Also add a minimal fail() helper to mempool.bats so guard failures report cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
0f05f73a23
commit
57a013bc66
@@ -27,7 +27,7 @@ The migration's aim, restated as **five pillars** (every app must satisfy all fi
|
||||
3. **Lifecycle bulletproof** — every app passes the full matrix
|
||||
(install / UI reachable / stop / start / restart / reinstall / reboot-survive
|
||||
/ archipelago-restart-survive / uninstall) **5× green on .228** — run ON the node
|
||||
(`ARCHY_ITERATIONS=5`; temporarily reduced from 20×, restore before final ship).
|
||||
(`ARCHY_ITERATIONS=5`).
|
||||
(Multinode / fleet → `docs/multinode-testing-plan.md`, separate.)
|
||||
before any release.
|
||||
4. **Data-driven apps** — install/uninstall needs only the app's manifest +
|
||||
@@ -41,7 +41,7 @@ The migration's aim, restated as **five pillars** (every app must satisfy all fi
|
||||
owned by the service user. Security is king.
|
||||
|
||||
**Per-app definition of done:** all five pillars hold → lifecycle matrix 5×
|
||||
(for now; was 20×) green on .228 (run ON the node) → catalog/registry updated (`app-catalog/catalog.json`
|
||||
green on .228 (run ON the node) → catalog/registry updated (`app-catalog/catalog.json`
|
||||
+ `releases/app-catalog.json`, rebuilt image pushed to the mirror) → tracker
|
||||
cell ticked. Only then move to the next app. (Fleet/multinode verification is a
|
||||
separate pass → `docs/multinode-testing-plan.md`.)
|
||||
@@ -80,7 +80,7 @@ cost hours of resync.
|
||||
archipelago` → `cp` binary → `start`.
|
||||
4. Validate: install fedimint-gateway → assert `fedimint-gateway-hash` (0600,
|
||||
archipelago-owned) + `.pw` generated → container starts healthy.
|
||||
5. Run `tests/lifecycle/run-20x.sh` for the gateway (do NOT touch knots/electrumx/lnd).
|
||||
5. Run `tests/lifecycle/run-gate.sh` for the gateway (do NOT touch knots/electrumx/lnd).
|
||||
6. Frontend fixes (separate from binary): see icon/rename below; rebuild neode-ui,
|
||||
ship `dist + catalog.json + assets` to `/opt/archipelago/web-ui` (chown 1000:1000).
|
||||
|
||||
@@ -168,7 +168,7 @@ v1.7.52 tags.
|
||||
Three production failures shipped on v1.7.90-alpha despite the existing harness,
|
||||
because nothing exercised the receive path, port-mapping drift, or secret
|
||||
completeness on a live node. New suites close those gaps (all run on the archy
|
||||
host, read-only, so they join `run.sh`/`run-20x.sh` automatically):
|
||||
host, read-only, so they join `run.sh`/`run-gate.sh` automatically):
|
||||
|
||||
| Suite | Failure it guards | Asserts |
|
||||
|---|---|---|
|
||||
@@ -196,9 +196,9 @@ ARCHY_PASSWORD=password123 tests/lifecycle/run.sh
|
||||
# Full + destructive (for the verification fleet):
|
||||
ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 tests/lifecycle/run.sh
|
||||
|
||||
# 5× release-gate run (for now; was 20× — restore before final ship):
|
||||
# 5× release-gate run:
|
||||
ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ITERATIONS=5 \
|
||||
tests/lifecycle/run-20x.sh
|
||||
tests/lifecycle/run-gate.sh
|
||||
```
|
||||
|
||||
To exercise the Phase 3.2 Quadlet-backend path on a target node without
|
||||
@@ -228,7 +228,7 @@ Goal: minimum-viable container subsystem.
|
||||
| `core/container/src/bitcoin_simulator.rs` | 219 | 0 | -219 | ○ couples with dev_orchestrator |
|
||||
| `core/container/src/port_manager.rs` | 175 | 0 | -175 | ○ couples with dev_orchestrator |
|
||||
| `core/archipelago/src/api/rpc/package/install.rs::install_bitcoincoin_rpc_repair` | ~150 | 0 | -150 | ◐ pending fold into orchestrator pre-start |
|
||||
| imperative `install_fresh` in prod_orchestrator | ~120 | 0 | -120 | ◐ Phase 3.2 wired behind `use_quadlet_backends` flag (default off); 3.3 in-place migration ✅; 3.4 health-gated startup (`Notify=healthy`) ✅ + `TimeoutStartSec=600` race fix ✅; 3.4a unit drift-sync each reconcile ✅; flip default after 20× green |
|
||||
| imperative `install_fresh` in prod_orchestrator | ~120 | 0 | -120 | ◐ Phase 3.2 wired behind `use_quadlet_backends` flag (default off); 3.3 in-place migration ✅; 3.4 health-gated startup (`Notify=healthy`) ✅ + `TimeoutStartSec=600` race fix ✅; 3.4a unit drift-sync each reconcile ✅; flip default after 5× green |
|
||||
|
||||
**Today: -270 LoC committed. Outstanding deletes possible: ~1,616 LoC** (if Phase 3 ships fully + dev_mode resolved).
|
||||
|
||||
@@ -251,7 +251,7 @@ We don't have a performance harness yet. Add as L6 lands:
|
||||
v1.7.52 ships only when ALL of:
|
||||
|
||||
1. ☐ Bitcoin-stops fix verified live on a fresh node (tests/lifecycle/bats/bitcoin-knots.bats fully ● after a cold install)
|
||||
2. ☐ `ARCHY_ITERATIONS=5 tests/lifecycle/run-20x.sh` returns 0 **run ON .228** (5× for now; full suite, ARCHY_ALLOW_DESTRUCTIVE=1) — 1× is GREEN (110/110), 5× in progress
|
||||
2. ☐ `ARCHY_ITERATIONS=5 tests/lifecycle/run-gate.sh` returns 0 **run ON .228** (5× for now; full suite, ARCHY_ALLOW_DESTRUCTIVE=1) — 1× is GREEN (110/110), 5× in progress
|
||||
3. ☐ Multinode/fleet (.198 + others) — tracked separately in `docs/multinode-testing-plan.md`, NOT a v1.7.52 single-node gate item
|
||||
4. ☐ The L3 `backend-survives-archipelago-restart` suite passes (= Phase 3 Quadlet shipped for backends)
|
||||
5. ☐ Cargo: 0 warnings, 0 unused, all tests green (sustained ✓ since 1c0df95f)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
#
|
||||
# Lifecycle tests for the electrumx package (containers are named
|
||||
# `electrumx` + `archy-electrs-ui`). Mirrors bitcoin-knots.bats /
|
||||
# lnd.bats so the 20× release-gate run exercises electrumx through
|
||||
# lnd.bats so the 5× release-gate run exercises electrumx through
|
||||
# the same state matrix.
|
||||
#
|
||||
# Tiers:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# tests/lifecycle/bats/lnd.bats
|
||||
#
|
||||
# Lifecycle tests for the lnd package. Mirrors bitcoin-knots.bats so the
|
||||
# 20× release-gate run exercises lnd through the same state matrix.
|
||||
# 5× release-gate run exercises lnd through the same state matrix.
|
||||
#
|
||||
# Tiers:
|
||||
# - Read-only (always runs): presence, state-reporting consistency, RPC reachable
|
||||
|
||||
@@ -14,6 +14,11 @@
|
||||
|
||||
load '../lib/rpc.bash'
|
||||
|
||||
# bats-assert is not loaded in this suite (only rpc.bash), so provide a minimal
|
||||
# `fail` so the `|| fail "..."` guards below report a real assertion failure
|
||||
# instead of an undefined-command status 127 that masks the actual reason.
|
||||
fail() { echo "$@" >&2; return 1; }
|
||||
|
||||
setup_file() {
|
||||
: "${ARCHY_PASSWORD:?Set ARCHY_PASSWORD env var to the UI password}"
|
||||
export ARCHY_FORCE_LOGIN=1
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
# - container down → skip (clean dependency report, no false-fail)
|
||||
# - container up → URL MUST return 200 with non-empty body
|
||||
#
|
||||
# Looped 20× via tests/lifecycle/run-20x.sh.
|
||||
# Looped 5× via tests/lifecycle/run-gate.sh.
|
||||
|
||||
load '../lib/rpc.bash'
|
||||
load '../lib/ui-probes.bash'
|
||||
|
||||
@@ -1,32 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/lifecycle/run-20x.sh — loop the lifecycle harness N times.
|
||||
# tests/lifecycle/run-gate.sh — loop the lifecycle harness N times (default 5×, the release gate).
|
||||
#
|
||||
# Each iteration: setup-teardown → run.sh (with the same args you'd pass
|
||||
# to run.sh) → setup-teardown. Tallies pass/fail per iteration and prints a
|
||||
# summary at the end. Returns non-zero if any iteration failed.
|
||||
#
|
||||
# Env:
|
||||
# ARCHY_ITERATIONS (default: 20)
|
||||
# ARCHY_ITERATIONS (default: 5)
|
||||
# ARCHY_FAIL_FAST=1 stop on first failed iteration
|
||||
# plus everything run.sh / lib/rpc.bash respects
|
||||
# (ARCHY_PASSWORD, ARCHY_HOST, ARCHY_SCHEME, ARCHY_ALLOW_DESTRUCTIVE,
|
||||
# ARCHY_ALLOW_CASCADE_DESTRUCTIVE, ARCHY_ALLOW_NOAUTH)
|
||||
#
|
||||
# Usage:
|
||||
# tests/lifecycle/run-20x.sh # 20× full bats/ suite
|
||||
# ARCHY_ITERATIONS=5 tests/lifecycle/run-20x.sh # 5× full suite
|
||||
# tests/lifecycle/run-20x.sh bitcoin-knots # 20× a single suite
|
||||
# tests/lifecycle/run-gate.sh # 5× full bats/ suite
|
||||
# ARCHY_ITERATIONS=5 tests/lifecycle/run-gate.sh # 5× full suite
|
||||
# tests/lifecycle/run-gate.sh bitcoin-knots # 5× a single suite
|
||||
#
|
||||
# Suggested release-gate invocation:
|
||||
# ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 \
|
||||
# tests/lifecycle/run-20x.sh
|
||||
# tests/lifecycle/run-gate.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$HERE"
|
||||
|
||||
ITER="${ARCHY_ITERATIONS:-20}"
|
||||
ITER="${ARCHY_ITERATIONS:-5}"
|
||||
if ! [[ "$ITER" =~ ^[1-9][0-9]*$ ]]; then
|
||||
echo "ARCHY_ITERATIONS must be a positive integer, got: $ITER" >&2
|
||||
exit 2
|
||||
@@ -2,7 +2,7 @@
|
||||
# tests/lifecycle/setup-teardown.sh
|
||||
#
|
||||
# Cleanup helper used between lifecycle test iterations. Run before AND after
|
||||
# a full bats pass (run-20x.sh handles this). Idempotent — safe to run any
|
||||
# a full bats pass (run-gate.sh handles this). Idempotent — safe to run any
|
||||
# time, on any host.
|
||||
#
|
||||
# Removes:
|
||||
|
||||
Reference in New Issue
Block a user