lfg2025/archy

Fork 0

Files

T

archipelagoandClaude Opus 5 8e814ca06a

Demo images / Build & push demo images (push) Failing after 2m22s

Details

feat(registry): move image and OTA references to the public domain

Replaces the registry host across 86 files: 309 references, covering all 40
app manifests, the orchestrator and container crates, the release and catalog
scripts, both demo-images workflows, the ISO builder, demo-deploy, and the
frontend marketplace data.

Verified the domain actually serves the registry before rewriting anything,
rather than assuming the web host implies the registry:
- TLS verifies clean, HTTP/2 on the web root
- an anonymous token grants a manifest fetch (HTTP 200) with no credentials
- skopeo inspect --no-creds resolves an image and lists its tags

That last check is the one that matters: an outside developer with no account
can now pull, which was the functional blocker for publishing at all.

Plain-HTTP references become HTTPS in the same pass, so OTA downloads stop
crossing the network in the clear.

Deliberately NOT rewritten:
- The public FIPS anchor on port 8444. It is a functional network endpoint
  every node dials to bootstrap the mesh — closer to Bitcoin Core's hardcoded
  seeds than to leaked infrastructure. The domain does resolve to the same
  host, so it could become a hostname, but that adds a DNS dependency to the
  path used precisely when things are broken. Worth a deliberate decision,
  not a side effect of this change.
- The companion APK on port 2100. The domain returns 404 for that path, so
  rewriting it would swap a working URL for a broken one. The Releases page
  does serve (200), which is where the plan already wants those binaries.
- releases/app-catalog.json, releases/manifest.json and release-manifest.json.
  These carry `signature` and `signed_by`; editing their contents invalidates
  the signature and the fleet refuses artifacts that fail verification. They
  were rewritten in a first pass and reverted — they must be regenerated and
  re-signed through the signing ceremony instead, which needs the mnemonic.

So the catalog still advertises the old host until that ceremony runs. Nodes
resolve images through the signed catalog, not the on-disk manifests, so this
commit alone does not change what a node pulls.

Verified: archipelago-container 75/75; every manifest still parses with a
top-level app block; no signed artifact modified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-07 11:31:20 -04:00

8.3 KiB

Raw Blame History

Bitcoin Multi-Version — Bulletproofing & Rollout (handoff)

Status 2026-06-29: code + images + catalog + frontend DONE on branch bitcoin-version-bulletproof (base commit 095a76cd, plus the catalog-generator

handoff follow-ups). .228 is the test node: binary + frontend + catalog are live there; its Knots chainstate is mid-reindex recovery (see §5). The fleet rollout (OTA binary+frontend, mirror catalog publish, :latest repoint) is the coordinated step the other agent owns — see §4. Pairs with docs/bitcoin-multi-version-design.md (the original design).

1. What was broken (root causes)

User report: "switched Knots to v29.3.knots20260508, version didn't update in the UI." Three stacked bugs, plus a data-corruption hazard:

Reconciler reverted the pin. prod_orchestrator::sync_quadlet_unit re-rendered the quadlet every reconcile tick using the manifest's :latest, ignoring the per-app pinned version → any switch silently reverted within one tick.
Entrypoint render bug. The renderer folded the manifest entrypoint: ["sh","-lc"] into Exec=. That only works when the image ENTRYPOINT is a passthrough shell wrapper. The versioned images use ENTRYPOINT ["bitcoind"], so Exec=sh -lc … became bitcoind sh -lc … → unexpected token 'sh' → crash loop.
Image USER divergence. The versioned images were built USER bitcoin (uid 1000); the legacy :latest ran as root. Chain data is owned by the data_uid (host 100101 / container uid 102). Root reads it via CAP_DAC_OVERRIDE (granted in the manifest); uid-1000 cannot → Error initializing block database.
Data hazard (already hit on .228). Repeated failed starts under mixed UIDs left bitcoind's two LevelDBs (blocks/index/ + chainstate/) truncated to KB stubs while the raw blocks/blk*.dat (797 GB) stayed intact. Recovery = bitcoind -reindex from local blocks (no re-download). The uniform-root image fix (below) removes the mixed-UID cause going forward; the proper switch flow was already data-safe (600s stop grace, clean stop→rm→recreate, conflict-stops the other impl — they share port 8332 + datadir /var/lib/archipelago/bitcoin).

2. What was fixed (all on the branch)

Renderer (core/archipelago/src/container/):
- prod_orchestrator.rs: factored resolve_catalog_image() (catalog/pinned-version → image) and call it in BOTH install_fresh and sync_quadlet_unit — the pin now survives reconcile.
- quadlet.rs: emit a real Entrypoint=<first> + Exec=<rest+cmd> instead of folding; exec_changed now also diffs Entrypoint= so the recreate fires. Validated against the live podman 5.4.2 quadlet generator.
Images (scripts/build-bitcoin-image.sh, apps/bitcoin-{knots,core}/Dockerfile): removed USER bitcoin → run as container-root like legacy (still 100% rootless: container-root maps to the unprivileged host service user; CAP_DAC_OVERRIDE from the manifest lets bitcoind read the data_uid-owned datadir). All images rebuilt root + pushed to the mirror (source.archipelago-foundation.org/lfg2025):
- Knots: 29.3.knots20260508, 29.3.knots20260507, 29.3.knots20260210, 29.2.knots20251110
- Core: 25.2 26.2 27.2 28.4 29.2 29.3 30.2 31.0 + latest (→31.0)
Catalog (scripts/generate-app-catalog.sh VERSIONS map + regenerated releases/app-catalog.json): Knots & Core versions[] populated; the generator now forces top-level version == the default entry's version (the 169ff2e2 invariant) regardless of the manifest version. Knots latest entry points at the newest dated image (29.3.knots20260508) so "Always use latest" = newest on fixed-binary nodes.
Frontend (neode-ui/):
- AppSidebar.vue: rename the latest option to "Always use the latest version" (no v prefix), fix right padding, and pickSelection() guarantees the bound value is a real option (fixes the blank dropdown).
- New components/InstallVersionModal.vue: full-screen version chooser shown from the App Store / Discover card install button for multi-version apps — app icon + "Install ", latest pre-selected. Wired in Discover.vue handleInstall.
- i18n keys: appDetails.alwaysUseLatestVersion, marketplace.installModalTitle/Hint.

3. Current live state on .228 (test node)

Binary with both renderer fixes: deployed (/usr/local/bin/archipelago).
New frontend bundle: deployed to /opt/archipelago/web-ui (hard-refresh to see it).
Updated catalog: placed at /var/lib/archipelago/app-catalog.json (local override — will refresh from the mirror's OLDER copy at the next hourly fetch until §4 publishes it).
Knots: bitcoin-knots service held stopped (package.stop, user_stopped); a detached bitcoin-knots-reindex container is rebuilding the index+UTXO (§5).

4. Remaining — coordinated fleet rollout (OTHER AGENT)

Do this together with the other workstream's release, AFTER both are ready:

Merge branch bitcoin-version-bulletproof into the release line.
Build + OTA the binary + frontend (these carry the renderer fix + UI). The renderer fix is a hard prerequisite for the new images everywhere — see fleet-safety below.
Publish the catalog to the mirror (push releases/app-catalog.json to gitea-vps2 main, the raw URL nodes fetch hourly). The current catalog is fleet-safe even before the binary lands: unpinned/auto-update nodes resolve via the manifest's floating :latest (still the legacy image); only explicit version selection (needs the new UI) uses the new root images.
Only AFTER the binary is fleet-wide: optionally repoint the bitcoin-knots:latest tag → 29.3.knots20260508 (root) and simplify the catalog latest entry back to the :latest tag. Do NOT repoint :latest before then — old-binary nodes fold Exec=sh -lc … and would crash on an ENTRYPOINT ["bitcoind"] image. (Core never worked on old binaries — it always shipped ENTRYPOINT ["bitcoind"] — so Core has no such constraint.)
Verify the full switch matrix on a healthy node (§6).

5. Finishing .228's reindex (OTHER AGENT owns this — not babysat by the original author)

The detached bitcoin-knots-reindex container runs the new root 29.3.knots20260508 image with -reindex -server=0 against /var/lib/archipelago/bitcoin. It holds the datadir lock, so the managed service (held stopped) can't collide. When it has connected blocks up to ~the prior tip (height ≥ ~955800) it's done; then:

# on .228 (SSH/sudo/UI pw all: <FLEET_PW>)
podman stop -t 600 bitcoin-knots-reindex && podman rm bitcoin-knots-reindex
# start the managed service via RPC (sets desired=running, clears user_stopped):
#   package.start {id: bitcoin-knots}  (POST https://127.0.0.1/rpc/v1, CSRF: echo csrf_token cookie as X-CSRF-Token)
# verify:
podman exec bitcoin-knots sh -lc '$(command -v bitcoind) --version | head -1'   # → v29.3.knots20260508
# RPC up → the Bitcoin UI populates; it syncs the gap to tip.

The "Bitcoin RPC connection refused (127.0.0.1:8332)" the UI shows is EXPECTED until this swap (reindex runs with RPC off).

6. Switch-matrix test plan (what "bulletproof" must prove)

On a healthy node, each step must end with bitcoind running + RPC answering + syncing, with NO Error initializing block database and NO data loss:

Knots: switch latest → 29.3.knots20260507 → 29.3.knots20260210 → back to latest.
Core: install latest; switch 31.0 → 28.4.0.
Knots ↔ Core (shared datadir/port): Knots→Core upgrade path (Core ≥ data version) and the reverse. Cross-major DOWNGRADES (e.g. 29.x data → Core 28.4) legitimately need a reindex — the UI already surfaces a downgrade warning; confirm it does and that confirming reindexes cleanly rather than crash-looping.
Reboot survival after each switch.

7. Notes / assumptions

"29.2" in the request doesn't exist as a Knots build (404 upstream); added as Bitcoin Core 29.2 (exists). Revisit if a Knots 29.2 was meant.
Reindex is unavoidable ONLY because .228's index was already corrupted by the pre-fix crash loop; a normal switch on the fixed binary does NOT reindex.
Creds for .228: SSH/sudo + UI/RPC all <FLEET_PW>.

8.3 KiB Raw Blame History