lfg2025/archy

Fork 0

Files

T

archipelagoandClaude Opus 5 8e814ca06a

Demo images / Build & push demo images (push) Failing after 2m22s

Details

feat(registry): move image and OTA references to the public domain

Replaces the registry host across 86 files: 309 references, covering all 40
app manifests, the orchestrator and container crates, the release and catalog
scripts, both demo-images workflows, the ISO builder, demo-deploy, and the
frontend marketplace data.

Verified the domain actually serves the registry before rewriting anything,
rather than assuming the web host implies the registry:
- TLS verifies clean, HTTP/2 on the web root
- an anonymous token grants a manifest fetch (HTTP 200) with no credentials
- skopeo inspect --no-creds resolves an image and lists its tags

That last check is the one that matters: an outside developer with no account
can now pull, which was the functional blocker for publishing at all.

Plain-HTTP references become HTTPS in the same pass, so OTA downloads stop
crossing the network in the clear.

Deliberately NOT rewritten:
- The public FIPS anchor on port 8444. It is a functional network endpoint
  every node dials to bootstrap the mesh — closer to Bitcoin Core's hardcoded
  seeds than to leaked infrastructure. The domain does resolve to the same
  host, so it could become a hostname, but that adds a DNS dependency to the
  path used precisely when things are broken. Worth a deliberate decision,
  not a side effect of this change.
- The companion APK on port 2100. The domain returns 404 for that path, so
  rewriting it would swap a working URL for a broken one. The Releases page
  does serve (200), which is where the plan already wants those binaries.
- releases/app-catalog.json, releases/manifest.json and release-manifest.json.
  These carry `signature` and `signed_by`; editing their contents invalidates
  the signature and the fleet refuses artifacts that fail verification. They
  were rewritten in a first pass and reverted — they must be regenerated and
  re-signed through the signing ceremony instead, which needs the mnemonic.

So the catalog still advertises the old host until that ceremony runs. Nodes
resolve images through the signed catalog, not the on-disk manifests, so this
commit alone does not change what a node pulls.

Verified: archipelago-container 75/75; every manifest still parses with a
top-level app block; no signed artifact modified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-07 11:31:20 -04:00

17 KiB

Raw Blame History

Bitcoin Multi-Version Support — Design

Status: design (2026-06-22) Goal: let a user choose which version of Bitcoin Core / Bitcoin Knots to install (latest pre-selected, older versions in a dropdown), and later switch versions or opt into auto-update — all manifest/catalog-driven, all served from our signed registry, rootless, with zero data loss across version changes.

See also: docs/registry-manifest-design.md (catalog distribution + signing this builds on), the production test gate (which must be green first), MEMORY → project_decoupled_app_updates, MEMORY → project_manifest_driven_north_star.

Scheduling: this is net-new scope. It lands after the production test gate (tests/lifecycle/run-20x.sh) is green on .228 + .198. The data- preservation invariant (downgrade vs. chainstate) is the highest risk here.

1. Where we are today

Image source / build

Thing	Today
`apps/bitcoin-core/Dockerfile`	`FROM bitcoin/bitcoin:24.0` — a community image, stale (manifest says 28.4), no project-official Docker image exists
`apps/bitcoin-knots/`	no Dockerfile — `:latest` is built/pushed by hand
Registry	`scripts/image-versions.sh` → `ARCHY_REGISTRY="source.archipelago-foundation.org/lfg2025"`; only `BITCOIN_KNOTS_IMAGE=…/bitcoin-knots:latest` pinned, no Core pin
Tags in registry	one tag per image. No historical versions.

Version pinning

apps/bitcoin-core/manifest.yml → …/bitcoin:28.4 (pinned).
apps/bitcoin-knots/manifest.yml → …/bitcoin-knots:latest (floating — a liability for reproducibility and for "switch back to the version I had").
core/archipelago/src/container/app_catalog.rs + app-catalog/catalog.json: signed, hourly-fetched, carries version (badge text) + image. catalog_image_override() overrides the manifest image only if same-repo. available_update_for_app() already ignores floating tags for update detection.

Install path

prod_orchestrator.rs::install_fresh() resolves the image as manifest image → catalog override → pull. There is no per-install version parameter — orchestrator.install(app_id) takes only the id.
RPC package.install (api/rpc/package/install.rs) accepts dockerImage / version params but for orchestrator-managed apps (bitcoin-core / bitcoin-knots are allowlisted) it ignores them and lets the orchestrator resolve.
Conflict guard (prod_orchestrator.rs ~1306–1325): core and knots may not run simultaneously. Must be preserved by everything below.

UI

Install is one-click, no modal (MarketplaceAppDetails.vue::installApp()).
Update badge + "Update to X" already exist (appDetails/AppHeroSection.vue, RPC package.update).
No Bitcoin-specific settings panel; all apps share AppSidebar.vue.
Per-app config persisted only at install time as containerConfig → /var/lib/archipelago/app-configs/<id>.json. No post-install set-config RPC.

2. Source-of-truth decision: official upstream → our registry

We use the official releases as upstream provenance, but nodes only ever pull from our registry. Nodes do not fetch bitcoin.org / GitHub at install time — that would break rootless/offline installs and the signed-registry trust model, and neither project publishes an official Docker image anyway.

Official sources (verified):

Impl	Index	Per-version asset pattern
Bitcoin Core	bitcoincore.org/en/releases · github bitcoin/bitcoin	`https://bitcoincore.org/bin/bitcoin-core-<ver>/bitcoin-<ver>-x86_64-linux-gnu.tar.gz` + `SHA256SUMS` + `SHA256SUMS.asc`
Bitcoin Knots	github bitcoinknots/bitcoin · bitcoinknots.org/files	`https://bitcoinknots.org/files/<maj>.x/<ver>/bitcoin-<ver>-x86_64-linux-gnu.tar.gz` (`<ver>` e.g. `29.3.knots20260508`)

Both ship signed binary tarballs with multi-builder Guix attestations (SHA256SUMS.asc). The build pipeline verifies these once, at build; our DHT Phase 0 registry signature then carries provenance to the fleet.

Knots version strings embed a build date (29.3.knots20260508). Treat the full string as the tag; surface a friendly 29.3 + date in the UI.

3. Design

Phase 0 — Reproducible, verified image pipeline (prerequisite)

New scripts/build-bitcoin-image.sh <impl> <version> that, per version:

Downloads the official tarball + SHA256SUMS(.asc) (GitHub release assets are an identical mirror → fallback).
Verifies SHA256 and the Guix/builder GPG signatures. Fail closed.
Builds a minimal rootless image: pin a small base, unpack bitcoind/bitcoin-cli. Keep the existing entrypoint probe (command -v bitcoind || find /opt -path '*/bin/bitcoind') so per-version layout differences don't break startup.
Tags + pushes :<version> and updates the default pin (:latest / :28.4-style) to the registry.

Curate, don't mirror everything. Publish a bounded set (proposal: current + last ~3 majors), e.g. Core 31.0, 30.0, 29.3, 28.4, 27.2 and Knots 29.3.knots…, 28.1.knots…, 27.1.knots…. log / document dropped versions — silent truncation reads as "all versions supported" when it isn't.

Also fixes existing debt: replaces the stale community FROM bitcoin/bitcoin:24.0 and gives Knots a real Dockerfile + non-floating tags.

Phase 1 — Version catalog (signed, registry-distributed)

Extend AppCatalogEntry (forward-compatible — no deny_unknown_fields, old nodes ignore it):

"bitcoin-core": {
  "version": "31.0",                 // default / latest (existing field)
  "image": "…/bitcoin:31.0",         // existing
  "versions": [                      // NEW
    { "version": "31.0", "image": "…/bitcoin:31.0", "default": true },
    { "version": "30.0", "image": "…/bitcoin:30.0" },
    { "version": "28.4", "image": "…/bitcoin:28.4", "deprecated": true, "eol": "2026-...." }
  ]
}

Published to releases/app-catalog.json, signed by the existing release-root mechanism. This is the single source of truth the UI reads for "what can I install / switch to," and third-party-registry apps inherit the capability for free. version/image stay as the default for back-compat.

Phase 2 — Install-time version selection

Orchestrator: add install_with_image(app_id, Option<image_tag>) (or an optional arg on install). When a tag is supplied, validate same-repo against the manifest (reuse image_without_registry_or_tag()), then override in install_fresh(). Default path unchanged. Preserve the core/knots conflict guard.
RPC: thread the selected version/image from package.install into the orchestrator for the allowlisted apps (the param is already received — just not forwarded).
UI: the first install modal in the app — latest pre-selected, dropdown of versions[], deprecated/EOL badges on old entries. On confirm, pass the chosen version to package.install.

Phase 3 — In-app version switch + auto-update toggle

UI: a Bitcoin "Version & Updates" card (conditional in AppSidebar.vue for bitcoin-core / bitcoin-knots): current version, a switch dropdown, and an auto-update-to-latest toggle.
Switch = controlled re-pull/recreate reusing the package.update machinery but targeting an arbitrary (incl. older) tag → effectively package.set-version.
Persistence: new package.set-config RPC writing the existing app-configs/<id>.json ({ pinnedVersion, autoUpdate }).
Auto-update: the existing hourly catalog check, when autoUpdate:true, triggers package.update to the catalog default. A pinned version suppresses the update badge.

4. Invariants & safety rails

Rootless only. Pipeline images and run path stay rootless; no Docker-socket, no privileged.
No data loss across version change. Preserve /var/lib/archipelago/bitcoin, secrets (bitcoin-rpc-password, …-rpcauth), ports, and the adoption container name on every install / switch / update.
⚠️ Downgrade vs. chainstate (highest risk). Bitcoin Core refuses to start on a chainstate written by a newer version unless reindexed (expensive, or data loss on a pruned node). The UI must warn loudly on downgrade; the orchestrator should gate/confirm it and never silently wipe. Pruned nodes can't simply -reindex.
Core ⇄ Knots switch stays governed by the existing conflict guard; treat an impl switch as distinct from a version switch.
Floating tags (latest) are never advertised as a selectable "version" and never counted as an available update (already handled by available_update_for_app).
Verify on a real node (.228 then .198) and pass run-20x before any tag.

5. Files / seams (no code yet)

Concern	File
Image build/push	new `scripts/build-bitcoin-image.sh`; `apps/bitcoin-core/Dockerfile`; new `apps/bitcoin-knots/Dockerfile`; `scripts/image-versions.sh`
Catalog schema	`core/archipelago/src/container/app_catalog.rs`; `releases/app-catalog.json` (+ `app-catalog/catalog.json`)
Install override	`core/archipelago/src/container/prod_orchestrator.rs` (`install` / `install_fresh`); `api/rpc/package/install.rs`; `api/rpc/dispatcher.rs`
Switch / set-config RPC	`api/rpc/package/update.rs`; new `package.set-config` handler; `app-configs/<id>.json`
Install modal	`neode-ui/src/views/MarketplaceAppDetails.vue`; new `…/marketplace/AppInstallModal.vue`
Version & Updates card	`neode-ui/src/views/appDetails/AppSidebar.vue`; `neode-ui/src/api/rpc-client.ts`; `neode-ui/src/types/api.ts`

6. Open questions

Curated version set — how many majors back do we host, and storage budget on the registry?
Multi-arch — fleet is x86_64 today; do any nodes need arm64 images?
Pruned-node downgrade policy — block outright, or allow with an explicit "this will require re-sync / may lose pruned data" confirmation?
Auto-update default — off (opt-in) for a consensus-critical app like Bitcoin? (Recommended: off, explicit opt-in.)
Knots date-suffix UX — how to display 29.3.knots20260508 cleanly.

17 KiB Raw Blame History Unescape Escape