feat(release): guard the catalog against publishing untrusted registry hosts
Encodes the sequencing rule that nearly shipped a fleet-wide outage today. The signed catalog is authoritative on every node — catalog_image_override makes its image refs beat the on-disk manifest. TRUSTED_REGISTRIES in the working tree describes a binary being built now; nodes run whatever was last shipped to them. Those two diverge for exactly as long as an OTA takes to reach the fleet, and that window is when regenerating the catalog silently breaks every install with "not from a trusted registry". Regenerating today would have done precisely that: the generator embeds each app's manifest, and those now name the new registry domain, which no deployed binary trusts. - releases/registry-trust-floor.json records the hosts DEPLOYED binaries trust, separately from what the source tree accepts, with the new domain parked under `pending` until an OTA carries it. The migration order is written down there rather than living in someone's memory. - scripts/check-catalog-registry-trust.py compares the catalog's hosts against that floor and explains the ordering fix when they diverge. - sign-catalog.sh runs it as a preflight BEFORE prompting for the mnemonic, so a bad catalog is refused at the last reversible moment. - CI runs it blocking, plus the drift report advisory (drift between a manifest landing and the next signed release is expected, since only the ceremony can close it). Also installs PyYAML in the manifests job. That job passed only because GitHub runners happen to ship ruby, which the validator used to require; it now needs python3+PyYAML. Verified: passes on the published catalog (2 hosts, both trusted); refuses a simulated full regenerate (79 refs on the untrusted domain) and blocks the ceremony without requesting the mnemonic. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
co-authored by
Claude Opus 5
Claude Opus 5
parent
7cf6980894
commit
d0af38e825
@@ -93,8 +93,25 @@ jobs:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install YAML parser
|
||||
run: python3 -m pip install --quiet pyyaml
|
||||
|
||||
- name: Validate manifests
|
||||
run: |
|
||||
for manifest in apps/*/manifest.yml; do
|
||||
./scripts/validate-app-manifest.sh --repo-audit "$manifest"
|
||||
done
|
||||
|
||||
# The signed catalog overrides on-disk manifests on every node, so a
|
||||
# catalog naming a registry host the deployed fleet does not trust breaks
|
||||
# every install fleet-wide. Blocking, and cheap.
|
||||
- name: Catalog registry trust floor
|
||||
run: python3 scripts/check-catalog-registry-trust.py
|
||||
|
||||
# Advisory: shows where the release catalog has fallen behind the
|
||||
# manifests in this repo. Not blocking, because the catalog can only be
|
||||
# updated through the signing ceremony, so drift is expected between a
|
||||
# manifest landing and the next signed release.
|
||||
- name: Catalog drift (advisory)
|
||||
continue-on-error: true
|
||||
run: python3 scripts/check-app-catalog-drift.py --catalog releases/app-catalog.json --release
|
||||
|
||||
Executable
+163
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Refuse to publish a catalog naming registry hosts the fleet cannot pull from.
|
||||
|
||||
The signed app catalog is authoritative for deployed nodes: `catalog_image_override`
|
||||
makes its image reference win over the on-disk manifest. So a catalog that names a
|
||||
registry host the *deployed* binaries do not trust turns every install into
|
||||
"not from a trusted registry" — fleet-wide, at publish time, with no local signal.
|
||||
|
||||
The subtlety this guard exists for: TRUSTED_REGISTRIES in the working tree
|
||||
describes a binary being built today. Nodes run what was shipped to them. Those
|
||||
two lists diverge for exactly as long as it takes an OTA to reach the fleet, and
|
||||
that window is when a catalog regeneration silently breaks everything.
|
||||
|
||||
So the floor is tracked explicitly in releases/registry-trust-floor.json and the
|
||||
catalog is checked against that, never against the source tree.
|
||||
|
||||
Usage:
|
||||
scripts/check-catalog-registry-trust.py # check the release catalog
|
||||
scripts/check-catalog-registry-trust.py --catalog path.json
|
||||
scripts/check-catalog-registry-trust.py --show # print current state
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterator
|
||||
|
||||
DEFAULT_CATALOG = "releases/app-catalog.json"
|
||||
DEFAULT_FLOOR = "releases/registry-trust-floor.json"
|
||||
IMAGE_POLICY = "core/archipelago/src/container/image_policy.rs"
|
||||
|
||||
|
||||
def iter_images(node: Any) -> Iterator[str]:
|
||||
"""Yield every value stored under an `image` key, at any depth."""
|
||||
if isinstance(node, dict):
|
||||
for key, value in node.items():
|
||||
if key == "image" and isinstance(value, str) and value:
|
||||
yield value
|
||||
else:
|
||||
yield from iter_images(value)
|
||||
elif isinstance(node, list):
|
||||
for item in node:
|
||||
yield from iter_images(item)
|
||||
|
||||
|
||||
def registry_host(image: str) -> str | None:
|
||||
"""Registry host of an image ref, or None for Docker Hub shorthand.
|
||||
|
||||
A ref's first segment is a registry only if it contains a '.' or ':'
|
||||
(docker.io, host:3000). Otherwise it is a Docker Hub namespace — `nginx`,
|
||||
`btcpayserver/btcpayserver` — which resolves via registries.conf, not an
|
||||
attacker-controlled host. This mirrors is_valid_docker_image() in
|
||||
image_policy.rs; keep the two in step.
|
||||
"""
|
||||
head = image.split("/", 1)[0]
|
||||
if "/" not in image:
|
||||
return None
|
||||
if "." in head or ":" in head:
|
||||
return head
|
||||
return None
|
||||
|
||||
|
||||
def source_trusted_registries(repo: Path) -> list[str]:
|
||||
"""TRUSTED_REGISTRIES as the working tree currently defines it (advisory)."""
|
||||
path = repo / IMAGE_POLICY
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
return []
|
||||
match = re.search(r"TRUSTED_REGISTRIES:\s*&\[&str\]\s*=\s*&\[(.*?)\];", text, re.S)
|
||||
if not match:
|
||||
return []
|
||||
body = match.group(1)
|
||||
hosts = re.findall(r'"([^"]+)"', body)
|
||||
# Entries may be consts (LEGACY_REGISTRY_HOST); resolve those too.
|
||||
for const in re.findall(r"\b([A-Z][A-Z0-9_]+)\b", body):
|
||||
const_match = re.search(rf'{const}:\s*&str\s*=\s*"([^"]+)"', text)
|
||||
if const_match:
|
||||
hosts.append(const_match.group(1))
|
||||
return sorted(set(hosts))
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--catalog", default=DEFAULT_CATALOG)
|
||||
parser.add_argument("--floor", default=DEFAULT_FLOOR)
|
||||
parser.add_argument("--repo", default=".")
|
||||
parser.add_argument("--show", action="store_true",
|
||||
help="print the floor, the source list and the catalog's hosts")
|
||||
args = parser.parse_args()
|
||||
|
||||
repo = Path(args.repo)
|
||||
catalog_path = repo / args.catalog
|
||||
floor_path = repo / args.floor
|
||||
|
||||
try:
|
||||
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
|
||||
except OSError as exc:
|
||||
print(f"ERROR: cannot read catalog: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
floor_doc = json.loads(floor_path.read_text(encoding="utf-8"))
|
||||
except OSError as exc:
|
||||
print(f"ERROR: cannot read trust floor: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
floor = set(floor_doc.get("hosts") or [])
|
||||
if not floor:
|
||||
print(f"ERROR: {args.floor} lists no hosts; refusing to pass vacuously.",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
hosts: dict[str, list[str]] = {}
|
||||
for image in iter_images(catalog):
|
||||
host = registry_host(image)
|
||||
if host:
|
||||
hosts.setdefault(host, []).append(image)
|
||||
|
||||
if args.show:
|
||||
print("trust floor (deployed binaries):")
|
||||
for h in sorted(floor):
|
||||
print(f" {h}")
|
||||
pending = floor_doc.get("pending") or {}
|
||||
if pending:
|
||||
print("pending (not yet in the fleet):")
|
||||
for h, meta in pending.items():
|
||||
print(f" {h} — trusted_from_binary={meta.get('trusted_from_binary')}")
|
||||
print("working-tree TRUSTED_REGISTRIES (advisory):")
|
||||
for h in source_trusted_registries(repo) or ["(could not parse)"]:
|
||||
print(f" {h}")
|
||||
print(f"catalog hosts ({catalog_path}):")
|
||||
for h in sorted(hosts):
|
||||
print(f" {h} ({len(hosts[h])} image refs)")
|
||||
|
||||
violations = sorted(set(hosts) - floor)
|
||||
if violations:
|
||||
print("")
|
||||
print("REFUSING: the catalog names registry hosts the deployed fleet does not trust.")
|
||||
for host in violations:
|
||||
examples = hosts[host][:3]
|
||||
print(f"\n {host} — {len(hosts[host])} image refs, e.g.")
|
||||
for ref in examples:
|
||||
print(f" {ref}")
|
||||
print("")
|
||||
print("Publishing this would make every install fail with")
|
||||
print('"not from a trusted registry" on every node in the field.')
|
||||
print("")
|
||||
print(f"Fix by ordering the migration — see the _comment in {args.floor}:")
|
||||
print(" ship a binary that trusts the host, confirm the fleet is on it,")
|
||||
print(" promote the host in the floor file, and only then regenerate.")
|
||||
return 1
|
||||
|
||||
print(f"OK: all {len(hosts)} registry host(s) in {args.catalog} are trusted by the deployed fleet.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -23,6 +23,18 @@ if [[ ! -x "$BIN" ]]; then
|
||||
fi
|
||||
SIGN=("$BIN" ceremony sign "$CATALOG")
|
||||
|
||||
# Preflight BEFORE asking for the mnemonic. Signing is the point of no return:
|
||||
# a signed catalog is authoritative for every node, and its image refs override
|
||||
# the on-disk manifests. If it names a registry host the deployed fleet does not
|
||||
# trust, every install fails "not from a trusted registry" — so catch that here
|
||||
# rather than after publication.
|
||||
if ! python3 "$REPO/scripts/check-catalog-registry-trust.py" --repo "$REPO"; then
|
||||
echo
|
||||
echo "✋ Refusing to sign. Nothing was changed and your mnemonic was not requested."
|
||||
exit 1
|
||||
fi
|
||||
echo
|
||||
|
||||
echo "════════════════════════════════════════════════════════════════"
|
||||
echo " Paste your 24-word release master mnemonic below, press Enter,"
|
||||
echo " then press Ctrl-D on a new line."
|
||||
|
||||
Reference in New Issue
Block a user