feat(registry-manifest): phase 2 — publisher embeds manifests into signed catalog

generate-app-catalog.sh gains opt-in EMBED_MANIFESTS=1: embeds each
apps/<id>/manifest.yml into its catalog entry's `manifest` field (whole document,
top-level app: preserved — exactly what the Rust side deserializes). Default off
so routine catalog regen is unchanged during the migration window; turn on
deliberately, then sign via the existing release-root ceremony. Verified: default
embeds 0; EMBED_MANIFESTS=1 embeds 40 manifests (generated_secrets preserved).

Adds a round-trip guard test: every shipped apps/*/manifest.yml must deserialize
+ validate through catalog_manifest_to_overlay (image apps accepted, build apps
defer to disk) — catches schema drift between disk manifests and the catalog path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-06-21 05:46:17 -04:00
co-authored by Claude Opus 4.8
parent 220666d3a9
commit 7bfbe8fe40
2 changed files with 82 additions and 2 deletions
+44 -2
View File
@@ -14,7 +14,16 @@
#
# Usage:
# scripts/generate-app-catalog.sh [output-path]
# EMBED_MANIFESTS=1 scripts/generate-app-catalog.sh # also embed full manifests
# # then publish: push releases/app-catalog.json to the OVH gitea (raw URL).
#
# EMBED_MANIFESTS (opt-in, default off): also embed each app's full
# apps/<id>/manifest.yml into its catalog entry's `manifest` field, so nodes can
# install from the signed registry alone (no OTA-shipped disk manifest). Consumed
# by container::app_catalog + the orchestrator's load_manifests overlay
# (origin-wins, disk = fallback). See docs/registry-manifest-design.md. Kept
# opt-in during the migration window so a routine catalog regen never changes
# what phase-1 nodes install until we deliberately turn it on.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -26,9 +35,16 @@ set -a
source "$ROOT/scripts/image-versions.sh"
set +a
UPDATED="$(date -u +%Y-%m-%d)" OUT="$OUT" python3 - <<'PY'
UPDATED="$(date -u +%Y-%m-%d)" OUT="$OUT" APPS_DIR="$ROOT/apps" \
EMBED_MANIFESTS="${EMBED_MANIFESTS:-}" python3 - <<'PY'
import glob
import json, os
try:
import yaml
except ImportError:
yaml = None
def img(var):
v = os.environ.get(var)
return v if v else None
@@ -121,6 +137,31 @@ for app_id, comps in STACK.items():
entry["images"] = images
apps[app_id] = entry
# Opt-in (EMBED_MANIFESTS): embed each app's full manifest so nodes install from
# the registry alone. The whole manifest document is embedded under `manifest`
# (top-level `app:` preserved) — that is exactly what the Rust side deserializes
# into an AppManifest. Apps not already in SINGLE/STACK get a new entry whose
# version comes from the manifest. A bad embed is harmless: the node validates and
# falls back to its disk manifest.
embedded = 0
apps_dir = os.environ.get("APPS_DIR")
if os.environ.get("EMBED_MANIFESTS") and apps_dir:
if yaml is None:
raise SystemExit("EMBED_MANIFESTS set but PyYAML is not available")
for path in sorted(glob.glob(os.path.join(apps_dir, "*", "manifest.yml"))):
with open(path) as fh:
data = yaml.safe_load(fh)
if not isinstance(data, dict) or not isinstance(data.get("app"), dict):
continue
app = data["app"]
app_id = app.get("id")
if not app_id:
continue
entry = apps.setdefault(str(app_id), {})
entry.setdefault("version", str(app.get("version", "")) or "0")
entry["manifest"] = data
embedded += 1
catalog = {
"schema": 1,
"updated": os.environ["UPDATED"],
@@ -130,5 +171,6 @@ catalog = {
with open(os.environ["OUT"], "w") as f:
json.dump(catalog, f, indent=2)
f.write("\n")
print(f"Wrote {os.environ['OUT']} with {len(apps)} apps")
suffix = f" (embedded {embedded} manifests)" if embedded else ""
print(f"Wrote {os.environ['OUT']} with {len(apps)} apps{suffix}")
PY