feat(apps): find out when an app has fallen behind upstream

Nodes offer an update when the signed catalog pins something newer than
what's running, and that machinery is fine. The missing step was the one
before it: nothing told *us* when upstream shipped. A pin could sit at
fedimintd v0.10.0 for months while every node in the fleet correctly and
confidently reported "up to date".

The reason nothing could tell us is that a manifest records only our
mirror — `source.archipelago-foundation.org/lfg2025/fedimintd:v0.10.0`
says nothing about the project it was mirrored from. So this adds an
optional `app.upstream` block naming the real source, and a script that
asks each one what it has released.

Running it answers the question that prompted this. Of 58 apps, 28 are
behind, including LND v0.18.4-beta against v0.21.2-beta, Bitcoin Core
28.4 against 31.1, and fedimintd/gatewayd v0.10.0 against v0.10.1.

Two choices worth stating. An app with no `upstream` block is reported
as UNTRACKED rather than skipped — a silent skip is how this stayed
invisible, and before this commit all 58 were silently skipped. And a
suggestion prefers our own tag variant: telling someone pinned to
`postgres:16.13-alpine` that the newest tag is `18.6-trixie` is true and
useless, because swapping the base image is a different decision from
bumping a version.

Five apps are deliberately left untracked (barkd, immich-postgres,
indeedhub-minio, lightning-stack, pine-whisper): I could not establish
their upstream with confidence, and a wrong `repo` produces a confident
wrong verdict, which is worse than an honest gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-17 08:04:33 -04:00
co-authored by Claude Opus 5
parent 59fffc809f
commit eb48eab946
55 changed files with 662 additions and 0 deletions
+3
View File
@@ -2,6 +2,9 @@ app:
id: aiui
name: AI Assistant
version: 0.1.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: Conversational AI interface for Archipelago. Quarantined — communicates only via context broker.
internal: true # System-managed, not shown in App Store
+6
View File
@@ -2,6 +2,12 @@ app:
id: alby-hub
name: Alby Hub
version: 1.23.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: getAlby/hub
description: Self-custodial Lightning wallet hub. Runs its own Lightning node on your Archipelago and connects your apps to it over Nostr Wallet Connect — one hub, every app pays through it.
category: money
+6
View File
@@ -2,6 +2,12 @@ app:
id: archy-btcpay-db
name: BTCPay Postgres
version: "15.17"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/postgres
description: Postgres backend for BTCPay and NBXplorer.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: archy-mempool-db
name: Mempool MariaDB
version: 11.4.10
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/mariadb
description: MariaDB backend for the mempool explorer stack.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: archy-mempool-web
name: Mempool Web
version: 3.0.1
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: mempool/mempool
description: Frontend web UI for mempool explorer.
container_name: mempool
+6
View File
@@ -2,6 +2,12 @@ app:
id: archy-nbxplorer
name: NBXplorer
version: 2.6.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: dgarage/NBXplorer
description: BTCPay blockchain indexer service.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: bitcoin-core
name: Bitcoin Core
version: 28.4.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: bitcoin/bitcoin
description: Reference Bitcoin Core node with dynamic prune/full-mode startup based on host disk.
container_name: bitcoin-core
+6
View File
@@ -2,6 +2,12 @@ app:
id: bitcoin-knots
name: Bitcoin Knots
version: 28.1.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: bitcoinknots/bitcoin
description: Full Bitcoin Knots node with dynamic prune/full-mode startup based on host disk.
container_name: bitcoin-knots
+3
View File
@@ -2,6 +2,9 @@ app:
id: bitcoin-ui
name: Bitcoin UI
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: |
Archipelago-native HTTP proxy + static site for interacting with the
Bitcoin Core / Bitcoin Knots JSON-RPC. Runs nginx inside a container
+3
View File
@@ -2,6 +2,9 @@ app:
id: botfights
name: BotFights
version: 1.2.11
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: Bot competition arena with 2-player arcade fighting mode. AI bots battle in trivia challenges while humans duke it out with controllers. Built for Bitcoiners.
category: community
+6
View File
@@ -2,6 +2,12 @@ app:
id: btcpay-server
name: BTCPay Server
version: 2.4.2
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: btcpayserver/btcpayserver
description: Self-hosted Bitcoin payment processor. Accept Bitcoin payments without intermediaries.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: core-lightning
name: Core Lightning (CLN)
version: 23.08.2
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: ElementsProject/lightning
description: Lightning Network implementation in C. Lightweight alternative to LND.
container:
+3
View File
@@ -2,6 +2,9 @@ app:
id: did-wallet
name: Web5 DID Wallet
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: Web5 wallet with Decentralized Identifier (DID) support. Manage your digital identity and Web5 assets.
container:
+3
View File
@@ -2,6 +2,9 @@ app:
id: electrs-ui
name: Electrs UI
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: |
Archipelago-native HTTP frontend for electrs/electrumx status. Runs
nginx inside a container, serves static assets, and proxies
+6
View File
@@ -2,6 +2,12 @@ app:
id: electrumx
name: ElectrumX
version: 1.18.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: spesmilo/electrumx
description: Electrum server indexing Bitcoin chain data for lightweight wallet queries.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: fedimint-clientd
name: Fedimint Client
version: 0.8.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: fedimint/fedimint-clientd
description: Fedimint ecash client daemon (fmcd). Lets the node hold Fedimint ecash and join federations; the wallet talks to it over a local REST API.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: fedimint-gateway
name: Fedimint Gateway
version: 0.10.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: fedimint/fedimint
description: Fedimint gateway service with automatic LND-or-LDK backend selection.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: fedimint
name: Fedimint Guardian
version: 0.10.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: fedimint/fedimint
description: Federated Bitcoin minting service with built-in Guardian UI. Privacy-preserving Bitcoin custody.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: filebrowser
name: File Browser
version: 2.27.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: filebrowser/filebrowser
description: Baseline Archipelago file manager service.
container:
+3
View File
@@ -2,6 +2,9 @@ app:
id: fips-ui
name: FIPS Mesh
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: |
Archipelago-native dashboard for the FIPS mesh transport. Runs nginx
inside a container with host networking, serves a static dashboard on
+6
View File
@@ -2,6 +2,12 @@ app:
id: gitea
name: Gitea
version: "1.23"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: go-gitea/gitea
description: Self-hosted Git service with built-in container registry, CI/CD, and package hosting.
category: development
+6
View File
@@ -2,6 +2,12 @@ app:
id: grafana
name: Grafana
version: 10.2.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: grafana/grafana
description: Analytics and monitoring platform. Visualize metrics and create dashboards.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: homeassistant
name: Home Assistant
version: 2026.7.3
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: home-assistant/core
description: Open source home automation platform. Control and monitor your smart home devices.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: immich-redis
name: Immich Redis
version: "7-alpine"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: valkey/valkey
description: Valkey (Redis-compatible) cache for Immich.
# Container named immich_redis (underscore) to match runtime per-app references
+6
View File
@@ -2,6 +2,12 @@ app:
id: immich
name: Immich
version: "2.7.4"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: immich-app/immich
description: Self-hosted photo and video backup with mobile apps and search.
# app_id "immich" = the user-facing launcher (matches the catalog entry's title
+3
View File
@@ -2,6 +2,9 @@ app:
id: indeedhub-api
name: IndeedHub API
version: "1.0.0"
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: IndeedHub backend API (Nostr auth, media, payments).
category: community
+3
View File
@@ -2,6 +2,9 @@ app:
id: indeedhub-ffmpeg
name: IndeedHub FFmpeg Worker
version: "1.0.0"
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: IndeedHub background media transcoding worker.
category: community
+6
View File
@@ -2,6 +2,12 @@ app:
id: indeedhub-postgres
name: IndeedHub Postgres
version: "16.13-alpine"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/postgres
description: Postgres database backend for IndeedHub.
category: community
+6
View File
@@ -2,6 +2,12 @@ app:
id: indeedhub-redis
name: IndeedHub Redis
version: "7.4.8-alpine"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/redis
description: Redis queue/cache backend for IndeedHub.
category: community
+6
View File
@@ -2,6 +2,12 @@ app:
id: indeedhub-relay
name: IndeedHub Nostr Relay
version: "0.9.0"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: scsibug/nostr-rs-relay
description: nostr-rs-relay backing IndeedHub's Nostr identity + comments.
category: community
+3
View File
@@ -2,6 +2,9 @@ app:
id: indeedhub
name: IndeeHub
version: "1.0.0"
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: Bitcoin documentary streaming platform featuring God Bless Bitcoin and other educational content about Bitcoin, sovereignty, and decentralized technology. Sign in with your Nostr identity.
category: community
+6
View File
@@ -2,6 +2,12 @@ app:
id: jellyfin
name: Jellyfin
version: 10.8.13
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: jellyfin/jellyfin
description: Free media server. Stream movies, music, and photos.
container:
+3
View File
@@ -2,6 +2,9 @@ app:
id: lnd-ui
name: LND UI
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: |
Archipelago-native HTTP frontend for LND. Runs nginx inside a
container and serves static assets. LND connection info is fetched
+6
View File
@@ -2,6 +2,12 @@ app:
id: lnd
name: LND
version: 0.18.4
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: lightningnetwork/lnd
description: Lightning Network implementation by Lightning Labs. Enables instant, low-cost Bitcoin payments.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: mempool-api
name: Mempool API
version: 3.0.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: mempool/mempool
description: Backend API for mempool explorer.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: mempool
name: Mempool Explorer
version: 3.0.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: mempool/mempool
description: Bitcoin mempool and blockchain explorer. Real-time transaction and block visualization.
container:
+3
View File
@@ -2,6 +2,9 @@ app:
id: morphos-server
name: MorphOS Server
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: MorphOS server platform. Decentralized application server.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: netbird-dashboard
name: NetBird Dashboard
version: "2.38.0"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: netbirdio/dashboard
description: NetBird management dashboard (SPA). Internal stack member served through the netbird proxy.
category: networking
+6
View File
@@ -2,6 +2,12 @@ app:
id: netbird-server
name: NetBird Server
version: "0.71.2"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: netbirdio/netbird
description: NetBird combined management / signal / relay server with an embedded identity provider and STUN. Backend for the self-hosted NetBird mesh VPN.
category: networking
+6
View File
@@ -2,6 +2,12 @@ app:
id: netbird
name: NetBird
version: "2.38.0"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/nginx
description: Self-hosted WireGuard mesh VPN control plane with dashboard, embedded identity provider, management API, signal, relay, and STUN. The user-facing entry point — a TLS proxy in front of the dashboard + server.
category: networking
+6
View File
@@ -2,6 +2,12 @@ app:
id: nextcloud
name: Nextcloud
version: "29"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: nextcloud/server
description: Your own private cloud. File sync, calendars, contacts.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: nostr-rs-relay
name: Nostr Relay (Rust)
version: 0.8.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: scsibug/nostr-rs-relay
description: High-performance Nostr relay written in Rust. Host your own decentralized social media relay and earn networking profits.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: phoenixd
name: phoenixd
version: 0.9.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: ACINQ/phoenixd
description: Headless Lightning daemon by ACINQ (the Phoenix wallet team). No screen of its own — it exposes a small local API that other apps and tools use to send and receive Lightning payments. Channel liquidity is managed automatically for a fee.
category: money
+6
View File
@@ -2,6 +2,12 @@ app:
id: photoprism
name: PhotoPrism
version: "240915"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: photoprism/photoprism
description: AI-powered photo management with facial recognition.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: pine-openwakeword
name: Pine Wake Word (openWakeWord)
version: "2.1.0"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: rhasspy/wyoming-openwakeword
description: Wyoming-protocol openWakeWord wake-word engine. Internal Pine voice-assistant stack member — lets Assist pipelines run wake-word detection on the node (groundwork for the custom "Yo Archy" wake word; stock models like "ok nabu" ship with the image).
category: home
+6
View File
@@ -2,6 +2,12 @@ app:
id: pine-piper
name: Pine Piper (TTS)
version: "2.2.2"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: rhasspy/wyoming-piper
description: Wyoming-protocol Piper text-to-speech engine. Internal Pine voice-assistant stack member — gives Home Assistant Assist a natural voice for spoken responses on the PineVoice satellite.
category: home
+6
View File
@@ -2,6 +2,12 @@ app:
id: pine
name: Pine
version: "1.3.0"
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: dockerhub
repo: library/nginx
description: A private voice assistant for your home. Pine runs speech-to-text (Whisper), text-to-speech (Piper) and wake-word detection (openWakeWord) on your own node and pairs with a PineVoice satellite speaker, so Home Assistant Assist works locally with nothing sent to the cloud. Ask it about your node — block height, sync, peers, Lightning balance — and, when a Claude API key is set, anything else.
category: home
+6
View File
@@ -2,6 +2,12 @@ app:
id: portainer
name: Portainer
version: 2.19.4
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: portainer/portainer
description: Container management web UI for the local Podman socket.
category: development
+3
View File
@@ -2,6 +2,9 @@ app:
id: router
name: Mesh Router
version: 1.0.0
# Built by this project — there is no upstream release feed to watch.
upstream:
kind: internal
description: Mesh routing and local network management. Provides device discovery, routing, and network topology visualization.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: searxng
name: SearXNG
version: 1.0.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: searxng/searxng
description: Privacy-respecting metasearch engine. Search the web without tracking.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: strfry
name: Strfry Nostr Relay
version: 0.9.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: hoytech/strfry
description: Lightweight Nostr relay written in C++. Alternative to nostr-rs-relay with lower resource usage.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: uptime-kuma
name: Uptime Kuma
version: 1.23.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: louislam/uptime-kuma
description: Self-hosted uptime monitoring.
container:
+6
View File
@@ -2,6 +2,12 @@ app:
id: vaultwarden
name: Vaultwarden
version: 1.30.0
# Where this app comes from, so scripts/check-upstream-releases.py can
# tell us when the pin below has fallen behind. Without it nothing can:
# container.image names our mirror, not the project it was mirrored from.
upstream:
kind: github
repo: dani-garcia/vaultwarden
description: Self-hosted password vault with zero-knowledge encryption.
container:
+40
View File
@@ -17,6 +17,45 @@ orchestrator code rather than a per-app installer, but it is not
manifest-declared, and the direction of travel is to replace each case with a
reusable manifest primitive.
## Upstream tracking
A node only offers an app update when the signed catalog pins a newer image
than the one running. That works — but nothing was telling *us* when upstream
had shipped something new, because a manifest records only our mirror
(`source.archipelago-foundation.org/lfg2025/fedimintd:v0.10.0`), which says
nothing about the project it was mirrored from. So a pin could sit still for
months while every node in the fleet correctly reported "up to date".
`upstream` closes that loop. It is metadata for the release process, never
read by the orchestrator:
```yaml
app:
id: fedimint
version: 0.10.0
upstream:
kind: github # github | dockerhub | internal | manual
repo: fedimint/fedimint
```
| `kind` | Meaning | Needs |
|--------|---------|-------|
| `github` | Watch a project's releases, then its tags. | `repo: owner/name` |
| `dockerhub` | Watch a Docker Hub repository's tags. | `repo: namespace/name` |
| `internal` | Built by this project — there is no upstream feed. | — |
| `manual` | Has releases, but not anywhere machine-readable. | `url:` for a human |
`scripts/check-upstream-releases.py` reads these and prints what is behind;
it exits non-zero when anything tracked has fallen behind, so a release pass
can gate on it. Export `GITHUB_TOKEN` first — a full sweep needs more than
GitHub's 60-per-hour anonymous quota.
An app with **no** `upstream` block is reported as `UNTRACKED` rather than
skipped: silently skipping unknowns is exactly how this gap stayed invisible.
Leaving it out is therefore fine and honest; guessing a wrong `repo` is not,
because a wrong source produces a confident wrong verdict.
## Top-level fields (`app:`)
| Field | Type | Required | Notes |
@@ -37,6 +76,7 @@ reusable manifest primitive.
| `devices` | list of string | — | Host device paths; must start with `/dev/`. |
| `interfaces` | map | — | Launch surfaces, keyed by name (`main`): `{ name, description, type, port, protocol, path }`. |
| `hooks` | LifecycleHooks | — | Allow-listed lifecycle hooks. See [Hooks](#hooks). |
| `upstream` | UpstreamSource | — | Where the app comes from, so release tooling can tell when the pin has fallen behind. See [Upstream tracking](#upstream-tracking). |
| _anything else_ | — | — | Unknown keys are absorbed into an `extensions` map (serde flatten) and treated as transitional metadata — e.g. `container_name`, `metadata`, `category`, `bitcoin_integration`, `lightning_integration`. These are **not** typed schema; do not rely on them being validated. |
## `container:` (ContainerConfig)
+340
View File
@@ -0,0 +1,340 @@
#!/usr/bin/env python3
"""Report which app pins have fallen behind their upstream project.
Why this exists
---------------
A node only offers an app update when the signed catalog pins a newer image
than the one running (`container/app_catalog.rs::available_update_for_app`).
That machinery works. What was missing is the step *before* it: nothing told
us when upstream had shipped something new, so a catalog pin could sit at
fedimintd v0.10.0 for months and every node in the fleet would correctly and
confidently report "up to date".
The reason nothing could tell us is that the manifests never recorded where an
app comes from. `container.image` names our *mirror*
(`source.archipelago-foundation.org/lfg2025/fedimintd:v0.10.0`), which says
nothing about the project it was mirrored from. So this script reads a new
optional `app.upstream` block (see docs/app-manifest-spec.md), asks that
source what its latest release is, and prints what is behind.
An app with no `upstream` block is reported as UNTRACKED rather than skipped.
A silent skip is how this gap stayed invisible in the first place.
Usage
-----
scripts/check-upstream-releases.py # check everything
scripts/check-upstream-releases.py fedimint lnd # check named apps
scripts/check-upstream-releases.py --offline # no network; coverage only
scripts/check-upstream-releases.py --json # machine-readable
Exit status is 1 when any tracked app is behind, so CI or the release
checklist can gate on it.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
import urllib.error
import urllib.request
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import yaml
REPO_ROOT = Path(__file__).resolve().parent.parent
APPS_DIR = REPO_ROOT / "apps"
CATALOG = REPO_ROOT / "releases" / "app-catalog.json"
USER_AGENT = "archipelago-upstream-check/1"
TIMEOUT = 20
# ── Version handling ───────────────────────────────────────────────────────
def version_parts(tag: str) -> tuple[int, ...] | None:
"""Numeric components of a version tag, for ordering only.
Mirrors `image_versions::parse_version_parts` on the node: accepts a
leading `v` and ignores a pre-release suffix. Returns None for opaque tags
(`RELEASE.2024-11-07T00-52-20Z`), which are reported but never *ordered* —
guessing an order for those is how you end up advertising a downgrade.
"""
if not tag:
return None
core = tag.strip().lstrip("vV").split("-")[0].split("+")[0]
if not re.fullmatch(r"\d+(\.\d+)*", core):
return None
return tuple(int(p) for p in core.split("."))
def tag_of(image: str) -> str:
"""The tag from an image reference, ignoring a registry port."""
if "@" in image: # digest pin — no tag to compare
return ""
last = image.rsplit("/", 1)[-1]
return last.split(":", 1)[1] if ":" in last else "latest"
# ── Upstream sources ───────────────────────────────────────────────────────
class RateLimited(RuntimeError):
"""GitHub refused because we are over the anonymous quota."""
def http_json(url: str, headers: dict[str, str] | None = None) -> Any:
hdrs = {"User-Agent": USER_AGENT, **(headers or {})}
# Anonymous GitHub allows 60 requests an hour, and a full sweep needs more
# than that. A token raises it to 5000 — worth exporting before a release
# pass, and the failure below says so rather than reporting every app as
# broken.
token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
if token and "api.github.com" in url:
hdrs["Authorization"] = f"Bearer {token}"
req = urllib.request.Request(url, headers=hdrs)
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as res: # noqa: S310
return json.loads(res.read().decode())
except urllib.error.HTTPError as e:
if e.code in (403, 429) and "rate limit" in (e.read().decode(errors="replace").lower()):
raise RateLimited(
"GitHub rate limit reached — export GITHUB_TOKEN and re-run"
) from e
raise
def latest_github(repo: str, current: str = "") -> str:
"""Newest release tag for `owner/name`.
Three sources, in descending order of what the project *means*: the marked
latest release, then the release list (many projects publish only
pre-releases, or never mark a latest), then plain git tags. The last one
matters more than it looks — electrumx, strfry and nostr-rs-relay all tag
releases without creating GitHub Release objects, and stopping at the
release list reported them as having "no orderable version tags" when they
were simply tagged instead.
"""
try:
return str(http_json(f"https://api.github.com/repos/{repo}/releases/latest")["tag_name"])
except (urllib.error.HTTPError, KeyError):
pass
try:
releases = http_json(f"https://api.github.com/repos/{repo}/releases?per_page=50")
best = _highest([r["tag_name"] for r in releases if not r.get("draft")], current)
if best:
return best
except urllib.error.HTTPError:
pass
tags = http_json(f"https://api.github.com/repos/{repo}/tags?per_page=100")
return _highest([t["name"] for t in tags], current)
def latest_dockerhub(repo: str, current: str = "") -> str:
"""Newest version-like tag on Docker Hub (`library/nginx`, `valkey/valkey`)."""
url = f"https://hub.docker.com/v2/repositories/{repo}/tags?page_size=100&ordering=last_updated"
results = http_json(url).get("results", [])
return _highest([t["name"] for t in results], current)
def _variant(tag: str) -> str:
"""The non-numeric suffix of a tag: `1.27-alpine` → `alpine`."""
core = tag.strip().lstrip("vV")
m = re.match(r"\d+(\.\d+)*[-.]?(.*)$", core)
return (m.group(2) if m else "").lower()
def _highest(tags: list[str], current: str = "") -> str:
"""The highest orderable tag, preferring our own variant.
Preferring the variant is what makes the answer actionable rather than
merely true: a node pinned to `postgres:16.13-alpine` is not helped by
being told the newest tag is `18.6-trixie`. Same version, different base
image — swapping it is a different decision from bumping a version.
"""
ranked = [(version_parts(t), t) for t in tags]
ranked = [(p, t) for p, t in ranked if p is not None]
if not ranked:
return ""
want = _variant(current)
if want:
same = [(p, t) for p, t in ranked if _variant(t) == want]
if same:
return max(same)[1]
return max(ranked)[1]
FETCHERS = {"github": latest_github, "dockerhub": latest_dockerhub}
# ── Manifest reading ───────────────────────────────────────────────────────
@dataclass
class AppPin:
app_id: str
manifest_version: str
image: str
upstream: dict[str, Any] = field(default_factory=dict)
catalog_image: str = ""
def load_apps(only: list[str]) -> list[AppPin]:
catalog_images: dict[str, str] = {}
if CATALOG.exists():
catalog = json.loads(CATALOG.read_text()).get("apps", {})
for app_id, entry in catalog.items():
image = entry.get("containers") or entry.get("image") or ""
catalog_images[app_id] = image if isinstance(image, str) else ""
pins: list[AppPin] = []
for path in sorted(APPS_DIR.glob("*/manifest.yml")):
try:
doc = yaml.safe_load(path.read_text()) or {}
except yaml.YAMLError as e:
print(f"warning: {path} is not valid YAML ({e})", file=sys.stderr)
continue
app = doc.get("app") or {}
app_id = app.get("id") or path.parent.name
if only and app_id not in only:
continue
pins.append(
AppPin(
app_id=app_id,
manifest_version=str(app.get("version") or ""),
image=str((app.get("container") or {}).get("image") or ""),
upstream=app.get("upstream") or {},
catalog_image=catalog_images.get(app_id, ""),
)
)
return pins
# ── Reporting ──────────────────────────────────────────────────────────────
def check(pin: AppPin, offline: bool) -> dict[str, Any]:
# The catalog pin is what nodes actually act on, so it is the number that
# matters; the manifest is the fallback for apps the catalog doesn't cover.
shipped_image = pin.catalog_image or pin.image
shipped = tag_of(shipped_image) or pin.manifest_version
row: dict[str, Any] = {
"app": pin.app_id,
"shipped": shipped,
"source": "catalog" if pin.catalog_image else "manifest",
"upstream_kind": pin.upstream.get("kind", ""),
"latest": "",
"status": "",
"note": "",
}
kind = pin.upstream.get("kind")
if not kind:
row["status"] = "UNTRACKED"
row["note"] = "no app.upstream block — nothing can tell us when this app moves"
return row
if kind == "internal":
row["status"] = "INTERNAL"
row["note"] = pin.upstream.get("note", "built by this project — no upstream to track")
return row
if kind == "manual":
row["status"] = "MANUAL"
row["note"] = pin.upstream.get("url", "check by hand")
return row
if kind not in FETCHERS:
row["status"] = "UNKNOWN-KIND"
row["note"] = f"unsupported upstream.kind {kind!r}"
return row
if offline:
row["status"] = "SKIPPED"
row["note"] = "offline"
return row
ref = pin.upstream.get("repo") or ""
if not ref:
row["status"] = "UNKNOWN-KIND"
row["note"] = f"upstream.kind {kind} needs a repo"
return row
try:
latest = FETCHERS[kind](ref, shipped)
except RateLimited as e:
# Distinct from ERROR: the pin may be perfectly current, we just
# couldn't ask. Reporting it as a failure would train people to ignore
# the column that matters.
row["status"] = "RATE-LIMITED"
row["note"] = str(e)
return row
except Exception as e: # noqa: BLE001 — any failure is "we couldn't ask"
row["status"] = "ERROR"
row["note"] = str(e)
return row
row["latest"] = latest
if not latest:
row["status"] = "ERROR"
row["note"] = "upstream published no orderable version tags"
return row
ours, theirs = version_parts(shipped), version_parts(latest)
if ours is None or theirs is None:
row["status"] = "UNCOMPARABLE"
row["note"] = "opaque tag — compare by hand"
elif theirs > ours:
row["status"] = "BEHIND"
elif theirs < ours:
row["status"] = "AHEAD"
row["note"] = "we ship newer than upstream's latest release"
else:
row["status"] = "CURRENT"
return row
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("apps", nargs="*", help="app ids to check (default: all)")
ap.add_argument("--offline", action="store_true", help="no network — report coverage only")
ap.add_argument("--json", action="store_true", help="machine-readable output")
args = ap.parse_args()
pins = load_apps(args.apps)
if not pins:
print("no manifests matched", file=sys.stderr)
return 2
rows = [check(p, args.offline) for p in pins]
if args.json:
print(json.dumps(rows, indent=2))
else:
width = max(len(r["app"]) for r in rows)
for r in sorted(rows, key=lambda r: (r["status"] != "BEHIND", r["app"])):
line = f"{r['app']:<{width}} {r['status']:<13} {r['shipped'] or '-':<18}"
if r["latest"]:
line += f"{r['latest']:<18}"
if r["note"]:
line += f" {r['note']}"
print(line.rstrip())
behind = [r["app"] for r in rows if r["status"] == "BEHIND"]
untracked = [r["app"] for r in rows if r["status"] == "UNTRACKED"]
print()
print(f"{len(rows)} apps · {len(behind)} behind · {len(untracked)} untracked")
if behind:
print("Behind: " + ", ".join(behind))
print("Bump the pin, regenerate and re-sign the catalog, and nodes will offer the update.")
if untracked:
print("Untracked apps cannot ever be reported as behind — add an app.upstream block.")
return 1 if any(r["status"] == "BEHIND" for r in rows) else 0
if __name__ == "__main__":
sys.exit(main())