lfg2025/archy
1
0
Fork 0
Code Issues 24 Pull Requests 4 Actions Packages Projects Releases 71 Wiki Activity

docs: write the three missing app-developer docs (secrets, quadlet, lifecycle)

Browse Source 
The open-source plan flagged three references as "the real gaps for app
developers", and the docs index named them as not-yet-written. Written now,
each from the code rather than stubbed:

- secrets.md — generated_secrets/secret_env: the two halves, the four kinds
  (hex16/hex32/base64/bcrypt) and which files each writes, the idempotent
  self-healing 0600 materialisation, and the rules a developer must not break
  (no hardcoded fallbacks, one canonical name, right encoding). From
  container/secrets.rs and the manifest schema.

- quadlet-compilation.md — manifest -> .container unit: the full directive
  mapping (including Secret= by reference, never value, and Pull=never), where
  units land (~/.config/containers/systemd, systemctl --user), the
  render/write/enable/disable lifecycle with write-if-changed, and how to
  inspect one. From container/quadlet.rs, scoped accurately to the companion-UI
  path it drives today.

- container-lifecycle.md — the level-triggered 30s reconciler: desired state
  from user-stopped/user-uninstalled/manifest set, the operations table, the
  self-heal-vs-respect-a-deliberate-stop rule, and migrations-never-destroy-data.
  From prod_orchestrator.rs and boot_reconciler.rs.

Index updated to link all three under App development and the "known gap" note
removed. Every link across the docs tree resolves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-07 18:22:26 -04:00
co-authored by Claude Opus 5
parent 460eccd368
commit 599787690a
4 changed files with 339 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/README.md
+3 -6
View File
@@ -22,18 +22,15 @@ disagree, the code wins and the doc is a bug.
- [App Developer Guide](app-developer-guide.md) — build and package a containerized app
- [App Manifest Specification](app-manifest-spec.md) — the manifest schema, field by field
- [Manifest → Quadlet unit](quadlet-compilation.md) — how a manifest compiles to a systemd-owned container unit
- [Container lifecycle](container-lifecycle.md) — the reconciler state machine: install/adopt/start/stop/self-heal
- [App secrets](secrets.md) — declaring, generating and injecting per-install credentials
- [Registry-Distributed Manifests](registry-manifest-design.md) — how manifests reach nodes via the signed catalog
- [Decentralized Marketplace Protocol](marketplace-protocol.md) — publishing apps via an external registry
- [Bitcoin RPC Relay](bitcoin-rpc-relay.md) — letting an external wallet reach the node's Bitcoin RPC
- [Companion Pairing QR](companion-pairing-qr.md) — the pairing handoff contract
- [TV input inside iframe apps](tv-input-iframe-apps.md) — keyboard/gamepad routing into embedded apps
> **Known gap (tracked):** three app-developer references named in the
> open-source plan are not yet written — how a manifest compiles to a
> Quadlet/systemd unit, the container-lifecycle reconciler state machine, and
> the `generated_secrets` materialisation flow. Until they land, the source of
> truth is the code: `core/archipelago/src/container/quadlet*.rs`,
> `prod_orchestrator.rs`, and `container::secrets` respectively.
## Design docs
docs/container-lifecycle.md
+108
View File
@@ -0,0 +1,108 @@
# Container lifecycle
How Archipelago keeps apps in the state you asked for — install, start, stop,
restart, uninstall — and how it self-heals without ever resurrecting something
you deliberately stopped. Source of truth:
`core/archipelago/src/container/prod_orchestrator.rs` and
`core/archipelago/src/container/boot_reconciler.rs`.
## The model: level-triggered, not fire-and-forget
Archipelago does not start a container and hope. A long-running **reconciler**
compares *desired state* (what the manifests and your explicit choices say
should be running) against *actual state* (what podman reports) and repairs the
difference. It is **level-triggered**: it acts on the current gap every tick, not
on a one-time event, so a container that dies, a unit that vanishes, or a reboot
that clears everything are all just "the gap is non-zero, close it".
The reconciler is spawned once at boot (`BootReconciler`) after an initial
`adopt_existing()` pass, and runs every **30 seconds**. It finishes an in-flight
pull or build before honouring a shutdown signal — it is never interrupted
mid-operation.
Concurrency: each app has its own async mutex guarding all mutating operations
against the reconciler, so a manual `stop` and a reconcile tick can't race, but
reconciles across different apps still run without serialising against each
other.
## Desired state has three inputs
For each app the reconciler asks: *should this be running right now?* The answer
comes from three durable signals, checked in this order:
1. **Explicitly user-stopped** (`user-stopped.json`). If you stopped an app, its
id is recorded and the reconciler leaves it down — it is **not** a gap to
repair. Cleared when you start it again. This is what makes a stop *stick*
across restarts and reboots.
2. **Explicitly uninstalled** (`user-uninstalled.json`). Same idea for uninstall:
a baseline app you removed stays removed, so self-heal can't reinstall it.
3. **Otherwise, the manifest set** — every catalog/disk app that isn't stopped or
uninstalled should be running.
Dependencies are pulled in: an app that is up requires its declared
dependencies, so they are kept up too — but a dependency you explicitly stopped
still stays stopped.
## The operations
All go through the orchestrator, all take the per-app lock, all are idempotent:
| Operation | What it does |
|-------------|--------------|
| **adopt** | At boot, take ownership of a pre-existing container **by name** rather than recreating it — preserves data, ports and identity across a daemon restart. |
| **install** | Materialise secrets → ensure image (build from a local Dockerfile or use a pre-pulled image) → create and start the container (via Quadlet where enabled). |
| **start / stop** | Bring the container up/down and record the desired-state change. A stop writes the app to `user-stopped.json`. |
| **restart** | Stop then start, preserving the container's data and identity. |
| **remove** | Stop and remove the container, **preserving `/var/lib/archipelago/<app>`, secrets, credentials and ports** — a reinstall or upgrade lands on the same data. |
| **upgrade** | Recreate at a new image while preserving data (see the version rules below). |
| **health** | Report the container's health from its declared `health_check`. |
## Self-heal vs. respecting your choice
The one rule that ties it together: **self-heal must never override a deliberate
stop or uninstall.**
- A container that disappeared while its siblings run — a wedged teardown, a
reboot that cleared it — is a hole to repair, and the reconciler rebuilds it
from the durable "was running" snapshot.
- A container that is down because you stopped or uninstalled it is a *choice*,
and the reconciler leaves it alone.
A small set of **baseline apps** are expected to exist from first boot and
self-heal when their container is missing — but the `user_stopped` /
`user_uninstalled` gates are checked first, so even a baseline app you turned off
stays off. Getting this wrong in either direction is a real bug: resurrecting a
stopped app ignores the operator, and failing to rebuild a crashed one is the
fire-and-forget failure the whole design exists to remove.
## Migrations never destroy data
Any recreate path — upgrade, reinstall, repair — preserves the app's data
directory, its generated secrets, its credentials, its ports, and the container
name used for adoption. An update that would roll a version *backwards* is
refused (see the version guard in `container::image_versions`): the update button
never offers a lower version than what is running, so a stale record cannot turn
into a downgrade. Version pins are honoured — a pinned app is not "updated" out
from under the operator by the catalog.
## Inspecting lifecycle state
Run as the archipelago service user:
```bash
# what podman actually has
podman ps -a --format '{{.Names}}\t{{.Status}}'
# the durable desired-state signals
cat /var/lib/archipelago/user-stopped.json
cat /var/lib/archipelago/user-uninstalled.json
# the reconciler's decisions
journalctl --user -u archipelago | grep -iE 'reconcile|adopt|install|user.stopped'
```
## Related
- [Manifest → Quadlet unit](quadlet-compilation.md) — how the unit the reconciler manages is generated
- [App secrets](secrets.md) — the `ensure_generated_secrets` tick that runs before start
- [App Manifest Specification](app-manifest-spec.md) — `health_check`, `dependencies`, `restart` fields
docs/quadlet-compilation.md
+121
View File
@@ -0,0 +1,121 @@
# Manifest → Quadlet unit
How an app manifest becomes a Podman [Quadlet](https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html)
`.container` unit that systemd owns, where the unit lands, and how to inspect one.
Source of truth: `core/archipelago/src/container/quadlet.rs`.
## Why Quadlet
Containers used to be fire-and-forget `tokio::spawn` blocks. If the daemon
crashed mid-spawn or the kernel reaped a parent cgroup, the container vanished
from `podman ps` and only a manual `podman run` brought it back. Quadlet removes
that whole class of failure: the unit lives on disk, **systemd owns
start/restart, and archipelago is just the provisioner**. This is the path that
runs the companion UI containers today (`archy-bitcoin-ui`, `archy-lnd-ui`,
`archy-electrs-ui`), and the validated path being flipped to default for apps.
## What gets generated
`Quadlet::from_manifest(manifest, name)` translates a manifest into a unit, and
`render()` produces the file. Every unit carries a header making clear it is not
hand-edited:
```ini
# Generated by archipelago. DO NOT EDIT.
# Edits are overwritten on the next reconcile.
[Unit]
Description=<app description>
After=network-online.target
Wants=network-online.target
Requires=<dependency>.service # one per declared dependency
After=<dependency>.service
[Container]
ContainerName=<name>
Image=<image ref>
Pull=never # image must be present locally already
Network=<host | pasta | slirp4netns | bridge name>
User=<uid> # when the manifest pins one
DropCapability=ALL # security default
AddCapability=<cap> # only capabilities the manifest opts into
PublishPort=<bind>:<host>:<container>/<proto>
Environment=<KEY>=<value> # non-secret env only
Secret=<secret_name>,type=env,target=<KEY> # secrets by REFERENCE, never value
Volume=<source>:<target><opts>
ReadOnly=true # when security.readonly_root
NoNewPrivileges=true # when security.no_new_privileges
HealthCmd=<cmd> # from the health_check block
[Service]
TimeoutStartSec=0
Restart=<always | on-failure> # from the restart policy
RestartSec=10 # 10s backoff caps a crash loop
[Install]
WantedBy=default.target
```
Two things to note in that mapping:
- **Secrets go in by reference, never by value.** A `secret_env` entry renders as
`Secret=<name>,type=env,target=<KEY>`, so podman injects the value at run time
from the node's secret store. The plaintext never appears in the unit file. See
[App secrets](secrets.md).
- **`Pull=never` is deliberate.** The provisioner does not pull images from here;
the image must already be local (pre-pulled or built). A missing image surfaces
immediately instead of retrying silently behind systemd's restart loop.
## Where units land
Rootless, per-user, under the archipelago service user (uid 1000, with linger
enabled so the units run without an active login):
```
~/.config/containers/systemd/<name>.container
```
Quadlet's systemd generator translates `<name>.container` into a
`<name>.service` unit at **daemon-reload** time. Everything is `systemctl --user`
— the system bus is never touched from this path.
## Lifecycle: render → write → enable → disable
The module does four things and nothing else:
1. **render** — manifest → unit text (above).
2. **write** — `tempfile + rename` so a partially-written unit is never visible to
systemd, and `write_if_changed` compares bytes first: if the rendered unit
matches what is on disk, nothing is touched — no daemon-reload, no restart
cascade. This is what makes a reconcile tick cheap and non-disruptive.
3. **enable** — `daemon-reload` then start the `.service`.
4. **disable** — stop and remove.
## Inspecting a unit
Run these **as the archipelago service user** (the units are in its user bus):
```bash
# the generated unit
cat ~/.config/containers/systemd/archy-bitcoin-ui.container
# what systemd made of it
systemctl --user cat archy-bitcoin-ui.service
systemctl --user status archy-bitcoin-ui.service
journalctl --user -u archy-bitcoin-ui.service
# after editing a unit by hand for debugging (it will be overwritten on reconcile)
systemctl --user daemon-reload
```
Because the unit is regenerated on every reconcile, the way to change a
container's shape is to change its **manifest** (and, for a catalog-covered app,
regenerate and re-sign the catalog), never to edit the `.container` file — the
`DO NOT EDIT` header is literal.
## Related
- [Container lifecycle](container-lifecycle.md) — the reconciler that drives this
- [App Manifest Specification](app-manifest-spec.md) — the manifest fields mapped above
- [App secrets](secrets.md) — how `Secret=` references resolve
- [ADR-001: Podman over Docker](adr/001-podman-over-docker.md)
docs/secrets.md
+107
View File
@@ -0,0 +1,107 @@
# App secrets
How an app declares a secret, how Archipelago materialises it, and how it
reaches the container — with the rules a developer must not break.
The whole point: **an app never ships a credential.** It declares the *shape* of
the secrets it needs, and the node generates a fresh, per-install value that
never leaves the node and is never logged. Source of truth:
`core/archipelago/src/container/secrets.rs` and the manifest schema in
`core/container/src/manifest.rs`.
## The two halves
A secret has a producer and a consumer, and they are separate manifest fields:
- **`generated_secrets`** — *produce* a random value into a file.
- **`secret_env`** — *inject* a file's contents into the container as an env var.
An app can use either alone. A generated secret with no consumer is just a file
on the node; a `secret_env` with no matching `generated_secrets` reads a file
that some other component (or the daemon) is expected to have written.
## Declaring a generated secret
```yaml
container:
generated_secrets:
- name: btcpay-db-password
kind: hex16
- name: fedimint-gateway-hash
kind: bcrypt
```
`name` is a **bare filename** under the node's secrets directory
(`/var/lib/archipelago/secrets/`). It is validated at manifest-load time — no
`/`, no `..` — so a manifest cannot write outside that directory.
`kind` chooses how the value is produced. Each kind is deterministic in *shape*
(the orchestrator knows exactly which files it will create) but random in value:
| `kind` | Value | Files written | Use for |
|---------|-----------------------------------------|-----------------------------------|---------|
| `hex16` | 16 random bytes, lowercase hex (32 ch) | `<name>` | service passwords, API tokens |
| `hex32` | 32 random bytes, lowercase hex (64 ch) | `<name>` | longer keys/cookies |
| `base64`| 32 random bytes, standard base64 (44 ch)| `<name>` | services that base64-decode their key (e.g. netbird relay `authSecret`) |
| `bcrypt`| a random password **and** its bcrypt hash| `<name>` (hash) + `<name>.pw` (plaintext) | server configured with a hash, client needs the plaintext |
`bcrypt` is the only kind that writes two files: `<name>` holds the bcrypt hash a
server is configured with, and `<name>.pw` holds the plaintext for any client
that must authenticate against it. A `secret_env` injects whichever of the two it
references.
## Injecting a secret into the container
```yaml
container:
secret_env:
- key: BTCPAY_DB_PASS
secret_file: btcpay-db-password
```
At apply time the orchestrator reads `/var/lib/archipelago/secrets/<secret_file>`
and sets `<key>` in the container's environment to its contents. The value is
never written into the manifest, the Quadlet unit, or any log line.
## How materialisation works
`ensure_generated_secrets()` runs on **every install and reconcile tick**, before
`secret_env` is resolved. It is idempotent and self-healing:
1. **Fast path.** If every target file for a secret already exists, is readable
by the service user, and is non-empty, it is left untouched. A secret is
generated **once** and then persists across restarts, updates and reinstalls —
this is what makes credentials stable (migrations never regenerate a working
secret out from under a database).
2. **Self-heal.** A target file that exists but is unreadable or empty — e.g.
left root-owned by a botched earlier write — is removed and recreated, owned
by the service user. The unlink uses the secrets directory's own write bit, so
recovery needs no privilege escalation.
3. **Write.** New values are written through an atomic `0600` writer: a temp file
in the same directory, fsynced, then renamed over the target, so a reader never
sees a half-written secret and the file is only ever readable by its owner.
Because it runs every tick and no-ops when the secret is healthy, calling it is
always safe; there is no separate "provision secrets" step to forget.
## Rules a developer must not break
- **Never hardcode a credential**, in the manifest or in code, even as a
fallback. A shared fallback password means everyone holding a copy of the repo
holds that credential. Declare `generated_secrets` instead.
- **Never log a secret.** `secret_env` values and the files under the secrets
directory stay out of logs, error messages and status output.
- **One canonical name.** The orchestrator, first-boot script, reconcile path and
any deploy tooling must all reference a secret by the *same* filename. A
producer writing `<app>-password` while the consumer reads `<app>-hash` yields a
service that authenticates against a credential nothing generated.
- **Pick the encoding the service expects.** `hex*` and `base64` decode to
different bytes; a service that base64-decodes its configured key must be given
a `base64` secret, or it will run with the wrong key material.
## Related
- [App Manifest Specification](app-manifest-spec.md) — the full manifest schema
- [ADR-009: Manifest-Level Container Security](adr/009-manifest-container-security.md)
- [Entropy Enforcement (KEY-05)](security/KEY-05-ENTROPY-ENFORCEMENT.md) — why secret
generation draws from an explicitly-named CSPRNG