Merge DATUM ngit proposal with preserved catalog entries

Reviewed ngit proposal 15ff5fb9 (pr/datum), head febdda968e. Preserve all existing catalog entries and normalize the new manifest memory limit to supported 512m syntax.

Validation: three configuration preservation tests and strict release catalog drift pass. Live miner shares, payout configuration and reboot acceptance remain separate.

app-catalog/catalog.json

@@ -685,6 +685,21 @@
       "tier": "optional",
       "repoUrl": "https://github.com/bencoin21/justworks.cash",
       "dockerImage": "localhost/archipelago-justworks:0.1.0"
+    },
+    {
+      "id": "datum",
+      "author": "OCEAN contributors",
+      "requires": [
+        "bitcoin-knots"
+      ],
+      "title": "DATUM",
+      "version": "0.4.1-beta.1",
+      "description": "Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM.",
+      "dockerImage": "localhost/archipelago-datum:0.4.1-beta.1",
+      "category": "bitcoin",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/datum.svg",
+      "repoUrl": "https://github.com/OCEAN-xyz/datum_gateway"
     }
   ]
 }

apps/datum/README.md

@@ -0,0 +1,86 @@
+# DATUM on Archipelago
+
+Packages OCEAN DATUM v0.4.1beta, pinned to upstream commit
+`5b061233a3d3323771b2be98e17f543e59346619`. The local build context must ship at
+`/opt/archipelago/docker/datum`; no published registry image is assumed.
+
+## First launch
+
+Install a Bitcoin node and allow it to synchronize, then install DATUM. Open its
+app tile and set your own Bitcoin payout address in DATUM's configuration page.
+The initial address is deliberately empty: upstream keeps the UI available while
+waiting for a valid address instead of mining to somebody else's address.
+The admin username is `admin`. The generated password is stored on the node at
+`/var/lib/archipelago/secrets/datum-admin-password`; retrieve it locally as the
+node administrator. Do not put it in miner passwords or share it with miners.
+
+Point miners at `stratum+tcp://<node-LAN-hostname>:23334`. Use a unique worker
+name for every miner, following upstream's payout/worker naming rules:
+https://github.com/OCEAN-xyz/datum_gateway/blob/v0.4.1beta/doc/usernames.md
+The default is pooled mining only; loss of the pool connection stops mining
+rather than silently switching to solo mining. DATUM's web UI reports template,
+Bitcoin and pool readiness; an HTTP health check only proves the UI is alive.
+
+## Stable connections
+
+Gashboard connects inside `archy-net` to `http://datum:7152`, using Podman's DNS
+alias. Never copy a container IP into either app's configuration. Bitcoin's DNS
+name is resolved from `BITCOIN_HOST` on each start, and the shared RPC secret and
+DATUM admin secret are refreshed without discarding the operator's settings.
+
+External miners connect to the **node**, not its container. Use a DHCP reservation
+on your router and a LAN DNS name if the miner supports DNS. Some miners do not
+support mDNS (`.local`); use the reserved LAN IP for those. Container DNS fixes
+container recreation, while the reservation prevents the node's DHCP address
+from moving. Neither setting requires host networking.
+
+Only Stratum is published directly. The admin UI is loopback-bound behind the
+Archipelago app gate and retains DATUM's admin authentication. The backend uses
+upstream's block notification polling fallback, so installing DATUM does not
+rewrite or restart Bitcoin to add a `blocknotify` command.
+
+## Data and validation
+
+Settings live in `/var/lib/archipelago/datum/config.json` with mode 0600. Preserve
+that directory and the platform secrets when uninstalling/reinstalling.
+
+Before catalog publication, validate install, setup, Bitcoin IBD and recovery,
+accepted shares from a real miner, stop/start, container recreation, preserved-data
+reinstall, backend restart and a controlled node reboot. Verify Gashboard recovers
+after DATUM receives a different container address. These live-node checks are
+separate from the local manifest/build checks and require a dedicated test node.
+
+## Local validation (2026-10-06)
+
+The pinned image builds on Linux/amd64. Its UI returns HTTP 200 while waiting
+for setup, `/clients` rejects unauthenticated requests, and its config is 0600.
+The container runs with read-only root, cap-drop ALL and no-new-privileges.
+Three config regression tests cover empty first-run payout, preserved payout
+policy (including explicit false settings), secret/DNS refresh and invalid input.
+Gashboard successfully polls this image using digest authentication and reconnects
+when its container IP changes. Manifest preflight and generated catalog drift
+checks pass. The catalog entries in this branch are review candidates; no signed
+catalog or image has been published. Real mining shares and full lifecycle acceptance remain required before release.
+
+## Operator-authorized node deployment (2026-10-06)
+
+Installed as rootless Podman apps on archi-dev-box and yaya-server, with the
+manifest and build context staged in the runtime payload. Both nodes report
+DATUM healthy. Chromium verified the normalized My Apps icon, title, Launch
+button, and embedded native UI with its Config navigation on both nodes.
+
+On yaya, a normal package restart recreated DATUM at 10.89.0.11 instead of
+10.89.0.9. Its configuration checksum was unchanged, and the still-running
+Gashboard resolved `datum` to the new address and resumed successful authenticated
+polling. Existing app container IDs remained unchanged on both hosts.
+
+The payout address remains unset. HTTP health proves that setup is available,
+not that Bitcoin/pool readiness or accepted mining shares have been established.
+No node reboot, IBD experiment, preserved-data reinstall, signed catalog release,
+or registry publication was performed in this deployment.
+
+The deployed platform drops manifest UI/icon metadata from its initial Installing
+placeholder, briefly showing a disk-only app under Services. Once scanned, both
+apps appear in My Apps with their declared icons and launch interfaces. A separate
+platform fix is being prepared; do not claim the installation-placeholder issue
+is fixed merely because the completed installation is displayed correctly.

apps/datum/manifest.yml

@@ -0,0 +1,84 @@
+app:
+  id: datum
+  name: DATUM
+  version: 0.4.1-beta.1
+  description: Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM.
+  upstream:
+    kind: github
+    repo: OCEAN-xyz/datum_gateway
+  container_name: datum
+  container:
+    build:
+      context: /opt/archipelago/docker/datum
+      dockerfile: Dockerfile
+      tag: localhost/archipelago-datum:0.4.1-beta.1
+    network: archy-net
+    network_aliases: [datum]
+    data_uid: "1000:1000"
+    derived_env:
+      - key: BITCOIN_RPC_HOST
+        template: "{{BITCOIN_HOST}}"
+    generated_secrets:
+      - name: datum-admin-password
+        kind: hex32
+    secret_env:
+      - key: BITCOIN_RPC_PASSWORD
+        secret_file: bitcoin-rpc-password
+      - key: DATUM_ADMIN_PASSWORD
+        secret_file: datum-admin-password
+  dependencies:
+    - app_id: bitcoin-knots
+    - storage: 1Gi
+  resources:
+    cpu_limit: 2
+    memory_limit: 512m
+    disk_limit: 1Gi
+  security:
+    capabilities: []
+    readonly_root: true
+    no_new_privileges: true
+    network_policy: isolated
+  ports:
+    - host: 7152
+      container: 7152
+      protocol: tcp
+      bind: 127.0.0.1
+      auth: gated
+    - host: 23334
+      container: 23334
+      protocol: tcp
+      auth: none
+      auth_rationale: Stratum mining clients require a raw TCP connection and cannot complete a browser login. Payout worker names are handled by DATUM; the administration UI uses a separate gated port.
+  volumes:
+    - type: bind
+      source: /var/lib/archipelago/datum
+      target: /data
+      options: [rw]
+    - type: tmpfs
+      target: /tmp
+      tmpfs_options: rw,noexec,nosuid,size=16m
+  health_check:
+    type: http
+    endpoint: http://localhost:7152
+    path: /
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  interfaces:
+    main:
+      name: DATUM Gateway
+      type: ui
+      port: 7152
+      protocol: http
+      path: /
+  bitcoin_integration:
+    rpc_access: admin
+    sync_required: true
+    pruning_support: true
+  metadata:
+    icon: /assets/img/app-icons/datum.svg
+    category: bitcoin
+    tier: optional
+    repo: https://github.com/OCEAN-xyz/datum_gateway
+    launch:
+      open_in_new_tab: false

core/archipelago/src/fips/app_ports.rs

@@ -14,6 +14,7 @@ pub const APP_LAUNCH_PORTS: &[u16] = &[
     3002,
     4080,
     5180,
+    7152,
     7778,
     8080,
     8081,

docker/datum/Dockerfile

@@ -0,0 +1,26 @@
+FROM debian:bookworm-slim@sha256:7c7b2c966bc9ee8cedfeef67e0e279108992c77681fa595db4a9d65c06ccc587 AS build
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    ca-certificates git build-essential cmake pkg-config libjansson-dev \
+    libmicrohttpd-dev libsodium-dev libcurl4-openssl-dev \
+    && rm -rf /var/lib/apt/lists/*
+WORKDIR /src
+# DATUM v0.4.1beta. Verify the commit as well as the tag.
+RUN git init && git remote add origin https://github.com/OCEAN-xyz/datum_gateway.git \
+    && git fetch --depth 1 origin refs/tags/v0.4.1beta \
+    && git checkout --detach FETCH_HEAD \
+    && test "$(git rev-parse HEAD)" = 5b061233a3d3323771b2be98e17f543e59346619 \
+    && cmake -DCMAKE_BUILD_TYPE=Release . && make -j2
+
+FROM debian:bookworm-slim@sha256:7c7b2c966bc9ee8cedfeef67e0e279108992c77681fa595db4a9d65c06ccc587
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    ca-certificates libjansson4 libmicrohttpd12 libsodium23 libcurl4 jq curl \
+    && rm -rf /var/lib/apt/lists/* \
+    && useradd --uid 1000 --create-home datum
+WORKDIR /app
+COPY --from=build /src/datum_gateway /app/datum_gateway
+COPY --from=build /src/www /app/www
+COPY entrypoint.sh /app/entrypoint.sh
+COPY configure.jq /app/configure.jq
+USER 1000:1000
+EXPOSE 7152 23334
+ENTRYPOINT ["sh", "/app/entrypoint.sh"]

docker/datum/configure.jq

@@ -0,0 +1,19 @@
+if type != "object" then error("Datum config must be an object") else . end
+| .bitcoind.rpcurl = ("http://" + env.BITCOIN_RPC_HOST + ":8332")
+| .bitcoind.rpcuser = "archipelago"
+| .bitcoind.rpcpassword = env.BITCOIN_RPC_PASSWORD
+# Use upstream's getbestblockhash fallback; no host bitcoind hooks needed.
+| .bitcoind.notify_fallback = true
+| .stratum.listen_addr = "0.0.0.0"
+| .stratum.listen_port = 23334
+| .mining.pool_address //= ""
+| .mining.coinbase_tag_primary //= "DATUM Gateway"
+| .mining.coinbase_tag_secondary //= "Archipelago"
+| .api.listen_port = 7152
+| .api.admin_password = env.DATUM_ADMIN_PASSWORD
+| .api.modify_conf = true
+| .logger.log_to_console = true
+| .logger.log_to_file = false
+| if .datum.pool_pass_workers == null then .datum.pool_pass_workers = true else . end
+| if .datum.pool_pass_full_users == null then .datum.pool_pass_full_users = true else . end
+| if .datum.pooled_mining_only == null then .datum.pooled_mining_only = true else . end

docker/datum/entrypoint.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+set -eu
+umask 077
+: "${BITCOIN_RPC_HOST:?Bitcoin host is required}"
+: "${BITCOIN_RPC_PASSWORD:?Bitcoin RPC password is required}"
+: "${DATUM_ADMIN_PASSWORD:?Datum admin password is required}"
+
+# Keep operator settings, including the payout address, across recreation.
+# Refresh platform-owned credentials and DNS names on every container start.
+config=/data/config.json
+if [ ! -e "$config" ]; then printf '{}\n' > "$config"; fi
+tmp=$(mktemp /data/config.json.XXXXXX)
+trap 'rm -f "$tmp"' EXIT HUP INT TERM
+jq -e -f /app/configure.jq "$config" > "$tmp"
+mv "$tmp" "$config"
+exec /app/datum_gateway --config "$config"

docker/datum/tests/test_config.py

@@ -0,0 +1,47 @@
+"""Config upgrades must preserve payout policy and reject broken input."""
+import json
+import os
+from pathlib import Path
+import subprocess
+import unittest
+
+FILTER = Path(__file__).resolve().parents[1] / 'configure.jq'
+
+
+def configure(value, host='bitcoin-core', password='new-rpc'):
+    return subprocess.run(['jq', '-e', '-f', str(FILTER)], input=json.dumps(value),
+                          text=True, capture_output=True,
+                          env={**os.environ, 'BITCOIN_RPC_HOST': host,
+                               'BITCOIN_RPC_PASSWORD': password,
+                               'DATUM_ADMIN_PASSWORD': 'test-admin'})
+
+
+class ConfigTests(unittest.TestCase):
+    def test_first_run_has_no_borrowed_payout_address(self):
+        result = configure({})
+        self.assertEqual(result.returncode, 0, result.stderr)
+        data = json.loads(result.stdout)
+        self.assertEqual(data['mining']['pool_address'], '')
+        self.assertTrue(data['datum']['pooled_mining_only'])
+        self.assertTrue(data['api']['modify_conf'])
+
+    def test_restart_preserves_payout_and_explicit_false_settings(self):
+        original = {'mining': {'pool_address': 'operator-address'},
+                    'datum': {'pool_pass_workers': False, 'pool_pass_full_users': False,
+                              'pooled_mining_only': False},
+                    'bitcoind': {'rpcurl': 'http://old-ip:8332', 'rpcpassword': 'old'}}
+        result = configure(original, password='quotes"and\\slashes')
+        self.assertEqual(result.returncode, 0, result.stderr)
+        data = json.loads(result.stdout)
+        self.assertEqual(data['mining']['pool_address'], 'operator-address')
+        self.assertEqual(data['datum'], original['datum'])
+        self.assertEqual(data['bitcoind']['rpcurl'], 'http://bitcoin-core:8332')
+        self.assertEqual(data['bitcoind']['rpcpassword'], 'quotes"and\\slashes')
+
+    def test_invalid_root_is_rejected(self):
+        for value in [None, [], 'broken', 1]:
+            self.assertNotEqual(configure(value).returncode, 0)
+
+
+if __name__ == '__main__':
+    unittest.main()

neode-ui/public/assets/img/app-icons/datum.svg

@@ -0,0 +1,15 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 100 100">
+  <!-- normalized by scripts/normalize-app-icon.py: margin=0.12 per side -->
+  <svg x="17.758" y="12.000" width="64.485" height="76.000" viewBox="0 0 28 33" preserveAspectRatio="xMidYMid meet">
+<svg width="28" height="33" viewBox="0 0 28 33" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M23.3296 4.17745C26.1789 6.96436 27.6035 10.9295 27.6035 16.0785C27.6035 21.2276 26.2113 25.1433 23.4209 27.9127C20.6335 30.6822 16.651 32.0669 11.4705 32.0669H0V0H11.1968C16.4361 0 20.4804 1.39345 23.3296 4.17745Z" fill="#1137F5"/>
+<mask id="mask0_3347_7483" style="mask-type:luminance" maskUnits="userSpaceOnUse" x="0" y="0" width="28" height="33">
+<path d="M23.3296 4.17745C26.1789 6.96436 27.6035 10.9295 27.6035 16.0785C27.6035 21.2276 26.2113 25.1433 23.4209 27.9127C20.6335 30.6822 16.651 32.0669 11.4705 32.0669H0V0H11.1968C16.4361 0 20.4804 1.39345 23.3296 4.17745Z" fill="white"/>
+</mask>
+<g mask="url(#mask0_3347_7483)">
+<path d="M37.9554 16.9571L15.9386 15.9214L37.7553 12.8232L37.6729 12.2472L15.8561 15.3454L36.6839 8.22105L36.4925 7.67123L15.6648 14.7956L34.75 3.90396L34.4557 3.40069L15.3675 14.2923L32.0274 0.0261431L31.6418 -0.41313L14.9849 13.8501L28.6071 -3.26986L28.145 -3.63058L14.5198 13.4952L24.6187 -5.86767L24.0978 -6.13531L13.9988 13.2247L20.2065 -7.67422L19.6414 -7.84004L13.4308 13.0647L15.5265 -8.62258L14.9407 -8.67786L12.845 13.0065L10.7522 -8.67786L10.1665 -8.62258L12.2593 13.0618L6.05158 -7.84004L5.48644 -7.67422L11.6941 13.2276L1.59522 -6.13531L1.07129 -5.86767L11.1731 13.4923L-2.45199 -3.63058L-2.91411 -3.26986L10.7081 13.8531L-5.94879 -0.41313L-6.33438 0.0261431L10.3225 14.2923L-8.76271 3.40069L-9.05705 3.90396L10.0282 14.7956L-10.7996 7.67123L-10.9909 8.22105L9.83683 15.3454L-11.9799 12.2472L-12.0623 12.8232L9.75442 15.9214L-12.2625 16.96L-12.233 17.5389L9.7809 16.5032L-11.6384 21.6378L-11.5001 22.2051L9.92219 17.0676L-10.1285 26.1178L-9.88416 26.6472L10.1636 17.5971L-7.78844 30.2341L-7.447 30.7083L10.505 18.0712L-4.70372 33.8385L-4.27692 34.24L10.9318 18.4727L-0.983223 36.8L-0.488727 37.1141L11.4263 18.7869L3.23471 39.0109L3.78219 39.2261L11.9738 19.0051L7.80292 40.3927L8.37983 40.5032L12.5507 19.1127V40.8989H13.1394V19.1127L17.3131 40.5032L17.8901 40.3927L13.7192 19.0051L21.9108 39.2261L22.4583 39.0109L14.2667 18.7898L26.1817 37.1141L26.6762 36.8L14.7582 18.4698L29.9699 34.24L30.3967 33.8385L15.188 18.0741L33.14 30.7083L33.4814 30.2341L15.5265 17.5971L35.5771 26.6472L35.8214 26.1178L15.7737 17.0676L37.1931 22.2051L37.3314 21.6378L15.9121 16.5032L37.926 17.5389L37.9554 16.9571Z" fill="white"/>
+</g>
+</svg>
+
+  </svg>
+</svg>

neode-ui/public/catalog.json

@@ -685,6 +685,21 @@
       "tier": "optional",
       "repoUrl": "https://github.com/bencoin21/justworks.cash",
       "dockerImage": "localhost/archipelago-justworks:0.1.0"
+    },
+    {
+      "id": "datum",
+      "author": "OCEAN contributors",
+      "requires": [
+        "bitcoin-knots"
+      ],
+      "title": "DATUM",
+      "version": "0.4.1-beta.1",
+      "description": "Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM.",
+      "dockerImage": "localhost/archipelago-datum:0.4.1-beta.1",
+      "category": "bitcoin",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/datum.svg",
+      "repoUrl": "https://github.com/OCEAN-xyz/datum_gateway"
     }
   ]
 }

neode-ui/src/views/appSession/generatedAppSessionConfig.ts

@@ -10,6 +10,7 @@ export const GENERATED_APP_PORTS: Record<string, number> = {
   "botfights": 9100,
   "btcpay-server": 23000,
   "cuprate-ui": 18091,
+  "datum": 7152,
   "electrs-ui": 50002,
   "electrumx": 50002,
   "fedimint": 8175,
@@ -59,6 +60,7 @@ export const GENERATED_APP_TITLES: Record<string, string> = {
   "core-lightning": "Core Lightning (CLN)",
   "cuprate": "Cuprate",
   "cuprate-ui": "Cuprate UI",
+  "datum": "DATUM",
   "electrs-ui": "Electrs UI",
   "electrumx": "ElectrumX",
   "fedimint": "Fedimint Guardian",