archy/docs/fips-compute-market-proposal.md

Proposal: an open AI compute market over FIPS

Status: exploration and proposal, 2026-10-25. Nothing here is implemented beyond what "Starting point" lists. Decisions for the maintainer are collected at the end.

Reference platform: AMD Ryzen AI Max+ 395 / 495 (Halo).

Builds on: feat/routstr-fips-llama (seller gateway, buyer, settings UI, earnings card) and its ledger, docs/routstr-fips-llama-integration.md.

The goal in one paragraph

Any Archipelago node can sell AI compute, whether its own model, a GPU box on its LAN, or capacity it buys from someone else. The reference platform for serving is an AMD Ryzen AI Max (Halo) node, which is both the Archipelago node and the AI server. Any Archipelago user can open AIUI, ask for a PDF or an image, and pay for the compute with ecash. Traffic goes over FIPS and payment over Cashu. It should feel like one open market, not a list of friends' addresses, and it should use the same model catalog conventions as the rest of the AI world (OpenAI-compatible APIs, Routstr's pricing and discovery), so it is not an Archipelago-only island.

Recommendation

1. Make the market Routstr's, and the transport ours. Routstr already defines the two hard parts: provider discovery (signed Nostr listings, kind 38421, "NIP-91") and per-request settlement (X-Cashu in, change out). Archipelago adds the missing third part: a FIPS endpoint in the listing. An Archipelago seller then appears in the same market as every public Routstr provider, and an Archipelago buyer sees both.
2. Make Routstr the single metering point on a seller node. The node's own GPU (on the Halo reference platform), LAN boxes (DGX Spark) and resold upstreams all become Routstr upstreams behind one gateway, one price list, one wallet and one earnings card. Do not build a second billing path.
3. Add an asynchronous job API before images. Synchronous HTTP with a bearer token is fragile for anything slow: Routstr v0.5.0 keeps the token if the caller hangs up. Images, PDFs over long documents and anything that runs for minutes need "quote, pay, job id, fetch result", where the result can be fetched again without paying again.
4. Render PDFs on the buyer's node. A PDF is text from a model plus local typesetting. Only the tokens need buying. Rendering locally is free, private and deterministic.

Starting point (verified on this branch, 2026-10-25)

• Seller gateway. /routstr/v1/models and /routstr/v1/chat/completions are served on the FIPS peer listener only, off until the operator turns selling on. Only X-Cashu and Content-Type are forwarded, and change returns in x-cashu.
• Buyer. Takes one chosen peer npub, journals each token before sending, reclaims change or an unspent token, and never resends a request.
• UI. Settings → System → AI over the mesh has selling on/off, the buy-from-peer address and an earnings card. AIUI labels peer models "· via mesh peer".
• First real run. Buyer archy-x250-beta paid seller archy-x250-pa3 over FIPS. Nine requests, 17 sats net, Minibits ecash, every change token received. Routstr on pa3 holds 17 sats for payout at 210.
• Discovery already exists for the public market. assistant/backends/routstr.rs reads kind 38421 listings from Nostr.
• Resale already exists inside Routstr. It has an upstream type routstr (another Routstr node) with a fee multiplier, alongside generic (OpenAI-compatible), ollama, openrouter and others.

Facts that shape the design:

Fact	Consequence
Routstr v0.5.0 does pay-and-refund accounting only for chat/completions, completions, embeddings and messages.	/v1/images/generations is not a settled endpoint today. Images need either chat models that output images, or an upstream change.
An X-Cashu request whose upstream times out, or whose caller disconnects, returns an error without the change.	The current mitigation is timeouts ordered so the seller always finishes first. The real fix is upstream, plus async jobs for slow work.
Routstr's own resale upstream pays with a Bearer API key and auto top-up.	A reseller keeps a prepaid balance at its upstream. That is acceptable, because it is the reseller's own money, but it is not per-request settlement and cannot cross our gateway, which only forwards X-Cashu.
The seller's FIPS source address is derived from the buyer's node key.	Purchases are pseudonymous but linkable: every seller sees the same buyer identity.
The Routstr container runs with network_policy: isolated on archy-net.	It cannot reach a LAN GPU box or dial a FIPS npub without a deliberate, narrow egress path.

Reference platform: AMD Ryzen AI Max ("Halo")

The reference box for Archipelago serving compute is AMD's Halo platform: the Ryzen AI Max+ 395 ("Strix Halo") and its successor, the 495. What makes it the right reference, using the 395's published figures (confirm the 495's when hardware is in hand):

• One machine is both node and AI server. x86-64 Linux, 16 Zen 5 cores, an integrated Radeon 8060S GPU (40 RDNA 3.5 compute units) and an XDNA 2 NPU. It runs the whole Archipelago stack and serves models, with no second box and no LAN egress rules.
• Up to 128 GB of unified memory, most of which the GPU can use. Models that need a workstation GPU elsewhere fit here: 70B-class dense models quantised, or large mixture-of-experts models.
• Memory bandwidth (about 256 GB/s) sets the speed, not compute. Generation speed is roughly bandwidth divided by the bytes of weights read per token. The best models to sell from a Halo node are therefore mixture-of-experts models, which read only their active experts per token, and mid-size dense models. Large dense models fit but generate slowly. The pricing suggestion and the catalog should say this rather than let operators discover it from complaints.
• Low power for the capability. That makes the cost-based price suggestion (section 1) competitive. A Halo node can undercut cloud prices on the models it runs well and still cover electricity.

What Archipelago needs for it

1. A GPU build of llama-server. The current image is CPU-only. It already loads its compute backends at runtime (GGML_BACKEND_DL), so the same image can also ship a Vulkan backend. llama.cpp then uses the iGPU when it is present and falls back to the CPU tiers when it is not. One image serves both pa3-class laptops and Halo boxes.
   • Vulkan first: it works on the iGPU through the standard Mesa (RADV) driver with nothing AMD-specific in the image.
   • ROCm/HIP later, as a measured alternative. Whether it is faster on this iGPU, for which model types, and on which ROCm version, should be benchmarked rather than assumed.
2. GPU access in a rootless container, without privileges. The manifest already allows devices: [/dev/dri]. Rootless Podman must also keep the host user's render/video group membership in the container (--group-add keep-groups), or the device node is unreadable. That is a new, narrow manifest capability to add and test, not privileged.
3. Memory sizing. How much of the unified memory the GPU may use is a host setting (BIOS or kernel parameters, not the container). The node should read what is actually available to the GPU and refuse to load a model that does not fit, rather than let llama-server fail at start.
4. Model sets sized for the platform. Ship a short, tested list per memory size (64 GB, 128 GB) with measured tokens per second and the context that fits. Do not let operators guess. Each entry is pinned by repository revision and checksum, as Qwen2.5-1.5B is now.
5. The NPU is not in scope yet. XDNA 2 runtimes on Linux are young, and llama.cpp does not use them. Revisit when a supported, OpenAI-compatible NPU path exists. The GPU is the serving path.
6. Images on the same box. Image generation (FLUX, SDXL) on this iGPU is possible through ROCm or Vulkan runtimes, with quality and speed to be measured. It fits the job API in section 4: a Halo node could sell both text and images from one machine.

Reference acceptance

A Halo node is "reference" when, from a clean install, the following works with no terminal step:

• llama-server starts on the GPU (verified from its log, not assumed);
• one of the tested models serves with an AIUI-sized prompt inside every timeout, with measured throughput shown in the catalog;
• selling is on, and a buyer elsewhere on FIPS completes paid requests with change returned;
• the earnings card shows the sales, and a payout reaches the node wallet.

Architecture

                         Nostr relays (kind 38421 listings, signed)
                     ┌──────────────────────────────────────────────┐
                     │ d, u=https://…, u=fips:npub1…, mint=…, models│
                     └───────▲──────────────────────────▲───────────┘
                    publish  │                          │ discover
┌────────────────────────────┴───────┐        ┌─────────┴─────────────────────────┐
│ SELLER NODE                        │  FIPS  │ BUYER NODE                        │
│                                    │◄──────►│                                   │
│ FIPS gateway (exact routes, opt-in)│        │ Market index (listings + catalogs │
│        │                           │        │   + own reputation records)       │
│        ▼                           │        │        │                          │
│ Routstr: prices, X-Cashu, change,  │        │ Buyer: quote, journal, X-Cashu,   │
│ wallet, payouts to node LN address │        │   receive change, never resend    │
│   │          │            │        │        │        │                          │
│   ▼          ▼            ▼        │        │ AIUI: model picker grouped by     │
│ local      LAN AI      resale      │        │ model, sorted by price/latency/   │
│ llama-     endpoints   upstreams   │        │ trust; PDF rendered locally       │
│ server     (DGX Spark, (Routstr    │        └───────────────────────────────────┘
│ (Halo      other Halo  over FIPS,  │
│  GPU)      boxes)      OpenRouter) │
└────────────────────────────────────┘

Three layers, each replaceable without touching the others:

Layer	Standard	Archipelago's part
Discovery	Routstr NIP-91 listings on Nostr	Publish with a FIPS endpoint; index and rank on the buyer
Transport	HTTPS / onion for the public market	FIPS for node-to-node: no public IP, no TLS certificates, no port forwarding
Settlement	Routstr X-Cashu per request, change in x-cashu	Buyer journal and reclaim; seller ledger and earnings; async jobs for slow work

1. Selling local AI: the node's GPU, and boxes on the LAN

What the operator should experience

A "Sell AI" flow in Settings, one screen per step:

1. What can this node sell? The node lists the sources it knows about:

   • the local llama-server app, if installed, and whether it runs on a GPU;
   • LAN endpoints the operator adds, with a "Scan my network" button that probes the usual OpenAI-compatible ports on hosts the operator picks;
   • resale upstreams.

   Scanning a LAN is intrusive, so it asks first and never runs on its own.

2. Which models, at what price? Each model the source reports gets a row: name, context size (read from the source, not typed), and a price. The default is a suggestion and the operator can always override it:

   • local hardware: from power draw and measured throughput (below);
   • resale: the upstream price plus a markup.

3. Who may buy? Three choices:

   • federation peers only;
   • any FIPS peer;
   • listed in the public market.

   Today's toggle covers "any FIPS peer"; the other two are new.

4. Earnings. The card that exists today, extended per source and model.

LAN endpoints as Routstr upstreams

A Halo reference node serves from its own GPU and needs none of this. LAN endpoints are for compute that lives on another machine: a DGX Spark, another Halo box, a workstation. To Routstr they are ordinary OpenAI-compatible servers; which runtime they use (vLLM, NIM, Ollama, llama.cpp, LM Studio, Lemonade) does not matter. Routstr's generic and ollama upstream types already handle them. What Archipelago must add:

• Narrow egress for the Routstr container. A new manifest capability, for example egress: [{ host: 192.168.1.40, port: 8000 }], rendered as firewall rules on the container's network. The default stays isolated. Never "allow the LAN": a compromised Routstr must not be able to reach a router admin page. Keys for the LAN servers live in node secrets, as llama-server's key does now.
• Model import and health. For each LAN source, read /v1/models, check a one-token completion, record the real context size, and show it unhealthy when it stops answering. A source that is off must disappear from the catalog, not answer with errors that cost buyers a round trip.
• Concurrency per source. A DGX Spark can serve several requests at once; pa3 cannot. The gateway's fixed two slots should become per-source limits, advertised so buyers can avoid a busy seller.

Pricing that means something

Local compute has a real marginal cost, and the operator should see it:

sats per million tokens ≈ (watts × hours per million tokens × electricity price
                           + hardware cost per hour) ÷ sats per unit currency

The node can measure tokens per second and, where available, read power draw, then suggest a price and show the margin over that cost. Routstr re-prices in sats from its own exchange-rate feeds. The node should show the operator both the currency price they set and today's sats price.

Measured today: pa3, a dual-core 2015 laptop, runs Qwen2.5-1.5B at about 5 tokens/s generation and 18 tokens/s prompt reading. A Halo node or a DGX Spark is in a different class, and the market should make that visible as latency and throughput figures, not just price.

2. Reselling

Two kinds of resale, with different risks:

Resale of	How it works	Risk to the reseller
A public provider (OpenRouter, Anthropic, another public Routstr)	Routstr upstream with the reseller's API key or prepaid balance	Prepaid balance (inventory); upstream outages; markup must cover the reseller's swap and mint fees
Another Archipelago node over FIPS	Routstr upstream pointing at that node's FIPS gateway	Change must flow through two hops; the slowest hop decides every timeout

FIPS resale needs two pieces that do not exist yet:

• A FIPS upstream for containers. Routstr cannot dial npub1…. The node should offer an internal endpoint on archy-net, for example http://fips-egress/peer/<npub>/v1/…, forwarding with the same rules as RoutstrFipsTarget: FIPS only, single delivery, X-Cashu only. Each app that may use it must say so in its manifest.
• Per-request payment upstream. Routstr's routstr upstream type uses a Bearer balance, and our gateway refuses Bearer on purpose (a Bearer token becomes a server-side account whose change never returns). Either:
  • contribute an X-Cashu mode to Routstr's routstr upstream, where the reseller pays each request from its own wallet and receives the change (preferred: settlement stays per-request end to end); or
  • allow Bearer on the gateway for named, federated resellers only, as a prepaid account with a visible balance and a refund button.

Resale chains need one rule: every hop's timeout must be shorter than the hop before it, or a slow model loses the buyer's change at the hop that hangs up first. With async jobs (section 4) this problem disappears.

3. The open market: discovery, choice and trust

Listing

Publish Routstr's existing listing with one additional endpoint form:

{
  "kind": 38421,
  "tags": [
    ["d", "archy-<node id>"],
    ["u", "fips:npub1…"],
    ["u", "https://…"],
    ["mint", "https://mint.minibits.cash/Bitcoin"],
    ["version", "0.5.0"]
  ],
  "content": "{\"name\":\"pa3\",\"about\":\"Local Qwen2.5 1.5B\"}"
}

• Keep the model catalog out of the event; fetch /v1/models from the endpoint. Prices change, and stale prices in a signed event invite disputes.
• Sign with a key bound to the node's FIPS identity, so fips:npub1… in a listing can be checked against the key that signed it. A listing whose signer cannot be tied to its FIPS endpoint is ignored, which stops someone advertising another node's address.
• Public Routstr clients ignore the fips: endpoint, so listing it causes no harm. If an operator also wants non-Archipelago buyers, the listing can carry an onion endpoint.
• Proposing fips: to Routstr as a documented endpoint scheme is worth doing early: it costs little and makes Archipelago sellers first-class there.

Choosing a provider

The buyer keeps a market index: listings, catalogs fetched over FIPS or HTTPS, and its own history. AIUI's picker then groups by model, not by provider:

Qwen2.5 1.5B Instruct     3 sellers   from 0.9 sats/1k tok   ~4 s/answer    via mesh
Llama 3.3 70B             5 sellers   from 14 sats/1k tok    ~2 s/answer    public + mesh
FLUX.1 schnell (image)    1 seller    21 sats/image           ~40 s          via mesh

• Selection happens before payment only. Choose by price, observed latency and trust, and fail over to another seller only when no token was sent. After a token is sent, the request has one owner and is never retried elsewhere. That rule is what keeps the buyer journal simple and correct.
• Per-seller caps. The Routstr budget stays the overall cap. Add an optional per-seller limit for sellers the buyer has not used before.

Trust and reputation

Most of what a buyer needs to know it can measure itself, from its own journal:

• answered vs refused;
• change returned correctly;
• charged vs quoted;
• latency.

Keep that locally first. Publishing it (for example as NIP-32 labels signed by the buyer node) is a later step, and only after thinking about what it reveals about the buyer.

Ways a seller can cheat, and the bound on each:

Seller misbehaviour	Bound today	Better bound
Keeps the token, returns nothing	Token size (8–13 sats here)	Small tokens; reputation; async jobs with receipts
Overstates usage	Token size	Buyer recounts tokens with the model's tokenizer and flags a gap
Serves a cheaper model than advertised	None	Spot checks against known outputs; reputation
Reads the prompts	None (the seller sees plaintext)	Buyer-side choice: sensitive work only to self or federation; confidential compute later

4. PDFs and images in AIUI

PDF: buy tokens, typeset locally

"Make me a PDF of …" breaks into:

1. the model writes structured content (Markdown, or an outline then sections);
2. the buyer node renders it (Typst or a headless HTML-to-PDF renderer as a small local app);
3. AIUI shows it in the PDF viewer it already has.

Only step 1 touches the market. Long documents should be generated section by section, so each paid request stays short and well inside every timeout. That is cheaper and more robust than one huge completion.

Images: two paths

Path	Works with Routstr v0.5.0?	Where it fits
Chat models that return images inside chat/completions (several image-output models on OpenRouter)	Yes, settled and refunded like text	Buying from the public market, or reselling it
Dedicated image generation (FLUX, SDXL via ComfyUI or diffusers on a Halo node or a DGX Spark)	No: /v1/images/generations has no settlement handler	Selling local image compute; needs the job API below

The job API (needed for images, video and long work)

POST /routstr/v1/jobs            X-Cashu: <token for the quoted maximum>
  → 202 { job_id, receipt, estimate_s }            (change, if any, in x-cashu)
GET  /routstr/v1/jobs/<job_id>   (no payment; the receipt authorises)
  → 202 { status: "running", progress }   or   200 { result, usage }

• The token is redeemed once, at submission. The answer is a signed receipt binding the job id, the seller's FIPS identity, the amount paid and the quoted output.
• Fetching the result is idempotent and can be retried over any transport, so a dropped connection costs nothing. This removes the "caller hung up, seller kept the token" failure entirely.
• If the seller cannot finish, it refunds by returning a token to whoever holds the receipt.
• Pricing per job: a flat per-image price, or per-megapixel or per-step. The quote is the maximum; change returns at completion.
• Results can be large (images, PDFs). The gateway's 4 MiB response limit becomes per-operation, and large results can be fetched in ranges.

This is the one piece that needs real design work with Routstr upstream. It is worth proposing to them rather than forking: their users have the same timeout problem with long completions.

5. Privacy

• Prompts are visible to the seller. That is true of every hosted AI service. The UI should say so plainly and let the buyer limit sensitive conversations to their own node or their federation.
• FIPS hides the buyer's IP address from sellers and the internet, but the buyer's node key is visible to every seller and links its purchases. Options:
  • per-purchase or rotating FIPS identities, if FIPS can support them (open question for the FIPS work);
  • Tor for purchases where unlinkability matters;
  • accept linkability and say so in the UI.
• Ecash is unlinkable at the mint, but the seller sees which mint the buyer uses and the token sizes. Standard token sizes (for example rounding up to powers of two) reduce fingerprinting at the cost of more change.

6. Money flows and custody

Buyer node wallet ──X-Cashu token──► Seller's Routstr wallet ──≥210 sats──► Seller node LN address
        ▲                                  │
        └──────── change (x-cashu) ────────┘

• Seller earnings sit in Routstr's wallet until the payout minimum. That is custodial inside the seller's own node, which is acceptable. It should be visible (it now is) and claimable on demand ("pay out now"), not only on a threshold.
• Both sides must accept a common mint, and every swap between mints costs fees. The listing's mint tags let the buyer prefer sellers that accept a mint it already holds, and that should be part of ranking.
• A reseller's float (prepaid balance at an upstream) needs a visible balance and a ceiling, like the Routstr AI budget on the buying side.

7. Roadmap

Each phase is useful alone, and each ends with a check on real nodes.

Phase	Delivers	Acceptance
0 (done)	One local model sold to one chosen peer over FIPS; buyer journal; earnings card	pa3 → beta, 17 sats net, change always received
1a. Halo reference node	llama-server with a Vulkan backend beside the CPU tiers; rootless GPU access (/dev/dri, keep-groups); GPU memory check; tested model set with measured throughput	The reference acceptance above, on a Ryzen AI Max+ 395
1b. Seller sources	LAN endpoints as Routstr upstreams with narrow egress; model import with real context sizes; per-model pricing and cost-based suggestions; per-source concurrency	A DGX Spark on the LAN sold through the Halo node's gateway; a source switched off disappears from the catalog within a minute
2. Market	Listing with a fips: endpoint and identity binding; buyer index; AIUI picker grouped by model across mesh and public sellers; per-seller caps; local reputation	A buyer with no configured peer finds pa3's model in AIUI and pays it; a forged listing pointing at pa3 is ignored
3. Resale	FIPS egress for containers; X-Cashu resale upstream (upstream contribution) or federated Bearer accounts; ordered timeouts across hops	Node A resells node B's model to node C; every hop's ledger agrees; change returns end to end
4. Jobs and media	Job API with receipts; image generation on LAN GPUs; local PDF rendering in AIUI	"Make an image" and "make a PDF" in AIUI, paid over FIPS; killing the buyer mid-job loses nothing
5. Hardening	Upstream fix for X-Cashu refund on timeout; privacy options; published reputation; lifecycle gate; base-image pins; catalog publication	Release-gate green; fault injection (seller restart, mint down, FIPS drop) never strands funds beyond one token

Phase 1 and the upstream conversation with Routstr (the fips: endpoint scheme, X-Cashu refund on timeout, X-Cashu resale, a job API) can start in parallel. The upstream items decide how much Archipelago has to carry itself.

8. Decisions for the maintainer

1. Market scope: list Archipelago sellers in the public Routstr market (recommended), or keep an Archipelago-only market for now?
2. Default audience for a new seller: federation only, any FIPS peer, or public? The safest default is federation only, with one tap to widen.
3. Resale settlement: push for X-Cashu resale upstream in Routstr (recommended), or accept Bearer accounts between federated nodes?
4. Images first via: chat models with image output (works now, buy-side only), or the job API with local FLUX on a Halo node (needed to sell images)?
5. Buyer privacy: accept that the node key links purchases, or make rotating FIPS identities a requirement before the public market?
6. Who carries upstream work: contribute the job API and refund fix to Routstr, or carry patches in our image (pinned and reviewed)?

Open questions to verify

• Can any two Archipelago nodes reach each other over FIPS through anchors, or only nodes on shared links? This decides whether "public market over FIPS" means global or regional.
• The 495's memory size, bandwidth and GPU, and whether llama.cpp's Vulkan backend supports it on day one (the 395 is supported).
• Vulkan vs ROCm on Strix Halo for the tested model set: speed, stability, and which ROCm release, if any, to pin.
• How much unified memory each Halo board lets the GPU use by default, and whether the node can read it from Linux reliably.
• Does Routstr's per-request request price field already work for flat pricing through the generic upstream? If so, simple per-call services could ship before the job API.
• How large do AIUI's instructions get with tools enabled? They set the minimum context any sold model needs. About 2.4k tokens was measured today; TinyLlama's 2k context could not take it.