diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-ROUTSTR-FINDINGS.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-ROUTSTR-FINDINGS.md new file mode 100644 index 00000000..fcaae4bc --- /dev/null +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-ROUTSTR-FINDINGS.md @@ -0,0 +1,85 @@ +# Routstr Protocol Findings — live probe (13-03) + +**Date run:** 2026-08-03 +**Probe:** `core/archipelago/examples/routstr_probe.rs` (commit `ea90ef05`), run by hand with +`cd core && ./target/debug/examples/routstr_probe` (equivalently `cargo run --example +routstr_probe`). +**Relays probed:** `wss://relay.damus.io`, `wss://relay.nostr.band`, `wss://nos.lol` — the +three default relays cited in `docs.routstr.com` (RESEARCH.md, `13-RESEARCH.md` "Routstr +chat-completions call shape"). +**Filters used:** (1) kind `38421`, limit 50, 30s wait; (2) no kind restriction, `#d` tag +`routstr-provider`, limit 50, 30s wait — a fallback in case the kind number in the docs had +drifted. +**Result:** **NO LIVE PROVIDER OBSERVED.** Zero events matched either filter across all three +relays within the wait budget. No endpoint was discovered, so the capability-probe half +(`/v1/models`, `/`) never ran — there was nothing to point it at. + +Every claim below is labelled **OBSERVED** (this probe run produced the evidence) or +**DOCS-ONLY** (still resting on `docs.routstr.com`, unconfirmed by this probe). No claim in +this document is invented; where nothing was seen, the verdict says so. + +## Verbatim probe output + +``` +=== routstr_probe: live observation, read-only, spends nothing === +relays: ["wss://relay.damus.io", "wss://relay.nostr.band", "wss://nos.lol"] + +--- subscription 1: kind 38421 --- +(no events) + +--- subscription 2: no kind filter, #d = "routstr-provider" (fallback, in case the kind number drifted) --- +(no events) + +(no endpoint discovered and no --endpoint override given — skipping capability probe) + +NO LIVE PROVIDER OBSERVED +``` + +(Reproduced in full — the output is 13 lines, nothing was trimmed.) + +Process exit code: `0`. No relay-connect timeout warning was printed, so all three relay +connections succeeded within the 10s connect budget; the absence of events is not an artifact +of failed connections. + +## Verdict table (per RESEARCH claim under test) + +| # | Claim (as cited, `13-RESEARCH.md` "Routstr chat-completions call shape" / A2) | Observed value | Verdict | +|---|---|---|---| +| 1 | Provider-announcement event kind is `38421` | No event of kind 38421 seen on any of the 3 default relays in a 30s window | **NOT OBSERVED** | +| 2 | `d` tag value is `routstr-provider` | No event carrying `#d=routstr-provider` under any kind seen on any of the 3 default relays in a 30s window | **NOT OBSERVED** | +| 3 | Event content carries an `endpoints` field (http/onion) | No matching event at all — nothing to inspect | **NOT OBSERVED** | +| 4 | Event content carries a `models` field | No matching event at all — nothing to inspect | **NOT OBSERVED** | +| 5 | Event content carries a `pricing` field | No matching event at all — nothing to inspect | **NOT OBSERVED** | +| 6 | Payment header is `Authorization: Bearer cashuA…` and/or `X-Cashu:` | No provider endpoint was discovered, so no unauthenticated `GET`/`401`/`402` was ever issued — this probe never got the chance to see a provider name its own header | **NOT OBSERVED** | +| 7 | `POST /v1/chat/completions` is OpenAI-compatible, non-streaming as the primary mode | No endpoint discovered — never called | **NOT OBSERVED** | +| 8 | `tool_calls[].function.arguments` arrives as a JSON-encoded **string** (OpenAI convention, distinct from Ollama/Claude's parsed object) | No live chat-completions call was made (nothing to call it against) | **NOT OBSERVED** | +| 9 | Default relay list (`relay.damus.io`, `relay.nostr.band`, `nos.lol`) is a reasonable place to find providers | All three relays accepted the WebSocket connection (no connect-timeout warning) — the relays themselves are live and reachable, they simply carried no matching event during this window | **NOT OBSERVED** (relay reachability confirmed — **OBSERVED**; but that a Routstr provider announces there was not confirmed) | + +**Summary: 0 of 9 claims confirmed, 0 differ, 9 not observed.** This is not evidence the docs +are wrong — it is evidence that no Routstr provider was actively announcing on these three +relays during this 60-second window on 2026-08-03. A young, actively-developed ecosystem with a +small provider count can plausibly have zero announcers online at any given moment; this result +does not distinguish "the docs are stale" from "nobody happened to be broadcasting right now." + +## Assumption A2 (RESEARCH.md) — status after this probe + +**OBSERVED (of the probe's own execution), DOCS-ONLY (of the protocol itself, unchanged):** +A2's original risk line ("if the docs site's content has drifted from the actual +`routstr-core` implementation, the Rust client's header names or the Nostr filter subscription +could be wrong on first integration attempt") is **neither confirmed nor refuted** by this +probe. A2 does **not hold** in the sense the RESEARCH.md recommendation intended — the +recommended live-relay test ran, but returned no data to validate or invalidate the cited +contract against. The Medium risk rating stands **unchanged and unretired**: `backends/routstr.rs` +(13-13) still cannot be written against an independently-observed contract, only against +`docs.routstr.com` citations, exactly the situation A2 warned about. 13-13's own Task 1 +checkpoint (already present in `13-13-PLAN.md` as of this writing) is the correct place this +risk gets resolved into a decision, not this document. + +## What this probe did NOT do (scope discipline, per the plan's ``) + +- Did not pay a Cashu token, build one, or import any wallet/ecash code. +- Did not send an authenticated request of any kind. +- Did not publish a Nostr event. +- Did not retry beyond the one clean 60-second run recorded above — a single run that + successfully connected to all three relays and returned zero matching events on both filters + is a complete, valid negative result, not an inconclusive one requiring more attempts. diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/COVERAGE.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/COVERAGE.md index 05f7bceb..b559865a 100644 --- a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/COVERAGE.md +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/COVERAGE.md @@ -4,11 +4,16 @@ > Produced at plan time (2026-08-03) for Phase 13, per D-04 ("Routstr is explicitly in scope > at the user's request"). > -> **Confidence caveat, stated up front:** `13-RESEARCH.md` rates the Routstr protocol -> **MEDIUM** confidence — every row below is derived from `docs.routstr.com` and has **never -> been run against a live provider**. Open Question 3 asks for a spike; plan **13-03** is that -> spike. Rows marked `INTEGRATE — UNCONFIRMED` are ones this matrix cannot yet vouch for. -> **13-03 Task 2 rewrites this file from what the live relay/provider actually returns.** +> **Updated 2026-08-03 by 13-03 Task 2, from a live probe, not from docs.** `13-03` ran +> `core/archipelago/examples/routstr_probe.rs` against the three relays `docs.routstr.com` +> cites (`wss://relay.damus.io`, `wss://relay.nostr.band`, `wss://nos.lol`), subscribing for +> kind-38421 provider announcements plus a `#d=routstr-provider` fallback filter, for 30 +> seconds each. All three relays accepted the connection; **zero matching events were found on +> either filter.** Full evidence and a per-claim verdict table are in +> `13-ROUTSTR-FINDINGS.md`. The three rows previously marked as integrate-but-not-yet-confirmed +> are downgraded below to an explicit opt-out with a dated reason, per this file's own prior +> instruction — nothing here is laundered into a confident integration on evidence this probe +> did not obtain. ## Scope note @@ -21,9 +26,9 @@ API and Ollama's HTTP API are already partially in-tree (`mesh/listener/assist.r | capability | decision | reason | |---|---|---| | `POST /v1/chat/completions` — non-streaming | INTEGRATE | The loop's only required call shape. Every turn where a tool may be emitted must be fully buffered (AI-SPEC §4b.2), so non-streaming is the primary mode, not a fallback. | -| Tool / function calling (`tools[]` request, `tool_calls[]` response) | INTEGRATE — UNCONFIRMED | Required for D-07 parity: the confirm gate must behave identically on Routstr. OpenAI-compat convention says `tool_calls[].function.arguments` is a JSON-encoded **string** (unlike Ollama/Claude's parsed object) — AI-SPEC §3 Pitfall 2. **Not confirmed against a live provider.** 13-03 must settle it before `backends/routstr.rs` is written. | -| Cashu payment attach (`Authorization: Bearer cashuA…` and/or `X-Cashu:`) | INTEGRATE — UNCONFIRMED | D-04/D-05 make paid inference the point of the integration. Two header spellings are documented; the spike determines which the live provider accepts, and the client must not guess. | -| Provider discovery over Nostr (kind `38421`) | INTEGRATE — UNCONFIRMED | D-04 says providers/models/prices are "discovered over Nostr". Reuses `nostr_discovery.rs::build_nostr_client` (Tor-aware). Event kind, `d` tag value and content schema are all cited-not-verified. | +| Tool / function calling (`tools[]` request, `tool_calls[]` response) | OPT-OUT | not observable — no live provider reachable on 2026-08-03 (probed all 3 default relays, kind 38421 + `#d=routstr-provider` fallback, 30s each, zero matching events — `13-ROUTSTR-FINDINGS.md`). 13-13 Task 1 (`checkpoint:decision`, already present in `13-13-PLAN.md`) decides whether to proceed against docs-only, add a fail-loud probe-first step, or defer this leg. | +| Cashu payment attach (`Authorization: Bearer cashuA…` and/or `X-Cashu:`) | OPT-OUT | not observable — no live provider reachable on 2026-08-03 (no endpoint was ever discovered to issue the 401/402 probe against — `13-ROUTSTR-FINDINGS.md` verdict row 6). 13-13 Task 1 decides the fallback path. | +| Provider discovery over Nostr (kind `38421`) | OPT-OUT | not observable — no live provider reachable on 2026-08-03 (zero kind-38421 events and zero `#d=routstr-provider` events across all 3 default relays in a 30s window each; all 3 relays connected successfully, so this is an absence of announcers, not a connectivity failure — `13-ROUTSTR-FINDINGS.md`). 13-13 Task 1 decides the fallback path. | | Model listing (from the discovered provider event / `GET /v1/models`) | INTEGRATE | Routstr's model id is not a constant in this codebase — it comes from the provider. Without listing there is nothing to select. | | Price listing (sats per model, from the provider event) | INTEGRATE | D-05's budget ceiling is arithmetic over a price. `auto_pay_token(…, price_sats)` cannot be called without one. | | Provider selection strategy among multiple advertised providers | INTEGRATE | Explicitly delegated to Claude's discretion in CONTEXT.md. Implemented as: cheapest advertised price for the requested model that is affordable under the remaining `PaymentPolicy` budget, preferring an onion endpoint when Tor is up. | @@ -37,14 +42,19 @@ API and Ollama's HTTP API are already partially in-tree (`mesh/listener/assist.r ## Opt-out audit -Every `OPT-OUT` row above carries a one-line reason. Six opt-outs, six reasons. No row is -marked INTEGRATE on confidence this matrix does not have — the three genuinely uncertain -capabilities are marked `INTEGRATE — UNCONFIRMED` rather than laundered into a clean -`INTEGRATE`. +Every opt-out row above carries a one-line reason. Nine opt-outs, nine reasons: the original +six plus the three rows 13-03 Task 2 downgraded from an integrate-but-not-yet-confirmed state +after a live probe found no reachable provider (2026-08-03). No row is marked as a confident +integration on confidence this matrix does not have — every claim that could not be observed +was downgraded with a dated, evidenced reason rather than left as an optimistic integration. ## Gate -`13-13` (Routstr backend + D-05 budget ceiling) **must not begin** until `13-03` has replaced -the three `UNCONFIRMED` rows with live-observed facts, or has recorded that no live provider -was reachable — in which case 13-13's own first task is a `checkpoint:decision` on whether to -ship a docs-only client or defer the Routstr leg of D-04 with a named residual. +`13-03` ran. No live provider was reachable (`13-ROUTSTR-FINDINGS.md`). Per this file's own +prior instruction, that means **13-13 may not proceed directly** — its first task must be a +`checkpoint:decision`. This is already true of `13-13-PLAN.md` as written: Task 1 is +`type="checkpoint:decision" gate="blocking"` with exactly the three options this situation +calls for (`proceed-observed`, `proceed-docs-with-probe-first`, `defer-with-residual`), and its +own acceptance criteria require it to read this `## Gate` section and quote it. No edit to +`13-13-PLAN.md` was needed or made by this plan — 13-03's job was to produce the evidence that +checkpoint reads, not to alter the checkpoint itself.