From 7134ae903d31ad6c02beee9ba6f3370ff5268bef Mon Sep 17 00:00:00 2001 From: archipelago Date: Mon, 3 Aug 2026 09:20:09 -0400 Subject: [PATCH] =?UTF-8?q?docs(13):=20research=20phase=20domain=20?= =?UTF-8?q?=E2=80=94=20AIUI=20conversational=20control?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verifies the AIUI-01 gating question against source (no tool-calling anywhere in this codebase today; Pine's HA intents are read-only Q&A, not an action-executing loop), surfaces a live unauthenticated Claude-proxy exposure (port 3142) and a same-origin iframe sandbox gap not previously named, and maps existing Cashu/Nostr primitives onto the Routstr integration. Co-Authored-By: Claude Opus 5 (1M context) --- .../13-RESEARCH.md | 518 ++++++++++++++++++ 1 file changed, 518 insertions(+) create mode 100644 .planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-RESEARCH.md diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-RESEARCH.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-RESEARCH.md new file mode 100644 index 00000000..254c2b84 --- /dev/null +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-RESEARCH.md @@ -0,0 +1,518 @@ +# Phase 13: AIUI — Conversational Node Control & Content Surfaces - Research + +**Researched:** 2026-08-03 +**Domain:** LLM tool-calling bridge (Rust), Cashu/Nostr paid-inference integration, cross-repo Vue embed architecture, media/content indexing +**Confidence:** MEDIUM-HIGH — the Rust and neode-ui sides are fully source-verified; the AIUI repo side is fully source-verified (cloned, read directly); Routstr protocol details are CITED from official docs (not independently protocol-tested against a live Routstr node, which does not exist in this environment). + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions + +**Where the agent loop lives** +- **D-01:** The agent loop (model call → tool call → result → model) runs **node-side in Rust**. The `archipelago` binary owns the loop, the tool registry, and the model key. AIUI becomes a thin chat client. Rationale: the key never reaches the browser; tool authorization sits where session auth already lives; Pine/voice can reuse the same tools later. Reversibility: costly. +- **D-02:** **One assistant, many front doors.** Extend the existing mesh assistant into a shared service: one tool registry, one backend selector, one place keys live. Mesh/LoRa, AIUI chat and (later) Pine voice are callers distinguished by permission scope. The existing peer-facing controls (`trusted_only`, `allowed_contacts`, `denied_askers`) are a per-caller scope mechanism that already exists. +- **D-03:** **Split by nature.** The node-side registry owns everything that reads or changes the node (system, bitcoin, network, wallet, files, media). The existing `ContextBroker` keeps only what must run in the browser (`navigate`, `open-app`, `launch-app`, `theme`) and remains the consent surface pushing `permissions:update`. +- **D-08:** Chat history lives **node-side in the per-node data dir** (`/var/lib/archipelago`), inheriting the node's backup/factory-reset/LUKS story. + +**Model backends** +- **D-04:** Backend chain is **local Ollama first, with Claude *and* Routstr as fallbacks**. Node data never leaves the node when a local model is available. Routstr is explicitly in scope at the user's request — an OpenAI-compatible endpoint paid per request in Cashu ecash, providers/models/prices discovered over Nostr. +- **D-05:** Routstr spending is authorized by a **prepaid budget the user sets**. Inference spends silently within the allowance, then stops and asks. The ceiling is hard. Reversibility: reversible. +- **D-07:** The **local model does get tools**, and every write needs confirmation regardless of backend. Backend choice stays a privacy decision, not a safety one. + +**Authority and sandboxing** +- **D-06:** Tools are a **curated allowlist of hand-written tools** — each with its own schema, permission category, and destructive/confirm flag. The model never sees the full RPC surface. No auto-generation from the dispatcher. Reversibility: reversible (adding tools is additive). +- **D-09:** First-cut authority is **reads within granted categories + app lifecycle (start/stop/restart) + settings writes**. Explicitly excluded from chat reach: keys, seeds, wallet spends, federation trust, factory reset. Reversibility: costly. +- **D-10:** **Tool authority never derives from content.** Peer-supplied text enters the context inside explicit untrusted-content delimiters marking it as data, not instructions. Pattern-stripping filters were considered and **rejected**. +- **D-11:** Write confirmations render **in neode-ui's trusted chrome, outside the iframe**, drawn by the host from the node's own description of the pending action — never by AIUI, never from model-authored text. Uses the Teleport-to-body modal pattern. Reversibility: costly — the load-bearing anti-spoofing property. +- **D-16:** All 10 permission categories (`apps`, `system`, `network`, `wallet`, `files`, `media`, `search`, `ai-local`, `notes`, `bitcoin`) **default closed** on a fresh node. +- **Hard constraint from Phase 10:** the `UNAUTHENTICATED_METHODS` hard-refuse gates and the loopback/auth boundaries must hold with AIUI on the other side of them. Not to be widened. + +**Content surfaces** +- **D-12:** **Feed the existing grids from Archy, replacing the LLM-synth source.** AIUI's design is kept exactly; what fills `FilmGrid`, `SongGrid`, `NewsGrid`, the detail views changes to real records. Reversibility: reversible — the grids are prop-driven. +- **D-13:** **Build a real music library** — albums, artists, tracks, tag/metadata extraction, an index that stays fresh. Lands as **its own wave inside Phase 13, not blocking the rest**. Reversibility: one-way — a persisted data model with a migration cost once nodes have indexed libraries. +- **D-14:** **IndeeHub and peer video are surfaced through the content + paid-unlock subsystem that already exists** (invoices, `X-Payment-Token`, Range streaming). No new payment rail. The cross-node "archipelago content source" is deferred. + +**Delivery and the two-repo split** +- **D-15:** AIUI is **built and shipped with the frontend, versioned and verified** — rsync path kept but made deliberate: AIUI's commit pinned, `VITE_BASE_PATH=/aiui/` enforced by the build script, a post-deploy check that fetches a live asset. Making AIUI a signed-catalog app was considered and **rejected** for this phase. +- **D-17:** AIUI **keeps its standalone mode**; embedded mode delegates to the node. Dev loop stays fast — no node required to work on the UI. +- **D-18:** **Push access to the AIUI repo is confirmed before planning starts** — CONFIRMED by the orchestrator per the phase brief; plan freely against `git.tx1138.com/lfg2025/AIUI` branch `development`. + +### Claude's Discretion +- What the music library indexes over (own filebrowser `Music` folder, peer audio, or both), the tag-extraction library, and where the index lives — within D-13's bounds. +- Streaming/token delivery for chat responses; context-window budgeting over node data. +- Which specific tools make the first curated allowlist, within D-09's authority ceiling. +- Routstr provider selection strategy among Nostr-advertised providers. +- Per-category mapping of the 10 permission categories onto individual tools. + +### Deferred Ideas (OUT OF SCOPE) +- Cross-node "archipelago content source" with payments (federation distribution + payments feature; own phase). +- AIUI Nostr integration polish (carried from Phase 2's deferred list). +- Reviving `ContentPanel.vue` and the plugin-renderer path (considered and rejected; dead-code cleanup, not architecture). +- AIUI's dev-only Vite middleware beyond what D-12 replaces (`vite-tmdb`, `vite-rss`, `vite-web-search`, `vite-fs`, `vite-music-search`, `vite-dev-chats`). +- Pine voice reusing the tool registry (D-02 makes it *possible*; wiring voice is not in this phase). +- **Not in scope (from the phase Domain block):** cross-node content distribution with payments; wallet spends, seed/key operations, federation trust changes, factory reset as chat-reachable actions; Nostr integration polish; reviving the dead `ContentPanel.vue` architecture. + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|------------------| +| AIUI-01 | Human-language node control — typed chat request reaches a real node action | §1 (gating question, verified), §2 (tool-calling loop), §4 (confirm gate) settle the mechanism; §"Curated Tool Allowlist" gives concrete RPC candidates | +| AIUI-02 | Conversational settings — system settings reachable by conversation, scoped to grants | Same tool-registry mechanism as AIUI-01; `system.settings.get`/`system.settings.set` are existing RPCs to wrap as tools | +| AIUI-03 | Content surfaces made real — peer files, music, IndeeHub movies, owned/paid content render live | §5 (content surfaces) maps `ContentItem`/`content.*` RPCs onto AIUI's `Film`/`Song`/`Podcast` prop shapes; §6 (music library) covers the one genuinely new data domain | +| AIUI-04 | Sandboxed by construction, permissioned by the user | §4 (confirm gate mechanism) + new §"Same-Origin Sandbox Gap" (a verified architectural finding: today's iframe embed has no hard browser-enforced boundary) | +| AIUI-05 | Delivery and build — AIUI reaches nodes through a real, verifiable update path | §7 (delivery) — current deploy scripts, the `VITE_BASE_PATH` requirement, and the missing live-asset check are all verified from source | +| AIUI-06 | Verified on device, embedded iframe, mobile included | §Validation Architecture | + + +## Summary + +AIUI today is not a dormant blank canvas waiting for wiring — it is an **actively working, unauthenticated, key-holding proxy straight to Anthropic**, running in production nginx config on every node that has had `setup-aiui-server.sh` run against it. The canonical `image-recipe/configs/nginx-archipelago.conf` proxies `/aiui/api/claude/` to a standalone Python HTTP server (`claude-api-proxy.py`, port 3142, its own systemd unit, its own `ANTHROPIC_API_KEY` env var — a *different* key than the Rust daemon's `data_dir/secrets/claude-api-key`) with **no session-cookie gate at all** ("API key managed by proxy, no session gate needed" — verified in the nginx config comment). This proxy bypasses the entire Rust JSON-RPC dispatcher: no `UNAUTHENTICATED_METHODS` gate, no CSRF check, no RBAC `role.can_access()` check, no involvement of `session::extract_session_cookie`. Anyone who can reach the node's web port can spend the node owner's Claude API budget with zero authentication. This is a pre-existing, currently-live exposure this phase's D-01 (move the agent loop node-side into the authenticated Rust surface) directly closes as a side effect of doing the phase correctly — but it needs to be named explicitly as a finding, because it is more severe than "chat can't act on the node" and is not mentioned in CONTEXT.md. + +Both halves of CONTEXT.md's central gating claim are **verified true against source**: `mesh/listener/assist.rs`'s `call_ollama` posts to Ollama's `/api/generate` (not `/api/chat`) with a bare prompt string and no `tools` field; `call_claude` posts to the Anthropic Messages API with a single user message and no `tool_use`/`tools` field. Both are single-shot Q&A, no loop. `dispatcher.rs` registers only `mesh.assistant-status`/`mesh.assistant-configure` — grep for `"pine.` across the entire dispatcher returns nothing; Pine has no RPC surface, and what CONTEXT.md called "the intent→action path Pine already proves" is, on inspection, Home Assistant's own `intent_script`/Assist framework seeded by `package/pine_ha.rs` — four **read-only** hardcoded intents (block height, peer count, sync status, Lightning balance) answered from REST-sensor state, not a Rust-side action-executing loop. HA's Claude conversation agent does get real LLM tool-calling via `llm_hass_api: ["assist"]`, but only over HA's own intents — none of which write to the node. So even Pine's voice path does not yet prove an action-taking loop; it proves Q&A-with-structured-intents at the HA layer. This corrects CONTEXT.md's framing and matters for scoping AIUI-01's "first tool-calling loop in this codebase" honestly. + +On the AIUI side (cloned, `development` branch, read directly): `useAI.ts`'s chat send path calls `streamClaude`/`streamOpenRouter` against `${BASE}api/claude/v1/messages` / `${BASE}api/openrouter` — i.e., exactly the nginx proxy above, or the browser's own vaulted API key. There is **no client-side tool-calling either**: the "Archy actions" AIUI's system prompt describes (`open-app`, `install-app`, `read-file`, `tail-logs`, `navigate`) are informational prose injected into the system prompt by `useArchy.ts`'s `buildArchyContext()`; only `readFile`/`tailLogs` are ever actually invoked by AIUI code, and both are called directly by UI components — never parsed out of a model response. AIUI has zero machinery today for turning an LLM's stated intent into an executed action; everything the model "does" today is either prose or a `[[tag:...]]` regex match consumed by `contentExtraction.ts`/`useContentPanel.ts` to render a content card. This phase must build the tool-calling protocol from scratch on both sides. + +A real anti-spoofing confirmation pattern already exists to extend, not invent: `contextBroker.ts`'s `install-app` handler dispatches a `CustomEvent('aiui:install-request')` for neode-ui's own UI to render a confirmation, then awaits `aiui:install-response` with a 60s timeout — this is D-11's mechanism today, just for one action type. It needs a second: `aiui-protocol.ts`'s `AIActionType` union has no `tool-call`/`confirm` member yet. + +For Routstr, the phase is not starting from zero on the payment side: `crate::swarm::payment::auto_pay_token` (used today by `streaming.prepare-payment`, the swarm content-payment path) already does exactly D-05's job — build a `cashuA` token for a given price against a set of `accepted_mints`, hard-capped by a `PaymentPolicy::with_budget`, degrading to `None` (never erroring) when unaffordable. Routstr's documented contract (CITED, not independently tested) accepts payment as `Authorization: Bearer cashuA...` or an `X-Cashu` header on an OpenAI-compatible `POST /v1/chat/completions`, and advertises providers via Nostr kind `38421` events — `nostr-sdk = "0.44"` is already a dependency with a working `build_nostr_client` (Tor-proxy aware) in `nostr_discovery.rs` to subscribe from. + +For content surfaces, `ArchyAppsGrid.vue` is confirmed dead (only referenced by dead `ContentPanel.vue` and its own test). All six of AIUI's "real data" Vite plugins are confirmed `configureServer`-only or `configureServer`+`configurePreviewServer`-only (verified per-file), so none run against the static `dist/` a node actually serves. `vite-fs.ts`'s hardcoded `/Users/dorian/Projects` is further confirmation this was never meant to reach a node. The grids' prop shapes (`Film`, `Song`, `Podcast`, etc. — all display-oriented with `posterUrl`/`coverUrl`/`sources[]`) do not match Archy's `ContentItem` (`id`, `filename`, `mime_type`, `access`, `availability`) — an adapter layer is required, not a straight pass-through. + +**Primary recommendation:** Build one new node-side "assistant" service module (extending, not replacing, `mesh/listener/assist.rs`'s backend-calling code) that owns a hand-written tool registry, a multi-turn loop per backend, and new RPC methods (`assistant.chat`, `assistant.confirm-tool`, `assistant.list-tools`, `assistant.history`) reached only via neode-ui's `contextBroker.ts` (new `chat:request`/`chat:response`/`tool:confirm-request` postMessage types) — never by AIUI fetching the RPC endpoint directly, even though nothing currently stops it (see the Same-Origin Sandbox Gap finding). Ship content surfaces as a straight `content.*` → grid adapter first (D-12, no new backend work beyond what exists), then land the music library (D-13) as its own wave using `lofty` for tag extraction. Treat the currently-live Claude proxy exposure and the iframe's lack of a hard sandbox boundary as findings the plan must explicitly decide how to handle (fix, accept-with-mitigation, or defer with a named risk) rather than silently working around. + +## Architectural Responsibility Map + +| Capability | Primary Tier | Secondary Tier | Rationale | +|------------|-------------|----------------|-----------| +| Agent loop (model call → tool call → result) | API/Backend (Rust daemon) | — | D-01: key + tool authority must stay server-side | +| Tool registry + permission scoping | API/Backend (Rust daemon) | — | D-06/D-09: curated allowlist, RBAC-adjacent, must not be derivable from the browser | +| Chat transport (AIUI ↔ Archy) | Browser/Client (postMessage bridge) | API/Backend (RPC over the neode-ui session) | AIUI has no session cookie path of its own by design; neode-ui's `contextBroker.ts` is the only channel today, and D-03 keeps it that way | +| Write-confirmation UI | Frontend Server / Browser (neode-ui trusted chrome) | — | D-11: must render outside the iframe, Teleport-to-body, drawn from node-authored text | +| Browser-only actions (navigate, open-app, theme) | Browser/Client (`ContextBroker`) | — | D-03: nothing server-side can perform a client-side navigation | +| Routstr payment (Cashu token build) | API/Backend (Rust daemon, `swarm::payment`) | — | Wallet/mint state is server-side; reuses existing `auto_pay_token` | +| Routstr provider discovery (Nostr) | API/Backend (Rust daemon, `nostr-sdk`) | — | Relay connections should route through the node's existing Tor-proxy-aware Nostr client, not the browser | +| Content surfaces (peer files, IndeeHub, paid content) | API/Backend (`content_server.rs`, `content.*` RPCs) | Browser/Client (AIUI grids, prop-adapted) | Data ownership and access control (`AccessControl::Paid`) must stay server-enforced; AIUI only renders | +| Music library index | Database/Storage (`/var/lib/archipelago`) + API/Backend (indexer) | Browser/Client (`SongGrid` consumer) | D-13: a persisted, migration-sensitive data model — indexing must not run in the browser | +| Media playback (Range streaming) | API/Backend (existing `/content/`, `/api/raw`, peer proxy) | Browser/Client (`GlobalAudioPlayer`) | Already solved; grids need data, not a new transport | +| AIUI static delivery | CDN/Static (nginx `/aiui/` location, frontend rsync) | — | D-15: built artifact, not a live service | + +## Standard Stack + +### Core + +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| `reqwest` | 0.11 (already in `archipelago/Cargo.toml`, `rustls-tls`+`socks`+`json`+`stream` features) | HTTP client for Ollama/Claude/Routstr calls | Already the codebase's only HTTP client; `socks` feature already present for Tor-proxied calls | +| `serde_json` | 1.0 (in-tree) | Tool-call schema construction, RPC params | Already universal in this codebase | +| `nostr-sdk` | 0.44 (in-tree, `nip04`+`nip44` features) | Routstr provider discovery (kind 38421 subscribe) | Already a dependency with a working Tor-aware client builder (`nostr_discovery.rs::build_nostr_client`) — no new crate needed | +| `tokio` | 1, `full` features (in-tree) | Async runtime for the multi-turn tool loop | Already universal | + +### Supporting (new, for the music library — D-13) + +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| `lofty` [ASSUMED — training-knowledge recommendation, not yet added to Cargo.toml; registry existence confirmed] | 0.24 (crates.io `max_version`, **VERIFIED: crates.io API**, 808K downloads, repo `github.com/Serial-ATA/lofty-rs`) | Read ID3/FLAC/M4A/OGG/WAV/APE tag metadata (title/artist/album/track/duration) in one unified API | Primary recommendation for D-13's tag extraction — broad multi-format support in one crate, avoids needing a separate parser per container format | + +### Alternatives Considered + +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| `lofty` (metadata-only) | `symphonia` (0.6, **VERIFIED: crates.io**, 9.4M downloads, `github.com/pdeljanov/Symphonia`) | Symphonia is a full audio *decoder* (needed for playback/transcoding, not tagging) — much heavier dependency surface for a job that's purely "read tags." Not needed here since playback already goes through the existing Range-streaming path, not server-side decode. | +| `lofty` (multi-format) | `id3` (1.17.1, **VERIFIED: crates.io**, 11.2M downloads, `codeberg.org/polyfloyd/rust-id3`) | ID3-only (MP3). Higher download count reflects broad MP3-tagging use elsewhere, not superiority for a library that must also cover FLAC/M4A/OGG. | +| Curated hand-written tool allowlist (D-06, locked) | Auto-generate tool schemas from `dispatcher.rs`'s method table | Explicitly rejected by D-06 — the model must never see the full RPC surface; every capability must be a deliberate decision | + +**Installation:** +```bash +# cargo add is run from core/ per CLAUDE.md +cd core && cargo add lofty --package archipelago +``` + +**Version verification:** `lofty` 0.24.0, `symphonia` 0.6.0, `id3` 1.17.1 confirmed live via the crates.io API (`crates.io/api/v1/crates/`) on 2026-08-03 — **VERIFIED: crates.io registry**, not merely a training-data guess. `reqwest`/`serde_json`/`nostr-sdk`/`tokio` versions read directly from `core/archipelago/Cargo.toml` — **VERIFIED: in-tree Cargo.toml**. + +## Package Legitimacy Audit + +| Package | Registry | Age | Downloads | Source Repo | Verdict | Disposition | +|---------|----------|-----|-----------|-------------|---------|-------------| +| `lofty` | crates.io | Long-running project (Serial-ATA/lofty-rs, active) | 808,246 total | github.com/Serial-ATA/lofty-rs | Not run through `gsd-tools query package-legitimacy check` in this session (tool unavailable in this environment) — manually checked: real GitHub org, active repo, substantial download count, no suspicious signals found | `[ASSUMED — recommend a `checkpoint:human-verify` before `cargo add`]` | +| `symphonia` | crates.io | Long-running (pdeljanov/Symphonia) | 9,452,628 total | github.com/pdeljanov/Symphonia | Same manual-check basis — not needed for this phase's scope (tagging only), listed for completeness | Not adopted — informational only | +| `id3` | crates.io | Long-running (rust-id3) | 11,202,064 total | codeberg.org/polyfloyd/rust-id3 | Same manual-check basis | Not adopted — informational only | + +**Packages removed due to [SLOP] verdict:** none. +**Packages flagged as suspicious [SUS]:** none by manual inspection, but `lofty` was not run through the automated `package-legitimacy check` seam (tool unavailable in this research session) — the plan must gate its `cargo add` behind a `checkpoint:human-verify` per the package-legitimacy protocol's own fallback rule for `[ASSUMED]` packages. + +## Architecture Patterns + +### System Architecture Diagram + +```text + Browser (neode-ui page, authenticated session) + ┌─────────────────────────────────────────────────────────────────────┐ + │ Chat.vue │ + │ ┌───────────────────────────────┐ postMessage (same-origin, │ + │ │