docs(13): full AIUI assessment + four-wave fix plan (security/mission/content)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
archipelago
+208
@@ -0,0 +1,208 @@
|
||||
# AIUI Assessment & Fix Plan — 2026-08-07
|
||||
|
||||
Full-stack audit of the AIUI work (browser bundle, bridge, host broker, Rust assistant,
|
||||
nginx boundary) against three questions the operator asked:
|
||||
|
||||
1. **Security** — can it leak sensitive node info to the AI model or outside parties?
|
||||
2. **Mission alignment** — nostr-first, Blossom, hashtree, FIPS, self-hosted AI?
|
||||
3. **Seed-vs-reality** — does the `/seed` promise survive contact with a real node?
|
||||
|
||||
Each finding below was verified against the working tree (many independently re-verified
|
||||
during the audit write-up). File refs are `aiui/` = `aiui/packages/app/src`,
|
||||
`core/` = `core/archipelago/src`, `ui/` = `neode-ui/src`.
|
||||
|
||||
---
|
||||
|
||||
## A. Security verdict
|
||||
|
||||
**The embedded chat path is genuinely well-engineered.** Grants are node-side,
|
||||
default-closed, 0600 (`core/assistant/grants.rs`). The tool registry is a hand-written
|
||||
allowlist with an excluded-authority test constant (`core/assistant/tools.rs:138-149`).
|
||||
`execute_tool` re-checks grants server-side per call (`core/assistant/loop_.rs:213-225`).
|
||||
The confirm gate is nonce-bound, node-authored, rendered in trusted chrome, and the iframe
|
||||
provably cannot open or resolve it (`core/assistant/confirm.rs`, `ui/.../contextBroker.ts:219-225`).
|
||||
Cloud egress is secret-screened on both cloud legs (BIP39 checksum-validated, secrets-dir
|
||||
corpus, fail-closed, 64KB cap — `core/assistant/egress.rs`). The `/aiui/` CSP pins the
|
||||
frame's network to its own prefix (verified live in a browser, 13-09-SUMMARY).
|
||||
|
||||
**The real exposure lives at the edges — six fixable items:**
|
||||
|
||||
| # | Severity | Finding | Evidence |
|
||||
|---|----------|---------|----------|
|
||||
| S1 | HIGH | `network_status` tool sends **WAN IP + Wi-Fi SSID** to cloud models while the operator toggle promises "no IP addresses" — consent against a false label | `core/assistant/tools.rs:357-369` → `network.diagnostics` → `core/api/rpc/router.rs:97,103`; label at `ui/stores/aiPermissions.ts:33-35` |
|
||||
| S2 | HIGH | Claude API key **plaintext in localStorage** (`aiui-settings`), bypassing the encrypted key-vault that exists for exactly this | `aiui/stores/settings.ts:28,51,60,66-68`; `aiui/composables/useAI.ts:277-278`; violates aiui/CLAUDE.md |
|
||||
| S3 | HIGH | **Standalone mode reachable on a node** (just drop `?embedded=true`) → full history + base64 images + anything pasted go to Anthropic through the forwarder **unscreened** (screen only runs in the Rust backends); forwarder also has **no rate limit** | `aiui/main.ts:34`, `aiui/composables/useAI.ts:264-327`; `core/.../model_proxy.rs` forwards verbatim |
|
||||
| S4 | MED | `/aiui/api/web-search` proxy is **unauthenticated** (no session check, straight to SearXNG :8888) — anyone on the LAN runs searches attributed to the node's IP; sibling `/app/searxng/` and `/app/ollama/` are likewise direct | `image-recipe/configs/nginx-archipelago.conf:130-139,580,812` vs session-gated `:97,:116` |
|
||||
| S5 | MED | Node-side `app_logs` enters model context **unredacted** (browser path redacts `password=`, long hex/base64; node tool only untrusted-wraps); a 40-char token or `rpcpassword=<32-hex>` passes the egress screen | `core/assistant/tools.rs:695-705` vs `ui/.../contextBroker.ts:1068-1078`; threshold at `core/assistant/egress.rs:174-187` |
|
||||
| S6 | MED | G-B2 turn-minimality **silently strips prior user turns** on cloud legs since 13-10 history replay — model sees its own answers without the questions; module doc now false | `core/assistant/egress.rs:74-88,309-311,353-356` vs `core/assistant/mod.rs:859-877` |
|
||||
|
||||
Lower: legacy broker actions beyond D-03's documented set (`read-file`/`tail-logs`/
|
||||
`search-web`/dead `install-app`); filebrowser JWT in JS-readable cookie (URL leak is fixed);
|
||||
`webSearchEnabled` defaults true in browser localStorage (operator decision: node-side,
|
||||
default-off — unimplemented); nginx drift can silently remove the CSP the boundary
|
||||
depends on (live Wikipedia traffic observed from dev3's embedded frame).
|
||||
|
||||
## B. Mission-alignment verdict
|
||||
|
||||
**Aligned in direction and custody model; weakest exactly where the mission is most
|
||||
distinctive.**
|
||||
|
||||
- **Nostr-first: STRONG where shipped.** NIP-07-only identity (no nsec ever in-browser),
|
||||
NIP-04 DMs, NIP-05, NIP-44 share encryption, NIP-50 search, NIP-65 relay lists, relay
|
||||
manager, kind:30023 conversation sharing. Stubs honestly labelled: NWC (no NIP-47
|
||||
signing), LNURL-auth (no signature check), zap QR (placeholder drawing), naddr
|
||||
(non-standard base64, self-consistent only).
|
||||
- **Node-side AI chain embodies the mission**: Ollama-first ("node data stays on-node this
|
||||
turn"), Claude gated, **Routstr = nostr-discovered, cashu-paid, Tor-preferred, hard
|
||||
prepaid budget with typed `BudgetExhausted` stop** (`core/assistant/backends/routstr.rs`,
|
||||
1163 lines). OpenRouter deliberately 404'd on-node.
|
||||
- **But**: Routstr is a *terminal fallback* with an unverified protocol (0/9 live claims,
|
||||
13-ROUTSTR-FINDINGS) and **no funding UX anywhere** (5fc82894 captured a todo, not code).
|
||||
Standalone AIUI defaults to Claude and ships a **dead, unselectable Ollama adapter**.
|
||||
- **Blossom: ABSENT** (design-doc intent for IndeeHub catalog only).
|
||||
**Hashtree: ABSENT** (zero footprint repo-wide).
|
||||
**FIPS: correctly inherited** as peer-content transport via the broker — AIUI should
|
||||
never speak it directly.
|
||||
- Centralized deps that should have node answers: mempool.space price polling from the
|
||||
browser (node has Bitcoin Core), TMDB/iTunes/OpenLibrary/Wikipedia/OSM enrichment
|
||||
(dev-middleware only, 404 on node), YouTube-embed "free films".
|
||||
|
||||
## C. Seed-vs-reality verdict
|
||||
|
||||
**The seeds are a standalone-mode demo. On a real node, almost everything that makes them
|
||||
rich is absent.** Two surface mechanisms exist and they fight each other:
|
||||
|
||||
- **Text extraction** (the seed path): tags like `[[film_ext:…]]` are taught only in
|
||||
AIUI's *standalone* SYSTEM_PROMPT (`aiui/composables/useAI.ts:80-119`) — the node's
|
||||
prompt contains zero tag vocabulary (verified by grep). `<recipe_ext>` and `<event_ext>`
|
||||
are taught in *neither* prompt — seed-only by construction. Extraction statically imports
|
||||
the mock libraries (`contentExtraction.ts:4-6`) and presents them to the model as
|
||||
"the user's library" — mock songs carry `open.spotify.com/track/example` dead links.
|
||||
- **Tool surfaces** (the real path): works for `content_list` only. `apps_list` returns a
|
||||
bare array so the broker's `s.data?.items` drops it silently; IndeeHub `films` scope
|
||||
items classify as `'excluded'` (no mime/filename) so that surface can never render;
|
||||
once Archy content activates it *blocks* seed extraction and wipes grids to
|
||||
"Nothing found" on every no-tool turn.
|
||||
- **Playback is broken for real content**: `usePlayer.play()` never checks
|
||||
`song.sources[]` — a real library track goes to (CSP-blocked) Wavlake and reports
|
||||
"Not found on Wavlake"; `FilmDetail` only plays YouTube sources. `media-src 'self'`
|
||||
would allow the real URLs — the UI never uses them.
|
||||
- **The auto-seeded "Exploring My Node" demo runs on first load with no dev guard** and
|
||||
persists fabricated node state — including `rpcpassword=archipelago123` in a fenced
|
||||
bitcoin.conf — into every fresh install's chat history (`aiui/stores/chat.ts:476-508`,
|
||||
`nodeDemoPrompts.ts:116`; chunk verified in prod bundle).
|
||||
- Enrichment is dead on node (dev middleware + CSP), so every poster/cover resolves to an
|
||||
SVG placeholder; degenerate metadata renders as "★ 0 · 0m"; recipe/app cards click to
|
||||
nowhere (details never mounted); seed fixture's own gold tests are RED (expects 10
|
||||
songs, extractor yields 6).
|
||||
|
||||
**Real on a node today**: own shared media (filename-derived cards, no art) and the
|
||||
indexed music library (real tags, no art, unplayable in-app) — both measured
|
||||
empty/unreachable on the demo node (INDEEHUB-CONTENT-INVENTORY: 0 items; 16 peers serving
|
||||
nothing).
|
||||
|
||||
---
|
||||
|
||||
## D. Fix plan — four waves
|
||||
|
||||
Ordering logic: stop the leaks first (they're live), then make real content actually
|
||||
render/play (the visible product), then mission alignment (the differentiator), then the
|
||||
already-proposed follow-on phases.
|
||||
|
||||
### Wave 0 — Security blockers (gate any wider demo/rollout)
|
||||
|
||||
| Item | Fix | Files |
|
||||
|---|---|---|
|
||||
| S1 | Strip `wan_ip`/`wifi_ssid` from the `network_status` tool result (or split a new, honestly-labelled category); make the toggle label true | `core/assistant/tools.rs`, `core/api/rpc/router.rs` |
|
||||
| S2 | Route `claudeApiKey` through the encrypted key-vault only; migrate-and-wipe any plaintext in `aiui-settings` | `aiui/stores/settings.ts`, `aiui/composables/useAI.ts`, `aiui/utils/key-vault.ts` |
|
||||
| S3 | When served from a node, standalone mode is off (node banner or hard gate); run `screen_outbound` in the model forwarder as defense-in-depth; add forwarder rate limit | `aiui/main.ts`, `core/.../model_proxy.rs`, reuse `core/assistant/egress.rs` |
|
||||
| S4 | Session-gate `/aiui/api/web-search` through the daemon (:5678) like the model proxies; audit `/app/searxng/` + `/app/ollama/` auth posture | `image-recipe/configs/nginx-archipelago.conf`, `core/.../model_proxy.rs` |
|
||||
| S5 | Port the broker's log redaction into the node `app_logs` tool before untrusted-wrap | `core/assistant/tools.rs`, mirror `ui/.../contextBroker.ts:1068-1078` |
|
||||
| S6 | Decide and implement: either screen full replayed history or document + fix the stale module doc; stop silently dropping user turns | `core/assistant/egress.rs`, `core/assistant/mod.rs` |
|
||||
| S7 | Kill the auto-seeded "Exploring My Node" demo in prod (`isDev` guard at minimum, delete the fabricated `rpcpassword` content entirely) | `aiui/stores/chat.ts:476-508`, `aiui/data/nodeDemoPrompts.ts` |
|
||||
|
||||
### Wave 1 — Content truth (make the real path rich; stop the fake one)
|
||||
|
||||
1. **One surface mechanism**: tool surfaces are the only path on-node; either teach the
|
||||
node prompt the tag vocabulary (cheap, brittle) or — preferred — extend `SURFACE_TOOLS`
|
||||
coverage and fix the two shape bugs: `apps_list` returns `{items:[…]}`; IndeeHub
|
||||
`films` scope carries a mime/filename hint so classification doesn't exclude it.
|
||||
(`core/assistant/tools.rs:55`, `core/api/rpc/container.rs:247`,
|
||||
`core/api/rpc/content.rs:1211`, `ui/.../contextBroker.ts:300-320`,
|
||||
`ui/.../archyContentAdapter.ts:231-242`)
|
||||
2. **Playback**: `usePlayer.play()` uses `song.sources[]` first (same-origin, CSP-legal),
|
||||
Wavlake only as fallback; `FilmDetail` plays node files via Range requests (the rules
|
||||
already exist — media streams via Range, never base64). (`aiui/composables/usePlayer.ts:235-277`,
|
||||
`aiui/components/content/FilmDetail.vue:134-136`)
|
||||
3. **Stop the mechanisms fighting**: `archyContentActive` must not latch on an *empty*
|
||||
granted bucket; no-tool turns must not wipe populated grids.
|
||||
(`aiui/composables/useContentPanel.ts:210-215,311-355`, `useArchy.ts:365-382`)
|
||||
4. **Mock quarantine**: mocks/fixtures out of the prod bundle (dynamic import behind
|
||||
`isDev`); SYSTEM_PROMPT never presents mocks as the user's library; `/seed` stays as a
|
||||
dev/demo command but is labelled as such. (`aiui/composables/contentExtraction.ts:4-6`,
|
||||
`useAI.ts:23-37`, bundle-verify by grep)
|
||||
5. **Honest rendering**: hide zero-value metadata (no "★ 0 · 0m"), empty states name the
|
||||
actual cause ("no media shared on this node yet" vs "No films match your search"),
|
||||
mount or remove the dead detail views (apps/recipes), delete or wire the seed-only
|
||||
surfaces (recipes/events) — no seed-only surface ships silently.
|
||||
6. **Enrichment production answer** (per operator decision in FOLLOW-ON-SCOPE-PROPOSAL):
|
||||
node-brokered enrichment/search endpoints, default-off, screened; fix the
|
||||
`/api/web-search` vs `/aiui/api/web-search` path bug and give RSS a node route or
|
||||
remove the feature flag. (`aiui/composables/useWebSearch.ts`, `useRssFetch.ts`)
|
||||
7. **Green fixtures**: seed tests pass (fix the 10-vs-6 song extraction or correct the
|
||||
fixture); add a regression test that a fresh prod bundle contains no mock hosts
|
||||
(spotify.com/track/example, cloud.example.com, plex://).
|
||||
|
||||
### Wave 2 — Mission alignment
|
||||
|
||||
1. **Routstr funding UX** (already a captured todo — schedule it): provider dropdown,
|
||||
in-chat funding, Settings modal, budget visible/raised only through trusted chrome.
|
||||
2. **Standalone self-hosting**: wire the shipped Ollama adapter into the selectable
|
||||
provider union; default standalone to local, Claude opt-in.
|
||||
(`aiui/adapters/ollama-adapter.ts`, `aiui/composables/useAI.ts:16,121-123`)
|
||||
3. **Price via the node**: replace browser mempool.space polling with a brokered value
|
||||
from the node's own Bitcoin Core (or drop the widget).
|
||||
4. **Honest stubs**: NWC/LNURL-auth/zap-QR either finished (NIP-47 signing is the blocker)
|
||||
or visibly labelled "preview" in the UI; replace the fake QR with a real one or remove it.
|
||||
5. **Blossom/hashtree**: do not fake. Document in the phase that both wait on the
|
||||
IndeeHub catalog layer (`docs/dht-distribution-design.md`); when that lands, AIUI media
|
||||
moves to blob-addressed storage. No AIUI-side work now.
|
||||
|
||||
### Wave 3 — Already-proposed follow-on (unchanged, from FOLLOW-ON-SCOPE-PROPOSAL)
|
||||
|
||||
Phase A (finish AIUI-02 settings + AIUI-05 delivery), Phase B (node-side relay bridge +
|
||||
enrichment proxy, setting-derived CSP — relays first), Phase C (nostr polish + zaps
|
||||
through the confirm gate). This plan's Wave 0/1 items slot *before* Phase B unblocks more
|
||||
egress — widen the pipe only after the leaks at the current diameter are closed.
|
||||
|
||||
---
|
||||
|
||||
## E. Coordination protocol (another agent is live in this tree)
|
||||
|
||||
- All AIUI-tree work happens in `archy-phase13/aiui/` on
|
||||
`gsd/phase-13-aiui-functional-conversational-node-control-and-content-surf`; node-side
|
||||
in `core/` and `image-recipe/` of the same worktree.
|
||||
- Stage by explicit path; never `git add -A`. Commit + push per unit of work (repo rule).
|
||||
- Wave 0 items S1/S5/S6 touch `core/assistant/` — check `git status`/`git log` for the
|
||||
other agent's in-flight work there before editing; if conflicted, take S2/S3/S4/S7
|
||||
(aiui/nginx side) first.
|
||||
- Do not touch `SongGrid.vue`, `ContentGridView.vue`, `packages/core/src/types/content.ts`
|
||||
(D-12 holds: design stays, only data sources change).
|
||||
|
||||
## F. Verification (per wave, on a real node)
|
||||
|
||||
- **Wave 0**: grant Network → ask "what's my network status" on a cloud leg → assert no
|
||||
WAN IP/SSID in the forwarded body (forwarder log + egress test); fresh profile → no
|
||||
`claudeApiKey` key in localStorage; `/aiui/` without `?embedded=true` on a node refuses
|
||||
or is screened; unauthenticated `curl /aiui/api/web-search` → 401; paste a
|
||||
checksum-valid BIP39 into standalone chat → blocked notice; fresh install → no seeded
|
||||
demo conversation.
|
||||
- **Wave 1**: with a seeded test library on .228/archi-dev-box: a song answer plays
|
||||
in-app from the real source URL; a film answer plays via Range; IndeeHub scope renders
|
||||
cards when content exists; apps tool renders the apps grid; no-tool turn leaves grids
|
||||
intact; prod bundle grep shows zero mock hosts; seed tests green.
|
||||
- **Wave 2**: fund Routstr from the AIUI dropdown end-to-end on regtest-scale budget;
|
||||
standalone default provider is Ollama; price widget works with browser devtools showing
|
||||
no mempool.space call.
|
||||
- Every wave: `tests/lifecycle/run-gate.sh` stays green on .228; AIUI verified in the real
|
||||
embedded iframe, desktop + mobile (AIUI-06 holds).
|
||||
Reference in New Issue
Block a user