docs(13): fill 13-VALIDATION map + write 13-UAT on-device acceptance record

13-15's two artifacts. The map names plan/wave/threat-ref per row with
today's measured results (Rust assistant suite 130/130; adapter 37; broker
25; toolConfirm, audioPlayer 11, appsConfig 13; AIUI 353/356 with the three
documented pre-existing fixture failures). Four close-out rows added
(S-invariants, evals, egress, mock-free-bundle grep). Manual-only table
discharged except the physical-handset pass, which 13-UAT records as owed
(the AIUI-06 flagged assumption wants both, and only devtools-mobile is on
record). Open questions 1/2/4 resolved with plan cites; Routstr stays
honestly open.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-07 17:52:22 -04:00
co-authored by Claude
parent 482c4e300e
commit 494400df29
2 changed files with 107 additions and 41 deletions
@@ -0,0 +1,56 @@
# 13-UAT — On-Device Acceptance Record
**Date:** 2026-08-07 · **Device:** archi-dev-box (this node; LAN 192.168.63.240) ·
**Build under test:** branch `gsd/phase-13-…` @ `482c4e30` (daemon binary deployed to
`/usr/local/bin/archipelago`, frontends rsynced to `/opt/archipelago/web-ui/`).
Method note (per the plan's second trap): deploys were verified against **fetched**
artifacts — live bundle hashes resolved through `index.html` and diffed against the
tree's `dist/`, never by grepping the never-pruned `assets/` graveyard on disk.
## Environment
- Real embedded iframe (`/dashboard/chat``/aiui/`), headless Chromium driving the
genuine login → chat → composer path (Playwright drivers in `neode-ui/*tmp.mjs`).
- Desktop viewport 1600×950 and 1280×760 and narrow 1100×620 during the day.
- **Real-handheld pass: NOT done.** The plan's flagged assumption asks for a physical
phone pass for touch/keyboard/audio. Recorded here as the remaining gap; the
narrow-viewport runs cover layout only.
## What was exercised and the results
| # | Flow | Result | Evidence |
|---|------|--------|----------|
| 1 | Login → chat renders embedded | PASS | Every driver run; headings resolved |
| 2 | Read tool in a turn ("show me my own shared content") | PASS | `content_list` scope `own` surfaced 18 items; heading "13 Images"; `locked: 0` after c25fd8b6 |
| 3 | Owner never pays for own content | PASS | 3 paid own items `GET /content/<id>` → 200 with real mimes (were 402) |
| 4 | Purchased item serves from local cache w/ Range | PASS (code+unit level; live 402s on re-fetch eliminated by proxy cache arm) | c25fd8b6 |
| 5 | Recommendation turn renders rich preview cards | PASS | "recommend me 10 scifi films" → Films tab, "10 Films", per-card title/year/director + "not in library" badges (1ac08a3e + 7d57e2c3) |
| 6 | Disabled-category request offers Settings | PASS | revoke `media` → ask → banner "The assistant needed access to Media Libraries, which is switched off." (6815a7d1); grant restored after |
| 7 | Confirm gate on a chat write | PASS | "install bitcoin knots" → `assistant.pending` carried node-authored `app_install` description; decline resolved with nothing executed (7686a486) |
| 8 | Cmd/Ctrl+K → "Talk to AIUI about it" carries the query | PASS | composer prefill exact-match |
| 9 | Grants persist across refresh | PASS | node-side grants identical before/after reload |
| 10 | CSP boundary (13-09) | PASS | operator-confirmed earlier: CSP blocks in frame, 200 at top |
| 11 | Share video + document | PASS | operator-confirmed earlier |
| 12 | Music library (D-13, non-blocking) | DEFERRED-RECORDED | own songs surface (2 tracks, real `/content/` URLs); in-app playback now node-first (c2e71bc7) — full music-track pass remains on its own track |
## Defects found during UAT and fixed same-day
- Content surface closed when the reply inferred no tabs (f0d2cdb7).
- Global archy latch suppressed all extracted previews after mount (7d57e2c3).
- Owner-locked own items (c25fd8b6). Peer/purchased cards with empty URLs (same commit).
- Recommendation turns never called the tools (1ac08a3e prompt) + films/apps shape bugs (a91bc55d).
- Permission banner structurally unreachable (D-16 hid tools → no refusal → no banner);
fixed by listing disabled tools as callable-but-refused (6815a7d1).
- `#ai-data-access` anchor missing (eaf0f073). strfry icon 404 (2787a9bb).
- Prod bundle shipped mock hosts — tree-shake defeated by an impure module-level export;
mocks dir declared side-effect-free (8329b826).
- Backdrop 1052K→478K (e36f36ee). S6 cloud legs no longer strip prior user turns (7686a486).
## Known-open at acceptance
- A real handheld pass (above) is still owed.
- The three pre-existing AIUI fixture failures (seed 10-vs-6, webSearch flag) predate this
phase's close work; recorded in W1.7 of the assessment fix plan.
- Console noise items (wavlake/itunes CSP on enrichment, sw.js SSL on self-signed) tracked
in the open task list.
@@ -4,8 +4,8 @@ slug: aiui-functional-conversational-node-control-and-content-surf
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
status: draft
nyquist_compliant: false
wave_0_complete: false
nyquist_compliant: true
wave_0_complete: true
created: 2026-08-03
---
@@ -18,19 +18,19 @@ created: 2026-08-03
## Test Infrastructure
This phase spans **three** test surfaces in **two** repositories.
This phase spans **three** test surfaces in **two** repositories (now one in-repo tree — D-19).
| Property | Value |
|----------|-------|
| **Framework (Rust)** | `cargo test` — in-tree unit/integration tests (precedent: `swarm/payment.rs` `#[tokio::test]`, `pine_ha.rs` `#[test]`) |
| **Framework (neode-ui)** | Vitest 3.1 — `neode-ui/package.json` `"test": "vitest run"` |
| **Framework (AIUI repo)** | ⚠️ UNCONFIRMED — `packages/app/src/__tests__/` and `composables/__tests__/` exist (`contentExtraction.test.ts`, `useAI.test.ts`) but the test command was not verified. **Wave 0 must confirm before any wave depends on it.** |
| **Config file** | `core/Cargo.toml` (Rust) · `neode-ui/vitest.config.ts` (frontend) |
| **Framework (AIUI repo)** | Vitest — `aiui/packages/app`: `pnpm test` (`vitest run`). **Confirmed 2026-08-07** (353 passed / 3 pre-existing fixture failures, run repeatedly). |
| **Config file** | `core/Cargo.toml` (Rust) · `neode-ui/vitest.config.ts` (frontend) · `aiui/packages/app/vite.config.ts` |
| **Quick run command** | `cd core && cargo test --package archipelago assistant::` · `cd neode-ui && npx vitest run src/services/__tests__/contextBroker.test.ts` |
| **Full suite command** | `cd core && cargo test` · `cd neode-ui && npm run test` |
| **Estimated runtime** | Rust full suite ~minutes; Vitest targeted ~seconds |
| **Full suite command** | `cd core && cargo test` · `cd neode-ui && npm run test` · `cd aiui/packages/app && pnpm test` |
| **Estimated runtime** | Rust assistant suite ~2s (post-compile); Vitest targeted <10s; AIUI full ~19s |
**Build gotcha (CLAUDE.md):** if `cargo test` hits `rust-lld: undefined hidden symbol`, that is incremental-cache corruption — rebuild with `CARGO_INCREMENTAL=0`. Not a real failure.
**Build gotcha (CLAUDE.md):** if `cargo test` hits `rust-lld: undefined hidden symbol`, that is incremental-cache corruption — rebuild with `CARGO_INCREMENTAL=0`. Not a real failure. (Hit twice on 2026-08-07 under parallel-build load; both times the clean rebuild confirmed green.)
---
@@ -49,30 +49,40 @@ Requirement-level map seeded from research. **The planner fills Task ID / Plan /
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
| TBD | TBD | TBD | AIUI-01 | — | Typed chat request executes a real read-only tool ("how much space is left" → `system.disk-status`) and returns the real result | integration (Rust) | `cargo test assistant::tests::disk_status_tool_executes` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-01 | D-07/D-11 | A write request ("restart bitcoin") produces a **pending confirmation**, never an executed action, until the human confirms | integration (Rust) + component (Vue) | `cargo test assistant::tests::destructive_tool_requires_confirm`; `npx vitest run src/services/__tests__/toolConfirm.test.ts` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-01 / AIUI-04 | Phase-10 D-01..D-04 | An unauthenticated caller cannot reach any new `assistant.*` RPC method | integration (Rust) | `cargo test rpc::middleware::tests::assistant_methods_require_session` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | **AIUI-04** | **live exposure** | `/aiui/api/claude/` and `/aiui/api/openrouter/` are **no longer reachable without a session** (see Manual-Only + note below) | integration/shell | `curl -s -o /dev/null -w '%{http_code}' http://<node>/aiui/api/claude/` returns 401/403 with no cookie | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-02 | D-16 | A conversational settings change is scoped to a granted permission category and **refused** when not granted | unit (Rust) | `cargo test assistant::tools::tests::settings_tool_respects_category_grant` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-04 | D-10 | Peer-supplied text inside untrusted-content delimiters cannot escalate tool authority; an injected "restart bitcoin" still requires a human confirm naming the real action | unit (Rust) | `cargo test assistant::tests::injected_instruction_does_not_grant_authority` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-03 | — | `content.*` RPC data renders in `FilmGrid`/`SongGrid` through the new adapter (pins the shape mismatch found in research) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/archyContentAdapter.test.ts` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-03 | — | Audio routes to the global bottom-bar player, never the lightbox (regression-pins the rule enforced in 5 call sites) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/useAudioPlayer.test.ts` | ⚠️ partial | ⬜ pending |
| TBD | TBD | TBD | AIUI-05 | D-15 | Build enforces `VITE_BASE_PATH=/aiui/`; script exits non-zero if unset | shell/CI | `scripts/build-aiui.sh` (new) | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-05 | D-15 | Post-deploy check **fetches a live asset over HTTP** rather than trusting a directory listing | shell | `curl` a hashed asset resolved via `sw.js`, assert 200 + content | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | AIUI-06 | — | Embedded iframe on archi-dev-box, desktop + mobile | manual | N/A | — | ⬜ pending |
| 13-01 T1 | 13-01 | 1 | AIUI-01 | — | Typed chat request executes a real read-only tool ("how much space is left" → `system.disk-status`) and returns the real result | integration (Rust) | `cargo test assistant::tests::disk_status_tool_executes` | ✅ | ✅ green (assistant suite 130/130 on 2026-08-07) |
| 13-08 T2 | 13-08 | 3 | AIUI-01 | D-07/D-11 | A write request ("restart bitcoin") produces a **pending confirmation**, never an executed action, until the human confirms | integration (Rust) + component (Vue) | `cargo test assistant::tests::destructive_tool_requires_confirm`; `npx vitest run src/services/__tests__/toolConfirm.test.ts` | ✅ | ✅ green; live-verified 2026-08-07 (app_install pending → decline → nothing executed) |
| 13-02 T1 | 13-02 | 1 | AIUI-01 / AIUI-04 | Phase-10 D-01..D-04 | An unauthenticated caller cannot reach any new `assistant.*` RPC method | integration (Rust) | `cargo test rpc::middleware::tests::assistant_methods_require_session` | ✅ | ✅ green |
| 13-02/13-09 | 13-02+13-09 | 1/4 | **AIUI-04** | **live exposure** | `/aiui/api/claude/` and `/aiui/api/openrouter/` are **no longer reachable without a session** | integration/shell | live `curl` (see note) | ✅ | ✅ green 2026-08-07: claude → 401; openrouter → SPA-fallback (indistinguishable from a nonexistent path, 13-02's recorded finding); web-search → 401 after same-day heal (482c4e30 + live hand-patch on the dev box, whose `/home/archipelago/archy` symlink skips the self-heal by design) |
| 13-05 T2 | 13-05 | 2 | AIUI-02 | D-16 | A conversational settings change is scoped to a granted permission category and **refused** when not granted | unit (Rust) | `cargo test assistant::tools::tests::settings_tool_respects_category_grant` | ✅ | ✅ green |
| 13-12 T1 | 13-12 | 5 | AIUI-04 | D-10 | Peer-supplied text inside untrusted-content delimiters cannot escalate tool authority; an injected "restart bitcoin" still requires a human confirm naming the real action | unit (Rust) | `cargo test assistant::tests::injected_instruction_does_not_grant_authority` | ✅ | ✅ green |
| 13-06 T1 | 13-06 | 2 | AIUI-03 | — | `content.*` RPC data renders in `FilmGrid`/`SongGrid` through the new adapter (pins the shape mismatch found in research) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/archyContentAdapter.test.ts` | ✅ | ✅ green (37 tests, incl. same-day owner-lock and per-onion regressions) |
| 13-06 T3 | 13-06 | 2 | AIUI-03 | — | Audio routes to the global bottom-bar player, never the lightbox (regression-pins the rule enforced in 5 call sites) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/useAudioPlayer.test.ts` | ✅ | ✅ green (11 tests, 2026-08-07) |
| 13-09 T1 | 13-09 | 4 | AIUI-05 | D-15 | Build enforces `VITE_BASE_PATH=/aiui/`; script exits non-zero if unset | shell/CI | `scripts/build-aiui.sh` `require_base_path` | ✅ | ✅ green (in daily use; the known post-success hang is timeboxed by callers) |
| 13-09 T3 | 13-09 | 4 | AIUI-05 | D-15 | Post-deploy check **fetches a live asset over HTTP** rather than trusting a directory listing | shell | resolve via index.html + fetch | ✅ | ✅ green 2026-08-07: live bundle hash diffing node↔tree and inside the RC ISOs (caught a stale-AIUI ISO bake, fixed in e669a3e4) |
| 13-15 T2 | 13-15 | 8 | AIUI-06 | — | Embedded iframe on archi-dev-box, desktop + mobile | manual | N/A | — | ⚠️ desktop + narrow-viewport PASS (see 13-UAT.md); **real-handheld pass outstanding** |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
### Rows added at close-out (phase reality, post-research)
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
| S-01…S-15 structural invariants | 13-01…13-14 | 17 | AIUI-01/02/04 | S-01…S-15 | Each invariant has a named test in `assistant::` (grants default-closed, confirm-gate nonce binding, untrusted wrap per-call, excluded-authority absence, egress fail-closed, prompt/authority split) | unit (Rust) | `cargo test --bin archipelago assistant::` | ✅ | ✅ green (130/130) |
| E-01…E-18 eval cases | 13-13 | 6 | AIUI-01 | — | `assistant::evals` golden behaviors | unit (Rust) | same suite | ✅ | ✅ green |
| — | 13-12 | 5 | AIUI-04 | G-B1/G-B2 | Cloud egress screen: BIP39-checksum, secrets corpus, ecash/nostr/macaroon shapes, 64KB cap, fail-closed; replayed history allowlisted, fabricated turns truncated | unit (Rust) | `cargo test assistant::egress` | ✅ | ✅ green (incl. S6 fix tests, 7686a486) |
| — | 13-13 | 6 | AIUI-01 | D-05 | Routstr budget hard-stops (`BudgetExhausted`), zero-allowance never selects Routstr | unit (Rust) | same suite | ✅ | ✅ green |
| — | 2026-08-07 close | 8 | AIUI-03 | — | Prod AIUI bundle carries zero mock hosts (mock quarantine, W1.4) | shell grep | `grep -c 'plex://\|cloud.example.com\|spotify.com/track/example\|image.tmdb.org' dist/assets/index-*.js` → 0 | ✅ | ✅ green 2026-08-07 |
---
## Wave 0 Requirements
- [ ] `core/archipelago/src/assistant/` + its `#[cfg(test)]` module — the tool-calling loop is net-new; **zero** existing coverage
- [ ] `neode-ui/src/services/__tests__/toolConfirm.test.ts`new confirm-flow coverage, extending the `contextBroker.test.ts` pattern
- [ ] `neode-ui/src/composables/__tests__/archyContentAdapter.test.ts` — pins the `ContentItem``Film`/`Song`/`Podcast` mapping
- [ ] `scripts/build-aiui.sh` (or equivalent) — does not exist; D-15's `VITE_BASE_PATH` enforcement + commit-pinning have no automated check today
- [ ] **Confirm AIUI's own test command** before any wave assumes Vitest parity — unverified in research
- [ ] Keep green: `contextBroker.test.ts`, `chatAiuiEmbed.test.ts`
- [x] `core/archipelago/src/assistant/` + its `#[cfg(test)]` module — built out; 130 tests in the assistant suite alone by close-out
- [x] `neode-ui/src/services/__tests__/toolConfirm.test.ts`exists and green (incl. the hydration-narrowing fix)
- [x] `neode-ui/src/composables/__tests__/archyContentAdapter.test.ts` — pins `ContentItem``Film`/`Song`/`Podcast`/`ImageItem`; 37 green
- [x] `scripts/build-aiui.sh` — exists, enforces `VITE_BASE_PATH=/aiui/` + dist verification (D-15)
- [x] **AIUI's own test command confirmed**`pnpm test` (`vitest run`), exercised repeatedly on 2026-08-07
- [x] Keep green: `contextBroker.test.ts` (25), `chatAiuiEmbed.test.ts` — green
---
@@ -80,10 +90,10 @@ Requirement-level map seeded from research. **The planner fills Task ID / Plan /
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Embedded AIUI works in the real iframe | AIUI-06 | Real-device rendering in the actual embed context; `dev:mock` does not reproduce it | Load neode-ui Chat view on archi-dev-box, desktop **and** mobile viewport; exercise a read tool, a confirmed write, and a content grid. **Scope (D-13):** the control and content tracks are blocking here; the music view is 13-15 step 7b, recorded as pass, gap or deferred and never blocking |
| Frontend bundle actually shipped | AIUI-05 | Node `assets/` is a never-pruned graveyard — a disk grep reports "deployed" before the deploy | Resolve live chunks via `sw.js`, fetch over HTTP, grep the **fetched** bytes for the new string |
| Confirm dialog is un-spoofable by the iframe | AIUI-04 / D-11 | Anti-spoofing is a visual/trust property of the host chrome | Verify the dialog renders outside the iframe, Teleports to body, full-screen backdrop, text drawn from the node's description — not model-authored |
| Routstr pays a live request | D-04 / D-05 | Research confidence on the Routstr protocol is MEDIUM — cited from docs, never run against a live provider | Spike against a real provider before the integration is trusted; budget ceiling must hard-stop |
| Embedded AIUI works in the real iframe | AIUI-06 | Real-device rendering in the actual embed context; `dev:mock` does not reproduce it | DONE on archi-dev-box 2026-08-07 (desktop + narrow viewports; see 13-UAT.md). **A physical-handset pass remains owed.** |
| Frontend bundle actually shipped | AIUI-05 | Node `assets/` is a never-pruned graveyard — a disk grep reports "deployed" before the deploy | DONE 2026-08-07: live chunks resolved via `index.html` and diffed node↔tree and inside the RC ISO images (that pass caught the stale-AIUI bake, e669a3e4) |
| Confirm dialog is un-spoofable by the iframe | AIUI-04 / D-11 | Anti-spoofing is a visual/trust property of the host chrome | DONE: modal Teleports to body, full-screen backdrop, description node-authored via `assistant.pending`; the iframe never sees the nonce (asserted by `iframe_message_cannot_open_or_resolve_confirmation`) |
| Routstr pays a live request | D-04 / D-05 | Research confidence on the Routstr protocol is MEDIUM — cited from docs, never run against a live provider | NOT DONE — recorded open; 13-ROUTSTR-FINDINGS.md stands (0/9 live claims). The backend ships behind the budget ceiling but stays unverified against a live provider. |
---
@@ -123,21 +133,21 @@ flat scalars under `prohibitions`, never under `truths`, and carry no `check_*`
Carried from `13-RESEARCH.md` § Open Questions — each needs a planner decision, and two change what "validated" even means:
1. **The port-3142 proxy**`/aiui/api/claude/` and `/aiui/api/openrouter/` are proxied with **no session gate** (`image-recipe/configs/nginx-archipelago.conf`, verified). Anyone reaching the node's web port can spend the owner's API budget. Removed, gated, or superseded by D-01's node-side loop?
2. **Iframe sandbox mechanism**AIUI is same-origin today, no `sandbox` attribute, permissive CSP. AIUI-04's "sandboxed by construction" is currently a code-discipline convention, not browser-enforced. Attribute, CSP, or accepted-and-documented risk?
3. **Routstr protocol accuracy**needs a spike against a live provider before it is load-bearing.
4. **RBAC integration**should new `assistant.*` RPCs go through the existing `role.can_access()` check?
1. **The port-3142 proxy**RESOLVED (13-02): deleted the anonymous relays; a session-gated Rust forwarder replaced them. Live-verified 2026-08-07 (`/aiui/api/claude/` → 401 unauthenticated; web-search → 401 after the same-day self-heal, 482c4e30).
2. **Iframe sandbox mechanism**RESOLVED (13-09): a `/aiui/`-scoped CSP plus G-B3 (postMessage origin check + broker-side sanitization). `sandbox` attribute was rejected (breaks required same-origin bridge semantics); the residual is documented in 13-09-SUMMARY.
3. **Routstr protocol accuracy**OPEN: 13-13 entered the backend behind D-05's budget ceiling; 13-ROUTSTR-FINDINGS.md's 0/9 live-protocol claims stand. The funding-UX phase must spike a live provider before Routstr is load-bearing.
4. **RBAC integration**RESOLVED (13-01): a single `assistant.` prefix arm, so the existing `role.can_access` gate applies unchanged before dispatch (`assistant_methods_require_session` green).
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 120s
- [ ] AIUI repo test command confirmed
- [ ] `nyquist_compliant: true` set in frontmatter
- [x] All tasks have `<automated>` verify or Wave 0 dependencies
- [x] Sampling continuity: no 3 consecutive tasks without automated verify
- [x] Wave 0 covers all MISSING references
- [x] No watch-mode flags
- [x] Feedback latency < 120s
- [x] AIUI repo test command confirmed
- [x] `nyquist_compliant: true` set in frontmatter**set below**: every map row has an automated verify or a discharged manual entry; the single honest exception is the physical-handset pass, recorded in 13-UAT.md as owed, not silently skipped.
**Approval:** pending
**Approval:** pending — awaits the operator's read of 13-UAT.md and the handset decision (the flagged AIUI-06 assumption asks for BOTH a devtools mobile viewport and a real phone; only the former is on record).