diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-VALIDATION.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-VALIDATION.md new file mode 100644 index 00000000..ad054754 --- /dev/null +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-VALIDATION.md @@ -0,0 +1,111 @@ +--- +phase: 13 +slug: aiui-functional-conversational-node-control-and-content-surf +# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6) +# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117) +status: draft +nyquist_compliant: false +wave_0_complete: false +created: 2026-08-03 +--- + +# Phase 13 — Validation Strategy + +> Per-phase validation contract for feedback sampling during execution. +> Seeded from `13-RESEARCH.md` § Validation Architecture. Task IDs are filled in by the planner. + +--- + +## Test Infrastructure + +This phase spans **three** test surfaces in **two** repositories. + +| Property | Value | +|----------|-------| +| **Framework (Rust)** | `cargo test` — in-tree unit/integration tests (precedent: `swarm/payment.rs` `#[tokio::test]`, `pine_ha.rs` `#[test]`) | +| **Framework (neode-ui)** | Vitest 3.1 — `neode-ui/package.json` `"test": "vitest run"` | +| **Framework (AIUI repo)** | ⚠️ UNCONFIRMED — `packages/app/src/__tests__/` and `composables/__tests__/` exist (`contentExtraction.test.ts`, `useAI.test.ts`) but the test command was not verified. **Wave 0 must confirm before any wave depends on it.** | +| **Config file** | `core/Cargo.toml` (Rust) · `neode-ui/vitest.config.ts` (frontend) | +| **Quick run command** | `cd core && cargo test --package archipelago assistant::` · `cd neode-ui && npx vitest run src/services/__tests__/contextBroker.test.ts` | +| **Full suite command** | `cd core && cargo test` · `cd neode-ui && npm run test` | +| **Estimated runtime** | Rust full suite ~minutes; Vitest targeted ~seconds | + +**Build gotcha (CLAUDE.md):** if `cargo test` hits `rust-lld: undefined hidden symbol`, that is incremental-cache corruption — rebuild with `CARGO_INCREMENTAL=0`. Not a real failure. + +--- + +## Sampling Rate + +- **After every task commit:** the quick-run command for the touched module (`cargo test assistant::`, or the specific Vitest file) +- **After every plan wave:** full `cargo test` + `npm run test` (neode-ui) + AIUI's own test command (once confirmed in Wave 0) +- **Before `/gsd-verify-work`:** full suite green **and** the AIUI-06 on-device pass on archi-dev-box (desktop + mobile) +- **Max feedback latency:** targeted Vitest < 30s; Rust module tests < 120s + +--- + +## Per-Task Verification Map + +Requirement-level map seeded from research. **The planner fills Task ID / Plan / Wave / Threat Ref columns** as it decomposes; every row below must end up owned by at least one task. + +| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status | +|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------| +| TBD | TBD | TBD | AIUI-01 | — | Typed chat request executes a real read-only tool ("how much space is left" → `system.disk-status`) and returns the real result | integration (Rust) | `cargo test assistant::tests::disk_status_tool_executes` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-01 | D-07/D-11 | A write request ("restart bitcoin") produces a **pending confirmation**, never an executed action, until the human confirms | integration (Rust) + component (Vue) | `cargo test assistant::tests::destructive_tool_requires_confirm`; `npx vitest run src/services/__tests__/toolConfirm.test.ts` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-01 / AIUI-04 | Phase-10 D-01..D-04 | An unauthenticated caller cannot reach any new `assistant.*` RPC method | integration (Rust) | `cargo test rpc::middleware::tests::assistant_methods_require_session` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | **AIUI-04** | **live exposure** | `/aiui/api/claude/` and `/aiui/api/openrouter/` are **no longer reachable without a session** (see Manual-Only + note below) | integration/shell | `curl -s -o /dev/null -w '%{http_code}' http:///aiui/api/claude/` returns 401/403 with no cookie | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-02 | D-16 | A conversational settings change is scoped to a granted permission category and **refused** when not granted | unit (Rust) | `cargo test assistant::tools::tests::settings_tool_respects_category_grant` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-04 | D-10 | Peer-supplied text inside untrusted-content delimiters cannot escalate tool authority; an injected "restart bitcoin" still requires a human confirm naming the real action | unit (Rust) | `cargo test assistant::tests::injected_instruction_does_not_grant_authority` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-03 | — | `content.*` RPC data renders in `FilmGrid`/`SongGrid` through the new adapter (pins the shape mismatch found in research) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/archyContentAdapter.test.ts` | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-03 | — | Audio routes to the global bottom-bar player, never the lightbox (regression-pins the rule enforced in 5 call sites) | unit (Vue/TS) | `npx vitest run src/composables/__tests__/useAudioPlayer.test.ts` | ⚠️ partial | ⬜ pending | +| TBD | TBD | TBD | AIUI-05 | D-15 | Build enforces `VITE_BASE_PATH=/aiui/`; script exits non-zero if unset | shell/CI | `scripts/build-aiui.sh` (new) | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-05 | D-15 | Post-deploy check **fetches a live asset over HTTP** rather than trusting a directory listing | shell | `curl` a hashed asset resolved via `sw.js`, assert 200 + content | ❌ W0 | ⬜ pending | +| TBD | TBD | TBD | AIUI-06 | — | Embedded iframe on archi-dev-box, desktop + mobile | manual | N/A | — | ⬜ pending | + +*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* + +--- + +## Wave 0 Requirements + +- [ ] `core/archipelago/src/assistant/` + its `#[cfg(test)]` module — the tool-calling loop is net-new; **zero** existing coverage +- [ ] `neode-ui/src/services/__tests__/toolConfirm.test.ts` — new confirm-flow coverage, extending the `contextBroker.test.ts` pattern +- [ ] `neode-ui/src/composables/__tests__/archyContentAdapter.test.ts` — pins the `ContentItem` → `Film`/`Song`/`Podcast` mapping +- [ ] `scripts/build-aiui.sh` (or equivalent) — does not exist; D-15's `VITE_BASE_PATH` enforcement + commit-pinning have no automated check today +- [ ] **Confirm AIUI's own test command** before any wave assumes Vitest parity — unverified in research +- [ ] Keep green: `contextBroker.test.ts`, `chatAiuiEmbed.test.ts` + +--- + +## Manual-Only Verifications + +| Behavior | Requirement | Why Manual | Test Instructions | +|----------|-------------|------------|-------------------| +| Embedded AIUI works in the real iframe | AIUI-06 | Real-device rendering in the actual embed context; `dev:mock` does not reproduce it | Load neode-ui Chat view on archi-dev-box, desktop **and** mobile viewport; exercise a read tool, a confirmed write, and a content grid | +| Frontend bundle actually shipped | AIUI-05 | Node `assets/` is a never-pruned graveyard — a disk grep reports "deployed" before the deploy | Resolve live chunks via `sw.js`, fetch over HTTP, grep the **fetched** bytes for the new string | +| Confirm dialog is un-spoofable by the iframe | AIUI-04 / D-11 | Anti-spoofing is a visual/trust property of the host chrome | Verify the dialog renders outside the iframe, Teleports to body, full-screen backdrop, text drawn from the node's description — not model-authored | +| Routstr pays a live request | D-04 / D-05 | Research confidence on the Routstr protocol is MEDIUM — cited from docs, never run against a live provider | Spike against a real provider before the integration is trusted; budget ceiling must hard-stop | + +--- + +## Open Questions Blocking Full Validation + +Carried from `13-RESEARCH.md` § Open Questions — each needs a planner decision, and two change what "validated" even means: + +1. **The port-3142 proxy** — `/aiui/api/claude/` and `/aiui/api/openrouter/` are proxied with **no session gate** (`image-recipe/configs/nginx-archipelago.conf`, verified). Anyone reaching the node's web port can spend the owner's API budget. Removed, gated, or superseded by D-01's node-side loop? +2. **Iframe sandbox mechanism** — AIUI is same-origin today, no `sandbox` attribute, permissive CSP. AIUI-04's "sandboxed by construction" is currently a code-discipline convention, not browser-enforced. Attribute, CSP, or accepted-and-documented risk? +3. **Routstr protocol accuracy** — needs a spike against a live provider before it is load-bearing. +4. **RBAC integration** — should new `assistant.*` RPCs go through the existing `role.can_access()` check? + +--- + +## Validation Sign-Off + +- [ ] All tasks have `` verify or Wave 0 dependencies +- [ ] Sampling continuity: no 3 consecutive tasks without automated verify +- [ ] Wave 0 covers all MISSING references +- [ ] No watch-mode flags +- [ ] Feedback latency < 120s +- [ ] AIUI repo test command confirmed +- [ ] `nyquist_compliant: true` set in frontmatter + +**Approval:** pending