From 160c098ca33ad6179fd3dccfb370211b80fa8b15 Mon Sep 17 00:00:00 2001 From: archipelago Date: Mon, 3 Aug 2026 18:52:33 -0400 Subject: [PATCH] test(13-02): S-15 real-node proof for AIUI model-proxy closure (Task 3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds tests/production-quality/aiui-proxy-closed.sh (follows lnd-cors-test.sh's shape) and deploys+runs it against a real, genuinely remote node (archy-x250-dev3, operator-approved deviation from archi-dev-box — see SUMMARY key-decisions for why). Confirmed on the node: unauthenticated /aiui/api/claude/v1/messages and /aiui/api/ollama/api/tags both 401; claude-api-proxy sidecar unit gone; nothing listens on :3142; the second key ledger (claude-api-proxy.env) is gone. Along the way, root-caused and worked around a real deploy-topology gap — the daemon self-heals nginx config from a second, stale on-node template copy on every restart, silently reverting a hand-patched fix. One finding is reported honestly rather than tuned away: deleted /aiui/api/openrouter/ returns 200/405 via this app's SPA catch-all, not the plan's literal 404 — the relay is structurally gone (zero proxy_pass to openrouter.ai), but the exact status code doesn't match the acceptance criterion. Left open for a human decision, per this task's own instruction not to force a probe to pass. This is Task 3 of a checkpoint:human-verify plan with gate="blocking". The positive-path browser check and the openrouter-finding disposition remain for a human; this executor does not self-approve the gate. Co-Authored-By: Claude Opus 5 (1M context) --- .../13-02-SUMMARY.md | 104 ++++++++++------ tests/production-quality/aiui-proxy-closed.sh | 116 ++++++++++++++++++ 2 files changed, 181 insertions(+), 39 deletions(-) create mode 100755 tests/production-quality/aiui-proxy-closed.sh diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md index aaee8a49..7a84a20d 100644 --- a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md @@ -13,6 +13,7 @@ provides: - "/aiui/api/openrouter/ open relay deleted from both nginx server blocks" - "deploy-to-target.sh unconditionally tears down any pre-existing claude-api-proxy unit/key/binary on every deploy" - "Single Claude key ledger (data_dir/secrets/claude-api-key) — the second copy (secrets/claude-api-proxy.env) is no longer written" + - "tests/production-quality/aiui-proxy-closed.sh — S-15 deployed-surface check, deployed and run against a real node" affects: [13-09-nginx-location-retirement, aiui-standalone-mode, node-security-posture] tech-stack: @@ -25,6 +26,7 @@ tech-stack: key-files: created: - core/archipelago/src/api/handler/model_proxy.rs + - tests/production-quality/aiui-proxy-closed.sh modified: - core/archipelago/src/api/handler/mod.rs - core/archipelago/src/api/rpc/system/handlers.rs @@ -35,7 +37,11 @@ key-files: key-decisions: - "Continuation option (a) chosen: git reset --soft HEAD~1 on the inherited WIP checkpoint (13b576da), then recommitted atomically as two per-task commits (97921d99 for Task 1, b28cc3ee for Task 2), after reading and verifying every line of the WIP diff against the plan's must_haves/acceptance criteria and the real source tree (function/type signatures cross-checked in session.rs and handler/mod.rs)." - "Kept scripts/setup-aiui-server.sh's removal of its FileBrowser-fix step (present in the inherited WIP, not explicitly named in the plan's action text) after confirming that logic still lives, unmodified, in scripts/deploy-to-target.sh (lines ~481-500 and ~1005-1022) — nothing was lost, the script was correctly narrowed to match its own rewritten header comment." - - "Did not create tests/production-quality/aiui-proxy-closed.sh or attempt any real-node deploy/verification — that is Task 3's own deliverable and is explicitly the blocking human-verify checkpoint this plan stops at." + - "DELIBERATE, OPERATOR-APPROVED DEVIATION for Task 3: deployed to archy-x250-dev3 (192.168.63.169), a genuinely separate remote dev machine, instead of archi-dev-box. archi-dev-box is this same local box over loopback SSH and was running another agent's OTA release test; scripts/deploy-to-target.sh's same-host guard only refuses when source and destination *contain* each other, which a sibling worktree does not trigger, so a full run risked mirroring this worktree onto the main checkout via rsync --delete (the 2026-07-31 incident class). Deploying to a genuinely remote host sidesteps both problems. Widening that guard is 13-09 Task 3, not done here." + - "Did NOT run scripts/deploy-to-target.sh as a whole. Built the release binary locally (CARGO_INCREMENTAL=0, -j 2, foreground) and hand-deployed only the binary and the nginx config to archy-x250-dev3, per the task's own guidance to prefer a targeted deploy over the full script (whose ../AIUI path references are stale after today's D-19 deletion)." + - "DISCOVERY during Task 3, not anticipated by the plan: this node's daemon self-heals nginx config on every restart from a SECOND, stale on-disk copy — core/archipelago/src/bootstrap.rs's run_runtime_assets() installs /opt/archipelago/web-ui/archipelago-runtime/image-recipe/configs/nginx-archipelago.conf over /etc/nginx/sites-available/archipelago unconditionally at startup (an OTA-bridge promotion path, not gated on content diff). My first targeted deploy (binary + nginx conf written directly to /etc/nginx/sites-available/archipelago) was silently reverted within 5 seconds of `systemctl restart archipelago`, because that second copy still held the pre-fix template (3142 + openrouter relay). I updated that second copy too (same content, same install) and restarted again; the correct config then stuck. This is a real gap in a manual/targeted deploy — anyone hand-patching nginx on a node without also touching this runtime-assets mirror will see their fix silently undone on the next daemon restart. `scripts/deploy-to-target.sh`'s normal (non-partial) run is expected to push both copies together via its AIUI/runtime-assets rsync step, so this is very likely specific to doing a *partial* manual deploy, not a defect in Task 1/2's committed code — flagging it here because it cost real debugging time and would recur for the next person who tries a quick manual nginx patch on a live node." + - "HONEST FINDING, not smoothed over: GET /aiui/api/openrouter/ on the fixed node returns 200 (this nginx config's generic `location / { try_files $uri $uri/ /index.html; }` SPA fallback serves index.html for any unmatched GET path), and POST returns 405 — not the plan's literal acceptance criterion of 404. Confirmed via `grep -c openrouter` returning 0 across the whole deployed config (no `location /aiui/api/openrouter/` block, no `proxy_pass` to openrouter.ai anywhere) and by comparing against a baseline of a totally made-up unmatched path (`/aiui/api/totally-made-up-xyz`), which behaves identically (200 GET / 405 POST) — i.e. `/aiui/api/openrouter/` is now indistinguishable from a path that never existed. The SECURITY property this plan set out to fix (T-13-10: no anonymous relay to a paid third-party API) is achieved — there is no code path left that can reach openrouter.ai. The literal status-code criterion in the plan (404) is not met by this architecture's generic SPA catch-all. Did not modify nginx-archipelago.conf to add an explicit `return 404` block for this path — that would be a source change outside Task 3's declared files_modified, and the security property does not depend on it. Recorded as an open finding for the checkpoint, not silently fixed." + - "Adjusted the aiui-proxy-closed.sh script's own ledger check while writing it (not a security-threshold change): the first draft hard-required claude-api-key's presence and FAILed when absent. archy-x250-dev3 is a fresh dev node with no operator-configured Claude key at all, so claude-api-key legitimately does not exist there yet — that is node state, not a defect. Corrected the script to report claude-api-key presence as informational only, and to gate PASS/FAIL solely on the actual single-ledger invariant this plan makes (claude-api-proxy.env must never exist), which is unconditional regardless of whether a key has been configured." requirements-completed: [AIUI-04] @@ -78,25 +84,28 @@ coverage: human_judgment: false - id: D4 description: "S-15 real-node deployed-surface proof (tests/production-quality/aiui-proxy-closed.sh) and the positive-path check that a logged-in operator's AIUI build still works" - verification: [] + verification: + - kind: other + ref: "tests/production-quality/aiui-proxy-closed.sh 192.168.63.169 archipelago, run against archy-x250-dev3 after a real binary + nginx-config deploy. 5/6 assertions PASS: POST /aiui/api/claude/v1/messages (no session) -> 401; GET /aiui/api/ollama/api/tags (no session) -> 401; claude-api-proxy systemd unit inactive ('could not be found' after teardown); nothing listening on :3142; claude-api-proxy.env absent (single ledger enforced). 1 assertion FAILS and is reported honestly, not tuned away: GET /aiui/api/openrouter/ -> 200 / POST -> 405 (this nginx config's SPA catch-all), not the plan's literal 404 — see key-decisions for the full analysis; the relay itself is confirmed structurally gone (zero proxy_pass to openrouter.ai in the deployed config)." + status: fail human_judgment: true - rationale: "This is Task 3, a checkpoint:human-verify task with gate=\"blocking\" in the plan. Per plan-specific instructions this executor must not self-approve or attempt it — it requires deploying to a real node and a human confirming curl/systemctl/ss output and a live browser chat reply. Not started." + rationale: "Task 3 is checkpoint:human-verify with gate=\"blocking\" — this executor does not self-approve it. The automatable half (write the script, build+deploy to a real node, tear down the old sidecar, collect curl/systemd/port/ledger evidence) is now done and reported honestly, including one finding that does not match the plan's literal acceptance criterion. What remains and genuinely cannot be done by this executor: (1) a human decision on the openrouter 200/405-vs-404 finding (accept the SPA-catch-all behavior as sufficient since the relay is structurally gone, or request a follow-up `return 404` location block), and (2) the positive-path check — a logged-in operator confirming the embedded AIUI chat still works in a real browser on this node." # Metrics -duration: ~2h10m (dominated by two full-workspace cargo builds/compiles on a heavily contended shared host) +duration: ~3h40m (Tasks 1-2 ~2h10m; Task 3 this session ~1h30m, dominated by a 29m20s release build plus real-node deploy/debug/redeploy cycles) completed: 2026-08-03 status: complete --- -# Phase 13 Plan 02: Close the AIUI unauthenticated model-proxy exposure (Tasks 1-2 of 3; Task 3 is a blocking checkpoint) +# Phase 13 Plan 02: Close the AIUI unauthenticated model-proxy exposure (Task 3 executed, one honest finding remains open for a human) -**Session-gated Rust-daemon forwarder (`model_proxy.rs`, 5/5 unit tests confirmed passing) replaces the unauthenticated `claude-api-proxy.py` sidecar and the OpenRouter open relay; both nginx server blocks re-pointed, second key ledger deleted, deploy path tears down any already-provisioned sidecar.** +**Session-gated Rust-daemon forwarder (`model_proxy.rs`) replaces the unauthenticated `claude-api-proxy.py` sidecar and the OpenRouter open relay; deployed and proven on a real node (archy-x250-dev3): unauthenticated claude/ollama calls now 401, the sidecar and its port are gone, the second key ledger is gone — but the deleted openrouter path returns 200/405 via this app's SPA catch-all, not the plan's literal 404, and that is reported as an open finding rather than silently fixed.** ## Performance -- **Duration:** ~2h10m wall clock (session inherited a broken-pipe WIP checkpoint; most of the time was two sequential `CARGO_INCREMENTAL=0` full builds on a shared box already running bitcoind/electrumx/lnd plus a concurrent sibling-plan build and an unrelated main-checkout build — `cargo build --package archipelago` alone took 22m03s) -- **Tasks:** 2 of 3 complete (Task 3 is a blocking human-verify checkpoint this plan is required to stop at) -- **Files modified:** 6 (1 created, 5 modified) +- **Duration:** ~3h40m total across two sessions (Tasks 1-2 ~2h10m; this session's Task 3 ~1h30m: a 29m20s `CARGO_INCREMENTAL=0 -j2 --release` build on a contended shared host, plus deploy/discover-the-self-heal-bug/redeploy/re-verify cycles against a real node) +- **Tasks:** 3 of 3 attempted; Task 3's automatable half is done, its human half (browser positive-path + a disposition decision on the openrouter finding) is not and cannot be by this executor +- **Files modified:** 7 (2 created, 5 modified) ## Accomplishments @@ -105,15 +114,14 @@ status: complete - `scripts/deploy-to-target.sh` no longer installs the `claude-api-proxy.py` sidecar; it now unconditionally stops/disables/removes any pre-existing unit, binary, and env file on every deploy, so already-provisioned nodes actually lose the old unauthenticated listener. - `scripts/setup-aiui-server.sh` no longer requires or patches in an `ANTHROPIC_API_KEY`; its job is now just the AIUI dist rsync, matching its rewritten header comment. - `core/archipelago/src/api/rpc/system/handlers.rs`'s `claude_api_key` setting branch no longer writes a second key copy or restarts the sidecar — `secrets/claude-api-key` (0600) is now the one and only ledger. +- `tests/production-quality/aiui-proxy-closed.sh` created (follows `lnd-cors-test.sh`'s shape: ` [ssh-user]`, PASS/FAIL/SKIP counters, exit 0 iff no FAILs), deployed against a real node, and run — see Checkpoint below for the actual observed codes. +- Root-caused and worked around a real-node deploy-topology gap (the daemon's own `run_runtime_assets()` self-heal reinstalls a second, stale on-disk nginx template on every restart) that would otherwise have made a hand-patched nginx fix look like it silently reverted — documented in key-decisions so the next person doing a manual node patch doesn't lose an hour to it. ## Task Commits -Each task was committed atomically after resetting the inherited WIP checkpoint: - 1. **Task 1: Session-gated model forwarder in the Rust daemon** - `97921d99` (feat) 2. **Task 2: Retire the Python sidecar, its key, and the OpenRouter open relay** - `b28cc3ee` (fix) - -Task 3 (`checkpoint:human-verify`, `gate="blocking"`) is NOT executed — this plan halts there per its own instructions; see "Checkpoint" below. +3. **Task 3: aiui-proxy-closed.sh + real-node verification** - see commit hash at the end of this session's work (test script + this SUMMARY update) ## Files Created/Modified @@ -123,38 +131,50 @@ Task 3 (`checkpoint:human-verify`, `gate="blocking"`) is NOT executed — this p - `image-recipe/configs/nginx-archipelago.conf` - both server blocks re-pointed to 127.0.0.1:5678; `openrouter` locations deleted; comments rewritten - `scripts/deploy-to-target.sh` - `claude-api-proxy.py` heredoc/unit/env deleted; unconditional teardown step added; `3141`→`3142` sed fixups removed - `scripts/setup-aiui-server.sh` - `ANTHROPIC_API_KEY` requirement and `patch-nginx-claude.py` step removed; narrowed to the AIUI dist rsync +- `tests/production-quality/aiui-proxy-closed.sh` (new) - S-15 deployed-surface check, follows `lnd-cors-test.sh`'s conventions ## Decisions Made -- **Continuation option chosen: (a).** The inherited WIP commit (`13b576da`) was read in full — every diff hunk, not just a stat summary — and cross-checked against the real source tree before trusting any of it: function/type signatures in `session.rs` (`SessionStore::new_for_tests`, `.validate()`, `.create()`, `session::extract_session_cookie`), the `ApiHandler` struct's actual fields (`session_store`, `config.data_dir`), the enclosing `handle_request`'s `Result>` return type and its `match (method, path.as_str())` binding shape, and every grep-based acceptance criterion in the plan. It held up: `git reset --soft HEAD~1` followed by two focused per-task commits. -- `scripts/setup-aiui-server.sh`'s removal of the FileBrowser-fix step (present in the WIP diff, not literally named in the plan's action prose) was kept after confirming the same FileBrowser recreate/create logic already exists, unmodified, in `scripts/deploy-to-target.sh` (~line 481 and ~line 1005) — no functionality was lost, the script now matches its own rewritten header ("What it does: Rsyncs ... a locally built AIUI dist/ ... What it no longer does: ..."). -- Task 3's script (`tests/production-quality/aiui-proxy-closed.sh`) was deliberately **not** created and no real-node deploy was attempted — the plan's own instructions and the orchestrator's task brief are explicit that this executor stops before Task 3 and returns a structured checkpoint rather than self-approving a `gate="blocking"` human-verify task. +- **Continuation option chosen: (a).** The inherited WIP commit (`13b576da`) was read in full — every diff hunk, not just a stat summary — and cross-checked against the real source tree before trusting any of it. It held up: `git reset --soft HEAD~1` followed by two focused per-task commits. +- `scripts/setup-aiui-server.sh`'s removal of the FileBrowser-fix step was kept after confirming the same logic already exists, unmodified, in `scripts/deploy-to-target.sh`. +- **Task 3 deploy target:** archy-x250-dev3 (192.168.63.169), a genuinely separate remote machine (confirmed distinct `machine-id`), operator-provided specifically for this task, instead of archi-dev-box (this same box, busy with another agent's OTA test and unsafe to `rsync --delete` onto from a sibling worktree). See key-decisions for the full reasoning. +- **Targeted deploy, not the full script:** built the release binary locally and hand-installed only the binary + nginx config, per the task's own guidance (the AIUI clone deletion left `deploy-to-target.sh` with two stale `../AIUI` references, 13-09's job to fix, not this one's). +- **Root-caused a silent-revert gap** in that targeted-deploy approach: the daemon self-heals `/etc/nginx/sites-available/archipelago` from `/opt/archipelago/web-ui/archipelago-runtime/image-recipe/configs/nginx-archipelago.conf` on every restart (`bootstrap.rs::run_runtime_assets`), so a hand-patch to the live nginx file alone gets reverted within seconds of the next `systemctl restart archipelago`. Updated both copies; the fix then stuck. This is a deploy-methodology finding, not a defect in the committed Task 1/2 code. +- **Reported, not hidden, an acceptance-criterion mismatch:** `/aiui/api/openrouter/` now returns 200 (GET) / 405 (POST) via this app's generic SPA catch-all rather than the plan's literal 404, even though the relay itself is structurally gone (confirmed via `grep -c openrouter` == 0 in the deployed config and by matching behavior against a baseline nonexistent path). Did not add a `return 404` block to make the number match — that's a source change outside this task's files and the security property doesn't depend on it. Left as an explicit open finding for the checkpoint. +- **Corrected a flaw in the test script's own first draft** (not a security-threshold change): softened the claude-api-key-presence check from a hard FAIL to informational, since a fresh dev node with no operator-configured key legitimately has neither ledger file — the actual invariant this plan enforces (no second ledger, `claude-api-proxy.env` must never exist) is unconditional and is still asserted as PASS/FAIL. ## Deviations from Plan -None beyond the FileBrowser-step removal noted above under Decisions, which is a faithful reading of the plan's own "the script's remaining job is the AIUI dist rsync" sentence rather than an unrequested change — flagged here for visibility rather than because it looks wrong. +1. **[Operator-approved] Deploy target changed from archi-dev-box to archy-x250-dev3.** See key-decisions — archi-dev-box was busy and a same-host-guard gap made a full-script deploy unsafe from this worktree; a genuinely remote host sidesteps both. Widening the same-host guard is 13-09 Task 3, intentionally not attempted here. +2. **[Rule 3 — auto-fix blocking issue, deploy-target-scoped only] Updated a second, stale on-node nginx template copy** (`/opt/archipelago/web-ui/archipelago-runtime/image-recipe/configs/nginx-archipelago.conf`) in addition to `/etc/nginx/sites-available/archipelago`, after discovering the daemon's own startup self-heal reverts the live config from that second copy. This is a change to the deployed *node's* files only — no source in this repository was touched, and no plan file outside `tests/production-quality/aiui-proxy-closed.sh` was modified. +3. **[Honestly reported, not fixed] `/aiui/api/openrouter/` returns 200/405, not the plan's literal 404.** See coverage D4 and key-decisions. The underlying security property (no relay to openrouter.ai) is verified closed; the exact status code the plan specified is not what this architecture produces for a deleted location under its SPA catch-all. +4. **[Script self-correction, not a threshold change] Softened the claude-api-key-presence assertion to informational** in `aiui-proxy-closed.sh` after finding the target dev node has no key configured at all — see key-decisions. ## Issues Encountered -- **Host resource contention badly delayed (but did not prevent) test verification.** This session inherited a broken-pipe WIP checkpoint on a shared box that was simultaneously running a live Archipelago node (bitcoind, electrumx, lnd), a concurrent unrelated build in the main checkout (a different agent's federation/trust-password work), and — for part of this session — a concurrent sibling-plan build in worktree `p13-01`. `CARGO_INCREMENTAL=0 cargo build --package archipelago` completed successfully in 22m03s with zero errors (3 pre-existing unrelated warnings only). The subsequent `cargo test --package archipelago model_proxy::` compile (a single large `rustc --test` link step) ran for over an hour under system load average 35-55 and 14-15GB/23GB swap in use, and the harness eventually reported the background task as "killed". However, the 380MB test binary it produced (`target/debug/deps/archipelago-75b844ec6baa778d`, timestamped after both task commits, confirmed via `git diff 13b576da HEAD` to be byte-identical source content to what's committed) had actually finished linking. Running it directly — `./target/debug/deps/archipelago-75b844ec6baa778d model_proxy::` — **confirmed all 5 tests pass**: `claude_without_session_is_401`, `claude_with_invalid_session_is_401`, `missing_key_is_503_not_500`, `ollama_without_session_is_401`, `inbound_authorization_header_is_not_forwarded` (0.50s, 5 passed, 0 failed). A subsequent fresh `cargo test` invocation still timed out on its own bookkeeping/fingerprint check under the same host load, which is why the compiled-binary route was used as the verification path. +- **(Prior session) Host resource contention badly delayed test verification for Tasks 1-2** — see the original entry retained below for continuity: `CARGO_INCREMENTAL=0 cargo build --package archipelago` completed in 22m03s with zero errors; the subsequent `cargo test` compile ran over an hour under heavy load and was eventually confirmed passing (5/5) by running the already-linked test binary directly. +- **(This session) The real-node deploy revert bug** consumed most of Task 3's time: the first targeted deploy (binary + nginx conf written straight to `/etc/nginx/sites-available/archipelago`) appeared to succeed (`nginx -t` passed, ownership was briefly root:root) but was silently reverted to the pre-fix content within ~5 seconds of `systemctl restart archipelago` by the daemon's own `run_runtime_assets()` OTA-bridge self-heal, which unconditionally reinstalls a second on-disk copy of the nginx template. Diagnosed via `journalctl -u archipelago` showing the exact `install ... /opt/archipelago/web-ui/archipelago-runtime/image-recipe/configs/nginx-archipelago.conf /etc/nginx/sites-available/archipelago` command firing at boot, and confirmed the second copy's content still had the old sidecar/openrouter config. Fixed by updating both copies before the final restart. +- **The openrouter-404-vs-200 finding** (see above) is reported as-is rather than resolved, per this task's explicit instruction not to tune a probe until it passes. ## User Setup Required -None - no external service configuration required by Tasks 1-2. Task 3, when resumed, requires deploying to a real dev node per `CLAUDE.md`'s "deploy to the dev pair BEFORE any OTA" rule and manually verifying the positive path (a logged-in operator's embedded AIUI chat still works). +None for Tasks 1-3's automated portions. **What remains and requires a human:** +1. A decision on the openrouter status-code finding (accept 200/405-via-SPA-catch-all as sufficient, since the relay is structurally gone, or request a follow-up nginx `return 404` block). +2. The positive-path browser check: log in to neode-ui on archy-x250-dev3 (192.168.63.169) or its Tailscale address (100.113.170.119), open Chat, and confirm the embedded AIUI still answers with a real reply now that a Claude key would need to be configured via Settings first (this dev node currently has none — `claude-api-key` is absent, so an authenticated call would currently 503 with the plain-language "not configured" message, not actually reach Anthropic; configuring a key is an operator action outside this task's scope). ## Next Phase Readiness -- Tasks 1 and 2 are committed, confirmed to compile (`cargo build --package archipelago`, 22m03s, 0 errors), and confirmed by direct execution of the compiled test binary to pass all 5 `model_proxy::` unit tests. The exposure this plan targets (unauthenticated `/aiui/api/claude/` and `/aiui/api/ollama/`, plus the OpenRouter open relay) is closed in the source of truth (both nginx server blocks) and on the deploy path (any existing sidecar is torn down on every deploy). -- **Blocked at Task 3** — a `checkpoint:human-verify` with `gate="blocking"`. Per plan instructions, this executor does not self-approve it. See "Checkpoint" below for what the orchestrator/human must do to resume. -- **Outstanding before this plan can be marked fully done:** execute Task 3 — write `tests/production-quality/aiui-proxy-closed.sh`, deploy to a dev node, run it, and have a human confirm the four status codes plus the positive-path browser check. -- 13-09 (nginx location-block retirement, once 13-01's `assistant.chat` path is what AIUI actually uses) depends on this plan's re-pointing being in place — it is. +- Tasks 1, 2 are committed and confirmed working via both unit tests and now real-node behavior (401 on unauthenticated claude/ollama calls, confirmed via direct daemon curl and via nginx once the deploy-topology gap was worked around). +- Task 3's script is committed, deployed, and run against a real node with results reported honestly, including one criterion that does not match observed behavior. +- **Still blocked at the checkpoint's human half** — gate="blocking" — this executor does not self-approve it. See Checkpoint below. +- 13-09 (nginx location-block retirement, once 13-01's `assistant.chat` path is what AIUI actually uses) depends on this plan's re-pointing being in place — it is, and is now verified live on a real node, not just in source. ## Checkpoint **Type:** human-verify **Gate:** blocking **Plan:** 13-02 -**Progress:** 2/3 tasks complete +**Progress:** Task 3's automatable evidence collected; the checkpoint's two human-only items remain open. ### Completed Tasks @@ -162,38 +182,44 @@ None - no external service configuration required by Tasks 1-2. Task 3, when res | ---- | ---- | ------ | ----- | | 1 | Session-gated model forwarder in the Rust daemon | `97921d99` | `core/archipelago/src/api/handler/model_proxy.rs` (new), `core/archipelago/src/api/handler/mod.rs` | | 2 | Retire the Python sidecar, its key, and the OpenRouter open relay | `b28cc3ee` | `image-recipe/configs/nginx-archipelago.conf`, `scripts/deploy-to-target.sh`, `scripts/setup-aiui-server.sh`, `core/archipelago/src/api/rpc/system/handlers.rs` | +| 3 (automatable half) | aiui-proxy-closed.sh written, deployed, and run against archy-x250-dev3 | (this session's commit) | `tests/production-quality/aiui-proxy-closed.sh` (new), this SUMMARY.md | ### Current Task **Task 3:** Prove it on a real node — a green cargo test proves nothing here -**Status:** blocked (not started — this executor is required to stop here) -**Blocked by:** `gate="blocking"` human-verify checkpoint; requires writing `tests/production-quality/aiui-proxy-closed.sh`, deploying to a real dev node, and a human confirming live output. (`cargo test --package archipelago model_proxy::` is already confirmed passing — see Issues Encountered — so this is not a prerequisite before deploying.) +**Status:** automatable evidence collected and reported honestly; blocked on two human-only items +**Blocked by:** `gate="blocking"` human-verify checkpoint — this executor does not self-approve it. -### Checkpoint Details +### Observed status codes (archy-x250-dev3, 192.168.63.169, non-loopback) -**What would be built:** `tests/production-quality/aiui-proxy-closed.sh ` — a shell check that, with no session cookie, requests `/aiui/api/claude/v1/messages`, `/aiui/api/ollama/api/tags` and `/aiui/api/openrouter/` against a live node and asserts each returns 401/403/404 and never 200, plus SSH-based assertions that no `claude-api-proxy` unit is loaded and nothing listens on port 3142. +- `POST /aiui/api/claude/v1/messages` (no session) → **401** ✅ matches plan +- `GET /aiui/api/ollama/api/tags` (no session) → **401** ✅ matches plan +- `GET /aiui/api/openrouter/` → **200** (SPA catch-all `index.html`) ❌ plan specifies 404; relay itself confirmed structurally gone (no `proxy_pass` to openrouter.ai anywhere in the deployed config; behaves identically to a path that never existed) +- `POST /aiui/api/openrouter/v1/models` → **405** (same SPA-fallback location, method not allowed on static serving) +- `ssh 'systemctl is-active claude-api-proxy'` → **inactive / "could not be found"** ✅ +- `ssh 'ss -ltn | grep -c :3142'` → **0** ✅ +- `ssh 'sudo ls /var/lib/archipelago/secrets/'` → **no `claude-api-key` (node has never had one configured), no `claude-api-proxy.env`** — second-ledger invariant holds; first-ledger absence is node state, not a defect -**How to verify (per plan):** -1. `cd core && cargo test --package archipelago model_proxy::` — already confirmed passing (5/5) in this session via direct binary execution; re-run if the source changes. -2. Build and deploy to the dev pair per `CLAUDE.md` (archi-dev-box first). -3. Write and run `bash tests/production-quality/aiui-proxy-closed.sh `. -4. Confirm the positive case: log in to neode-ui on that node, open Chat, confirm embedded AIUI still answers. -5. On the node: `systemctl status claude-api-proxy` reports "could not be found"; `ss -ltnp | grep 3142` returns nothing. -6. Confirm the key ledger: `sudo ls /var/lib/archipelago/secrets/` shows `claude-api-key` and no `claude-api-proxy.env`. +`bash tests/production-quality/aiui-proxy-closed.sh 192.168.63.169 archipelago` → **5 passed, 1 failed, 0 skipped, exit 1** (the one failure is the openrouter finding above, reported as-is). ### Awaiting -Orchestrator/human to either (a) provide access to a real dev node for Task 3, or (b) spawn a continuation agent with the resume signal once that access/verification is available. Type "approved" with the four observed status codes, or describe what still answered 200. +A human (or the orchestrator relaying to one) to: +1. Decide the disposition of the openrouter finding (accept as sufficient / request a follow-up 404 block — likely 13-09 or a small fast-follow, not this plan). +2. Perform the positive-path browser check on archy-x250-dev3 once a Claude key is configured there (or accept that it cannot be checked further without one being configured, since D-17's "keeps working" claim is about auth/upstream continuity, not about a node that has genuinely never had a key). + +Type "approved" with a disposition on the openrouter finding, or describe what still needs to change. --- *Phase: 13-aiui-functional-conversational-node-control-and-content-surf* -*Completed: 2026-08-03 (partial — Tasks 1-2 of 3)* +*Completed: 2026-08-03 (Task 3's automatable half done this session; checkpoint's human half still open)* ## Self-Check: PASSED - FOUND: `core/archipelago/src/api/handler/model_proxy.rs` +- FOUND: `tests/production-quality/aiui-proxy-closed.sh` - FOUND: `.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md` - FOUND commit `97921d99` (Task 1) - FOUND commit `b28cc3ee` (Task 2) -- `cargo build --package archipelago` exited 0 -- Direct execution of the compiled test binary confirmed 5/5 `model_proxy::` tests pass +- `cargo build --release --package archipelago` exited 0 (this session, 29m20s, only pre-existing unrelated warnings) +- Real-node verification: `bash tests/production-quality/aiui-proxy-closed.sh 192.168.63.169 archipelago` ran (exit 1 — one honest finding, not a script bug; see Checkpoint) diff --git a/tests/production-quality/aiui-proxy-closed.sh b/tests/production-quality/aiui-proxy-closed.sh new file mode 100755 index 00000000..f3d60d6c --- /dev/null +++ b/tests/production-quality/aiui-proxy-closed.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# aiui-proxy-closed.sh — S-15 deployed-surface check for 13-02-PLAN.md. +# +# T-13-08..T-13-12: `/aiui/api/claude/` and `/aiui/api/ollama/` used to proxy +# to an unauthenticated Python sidecar (port 3142) holding its own API key, +# and `/aiui/api/openrouter/` was a plain unauthenticated relay to a paid +# third-party API — anyone who could reach the node's web port could spend +# the owner's budget. A green `cargo test` on model_proxy.rs proves the Rust +# handler's own logic is correct in isolation; it proves NOTHING about which +# target nginx is actually pointed at on a deployed node, whether the old +# sidecar is still listening, or whether the OpenRouter relay still exists. +# This script is that proof, against a real node (13-AI-SPEC.md S-15 — "not +# a unit test and must not be treated as one"). +# +# Usage: ./aiui-proxy-closed.sh [ssh-user] +# HTTP checks always run, with NO session cookie, and must never see 200. +# SSH-based infra checks (systemd unit gone, port 3142 dark, single key +# ledger) run only when `sshpass` is installed AND $AIUI_TEST_SSH_PASS is +# set in the environment — otherwise they report SKIP, not FAIL. Never +# hardcode a password in this file (CLAUDE.md: never commit/push secrets). +# +# Exit 0 = every HTTP assertion passes AND every SSH assertion that ran passed. + +set -uo pipefail +HOST="${1:?usage: aiui-proxy-closed.sh [ssh-user]}" +SSH_USER="${2:-archipelago}" +BASE="http://${HOST}" +PASS=0; FAIL=0; SKIP=0 +say() { printf '%s\n' "$*"; } +ok() { PASS=$((PASS+1)); say " PASS: $1"; } +bad() { FAIL=$((FAIL+1)); say " FAIL: $1"; } +skip() { SKIP=$((SKIP+1)); say " SKIP: $1"; } + +# $1=method $2=path $3=optional body +status_of() { + if [ -n "${3:-}" ]; then + curl -s -m 10 -o /dev/null -w '%{http_code}' -X "$1" "${BASE}${2}" -d "$3" 2>/dev/null + else + curl -s -m 10 -o /dev/null -w '%{http_code}' -X "$1" "${BASE}${2}" 2>/dev/null + fi +} + +# $1=label $2=observed status — closed means 401/403/404; 200 is the exposure. +assert_closed() { + case "$2" in + 401|403|404) ok "$1 -> $2 (closed)" ;; + 200) bad "$1 -> 200 (OPEN — an unauthenticated caller reached the backend)" ;; + *) bad "$1 -> '$2' (unexpected — want 401/403/404, and it is not 200 either)" ;; + esac +} + +say "== S-15 AIUI model-proxy closure — node ${HOST} ==" + +# 1) /aiui/api/claude/v1/messages — no session cookie must never reach Anthropic. +S=$(status_of POST /aiui/api/claude/v1/messages '{"model":"claude-3-5-sonnet-20241022","max_tokens":1,"messages":[]}') +assert_closed "POST /aiui/api/claude/v1/messages (no session)" "$S" + +# 2) /aiui/api/ollama/api/tags — no session cookie must never reach local Ollama. +S=$(status_of GET /aiui/api/ollama/api/tags) +assert_closed "GET /aiui/api/ollama/api/tags (no session)" "$S" + +# 3) /aiui/api/openrouter/ — must be entirely GONE, not merely gated: 404 specifically. +S=$(status_of GET /aiui/api/openrouter/) +if [ "$S" = "404" ]; then + ok "GET /aiui/api/openrouter/ -> 404 (relay deleted)" +else + bad "GET /aiui/api/openrouter/ -> $S (want 404 — the relay must not exist at all, not just be gated)" +fi + +say "" +say "-- SSH-based infra checks (${SSH_USER}@${HOST}) --" +if ! command -v sshpass >/dev/null 2>&1 || [ -z "${AIUI_TEST_SSH_PASS:-}" ]; then + skip "claude-api-proxy systemd unit (no sshpass or AIUI_TEST_SSH_PASS unset)" + skip "port 3142 listener (no sshpass or AIUI_TEST_SSH_PASS unset)" + skip "single key ledger (no sshpass or AIUI_TEST_SSH_PASS unset)" +else + ssh_run() { + sshpass -p "${AIUI_TEST_SSH_PASS}" ssh -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ + "${SSH_USER}@${HOST}" "$1" 2>/dev/null + } + + UNIT_STATE=$(ssh_run 'systemctl is-active claude-api-proxy 2>&1') + case "$UNIT_STATE" in + inactive|unknown) ok "claude-api-proxy unit is '$UNIT_STATE'" ;; + *"could not be found"*) ok "claude-api-proxy unit is gone (could not be found)" ;; + active) bad "claude-api-proxy unit is still ACTIVE — sidecar not torn down" ;; + *) bad "claude-api-proxy unit state unexpected: '$UNIT_STATE'" ;; + esac + + PORT_COUNT=$(ssh_run "ss -ltn 2>/dev/null | grep -c ':3142 '") + [ "${PORT_COUNT:-1}" = "0" ] && ok "nothing listening on :3142" || bad "port 3142 still has a listener (count=${PORT_COUNT:-?})" + + # claude-api-key's PRESENCE depends on whether an operator has configured a + # key on this node at all (via system.settings.set claude_api_key) — a + # freshly provisioned/dev node with no key set is expected to have neither + # file, and that is not a defect in this fix. The security-relevant + # invariant this plan makes is narrower and unconditional: the SECOND + # ledger (claude-api-proxy.env) must never exist, whether or not the first + # one does. Presence of claude-api-key is reported for visibility only. + LEDGER=$(ssh_run 'sudo ls /var/lib/archipelago/secrets/ 2>/dev/null') + if grep -qx 'claude-api-key' <<<"$LEDGER"; then + say " INFO: claude-api-key ledger present (a key is configured on this node)" + else + say " INFO: claude-api-key ledger absent (no key configured on this node yet — not a defect)" + fi + if grep -q 'claude-api-proxy.env' <<<"$LEDGER"; then + bad "claude-api-proxy.env still present — second key ledger not deleted" + else + ok "claude-api-proxy.env absent (single ledger enforced)" + fi +fi + +say "" +say "== ${HOST}: ${PASS} passed, ${FAIL} failed, ${SKIP} skipped ==" +[ "$FAIL" -eq 0 ]