From 6d4bf42a0835bdf2e289299251df93bfee882595 Mon Sep 17 00:00:00 2001 From: archipelago Date: Mon, 3 Aug 2026 14:59:28 -0400 Subject: [PATCH] =?UTF-8?q?docs(13-02):=20SUMMARY=20for=20Tasks=201-2=20?= =?UTF-8?q?=E2=80=94=20halting=20at=20Task=203's=20blocking=20checkpoint?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tasks 1 (session-gated model forwarder, 97921d99) and 2 (retire the Python sidecar/OpenRouter relay, b28cc3ee) are committed, cargo build --package archipelago succeeds, and all 5 model_proxy:: unit tests are confirmed passing (via direct execution of the compiled test binary, since a fresh `cargo test` invocation was too slow to complete under severe host resource contention — see the SUMMARY's Issues Encountered for the full account). Task 3 (checkpoint:human-verify, gate="blocking" — real-node curl/systemd proof, S-15) is intentionally NOT executed. Per the plan and this executor's instructions, it halts here and returns a structured checkpoint rather than self-approving. Co-Authored-By: Claude Opus 5 (1M context) --- .../13-02-SUMMARY.md | 199 ++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 .planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md diff --git a/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md new file mode 100644 index 00000000..aaee8a49 --- /dev/null +++ b/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md @@ -0,0 +1,199 @@ +--- +phase: 13-aiui-functional-conversational-node-control-and-content-surf +plan: 02 +subsystem: api +tags: [session-auth, reverse-proxy, nginx, rust, security, aiui, anthropic, ollama] + +requires: + - phase: 10-key-material-hardening + provides: "data_dir/secrets/claude-api-key single-ledger pattern, is_authenticated idiom" +provides: + - "Session-gated forwarder (core/archipelago/src/api/handler/model_proxy.rs) for /aiui/api/claude/* and /aiui/api/ollama/*" + - "Both nginx server blocks re-pointed to the Rust daemon (127.0.0.1:5678) instead of the unauthenticated Python sidecar (127.0.0.1:3142)" + - "/aiui/api/openrouter/ open relay deleted from both nginx server blocks" + - "deploy-to-target.sh unconditionally tears down any pre-existing claude-api-proxy unit/key/binary on every deploy" + - "Single Claude key ledger (data_dir/secrets/claude-api-key) — the second copy (secrets/claude-api-proxy.env) is no longer written" +affects: [13-09-nginx-location-retirement, aiui-standalone-mode, node-security-posture] + +tech-stack: + added: [] + patterns: + - "HTTP path dispatch auth-gate idiom: `if !self.is_authenticated(...).await { return Ok(Self::unauthorized()) }` before any upstream/handler work, same as the existing /ws/db, /ws/remote-input, /ws/remote-relay arms" + - "Streamed (not buffered) upstream response via Body::wrap_stream(resp.bytes_stream()), matching proxy.rs's peer Range-streaming shape" + - "Header allowlisting on forward: only content-type/accept copied inbound; extra_headers supplies the outbound key/version pin — inbound authorization/x-api-key/cookie are never read" + +key-files: + created: + - core/archipelago/src/api/handler/model_proxy.rs + modified: + - core/archipelago/src/api/handler/mod.rs + - core/archipelago/src/api/rpc/system/handlers.rs + - image-recipe/configs/nginx-archipelago.conf + - scripts/deploy-to-target.sh + - scripts/setup-aiui-server.sh + +key-decisions: + - "Continuation option (a) chosen: git reset --soft HEAD~1 on the inherited WIP checkpoint (13b576da), then recommitted atomically as two per-task commits (97921d99 for Task 1, b28cc3ee for Task 2), after reading and verifying every line of the WIP diff against the plan's must_haves/acceptance criteria and the real source tree (function/type signatures cross-checked in session.rs and handler/mod.rs)." + - "Kept scripts/setup-aiui-server.sh's removal of its FileBrowser-fix step (present in the inherited WIP, not explicitly named in the plan's action text) after confirming that logic still lives, unmodified, in scripts/deploy-to-target.sh (lines ~481-500 and ~1005-1022) — nothing was lost, the script was correctly narrowed to match its own rewritten header comment." + - "Did not create tests/production-quality/aiui-proxy-closed.sh or attempt any real-node deploy/verification — that is Task 3's own deliverable and is explicitly the blocking human-verify checkpoint this plan stops at." + +requirements-completed: [AIUI-04] + +coverage: + - id: D1 + description: "Session-gated forwarder for /aiui/api/claude/* and /aiui/api/ollama/* — unauthenticated/invalid-session requests get 401 before any upstream call, missing key returns 503 (never 500, never the key path), inbound authorization/x-api-key/cookie headers are never forwarded upstream" + requirement: "AIUI-04" + verification: + - kind: unit + ref: "core/archipelago/src/api/handler/model_proxy.rs#model_proxy::tests::claude_without_session_is_401" + status: pass + - kind: unit + ref: "core/archipelago/src/api/handler/model_proxy.rs#model_proxy::tests::ollama_without_session_is_401" + status: pass + - kind: unit + ref: "core/archipelago/src/api/handler/model_proxy.rs#model_proxy::tests::claude_with_invalid_session_is_401" + status: pass + - kind: unit + ref: "core/archipelago/src/api/handler/model_proxy.rs#model_proxy::tests::missing_key_is_503_not_500" + status: pass + - kind: unit + ref: "core/archipelago/src/api/handler/model_proxy.rs#model_proxy::tests::inbound_authorization_header_is_not_forwarded" + status: pass + human_judgment: false + - id: D2 + description: "Both nginx server blocks re-pointed from the Python sidecar (127.0.0.1:3142) to the Rust daemon (127.0.0.1:5678); /aiui/api/openrouter/ deleted from both blocks" + requirement: "AIUI-04" + verification: + - kind: other + ref: "grep -c openrouter image-recipe/configs/nginx-archipelago.conf == 0; grep -c 127.0.0.1:3142 == 0; grep -c 'location /aiui/api/claude/' == 2" + status: pass + human_judgment: false + - id: D3 + description: "deploy-to-target.sh removes any pre-existing claude-api-proxy unit/key/binary unconditionally on every deploy; setup-aiui-server.sh no longer requires or patches in an ANTHROPIC_API_KEY; handlers.rs no longer writes the second key ledger or restarts the sidecar" + requirement: "AIUI-04" + verification: + - kind: other + ref: "grep -c 'PORT = 3142' scripts/deploy-to-target.sh == 0; grep -c claude-api-proxy core/archipelago/src/api/rpc/system/handlers.rs == 0; grep -c secrets/claude-api-key handlers.rs >= 1; cargo build --package archipelago exits 0" + status: pass + human_judgment: false + - id: D4 + description: "S-15 real-node deployed-surface proof (tests/production-quality/aiui-proxy-closed.sh) and the positive-path check that a logged-in operator's AIUI build still works" + verification: [] + human_judgment: true + rationale: "This is Task 3, a checkpoint:human-verify task with gate=\"blocking\" in the plan. Per plan-specific instructions this executor must not self-approve or attempt it — it requires deploying to a real node and a human confirming curl/systemctl/ss output and a live browser chat reply. Not started." + +# Metrics +duration: ~2h10m (dominated by two full-workspace cargo builds/compiles on a heavily contended shared host) +completed: 2026-08-03 +status: complete +--- + +# Phase 13 Plan 02: Close the AIUI unauthenticated model-proxy exposure (Tasks 1-2 of 3; Task 3 is a blocking checkpoint) + +**Session-gated Rust-daemon forwarder (`model_proxy.rs`, 5/5 unit tests confirmed passing) replaces the unauthenticated `claude-api-proxy.py` sidecar and the OpenRouter open relay; both nginx server blocks re-pointed, second key ledger deleted, deploy path tears down any already-provisioned sidecar.** + +## Performance + +- **Duration:** ~2h10m wall clock (session inherited a broken-pipe WIP checkpoint; most of the time was two sequential `CARGO_INCREMENTAL=0` full builds on a shared box already running bitcoind/electrumx/lnd plus a concurrent sibling-plan build and an unrelated main-checkout build — `cargo build --package archipelago` alone took 22m03s) +- **Tasks:** 2 of 3 complete (Task 3 is a blocking human-verify checkpoint this plan is required to stop at) +- **Files modified:** 6 (1 created, 5 modified) + +## Accomplishments + +- `core/archipelago/src/api/handler/model_proxy.rs` created: re-derives session auth from the request's own cookie (does not trust nginx), forwards authenticated Claude calls to `https://api.anthropic.com/` using the node's single key ledger and authenticated Ollama calls to `http://127.0.0.1:11434/`, streams responses instead of buffering, and never forwards inbound `authorization`/`x-api-key`/`cookie` headers upstream. +- Both nginx server blocks (`image-recipe/configs/nginx-archipelago.conf`, ~line 49 and ~line 961) re-pointed from the unauthenticated Python sidecar (port 3142) to the Rust daemon (127.0.0.1:5678), and the `/aiui/api/openrouter/` open relay deleted from both. +- `scripts/deploy-to-target.sh` no longer installs the `claude-api-proxy.py` sidecar; it now unconditionally stops/disables/removes any pre-existing unit, binary, and env file on every deploy, so already-provisioned nodes actually lose the old unauthenticated listener. +- `scripts/setup-aiui-server.sh` no longer requires or patches in an `ANTHROPIC_API_KEY`; its job is now just the AIUI dist rsync, matching its rewritten header comment. +- `core/archipelago/src/api/rpc/system/handlers.rs`'s `claude_api_key` setting branch no longer writes a second key copy or restarts the sidecar — `secrets/claude-api-key` (0600) is now the one and only ledger. + +## Task Commits + +Each task was committed atomically after resetting the inherited WIP checkpoint: + +1. **Task 1: Session-gated model forwarder in the Rust daemon** - `97921d99` (feat) +2. **Task 2: Retire the Python sidecar, its key, and the OpenRouter open relay** - `b28cc3ee` (fix) + +Task 3 (`checkpoint:human-verify`, `gate="blocking"`) is NOT executed — this plan halts there per its own instructions; see "Checkpoint" below. + +## Files Created/Modified + +- `core/archipelago/src/api/handler/model_proxy.rs` (new, 400 lines) - session-gated forwarder + its own `#[cfg(test)]` suite +- `core/archipelago/src/api/handler/mod.rs` - `mod model_proxy;` + one new path-dispatch arm gating `/aiui/api/claude/` and `/aiui/api/ollama/` +- `core/archipelago/src/api/rpc/system/handlers.rs` - `claude_api_key` branch no longer writes the second ledger / restarts the sidecar +- `image-recipe/configs/nginx-archipelago.conf` - both server blocks re-pointed to 127.0.0.1:5678; `openrouter` locations deleted; comments rewritten +- `scripts/deploy-to-target.sh` - `claude-api-proxy.py` heredoc/unit/env deleted; unconditional teardown step added; `3141`→`3142` sed fixups removed +- `scripts/setup-aiui-server.sh` - `ANTHROPIC_API_KEY` requirement and `patch-nginx-claude.py` step removed; narrowed to the AIUI dist rsync + +## Decisions Made + +- **Continuation option chosen: (a).** The inherited WIP commit (`13b576da`) was read in full — every diff hunk, not just a stat summary — and cross-checked against the real source tree before trusting any of it: function/type signatures in `session.rs` (`SessionStore::new_for_tests`, `.validate()`, `.create()`, `session::extract_session_cookie`), the `ApiHandler` struct's actual fields (`session_store`, `config.data_dir`), the enclosing `handle_request`'s `Result>` return type and its `match (method, path.as_str())` binding shape, and every grep-based acceptance criterion in the plan. It held up: `git reset --soft HEAD~1` followed by two focused per-task commits. +- `scripts/setup-aiui-server.sh`'s removal of the FileBrowser-fix step (present in the WIP diff, not literally named in the plan's action prose) was kept after confirming the same FileBrowser recreate/create logic already exists, unmodified, in `scripts/deploy-to-target.sh` (~line 481 and ~line 1005) — no functionality was lost, the script now matches its own rewritten header ("What it does: Rsyncs ... a locally built AIUI dist/ ... What it no longer does: ..."). +- Task 3's script (`tests/production-quality/aiui-proxy-closed.sh`) was deliberately **not** created and no real-node deploy was attempted — the plan's own instructions and the orchestrator's task brief are explicit that this executor stops before Task 3 and returns a structured checkpoint rather than self-approving a `gate="blocking"` human-verify task. + +## Deviations from Plan + +None beyond the FileBrowser-step removal noted above under Decisions, which is a faithful reading of the plan's own "the script's remaining job is the AIUI dist rsync" sentence rather than an unrequested change — flagged here for visibility rather than because it looks wrong. + +## Issues Encountered + +- **Host resource contention badly delayed (but did not prevent) test verification.** This session inherited a broken-pipe WIP checkpoint on a shared box that was simultaneously running a live Archipelago node (bitcoind, electrumx, lnd), a concurrent unrelated build in the main checkout (a different agent's federation/trust-password work), and — for part of this session — a concurrent sibling-plan build in worktree `p13-01`. `CARGO_INCREMENTAL=0 cargo build --package archipelago` completed successfully in 22m03s with zero errors (3 pre-existing unrelated warnings only). The subsequent `cargo test --package archipelago model_proxy::` compile (a single large `rustc --test` link step) ran for over an hour under system load average 35-55 and 14-15GB/23GB swap in use, and the harness eventually reported the background task as "killed". However, the 380MB test binary it produced (`target/debug/deps/archipelago-75b844ec6baa778d`, timestamped after both task commits, confirmed via `git diff 13b576da HEAD` to be byte-identical source content to what's committed) had actually finished linking. Running it directly — `./target/debug/deps/archipelago-75b844ec6baa778d model_proxy::` — **confirmed all 5 tests pass**: `claude_without_session_is_401`, `claude_with_invalid_session_is_401`, `missing_key_is_503_not_500`, `ollama_without_session_is_401`, `inbound_authorization_header_is_not_forwarded` (0.50s, 5 passed, 0 failed). A subsequent fresh `cargo test` invocation still timed out on its own bookkeeping/fingerprint check under the same host load, which is why the compiled-binary route was used as the verification path. + +## User Setup Required + +None - no external service configuration required by Tasks 1-2. Task 3, when resumed, requires deploying to a real dev node per `CLAUDE.md`'s "deploy to the dev pair BEFORE any OTA" rule and manually verifying the positive path (a logged-in operator's embedded AIUI chat still works). + +## Next Phase Readiness + +- Tasks 1 and 2 are committed, confirmed to compile (`cargo build --package archipelago`, 22m03s, 0 errors), and confirmed by direct execution of the compiled test binary to pass all 5 `model_proxy::` unit tests. The exposure this plan targets (unauthenticated `/aiui/api/claude/` and `/aiui/api/ollama/`, plus the OpenRouter open relay) is closed in the source of truth (both nginx server blocks) and on the deploy path (any existing sidecar is torn down on every deploy). +- **Blocked at Task 3** — a `checkpoint:human-verify` with `gate="blocking"`. Per plan instructions, this executor does not self-approve it. See "Checkpoint" below for what the orchestrator/human must do to resume. +- **Outstanding before this plan can be marked fully done:** execute Task 3 — write `tests/production-quality/aiui-proxy-closed.sh`, deploy to a dev node, run it, and have a human confirm the four status codes plus the positive-path browser check. +- 13-09 (nginx location-block retirement, once 13-01's `assistant.chat` path is what AIUI actually uses) depends on this plan's re-pointing being in place — it is. + +## Checkpoint + +**Type:** human-verify +**Gate:** blocking +**Plan:** 13-02 +**Progress:** 2/3 tasks complete + +### Completed Tasks + +| Task | Name | Commit | Files | +| ---- | ---- | ------ | ----- | +| 1 | Session-gated model forwarder in the Rust daemon | `97921d99` | `core/archipelago/src/api/handler/model_proxy.rs` (new), `core/archipelago/src/api/handler/mod.rs` | +| 2 | Retire the Python sidecar, its key, and the OpenRouter open relay | `b28cc3ee` | `image-recipe/configs/nginx-archipelago.conf`, `scripts/deploy-to-target.sh`, `scripts/setup-aiui-server.sh`, `core/archipelago/src/api/rpc/system/handlers.rs` | + +### Current Task + +**Task 3:** Prove it on a real node — a green cargo test proves nothing here +**Status:** blocked (not started — this executor is required to stop here) +**Blocked by:** `gate="blocking"` human-verify checkpoint; requires writing `tests/production-quality/aiui-proxy-closed.sh`, deploying to a real dev node, and a human confirming live output. (`cargo test --package archipelago model_proxy::` is already confirmed passing — see Issues Encountered — so this is not a prerequisite before deploying.) + +### Checkpoint Details + +**What would be built:** `tests/production-quality/aiui-proxy-closed.sh ` — a shell check that, with no session cookie, requests `/aiui/api/claude/v1/messages`, `/aiui/api/ollama/api/tags` and `/aiui/api/openrouter/` against a live node and asserts each returns 401/403/404 and never 200, plus SSH-based assertions that no `claude-api-proxy` unit is loaded and nothing listens on port 3142. + +**How to verify (per plan):** +1. `cd core && cargo test --package archipelago model_proxy::` — already confirmed passing (5/5) in this session via direct binary execution; re-run if the source changes. +2. Build and deploy to the dev pair per `CLAUDE.md` (archi-dev-box first). +3. Write and run `bash tests/production-quality/aiui-proxy-closed.sh `. +4. Confirm the positive case: log in to neode-ui on that node, open Chat, confirm embedded AIUI still answers. +5. On the node: `systemctl status claude-api-proxy` reports "could not be found"; `ss -ltnp | grep 3142` returns nothing. +6. Confirm the key ledger: `sudo ls /var/lib/archipelago/secrets/` shows `claude-api-key` and no `claude-api-proxy.env`. + +### Awaiting + +Orchestrator/human to either (a) provide access to a real dev node for Task 3, or (b) spawn a continuation agent with the resume signal once that access/verification is available. Type "approved" with the four observed status codes, or describe what still answered 200. + +--- +*Phase: 13-aiui-functional-conversational-node-control-and-content-surf* +*Completed: 2026-08-03 (partial — Tasks 1-2 of 3)* + +## Self-Check: PASSED + +- FOUND: `core/archipelago/src/api/handler/model_proxy.rs` +- FOUND: `.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-02-SUMMARY.md` +- FOUND commit `97921d99` (Task 1) +- FOUND commit `b28cc3ee` (Task 2) +- `cargo build --package archipelago` exited 0 +- Direct execution of the compiled test binary confirmed 5/5 `model_proxy::` tests pass