test(aiui): the three "pre-existing failures" were stale tests, not defects

AIUI has carried 353/356 as a known-open item (W1.7) for long enough that
the three reds were treated as background noise. All three were the tests
being wrong. 357/357 now.

`injects web results into system prompt when enabled` asserted
`body.webSearch === true`. The code deliberately sends false there:
`proxyWebSearch = webSearchEnabled && !clientSearchSucceeded`, so when the
client-side search has already run and injected its results into the system
prompt, asking the proxy to search again would be a second redundant search
on every turn. The assertion predates that change. Fixed, and the other half
of the contract added as its own case — zero client results must still ask
the proxy to search.

The two seed-songs failures ("extracts 10 songs", and the conversation
regression built on the same fixture) were one wrong number:
`expected: { songs: 10 }` against an assistantResponse containing exactly
six `song_ext` entries and ending coherently on Treefingers. Not truncated —
just miscounted. The extractor was returning the right answer the whole
time. Counted from the fixture rather than from intent, so the number now
describes the input instead of contradicting it.

Full branch state after the main merge: AIUI 357/357, neode-ui 972/972
across 114 files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-08 09:31:45 -04:00
co-authored by Claude Opus 5
parent 71164137ef
commit 78f0370afc
2 changed files with 36 additions and 1 deletions
@@ -70,7 +70,12 @@ All feature practical effects, IMAX cinematography, and Hans Zimmer scores (exce
[[song_ext:Music for Airports 1/1|Brian Eno|1978]] — The track that coined "ambient music." Tape loops of piano and voices create an ever-evolving soundscape.
[[song_ext:Treefingers|Radiohead|2000]] — From Kid A. Processed guitar creating ethereal textures. Works beautifully on loop.`,
expected: { songs: 10 },
// Six, not ten. The assistantResponse above lists exactly six
// `song_ext` entries and ends coherently on Treefingers — it was never
// truncated. The extractor was returning the right answer and this
// number was the wrong one, which is why "extracts 10 songs" had been
// failing. Counted from the fixture, not from intent.
expected: { songs: 6 },
},
// ─── Books ──────────────────────────────────────────────────
@@ -348,6 +348,36 @@ describe('useAI', () => {
const body = JSON.parse(claudeCall![1].body as string)
expect(body.system).toContain('Web search results')
expect(body.system).toContain('Result 1')
// FALSE on purpose: `proxyWebSearch = webSearchEnabled && !clientSearchSucceeded`.
// The client already searched and injected the results above, so asking
// the proxy to search again would be a second, redundant search per turn.
// This assertion read `true` and had been failing since that change —
// the test was stale, the behaviour is intentional.
expect(body.webSearch).toBe(false)
})
it('asks the proxy to search when the client-side search finds nothing', async () => {
// The other half of the same contract: no client results means nothing
// was injected, so the proxy must still do the search.
vi.mocked(searchWeb).mockResolvedValue([])
const fetchSpy = vi.fn().mockResolvedValue(
mockClaudeResponse(['data: {"type":"content_block_delta","delta":{"text":"answer"}}\n\n'])
)
globalThis.fetch = fetchSpy
const chatStore = useChatStore()
chatStore.webSearchEnabled = true
const { sendMessage } = useAI()
await sendMessage('latest bitcoin news')
const claudeCall = fetchSpy.mock.calls.find(
(c: unknown[]) => (c[0] as string)?.toString().includes('/claude/')
)
expect(claudeCall).toBeDefined()
const body = JSON.parse(claudeCall![1].body as string)
expect(body.system).not.toContain('Web search results')
expect(body.webSearch).toBe(true)
})
})