450 lines
14 KiB
Markdown
450 lines
14 KiB
Markdown
# Testing Patterns
|
||
|
||
**Analysis Date:** 2026-07-29
|
||
|
||
## Test Framework
|
||
|
||
**Frontend:**
|
||
- Runner: Vitest 3.1.1
|
||
- Config: `neode-ui/vitest.config.ts`
|
||
- Environment: jsdom (DOM testing in Node.js)
|
||
- Globals: enabled (`globals: true`) — `describe`, `it`, `expect` available without imports
|
||
- Assertion library: built-in Vitest assertions (compatible with Jest)
|
||
|
||
**Backend (Rust):**
|
||
- Framework: built-in `#[test]` attribute and `cargo test`
|
||
- Command: `cd core && cargo test --workspace --bins`
|
||
|
||
**E2E (Browser):**
|
||
- Framework: Playwright 1.58.2
|
||
- Config: implicit (tests in `neode-ui/e2e/` directory)
|
||
|
||
**Shell Integration Tests:**
|
||
- Framework: Bats (Bash Automated Testing System)
|
||
- Location: `tests/lifecycle/bats/`
|
||
- Config files: `tests/lifecycle/lib/rpc.bash` (RPC wrapper helpers)
|
||
|
||
**Run Commands:**
|
||
```bash
|
||
# Unit tests (Vitest)
|
||
npm run test # Run all tests once
|
||
npm run test:watch # Watch mode, re-run on file changes
|
||
|
||
# Rust tests
|
||
cd core && cargo test --workspace --bins
|
||
|
||
# Specific Vitest suite
|
||
npm run test -- src/composables/__tests__/useFileType.test.ts
|
||
|
||
# Shell lifecycle tests (from repo root)
|
||
ARCHY_PASSWORD=password123 tests/lifecycle/run.sh # Read-only tests
|
||
ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 tests/lifecycle/run.sh # Include destructive
|
||
|
||
# Release gate (5× iterations, must run ON the target node)
|
||
ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 ARCHY_ITERATIONS=5 \
|
||
tests/lifecycle/run-gate.sh
|
||
```
|
||
|
||
## Test File Organization
|
||
|
||
**Location:**
|
||
- Frontend: co-located with source in `__tests__/` subdirectories
|
||
- Example: `src/composables/useFileType.ts` → `src/composables/__tests__/useFileType.test.ts`
|
||
- Example: `src/api/rpc-client.ts` → `src/api/__tests__/rpc-client.test.ts`
|
||
- E2E: separate `e2e/` directory at root of frontend
|
||
- Shell: `tests/lifecycle/bats/` directory
|
||
|
||
**Naming:**
|
||
- Vitest: `*.test.ts` or `*.spec.ts` suffix (`.test.ts` preferred)
|
||
- Playwright: `*.spec.ts` suffix
|
||
- Bats: `*.bats` suffix
|
||
- Rust unit: same file with `#[test]` functions at the bottom or in submodules
|
||
|
||
**Structure:**
|
||
```
|
||
neode-ui/
|
||
├── src/
|
||
│ ├── composables/
|
||
│ │ ├── useFileType.ts
|
||
│ │ └── __tests__/
|
||
│ │ ├── useFileType.test.ts
|
||
│ │ ├── useNavSounds.test.ts
|
||
│ │ └── ... (other composable tests)
|
||
│ ├── api/
|
||
│ │ ├── rpc-client.ts
|
||
│ │ └── __tests__/
|
||
│ │ └── rpc-client.test.ts
|
||
│ └── stores/
|
||
│ ├── controller.ts
|
||
│ └── (no tests found for stores in exploration)
|
||
├── e2e/
|
||
│ ├── app-launch.spec.ts
|
||
│ ├── intro-experience.spec.ts
|
||
│ └── visual-regression.spec.ts
|
||
```
|
||
|
||
## Test Structure
|
||
|
||
**Vitest Suite Organization:**
|
||
|
||
```typescript
|
||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||
import { ref } from 'vue'
|
||
import { getFileCategory, useFileType, formatSize, formatDate } from '../useFileType'
|
||
|
||
describe('getFileCategory', () => {
|
||
it('returns folder for directories', () => {
|
||
expect(getFileCategory('', true)).toBe('folder')
|
||
expect(getFileCategory('jpg', true)).toBe('folder')
|
||
})
|
||
|
||
it('identifies image extensions', () => {
|
||
expect(getFileCategory('jpg', false)).toBe('image')
|
||
expect(getFileCategory('png', false)).toBe('image')
|
||
})
|
||
})
|
||
|
||
describe('useFileType', () => {
|
||
it('returns correct category and computed values for an image', () => {
|
||
const ext = ref('jpg')
|
||
const isDir = ref(false)
|
||
const result = useFileType(ext, isDir)
|
||
|
||
expect(result.category.value).toBe('image')
|
||
expect(result.isImage.value).toBe(true)
|
||
})
|
||
|
||
it('reacts to ref changes', () => {
|
||
const ext = ref('jpg')
|
||
const isDir = ref(false)
|
||
const result = useFileType(ext, isDir)
|
||
|
||
expect(result.category.value).toBe('image')
|
||
ext.value = 'mp3'
|
||
expect(result.category.value).toBe('audio')
|
||
})
|
||
})
|
||
```
|
||
|
||
**Patterns:**
|
||
- `describe()` blocks group related tests by function or component
|
||
- `it()` blocks test a single behavior (flat structure, no nesting of describe blocks observed)
|
||
- `beforeEach()` / `afterEach()` hooks for setup/teardown per test
|
||
- `beforeAll()` / `afterAll()` hooks for suite-level setup (e.g., login in bats tests)
|
||
- Assertions use `expect(actual).toBe(expected)` or `expect(actual).toEqual(object)`
|
||
|
||
**Playwright E2E Structure:**
|
||
|
||
```typescript
|
||
import { expect, test, type Page } from '@playwright/test'
|
||
|
||
async function login(page: Page) {
|
||
await page.goto('/login', { waitUntil: 'domcontentloaded' })
|
||
await page.evaluate(() => {
|
||
localStorage.setItem('neode_intro_seen', '1')
|
||
})
|
||
// ... fill form, submit
|
||
await page.waitForURL('**/dashboard**', { timeout: 20_000 })
|
||
}
|
||
|
||
test('installed app launch opens reachable app URL', async ({ page, context, baseURL }) => {
|
||
test.skip(!EXPECTED_URL, 'Set ARCHY_EXPECTED_LAUNCH_URL for launch qualification')
|
||
|
||
await login(page)
|
||
await page.goto('/dashboard/apps', { waitUntil: 'domcontentloaded' })
|
||
|
||
const appCard = page.locator('[data-controller-container]', {
|
||
has: page.getByRole('heading', { name: APP_CARD_TITLE, exact: true }),
|
||
}).first()
|
||
|
||
await appCard.waitFor({ timeout: 30_000 })
|
||
await expect(appCard.locator('button')).toBeVisible()
|
||
})
|
||
```
|
||
|
||
**Bats Shell Test Structure:**
|
||
|
||
```bash
|
||
#!/usr/bin/env bats
|
||
# tests/lifecycle/bats/bitcoin-knots.bats
|
||
|
||
load '../lib/rpc.bash'
|
||
|
||
setup_file() {
|
||
: "${ARCHY_PASSWORD:?Set ARCHY_PASSWORD env var to the UI password}"
|
||
export ARCHY_FORCE_LOGIN=1
|
||
rpc_login
|
||
unset ARCHY_FORCE_LOGIN
|
||
}
|
||
|
||
teardown_file() {
|
||
rpc_logout_local
|
||
}
|
||
|
||
@test "container-list includes bitcoin-knots" {
|
||
run rpc_result container-list
|
||
[ "$status" -eq 0 ]
|
||
echo "$output" | jq -e '.[] | select(.name == "bitcoin-knots")' >/dev/null
|
||
}
|
||
|
||
@test "container-status returns a valid status object" {
|
||
run rpc_call container-status '{"app_id":"bitcoin-knots"}'
|
||
[ "$status" -eq 0 ]
|
||
}
|
||
```
|
||
|
||
## Mocking
|
||
|
||
**Framework (Vitest):** `vi` from Vitest; global stub support
|
||
|
||
**Patterns:**
|
||
|
||
```typescript
|
||
const mockFetch = vi.fn()
|
||
vi.stubGlobal('fetch', mockFetch)
|
||
|
||
// Import after stubbing
|
||
const { rpcClient } = await import('../rpc-client')
|
||
|
||
// In tests:
|
||
mockFetch.mockResolvedValueOnce(jsonResponse({ result: { did: 'did:key:z123' } }))
|
||
mockFetch.mockRejectedValueOnce(new Error('fetch failed'))
|
||
|
||
// Assertions on mock calls:
|
||
expect(mockFetch).toHaveBeenCalledOnce()
|
||
const [url, init] = mockFetch.mock.calls[0]!
|
||
expect(url).toBe('/rpc/v1')
|
||
expect(init.method).toBe('POST')
|
||
```
|
||
|
||
**Vue Test Utils:**
|
||
- Component mounting: `mount(Component, { global: { mocks: { $ver: displayVersion } } })`
|
||
- Props tested by passing to mount options
|
||
- Events tested by listening to emitted events
|
||
|
||
**What to Mock:**
|
||
- External HTTP requests (fetch, axios)
|
||
- Timers (for timeout logic; `vi.useFakeTimers()`)
|
||
- Global objects (localStorage, console, window.location)
|
||
|
||
**What NOT to Mock:**
|
||
- Vue reactivity (ref, computed) — these are core to component behavior
|
||
- RPC client methods in component tests — prefer integration-style testing
|
||
- Built-in assertions (expect) — always available
|
||
- Pinia stores in unit tests of composables that use them — store directly if needed
|
||
|
||
## Fixtures and Factories
|
||
|
||
**Test Data:**
|
||
|
||
```typescript
|
||
function jsonResponse(body: unknown, status = 200, statusText = 'OK'): Response {
|
||
return {
|
||
ok: status >= 200 && status < 300,
|
||
status,
|
||
statusText,
|
||
json: () => Promise.resolve(body),
|
||
// ... other Response properties
|
||
}
|
||
}
|
||
|
||
// Usage:
|
||
mockFetch.mockResolvedValueOnce(jsonResponse({ result: { did: 'did:key:z123' } }))
|
||
mockFetch.mockResolvedValueOnce(jsonResponse(null, 502, 'Bad Gateway'))
|
||
```
|
||
|
||
**Location:**
|
||
- Fixtures (test data, helper functions) defined inline in test files or in small helper modules
|
||
- No central fixture factory observed; each test file is self-contained
|
||
- Shell test helpers in `tests/lifecycle/lib/rpc.bash` (RPC wrapper for bats)
|
||
|
||
## Coverage
|
||
|
||
**Requirements:**
|
||
- Frontend: 80% branch coverage (set in `vitest.config.ts` thresholds)
|
||
- Rust: no explicit threshold; pragmatic testing of public APIs
|
||
- Shell: coverage tracked per app in `tests/lifecycle/TESTING.md` (lifecycle matrix)
|
||
|
||
**View Coverage:**
|
||
|
||
```bash
|
||
# Generate coverage report
|
||
npm run test -- --coverage
|
||
|
||
# Output formats: text, text-summary, html
|
||
# Config in vitest.config.ts: reporter: ['text', 'text-summary']
|
||
```
|
||
|
||
**Coverage Scope (Frontend):**
|
||
- Included: `src/api/*.ts`, `src/stores/*.ts`, `src/composables/*.ts`, `src/utils/*.ts`, `src/services/*.ts`, `src/router/*.ts`
|
||
- Excluded: test files (`src/**/__tests__/**`), type definitions (`*.d.ts`), entry point (`src/main.ts`)
|
||
|
||
## Test Types
|
||
|
||
**Unit Tests (Vitest):**
|
||
- Scope: individual functions, composables, utility modules
|
||
- Approach: fast, isolated, mock external dependencies
|
||
- Example: `useFileType.test.ts` tests `getFileCategory`, `useFileType`, `formatSize`, `formatDate` independently
|
||
- Latency: ~5s for full suite; individual tests <1s
|
||
|
||
**Integration Tests (Vitest + RPCClient):**
|
||
- Scope: RPC client with mocked fetch, authentication flows, retry logic
|
||
- Approach: more complex setup, test interactions between layers
|
||
- Example: `rpc-client.test.ts` tests 70+ scenarios (login, TOTP, federation, package operations)
|
||
- Latency: ~30s for full suite
|
||
|
||
**E2E Tests (Playwright):**
|
||
- Scope: real browser, real app instance, user journeys (login → navigate → interact)
|
||
- Approach: full app stack running; no mocks of UI layer
|
||
- Example: `app-launch.spec.ts` tests app card discovery and launch via button click
|
||
- Latency: 30–120s per test depending on app startup time
|
||
|
||
**Lifecycle Tests (Bats):**
|
||
- Scope: container operations (install, start, stop, restart, uninstall) on a live node
|
||
- Approach: RPC calls to backend, shell commands for verification, destructive operations tier-gated
|
||
- Tiers:
|
||
- L0 unit: Rust unit tests (cargo test)
|
||
- L1 RPC: JSON-RPC API responses (bats + rpc.bash)
|
||
- L2 UI: HTTP probe of app URLs (bats + ui-probes.bash)
|
||
- L3 lifecycle survival: container restart/reboot survival (bats, gated)
|
||
- Latency: 30–120s per suite depending on tier and container startup
|
||
|
||
## Common Patterns
|
||
|
||
**Async Testing (Vitest):**
|
||
|
||
```typescript
|
||
it('makes a successful RPC call and returns the result', async () => {
|
||
mockFetch.mockResolvedValueOnce(jsonResponse({ result: { did: 'did:key:z123' } }))
|
||
|
||
const result = await rpcClient.call<{ did: string }>({
|
||
method: 'node.did',
|
||
params: {},
|
||
})
|
||
|
||
expect(result).toEqual({ did: 'did:key:z123' })
|
||
expect(mockFetch).toHaveBeenCalledOnce()
|
||
})
|
||
```
|
||
|
||
- `async` keyword on test function
|
||
- `await` for async operations
|
||
- No explicit promise handling; expect called after `await` completes
|
||
- Timeouts set via test config or `{ timeout: N }` in individual tests
|
||
|
||
**Error Testing (Vitest):**
|
||
|
||
```typescript
|
||
it('throws after max retries on persistent 502', async () => {
|
||
mockFetch.mockResolvedValue(jsonResponse(null, 502, 'Bad Gateway'))
|
||
|
||
await expect(rpcClient.call({ method: 'test' })).rejects.toThrow('HTTP 502: Bad Gateway')
|
||
expect(mockFetch).toHaveBeenCalledTimes(3)
|
||
})
|
||
|
||
it('throws immediately on non-retryable HTTP errors', async () => {
|
||
mockFetch.mockResolvedValueOnce(jsonResponse(null, 401, 'Unauthorized'))
|
||
|
||
await expect(rpcClient.call({ method: 'test' })).rejects.toThrow('Session expired')
|
||
expect(mockFetch).toHaveBeenCalledOnce()
|
||
})
|
||
```
|
||
|
||
- `expect(...).rejects.toThrow(message)` for expected rejections
|
||
- Mock returns set per-call (`mockResolvedValueOnce`, `mockResolvedValue`)
|
||
- Retry logic verified via call count assertions (`toHaveBeenCalledTimes`)
|
||
|
||
**Timer Mocking (Vitest):**
|
||
|
||
```typescript
|
||
beforeEach(() => {
|
||
vi.useFakeTimers({ shouldAdvanceTime: true })
|
||
})
|
||
|
||
afterEach(() => {
|
||
vi.useRealTimers()
|
||
})
|
||
|
||
it('retries on 502 Bad Gateway and eventually succeeds', async () => {
|
||
mockFetch
|
||
.mockResolvedValueOnce(jsonResponse(null, 502, 'Bad Gateway'))
|
||
.mockResolvedValueOnce(jsonResponse({ result: 'ok' }))
|
||
|
||
const result = await rpcClient.call({ method: 'test' })
|
||
|
||
expect(result).toBe('ok')
|
||
expect(mockFetch).toHaveBeenCalledTimes(2)
|
||
})
|
||
```
|
||
|
||
- Fake timers enable testing of timeout/retry delays without blocking real time
|
||
- `shouldAdvanceTime: true` auto-advances clock for non-blocking tests
|
||
- Clean up with `vi.useRealTimers()` after each test
|
||
|
||
**Vue Component Testing (Vue Test Utils):**
|
||
|
||
```typescript
|
||
it('returns correct values for audio', () => {
|
||
const ext = ref('mp3')
|
||
const isDir = ref(false)
|
||
const result = useFileType(ext, isDir)
|
||
|
||
expect(result.category.value).toBe('audio')
|
||
expect(result.isAudio.value).toBe(true)
|
||
expect(result.isImage.value).toBe(false)
|
||
expect(result.iconColor.value).toBe('text-orange-400')
|
||
})
|
||
```
|
||
|
||
- Refs created with `ref()` passed as test inputs
|
||
- Computed values accessed via `.value`
|
||
- No mount overhead for pure composable logic
|
||
|
||
**Playwright Browser Testing:**
|
||
|
||
```typescript
|
||
test('installed app launch opens reachable app URL', async ({ page, context, baseURL }) => {
|
||
await login(page)
|
||
await page.goto('/dashboard/apps', { waitUntil: 'domcontentloaded' })
|
||
|
||
const appCard = page.locator('[data-controller-container]', {
|
||
has: page.getByRole('heading', { name: APP_CARD_TITLE, exact: true }),
|
||
}).first()
|
||
|
||
await appCard.waitFor({ timeout: 30_000 })
|
||
await expect(appCard.locator('button')).toBeVisible()
|
||
})
|
||
```
|
||
|
||
- Locators used to find elements (CSS selector, role, text)
|
||
- Wait timeouts on slow networks (30s for app startup)
|
||
- `waitUntil: 'domcontentloaded'` or `'networkidle'` for page load
|
||
- Screenshots and video recording available via config
|
||
|
||
## Test Configuration Details
|
||
|
||
**Vitest Config (`vitest.config.ts`):**
|
||
- Environment: jsdom
|
||
- Globals: enabled (no imports needed)
|
||
- Setup file: `vitest.setup.ts` (mocks global Vue config like `$ver`)
|
||
- Coverage provider: v8
|
||
- Coverage threshold: 80% branches
|
||
- Excluded from coverage: tests, types, main.ts
|
||
|
||
**Playwright Config (implicit, environment variables used):**
|
||
- Base URL: derived from `VITE_*` env vars in dev
|
||
- Timeouts: per-test overrides via `{ timeout: N }`
|
||
- Retry: 0 (no automatic retries; explicit in tests via polling)
|
||
- Config environment variables: `ARCHY_PASSWORD`, `ARCHY_APP_ID`, `ARCHY_EXPECTED_LAUNCH_URL`
|
||
|
||
**Shell Test Config (environment variables):**
|
||
- `ARCHY_PASSWORD`: login password (required)
|
||
- `ARCHY_ALLOW_DESTRUCTIVE`: enable stop/start/restart/uninstall tests
|
||
- `ARCHY_ALLOW_CASCADE_DESTRUCTIVE`: enable uninstall/reinstall on throwaway app
|
||
- `ARCHY_ITERATIONS`: loop count for release gate (5× for production readiness)
|
||
- `ARCHY_FORCE_LOGIN`: fresh RPC token per test file
|
||
|
||
---
|
||
|
||
*Testing analysis: 2026-07-29*
|