docs(qr): scanner snappiness research + companion-dev handover; 10/s native decode
All checks were successful
Demo images / Build & push demo images (push) Successful in 4m16s

Research findings and a concrete split of work: web-side items for this
repo (pre-warm camera, torch toggle, continuous focus, keep-stream-
alive) and a native handover list for the companion dev (pre-warmed
CameraX + ML Kit, QR-only format, 720p keep-latest analysis, torch,
zoom nudge, haptic dismiss) with acceptance criteria and how to
measure. Quick win landed now: live scan runs at 10 scans/sec when the
platform has a native BarcodeDetector, keeping 4/s only for the
JS-worker fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
archipelago 2026-07-29 10:51:39 -04:00
parent 8bea3707ca
commit 21de734385
2 changed files with 87 additions and 3 deletions

View File

@ -0,0 +1,82 @@
# QR scanner snappiness — research + companion-dev handover
*2026-07-29. Owner: web side = node repo (this doc's "web" items); native side =
companion app dev (Mac). Backlog origin: UNIFIED-TASK-TRACKER "optimise
companion QR scan (quicker start/decode, low-light)".*
## Where scanning happens today
| Path | Stack | Used when |
|---|---|---|
| Web live scan | nimiq `qr-scanner` 1.4.x over `getUserMedia`, in `WalletScanModal.vue` | HTTPS browsers / secure contexts |
| Photo fallback | `<input capture>` photo → `BarcodeDetector` if present, else `qr-scanner.scanImage` multi-pass (`decodePhotoRobust`) | Plain-http (LAN) where `getUserMedia` doesn't exist |
| Native scan | `ArchipelagoQr` JS bridge → companion's native scanner (0.5.22 fixed dense invoice QRs) | Inside the companion app |
## What makes it feel slow (ranked)
1. **Camera cold-start** — the stream starts only after the user reaches the
scan pane; on phones `getUserMedia` + first frame is routinely 6001500ms,
and the native path pays a similar CameraX bind + ML Kit model cold-start.
2. **Decode cadence** — web live scan was capped at 4 scans/sec (WebView
preview lagged at 10/s when decoding on the JS worker). A hand-held code
therefore waits up to 250ms *after* it's already sharp and centered.
3. **Low light / focus hunting** — no torch control anywhere; no explicit
continuous-focus request. Dense LN invoices need sharpness more than
resolution.
4. **Dense-QR decode budget** — big bolt11/catalog QRs push the JS decoder
hard; the native ML Kit path is far better at these (proven by 0.5.22).
## Web side (node repo — can be done here)
- ✅ DONE (2026-07-29): scan at **10/s when `BarcodeDetector` exists** (Chrome/
Android WebView decode natively — cheap), keep 4/s only for the JS-worker
fallback.
- **Pre-warm the camera**: start `getUserMedia` the moment the modal opens
(action pane), not when the scan pane is reached — hide the preview until
needed. Saves the entire cold-start from the user's perceived timeline.
- **Torch toggle**: `qr-scanner` exposes `hasFlash()/turnFlashOn()` — add a 🔦
button on the scan pane (it silently no-ops where unsupported).
- **Continuous focus + modest resolution**: pass constraints
`{ focusMode: 'continuous', width: { ideal: 1280 } }` — 720p-class frames
start faster AND decode faster than 1080p+, with no loss for QR density
that matters to us.
- **Don't stop/start between panes**: returning from amount → scan currently
re-inits the scanner; keep the (paused) stream alive while the modal lives.
## Native side (companion dev handover)
The `ArchipelagoQr` bridge overlay is the right architecture — these are
tuning items inside the native scanner activity:
1. **Pre-warm CameraX + ML Kit**: bind the camera provider and instantiate
`BarcodeScanning.getClient(...)` when the WebView *requests* the overlay —
or even when the wallet modal opens (add a `ArchipelagoQr.prewarm()` bridge
method; the web side will call it if present). ML Kit's first-inference
model load is 100300ms — pay it before the user aims.
2. **Restrict formats**: `BarcodeScannerOptions` with `FORMAT_QR_CODE` only —
skipping the other symbologies measurably cuts per-frame latency.
3. **Analysis resolution ≈ 1280×720** with
`ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST` — never queue stale frames;
decode the newest one only.
4. **Continuous autofocus + tap-to-focus** on the preview, and a **torch
toggle** (low-light was an explicit user complaint).
5. **`zoomRatio` nudge for small codes**: if no hit after ~2s, step zoom to
1.5× — helps distant/small printed codes without user action.
6. **Success haptic + instant dismiss**: vibrate on decode and close the
overlay immediately; perceived speed is heavily back-loaded.
7. Optional: **ML Kit `enableAllPotentialBarcodes` off** and skip inverted
scans unless first pass fails (inverted QRs are rare; halves work).
## Acceptance criteria
- Cold open → first successful scan of a normal invoice QR in **< 2s** on the
companion app, **< 3s** in a mobile browser.
- Dense (700+ char) bolt11 QR decodes in **< 1.5s** once framed, both paths.
- Dim-room scan succeeds with the torch toggle without leaving the scanner.
## Verification notes for whoever implements
- Measure with a timestamp log: overlay-requested → camera-first-frame →
decode-success. The three deltas map 1:1 onto items above.
- Web `BarcodeDetector` presence differs per WebView/Play-Services build —
keep the JS-worker fallback path intact.

View File

@ -386,9 +386,11 @@ async function startScanning() {
returnDetailedScanResult: true,
highlightScanRegion: false,
preferredCamera: 'environment',
// 4/s decodes plenty fast for a hand-held QR while keeping the
// preview smooth on phone WebViews (10/s visibly lagged the video).
maxScansPerSecond: 4,
// With a native BarcodeDetector (Chrome/Android WebView) decoding is
// hardware-cheap scan at 10/s for a snappier lock-on. The 4/s cap
// remains for the JS-worker fallback, where 10/s visibly lagged the
// preview on phone WebViews.
maxScansPerSecond: 'BarcodeDetector' in window ? 10 : 4,
}
)
await qrScanner.value.start()