From 21de734385e8f7bd4286954a6e8b56ca41487b5a Mon Sep 17 00:00:00 2001 From: archipelago Date: Wed, 29 Jul 2026 10:51:39 -0400 Subject: [PATCH] docs(qr): scanner snappiness research + companion-dev handover; 10/s native decode Research findings and a concrete split of work: web-side items for this repo (pre-warm camera, torch toggle, continuous focus, keep-stream- alive) and a native handover list for the companion dev (pre-warmed CameraX + ML Kit, QR-only format, 720p keep-latest analysis, torch, zoom nudge, haptic dismiss) with acceptance criteria and how to measure. Quick win landed now: live scan runs at 10 scans/sec when the platform has a native BarcodeDetector, keeping 4/s only for the JS-worker fallback. Co-Authored-By: Claude Fable 5 --- docs/qr-scanner-snappiness-handover.md | 82 +++++++++++++++++++++ neode-ui/src/components/WalletScanModal.vue | 8 +- 2 files changed, 87 insertions(+), 3 deletions(-) create mode 100644 docs/qr-scanner-snappiness-handover.md diff --git a/docs/qr-scanner-snappiness-handover.md b/docs/qr-scanner-snappiness-handover.md new file mode 100644 index 00000000..f53cd4a7 --- /dev/null +++ b/docs/qr-scanner-snappiness-handover.md @@ -0,0 +1,82 @@ +# QR scanner snappiness — research + companion-dev handover + +*2026-07-29. Owner: web side = node repo (this doc's "web" items); native side = +companion app dev (Mac). Backlog origin: UNIFIED-TASK-TRACKER "optimise +companion QR scan (quicker start/decode, low-light)".* + +## Where scanning happens today + +| Path | Stack | Used when | +|---|---|---| +| Web live scan | nimiq `qr-scanner` 1.4.x over `getUserMedia`, in `WalletScanModal.vue` | HTTPS browsers / secure contexts | +| Photo fallback | `` photo → `BarcodeDetector` if present, else `qr-scanner.scanImage` multi-pass (`decodePhotoRobust`) | Plain-http (LAN) where `getUserMedia` doesn't exist | +| Native scan | `ArchipelagoQr` JS bridge → companion's native scanner (0.5.22 fixed dense invoice QRs) | Inside the companion app | + +## What makes it feel slow (ranked) + +1. **Camera cold-start** — the stream starts only after the user reaches the + scan pane; on phones `getUserMedia` + first frame is routinely 600–1500ms, + and the native path pays a similar CameraX bind + ML Kit model cold-start. +2. **Decode cadence** — web live scan was capped at 4 scans/sec (WebView + preview lagged at 10/s when decoding on the JS worker). A hand-held code + therefore waits up to 250ms *after* it's already sharp and centered. +3. **Low light / focus hunting** — no torch control anywhere; no explicit + continuous-focus request. Dense LN invoices need sharpness more than + resolution. +4. **Dense-QR decode budget** — big bolt11/catalog QRs push the JS decoder + hard; the native ML Kit path is far better at these (proven by 0.5.22). + +## Web side (node repo — can be done here) + +- ✅ DONE (2026-07-29): scan at **10/s when `BarcodeDetector` exists** (Chrome/ + Android WebView decode natively — cheap), keep 4/s only for the JS-worker + fallback. +- **Pre-warm the camera**: start `getUserMedia` the moment the modal opens + (action pane), not when the scan pane is reached — hide the preview until + needed. Saves the entire cold-start from the user's perceived timeline. +- **Torch toggle**: `qr-scanner` exposes `hasFlash()/turnFlashOn()` — add a 🔦 + button on the scan pane (it silently no-ops where unsupported). +- **Continuous focus + modest resolution**: pass constraints + `{ focusMode: 'continuous', width: { ideal: 1280 } }` — 720p-class frames + start faster AND decode faster than 1080p+, with no loss for QR density + that matters to us. +- **Don't stop/start between panes**: returning from amount → scan currently + re-inits the scanner; keep the (paused) stream alive while the modal lives. + +## Native side (companion dev handover) + +The `ArchipelagoQr` bridge overlay is the right architecture — these are +tuning items inside the native scanner activity: + +1. **Pre-warm CameraX + ML Kit**: bind the camera provider and instantiate + `BarcodeScanning.getClient(...)` when the WebView *requests* the overlay — + or even when the wallet modal opens (add a `ArchipelagoQr.prewarm()` bridge + method; the web side will call it if present). ML Kit's first-inference + model load is 100–300ms — pay it before the user aims. +2. **Restrict formats**: `BarcodeScannerOptions` with `FORMAT_QR_CODE` only — + skipping the other symbologies measurably cuts per-frame latency. +3. **Analysis resolution ≈ 1280×720** with + `ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST` — never queue stale frames; + decode the newest one only. +4. **Continuous autofocus + tap-to-focus** on the preview, and a **torch + toggle** (low-light was an explicit user complaint). +5. **`zoomRatio` nudge for small codes**: if no hit after ~2s, step zoom to + 1.5× — helps distant/small printed codes without user action. +6. **Success haptic + instant dismiss**: vibrate on decode and close the + overlay immediately; perceived speed is heavily back-loaded. +7. Optional: **ML Kit `enableAllPotentialBarcodes` off** and skip inverted + scans unless first pass fails (inverted QRs are rare; halves work). + +## Acceptance criteria + +- Cold open → first successful scan of a normal invoice QR in **< 2s** on the + companion app, **< 3s** in a mobile browser. +- Dense (700+ char) bolt11 QR decodes in **< 1.5s** once framed, both paths. +- Dim-room scan succeeds with the torch toggle without leaving the scanner. + +## Verification notes for whoever implements + +- Measure with a timestamp log: overlay-requested → camera-first-frame → + decode-success. The three deltas map 1:1 onto items above. +- Web `BarcodeDetector` presence differs per WebView/Play-Services build — + keep the JS-worker fallback path intact. diff --git a/neode-ui/src/components/WalletScanModal.vue b/neode-ui/src/components/WalletScanModal.vue index 73c4c403..3fe113d0 100644 --- a/neode-ui/src/components/WalletScanModal.vue +++ b/neode-ui/src/components/WalletScanModal.vue @@ -386,9 +386,11 @@ async function startScanning() { returnDetailedScanResult: true, highlightScanRegion: false, preferredCamera: 'environment', - // 4/s decodes plenty fast for a hand-held QR while keeping the - // preview smooth on phone WebViews (10/s visibly lagged the video). - maxScansPerSecond: 4, + // With a native BarcodeDetector (Chrome/Android WebView) decoding is + // hardware-cheap — scan at 10/s for a snappier lock-on. The 4/s cap + // remains for the JS-worker fallback, where 10/s visibly lagged the + // preview on phone WebViews. + maxScansPerSecond: 'BarcodeDetector' in window ? 10 : 4, } ) await qrScanner.value.start()