feat(kiosk): companion remote drives app iframes via trusted CDP input
Demo images / Build & push demo images (push) Successful in 3m11s

Companion tap/scroll/type now works INSIDE cross-origin app iframes and
kiosk tabs. The web relay synthesizes untrusted DOM events in the top
document, which can never cross an origin boundary — so apps served
through the appgate were dead to the remote. The kiosk Chromium now
exposes a loopback-only CDP port (default origin check intact, no
--remote-allow-origins) and a backend bridge (api/handler/cdp.rs)
dispatches validated companion input as Input.dispatchKeyEvent /
dispatchMouseEvent / mouseWheel — trusted events that hit-test through
any frame, move real focus, and insert text like a physical device.

- Session keeper self-heals across kiosk Chromium restarts; inert on
  nodes without a kiosk unit (falls back to the existing relay path).
- The kiosk relay subscriber self-tags (?kiosk=1) and the backend mutes
  its key/click/scroll messages while the bridge is live, so input never
  applies twice; cursor moves still flow for the on-screen cursor.
- While companion input is active the native OS pointer is hidden
  (cursor:none, auto-restores 30s after the last event) so the dead
  physical-mouse cursor doesn't sit next to the virtual one.
- docs/tv-input-iframe-apps.md scope note updated: gamepad keys stay on
  uinput; CDP is for companion pointer/typing only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-08-16 05:28:53 -04:00
co-authored by Claude Fable 5
parent 876ecc4bdf
commit 3a3077529b
8 changed files with 517 additions and 7 deletions
+9
View File
@@ -1,4 +1,5 @@
mod blob;
mod cdp;
mod content;
mod dwn;
mod model_proxy;
@@ -51,6 +52,10 @@ pub struct ApiHandler {
/// to the phone's default browser. Lets "open in external browser" apps —
/// which the kiosk can't usefully open itself — launch on the controller.
external_open_tx: broadcast::Sender<String>,
/// Bridge that dispatches companion input into the local kiosk Chromium
/// as trusted CDP events (reaches inside cross-origin app iframes).
/// Inert (never connects) on nodes without a kiosk.
cdp_bridge: cdp::CdpBridge,
/// Content-addressed blob store for attachments shared over mesh/federation.
blob_store: Arc<BlobStore>,
/// Our own node pubkey (hex) — used to self-sign debug/test capabilities.
@@ -79,6 +84,7 @@ impl ApiHandler {
);
let (input_relay_tx, _) = broadcast::channel(64);
let (external_open_tx, _) = broadcast::channel(16);
let cdp_bridge = cdp::CdpBridge::spawn();
// Derive a blob-store capability key from the node's Ed25519 signing
// key. SHA-256 domain-separated so rotating the identity rotates
@@ -109,6 +115,7 @@ impl ApiHandler {
session_store,
input_relay_tx,
external_open_tx,
cdp_bridge,
blob_store,
self_pubkey_hex,
})
@@ -402,6 +409,7 @@ impl ApiHandler {
req,
self.input_relay_tx.clone(),
self.external_open_tx.subscribe(),
self.cdp_bridge.clone(),
)
.await;
}
@@ -416,6 +424,7 @@ impl ApiHandler {
req,
self.input_relay_tx.subscribe(),
self.external_open_tx.clone(),
self.cdp_bridge.clone(),
)
.await;
}