Skip to main content

Module computer_use

Module computer_use 

Source
Expand description

X11 backend for the Computer Use HTTP API: XTEST injection and one-shot root screenshots over a private x11rb connection, so the agent can drive an X session with no active capture and no shared state with any streaming capture thread (the X server itself serializes).

Keysyms the active layout cannot produce (after base+AltGr resolution, which stays the preferred path) are typed through a transient remap: under XGrabServer the keymap is re-fetched, the needed keysyms are bound onto all-NoSymbol spare keycodes with ONE XChangeKeyboardMapping, and the key sequence is injected; after a settle window (which lets the focused client re-fetch the bound map — see type_with_transient_binds) one conditional XChangeKeyboardMapping under a second grab puts the spares back — two MappingNotify broadcasts per action. This coexists with a selkies session’s own spare-keycode overlay allocator on the same server: spares here are chosen DESCENDING from the top of the keycode range while selkies allocates ASCENDING, each grab re-fetches the keymap so every binding selkies already made is respected, the restore clears only keycodes still carrying our content, and selkies’ foreign-change invalidation (fired by each MappingNotify) is self-healing because its held keys live on BOUND (non-NoSymbol) keycodes an all-NoSymbol spare can never steal.

Structs§

CuX11Backend