Module computer_use 

HTTP server implementing the Anthropic Computer Use spec for AI agent desktop control. HTTP server implementing the Anthropic Computer Use specification.

Enabled by setting the PIXELFLUX_CU environment variable to the listen port. The server handles POST /computer-use requests for screenshots, mouse/keyboard injection, scrolling, and cursor position queries. Actions run against a [CuBackend], resolved per request: the Wayland compositor owned by this process when one is registered, otherwise the X server named by DISPLAY (XTEST injection on a private connection, no active capture required).

The same server also exposes the built-in MP4 recorder at /record_start, /record_stop and /record_status, so a headless script can drive a session and record it over plain HTTP.

Structs

CuWaylandBackend

Wayland implementation: every primitive is a ThreadCommand on the compositor’s calloop channel, so injection and readback serialize naturally with rendering and encoding.

Enums

CuButton

Semantic mouse-button vocabulary of the action layer; each backend maps it to its native codes (evdev BTN_ on Wayland, X core buttons 1/2/3 on X11).

Traits

CuBackend

The desktop-side primitives one Computer Use action decomposes into. Keycodes are X/xkb keycodes (evdev + 8) — the numbering both the smithay seat and XTEST consume — and coordinates are framebuffer/root pixels, already clamped by the action layer.

Functions

encode_png_rgba

Turn a raw framebuffer into a PNG the Computer Use agent can actually look at.

register_wayland_backend

Make the Wayland compositor the preferred CU backend: once a live calloop sender is registered, every subsequent request routes to it instead of an X11 connection.

run_cu_server

Expose the captured desktop to an AI agent over HTTP, so a Computer Use client can drive the session much as a human viewer would.

set_app_wayland_display

Set (or clear, with None/empty) the app compositor socket for CU text injection.

spawn_cu_from_env

Start the CU server if PIXELFLUX_CU names a bind (the standalone fallback; a selkies-managed session passes the setting through start_cu_server).

start_cu_server

Start the CU server on bind: a bare port listens on all interfaces (the container-deployment default), host:port scopes it. Guarded so that exactly one server binds per process no matter how many call sites (module import, Wayland compositor init, the selkies setting) race to spawn it; backend selection stays per-request, so a server bound at import serves a compositor that only starts later.