Crate pixelflux 

pixelflux

A high-performance screen capture and encoding pipeline exposed as a Python extension via PyO3. It supports two independent backends — X11 (XShm + XFixes) and Wayland (a headless Smithay compositor) — and a shared encoding layer that dispatches to software (striped JPEG, and H.264 through the build’s software encoder: libx264 with the gpl feature, OpenH264 without) or hardware (NVENC, VA-API) encoders based on the available GPU and operator settings.

Crate structure

Module	Purpose
encoders	Encoder backends: software H.264 (libx264 or OpenH264) / JPEG, NVENC, VA-API, watermark overlay
wayland	Headless Smithay compositor, cursor rendering
x11	X11/XShm capture loop, XFixes out-of-band cursor monitor, and stripe dispatch
pipeline	Frame-processing policy shared by both backends (send/QP/keyframe decisions)
recording_sink	Unix-socket H.264 fan-out for external recording
recorder	Built-in MP4 recorder (fMP4 muxer + Python/env/REST control surfaces)
computer_use	HTTP API for AI-agent desktop control (screenshots, input injection)
nvgpufilter	Multi-GPU NVENC device filtering via ioctl
webcam	Virtual camera: client webcam uplink decoded into a V4L2 device (interposer ring, v4l2loopback, PipeWire node)

Data flow

Python  ──►  CaptureSettings  ──►  X11 / Wayland backend
                                        │
                                   frame pixels
                                        │
                                   ┌────┴────┐
                                   │ Encoder │  (NVENC / VAAPI / x264 or OpenH264 / JPEG)
                                   └────┬────┘
                                        │
                                  EncodedStripe(s)
                                        │
                                   Python callback

Re-exports

pub use encoders::nvenc;

pub use encoders::software::StripeState;

pub use encoders::vaapi;

Modules

computer_use

HTTP server implementing the Anthropic Computer Use spec for AI agent desktop control. HTTP server implementing the Anthropic Computer Use specification.

encoders

nvgpufilter

Multi-GPU NVENC device filtering via kernel ioctl. Multi-GPU NVENC GET_ATTACHED_IDS / GET_PROBED_IDS ioctl filter — it exists so a container handed only a subset of the host’s GPUs can still open an NVENC session.

pipeline

Frame-processing policy shared by the X11 and Wayland backends. Frame-processing policy shared by the two capture backends. It lives in its own module for one reason: the Wayland path (dmabuf, compositor damage) and the X11 path (host-ARGB, stripe-hash damage) capture pixels in completely different ways, but a viewer must never be able to tell which one produced a frame — a paint-over refresh or a recovery keyframe has to behave identically either way. Keeping the decision logic here, source-agnostic, is what guarantees it.

recorder

Built-in MP4 recorder: independent capture-to-file with Python/env/REST control. Built-in MP4 recorder: an independent pipeline that captures the desktop to a ready-to-play fragmented MP4 without needing any connected client.

recording_sink

Unix-socket H.264 recording fan-out for external capture tools. Unix-socket H.264 fan-out for external recording.

wayland

Headless Wayland compositor and cursor rendering. Wayland backend: a headless Smithay compositor that stands in for a real display server.

webcam

Virtual camera: the browser’s webcam, delivered to applications as a V4L2 capture device.

x11

X11/XShm capture loop, stripe dispatch, and per-stripe change detection. X11 host capture: grab the root window into host memory as BGRA, composite the XFixes hardware cursor and the watermark on the CPU, and feed each frame to pipeline::X11Pipeline, which owns damage/stripe/encode. The grab goes through a shared-memory segment (XShm via x11rb) rather than a plain GetImage for one reason: a full-screen frame is far too large to copy through the X protocol socket every tick, so XShm has the server write the pixels straight into memory this process already has mapped.

Structs

LiveTunables

The per-frame decision/quality knobs every encoder re-reads from the settings on each tick, so they retune a running capture with no encoder re-init: x264 reconfigures, NVENC CQP retargets, VAAPI re-opens only its codec ctx, JPEG is stateless. Applied on the thread that owns the settings copy. Structural switches (encoder, chroma, RC mode, device) still need a capture restart.

RustCaptureSettings

The full set of capture + encode parameters the Python layer hands to the Rust backend.

VirtualKeyboardUnavailable

The target compositor does not advertise zwp_virtual_keyboard_manager_v1.

WlEncodeControls

Cross-thread controls for the Wayland encode thread, the X11 Controls scheme: the UpdateRate handler stores the current values then flips rate_dirty with Release; the encode thread swaps it with Acquire and re-reads the payload, never seeing it half-applied. force_idr is swapped just before each encode, so an on-demand keyframe lands on the frame ALREADY in flight instead of waiting one pipeline stage for the next publish.

WlEncodeStats

Shared capture stats: whichever thread owns the encoders counts frames/stripes and composes desc + n_stripes (the encoder half of the 1 s debug log line); the calloop log loads, prints and resets the counters.

WlFrame

One captured host-pixel frame in flight from the calloop (render/readback) to the Wayland encode thread: the pixels plus the per-frame inputs of the encode dispatch (damage, overlay animation); the IDR request travels separately via the controls atomic.

WlFramePool

Render->encode handoff for the Wayland readback paths, mirroring the X11 FramePool’s single-slot non-dropping design with one deliberate difference: the calloop thread is also the compositor + input dispatcher, so it must NEVER block on the pool. try_begin hands out a buffer only while the publish slot is empty, so publish cannot block, and a saturated encoder throttles capture by SKIPPING ticks (compositor damage accumulates via buffer age, so nothing is lost). Every published frame is encoded, in order: the H.264 reference chain stays contiguous exactly as on X11.

Enums

CompositionElements

ThreadCommand

Control messages sent from the Python-facing methods to the capture thread.

Type Aliases

OutputDesc

One live output as (id, x, y, width, height, scale, capturing). Sizes are physical pixels and (x, y) is the layout offset.

WindowDesc

One mapped window as (window_id, title, app_id, output_id, parked). A parked window has no output yet — a nested session’s spare screens.