Crate pixelflux 

pixelflux

A high-performance screen capture and encoding pipeline exposed as a Python extension via PyO3. It supports two independent backends — X11 (XShm + XFixes) and Wayland (a headless Smithay compositor) — and a shared encoding layer that dispatches to software (striped JPEG, H.264 through the build’s software encoder — libx264 with the gpl feature, OpenH264 without — and the other codecs through the software encoders the build links: x265 or kvazaar, libvpx, SVT-AV1) or hardware (NVENC, VA-API) encoders based on the available GPU and operator settings.

Crate structure

Module	Purpose
encoders	Encoder backends: software H.264 (libx264 or OpenH264) / JPEG, NVENC, VA-API, watermark overlay
wayland	Headless Smithay compositor, cursor rendering
x11	X11/XShm capture loop, XFixes out-of-band cursor monitor, and stripe dispatch
pipeline	Frame-processing policy shared by both backends (send/QP/keyframe decisions)
recording_sink	Unix-socket H.264 fan-out for external recording
recorder	Built-in MP4 recorder (fMP4 muxer + Python/env/REST control surfaces)
computer_use	HTTP API for AI-agent desktop control (screenshots, input injection)
nvgpufilter	Multi-GPU NVENC device filtering via ioctl
webcam	Virtual camera: client webcam uplink decoded into a V4L2 device (interposer ring, v4l2loopback, PipeWire node)

Data flow

Python  ──►  CaptureSettings  ──►  X11 / Wayland backend
                                        │
                                   frame pixels
                                        │
                                   ┌────┴────┐
                                   │ Encoder │  (NVENC / VAAPI / x264 or OpenH264 / JPEG)
                                   └────┬────┘
                                        │
                                  EncodedStripe(s)
                                        │
                                   Python callback

Re-exports

pub use encoders::nvenc;

pub use encoders::software::StripeState;

Modules

computer_use

HTTP server implementing the Anthropic Computer Use spec for AI agent desktop control. HTTP server implementing the Anthropic Computer Use specification.

encoders

Encoder backends and the codec identities, wire framing, and rate-control policy they share. Encoder backends and what they share: the codec identities and wire framing, the rate-control policy, which software encoder a build resolves each codec to, and which hardware encoder a render node serves each with.

log

The debug switch behind every backend’s tagged line. The verbosity switch behind every backend’s tagged line.

nvgpufilter

Multi-GPU NVENC device filtering via kernel ioctl. Multi-GPU NVENC GET_ATTACHED_IDS / GET_PROBED_IDS ioctl filter — it exists so a container handed only a subset of the host’s GPUs can still open an NVENC session.

pace

When a capture is due a frame, shared by the X11 and Wayland backends. The frame pacing every capture backend shares: when a display is due a frame, and how far a change that just happened may bring the next one forward.

pipeline

Frame-processing policy shared by the X11 and Wayland backends. Frame-processing policy shared by the two capture backends. It lives in its own module for one reason: the Wayland path (dmabuf, compositor damage) and the X11 path (host-ARGB, stripe-hash damage) capture pixels in completely different ways, but a viewer must never be able to tell which one produced a frame — a paint-over refresh or a recovery keyframe has to behave identically either way. Keeping the decision logic here, source-agnostic, is what guarantees it.

pipewire

Run-time libpipewire binding and SPA pod encoding shared by the webcam sink and host capture. Run-time binding to libpipewire-0.3 and the SPA pod encoding shared by every PipeWire user in pixelflux: the webcam sink (a Video/Source node) and the portal screen capture (a Video/Input stream). The library is loaded with dlopen, so pixelflux keeps no build-time or load-time dependency on PipeWire and each user simply stays off where the library or the daemon is absent. Pods are built and read by hand against the SPA wire layout: 8-byte aligned size, type, payload cells, objects carrying key, flags, cell properties, and a choice cell whose first value is the default (and, once fixated, the only) value.

recorder

Built-in MP4 recorder: independent capture-to-file with Python/env/REST control. Built-in MP4 recorder: an independent pipeline that captures the desktop to a ready-to-play fragmented MP4 without needing any connected client.

recording_sink

Unix-socket H.264 recording fan-out for external capture tools. Unix-socket video fan-out for external recording.

report

What each capture streams and how it got there, as values a caller reads. What a capture streams and how it got there, for the caller to show its user: the capture path and whether it is zero-copy, the encoder and whether it is hardware, the GPU behind it, and the reason a faster path was declined. The same facts the tagged log lines state, kept as values so nobody has to read a log to learn them.

uinput

Kernel uinput devices, the first rung of host-capture input injection. Kernel input devices for host-capture injection, over /dev/uinput.

wayland

Headless Wayland compositor and cursor rendering. Wayland backend: a headless Smithay compositor that stands in for a real display server.

webcam

Virtual camera: the browser’s webcam, delivered to applications as a V4L2 capture device.

x11

X11/XShm capture loop, stripe dispatch, and per-stripe change detection. X11 host capture: grab the root window into host memory as BGRA, composite the XFixes hardware cursor and the watermark on the CPU, and feed each frame to pipeline::X11Pipeline, which owns damage/stripe/encode. The grab goes through a shared-memory segment (XShm via x11rb) rather than a plain GetImage for one reason: a full-screen frame is far too large to copy through the X protocol socket every tick, so XShm has the server write the pixels straight into memory this process already has mapped.

Structs

LiveTunables

The per-frame decision/quality knobs every encoder re-reads from the settings on each tick, so they retune a running capture: x264, x265, and libvpx reconfigure, NVENC and VA-API retarget their constant quantizer, kvazaar and SVT-AV1 re-open, JPEG is stateless. Applied on the thread that owns the settings copy. Structural switches (encoder, chroma, RC mode, device) still need a capture restart.

RustCaptureSettings

The full set of capture + encode parameters the Python layer hands to the Rust backend.

VirtualKeyboardUnavailable

The target compositor does not advertise zwp_virtual_keyboard_manager_v1.

WlEncodeControls

Cross-thread controls for the Wayland encode thread, the X11 Controls scheme: the UpdateRate handler stores the current values then flips rate_dirty with Release; the encode thread swaps it with Acquire and re-reads the payload, never seeing it half-applied. force_idr is swapped just before each encode, so an on-demand keyframe lands on the frame ALREADY in flight instead of waiting one pipeline stage for the next publish. reference_reports are what clients said of frames (lost, or held by every one of them), drained ahead of the same encode.

WlEncodeStats

Shared capture stats: whichever thread owns the encoders counts frames/stripes and composes desc + n_stripes (the encoder half of the 1 s debug log line); the calloop log loads, prints, and resets the counters.

WlFrame

One captured host-pixel frame in flight from the calloop (render/readback) to the Wayland encode thread: the pixels plus the per-frame inputs of the encode dispatch (damage, overlay animation); the IDR request travels separately via the controls atomic.

WlFramePool

Render->encode handoff for the Wayland readback paths, mirroring the X11 FramePool’s single-slot non-dropping design with one deliberate difference: the calloop thread is also the compositor + input dispatcher, so it must NEVER block on the pool. try_begin hands out a buffer only while the publish slot is empty, so publish cannot block, and a saturated encoder throttles capture by SKIPPING ticks (compositor damage accumulates via buffer age, so nothing is lost). Every published frame is encoded, in order: the H.264 reference chain stays contiguous exactly as on X11.

Enums

CompositionElements

ThreadCommand

Control messages sent from the Python-facing methods to the capture thread.

Type Aliases

OutputDesc

One live output as (id, x, y, width, height, scale, capturing). Sizes are physical pixels and (x, y) is the layout offset.

WindowDesc

One mapped window as (window_id, title, app_id, output_id, parked). A parked window has no output yet — a nested session’s spare screens.