Skip to main content

decide_hw_fullframe

Function decide_hw_fullframe 

Source
pub fn decide_hw_fullframe(
    st: &mut StripeState,
    settings: &RustCaptureSettings,
    frame_counter: u16,
    damage: Damage,
    is_animated: bool,
    requested_idr: bool,
    encoder: EncoderQuality,
) -> HwFrameDecision
Expand description

The send / quality / keyframe policy every full-frame encoder obeys. A constant-quality session takes decide_constant_quality; what follows is the constant-rate one.

The GOP is left infinite and an IDR is forced only when a consumer genuinely needs a fresh decode entry point or a still screen is cleaned up. Every full-frame encoder shares this one function so they cannot drift apart; the striped software path applies the same policy per stripe inside encode_cpu.

  1. Cleanup (cleanup_due): once the content stops changing, whatever Turbo sends, a refresh and, after a large change and a longer stillness, a key frame where the encoder’s cleanup ends on one (EncoderQuality::keys), each encoded at the paint-over quantizer held for it (hold_qp, which the encoder applies under any rate control; a refresh of a constant-rate session coarsened to held_refresh_quality, a key frame capped by the encoder at HELD_KEY_BUDGET_S) and followed by a recovery burst. A constant-rate session whose rate control reports its frames’ bytes (EncoderQuality::converges) is cleaned up through that rate control instead, NVENC’s like this: the frames keep flowing, each within the rate control’s budget, until it codes the screen at the paint-over quantizer or finer in a frame under CONVERGED_SHARE of the budget (convergence), for CONVERGE_S at most. A frame held at the paint-over quantizer the size of a whole refresh would otherwise queue for up to half a second on a link near the stream’s rate; the rate control reaches the same picture in about a second at the default rate with every frame within its budget. Only where it has not converged within FALLBACK_TRIGGERS periods, or stalls short of the paint-over quality (STALL_TRIGGERS periods of small frames at a coarser quantizer), as where the rate is low for the resolution, is the picture refreshed at held_refresh_quality, which ends the cleanup: a band a frame where the encoder holds one (FIRST_BAND, BAND_BUDGETS) until the sweep has covered it or motion ends it, else one held frame. A sweep whose rest would take it past REFINE_S at its bands’ pace, once they are sized to the budget, is coded a step coarser a band until the pace fits, and ends where that reaches the rate control’s own quantizer: dense text at 0.1 and 0.25 Mbit/s never finished in 40 s at the paint-over one. At 2 Mbit/s at 1080p a key frame capped to HELD_KEY_BUDGET_S of the target came out coarser than the picture it cleaned (35.9 dB), where the refresh reached 44.2. A rate control whose frames of the still screen run over OVERSHOOT_BUDGETS for a trigger period is pinned at its coarsest quantizer and refines nothing within the rate, and every frame of a sweep would carry the rest of the picture at that cost: NVENC’s at 0.1 Mbit/s codes a still 1080p screen of texture in 16-budget frames, and its sweep sent 4.7 MB in 40 s without ending. Such a screen is refreshed at once in one held frame, coarsened to what HELD_KEY_BUDGET_S buys at those frames’ size and coded again to fit it where it comes out past HELD_REFRESH_LIMIT_S, as dense text does, and a burst that has no refresh to fall back to ends. A session that reports its frames’ bytes but holds no quantizer at a constant rate (VA-API, x265, kvazaar, libvpx’s VP9) has no refresh to fall back to, so its frames flow, each the rate control’s, until the picture is done: where the session measures its reconstruction against its source (VA-API, plateau) until that stops improving, else after a trigger period of frames at the paint-over quantizer or once the finest quantizer it has coded at has stood for LEVEL_S; and in either case after STALL_TRIGGERS periods of small frames or REFINE_S. A fixed burst left such a screen where the motion did (19 to 27 dB at 2 Mbit/s at 1080p on iHD, x265, and SVT-AV1, each frame within its budget), and a reported quantizer alone ends it early or never: iHD’s H.264 average reads 26 through a refinement from 23 to 50 dB. That cleanup then counts as settled (StripeState::settled), and only a change of more than a small area arms another, so a caret blinking on a clean screen costs its own frames and no more. A session that reports no bytes (Tegra, a stateful V4L2 device) gets the refresh and burst at its rate control’s own quality, and no key frame. A change of more than a small area ends a cleanup through the rate control; a caret blinking or a clock ticking on the screen does not, so constant low motion is cleaned up the same way.
  2. Recovery keyframe: a requested or scheduled IDR, at the rate control’s own quality: it answers a join or a loss, often on a link that just fell behind, where a key frame the size of a cleanup would only fall behind again. On a still screen it opens the burst, which lasts until the rate control converges where the cleanup runs through it, or is pinned (OVERSHOOT_BUDGETS).
  3. Recovery burst: the frames after a keyframe or cleanup on a still screen keep flowing so rate control settles, until motion resumes: after a cleanup held at the refresh’s quantizer where the paint-over one still improves on the rate control’s, so they go on refining the picture (a key frame capped to its budget below the paint-over quality included), and so after any keyframe of a constant-quality session; after a recovery key frame of a constant-rate session they are the rate control’s.
  4. Motion, Turbo (video_streaming_mode), and animated overlays send at the session’s quality; a still screen outside all of the above sends nothing.

§Arguments

  • st - Per-region state carrying the cleanup and burst bookkeeping.
  • settings - Capture settings; reads CRF, paint-over CRF, burst frames, trigger frames, streaming mode, and keyframe interval.
  • frame_counter - Current frame number (wrapping u16).
  • damage - What changed, from compositor damage, the X server, the driver, or a content hash.
  • is_animated - Forces a send for animated overlays.
  • requested_idr - On-demand IDR request (client join / reset / recording cadence).
  • encoder - What the encoder says of its frames (EncoderQuality).

§Returns

HwFrameDecision with send, force_idr, target_qp (the session quality index), hold_qp (the quantizer a cleanup holds its frame at), and hold_band (the share of the picture it covers).