feat(engine,producer): verified interleaved parallel drawElement streaming (opt-in)#2056
Open
vanceingalls wants to merge 2 commits into
Open
feat(engine,producer): verified interleaved parallel drawElement streaming (opt-in)#2056vanceingalls wants to merge 2 commits into
vanceingalls wants to merge 2 commits into
Conversation
…aming (opt-in) Step 2 of the DE engagement plan: multi-worker drawElement capture through the streaming encoder, with the full runtime self-verification net riding along — the confinement rule that kept the parallel clamp in place is now satisfied on this path. Opt-in via HF_DE_PARALLEL_STREAM=true; default routing (including the #2026 single-worker inversion) is unchanged. Mechanism: - distributeFramesInterleaved + WorkerTask.frameStride: worker i captures frames i, i+N, i+2N... — seek-based capture makes stride free and the ordered writer's reorder window shrinks from totalFrames/N to N (contiguous chunks serialize workers behind the writer). - Depth-2 pipelined worker-encode produce in the parallel worker loop (the same shape as the sequential loop; frame k's in-page encode overlaps k+stride's produce). HF_DE_PAR_DEBUG=1 traces the first frames per worker. - Drain guard extracted to createDrainFrameGuard (session-parameterized): every parallel frame gets the SAME blank-guard + PSNR self-verify as the sequential drain, against its owning worker's pre-injection ground truth (all sessions arm identical sample indices from CaptureOptions.compositionDurationSeconds). - FrameReorderBuffer.abort(err): a failed worker (e.g. verification error) rejects all parked and future waiters — without this, peers park forever in waitForFrame and the pool (which awaits ALL workers before surfacing errors) deadlocks. Found by the verify-trip test; unit-tested. - The typed DrawElementVerificationError is preserved past the pool's error-string flattening so the orchestrator's verify-retry recognizes it. - Static-dedup stride hazard fixed: lastEncodeResult reuse now requires EVERY frame in (lastEncodeResultFrame, i] to be predicted-static (sequential capture reduces to the old has(i) check). - Workers get separate browser PROCESSES under the flag: pages co-tenant in one browser starve non-active pages of BeginFrames on the paint-wait path (measured 86s vs 30s on a 3,245-frame rAF comp). Validation: - Happy path W3: verify samples pass across workers (4x inf on the 2,381f comp), output vs single-worker DE = 59.3dB (encode noise floor) — the interleave + dedup-stride produce identical pixels. - Verify-trip (marginal comp + HF_DE_VERIFY_MIN_DB=45): fails at frame 649 (32.2dB < 45), peers abort instead of deadlocking, whole render retries via parallel screenshot, RENDER_OK in 42.6s. - Canary suite 7/7 with the flag off (default paths untouched); producer orchestrator tests 99/99; engine suite 909 passed (14 pre-existing main failures, stash-A/B verified); reorder-buffer abort unit tests. Perf note: capture-only parallel speedup measured 1.38x (W2) / 1.52x (W3) over single-worker DE in the spike; end-to-end numbers on this machine are currently noisy (separate-browser init overhead + bench load) — clean benchmarks before any default routing change. The flag stays explicit opt-in; promoting it into the router replaces the #2026 W=1 pin for the same cohort. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inframe capture mode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
clamp:parallelexists because parallel capture had no runtime self-verification. This PR builds the verified path: multi-worker drawElement through the streaming encoder with the full safety net riding along — opt-in viaHF_DE_PARALLEL_STREAM=true, default routing (including the #2026 inversion) untouched.Mechanism
distributeFramesInterleaved,WorkerTask.frameStride): worker i takes framesi, i+N, i+2N…— seek-based capture makes stride free, and the ordered writer's reorder window shrinks fromtotalFrames/NtoN. (Contiguous chunks serialize workers behind the writer — the original stall.)createDrainFrameGuard(session-parameterized) — every parallel frame gets the same blank-guard + PSNR self-verify as the sequential drain, against its owning worker's pre-injection ground truth (all sessions arm identical sample indices).FrameReorderBuffer.abort(err): a failed worker rejects all parked/future waiters. Without it, a verification failure in one worker deadlocks its peers forever (the pool awaits ALL workers before surfacing errors) — found by the verify-trip test, unit-tested.DrawElementVerificationErrorsurvives the pool's error-string flattening so the orchestrator's verify-retry fires.lastEncodeResultreuse now requires every frame in(lastEncodeResultFrame, i]to be predicted-static — sequential behavior unchanged, stride-N reuse safe.Validation
HF_DE_VERIFY_MIN_DB=45): fails at frame 649 (32.2 dB), peers abort instead of deadlocking, render retries via parallel screenshot, RENDER_OK in 42.6s.Perf — clean benchmark complete, VERDICT: promote
36-run benchmark (4 comps × {single, par2, par3} × 3 reps, interleaved order) on a quiet machine (load1 held 4–6 throughout, hard-fail load gate, 0 gate timeouts). 4/4 verify + 0 fallback on every run, max within-cell drift 8.3% (no monotonic drift — confirms the machine was actually quiet).
0768f038(GSAP graphics)398c0655(rAF, high-variance)07856d80(39% static/dedup-heavy)13ce38c6(915f crossover probe)2/3 comps ≥2,000 frames clear the pre-agreed 1.25× bar on totals with spread <20%; the dedup-heavy comp doesn't (1.16×), which is mechanically expected — static-frame dedup already skips the capture work parallelism would split. No comp anywhere shows par3 < single; par2 is monotonically weaker than par3 on every comp (real parallel scaling, not noise).
Merging this as opt-in (
HF_DE_PARALLEL_STREAM=true, default routing unchanged). Next: a separate default-off router PR preferring verified parallel DE over the #2026 single-worker inversion above a measured frame threshold, then a 2-week telemetry soak (revert rate ≤2%,de_verify_min_dbp10 stable) before any default flip.🤖 Generated with Claude Code