Why CineQC scores the way it does
How do you judge a video that was never filmed? CineQC is a broadcast QC discipline translated for AI-generated video. It was written by Marc Warfield — two-time Sports Emmy-winning editor, Emmy judge since 2017, with 20+ years cutting elite sports for Olympic Channel, ESPN, Sky Sports, and BT Sport — now applying that eye to AI video evaluation, RLHF, and human-in-the-loop feedback.
Everything described here runs 100% client-side. Clips are decoded and scored in the browser — no upload, no server, no account. That's deliberate: evaluators at frontier labs often work with unreleased model outputs under NDA, and a QC tool is only usable there if nothing ever leaves the machine.
The problem this rubric solves
AI video evaluation mostly happens by vibes — "this one looks better." Broadcast QC solved that problem decades ago: measurable signals are measured, subjective calls are scored against anchored definitions, and every verdict leaves a paper trail. CineQC applies that discipline to generated video.
Measure what's measurable
Flicker, stutter, drift, sharpness, motion coherence, structure, and loudness are signals, not opinions. Machines score them per frame.
Anchor what's subjective
Anatomy, physics, text, continuity, composition, and prompt adherence get 1–5 scales with written anchors at every step, so two reviewers land on the same number.
Weight for the delivery target
A broadcast master and a social clip fail differently. Weight presets re-blend the same scores for the destination that matters.
Export evidence, not vibes
Every pass ends in a defensible artifact: JSON, a PDF scorecard, a shareable report — or RLHF preference pairs for training pipelines.
The standards behind the numbers
Where a published standard exists, CineQC implements it. Where none exists — most of AI-video failure — the thresholds come from broadcast delivery practice.
EBU R128 / ITU-R BS.1770-4
Audio loudness is measured as integrated LUFS to the same standard broadcasters deliver against — not peak level, not a guess.
Per-shot measurement
Scene cuts are detected first and every metric resets per shot, so an edit never reads as flicker or drift. Standard practice in file-based QC.
Anchored 1–5 scales
Each rubric step has a written definition — the method behind repeatable subjective scoring, and the backbone of inter-rater reliability.
Composite verdict
Scored categories blend into 0–100: 80+ is PASS, 60–79 CONDITIONAL, below 60 FAIL. Letter grades follow the familiar 90/80/70/60 breaks.
Broadcast Delivery preset
Temporal stability weighted 1.4, continuity 1.3, anatomy, text, and composition 1.2 — instability and continuity breaks are what get masters rejected.
Research Eval preset
Anatomy, physics, and prompt adherence weighted 1.4 — the failure modes that matter when the question is "how good is this model," not "can this air."
The automated pass is deliberately honest about its limits: no algorithm here claims to judge anatomy, physics, or meaning. Those calls belong to the human pass — automation's job is to make them faster and better-evidenced.
Seven categories, anchored at both ends
The full anchor text ships in the tool. The scale's extremes show how each category is defined — a 5 is not "looks good," it's a specific claim.
- Temporal Stability — 1: severe flicker, pumping, or judder throughout · 5: rock-steady exposure and motion cadence. Pre-filled from the automated pass; the eye can override.
- Anatomical / Object Integrity — 1: broken anatomy in most frames · 5: anatomy and objects hold up under scrutiny.
- Physics Plausibility — 1: floating, impossible collisions, warping · 5: motion and dynamics read as physically real.
- Text & Logo Integrity — 1: text garbled or invented throughout · 5: all text and logos crisp and accurate.
- Continuity — 1: lighting, props, and shadows change constantly · 5: consistent lighting, shadow, and props across the shot.
- Composition & Framing — 1: broken horizons, cut-off subjects · 5: deliberate, broadcast-clean composition.
- Prompt Adherence (optional) — 1: ignores the prompt · 5: faithfully realizes it.
An Emmy jury doesn't score "I liked it." It scores against criteria, and so does every usable human-feedback pipeline. Anchored scales are what make one evaluator's 4 mean the same thing as another's — the difference between opinion and data.
From QC report to training data
A scored evaluation is also a preference judgment. CineQC exports A/B comparisons and batch runs as RLHF-ready JSONL — one {chosen, rejected} pair per line, with scores, margin, and rationale attached.
Preference pairs
The higher-scoring clip is chosen, with per-metric scores for both sides, the winning margin, and the primary reason it won — the format preference-tuning pipelines consume.
Confidence-gated
Pairs are labeled high (margin > 20), medium (10–20), or low. Low-margin pairs are excluded from batch export — a near-tie teaches a model nothing.
Limitations, stated plainly
CineQC does not claim to replace human judgment. It automates what machines measure well — luma, motion, color, loudness — and structures what humans judge best: anatomy, physics, meaning. The composite grade is a starting point, not a verdict. The final call is always the evaluator's.
Fast mode trades accuracy for speed (~3× faster) by sampling fewer frames — built for triage, not final delivery assessment. Precision mode goes the other way: it steps the clip seek-by-seek so every source frame is captured regardless of machine load, and the same clip always produces the same scores. Broadcast-destined content gets the precision pass.
Trust, but verify.
The tool is the methodology, running. Bring a clip and watch it happen. For evaluation partnerships, methodology consulting, or training-pipeline integration: marc.warfield@dynamicvibe.net
Open CineQC Read the user guide