Applied Creative AI Engineering — evidence

Temporal detector versus heuristics

A learned detector trained on a local RTX 4080, benchmarked against hand-written measurements on identical held-out clips. The heuristics win.

Bounded learned signals in CineQC

Adding an optional model to a shipped tool so that every failure path leaves the existing behaviour untouched.

Reproducible media tooling

Builds a CLI that turns a media file into an exact JSON description plus derived artifacts, treating external binaries as a trust boundary.

• capstones/media-probe/README.md
• capstones/media-probe/src/media_probe/probe.py

• Reads the first video stream only; multi-stream and multi-track files are out of scope.
• Sampling near the tail clamps to the last decodable frame, so the returned frame is the closest one rather than an exact timestamp match.

Measured image and video quality signals

Implements five artifact metrics that separate controlled defects from clean footage by 62-80 points, each publishing its raw signal, per-step series, normalisation constant, and threshold.

• artifact-lab.html
• data/artifact-lab-report.json
• capstones/artifact-lab/README.md

• Measured on 48 controlled synthetic clips where exactly one property varies at a time; not an estimate of field accuracy.
• No numerical agreement with browser CineQC is claimed, because the two have never been run against a shared fixture set.

Dataset design and leakage control

Designs a grouped split by scene so overlapping windows and shared backgrounds cannot leak across train, validation, and held-out sets, and proves the property with tests rather than convention.

• capstones/temporal-detector/DATASET_CARD.md
• labs/fixtures/manifest.json

• Eight synthetic scenes. A held-out result rests on two scenes and twelve clips; differences of a few points are noise.
• Class balance is 5:1 towards defects, the opposite of most production footage.

Training, evaluation, and honest benchmarking

Trains a PyTorch detector on a real local GPU with a recorded environment, and benchmarks it against a heuristic baseline on identical held-out clips - reporting that the heuristics win.

• case-study-temporal-detector.html
• capstones/temporal-detector/MODEL_CARD.md
• capstones/temporal-detector/runs/local-rtx4080/run.json
• capstones/temporal-detector/runs/local-rtx4080/predictions.json
• capstones/temporal-detector/runs/local-rtx4080/benchmark/benchmark.json

• The learned detector loses to the heuristics on this fixture set: F1 0.90 against 1.00 on twelve held-out clips.
• The model has never seen a frame of real footage, and does not name which defect it found.
• Hyperparameters were chosen on validation only; validation F1 plateaued at 0.926 across every setting tried.

Scoped decision-making under constraint

Selects one semantic quality target from three candidates by a rule written before the answer, rejecting the highest-value candidate on lawful-example grounds and recording why.

• capstones/semantic-evaluator/selection-matrix.json
• capstones/semantic-evaluator/MODEL_SYSTEM_CARD.md
• capstones/semantic-evaluator/results/metrics.json

• The evaluator only reads the alphabet it was given; on an unfamiliar typeface it will call correct text garbled.
• Its single held-out error is a fixture labelling ambiguity, not a detector failure.
• Nothing here says anything about text in real generated media.

Bounded integration of a model into production code

Adds optional learned signals to a shipped tool behind a default-off flag, where every failure path preserves the existing heuristics and routes to human review.

• case-study-cineqc-hybrid.html
• capstones/cineqc-hybrid-api/README.md

• The service's two analysers are stand-ins with the real models' signature; the trained checkpoints are not yet served.
• The service has no authentication or rate limiting and is not deployed.
• The browser sends only the luma timeline, which is why semantic confidence stays low enough to route to review.