Milestone 4 of 12

Pixels, arrays, shapes, and dtype

Make every frame transformation legible.

Frame contract

shapedtype + colorsafe array

A frame is not “an image” to Python. It is a rectangular array with an axis order, numeric type, value range, color order, and memory relationship. This milestone makes that contract visible before you build a detector.

Required route: diagnostic and model (45 min), NumPy core (120 min), sandbox lab (80 min), failure clinic (35 min), CineQC transfer (35 min), evidence and summary (45 min). Total: 6 hours, including the selected NumPy reading. Optional references sit outside that total.

Entry diagnostic

Predict every printed shape and dtype before running the block. Then explain why selecting one pixel removes two axes while selecting one channel preserves height and width.

Runtime: Browser lab · Pyodide with NumPy · 10 min.

import numpy as np

frame = np.zeros((1080, 1920, 3), dtype=np.uint8)
print(frame.shape, frame.dtype)
print(frame[20, 30].shape)
print(frame[:, :, 0].shape)

The answers are (1080, 1920, 3) uint8, (3,), and (1080, 1920).

Self-check

If frames.shape is (48, 1080, 1920, 3), which axis is time, and what shape does frames[0] have?

Why this matters in CineQC

A detector can run without crashing while reading the wrong channel, overflowing uint8, or comparing CHW data as HWC. Those are silent evidence failures. CineQC needs each transformation to declare five things: shape, axis meaning, dtype, numeric range, and color order.

Print those facts at boundaries—after decode, before a metric, before a model, and before save. Shape-printing is not beginner scaffolding; it is an audit trail.

Visual model — T × H × W × C

Tensor axis cards show batch, channel, height, and width in a 32 by 3 by 224 by 224 shape.

Use the axis card:

• one image: H × W × C, normally (height, width, 3);
• a clip batch: T × H × W × C;
• a PyTorch image: C × H × W after transpose(2, 0, 1);
• Pillow uses RGB; OpenCV decode commonly returns BGR;
• uint8 usually means 0..255; model floats commonly mean 0.0..1.0.

Tiny arrays make values inspectable; frame-sized arrays prove the operation scales.

Runtime: Browser lab · Pyodide with NumPy · 15 min.

import numpy as np

tiny = np.array([[[255, 0, 0], [0, 255, 0]]], dtype=np.uint8)
frame = np.zeros((1080, 1920, 3), dtype=np.uint8)
print("tiny", tiny.shape, tiny[0, 0].tolist())
print("frame", frame.shape, frame.nbytes)

Create, index, and name axes

np.array preserves supplied values; zeros, ones, and full create predictable test frames. Indexing fixes an axis; slicing keeps it. Always pair an operation with an expected shape.

Runtime: Browser lab · Pyodide with NumPy · 15 min.

import numpy as np

patch = np.arange(24, dtype=np.uint8).reshape(2, 4, 3)
row = patch[0]          # (4, 3)
crop = patch[:, 1:3]   # (2, 2, 3)
red = patch[..., 0]    # (2, 4)
print(patch.shape, row.shape, crop.shape, red.shape)

The ellipsis means “all preceding axes.” It makes the final channel selection explicit.

Broadcasting and dtype

Broadcasting aligns dimensions from the right. A three-value color offset can apply to every pixel because (3,) aligns with the channel axis of (H, W, 3). Convert before arithmetic: uint8(250) + uint8(20) can wrap instead of producing 270.

Runtime: Browser lab · Pyodide with NumPy · 20 min.

import numpy as np

pixel = np.array([250, 10, 100], dtype=np.uint8)
offset = np.array([20, -5, 10], dtype=np.int16)
safe = np.clip(pixel.astype(np.int16) + offset, 0, 255).astype(np.uint8)
print(safe.tolist(), safe.dtype)

The expected result is [255, 5, 110], not a wrapped red value.

Self-check

Why can an array shaped (3,) broadcast across (1080, 1920, 3), while (1080,) cannot?

Copies, views, and color order

Basic slices are usually views: they share memory with the source. Use .copy() before annotating a crop that must not alter its decoded frame. Fancy indexing usually returns a copy, but relying on memory folklore is fragile; test with np.shares_memory.

Runtime: Browser lab · Pyodide with NumPy · 20 min.

import numpy as np

rgb = np.array([[[255, 20, 5]]], dtype=np.uint8)
crop = rgb[:, :, :]
safe_crop = crop.copy()
bgr = rgb[..., ::-1].copy()
chw = np.transpose(rgb, (2, 0, 1))
print(np.shares_memory(rgb, crop), np.shares_memory(rgb, safe_crop))
print(bgr[0, 0].tolist(), chw.shape)

RGB [255, 20, 5] becomes BGR [5, 20, 255]; CHW becomes (3, 1, 1).

Normalize and restore deliberately

Normalize by converting to float first. Restore by clipping, scaling, and rounding; write the convention into the function name or docstring.

Runtime: Local Python 3.12 · NumPy 2.x · 15 min.

import numpy as np

frame_u8 = np.full((1080, 1920, 3), 128, dtype=np.uint8)
frame_01 = frame_u8.astype(np.float32) / 255.0
restored = np.rint(np.clip(frame_01, 0.0, 1.0) * 255).astype(np.uint8)
print(frame_01.shape, frame_01.dtype, float(frame_01.min()), float(frame_01.max()))
print(np.array_equal(frame_u8, restored))

Sandbox lab

Use the shared browser Python runner as a scratch editor: NumPy is supported, but there is not yet a guided M04 exercise pack. Paste the blocks above one at a time. Build a 4 × 6 × 3 synthetic frame, crop it, convert HWC to CHW and back, then record every shape. Repeat on a (1080, 1920, 3) zero frame to prove the contract is frame-sized.

Failure clinic

The following block intentionally demonstrates overflow and source mutation. Its wrong outputs are the lesson; repair with a wider dtype and .copy().

Runtime: Browser lab · Pyodide with NumPy · expected wrong values, then repair · 20 min.

import numpy as np

source = np.array([[[250, 20, 5]]], dtype=np.uint8)
view = source[:, :, :]
view[0, 0, 0] += np.uint8(10)
print(source[0, 0].tolist())  # wrapped red and mutated source

Also diagnose the classic blue-face symptom: RGB data was interpreted as BGR, or vice versa. Do not “fix” it by swapping until the decode and save contracts are named.

CineQC transfer

Add a boundary log to one existing metric: name, shape, dtype, min, max, and color_order. Assert three channels before a color metric and two dimensions before a grayscale metric. Store raw scores before rounding so calibration later uses evidence, not display values.

Self-check

A slice changes when its source is annotated. Which single method makes ownership explicit, and how would you prove memory is no longer shared?

Evidence gate

Submit an array notebook or script with two paired cases: a tiny inspectable array and a frame-sized array.

1. Annotated transitions: create, crop, broadcast, RGB/BGR swap, HWC/CHW transpose, normalize, and restore; print shape, dtype, range, and color order after each step.
2. Safe arithmetic: demonstrate the uint8 overflow, then repair it with conversion and clipping.
3. Memory contract: demonstrate a mutating view and a safe copy, using np.shares_memory as proof.

The proof passes when another engineer can predict each intermediate array without opening an image viewer.

One-page summary

• Name every axis: time, height, width, channel.
• Pair tiny arrays with real frame-sized arrays.
• Indexing can remove axes; slicing preserves them.
• Broadcasting aligns dimensions from the right.
• Convert before arithmetic; clip before returning to uint8.
• HWC/CHW and RGB/BGR are contracts, not cosmetic choices.
• Slices may share memory; copy before isolated mutation.