Week 1 of 12 · Foundations

Synthesis & the Judgment Ritual

Lock in Week 1, prove it with a self-quiz, and adopt the one habit you'll use every day for the rest of the track

Day 5 ~60 minutes Review

Day 5 of 66

What you now know

One week in, you can already see the machine most people who use AI never see. Say this back without looking:

The Week 1 spine

Models learn behavior from human data; improving data is how you improve the model (data-centric AI).
Three layers: pretraining (knowledge) → SFT (behavior) → preference/RLHF (judgment).
The recipe: SFT → reward model from comparisons → RLHF; comparisons train the grader, so their quality compounds.
Four data products: SFT, preference, evaluation, red-team — and preference ≠ evaluation (don't contaminate).
Know the names DPO and Constitutional AI / RLAIF and what they change about data collection.

Self-quiz (no notes)

Answer out loud or in writing. If any answer is shaky, reread that day before moving to Week 2.

A model is great at facts but rude and unsafe. Which training stage is responsible, and what data fixes it?
You rank four images best-to-worst. What does that data train — the assistant, or something else?
Why is it dangerous to train on your evaluation set?
A teammate says "let's just use a bigger model." Give the data-centric counter-question.
In one sentence: what does a Human Data Lead do that a tutor doesn't?

Answer key (peek after you try)

1) Pretraining gave knowledge; SFT + RLHF give behavior/safety — fix with demonstrations + preference/safety data. 2) It trains the reward model (the grader), not the assistant directly. 3) It's contamination — your measurement is no longer honest; you can't tell if the model truly improved. 4) "What data is the model learning X from, and is it consistent and well-specified?" 5) A lead designs and measures the data (rubrics, collection, quality, reporting); a tutor produces it.

The Judgment Ritual

This is the habit you'll carry through all 66 days — and into the job. Every later lesson links back here. When you face an ambiguous labeling or judgment call, do not guess silently. Run the ritual:

The Ritual — memorize it

Check the guideline. Is this case actually covered? Re-read the relevant rule and examples.
Check the gold examples. Is there a labeled exemplar close to this case? Match against it, not your gut.
If still unclear — flag, don't guess. Route it to adjudication and label it as ambiguous. A guessed label is invisible noise; a flagged one is a fixable signal.
Write down the ambiguity. One line: "Case X is unclear because the guideline doesn't say Y." This note is how guidelines improve.

Why this is leadership, not box-ticking

Disagreement is data. Every ambiguous case you document is a precise pointer to a hole in the guideline — and fixing guidelines is the highest-leverage thing a data lead does (Week 2). The tutor who silently guesses produces noise. The one who flags and writes it down produces the roadmap for improving the entire operation. Be the second one.

The lead move

The Judgment Ritual makes you reliable. The lead version makes the team reliable: you own the loop that turns every flagged ambiguity into a guideline revision, then proves agreement climbing afterward. Disagreement stops being friction and becomes your roadmap for improving the whole operation.

Say this in an interview: "I treat disagreement as data. Every flagged ambiguous case is a precise pointer to a hole in the guideline, and closing those holes is the highest-leverage work I do — I can show inter-annotator agreement climbing as I do it."

Close the week

Wrap-up

Before Week 2

Self-quiz answers all solid (or reread the weak day)
Week 1 summary written in your own words
The Judgment Ritual copied somewhere you'll see it during work
One question you most want this track to answer, written down

Up next — Week 2: Rubrics & Annotation Guidelines

You'll write a real rubric for an image/video quality task you already know — weighted dimensions, a safety gate, worked examples, edge cases — and turn it into runnable scoring code. It's your first portfolio artifact. (Lesson pages for Week 2 are being authored at the same gold-standard as this week — the dashboard will light them up as they ship.)

Week 1 Takeaways

You can map any task to the training loop and name its downstream effect.
You know the canonical recipe and the modern variants by name.
You have the Judgment Ritual — your daily operating habit.
You've started thinking data-centric: fix the data, not the model.