Why a $100M model is, underneath, a pile of human judgments — and why that makes your job the important one
Day 1 of 66
A large language model is not programmed. Nobody writes the rules for "be helpful" or "this image matches the prompt." Instead, the model's behavior is learned from examples that humans produced or judged. Change the data, and you change the model. That is the entire premise of your future job — and of this track.
Modern AI is data-centric. Once a lab has a strong model architecture, the fastest way to make the model better is almost never to change the model — it's to improve the data it learns from. The people who design, collect, and quality-control that data are the bottleneck on model quality. A Human Data Lead owns that bottleneck.
You already live inside this loop. Every time you rate a Grok image, rewrite a weak caption, or flag a video that doesn't match its prompt, you are manufacturing the raw material a model trains on. Today we zoom out so you can see the whole factory — not just your station on the line.
Almost everything an assistant like Grok or ChatGPT does traces back to three kinds of data, layered in order. Keep this mental model; we'll refer to it for 12 weeks.
The model first reads a huge slice of the internet, books, and code, learning to predict the next token. This gives it raw knowledge and language ability but no manners — a base model will happily continue "How do I make a bomb?" because it's just predicting plausible text. Pretraining is mostly not human-annotated; it's scraped and filtered. Humans matter here for data curation and filtering, not labeling.
Next, humans write demonstrations: ideal answers to prompts. "Here's a good response to this request." The model imitates them. This is where a base model becomes an assistant. SFT data is expensive because a person has to author the gold-standard answer. This is human data.
Finally, humans compare outputs: "Response A is better than Response B." Those comparisons train a reward model, which then steers the main model via reinforcement learning (RLHF). This is how models get aligned to nuanced human taste — helpfulness, tone, harmlessness. This is the human data you produce most often as a tutor.
The famous InstructGPT result (OpenAI, 2022) made this undeniable: a 1.3-billion-parameter model trained with human feedback was preferred by people over the 175-billion-parameter base model — 100× larger — that lacked it. The human data mattered more than 100× the size.
Pretraining gives the model knowledge. SFT gives it behavior. Preference/RLHF gives it judgment. Humans are the source of the last two — and increasingly of the quality filtering on the first.
As an AI image/video tutor at xAI, most of your output feeds stages 2 and 3 — and, because it's multimodal, some of the hardest, least-solved parts of them:
| When you… | You're producing… | The model learns… |
|---|---|---|
| Write or fix an ideal caption / response | SFT demonstration | how to do the task |
| Pick the better of two generated images | Preference comparison | which output is better |
| Score one output on a quality scale | Evaluation / rating data | how good it currently is |
| Flag an unsafe or policy-violating output | Safety / red-team data | what not to do |
Here's the leap this track is about: a tutor executes one of these well. A lead designs all of them — decides which kind of data the model needs next, writes the rubric the tutors follow, sets how many comparisons get collected, measures whether the resulting data is any good, and reports it upward. Same domain, different altitude.
Andrew Ng popularized the framing that an AI system is Code + Data, and that for most teams the data half is where the wins are. His often-quoted figure: roughly 99 of 100 research papers focus on models, but in practice systematically improving the data is what moves accuracy. In one example, a data-centric approach improved a defect-detection task by ~17% while model tweaks added 0%.
The skills that make models better — clean rubrics, high inter-annotator agreement, well-designed preference collection, honest evals — are human-data skills, not modeling skills. You don't need to train a neural network to be the person who makes the model better. That's the door you're walking through.
The full curated, verified resource list for this week is at the bottom of the page — but the two above are today's required watch/read.
A tutor produces one kind of data well. A lead decides which kind the model needs next, and can defend the call — more SFT demonstrations won't fix a taste problem that needs preference data. The altitude jump is from "I label accurately" to "I choose what gets labeled, and why."
Say this in an interview: "I don't just rate outputs — I think about which data product the model actually needs next, and what each label teaches the reward model downstream. That's the decision that moves the metric, and it's the one I want to own."