Build the mental model in code: classify your real xAI tasks by the training data they produce
Day 3 of 66
Because the fastest way to lock in a mental model is to make it operate on your own real work. Today you'll write a tiny Python tool that takes tasks you've actually done as a tutor and classifies each one by which kind of training data it produces. By the end you'll never look at a task the same way — you'll see the model behind it.
This script only uses input(), a dictionary, and a loop. If you've never run Python: install it from python.org, save the code as taxonomy.py, open a terminal in that folder, and run python taxonomy.py. The real exercise is the thinking, not the syntax — Week 7 makes you fluent in the data stack.
Every human-data task you'll ever be handed is fundamentally one of these. Memorize the distinction — it's the spine of the whole role.
| Product | The human action | Trains… | Cost |
|---|---|---|---|
| SFT demonstration | Author the ideal answer | the model's behavior | High (you create) |
| Preference comparison | Rank/choose among outputs | the reward model | Medium (you judge) |
| Evaluation rating | Score against a rubric | nothing — it measures | Medium |
| Red-team / safety | Provoke & flag failures | what to refuse / avoid | High (skilled, taxing) |
Preference data and evaluation data look identical from the chair — you're judging outputs — but they serve opposite purposes. Preference data trains the model (it goes into the reward model). Evaluation data measures the model (it must be held out and never trained on). Mixing them up — letting eval data leak into training — is called contamination, and it's one of the costliest mistakes a data org can make. We'll dedicate Week 6 to it.
The "Try This — On Real Data" box below (injected for build days) contains the full taxonomy.py. Type it in — don't paste — so you read every line. Then run it against five tasks you genuinely did at xAI in the last month.
As you classify each, write a second note: "If I did this task carelessly, the model would end up ___." That sentence is the difference between a labeler and a lead — a lead always knows the downstream consequence of a label.
Look at your five results. If four of them are the same product (likely "preference" or "evaluation"), you've just found your current lane — and the three other products are the capability gap this track closes. Screenshot the output; it's the first entry in your portfolio.
Knowing a label's downstream consequence is the floor. The lead move is designing the task so the whole team can't produce the wrong product by accident — keeping preference and evaluation data in separate pipelines so contamination is structurally impossible, not just discouraged. You engineer the mistake out instead of policing it.
Say this in an interview: "I keep preference and eval data in separate pipelines by design, so eval can never leak into training. Contamination is one of the costliest mistakes a data org can make, so I make it structurally impossible rather than relying on people to remember."