Pick one real model-quality problem and turn it into a director-grade goal, a measurable metric, and a map of every artifact you've already built
Day 56 of 66
For eleven weeks you built parts: a rubric, an agreement method, a preference dataset, a multimodal rubric, an eval suite, a pandas analysis, a pipeline design, a QA audit, a calibration playbook, a safety taxonomy. This week you assemble them into a single object — a data program you could hand to a director and a team and say "run this." Today is the foundation: scope it.
Scoping is not "pick a topic." It's deciding, on purpose, what problem you will solve, how you'll know you solved it, what data that requires, and what you will deliberately not do. A program with a fuzzy scope spends its budget everywhere and moves nothing. A scoped program points every artifact at one measurable target.
Choose a single, concrete model-quality problem you could plausibly be handed at xAI. Resist the urge to fix everything. The best capstone problems are narrow, measurable, and tied to a real Grok capability you understand. A strong default:
Raise Grok's image prompt-adherence pass-rate from 72% to 85% in 6 weeks. It's specific (prompt adherence, not "quality"), it has a baseline (72%), a target (85%), and a deadline (6 weeks). Use this one, or swap in your own with the same shape — e.g. "cut over-refusals on benign image edits by half," or "raise video temporal-consistency pass-rate 10 points."
Pick something where you genuinely know the failure modes — that's where your tutoring background becomes leadership credibility. You've seen the adherence failures; that's why your program will target the right data.
A goal a director can approve is one sentence with a number and a date. A success metric is how you'll prove you hit it — and it must be measurable without asking anyone's opinion.
The goal is the outcome you want: "85% prompt-adherence pass-rate." The metric is the instrument that reads the goal: "weekly gold-set accuracy on a held-out adherence eval, plus human-eval win-rate vs. the current model." A goal without a metric is a wish. A metric without a goal is busywork.
It runs the same way every week (so you can see a trend), it's measured on data the program never trained on (so it can't be gamed — this is the contamination lesson from Week 6), and it moves only when the model actually improves. If your metric can go up while the model gets worse, it's the wrong metric.
"Success = weekly gold-set accuracy on a held-out prompt-adherence eval reaches 85%, confirmed by a human-eval win-rate above 60% against the current model." Two readings — one cheap and automated (the gold set, run weekly), one expensive and trusted (the human eval, run at milestones). The cheap one watches the trend; the expensive one is the verdict.
Now work backward from the metric. To move prompt adherence, what human data must you produce? Be specific about kind, volume, and target.
If your data list has more than four kinds of data, you're scoping a department, not a program. Cut to the two or three that most directly move the metric. The discipline of saying "not this, not now" is the lead skill that separates a plan from a wishlist.
This is the move that makes the capstone a portfolio and not an essay. Every part of the program should point at something you already built. Fill this map for your chosen problem:
| Program part | Reuses the artifact from… | What it contributes |
|---|---|---|
| Quality definition / scoring | Rubric (Week 2) + multimodal rubric (Week 5) | what "adherence pass" means, in writing |
| Agreement / gold sets | Agreement methods (Week 3) | proof the labels are trustworthy |
| Preference dataset | Preference pairs (Week 4) | the training fuel, targeted at failures |
| Evaluation | Eval suite + LLM-judge (Week 6) | the held-out metric that proves it worked |
| Analysis | pandas analysis (Week 7) | finding the failure clusters in the data |
| Flow / sampling / gold injection | Pipeline design (Week 8) | how items get labeled at quality |
| Ongoing quality | QA / drift audit (Week 9) | the weekly heartbeat + drift alerts |
| Team operation | Calibration + leadership playbook (Week 10) | how the people stay aligned |
| Safety gate | Safety taxonomy / triage (Week 11) | the harm floor the push must not cross |
When this table is full, you have proven something quietly powerful: the program isn't a fantasy, it's an assembly. Every claim has an artifact behind it. That's the difference between someone who has read about running a data operation and someone who has the pieces on the bench.
End today with a single crisp paragraph — the kind a director skims in fifteen seconds and approves. It should answer: what's broken, by how much, what you'll do, and how you'll know it worked.
A tutor is handed a goal and produces data against it. A lead writes the goal: turns "Grok's adherence feels off" into "72% to 85% in six weeks, read on a held-out gold set," then names the two or three data products that move that number and the dozen they'll deliberately not chase.
Say this in an interview: "I don't start from 'collect more data.' I scope one measurable problem with a baseline and a date, then work backward to the smallest data list that moves it — and I'm just as clear about what I'm choosing not to do."