Turn your next real shift into a field study — and start seeing quality problems the way a lead does
Day 4 of 66
When a model underperforms, there are two instincts. The model-centric instinct says "change the model — more parameters, new architecture, more training." The data-centric instinct says "the model is fine; the data is noisy, inconsistent, or missing cases — fix that." Andrew Ng's central argument is that for most mature systems, the data-centric path is faster, cheaper, and the one most teams neglect.
Imagine two tutors scoring the same images. If they systematically disagree about what "good prompt adherence" means, the model receives contradictory signal — for the same image it's told both "good" and "bad." The model can't learn a clean boundary from noisy labels; it averages the confusion. Ng's research showed that tightening label consistency often beats both adding data and changing the model. This is why your Week 2 (rubrics) and Week 3 (agreement) exist: consistency is a lever you control.
When someone says "the model is bad at X," a data lead doesn't reach for the model. They ask: What data taught it X? Was that data consistent? Did the rubric cover the cases it's failing on? Is our eval even measuring X honestly? Those four questions are your job in a sentence.
This is a "Field Day" — the work happens during your actual xAI shift. You're going to do your normal tutoring, but with a second layer of attention running. You're observing the system, not just doing the task.
Record your observations in general terms in your own notebook — the mechanics and your reasoning, not confidential prompts, customer data, or anything under NDA. The transferable skill is the habit of systems observation, and that's fully yours to keep.
A tutor proposes one guideline fix. A lead turns "the model is bad at X" into a data investigation: quantify the inconsistency, trace it to the rubric clause that allows it, ship the fix, and report the lift as a number. The reflex becomes a repeatable diagnostic the team runs — not a one-off observation.
Say this in an interview: "When someone says 'the model is bad at X,' I don't reach for the model. I ask what data taught X, whether it was consistent, whether the rubric covered the failing cases, and whether the eval even measures X honestly — and then I fix the data and show the lift."