Build your first eval
An eval is a dataset plus a grader plus a bar. That's it — and each of the three is a place where testers' instincts matter more than ML knowledge. Let's build one properly.
8 steps · 6 minThe anatomy
Three parts. The DATASET: examples of real inputs, each with a checkable expectation. The GRADER: something that scores each output against that expectation. The BAR: the pass rate you require before shipping. Weak evals fail at one of these three — usually the dataset.
Datasets are edge-case work
A good eval set looks like your test-case instincts, written as data: happy paths for baseline, boundaries where behaviour should flip, traps where the RIGHT answer is refusing ('no deadline in this text — suggest nothing'), and real failures harvested from production. Twenty diverse examples beat two hundred easy ones.
The trap cases matter most — they’re the only way to catch a model that confidently invents answers.
Which example adds the MOST value to an eval set for “AI suggests a reminder time from task text”?
Graders, cheapest first
Grade with the dumbest thing that works. Exact/contains checks for constrained outputs. CODE checks for properties: 'suggested time < deadline', 'response is valid JSON', 'no email addresses present'. Only when quality is genuinely subjective — tone, helpfulness — do you reach for a rubric or an LLM judge.
Code-checkable properties hide everywhere: length limits, required fields, forbidden content, ordering. Hunt those before writing any rubric.
“The AI reply must never promise a refund.” What’s the right grader?
Setting the bar
The bar is a product decision wearing a number. 99.9% for 'never promise refunds' (and a guardrail besides). Maybe 85% for 'suggestion is genuinely helpful'. The bar encodes how much failure this feature can afford — which depends on who's hurt when it misses, not on what feels impressive.
Your eval runs on every prompt change. This week: 91%, last week: 96%, bar: 90%. Ship the prompt change?
The tester’s edge
Notice what needed ML expertise here: nothing. Dataset design is edge-case thinking. Grader choice is the automation pyramid. The bar is severity judgment. Evals are a testing discipline that happens to point at a model — which is why testers who learn them get very valuable, very fast.
Next: what happens when the grader itself is a model — and how it lies to you.
Lesson complete.
Dataset from real failures, the dumbest grader that works, a bar with a reason. You can now build the instrument — next lesson, you’ll learn to distrust it properly.