Lesson 3 of 6
Evals as a product concern
7 min read
You ship a shiny new model version. The demo answers are gorgeous. A week later, support is on fire — it quietly got worse at the boring cases nobody demoed. How could you have known?
The demo lies; the test set doesn't
A demo shows you the best case. Real users hit the boring, awkward, edge cases — and that's exactly where a change can silently break things. The fix is an eval: a fixed set of real test cases with known-good answers that you run the product against, so you can measure quality instead of guessing from a good-looking demo.
An eval turns 'it feels better' into a number. It's how you catch a regression — a change that quietly makes some cases worse — before your users do.
Run it on every change
Evals earn their keep when they're automatic: every time you change the model, the prompt, or a setting, re-run the test set and compare to the last version. If three cases that used to pass now fail, you see it before you ship — not from an angry customer. Evals aren't an engineering nicety; they're the product's quality gate.
Ship changes through the eval, not around it. 'The demo looked great' is not a release criterion.
Keep your test set honest. Fill it with the real, hard, boring cases your users actually send — not the flattering ones that make the score look good. An eval you can't fail teaches you nothing.
The shape of it
- —A demo shows the best case; an eval measures the real ones.
- —Build a fixed test set of hard, real cases with known-good answers.
- —Re-run it on every change and compare versions to catch regressions early.
Before shipping a new model version, how do you know it didn't quietly get worse?
Continue in the app
Take the whole Measuring & Governing AI Products course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.