Lesson 2 of 6
The usefulness threshold
7 min read
The most powerful model scores highest on every benchmark. It's also the slowest and the most expensive. Do you ship it?
Good enough, on purpose
Every AI feature has a usefulness threshold — the bar where the output is good enough to do the job. Above the bar, more quality is often invisible to the user; below it, the feature is useless no matter how cheap. The product question is never 'which model is best?' It's 'which is the cheapest, fastest model that clears the bar?'
Quality, latency, and cost are one trade-off, not three. You pick the point that clears your usefulness bar for the least money and wait.
Right-size, then re-check
A bigger model costs more per answer and often replies slower — and users feel a slow reply as a worse product. So right-size: start from the smallest model that passes, and only move up when the quality genuinely falls short. Then re-check as models improve — last quarter's flagship job might run fine on this quarter's cheap model.
The right model is the smallest one that clears the bar. Paying for quality your users can't feel is just a bigger bill.
Latency is a feature. A 'better' answer that arrives two seconds late can score worse with users than a good-enough answer that's instant. Weigh speed alongside quality and cost, not after them.
The shape of it
- —Set a usefulness threshold: the quality bar where the feature does its job.
- —Pick the cheapest, fastest model that clears the bar — not the most powerful one.
- —Re-check as models change; today's cheap model may clear yesterday's hard bar.
A cheaper, faster model answers 'well enough' for your task at a fraction of the flagship's cost. What's the right call?
Continue in the app
Take the whole Measuring & Governing AI Products course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.