Lesson 5 of 6
Confidence isn't correctness
5 min read
'I'm certain.' 'Without a doubt.' 'This is well established.' An AI can wrap a wrong answer in the language of total confidence — and it sounds exactly like it does when it's right.
The tone is generated, just like the answer
When a model writes 'definitely' or 'I'm certain,' it isn't reporting a measurement of its own reliability. Those words are just more likely-next-words, picked because confident text tends to follow this kind of question. The sureness is part of the writing — not a readout of how right it is.
How well stated confidence matches real accuracy has a name — calibration — and most chatbots are badly calibrated. Loud and wrong is very much on the menu.
So confidence can't be your filter
If 'sounds sure' were a reliable signal, you could just trust the confident answers and double-check the hedged ones. But sureness and correctness come loose from each other: some of its most emphatic answers are wrong, and some hesitant ones are spot on. Sorting by confidence doesn't sort by truth.
Flip the usual instinct. A hedge ('I think', 'you should verify') is often a model being honest — sometimes more trustworthy than a booming, unqualified claim.
What to take away
- —Confident phrasing is style, not evidence of accuracy.
- —Models are poorly calibrated: they can sound just as sure when wrong as when right.
- —Judge an answer by whether it checks out — never by how sure it sounds.
Two AI answers to a factual question: one hedges ('I believe it's around…'), one is emphatic ('It is definitely X'). Which deserves more trust?
Continue in the app
Take the whole When AI Gets It Wrong course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.