Talking to a simulated expert

When an assistant answers you, it feels like there's someone in there — a knowledgeable, patient expert. There isn't. So who, exactly, are you talking to?

Not a mind — a performance

Remember what finetuning did: it showed the model thousands of example conversations, each ending in an ideal, helpful answer, and had it imitate them. So when you ask a question, the model isn't consulting a mind or looking anything up — it's playing a character: the helpful assistant it saw over and over in those examples. Its reply is its best impression of what that character would say next. And a character is assembled from its sources. Below, cast that character yourself — turn up how much the assistant leans on each kind of writer it learned from, and watch the very same answer change voice.

Whoever wrote the examples

This has a sharp consequence. If the assistant is an impression of the people who wrote its examples, then who wrote them decides who it becomes. Swap in more careful, hedging writers and it hedges; feed it brash, certain ones and it turns brash. The same base model can be cast as a terse coder, a patient tutor, or a breezy companion — nothing new is learned about the world, only whose voice to borrow. Rewrite the guidelines those writers follow — tell them to hedge more, or to never refuse — and the finished assistant shifts to match, though not a single fact in its head has changed. The writers usually follow guidelines — be helpful, be honest, admit what you don't know — so a well-made assistant picks up that steady, reasonable persona. But notice what rode in with it: that calm, confident tone.

Tone is not truth

Here's the trap. A well-played expert sounds sure — that steady, confident voice is part of the performance, copied straight from writers who wrote that way. But sounding sure and being right are two different dials, and only one of them is yours to turn. The scene below now shows two meters. Drag the sliders and watch sounds sure swing all the way up or down. Then keep an eye on the other meter — actually right. It doesn't move. The fact was fixed before you touched anything; the voice is yours to shape, the facts are not.

Why this is the trap

Why does this matter so much? Because we read confidence as a signal — a sure human voice usually means someone who checked. With a simulated expert that shortcut breaks: the model was trained to sound like a confident helper whether or not it actually knows. Push it to the limit and you get the strangest case — an answer wrong in every detail, delivered in the warmest, most authoritative voice in the room. The fix isn't to distrust the assistant on principle; it's to stop reading its tone as evidence. Judge the claim, not the confidence. In practice the tell is simple: the more polished and certain an answer sounds, the more it deserves the same check you would give a stranger's confident claim — because that polish was rehearsed, not earned.

Recap

• —The assistant is a character the model learned to play by imitating example conversations — not a mind or a database
• —Whoever wrote those examples, and the guidelines they followed, shape its personality, values, and voice
• —Its confident tone is part of the performance — a dial that can be turned, separate from whether the answer is true
• —So confidence is never a fact-check: weigh the claim itself, not how sure it sounds