Lesson 5 of 8
Why AI forgets the conversation
10 min read
You tell an AI your name, chat for a while, then ask a follow-up — and it acts as if it never heard your name at all. It didn't get distracted. So what actually happened?
It re-reads the whole chat, every time
An AI has no memory of you between messages. Each time you hit send, it doesn't recall the conversation — it re-reads it. The entire chat so far is laid out in front of it, like notes spread across a desk, and it predicts the next words from what's on that desk. Nothing is filed away on the side; the conversation itself is the only thing it ever looks at.
Everything an AI can use right now sits in one place — its context window. Picture it as a desk: the AI re-reads the whole desk on every reply, because it has no memory of the chat apart from what's lying on it.
The desk has an edge
But the desk is only so big. As the conversation grows, message after message piles on — and once it's full, something has to give. The oldest notes slide off the far edge to make room for the new ones. They aren't tucked into a drawer; they're simply gone from view. The AI keeps reading the desk just as before — it just can't see what fell off.
When the context window fills, the oldest part drops out of view to make room. That's the "forgetting": not a broken memory, but a full desk pushing the start off the edge.
Look inside: a fixed number of slots
The desk has an edge — but how big is it, exactly? Inside, the window is a fixed amount of room, cut into slots. Every message you send takes up a few slots; a longer one takes more. Before each reply the AI re-reads everything in the room, top to bottom. And here's the mechanism behind the forgetting: when new messages need more room than there is, the oldest slots are dropped to make space — no choice, no drawer, just gone.
That room is the context window, and the slots are tokens — roughly, chunks of words. A window holds a fixed number of them, its context length. Nothing is remembered outside it; the AI only ever works with what's in the room right now.
You'd think a bigger window fixes everything — and it helps. But every slot the AI re-reads costs a little more time and money, so windows aren't endless, and a long enough chat fills any of them. That's why even huge-window models still drift on a marathon conversation. The reliable fix isn't a bigger desk; it's putting what matters back in the room — restating the key facts so they sit in the slots the AI is actually reading.
So that's why it forgets your name
Your name sat on the desk the whole time you'd just mentioned it — and for a while after. But a long conversation keeps shoving notes off the edge, and eventually that early message goes with them. Ask "what's my name?" now and the AI honestly can't see it. It isn't being careless; the information simply isn't in front of it anymore.
An AI only knows what's currently on the desk. Once something scrolls out of the context window, it's as if it was never said — so the fix is simply to put it back in view.
In a long chat, if something matters — a key instruction, a fact about you — restate it now and then. Bringing it back onto the desk is what keeps the AI working with it. And starting a fresh chat clears the desk completely.
Recap
- —An AI has no memory between messages — it re-reads the whole conversation each time it replies
- —That conversation sits in a fixed-size context window, like a desk with limited space
- —When the desk fills, the oldest messages slide off and the AI can no longer see them — that's why it "forgets"
You're deep in a long chat and want the AI to keep following one important instruction you gave at the very start. Best move?
Continue in the app
Take the whole AI from Scratch course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.