Lesson 5 of 7
Context engineering
8 min read
Your assistant nailed the task ten messages ago. Now it's ignoring the one instruction that mattered. It didn't get dumber — it ran out of room.
The window is a budget, not a memory
Everything the model sees on a call — system prompt, history, retrieved documents, tool results — shares one fixed space: the context window, measured in tokens. It is not long-term memory; it's a desk of a fixed size. Pile on too much and the oldest or least-important material falls off the edge.
The context window is a fixed budget. What you put in it — and what you leave out — decides what the model can actually use.
Curate what goes in
This is context engineering: deciding, on every call, what deserves a slot. Don't dump the whole database — retrieve the few passages that matter. Don't resend a hundred messages — summarise the old ones and keep the recent turns. Put durable rules in the system prompt so they never scroll away. More context isn't better; the right context is.
The skill isn't stuffing the window — it's curating it: the right facts, summarised history, durable rules up top, and nothing that just burns budget.
A bloated window costs you twice: it's slower and pricier to process, and models attend worse to a needle buried in a haystack. Lean context often beats full context on both speed and quality.
The shape of it
- —Everything the model sees shares one fixed token budget — the context window.
- —It's working space, not memory: overflow drops the oldest or least-relevant material.
- —Curate each call: retrieve what's relevant, summarise old turns, pin durable rules.
- —More context isn't better — the right context, within budget, is.
Your support bot keeps forgetting its 'never share pricing' rule deep in long chats. What's the fix?
Continue in the app
Take the whole Building AI Apps with the API course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.