Lesson 3 of 7
The RAG pipeline
7 min read
You've got a 200-page handbook and one question. You can't paste the whole thing into the model. So what actually happens between the document and the answer?
Four steps, once you see them
Every RAG system runs the same assembly line. Chunk the documents into passages. Embed each chunk into a vector and store it. When a question arrives, embed the query and retrieve the few nearest chunks. Then generate an answer from those chunks. Step through it and the whole thing stops being a black box.
The RAG pipeline is chunk → embed → retrieve top-k → generate. The same four steps, every time.
Top-k: only the best few
You don't feed the model everything — you feed it the top-k: the k nearest chunks to the question, usually a small handful. Too few and you miss the answer; too many and you bury it in noise (and pay for the extra tokens). Choosing k and the chunk size is most of the tuning in a real system.
Retrieval returns the top-k nearest chunks — enough to answer, few enough to stay sharp and cheap.
Chunk size is a real trade-off: small chunks pinpoint the answer but can lose context; big chunks keep context but dilute the match. Start around a paragraph and tune from there.
The shape of it
- —Offline: chunk the docs, embed each chunk, and store the vectors.
- —At query time: embed the question, retrieve the top-k nearest chunks.
- —Generate the answer from those chunks — grounded in real passages.
Your RAG bot retrieves the right chunk, but the answer misses a key detail that was in the paragraph right before it. What's the most likely fix?
Continue in the app
Take the whole RAG & Search course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.