Lesson 4 of 7
Vector databases
6 min read
You've embedded a million chunks. A question comes in. Comparing it against all million vectors, one by one, on every single query — is that really the plan?
A database built for nearness
A vector database stores your embeddings and answers one kind of question fast: which stored vectors are nearest to this one? A brute-force scan works for a few thousand, but not for millions on every query. So a vector database uses an index — a smart shortcut — to find the nearest neighbors without checking them all.
A vector database stores embeddings and finds nearest neighbors fast — the retrieval half of RAG at scale.
Fast enough, not perfect
The trick is approximate nearest neighbor search: it trades a sliver of accuracy for a huge speed-up, returning almost always the true nearest chunks in milliseconds instead of seconds. Alongside the vectors it stores metadata — source, date, author — so you can filter ("only this project's docs") before or after the search.
Approximate search trades a little accuracy for speed; metadata lets you filter to the right subset.
You don't always need a dedicated vector database. For a few thousand chunks, an in-memory search or a vector extension of a database you already run is often simpler. Reach for a specialized one when scale or filtering demands it.
The shape of it
- —A vector database stores embeddings and finds nearest neighbors fast.
- —An index (approximate search) avoids scanning every vector on each query.
- —Metadata stored alongside lets you filter by source, date, or project.
Your prototype compares each query against all 5,000 chunks and it's plenty fast. Your boss asks why you'd ever need a vector database. The honest answer?
Continue in the app
Take the whole RAG & Search course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.