Lesson 7 of 7
You built the thing
6 min read
Seven steps ago a string couldn't even enter the network. Now you've got an assistant. What actually happened?
The whole pipeline
Step back and the whole thing is a short list. Every model you've ever chatted with is some version of exactly this pipeline — the labs just run it at enormous scale.
- —Tokens & embeddings — text becomes token ids, then learnable vectors.
- —Attention — tokens read each other; meaning gets mixed from context.
- —The block — attention + feed-forward + residual + norm, stacked deep.
- —Pretraining — predict the next token over unlabeled text; tune every weight.
- —Scale — grow size and data together; new abilities emerge.
- —Fine-tuning — a short round of examples turns the predictor into an assistant.
Why building it changes how you use it
Knowing the pipeline is practical, not academic. You know why a model hallucinates (it's predicting likely tokens, not looking up facts), why context matters (attention only sees what's in the window), and why fine-tuning changes behaviour but not knowledge. The Transformer you assembled and the fine-tuning round on top are the two ideas behind almost every model shipping today.
Want the real code? Sebastian Raschka's Build a Large Language Model (From Scratch) implements every step here in plain PyTorch — this course is the map, the book is the territory.
Which single loop does the heavy lifting in both pretraining and fine-tuning?
Continue in the app
Take the whole Build an LLM from Scratch course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.