Course

Build an LLM from Scratch

The capstone. You've used models; now build one on paper. Following Sebastian Raschka's *Build a Large Language Model (From Scratch)*, this course walks the pipeline a real implementation runs: text becomes tokens and embeddings, self-attention lets tokens read each other, a stack of Transformer blocks refines them, pretraining tunes millions of weights by predicting the next token, scale decides how good it gets, and a short round of fine-tuning turns a raw text predictor into an instruction-follower. Seven short lessons, every step drawn — the same mechanism, no formulas, no hand-waving.

7 lessonsAdvanced56 min

Start the course Get the app

Course lessons

Text → tokens → embeddings

The first thing your code does: cut text into tokens, then hand each one a vector the model can actually do math on.

›

Coding attention

Self-attention by hand: let every token look back at the ones before it and pull in what matters.

›

Assembling a GPT block

Wrap attention and a small feed-forward network into one repeatable Transformer block — then stack it.

›

Pretraining on unlabeled text

The training loop: predict the next token, measure how wrong, nudge every weight, repeat over a mountain of text.

›

Scale: data, params, compute

Two knobs decide how good the model gets — model size and training data — and they only pay off together.

›

Fine-tuning to follow instructions

A short, cheap round of training on example instruction–response pairs turns a raw predictor into an assistant.

›

You built the thing

The whole pipeline, end to end — from a raw string to an instruction-following model — in one picture.

›

Take the full course

Learn it properly — tracked & personalized

Follow the whole course as a guided journey — sequential lessons, progress, streaks and a personalized path, all in the app.

Get the app