Course

Inference & Hardware

You can call a model in one line. Running one — fast, cheap, at scale — is a different craft. This is the engineer's tour of inference: what a GPU actually does that a CPU can't, what you're really paying NVIDIA for, why generating tokens is slow, and the practical levers that make it fast and affordable — serving engines like vLLM and Ollama, quantization, and right-sizing the model to the job. Seven short lessons, every screen drawn, no hype and no GPU spec sheets to memorize.

7 lessonsIntermediate45 min

Start the course Get the app

Course lessons

What a GPU actually does

A CPU races through tasks one at a time; a GPU does thousands of small ones at once. That difference is the whole game.

›

What NVIDIA actually does

It's not just the chips. The quiet moat is CUDA — the software layer everyone builds on.

›

Why inference is slow & expensive

A model writes one token at a time, and 'thinking' models write many you never see — each one is real compute you pay for.

›

Serving engines

A serving engine is the software that runs a model efficiently — batching many requests, reusing memory, keeping the GPU busy.

›

Quantization

Store each weight in fewer bits and the model shrinks and speeds up — with only a small drop in quality, until you go too low.

›

Cutting your inference bill

The cost of serving is a formula with a few knobs — model size, tokens, batching, caching. Turn them deliberately.

›

Serving it fast and cheap

The whole course in one loop: the hardware, why it's slow, and the levers that make it fast and affordable.

›

Take the full course

Learn it properly — tracked & personalized

Follow the whole course as a guided journey — sequential lessons, progress, streaks and a personalized path, all in the app.

Get the app