Lesson 1 of 7
What a GPU actually does
7 min read
Your laptop's main chip is a genius. So why does running a model need a completely different kind of chip — one built from thousands of tiny, almost dumb cores?
One fast worker vs a huge crew
A CPU is a few very fast, very clever cores — brilliant at doing one complicated thing after another. A GPU is the opposite: thousands of simple cores that all do easy maths at the same time. A model's core operation is exactly that easy maths — multiplying long lists of numbers — but there's an enormous amount of it. Hand that to a CPU and it grinds through the queue; hand it to a GPU and the crew splits it up and finishes together.
A GPU isn't a faster CPU. It's a different shape: many small cores doing the same simple maths in parallel — which is exactly the shape of the work a model does.
Why the model loves parallel
Running a model means multiplying its input by millions of weights, over and over. None of those multiplications need to wait for each other — they can all happen at once. That's the definition of a parallel workload, and it's why a GPU can be tens of times faster than a CPU on the same job, even though each individual GPU core is slower.
The work is embarrassingly parallel: millions of independent multiplications. A GPU exists to do independent work simultaneously — a near-perfect match.
Under the hood, this is why GPUs matter for graphics too: drawing a screen means shading millions of pixels independently — the same 'lots of easy maths at once' shape that a model's maths has.
The shape of it
- —A CPU has a few clever cores; a GPU has thousands of simple ones.
- —A model's maths is millions of independent multiplications — perfect for doing all at once.
- —That's why inference runs on GPUs: the work is parallel, and a GPU is built for parallel.
Why does running a model favour a GPU over a fast CPU?
Continue in the app
Take the whole Inference & Hardware course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.