Lesson 6 of 7
Bigger, predictably better
9 min read
A tiny AI and a giant one run the same design — the giant just has more of everything. And that's one of the strangest truths of modern AI: to make it smarter, you often don't need a cleverer idea. You just make it bigger.
Bigger — and predictably so
Researchers noticed something almost eerie. As you scale a model up — more parameters, more training text, more computing power — how wrong it is falls along a smooth, predictable curve. Not in random jumps: a curve so regular you can forecast roughly how good a model will be before you've built it. Drag the size below and watch the error slide down that curve.
Scale buys accuracy along a predictable curve. That's why you can decide, before spending the money, that a bigger model is worth building — the gains are forecastable.
Two knobs, not one
"Bigger" sounds like a single dial. It isn't. Making a model larger has two knobs that must move together: the size of the model and the amount of text it trains on. Below, drag them yourself. A huge model fed too little text is wasted — it stays about as wrong as a tiny one. Push both up together and the error finally falls and the skills switch on. Grow just one and a warning appears: you're paying for scale you can't use.
Scale isn't one lever — it's a balance. A giant model starved of data, or a mountain of data with a tiny model, both waste the effort. The gains come only when the pieces grow in step.
And new abilities just appear
Here's the surprising part, the one you just saw switch on. The smooth curve hides sudden gifts: cross a certain size and the model can abruptly do something it flat-out couldn't before — spell reliably, add numbers, follow a multi-step instruction. These are emergent abilities — skills nobody wrote in, that simply appear once the model is big enough.
Scale doesn't just sharpen what a model already does — past certain sizes it hands the model whole new skills, as a side effect of being bigger. That's what makes each new generation feel like a jump, not a tune-up.
Why this shaped the whole field
This is why the biggest labs pour fortunes into ever-larger models and the data centres to train them: scale is the one lever that reliably pays off. It's also why nobody can fully predict a new model — you can read the error curve in advance, but exactly which new ability pops out at which size is far harder to call. Predictable in general, surprising in the details.
Scale is the field's most dependable lever — and its biggest source of surprises. The curve is smooth; the new abilities hiding along it are not.
Under the hood, "bigger" has three knobs, not one: the number of parameters, the amount of training data, and the compute spent training. They have to grow together — a giant model starved of data is wasted — and working out the right balance between them is its own line of research.
Recap
- —As you scale a model up (more parameters, data, and compute) its error falls along a smooth, predictable curve — scaling laws
- —Past certain sizes, brand-new emergent abilities appear that smaller models simply didn't have
- —Scale is the field's most reliable lever — forecastable in general, surprising in exactly which skill emerges when
A team's current model keeps failing a task they need it to do reliably. Based on how scaling works, what's the most reasonable bet to fix it?
Continue in the app
Take the whole How AI Really Works course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.