Lesson 4 of 7
Practice makes perfect
11 min read
You can learn tennis by watching a coach, but you don't get good until you play — badly, for hours, keeping whatever wins the point. It turns out AI is much the same. So how does a model practise?
From copying to practising
Finetuning taught the model by imitation — copy the human's answer. But imitation has a ceiling: you can only ever be as good as the examples, and for a hard problem nobody may have written the perfect steps. So for problems with a checkable answer — a maths result, a piece of code that either runs or doesn't — we switch methods. Instead of showing the model the answer, we let it try.
Imitation copies what humans did, so it can't beat them. To go further, the model has to stop copying and start practising — on problems where success can actually be checked.
Keep what works
The practice loop is this. Give the model a problem and let it generate many attempts — dozens of tries, each reasoning it through a little differently. Check which ones reached the right answer. Then nudge the model to make the successful kinds of attempts more likely next time, and the failures less likely. Repeat across thousands of problems. This is reinforcement learning: learning from the outcome of its own tries, not from a human's example. Run a few rounds below.
Nobody hands the model the solution. It discovers its own by trying in bulk and reinforcing whatever worked — so with practice it gets steadily better, and can find strategies no human ever demonstrated.
What actually changes inside
Firing off attempts is only half the loop. The other half is what practice does to the model afterwards. Picture the model as having habits — approaches it reaches for when it sees a problem. Below it starts with a bad habit: mostly rough guessing. Each practice round it tries in bulk, keeps whatever reached the answer, and shifts its habits toward those. Round after round the approach that works takes over — and it got there without anyone ever showing it the answer.
Practice doesn't just grade attempts — it reshapes the model's habits toward whatever succeeds. That's how it climbs from a bad start to a good approach on its own.
Why practice can beat its teachers
Here's the whole payoff. Copying is capped at the copied — you can't outdo the examples you imitate. But a model that keeps whatever actually works isn't limited to approaches a human wrote down; it can stumble onto its own and keep them precisely because they win. Give it a problem whose answer can be checked and enough rounds of practice, and it can end up better than anything it was first shown — which is exactly where this course is heading.
The many attempts per problem are often called rollouts, and reinforcing the good ones is how a model learns to solve maths and coding problems well past what it saw in finetuning. Because it's discovering its own methods, its reasoning can end up looking a little alien — which is exactly the next lesson.
Recap
- —Finetuning learns by imitation — copying human answers — which caps the model at human level
- —Reinforcement learning lets it practise: generate many attempts, check them, reinforce the ones that reach the answer
- —It learns from the outcome of its own tries, so it can discover strategies no human showed it
- —Practice reshapes the model's habits toward the approaches that work — so it can climb from a bad start on its own
A model is being improved on maths problems that each have a definite right answer. Which approach best matches reinforcement learning?
Continue in the app
Take the whole How AI Learns to Think course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.