Lesson 3 of 7
Inside the black box
9 min read
You've probably heard that AI runs on a neural network with billions of parameters. It sounds like a sealed box humming with magic. But between you hitting enter and the answer appearing, something surprisingly orderly happens — and it's worth seeing.
First, words become numbers
A computer can't do arithmetic on the word cat. So the very first thing the model does is turn each token into a long list of numbers — think of it as a set of coordinates that place the token's meaning in a vast space. Words used in similar ways land near each other. From here on, the model isn't juggling words at all; it's doing maths on these number-lists.
Past this first step, the words are gone. Everything the model does next is arithmetic on numbers — right up until the final step, where numbers turn back into a word.
Then a tall stack of layers
Those number-lists travel up through the model's main machine: a transformer, a tall stack of near-identical layers. Each layer reads the current numbers, refines them a little — sharpening what each word means in this particular sentence — and passes the improved version up to the next layer. Real models stack dozens of them. Run it below and watch the computation climb, one layer at a time.
No single layer holds the answer. The understanding is built up gradually — a small refinement at each layer — all the way to the top.
Out comes a guess
At the very top, the model takes the final numbers for the last token — the one right before the blank — and turns them back into words: a score for every possible next token, which becomes a probability. Steering all those refinements are the model's parameters — the billions of numbers that were tuned during training. That's what "a 70-billion-parameter model" is counting.
The whole black box, in one line: words → numbers → refine, refine, refine → numbers → next word. Enormous in scale, but not magic in shape.
Under the hood, that first words-to-numbers step is called an embedding, and each layer really does two jobs: letting the tokens share information with each other, and then thinking about each token on its own. That first job — deciding which earlier words each word should listen to — is attention, and it's the whole of the next lesson.
Recap
- —The model first turns each token into a list of numbers — after that, it's all arithmetic, no words
- —Those numbers climb a transformer: a tall stack of layers, each refining them a little
- —At the top, the last token's numbers become a probability for the next word — all steered by billions of trained parameters
A model is described as having 40 layers. Which best captures what those 40 layers do to your text?
Continue in the app
Take the whole How AI Really Works course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.