Glossary
What is a token?
The unit an AI actually reads and writes: a chunk of text — often a whole common word, sometimes just a piece of a longer or rarer one. A model sees your text as a sequence of these chunks, never as individual letters.
Models don't see letters or whole sentences. They break text into tokens — on average about ¾ of a word in English — and predict the next token, over and over. That splitting step is called "tokenization".
1 word · 3 tokens — the model handles each piece separately.
Tokens matter for two very practical reasons: cost and limits. You're usually billed per token, and every model can only hold a fixed number of tokens in view at once — its context window.
Related terms
Appears in these lessons
See it live
Don't just read the definition — play with it
Watch text split into tokens in real time — it's a lesson in the How AI Really Works course, free to try in the app.