The concept
Token: a small chunk of text, often a whole common word, part of a longer word, a space plus a word, or a punctuation mark. A large language model (LLM) splits your text into tokens, turns each into a number, and works only with those numbers.
Why not whole words? There are too many (names, typos, product codes). A vocabulary of reusable pieces can build any text, even words the model has never seen.
Rule of thumb for English: 1 token ≈ 4 characters, or about ¾ of a word. So 100 tokens ≈ 75 words. Other languages, code and numbers often use more tokens. Every model has its own tokenizer, so counts vary.
Walkthrough: a bank's complaint inbox
A customer emails: "This is unbelievable. I was charged an overdraft fee twice!"
A tokenizer might split it like this (exact splits depend on the model):
- "This" " is" → 2 tokens (common words stay whole, the space rides along)
- " unbelievable" → maybe " un" + "belie" + "vable" → 3 tokens
- " overdraft" → maybe " over" + "draft" → 2 tokens
- "." and "!" → 1 token each
Roughly 15 tokens for a 10-word sentence.
Now scale it. The bank wants AI to summarize and tag 1,000 complaint emails a day.
- Say each email plus instructions ≈ 500 tokens in.
- 1,000 × 500 = 500,000 input tokens.
- Say each summary ≈ 150 tokens out → 150,000 output tokens.
- Imagine a price of $3 per million input tokens and $15 per million output tokens (illustrative; real prices vary a lot by model).
- Input: 0.5 × $3 = $1.50. Output: 0.15 × $15 = $2.25. About $3.75 a day.
Notice: output tokens often cost more per token than input. Asking for shorter answers saves money.
Why tokens matter
- Cost: most AI APIs bill per token, in and out.
- Limits: every model can only hold so many tokens at once (Day 20).
- Quirks: the model sees "belie" + "vable", not letters, which is one reason letter-counting tasks can trip it up.
Where you see it at work
- Banking: budgeting a complaint-triage or call-summary project.
- Insurance: estimating cost to summarize thousands of claim files.
- Product management: writing the business case. "Tokens per request × requests per day × price" is your first cost model.
- Business analysis: setting a max reply length in the requirements.
In one sentence
LLMs read text as tokens (about ¾ of a word each in English), and tokens drive both your bill and your limits.