The concept
Transformer: the neural network design (Day 9) behind today's large language models. Introduced in 2017, its key idea is attention.
Attention: for every token (Day 16), the model asks "which other tokens in this text matter most for understanding me?" and gives each one a weight. Then it blends in information from the high-weight tokens.
Two things made this a breakthrough:
- Every word looks at every other word, not just its neighbors.
- It happens in parallel, all words at once, instead of reading left to right one by one. That parallel math is exactly what GPUs are good at (Day 13).
Walkthrough: what does "it" mean?
Sentence 1: "...returned the laptop because it was broken."
- The model processes "it" and scores every other word.
- Words like "broken" fit things, not people. So "laptop" gets a high weight. Say laptop 0.7, customer 0.1, everything else shares the rest.
- "it" now carries the meaning "the laptop".
Sentence 2: "...returned the laptop because it was too expensive."
- "too expensive" still points to a thing with a price. Laptop again gets most of the weight.
- Swap the ending to "because she changed her mind" and "she" would attend to "customer" instead.
The weights aren't hand-written rules. The model learned them from huge amounts of text (Day 18).
Attention heads
A transformer runs many attention "heads" side by side. Each head can learn to track a different kind of relationship:
- one may link pronouns to nouns ("it" → laptop)
- one may link reasons to actions ("because" → returned)
- one may track descriptions ("broken" → laptop)
No one assigns these roles; heads learn them. Together they build a rich picture of each word in context. Stack dozens of these layers and you get a modern LLM.
Where you see it at work
- Retail: reading return reasons and reviews. "It stopped charging after a week" is understood as a product defect, not a shipping problem.
- Banking: telling "I disputed the charge because it was fraud" from "...because it was a duplicate".
- Insurance: linking "the other driver" vs "my car" correctly in a claim description.
- Customer experience: routing by meaning, not keywords.
In one sentence
Transformers use attention so every word can look at every other word at once, which is how AI understands words in context.