🎯 Core Goals
- Tie it all together: distance is how LLMs decide what word comes next.
- See how LLMs find words that are “close” to the context.
- Understand the complete picture: autocomplete → distance → generation.
LLMs generate text by finding words that are “close” to the context. “The delicious ___” → pizza is close, cat is far → LLM picks pizza.
👁️ Visuals & Interactives
"The delicious..." _
Next Word Probability (Distance)
🍕 pizza
44%
🍰 cake
33%
🐱 cat
2%
others ...
21%
The LLM maps "The delicious" into meaning-space and picks the mathematically closest valid next word.
📝 Key Concepts
- Context: The prompt establishes the initial coordinate in meaning-space.
- Distance Calculation: The LLM measures the distance to every word in its vocabulary.
- Closest Word Output: It statistically selects the closest valid word as the “next” word.
- The Cycle: This connects back to the super-powered autocomplete concept—this is exactly how the generation process works.
This is why LLMs can’t “go back and edit” as they write—they only look forward, predicting the next token based on everything that came before it.
LLMs don’t “choose” words based on understanding. They calculate distances and pick statistically. No thinking, just math.
🧠
QUIZ
When an LLM generates text, what happens at each step?
It plans the entire sentence first, then writes it all at once
It looks up a pre-written response from its training data
It calculates the distance to every word in its vocabulary and picks the most likely one