🎯 Core Goals
- Introduce the concept of “distance” between words.
- Understand that similar words are “close” in meaning-space.
- See how LLMs use this distance to understand relationships.
Words have “distance” between them based on meaning. Similar words like “lion” and “tiger” are close. Unrelated words like “lion” and “banana” are far.
👁️ Visuals & Interactives
🧠
Find the Odd One Out
🍎
Apple
🍌
Banana
🍊
Orange
🐕
Dog
📝 Key Concepts
Just for the record: “Meaning-space” isn’t the proper technical term. Data scientists use jargon like “Semantics”, “Vector Embeddings”, and “Metric Space”. But for our intuition, “Meaning-space” works perfectly! We’ll cover the real jargon later.
- Meaning-Space: Think of it as a giant map where every word has a specific coordinate. Words with similar meanings are located near each other.
- Contextual Neighbors: The distance between words is how an LLM can understand that “the doctor walked into the room” and “the nurse walked into the room” are likely talking about a similar situation.
How do you think an LLM would know that “doctor” and “nurse” are related? What patterns might it have seen in training data?
🧠
QUIZ
In an LLM's "meaning-space," which two words would be closest together?
"Doctor" and "doctrine" — they share most of the same letters
"Doctor" and "nurse" — they share similar meaning and context
"Doctor" and "calendar" — doctors use calendars for appointments