🎯 Core Goals
- Teach that embeddings capture meaning as coordinates in space.
- Show through a hands-on exercise how distance between coordinates reveals similarity.
Embeddings turn the meaning of words and sentences into coordinates. Similar meaning = nearby coordinates. That’s how one popular type of RAG retrieval — semantic search — finds relevant documents without keyword matching.
Words as Points in Space
When we explored word distance earlier, we established that some words are “closer” in meaning than others — king and queen are neighbors; king and banana are strangers.
Embeddings make that intuition precise. Every word, sentence, or document gets converted into a list of numbers — its coordinates in a multi-dimensional space. Two documents with similar meaning will have coordinates that are close together.
This is what makes semantic RAG retrieval work: instead of matching keywords, it finds documents whose coordinates are nearest to your question’s coordinates.
🗺️ Try It: Distance on a Map
Here are four words plotted on a simple XY graph:
Words as Coordinates
Click any word to see how far it is from the others
👆 Click an animal to see its distances to all others
These words all mean "cat" — but a keyword search for "cat" finds zero of them:
Semantic search finds them all — their embeddings land near 🐱 on the map.
Click any animal to see its distances to the others. Notice how the three felines cluster together — lion, cat, and tiger all share the same region of the map — while banana lands far away in its own corner.
The math confirms what intuition already knew — animals in the same category share far more in common than animals and fruit.
Scaling Up
In this exercise, we used 2 dimensions. Real embeddings use hundreds or thousands of dimensions — capturing not just one aspect of meaning, but many simultaneously: topic, sentiment, formality, domain, and more.
The math stays exactly the same (distance between points). The space just gets much bigger.
When you search and find relevant results without using the exact right keywords, that’s embeddings at work. “Company vacation policy” finds docs about “PTO” and “annual leave” because they’re close in meaning — not because the words match.
Why This Enables Semantic RAG Retrieval
RAG is a pattern: retrieve relevant documents, then have the LLM answer from them. The retrieval step can use many methods. This is the one most people picture when they say “RAG” — semantic search powered by embeddings.
Vector databases store embeddings for every document in your knowledge base. When a question arrives:
- Convert the question to an embedding (a set of coordinates)
- Find documents with the closest embeddings
- Return the top matches — ranked by meaning, not keyword overlap
Sarah searching for “construction delay cases” finds all the relevant cases, even if some used the phrase “contractor failed to meet the deadline.”
But the retrieval step could also use keyword search, a SQL query, or a hybrid of multiple methods. The RAG pattern stays the same — only the search engine changes.
📝 Key Concepts
- Embeddings: Numbers that capture the meaning of text as coordinates
- Similar meaning → close coordinates → small distance
- Vector databases: Store embeddings for fast similarity lookup at scale
- Semantic search: Find by meaning, not by matching exact words — one retrieval method within the RAG pattern
- Dimensions: Real embeddings use hundreds of dimensions — same concept, much bigger space
How do vector embeddings enable semantic search?