Word Embeddings: Turning Words into Vectors
Word embeddings turn words into numerical vectors, like coordinates on a map of meaning. Words with similar meanings, like "king" and "queen," are placed close together in this vector space. This is fundamental for text analysis in machine learning, allowing models to grasp semantic relationships instead of just matching text. The footgun is assuming the vector's individual numbers are human-interpretable; they are abstract features learned from data.
### The mental model A word embedding is a way to represent words numerically. Think of it as assigning every word in a vocabulary a specific coordinate in a multi-dimensional space. In this space, the distance and direction between words capture their semantic relationships. "King" isn't just a string of letters; it's a point, and the path from "king" to "queen" is mathematically similar to the path from "man" to "woman".
### How it works Embeddings are learned, not manually assigned. A model is trained on a massive text corpus by learning to predict a word given its surrounding context words. To do this efficiently, the model creates a dense vector of real numbers (e.g., a list of 300 numbers) for each word. Words that frequently appear in similar contexts, like "coffee" and "tea," will end up with very similar vectors because the model needs to make similar predictions for both.
### When to use it * **Semantic Search:** Finding documents that are conceptually related, not just matching keywords. * **Text Classification:** Determining the sentiment or topic of a document by understanding the nuance of the words used. * **Machine Translation:** Mapping words from one language to another by finding equivalent points in their respective embedding spaces.
### When NOT to use it * **When you need explainable rules:** The dimensions of an embedding vector are not human-interpretable. You can't point to one number and say it represents "animal-ness." The meaning is encoded in the vector as a whole. * **For simple string matching:** If you just need to find exact occurrences of a word, simpler methods like keyword search are more efficient.
### One canonical example The most famous example demonstrates the linear relationships embeddings can capture: `vector('king') - vector('man') + vector('woman')` results in a vector that is extremely close to `vector('queen')`. This shows the model learned the concepts of gender and royalty as directions in the vector space.
Read the original → en.wikipedia.org
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.