Word2Vec: Word Meaning as a Point in Space
Word2Vec turns words into numerical vectors, where semantic similarity becomes spatial proximity. It powers synonym detection and analogy tasks by learning from a word's context in a large text corpus.
WHY IT EXISTS Computers understand numbers, not words. To perform any meaningful computation on text, we need to represent words numerically in a way that captures their meaning. Simple methods like assigning a unique ID to each word fail because they treat every word as equally different from every other, losing all semantic relationships.
THE MENTAL MODEL Think of Word2Vec as creating a multi-dimensional map of language. Each word is a point on this map. Words with similar meanings, like "cat" and "kitten," are placed close together. The directions on this map also have meaning; moving in a certain direction might correspond to changing from a country to its capital, or from a masculine form to a feminine one.
HOW IT WORKS Word2Vec learns these representations by processing a massive text corpus. It uses a shallow neural network with one of two objectives: Continuous Bag-of-Words (CBOW), which predicts a word from its surrounding context, or Skip-gram, which predicts the surrounding context from a single word. The prediction task itself is just a training method; the real output is the set of learned weights from the network's hidden layer, which become the vector representation for each word.
WHEN TO USE IT Use Word2Vec to create foundational word embeddings for tasks like text classification, sentiment analysis, and synonym detection. It's a powerful and efficient way to give a model a basic understanding of word relationships before tackling a more complex task.
WHEN NOT TO USE IT Word2Vec creates only one vector per word, so it cannot handle words with multiple meanings (e.g., "bank" as a river bank vs. a financial institution). It also has no understanding of word order or sentence structure. For tasks requiring deep contextual understanding, modern transformer-based models like BERT are superior.
ONE CANONICAL EXAMPLE The most famous demonstration is solving analogies with vector math. After training, the model can compute that the vector for 'king' minus the vector for 'man' plus the vector for 'woman' results in a new vector that is closest in the 'meaning space' to the vector for 'queen'. This shows it learned abstract relationships like gender and royalty from text alone.
Read the original → en.wikipedia.org
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.