Vector Embeddings: Turning Meaning into Math

Why it exists

Computers understand numbers, not abstract concepts like "meaning" or "similarity." To enable machines to find related items—like images with similar content or articles about the same topic—we needed a way to translate complex, unstructured data into a numerical format that can be compared mathematically.

The mental model

Think of an embedding as a coordinate in a high-dimensional "meaning space." Just as the map coordinate (4, 2) is close to (4, 3), the vector for the word "king" is close to the vector for "queen" in this meaning space. The embedding process is what draws this map, placing related concepts near each other.

How it works

A trained model, often a neural network, learns to map an item (like a word or image) to a fixed-size list of real numbers, called a vector. The model is trained on a massive dataset, learning to place items that appear in similar contexts close together in the resulting vector space. The distance between two vectors (often measured by cosine similarity) then becomes a proxy for the semantic similarity of the original items.

When to use it

Use embeddings when you need to operate on the meaning of data, not just its literal value. This is crucial for semantic search (finding documents that mean the same thing, not just share keywords), recommendation engines ("users who liked this also liked..."), and data clustering (grouping similar items automatically).

When not to use it

Avoid embeddings for tasks requiring exact, deterministic matching. For finding a user by a unique ID or retrieving a document by its exact title, a traditional database index or hash table is far more efficient and correct. Embeddings are for probabilistic, similarity-based tasks.

One canonical example

In natural language processing, a model might learn embeddings for words. After training, it could perform vector arithmetic that reflects meaning, such as vector('king') - vector('man') + vector('woman') resulting in a vector very close to vector('queen'). This shows it has captured underlying semantic relationships, not just memorized words.