Embeddings

A list of numbers that captures meaning. Two pieces of text that mean similar things produce similar lists.

What embeddings actually do

An embedding model takes a piece of text — a word, a sentence, a whole paragraph — and turns it into a list of numbers. Sounds boring; the magic is in the geometry. Pieces of text that mean similar things end up with similar lists, even when they share zero words. "Cancel my plan" and "how do I terminate my subscription?" become near-neighbours. "The weather in Paris" lives somewhere else entirely.

Once meaning is geometry, "find me text that means roughly this" becomes "find me the nearest pins on the map" — a problem computers solve in milliseconds across millions of items.

That's the whole concept. Below: how the geometry is actually trained and used.

When you'd reach for it

Anytime you want a system to compare meaning rather than wording:

• Semantic search over docs, tickets, code, transcripts — finding articles a keyword search would miss.
• Recommendations — "people who liked this also liked..." without a manual taxonomy.
• Deduplication and clustering — grouping the 50,000 customer complaints into themes.
• RAG(Retrieval-Augmented Generation) — embeddings are the retrieval half of retrieval-augmented generation.

You wouldn't reach for embeddings when exact strings matter — order numbers, SKUs, error codes. For those, plain keyword search wins.

How it works under the hood

You feed a piece of text into an embedding model (a small specialised model, separate from the chat LLM). It returns a fixed-length vector of floats — typically 768, 1024, 1536, or 3072 numbers:

"refunds take 5 business days"   →  [0.024, -0.118, 0.503, …]    (e.g. 1024 numbers)
"how long until I get my money?" →  [0.031, -0.105, 0.488, …]
"the weather in Paris"           →  [0.402, 0.221, -0.114, …]

To measure how close two pieces of text are, almost everyone uses cosine similarity: the cosine of the angle between two vectors. 1.0 = identical direction (very similar), 0 = unrelated, −1 = opposite. The actual length of the vector doesn't matter — only its direction.

Embedding models learn this geometry through contrastive training: shown sentence pairs, taught to push related pairs together in space and unrelated ones apart. At scale, this produces a high-dimensional space where geometry encodes meaning.

Where it bites in real life

Common embedding models

Under the hood (optional)

If you ever build with embeddings: send a list of texts to an embedding model, get back a list of float-arrays, compute cosine similarity between any two. About 12 lines of Python with the Voyage SDK.

from voyageai import Client
voyage = Client()
 
texts = [
    "refunds take 5 business days",
    "how long until I get my money?",
    "the weather in Paris",
]
 
resp = voyage.embed(texts, model="voyage-3")
vectors = resp.embeddings   # list of lists of floats
 
import numpy as np
def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
 
print(cosine(vectors[0], vectors[1]))   # ~0.85 — very related
print(cosine(vectors[0], vectors[2]))   # ~0.10 — unrelated

Further reading

• The Illustrated Word2vec (Jay Alammar) — the visual explainer of where embeddings come from.
• OpenAI — Embeddings guide — practical tutorial with code.
• Anthropic — Embeddings (Voyage) — using Voyage embeddings with Claude pipelines.
• MTEB leaderboard — independent comparison of embedding models across many retrieval tasks.