Vector databases

A database optimised for one question: "given this point, find the nearest points to it — fast." The storage layer for embeddings.

What a vector DB actually does

A vector database has one job: store millions of embeddings and, given a new one, return the nearest neighbours fast. That's it. It's the storage layer that makes embeddings practical at scale — without it, every query would mean comparing the question to every document one by one.

Once you can ask "what's near this point?" in milliseconds across millions of items, semantic search, recommendations, and RAG(Retrieval-Augmented Generation) all become real products instead of demos.

That's the whole concept. Below: the algorithms that make it fast and the production-grade recipe.

When you'd reach for it

Whenever you have a lot of embeddings and need to query them online:

• A support bot that searches across thousands of help articles per query.
• A product recommender comparing a user vector against millions of items.
• A multi-tenant SaaS where each customer's documents need to be searchable but kept separate.
• Any RAG pipeline beyond the prototype stage.

For a few hundred items, you don't need a vector DB — a plain array and a for loop is fine. The DB earns its keep at scale and when filtering by metadata matters.

How it's actually built

The naive approach — compute cosine similarity between your query vector and every stored vector — is O(n) per query. At 10M chunks × 1024 dimensions, that's billions of multiplications per query. Too slow.

Vector databases use ANN(Approximate Nearest Neighbour) algorithms — primarily HNSW(Hierarchical Navigable Small World) or IVF(Inverted File index). They give up exact correctness for ~100× speedup, with recall typically above 95%. Great trade for retrieval.

What a row in a vector DB looks like:

• A vector (the embedding).
• An ID.
• Metadata for filtering: tenant_id, source, date, language.
• Often, the original text.

INSERT INTO chunks (id, embedding, text, source, tenant_id)
VALUES
  ('c1', '[0.024,…]', 'Refunds take 5 days…',     'kb/refunds.md',  'acme'),
  ('c2', '[-0.077,…]', 'Cancel your sub at…',     'kb/billing.md',  'acme');

Querying:

SELECT id, text
FROM chunks
WHERE tenant_id = 'acme'              -- metadata filter
ORDER BY embedding <=> '[0.018,…]'    -- cosine distance to query
LIMIT 5;

Where it bites in real life

The landscape

Under the hood (optional)

If you set up your own vector DB, this is roughly what it looks like with Postgres + pgvector + Voyage embeddings — a few SQL lines for setup, a small Python query at runtime.

-- One-time setup
CREATE EXTENSION vector;
CREATE TABLE chunks (
  id        BIGSERIAL PRIMARY KEY,
  text      TEXT NOT NULL,
  source    TEXT,
  tenant_id TEXT NOT NULL,
  embedding VECTOR(1024)             -- voyage-3 dimensions
);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunks (tenant_id);

# Query
query_emb = voyage.embed(["how do I cancel?"], model="voyage-3").embeddings[0]
cur.execute("""
  SELECT id, text
  FROM chunks
  WHERE tenant_id = %s
  ORDER BY embedding <=> %s::vector
  LIMIT 10
""", (tenant_id, query_emb))

Further reading

• pgvector — official repo & docs — the most-deployed vector store in the world.
• Pinecone — Vector database concepts — clear non-tutorial explanation of HNSW/IVF.
• Cohere — Rerank — official guide to rerankers as the second stage of retrieval.
• Anthropic — Contextual retrieval — modern technique that significantly improves chunk-level retrieval quality.