What Is a Vector Database?

Storage built around one question: which vectors are closest to this one? Why that needs different machinery than a normal database.

A conventional database answers exact-match and range questions. WHERE user_id = 42. WHERE date > '2026-01-01'. Indexes make these fast by sorting and partitioning on values.

A vector database answers a different question: which stored vectors point most nearly in the same direction as this one?

That question does not decompose into sorting, because embeddings have hundreds or thousands of dimensions and there is no single axis to order by. Different question, different machinery.

Why exact search does not scale

The naive approach works and is worth knowing: compare the query against every stored vector, keep the closest.

For 10,000 vectors this is fine — genuinely fine, and a plain array plus a loop is a legitimate solution at that scale. For 10 million vectors of 1,536 dimensions each, every query means over 15 billion multiplications. Too slow for interactive use.

So production systems use approximate nearest neighbour search: accept a small chance of missing the true closest match in exchange for orders-of-magnitude speedup.

The tradeoff is explicit and tunable. Recall — the fraction of true nearest neighbours actually found — is a dial you set against latency. In RAG this is usually an easy trade, because a reranker is going to reorder the candidates anyway and occasional misses at the margin matter little.

How approximate search works

Two dominant approaches.

HNSW builds a layered graph. Each vector is a node connected to its neighbours, with sparse long-range links in upper layers and dense local links below. A search enters at the top, takes big jumps toward the target, then descends into progressively finer layers — like navigating with a country map, then a city map, then a street map. Fast and high-recall, at substantial memory cost since the graph lives in RAM.

IVF partitions vectors into clusters. Each query is compared against cluster centroids, then searched exhaustively within the few nearest clusters only. Cheaper on memory, and the recall cost shows up at cluster boundaries — a true neighbour sitting just across a boundary gets missed.

Quantization composes with both, storing vectors at reduced precision. Large memory savings, small accuracy cost, frequently worth it.

What these systems provide beyond search

The search algorithm is the interesting part but rarely the reason to adopt a product. What you actually get:

Metadata filtering. Restricting by date, source, or permissions alongside vector search. Harder than it sounds — filtering after search can leave you with too few results, filtering before search can wreck the index structure. Quality of filtered search is a genuine differentiator between systems.

Hybrid search. Combining vector similarity with keyword matching, which reliably outperforms either alone. Vectors handle meaning; keywords handle exact terms like error codes and function names.

Updates and deletes. Graph indexes do not love mutation. Handling churn without full rebuilds is real engineering.

Operational basics. Persistence, replication, backups, access control.

Choosing

The honest hierarchy, cheapest first:

Under ~10,000 vectors: compute similarity in application code. No database. This is a completely respectable answer and covers a lot of real projects.

Existing relational database: PostgreSQL with pgvector adds vector search to a database you already run. Transactions, joins, and one system to operate. Usually the right choice up to a few million vectors, and the option people skip past too readily.

Embedded vector library: an in-process index with no server. Good for local applications and single-node deployments.

Dedicated vector database: justified at large scale, high query volume, or when you need advanced filtering and hybrid search out of the box.

The common mistake is starting at the bottom of that list. Vector search is a feature, not an architecture, and adopting a distributed system for 50,000 chunks buys operational burden and no capability.

What to remember

• Vector databases answer “which vectors are nearest,” which conventional indexes cannot express.
• Exact search is fine at small scale; large scale needs approximate search, trading recall for speed on a tunable dial.
• HNSW (layered graph, fast, memory-hungry) and IVF (clustering, cheaper, boundary misses) are the main algorithms.
• Metadata filtering and hybrid search matter more in practice than the algorithm choice.
• Start with application code or pgvector; adopt a dedicated system when scale actually demands it.

Next: How Similarity Search Actually Works