Vector Index Tuning: HNSW vs IVF

The Problem#

Vector search is rarely slow because the database is broken. It is usually slow because the index was chosen or tuned without measuring the recall-latency trade-off.

Why It Matters#

Approximate nearest-neighbor indexes intentionally trade accuracy for speed. The right settings depend on corpus size, vector dimension, filter selectivity, write rate, and latency target.

Core Concepts#

HNSW builds a navigable graph of nearby vectors. It often gives strong recall and low latency, but uses more memory and can be expensive to build. IVF partitions vectors into lists and searches a subset of them. It can be cheaper, but tuning probe count is critical.

Implementation#

Benchmark against exact search on a sample:

for each query:
  exact_top_10 = brute_force(query)
  ann_top_10 = index_search(query, params)
  recall = overlap(exact_top_10, ann_top_10) / 10

Tune for a target such as recall@10 >= 0.95 under your latency budget.

Example#

Assume you have 5 million support-doc chunks and a 120 ms retrieval budget. You test 100 real user queries against exact search and compare two ANN configurations:

HNSW ef_search=40   recall@10=0.91   p95=42ms
HNSW ef_search=100  recall@10=0.96   p95=83ms
IVF probes=8        recall@10=0.84   p95=31ms
IVF probes=32       recall@10=0.93   p95=76ms

If the product requires high answer quality, HNSW ef_search=100 is the better default. If the endpoint is autocomplete and latency matters more, the lower-latency setting may be acceptable. The key is that the decision is based on measured recall, not only speed.

Common Mistakes#

• Tuning only one query type.
• Measuring latency without measuring recall.
• Ignoring memory cost during HNSW build.
• Adding metadata filters that make the selected index ineffective.

Production Considerations#

Rebuild indexes after embedding model changes. Keep benchmark datasets from real queries, not synthetic examples, because retrieval difficulty varies by domain.

Security#

Filter by tenant or access policy before returning results. If the vector database cannot filter efficiently, partition indexes by tenant or sensitivity class.

Performance#

HNSW query-time settings usually trade latency for recall. IVF probe counts do the same. Tune using p95 and p99 latency, not only averages.

Summary#

Vector index tuning is a measurement problem. Compare HNSW and IVF with real queries, exact-search baselines, recall targets, and production latency budgets.