Vector Indices (HNSW)

Meaning as Mathematical Coordinates

How do you explain the meaning of "Queen" to a computer? You don't. You convert it into coordinates. Modern embedding models (like OpenAI's text-embedding-3 or BERT) convert text into "vectors" fixed-length arrays of numbers that position concepts in a multi-dimensional semantic space. In this space, similarity isn't about shared letters; it's about proximity. "Cat" and "Kitten" are neighbors, even though they share no characters.

The Brute Force Problem

To find the closest match in this space, the naive approach is to measure the distance from your query vector to every single document vector in your database. This is called "Exact kNN". It works perfectly for small datasets, but the math is heavy. With 100 million documents and 768 dimensions per vector, a single query requires ~76 billion floating-point operations. That takes seconds far too slow for search.

HNSW: Hierarchical Navigable Small World

To solve the scalability problem, we trade a tiny bit of accuracy for massive speed. HNSW is the state-of-the-art algorithm for this. It builds a multi-layered graph, similar to a highway system. The top layers are "expressways" with few stops (long links) that let you jump across the semantic universe quickly. As you get closer to your destination, you drop down to "local roads" (dense connections) to find the exact neighborhood.

What is Actually Stored? (HNSW Internals)

It's easy to think of an index as "just the vectors". But HNSW is a graph. On disk, it stores:

How is it built? One vector at a time. When you insert a document, the system "searches" for its nearest neighbors in the existing graph and draws edges to them. This greedy construction is fast but approximate.

Where It Fits in the Search System

A common misconception is that vector search replaces the whole search engine. It doesn't. It replaces the Recall Stage (generating candidates).

Vector Search in Elasticsearch

Elasticsearch supports vector search natively through the dense_vector field type. You need to generate embeddings externally (using models like sentence-transformers or OpenAI) and store them alongside your documents. Here's how to configure the mapping and run a kNN query.

PUT /products
{
  "mappings": {
    "properties": {
      "embedding": {
        "type": "dense_vector",
        "dims": 768,           // Match your model output
        "index": true,          // Enable HNSW indexing
        "similarity": "cosine"  // or "dot_product", "l2_norm"
      }
    }
  }
}

POST /products/_search
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.12, 0.45, -0.33, ...],  // 768 dims
    "k": 10,                                // Return top 10
    "num_candidates": 100                  // Speed vs accuracy
  }
}

HNSW Parameters (Tuning Guide)

HNSW has three knobs that control the "Triangle of Trade-offs": Speed, Accuracy (Recall), and Memory. Understanding these is key to tuning your cluster.

Hybrid Search: Keywords + Vectors

{
  "query": {
    "match": { "title": "wireless keyboard", "boost": 0.3 }
  },
  "knn": {
    "field": "title_vector",
    "query_vector": [...],
    "k": 50,
    "boost": 0.7
  }
}
// final_score = 0.3 × BM25 + 0.7 × vector_similarity

Memory & Quantization

Vectors are heavy. A single 768-dimensional vector takes 3KB (floats). Multiply by 100 million documents, and you need 300GB of RAM just for vectors. This gets expensive fast.Quantization solves this by compressing 32-bit floats into 8-bit integers (or even 1-bit binaries), sacrificing a tiny bit of precision for massive memory savings.

Common Pitfalls

Implementing vector search is deceptive. It looks easy ("just index the vectors!"), but the complexity hides in the operational details. Here are the two ways teams most often destroy their search relevance.

When Vector Search Fails

Vector search is magical, but it's not a silver bullet. If you remove keyword search entirely, you will likely degrade the user experience in these specific scenarios:

Performance Benchmarks

How fast is HNSW? Incredibly fast. Because search is logarithmic `O(log N)`, doubling the data size only adds a tiny constant amount of time. You can search billions of vectors in milliseconds.