Tuning Elasticsearch kNN recall: num_candidates, oversampling and rescoring

Elasticsearch num_candidates sets how many nearest-neighbour candidates each shard collects before it keeps the top k. Raising it improves recall on int8_hnsw fields. On bbq_hnsw fields it barely helps past k × oversample, because only that many candidates get rescored against the raw vectors, so rescore_vector.oversample is the knob that moves recall there. In our sweep on Elasticsearch 9.5.5 with 57,638 FiQA vectors, the default bbq_hnsw query returned 77% of the true top 10; raising oversample to 8 with num_candidates: 1000 brought that to 96%.

Older guides predate the 9.1 change that made BBQ the default for 384+ dimension vectors and tied its default candidate count to oversampling, so their defaults are now wrong. Everything below comes from a run on docker.elastic.co/elasticsearch/elasticsearch:9.5.5 (Lucene 10.5.1, single node, security off) unless it is labelled otherwise.

What k, num_candidates and oversample control

The three parameters act at different stages of one kNN query. The knn query reference describes the flow: each shard retrieves num_candidates candidates, rescores the top k × oversample of them with the original vectors, and returns its top k to the coordinator, which merges them into a global top k.

When rescoring is on, Elasticsearch silently raises the candidate count: num_candidates becomes max(num_candidates, k × oversample). That one rule explains most of the results below.

sequenceDiagram
  participant C as Coordinator
  participant S as Shard
  participant G as Segment HNSW graphs
  C->>S: knn k=10, num_candidates, oversample
  S->>G: explore max of num_candidates and k x oversample
  G-->>S: candidates scored on quantized vectors
  S->>S: rescore top k x oversample on float vectors
  S-->>C: top k
  C->>C: merge shard results into global top k

Elasticsearch num_candidates defaults on 9.5.5

The 1.5 × k default is older than 9.1. Elastic explains why it chose it over max(100, k) in Simplifying kNN search. What 9.1 changed is the BBQ case. Elastic's 9.1 release post (July 2025) says "we set the default num_candidates for BBQ to max(1.5k, oversamplek)", which works out to 30 at k: 10.

We created both field types with only type set and read the mapping back:

{"t-bbq":{"mappings":{"properties":{"emb":{"type":"dense_vector","dims":384,"index":true,
  "similarity":"cosine","index_options":{"type":"bbq_hnsw","m":16,"ef_construction":100,
  "rescore_vector":{"oversample":3.0}}}}}},
 "t-int8":{"mappings":{"properties":{"emb":{"type":"dense_vector","dims":384,"index":true,
  "similarity":"cosine","index_options":{"type":"int8_hnsw","m":16,"ef_construction":100}}}}}}

bbq_hnsw carries a mapping-level oversample of 3.0. int8_hnsw has none, so it does not rescore unless the query asks. To confirm the effective candidate count on BBQ, we compared result sets: on both datasets, omitting num_candidates, setting it to 10 and setting it to 20 returned identical recall to oversample 3.0 with num_candidates: 30 (0.7738 on FiQA, 0.918 on SciFact).

The tuning page lists bbq_disk as the 9.4+ default "when the license allows". On self-managed 9.5.5 we still got bbq_hnsw; the BBQ post shows why.

The validation errors, and a doc conflict

The docs disagree on the oversample range. The dense_vector reference says "Greater than 1.0 and less than 10.0", while the knn query reference says >= 1f. 9.5.5 applies different rules to the mapping and the query:

The query has no upper bound, and the mapping accepts exactly 1.0 despite its own error text. Read the nested caused_by when a query fails; the root cause line for 0.5 tells you nothing.

Measuring recall against an exact script_score baseline

Recall@10 is the share of the true 10 nearest neighbours that the kNN query returns. The truth comes from an exact brute-force query, which Elastic's exact kNN guide writes as a script_score. It reads the raw float vectors, so it works on quantized fields too:

POST fiqa-bbq-hnsw/_search
{
  "size": 10,
  "_source": false,
  "query": {
    "script_score": {
      "query": { "match_all": {} },
      "script": {
        "source": "cosineSimilarity(params.qv, 'emb') + 1.0",
        "params": { "qv": [0.0213, -0.0472, "...384 floats"] }
      }
    }
  }
}

For the first FiQA query, the kNN search (num_candidates: 100, default oversample) reported took: 1 and the exact query took: 13, with the same 10 IDs in the same order. The scores differ by design: kNN with cosine returns (1 + cos) / 2, so its 0.9004 is the exact query's 1.8009 halved. Compare IDs, never scores. We also cross-checked the script_score top 10 against NumPy brute force on every query: overlap was 1.0.

flowchart LR
  Q([Query vectors]) --> E["Exact script_score<br/>top 10 per query"]
  Q --> K["knn query<br/>num_candidates x oversample grid"]
  E --> T[(Ground truth IDs)]
  K --> R[Returned IDs]
  T --> M["recall@10 = overlap / 10"]
  R --> M
  M --> O([Recall and latency per config])

The sweep script, trimmed to the parts that matter:

import json, time, statistics, requests

ES, K = "http://localhost:9212", 10
s = requests.Session()

def search(index, body):
    r = s.post(f"{ES}/{index}/_search", json=body)
    r.raise_for_status()
    return [h["_id"] for h in r.json()["hits"]["hits"]]

def exact(index, qv):
    return search(index, {"size": K, "_source": False, "query": {"script_score": {
        "query": {"match_all": {}},
        "script": {"source": "cosineSimilarity(params.qv, 'emb') + 1.0",
                   "params": {"qv": qv}}}}})

def knn(index, qv, nc, oversample):
    q = {"field": "emb", "query_vector": qv, "k": K}
    if nc is not None:
        q["num_candidates"] = nc
    if oversample is not None:
        q["rescore_vector"] = {"oversample": oversample}
    t = time.perf_counter()
    ids = search(index, {"size": K, "_source": False, "knn": q})
    return ids, (time.perf_counter() - t) * 1000

def sweep(index, queries, truth, nc, oversample):
    for qv in queries[:50]:                      # warm-up
        knn(index, qv, nc, oversample)
    recall, lat = [], []
    for qv, t in zip(queries, truth):
        ids, ms = knn(index, qv, nc, oversample)
        recall.append(len(set(ids) & set(t)) / K)
        lat.append(ms)
    lat.sort()
    return sum(recall) / len(recall), statistics.median(lat), lat[int(0.95 * len(lat))]

The sweep: setup and results

FiQA recall@10 as num_candidates grows:

xychart-beta
  title "FiQA recall@10 vs num_candidates, ES 9.5.5"
  x-axis ["10", "20", "50", "100", "200", "500", "1000"]
  y-axis "Recall@10" 0.7 --> 1.0
  line [0.7164, 0.8228, 0.909, 0.9398, 0.9594, 0.9678, 0.9704]
  line [0.8296, 0.8296, 0.9202, 0.9544, 0.976, 0.9868, 0.9896]
  line [0.7738, 0.7738, 0.8088, 0.839, 0.8492, 0.859, 0.8622]

Blue is int8_hnsw without rescoring, grey is int8_hnsw with oversample 2, yellow is bbq_hnsw at its default oversample of 3. The flat start on the grey and yellow lines is the max(num_candidates, k × oversample) floor.

Holding num_candidates at 1000 and moving only oversample on bbq_hnsw:

xychart-beta
  title "FiQA bbq_hnsw recall@10 by oversample, num_candidates 1000"
  x-axis ["0", "2", "3", "5", "8", "10"]
  y-axis "Recall@10" 0.5 --> 1.0
  bar [0.5986, 0.7924, 0.8622, 0.9234, 0.957, 0.9684]

SciFact, the smaller corpus, tells the same story at higher levels: int8_hnsw reached 0.982 at num_candidates: 100 without rescoring and 1.0 at 1000 with oversample 2; bbq_hnsw sat at 0.918 by default and 0.974 with oversample 5 and num_candidates: 100.

What the latencies show

At these corpus sizes on a laptop, the latency signal is small and noisy. In the main FiQA sweep, p50 stayed between 1.0 and 2.6 ms across every config with no clean trend. A second run going up to num_candidates: 10000 moved p50 from 3.7 ms at 100 to 5.6 ms at 10,000 on int8_hnsw, and from 3.4 to 5.1 ms on bbq_hnsw at oversample 10. The baseline itself shifted by 2 to 3 ms between those two runs, so trust only the within-run direction: more candidates cost more, and on 57k vectors the cost is a few milliseconds. On a production corpus with tens of millions of vectors the slope will be steeper. Measure there, with your own queries.

Reading the results

On int8_hnsw, num_candidates is the dial. int8 quantization caps recall at about 0.97 to 0.98 without rescoring; oversample 2 removes that cap, reaching 0.990 at 1000 candidates and 1.0 at 10,000.

On bbq_hnsw, oversample is the dial. Going from 100 to 10,000 candidates at the default oversample of 3 moved FiQA recall from 0.839 to 0.870, while going from oversample 3 to 8 at 1000 candidates moved it from 0.862 to 0.957. With 1-bit codes, the candidate ordering is coarse. True neighbours are usually found, then ranked outside the top k × oversample and never rescored. With rescoring off entirely, bbq_hnsw returned about 60% of the true top 10 at best.

Our recommendation: on bbq_hnsw, start at oversample 5 with num_candidates set to 10 × k, then raise oversample until recall meets your target. Set the final oversample in the mapping so every client gets it. On int8_hnsw, add oversample 2 and tune num_candidates. In both cases, measure against the script_score baseline before you change anything in production.

Other levers: segments, page cache and shards

Segments. Each segment has its own HNSW graph and is searched separately, so more segments means more total candidates. We indexed FiQA into bbq_hnsw with a refresh after every 1,000 docs, then force-merged:

Merging cut recall by 5.7 points at the default settings, because the same num_candidates now covers one graph instead of twelve. Re-run the recall test after any merge, and tune on the segment layout you serve. Elastic's production kNN guidance prefers raising index.merge.policy.max_merged_segment over force merging and notes kNN "can be up to several times faster if there are fewer segments".

Page cache. The same page says to preload only what must stay in RAM: for quantized HNSW, the codes (.veq or .veb) plus the graph (.vex), never raw .vec files. The vector sizing calculator gives the byte counts. Cold page cache will swamp any num_candidates effect.

Shards. num_candidates applies per shard, so five shards explore five times the candidates for the same query. Expect recall to rise with shard count while per-query work multiplies; we tested on one shard only. Fix shard count first, then tune.

The non-vector parts of the request matter too, such as filters and _source fetching. Our Elasticsearch query performance guide covers those, and hybrid search scoring explained covers fusing kNN with BM25. On OpenSearch the equivalent knobs differ; see our Elasticsearch vs OpenSearch comparison and the OpenSearch knn_vector mapping guide.

Checklist

• Read GET index/_mapping and note index_options.type and any rescore_vector
• Build exact script_score ground truth for 300 to 1,000 real queries
• Sweep on the segment layout and shard count you serve
• On bbq_hnsw, raise oversample first; keep num_candidates >= k × oversample
• On int8_hnsw, add oversample: 2 and raise num_candidates
• Record p50 and p95 on production hardware, not a laptop
• Put the chosen oversample in the mapping and re-test after merges or reindexing

Where we can help

We run this recall test on client clusters before changing index types or query settings, so the trade-off is measured on their data. Our Elasticsearch query performance work for AlphaSense made their search 90% faster; that engagement was general query tuning, separate from kNN. If your vector search returns plausible but wrong neighbours, see our Elasticsearch consulting service.