Reciprocal rank fusion in Vespa: hybrid search in global-phase, and the rerank-count trap

Vespa reciprocal rank fusion is a global-phase expression: reciprocal_rank_fusion(text_score, closeness(field, embedding)) inside a global-phase block of a rank profile, with a YQL query that ORs userQuery() and nearestNeighbor. The fusion only sees the top rerank-count hits, which default to 100 per schema in total (not per content node), chosen by your first-phase expression. Every hit below that line keeps its first-phase order and is never fused, so a low rerank-count or a one-sided first-phase silently removes documents that only one retriever liked.

Everything below was run on Vespa 8.778.45 in Docker on 31 October 2026, with the e5-small-v2 embedder from the official samples. The corpus is 17 short help-centre articles written for this post, the same fictional cloud product as our hybrid search scoring post, so read the rankings as a worked demonstration of the mechanics. They are not a benchmark.

What global-phase is and where it runs

Vespa ranks in up to three phases. First-phase runs on the content nodes for every matched document. Second-phase, if you define one, reranks the best first-phase hits on each content node. Global-phase runs in the stateless container after the hits from all content nodes are merged, which is why it can do cross-hit work like normalization and rank fusion.

The December 2023 newsletter dates global-phase to Vespa 8.246, alongside the normalizers normalize_linear, reciprocal_rank and reciprocal_rank_fusion. Those three are only valid in global-phase. You cannot rank-fuse on a content node, because a content node only sees its own documents.

flowchart LR
  Q["Query: userQuery OR nearestNeighbor"] --> CN1["Content node 1<br/>match + first-phase"]
  Q --> CN2["Content node 2<br/>match + first-phase"]
  CN1 --> M["Container merges hits<br/>by first-phase score"]
  CN2 --> M
  M --> W["Top rerank-count hits<br/>default 100 per schema"]
  M --> R["All other fetched hits"]
  W --> G["global-phase:<br/>reciprocal_rank_fusion"]
  R --> S["Linear rescale of<br/>first-phase score"]
  G --> P["Sort, then trim to<br/>offset + hits"]
  S --> P

The Vespa hybrid search setup

The embedder goes in services.xml. This is the e5-small-v2 ONNX model the pyvespa getting-started guide uses, with the query: and passage: prefixes that e5 expects, configured through <prepend> as the embedding reference shows:

<container id="default" version="1.0">
  <component id="e5" type="hugging-face-embedder">
    <transformer-model url="https://data.vespa-cloud.com/sample-apps-data/e5-small-v2-int8/e5-small-v2-int8.onnx"/>
    <tokenizer-model url="https://data.vespa-cloud.com/sample-apps-data/e5-small-v2-int8/tokenizer.json"/>
    <prepend>
      <query>query: </query>
      <document>passage: </document>
    </prepend>
  </component>
  <document-api/>
  <search/>
</container>

The schema has two BM25-enabled text fields and a synthetic tensor field that embeds title plus body at feed time, indexed with HNSW:

schema doc {
    document doc {
        field title type string {
            indexing: index | summary
            index: enable-bm25
        }
        field body type string {
            indexing: index | summary
            index: enable-bm25
        }
    }

    field embedding type tensor<float>(x[384]) {
        indexing: input title . " " . input body | embed e5 | attribute | index
        attribute {
            distance-metric: angular
        }
        index {
            hnsw {
                max-links-per-node: 16
                neighbors-to-explore-at-insert: 100
            }
        }
    }

    fieldset default {
        fields: title, body
    }

    rank-profile bm25 {
        inputs {
            query(q) tensor<float>(x[384])
        }
        function text_score() {
            expression: bm25(title) + bm25(body)
        }
        first-phase {
            expression: text_score
        }
    }

    rank-profile rrf inherits bm25 {
        first-phase {
            expression: closeness(field, embedding)
        }
        global-phase {
            expression: reciprocal_rank_fusion(text_score, closeness(field, embedding))
            rerank-count: 100
        }
        match-features {
            text_score
            closeness(field, embedding)
        }
    }
}

The rrf profile mirrors the official examples. Both the learn.vespa.ai hybrid search lesson and the pyvespa getting-started guide use closeness(field, embedding) as first-phase and fuse it with a BM25 sum in global-phase. The lesson sets rerank-count: 100, and pyvespa sets rerank_count=1000. Neither explains the difference, and that difference is the subject of this post. I added match-features only so the response shows both inputs per hit.

The query ORs a lexical match with an approximate nearest-neighbour match and embeds the query text in the container:

POST /search/
{
  "yql": "select * from doc where userQuery() or ({targetHits:10}nearestNeighbor(embedding, q))",
  "query": "X-RateLimit-Remaining header",
  "ranking": "rrf",
  "input.query(q)": "embed(e5, @query)",
  "hits": 10
}

Note that targetHits is per content node. The YQL reference recommends totalTargetHits, which is resolved across the nodes in a group.

How rerank-count actually works

The phased ranking guide states that "By default, global-phase ranking runs on the 100 globally best hits for a schema". The schema reference adds that the default is applied to one schema at a time, so a query over two schemas may rerank 100 hits per schema. You can override it per request with ranking.globalPhase.rerankCount, which is what we used for the tests below.

Global-phase and second-phase count differently, and the names make it easy to mix them up:

Two details are missing from the guide, so we read the container source in the vespa-engine repository. In ClusterSearcher, the container asks the content nodes for max(offset + hits, rerankCount) hits, reranks the first rerankCount of them, then trims back to your page. In ResultReranker, the hits that were fetched but not reranked are rescaled with a linear function (RangeAdjuster) so that none scores above the worst reranked hit. Their relative order is still their first-phase order. The schema reference documents this rescoring for second-phase, and its global-phase rank-score-drop-limit entry mentions "a re-scored rank score", which matches what the code does.

sequenceDiagram
  participant C as Client
  participant Ctr as Container
  participant N as Content nodes
  C->>Ctr: hits=10, rerankCount=5
  Ctr->>N: fetch max of offset+hits and rerankCount, here 10
  N-->>Ctr: 10 hits sorted by first-phase, with match-features
  Ctr->>Ctr: rank text_score and closeness within the top 5 only
  Ctr->>Ctr: RRF score for those 5
  Ctr->>Ctr: rescale first-phase score of hits 6 to 10
  Ctr-->>C: 10 hits

The rerank-count trap, on a real query

The query X-RateLimit-Remaining header has one obvious answer, a release-notes article that mentions the header in passing. Its first-phase inputs, from match-features:

"Use the REST API" is a lexical-only document: third for BM25, last of the 11 matches for vector closeness. Because first-phase is closeness, it sorts last in the merged list. Here are the top seven at two rerank-count values:

With 100, all 11 matches are fused and "Use the REST API" ranks seventh. With 5, it is outside the window, is never fused, keeps its last-place first-phase position and drops out of the top 10 entirely. The trace at trace.level: 3 confirms the window: Use global-phase from [Optional[schema 'doc']] to re-rank 5 hits.

The arithmetic matches the docs' formula, 1 / (k + rank) with k = 60:

Two things in that table are easy to miss.

First, ranks are computed inside the window. ERR_429 is fourth for BM25 across all matches but third within the top 5, because the REST API article is no longer there to outrank it. Changing rerank-count changes the RRF scores of the hits that do get fused.

Second, a document with text_score 0 still earns text-side credit. "Cancel your subscription" contains none of the query terms and matched through nearestNeighbor only, yet reciprocal_rank(text_score) gave it rank 4 in the window and 1/64. Vespa ranks every hit in the window on every input, with ties taking consecutive ranks. That differs from list-based RRF in Elasticsearch or OpenSearch, where a document absent from one retriever's list gets nothing from it.

The hits outside the window are rescaled as the source describes. Both ranges are below 1, so RangeAdjuster uses a scale of 1 and shifts each score by the gap between the lowest reranked first-phase score (0.603524) and the lowest RRF score (0.030769). "API rate limits by plan" has closeness 0.603320, so it lands at 0.603320 - 0.603524 + 0.030769 = 0.030565, just under ERR_500. Those positions are pure vector order.

On a 17-document corpus a window of 5 looks contrived. The same arithmetic applies to a million-document index with the default 100: userQuery() can match tens of thousands of documents, and the hits outside the top 100 by closeness never reach the fusion.

Fixing it: first-phase picks the candidates

rerank-count is a candidate-selection setting, and the first-phase expression is what fills the window. With closeness alone in first-phase, lexical-only hits are always the first to be cut. We added a profile whose first-phase combines both signals:

rank-profile rrf_sum_first inherits rrf {
    first-phase {
        expression: text_score + 10 * closeness(field, embedding)
    }
}

The weight 10 is a rough scale match for this corpus, nothing more. With rerankCount: 5, this profile put "Use the REST API" into the window and it ranked fifth (0.031258). The fused order then matched the rerankCount: 100 result except that the REST API article moved up from seventh, because it now competes only against window members. The first-phase sum never reaches the user here. It only decides which hits get fused.

The other lever is the count itself. The cost of a larger rerank-count is that the content nodes return more hits with their match-features, and the container evaluates the global-phase expression for each. For RRF the expression is cheap arithmetic. We did not measure latency at different values here, so measure it on your own cluster before picking 1,000 because pyvespa does.

normalize_linear instead of RRF

normalize_linear(x) rescales an input to [0, 1] over the hits in the window, with (x - min) / (max - min). A weighted version of our profile:

global-phase {
    expression: 0.5 * normalize_linear(text_score) + 0.5 * normalize_linear(closeness(field, embedding))
    rerank-count: 100
}

It has the same window dependence as RRF, and min and max move with it. On our query, ERR_429 scored 0.201499 with 100 hits and 0.146638 with 5. With 5, ERR_500 normalized to exactly 0 on closeness and the hits below it went negative after rescaling (-0.000204 for the next one). Negative scores are expected here, but they will trip any client code that treats scores as probabilities.

Our recommendation is the same as on Elasticsearch and OpenSearch: start with RRF, and move to weighted normalization only once you can measure the weights.

How to evaluate your rerank-count

Use the query-time override to compare windows without redeploying:

{ "ranking": "rrf", "ranking.globalPhase.rerankCount": 1000, "hits": 10 }

1. Take a sample of real queries, with both identifier lookups and paraphrased questions in it.
2. Run each at your production rerank-count and at ten times that value.
3. Count how often the top 10 differs. If it differs, your window is cutting real candidates. Raise the count or fix first-phase before you tune anything else.
4. Check which retriever the lost hits came from. If they are all lexical-only or all vector-only, the cause is first-phase, and more hits only hides it.
5. Keep targetHits (or totalTargetHits) and rerank-count consistent. With targetHits per node and several content nodes, the vector side alone can supply more candidates than a global window of 100 can hold.

The same mechanics apply to the other engines we cover. Elasticsearch has rank_window_size for its RRF retriever, as our hybrid scoring comparison shows, and the hybrid search playground lets you change window size and fusion method and watch the order move. Candidate depth matters even more once vectors are quantized, as in the BBQ HNSW vs BBQ disk comparison and the Qdrant TurboQuant guide, because the approximate candidates are less exact to begin with. If you are weighing Vespa against Elasticsearch for hybrid ranking in general, the Elasticsearch vs Vespa comparison covers that, and for RAG, see why an agent is only as good as its retriever.

Where we can help

We design Vespa rank profiles, tune first-phase and rerank-count against labelled queries, and set up the evaluation so changes show up as numbers before they ship. See our Vespa consulting service.