Vespa phased ranking: first-phase, second-phase and global-phase explained

Vespa phased ranking splits a rank profile into up to three expressions that run on shrinking sets of hits. first-phase runs on the content nodes for every matched document, second-phase reranks the best first-phase hits on each content node (100 per node by default), and global-phase runs in the stateless container on the best hits after all nodes are merged (100 per schema by default), which is where an ONNX cross-encoder belongs. Put cheap features in first-phase, priors and learned models in second-phase, and the expensive model in global-phase with a small rerank-count.

Everything below was run on Vespa 8.778.45 in Docker on 31 October 2026. The application had two content nodes (distribution keys 0 and 1) on one host with redundancy 1, so per-node behaviour is visible. The corpus is 24 short help-centre articles written for this post, the same fictional product as our reciprocal rank fusion post. The node split was 11 and 13 documents. Read the rankings as a demonstration of the mechanics. They are not a benchmark.

Cost per document decides the phase

A ranking expression costs about the same for each document it scores, so the question for each feature is how many documents can afford it. The phased ranking guide states the contract: the first-phase expression "is computed for every document retrieved by the query", second-phase is "evaluated for the 100 best hits from the first-phase ranking per content node", and global-phase "will run in the stateless container after gathering the best hits from the content node phases".

Defaults are from the schema reference. The reference recommends total-rerank-count over rerank-count for second-phase, since the total stays the same when you add content nodes.

The arithmetic for an illustrative cluster shows why the split matters. Take 2 content nodes, a query that matches 200,000 documents on each, second-phase rerank-count: 100 and global-phase rerank-count: 50:

flowchart TD
  M["Matching on 2 content nodes<br/>200,000 matches per node"] --> F["first-phase<br/>400,000 evaluations"]
  F --> S["second-phase<br/>rerank-count 100 per node = 200 evaluations"]
  S --> MG["Container merges hits<br/>sorted by second-phase score"]
  MG --> G["global-phase<br/>rerank-count 50 = 50 cross-encoder inferences"]
  G --> R["Response: offset + hits"]

Global-phase scores 1 document for every 8,000 that first-phase scored. Adding a third content node raises the second-phase count to 300 with rerank-count, and leaves it at the configured total with total-rerank-count. The global-phase count does not move with node count at all.

A Vespa rank profile with all three phases

The services.xml defines two components. The e5-small-v2 embedder from the Vespa samples produces the query and document vectors, and a hugging-face-tokenizer produces token ids for the cross-encoder. The tokenizer and model are cross-encoder/ms-marco-MiniLM-L-6-v2 as exported to ONNX on Hugging Face by Xenova, the same files the msmarco-ranking sample app downloads into models/.

<container id="default" version="1.0">
  <nodes><node hostalias="node1"/></nodes>
  <component id="e5" type="hugging-face-embedder">
    <transformer-model url="https://data.vespa-cloud.com/sample-apps-data/e5-small-v2-int8/e5-small-v2-int8.onnx"/>
    <tokenizer-model url="https://data.vespa-cloud.com/sample-apps-data/e5-small-v2-int8/tokenizer.json"/>
    <prepend>
      <query>query: </query>
      <document>passage: </document>
    </prepend>
  </component>
  <component id="tokenizer" type="hugging-face-tokenizer">
    <model path="models/tokenizer.json"/>
  </component>
  <document-api/>
  <search/>
</container>

<content id="docs" version="1.0">
  <redundancy>1</redundancy>
  <documents>
    <document type="doc" mode="index"/>
  </documents>
  <nodes>
    <node hostalias="node1" distribution-key="0"/>
    <node hostalias="node1" distribution-key="1"/>
  </nodes>
</content>

The schema stores an updated timestamp (seconds since epoch) and a views count as attributes, embeds title plus body into an HNSW-indexed tensor, and tokenizes the same text into a tokens attribute so the cross-encoder never re-tokenizes documents at query time:

schema doc {
    document doc {
        field title type string {
            indexing: index | summary
            index: enable-bm25
        }
        field body type string {
            indexing: index | summary
            index: enable-bm25
        }
        field updated type long {
            indexing: attribute | summary
        }
        field views type int {
            indexing: attribute | summary
        }
    }

    field embedding type tensor<float>(x[384]) {
        indexing: input title . " " . input body | embed e5 | attribute | index
        attribute {
            distance-metric: angular
        }
    }

    field tokens type tensor<float>(d0[128]) {
        indexing: input title . " " . input body | embed tokenizer | attribute
    }

    fieldset default {
        fields: title, body
    }

    onnx-model cross_encoder {
        file: models/model.onnx
        input input_ids: input_ids
        input attention_mask: attention_mask
        input token_type_ids: token_type_ids
    }

    rank-profile phased {
        inputs {
            query(q) tensor<float>(x[384])
            query(q_tokens) tensor<float>(d0[32])
        }

        function lexical() {
            expression: bm25(title) + bm25(body)
        }
        function semantic() {
            expression: closeness(field, embedding)
        }
        function fresh() {
            expression: freshness(updated)
        }
        function popular() {
            expression: log10(1 + attribute(views))
        }

        function input_ids() {
            expression: tokenInputIds(128, query(q_tokens), attribute(tokens))
        }
        function token_type_ids() {
            expression: tokenTypeIds(128, query(q_tokens), attribute(tokens))
        }
        function attention_mask() {
            expression: tokenAttentionMask(128, query(q_tokens), attribute(tokens))
        }
        function ce_score() {
            expression: onnx(cross_encoder){d0:0,d1:0}
        }

        first-phase {
            expression: lexical + 10 * semantic
        }
        second-phase {
            expression: firstPhase + 2 * fresh + 0.5 * popular
            rerank-count: 3
        }
        global-phase {
            expression: ce_score
            rerank-count: 4
        }
        match-features {
            lexical
            semantic
            fresh
            popular
            firstPhase
            firstPhaseRank
        }
        summary-features {
            attribute(views)
        }
    }
}

The windows of 3 and 4 are deliberately tiny so a 24-document corpus shows them. Production values are discussed below.

Our first deploy named the function cross_encoder, the same as the onnx-model, and the config server rejected it:

Invalid application: Invalid schema 'doc': Cycle in ranking expression function 'cross_encoder' called from: [cross_encoder[]]

onnx(cross_encoder) resolved to the function rather than the model. Give the wrapper function a different name.

The rank features in each phase

Definitions are from the rank features reference. freshness lives in second-phase here because it is a prior: it should reorder close candidates and has no business choosing them. bm25 is cheap enough for every match; the reference describes it as "about 3-4 times faster than nativeRank".

The query ORs a weakAnd text match with a nearest-neighbour match, and fills both query tensors in the container:

POST /search/
{
  "yql": "select title, views from doc where userQuery() or ({targetHits:10}nearestNeighbor(embedding, q))",
  "query": "why are my webhook deliveries failing",
  "type": "weakAnd",
  "ranking": "phased",
  "input.query(q)": "embed(e5, @query)",
  "input.query(q_tokens)": "embed(tokenizer, @query)",
  "hits": 10
}

Second-phase rerank-count is per content node

The first hit of the response, with its matchfeatures:

{
  "id": "index:docs/1/cfcd20844a73891c0bfe68f1",
  "relevance": 6.13296365737915,
  "source": "docs",
  "fields": {
    "matchfeatures": {
      "firstPhase": 10.171252803417815,
      "firstPhaseRank": 1.0,
      "fresh": 0.944383487654321,
      "lexical": 3.3287672128505252,
      "popular": 3.079543007402906,
      "semantic": 0.6842485590567289
    },
    "title": "Webhook deliveries failing with 410 Gone",
    "views": 1200
  }
}

firstPhaseRank is the tell. Across the 21 matches, exactly six hits carried a value, and the values were 1, 2, 3 twice over: three per node, as rerank-count: 3 says. The other 15 returned the feature's default (the maximum double) because second-phase never scored them. Here is the top six, with the second-phase score taken from the same query run with ranking.globalPhase.rerankCount: 0:

You can check the second-phase column by hand. For the top hit, 10.1713 + 2 × 0.9444 + 0.5 × 3.0795 = 13.5999.

Second-phase reordered the candidates on priors. "Webhook retry schedule" (12 days old) rose from fifth in the merged first-phase order to second, and "Troubleshoot failed webhook deliveries" fell from second to fourth, because at 200 days its freshness is 0. "Monitor your usage quota" held third on a 35-day-old timestamp and 2,200 views. Global-phase took the top four by second-phase score, ran the cross-encoder on them, and returned a logit of -2.52 for that article, which pushed it below the three that answer the question. Hits 5 and 6 kept their second-phase order but were rescaled under the worst cross-encoder score (-4.40 and -6.26 in the response). The schema reference documents this: hits not reranked "might be re-scored using a linear function to avoid a greater rank score than the worst reranked hit". The global-phase trace at trace.level: 3 confirms the window: Use global-phase from [Optional[schema 'doc']] to re-rank 4 hits.

total-rerank-count is split by document share

We then overrode the window per query and counted the hits with a firstPhaseRank:

rerankCount multiplies by node count. totalRerankCount came out one above the total on every run. The reference only says each node gets "the node's share of total-rerank-count". The node metrics showed 11 and 13 active documents, and rounding up each node's proportional share reproduces every row: for a total of 4, 4 × 13/24 = 2.17 rounds up to 3, and 4 × 11/24 = 1.83 rounds up to 2. We did not find the formula in the source, so treat it as observed behaviour on 8.778.45. The practical point holds either way: total-rerank-count is approximately the total, and it does not double when you double the cluster.

Global-phase rerank-count counts differently again: per schema, after the merge, independent of node count. Our RRF post covers the trap of a window too small for the fusion inputs, so we won't repeat it here.

ONNX ranking in global-phase

Raising the global window from 4 to 10 brought "Allowlist our IP addresses" (a firewall blocking deliveries) to fourth with a logit of 0.81, and "Release notes: September" (delivery logs) to fifth at -1.54. Both moved above "Monitor your usage quota". This is the reason to have a cross-encoder at all: it reads query and document together, and it disagreed with the popularity and freshness priors.

Cost is what bounds the window. The cross-encoder guide says inference cost is linear in rerank-count and quadratic in sequence length, and it also grows with model size. Its examples use rerank-count: 25 and recommend a GPU for models above 30 to 40M parameters. Our sequence length of 128 tokens (via tokenInputIds(128, ...)) covers short help articles; long passages need chunking or a larger limit. Note that tokenInputIds assumes BERT special tokens 101 and 102. For a RoBERTa-based model such as BAAI/bge-reranker-base, the guide uses customTokenInputIds(1, 2, ...) and drops token_type_ids.

We did not measure latency per window size on this 24-document corpus, because a laptop number for 4 versus 10 inferences tells you nothing about your hardware. Measure it on your cluster with presentation.timing: true while you vary ranking.globalPhase.rerankCount.

Debugging with match-features and summary-features

The two feature lists look alike and are computed at different times in the query protocol:

sequenceDiagram
  participant Ctr as Container
  participant N as Content nodes
  Ctr->>N: match protocol phase
  N->>N: first-phase on all matches, second-phase on top hits
  N-->>Ctr: top hits with match-features
  Ctr->>Ctr: merge, global-phase on top rerank-count
  Ctr->>N: fill protocol phase for the final page
  N-->>Ctr: document summaries with summary-features

match-features are computed in the match phase and travel with each hit, so global-phase can reuse them, and the values are the ones the ranking actually used. summary-features are recalculated in the fill phase for the hits that made the final page. In our run, summary-features { attribute(views) } only appeared with select *: the response held "attribute(views)": 1200.0 plus "vespa.summaryFeatures.cached": 0.0, and it was absent from the select title, views response above. Both lists accept only rank features or function names. The reference notes that compound expressions must be wrapped in a function, which is why lexical and fresh are functions here.

For a one-off look at everything, ranking.listFeatures: true dumps the profile's rank features. For day-to-day work, keep a match-features list that names every input of every phase, plus firstPhaseRank, in a debug profile that inherits the production one.

Evaluating a phased ranking change

1. Log match-features for a sample of real queries, so a ranking regression can be traced to the input that moved.
2. Change one phase at a time. A first-phase change alters which hits reach second-phase, so it can move results that the later phases were already sorting correctly.
3. For each window, compare the top 10 at the production count and at 10 times that count with the query overrides. If the top 10 changes, the window is cutting real candidates. The same candidate-depth question exists for kNN in Elasticsearch, see num_candidates and oversampling.
4. Score the variants on labelled queries before shipping. The metrics and judgement process in our relevancy techniques post apply unchanged.

Our recommendation for a new Vespa rank profile: bm25 plus closeness in first-phase, priors in second-phase with total-rerank-count sized from your evaluation (start at the default of 100 per node), and a cross-encoder in global-phase with rerank-count between 25 and 100, set by measured latency. Teams coming from Elasticsearch tend to ship a single first-phase expression. That works, and it leaves Vespa's main lever unused. The Elasticsearch vs Vespa comparison covers where that lever is worth a migration, and the hybrid search scoring explainer and hybrid search playground cover the fusion side.

Where we can help

We design Vespa rank profiles phase by phase and size each window against labelled queries and a latency budget, including ONNX rerankers in global-phase. See our Vespa consulting service.