Elasticsearch Query Performance Optimization: A Complete Guide

The fastest ways to speed up Elasticsearch queries are to put non-scoring clauses in filter context, map exact-match fields as keyword, return only the fields and results you need, paginate with search_after instead of deep from/size, keep shards between 10GB and 50GB, and use the Profile API to find what is actually slow. This guide covers each technique, from first principles to advanced settings, with configuration examples.

Why does query performance matter?

Slow queries hurt in two places at once. Users notice latency in search boxes, autocomplete and dashboards, and slow queries consume more CPU and memory per request, so you need more nodes to serve the same traffic. Measure your own latency percentiles (p50, p95, p99) before changing anything, so you know which changes help.

How does Elasticsearch execute a query?

Before optimizing, understand how Elasticsearch executes queries:

1. Query phase

1. The coordinating node receives the query
2. The query is sent to a copy of every relevant shard
3. Each shard executes the query locally
4. Each shard returns its top N document IDs and scores to the coordinator

2. Fetch phase

1. The coordinator merges the shard results and picks the final top documents
2. Fetch requests are sent to the shards holding those documents
3. Full documents are assembled
4. Results are returned to the client

Most optimizations below either make the query phase cheaper per shard, touch fewer shards, or make the fetch phase move less data.

What are the top 10 query optimization techniques?

1. Use filter context instead of query context

Before (scores clauses that don't need scoring):

{
  "query": {
    "bool": {
      "must": [
        { "term": { "status": "active" }},
        { "range": { "price": { "gte": 100 }}}
      ]
    }
  }
}

After (filter context):

{
  "query": {
    "bool": {
      "filter": [
        { "term": { "status": "active" }},
        { "range": { "price": { "gte": 100 }}}
      ]
    }
  }
}

Filter context:

• Is a yes/no match with no relevance score calculated, so it runs faster than query context
• Is cached automatically when a filter is used frequently
• Is the right place for exact matches, ranges and existence checks

2. Optimize field data types

Choosing the right field type has a large performance impact:

Text vs keyword:

• text: Use for full-text search (analyzed)
• keyword: Use for exact matching, aggregations and sorting

Example mapping:

{
  "mappings": {
    "properties": {
      "email": { "type": "keyword" },
      "description": { "type": "text" },
      "status": { "type": "keyword" },
      "price": { "type": "float" },
      "created_at": { "type": "date" }
    }
  }
}

3. Reduce result set size

Use _source filtering:

{
  "_source": ["id", "title", "price"],
  "query": { "match_all": {} }
}

Returning only the fields you need reduces the work in the fetch phase and the bytes sent over the network, which matters most for large documents. Likewise, keep size to what you will display.

4. Leverage caching

Elasticsearch has several caches worth understanding:

Node query cache:

• Caches the results of frequently used filters
• Automatically managed, enabled by default (index.queries.cache.enabled)

Shard request cache:

• Caches the results of whole search requests per shard
• By default only caches requests with size=0 (aggregations, hit counts, suggestions), which makes it a natural fit for dashboards
• Use ?request_cache=true to cache other requests explicitly
• Invalidated automatically when the index refreshes, so it helps most on indices that don't change often

Filesystem cache:

• Not an Elasticsearch cache, but the most important one: leave plenty of memory outside the heap so hot index data stays in the OS page cache

Field data cache:

• Used for aggregations and sorting on text fields (avoid this; use keyword)
• Monitor with: GET /_stats/fielddata

5. Optimize aggregations

Request only as many buckets as you will use:

Expensive aggregation:

{
  "aggs": {
    "categories": {
      "terms": { "field": "category", "size": 10000 }
    }
  }
}

Cheaper aggregation:

{
  "size": 0,
  "aggs": {
    "categories": {
      "terms": { "field": "category", "size": 100 }
    }
  }
}

Setting "size": 0 skips fetching hits you don't need and makes the request eligible for the shard request cache. If you need every bucket, page through them with a composite aggregation instead of one huge terms aggregation.

6. Paginate deep result sets with search_after and a point in time

Don't do this for large datasets:

{ "query": { "match_all": {} }, "from": 9000, "size": 1000 }

By default you can't page past 10,000 hits with from and size, and deep pages get more expensive because every shard must collect from + size results. Elastic no longer recommends the scroll API for deep pagination. Use search_after with a point in time (PIT) instead:

POST /my-index/_pit?keep_alive=1m

GET /_search
{
  "size": 100,
  "query": { "match_all": {} },
  "pit": { "id": "<pit_id>", "keep_alive": "1m" },
  "sort": [{ "@timestamp": "asc" }]
}

For each next page, pass the sort values of the last hit as search_after. The PIT keeps results consistent while you page and adds a tiebreaker sort automatically.

7. Avoid wildcard queries on text fields

Very slow on large indices:

{ "query": { "wildcard": { "description": "*search*" }}}

Better alternatives:

• Use index_prefixes on the field to make prefix queries fast
• Use an n-gram tokenizer for infix matching
• Use match_phrase_prefix or search_as_you_type for type-ahead
• Use the completion suggester for autocomplete

8. Optimize index structure

Shard sizing:

• Aim for shard sizes between 10GB and 50GB, and fewer than 200 million documents per shard
• Too many small shards add overhead to every search and to the cluster state
• Too few very large shards limit parallelism and slow down recovery

Rather than a formula, start from your data size and growth rate, pick a shard count that lands in that range, and use rollover or ILM for time-based data so shards stay in range as data grows.

9. Use the Profile API

Identify bottlenecks:

{
  "profile": true,
  "query": { "match": { "title": "elasticsearch" }}
}

Look for:

• Which query component dominates the time (build_scorer, next_doc, advance, score)
• Expensive rewrites, such as wildcard or fuzzy queries expanding to many terms
• Large collector or aggregation times

Kibana's Search Profiler visualizes the same output.

10. Keep scripts out of the hot path

Script-based sorting, script aggregations and script_score run per document. Where you can, compute values at index time (an ingest pipeline or your indexing code) and store them as regular fields.

How do you monitor query performance?

Key metrics to track

# Query latency
GET /_nodes/stats/indices/search

# Cache hit rates
GET /_stats/request_cache,query_cache

# Slow query log
PUT /my-index/_settings
{
  "index.search.slowlog.threshold.query.warn": "500ms",
  "index.search.slowlog.threshold.query.info": "200ms"
}

Alerting thresholds

Set thresholds from your own baseline rather than generic numbers. A useful pattern:

• Alert when p95 or p99 query latency rises well above its normal range for that time of day
• Watch cache hit rates and evictions after mapping or query changes
• Track search thread pool queue and rejections, which show when the cluster can't keep up
• Track field data memory and evictions, which usually point to aggregating on the wrong field type

What are the advanced optimization techniques?

1. Index sorting

Pre-sort segments to speed up conjunctions and queries sorted on the same field, at the cost of somewhat slower indexing:

{
  "settings": {
    "index": {
      "sort.field": "timestamp",
      "sort.order": "desc"
    }
  }
}

2. Adaptive replica selection

Elasticsearch routes each search to the best-performing shard copy by default (cluster.routing.use_adaptive_replica_selection). Leave it on; if you see uneven load, check whether someone disabled it.

3. Force-merge read-only indices

For indices that no longer receive writes, such as older time-based indices, force-merging to a single segment makes searches more efficient. Don't force-merge indices you still write to.

4. Move rarely searched data to the cold or frozen tier

The old _freeze API was removed in Elasticsearch 8.0, and 9.x can no longer read frozen indices at all. For data you rarely query, use ILM to move it to the cold or frozen data tiers, where the frozen tier serves searches from searchable snapshots.

What are the most common pitfalls?

1. Deep pagination with from/size: Use search_after with a PIT
2. Too many shards: Each shard adds overhead. Aim for 10-50GB per shard
3. Aggregating on text fields: Use keyword (or a keyword subfield)
4. Hand-tuning the heap: Elasticsearch sizes the heap automatically by default. If you set it manually, keep it at or below 50% of memory and below the compressed-oops threshold (26GB is safe on most systems)
5. Scripts in production queries: Precompute at index time where you can

How do you benchmark performance?

Rally - the official benchmarking tool

# Install
pip3 install esrally

# Benchmark an existing cluster
esrally race --track=geonames --target-hosts=localhost:9200 --pipeline=benchmark-only

Custom load testing

A quick sanity check with the Python client (use Rally for real load tests):

import time
from elasticsearch import Elasticsearch

es = Elasticsearch('http://localhost:9200')

start = time.time()
for i in range(1000):
    es.search(index='products', query={'match': {'title': 'laptop'}})
end = time.time()

print(f'1000 queries in {end-start:.2f}s')
print(f'Average: {(end-start)/1000*1000:.2f}ms per query')

What about time-series data and runtime fields?

1. Use a time series data stream (TSDS) for metrics

PUT _index_template/my-metrics-template
{
  "index_patterns": ["metrics-myapp-*"],
  "data_stream": {},
  "priority": 500,
  "template": {
    "settings": {
      "index.mode": "time_series"
    },
    "mappings": {
      "properties": {
        "host.name": { "type": "keyword", "time_series_dimension": true },
        "cpu.usage": { "type": "half_float", "time_series_metric": "gauge" },
        "@timestamp": { "type": "date" }
      }
    }
  }
}

A TSDS routes all data points of a time series to the same shard using dimension fields. Elastic reports that in its benchmarks, metrics stored in a TSDS used 70% less disk space than a regular data stream, and notes that the exact impact varies by dataset.

2. Use runtime fields for flexibility, not speed

{
  "runtime_mappings": {
    "day_of_week": {
      "type": "keyword",
      "script": {
        "source": "emit(doc['@timestamp'].value.getDayOfWeekEnum().toString())"
      }
    }
  }
}

Runtime fields are computed by a script at query time. They are great for exploring data or adding a field without reindexing, but they cost more per query than an indexed field. If a runtime field becomes part of a hot query, index it.

When should you get expert help?

Consider professional Elasticsearch consulting if:

• Queries consistently miss your latency targets despite the basics above
• Cluster CPU or heap stays saturated under normal traffic
• Out-of-memory errors persist despite heap tuning
• You can't handle traffic spikes
• You're planning a major version upgrade, for example to 9.x

Conclusion

Query performance optimization is not a one-time task; it's an ongoing process:

1. Monitor continuously: Track p95/p99 latencies
2. Optimize iteratively: Start with highest-impact changes
3. Test rigorously: Use Rally or custom benchmarks
4. Document changes: Know what worked and why

Next steps

1. Run the Profile API on your slowest queries
2. Move non-scoring clauses to filter context
3. Fix field mappings (text vs keyword)
4. Enable request caching for repeated aggregation queries
5. Set up monitoring and alerting

Need help optimizing your Elasticsearch cluster? See our Elasticsearch consulting services.

Sources

• Elastic documentation, Query and filter context
• Elastic documentation, Tune for search speed
• Elastic documentation, Size your shards
• Elastic documentation, Shard request cache
• Elastic documentation, Paginate search results
• Elastic documentation, Search shard routing
• Elastic documentation, JVM settings
• Elastic documentation, Set up a TSDS
• Elastic documentation, Time series data streams
• Elastic documentation, Data tiers
• Elasticsearch 9.0 release notes, Remove the ability to read frozen indices