nmslib is deprecated in OpenSearch: how to migrate k-NN indices to Faiss or Lucene

The nmslib engine was deprecated in the OpenSearch k-NN plugin 2.19, and since 3.0 OpenSearch rejects any new index that uses it. Existing nmslib indices still load and answer queries after the upgrade, but you cannot change a field's engine in place: create a new index on Faiss or Lucene, _reindex into it, verify counts and recall, then swap the alias. We recommend Faiss HNSW for almost every migration, and Lucene only for small indices under the docs' 10 million vector guideline.

Everything below was run on OpenSearch 3.9.0 (Lucene 10.5.1, k-NN 3.9.0.0), the current release as of 31 October 2026. The source index was created on 2.19.7, and a second one on 2.17.1, then both data directories were started on 3.9.0. The corpus is illustrative: 5,000 random 8-dimension vectors with a category keyword. It is big enough to show every request and error, and too small to say anything about latency.

What changed, by version

The engine class in the plugin source is annotated @Deprecated(since = "2.19.0", forRemoval = true). Plan as if a future major version will stop reading nmslib segments, even though 3.9.0 still does.

The error you get on OpenSearch 3.x

Creating an nmslib field on 3.9.0:

PUT products-nmslib-test
{
  "settings": { "index.knn": true },
  "mappings": { "properties": {
    "embedding": { "type": "knn_vector", "dimension": 8,
      "method": { "name": "hnsw", "engine": "nmslib", "space_type": "cosinesimil" } } } }
}

{
  "error": {
    "root_cause": [{
      "type": "mapper_parsing_exception",
      "reason": "Failed to parse mapping [_doc]: nmslib engine is deprecated in OpenSearch  and cannot be used for new index creation in OpenSearch from  3.0.0."
    }],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "nmslib engine is deprecated in OpenSearch  and cannot be used for new index creation in OpenSearch from  3.0.0."
    }
  },
  "status": 400
}

The double spaces are in the source string, so match on a substring if you grep logs for it. The usual trigger is an index template, or application code that creates indices on startup, still carrying "engine": "nmslib". The old index is fine; the next rollover or the next test environment is what breaks.

GitLab is a good example of how this happens. In December 2024 their merge request 175261 pinned nmslib explicitly, because 2.18 had moved the default to Faiss and Faiss did not support cosine yet. That fix was correct for 2.18 and is exactly what 3.0 rejects.

Find every nmslib field before you upgrade

The obvious check is GET _all/_mapping filtered on method.engine. It misses two cases: knn_vector fields inside objects, and fields created before 2.18 with no method block at all. We created this on 2.17.1:

PUT legacy
{
  "settings": { "index.knn": true },
  "mappings": { "properties": { "embedding": { "type": "knn_vector", "dimension": 8 } } }
}

The stored mapping is just {"type":"knn_vector","dimension":8}, with no engine anywhere, and GET _plugins/_knn/stats reported "nmslib_initialized": true after the first document. After a restart on 3.9.0 the mapping still names no engine and the index still serves nmslib queries. A mapping-only filter reports nothing to migrate.

This script walks every mapping, recurses into objects, and uses index.version.created_string to resolve the default engine:

import json, sys, urllib.request
OS = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:9200"
get = lambda p: json.load(urllib.request.urlopen(OS + p))
mappings = get("/_all/_mapping")
settings = get("/_all/_settings?human&filter_path=*.settings.index.version.created_string")

def walk(props, path=""):
    for name, f in props.items():
        if f.get("type") == "knn_vector":
            yield path + name, f
        if "properties" in f:
            yield from walk(f["properties"], path + name + ".")

for index, body in mappings.items():
    created = settings.get(index, {}).get("settings", {}).get("index", {}) \
                      .get("version", {}).get("created_string", "?")
    for field, f in walk(body["mappings"].get("properties", {})):
        engine = f.get("method", {}).get("engine")
        if "model_id" in f:
            engine = "model:" + f["model_id"]   # trained model: check the model's engine
        elif engine is None:
            # no engine in the mapping: the default applied at creation time
            engine = "nmslib (default)" if tuple(map(int, created.split(".")[:2])) < (2, 18) else "faiss (default)"
        flag = "MIGRATE" if engine.startswith("nmslib") else ""
        print(f"{index:30} {field:25} created={created:8} engine={engine:18} {flag}")

Output against our two 3.9.0 clusters:

products-v1     embedding   created=2.19.7   engine=nmslib             MIGRATE
products-v2     embedding   created=3.9.0    engine=faiss
products-v2-lucene embedding created=3.9.0  engine=lucene
legacy          embedding   created=2.17.1   engine=nmslib (default)   MIGRATE

Run it on the 2.x cluster before the upgrade, and again against your index templates (GET _index_template), since a template is what will try to create the next nmslib index.

OpenSearch Faiss vs Lucene: which engine to migrate to

The engine recommendations say to "select Faiss for large-scale use cases" and describe Lucene as "a good option for smaller deployments". The relevant rows, from the 3.9 docs and our runs:

One contradiction to know about: the same engines table lists Faiss filtering as "Post-filter", while the filtering docs say Faiss HNSW has supported efficient filtering since 2.9. The filtering docs are right. A knn query with filter inside the clause returned 10 of 10 hits on our Faiss index on 3.9.0.

That filter is also the best reason to migrate early. The same query against the nmslib index fails:

failed to create query: Engine [NMSLIB] does not support filters

On nmslib you can only filter after the k-NN search, with a bool filter or post_filter. Our category filter matches a quarter of the corpus, and the post-filtered query returned 2 hits out of 10 requested. That shortfall is the classic nmslib complaint, and moving the filter inside the clause on Faiss or Lucene fixes it.

Our recommendation is Faiss HNSW. It is the default, so an index that omits the engine lands there anyway, and it is the engine disk mode and the quantization encoders are built around. Pick Lucene when the index will stay well under 10 million vectors and you would rather keep vectors as ordinary Lucene segments. Trained models (IVF, PQ) are Faiss only.

flowchart TD
  A["nmslib field found"] --> B{"space_type l1 or linf?"}
  B -- yes --> C["No HNSW equivalent on Faiss or Lucene.<br/>Re-evaluate the metric with your own queries"]
  B -- no --> D{"Under 10M vectors and<br/>staying there?"}
  D -- yes --> E{"Need quantization<br/>or disk mode?"}
  E -- no --> F["Lucene HNSW"]
  E -- yes --> G["Faiss HNSW"]
  D -- no --> G
  C --> D

Space-type mapping gotchas

On 2.17.1 (our run) and 2.18 (GitLab's report), Faiss with cosine fails with this error:

Validation Failed: 1: "hnsw" with "faiss" configuration does not support space type: "cosinesimil".;

On 3.9.0 the same mapping works. l1 still fails on both engines, with "hnsw" with "faiss" configuration does not support space type: "l1". and the same message for Lucene. If you have an l1 or linf field, the migration is a relevance change rather than a reindex, and needs an offline evaluation first.

Faiss cosine has one side effect the docs flag: vectors "are automatically normalized to unit length during indexing". Combined with index.knn.derived_source.enabled, which defaults to true for k-NN indices created on 3.x, the vector you read back from _source is the normalized one. Document 4415 came back from the nmslib index as [-0.1187, -0.0943, 0.5059, ...] and from the Faiss index as [-0.093887866, -0.074588254, 0.4001506, ...]. Scores were identical (0.97389656 for the top hit on both). If anything downstream reads raw vectors back out of OpenSearch, for example a later reindex into an l2 or innerproduct field, re-embed from the source of truth instead. Lucene cosine returned the original values.

Step-by-step migration from nmslib to Faiss

Assume your application reads and writes through an alias, products, pointing at products-v1. If it addresses the index name directly, add the alias first; that is a one-request change and makes every later cutover cheap.

1. The mapping before and after

Before, created on 2.19.7:

PUT products-v1
{
  "settings": { "index.knn": true, "index.knn.algo_param.ef_search": 100 },
  "mappings": { "properties": {
    "title":    { "type": "text" },
    "category": { "type": "keyword" },
    "embedding": { "type": "knn_vector", "dimension": 8,
      "method": { "name": "hnsw", "engine": "nmslib", "space_type": "cosinesimil",
                  "parameters": { "m": 16, "ef_construction": 128 } } } } }
}

After, on 3.9.0:

PUT products-v2
{
  "settings": { "index.knn": true, "number_of_replicas": 0, "refresh_interval": "-1" },
  "mappings": { "properties": {
    "title":    { "type": "text" },
    "category": { "type": "keyword" },
    "embedding": { "type": "knn_vector", "dimension": 8, "space_type": "cosinesimil",
      "method": { "name": "hnsw", "engine": "faiss",
                  "parameters": { "m": 16, "ef_construction": 128, "ef_search": 100 } } } } }
}

Keep m and ef_construction the same, so the only variable is the engine. ef_search moves from the index setting, which the settings docs say is in the deprecation process, into the method parameters. Replicas off and refresh disabled make the bulk load cheaper; you restore both before cutover.

Check native memory before you build. The docs estimate HNSW at 1.1 * (4 * dimension + 8 * m) bytes per vector. For an illustrative 10 million vectors at 768 dimensions with m: 16, that is 1.1 * (3072 + 128) = 3,520 bytes, or 35.2 GB, doubled to 70.4 GB with one replica. During the migration both indices are resident, and native indices only get the share of RAM set by circuit_breaker_limit (50% by default). The vector sizing calculator runs the same arithmetic for your own dimensions, vector count and quantization.

2. Reindex

POST _reindex?slices=auto&requests_per_second=2000&wait_for_completion=false
{
  "source": { "index": "products-v1", "size": 1000 },
  "dest":   { "index": "products-v2" }
}

GET _tasks/NS3zDjNmSOSwSZt6Ivb3Ag:193
{ "completed": true,
  "response": { "total": 5000, "created": 5000, "batches": 5, "failures": [] } }

slices=auto parallelizes per shard. requests_per_second throttles it (the default -1 means unthrottled), which matters because every batch triggers graph building on the destination. Writes that arrive during the reindex do not reach products-v2. Either pause them, dual-write from the application, or run a second _reindex with a range query on an update timestamp after the first one finishes.

3. Verify counts and recall

Restore the settings, refresh, and compare counts:

PUT products-v2/_settings
{ "index": { "refresh_interval": null, "number_of_replicas": 1 } }

Both products-v1/_count and products-v2/_count returned 5,000. Then compare each engine's approximate top 10 against exact scoring, using real queries from your logs:

def exact(index, v, k=10):
    return search(index, {"size": k, "_source": False, "query": {"script_score": {
        "query": {"match_all": {}},
        "script": {"source": "knn_score", "lang": "knn", "params": {
            "field": "embedding", "query_value": v, "space_type": "cosinesimil"}}}}})
# recall@10 = |ann_top10 ∩ exact_top10| / 10, averaged over the query set

On our 5,000-vector toy index both engines scored 1.000 recall@10 over 100 queries. Expect that on any index this small; it proves the mapping is correct, not that the engines are equal. On a production-sized index, a gap between old and new recall points to ef_search or ef_construction, which you tune before cutover.

4. Swap the alias, keep the rollback

POST _aliases
{ "actions": [
  { "remove": { "index": "products-v1", "alias": "products" } },
  { "add":    { "index": "products-v2", "alias": "products", "is_write_index": true } }
] }

Both actions apply atomically. Rollback is the same request with the index names reversed, so keep products-v1 read-only and resident until production traffic has run on products-v2 for a full business cycle. The same alias pattern is the core of our zero-downtime deployment guide, and it carries over to OpenSearch unchanged. After the swap, move any bool or post_filter vector filters inside the knn clause.

flowchart LR
  A["Audit mappings<br/>and templates"] --> B["Create products-v2<br/>Faiss, replicas 0"]
  B --> C["_reindex<br/>slices=auto, throttled"]
  C --> D["Catch-up writes"]
  D --> E{"Counts and<br/>recall@10 match?"}
  E -- no --> F["Tune ef_search,<br/>reindex again"]
  F --> C
  E -- yes --> G["Restore replicas<br/>and refresh"]
  G --> H["Atomic alias swap"]
  H --> I{"Production OK?"}
  I -- no --> J["Swap alias back<br/>to products-v1"]
  I -- yes --> K["Update templates,<br/>delete products-v1"]

Update the index template in the same change as the alias swap. Otherwise the template is still carrying nmslib, and the next index it creates fails with the 400 above.

Upgrade order: migrate on 2.19 or on 3.x?

Both work. Our 2.19.7 and 2.17.1 nmslib indices opened on 3.9.0 after a full restart, and _reindex into Faiss ran on 3.9.0. Migrating on 2.19 before the upgrade is safer, because Faiss cosine and efficient filtering are already there and the upgrade then carries no nmslib indices at all. Migrating after means testing two changes at once. If you are also comparing engines at this point, our Elasticsearch vs OpenSearch comparison and the Elasticsearch to OpenSearch migration guide cover the wider decision. For vector memory on the new index, the quantization trade-offs in Elasticsearch BBQ HNSW vs BBQ disk and the Qdrant quantization guide map closely onto Faiss sq and on_disk mode.

Where we can help

We run OpenSearch upgrades and k-NN migrations for teams with vectors in production, from the nmslib audit to the recall check against real queries before cutover. If you are planning a 2.x to 3.x upgrade, see our OpenSearch consulting service.