Vector Search and the k-NN Plugin

A match query asks "which documents contain these words?" A vector query asks a different and stranger question: "which documents are closest to this point in a 1,024-dimensional space?" That single change — from matching tokens to measuring geometric distance — is what powers semantic search, retrieval-augmented generation (RAG), recommendations, image similarity, deduplication, and anomaly detection. The text "a small domestic feline" and the text "kitten" share almost no tokens, so BM25 scores them as unrelated. Run both through an embedding model and they land next to each other in vector space. Vector search finds that neighbor; lexical search never will.

OpenSearch does this through the k-NN plugin (opensearch-project/k-NN), a separate repository that adds a knn_vector field type, a knn query, three approximate-nearest-neighbor engines, and a native C++ layer reached over JNI. This section is the advanced, vector-and-scale layer of the curriculum. It assumes you have done the OpenSearch warm-up, understand the plugin architecture, and have read the Lucene chapter on HNSW vector search — because k-NN's Lucene engine is that HNSW format, and its native engines reimplement the same idea in C++.

Note: "k-NN" is k-nearest-neighbors. In OpenSearch the plugin name is literally k-NN (repo) / knn (the field type, query, and REST namespace). We use "k-NN" and "vector search" interchangeably.

Why vector search matters

Workload	The question	Why lexical search fails	What vectors do
Semantic search	"find docs about X"	synonyms, paraphrase, no shared tokens	embed query + docs, find nearest
RAG	"retrieve context for an LLM prompt"	the prompt rarely shares keywords with the answer	nearest-neighbor retrieval over chunk embeddings
Recommendations	"items like this item"	"like" is not a keyword	nearest neighbors in an item-embedding space
Image / audio similarity	"find similar media"	there are no tokens at all	embed media, ANN over the vectors
Deduplication / anomaly	"near-duplicate or outlier?"	edit distance misses semantic dupes	distance threshold (radial search)

The mechanism is always the same: an embedding model (often run outside OpenSearch, or inside it via the ml-commons plugin and a neural-search pipeline) turns text or media into a fixed-length float array — the vector. You index those vectors, then at query time embed the query the same way and ask k-NN for the k closest stored vectors by a distance metric (l2, cosinesimil, innerproduct, …). "Closest" is the whole game.

The hard part is scale. Computing the exact distance from your query to every one of 100 million stored vectors per search is too slow. So k-NN uses approximate nearest neighbor (ANN) algorithms — chiefly HNSW (a navigable small-world graph) and IVF (inverted-file clustering) — that trade a little recall for orders of magnitude less work. Everything downstream (engines, quantization, native memory, disk-based search) exists to make ANN fast, accurate, and affordable enough to run in production.

How k-NN relates to core OpenSearch

The k-NN plugin is an ordinary OpenSearch Plugin — the same machinery from the plugin architecture deep dive, with two twists that make it unusual.

flowchart LR
    subgraph core["OpenSearch core (published artifacts)"]
        EP["extension interfaces<br/>MapperPlugin / SearchPlugin / ActionPlugin /<br/>EnginePlugin / ScriptPlugin / ..."]
    end
    subgraph knn["k-NN plugin repo (org.opensearch.knn)"]
        J["src/main/java<br/>field type, query, engines, codec"]
        N["jni/ (CMake C++)<br/>faiss + nmslib wrappers"]
    end
    J -->|implements| EP
    J <-->|JNI| N
    N -->|links| FAISS["faiss / nmslib<br/>(native ANN libraries)"]

It builds against published OpenSearch artifacts, not the core source tree. Like security, sql, and ml-commons, k-NN lives in its own repo with its own release cadence, version-locked to a specific OpenSearch version (opensearch.version in its Gradle build). Confirm the version it targets:
```
cd ~/src/k-NN
grep -n "opensearch.version" build.gradle    # e.g. opensearch_version = "3.6.0-SNAPSHOT"
```
It has a native build under jni/. Pure-Java plugins are the norm; k-NN is not pure Java. Under jni/ is a CMake project that compiles C++ glue (faiss_wrapper.cpp, nmslib_wrapper.cpp) and links the faiss and nmslib ANN libraries pulled in as git submodules under jni/external/. The Java side calls into it through org.opensearch.knn.jni.JNIService → FaissService / NmslibService. This is the only part of the OpenSearch ecosystem most contributors meet where a bug might be in C++, a memory leak might be off-heap, and a build failure might be CMake.
```
ls ~/src/k-NN/jni/                 # CMakeLists.txt, src/, external/, include/
ls ~/src/k-NN/jni/external/        # faiss  nmslib  (git submodules)
```

Everything else is recognizable plugin work: KNNVectorFieldMapper is a MapperPlugin contribution, KNNQueryBuilder is a SearchPlugin contribution, the warmup and stats endpoints are ActionPlugin contributions. If you have built an in-repo plugin, you already know 80% of the shape — the native layer is the new 20%.

The three engines at a glance

The engine mapping parameter on a knn_vector field picks which ANN implementation builds and searches the graph. There are three, and choosing among them is the single most consequential modeling decision in vector search.

Engine	Status	Algorithms	Where it runs	Notes
`faiss`	DEFAULT	HNSW, IVF	native C++ (off-heap) via JNI	quantization (PQ/SQ/BQ), disk-based ANN, byte vectors
`lucene`	supported	HNSW	pure Java, on JVM heap (Lucene's own format)	tight filtering integration, scalar quantization, no JNI
`nmslib`	DEPRECATED	HNSW only	native C++ via JNI	new nmslib indices blocked since 3.0; existing indices still read for BWC

# Confirm in the source: the engine enum, the default, and what's deprecated.
cd ~/src/k-NN
grep -nE "FAISS|LUCENE|NMSLIB|DEFAULT|DEPRECATED" \
  src/main/java/org/opensearch/knn/index/engine/KNNEngine.java | head

You will find public static final KNNEngine DEFAULT = FAISS; and a DEPRECATED_ENGINES set containing NMSLIB. The deprecation is real and version-gated: nmslib was deprecated in OpenSearch 2.19 and creating new nmslib indices is blocked in 3.0. There is an active meta-issue for what comes next — [META] Supporting New Vector Engine in OpenSearch (k-NN #2605) (https://github.com/opensearch-project/k-NN/issues/2605) — which is the strategic context for why the engine layer is being restructured. The dedicated Engines chapter compares all three in depth.

How this section is organized

Read in order. Each concept chapter has companion labs; do the concept first, then the lab that exercises it.

Concept chapters

#	Chapter	What you learn
1	Overview (this page)	why vectors, how k-NN relates to core, the three engines
2	k-NN Warm-Up: From User to Contributor	run k-NN as a user, then bridge every behavior to source
3	k-NN Plugin Architecture	`KNNPlugin`, the module layout, the custom codec, native memory
4	Engines: Faiss, Lucene, and NMSLIB	deep compare; the capability matrix; how engine choice flows to segment files
5	Algorithms: HNSW, IVF, and PQ	the ANN algorithms themselves and their parameters
6	Native Integration, JNI, and Memory	the C++ build, the JNI boundary, off-heap memory and the circuit breaker
7	The k-NN Query Path	`KNNQueryBuilder` → `KNNQuery` → `KNNWeight`, filtering, rescoring
8	Quantization and Disk-Based ANN	byte/FP16/PQ/BQ, `on_disk` mode, `compression_level`, rescoring

Labs

#	Lab	Deliverable
K1	Build the k-NN plugin from source	a working native + Java build, plugin installed in a local node
K2	Trace a k-NN query	a stack trace from REST down to the JNI call
K3	The `knn_vector` field type	read and modify the field mapper
K4	Build it — a custom k-NN feature	a real feature added to the plugin
K5	Fix it — a real k-NN issue	a PR-shaped fix against a labeled issue
K6	Benchmark recall and latency	a recall/latency report across engines and parameters

The contribution landscape

k-NN is one of the most active OpenSearch plugin repos and one of the most welcoming to new contributors, precisely because the surface is broad: Java field-mapping work, Lucene-codec work, C++/JNI work, quantization math, benchmarking, and cutting-edge GPU and remote-build research all live in one repo. A few live threads to orient you:

The next vector engine — k-NN #2605 (META): how the engine abstraction evolves after nmslib's deprecation.
GPU acceleration — k-NN #2293 (RFC, NVIDIA cuVS / CAGRA) and remote index build k-NN #2294 (RFC): offload segment graph construction to a remote GPU/CPU fleet.
Native memory circuit breaker — k-NN #1582: rearchitecting the off-heap memory guard. A great window into the off-heap memory story.

For the full catalog of issues, RFCs, and where to start contributing across OpenSearch, Lucene, and k-NN, see Real Issues, RFCs, and Where to Contribute. For everything that does not have a verified issue number here, do not invent one — search GitHub directly:

gh issue list --repo opensearch-project/k-NN --label "good first issue" --state open
gh issue list --repo opensearch-project/k-NN --search "faiss filter rescore in:title"

What this builds on

This section is the application of the Lucene chapter to a production subsystem. Before the architecture chapter, you should be comfortable with:

HNSW Vector Search in Lucene — the graph algorithm, KnnFloatVectorField, Lucene99HnswVectorsFormat, the .vec/.vex/.vem segment files. The k-NN lucene engine is a thin wrapper over exactly this; the faiss and nmslib engines are the same idea implemented in C++ with their own segment files.
SIMD and the Panama Vector API — why distance computations are fast, which is the same reason faiss is fast (it uses AVX2/AVX-512; see the PlatformUtils.isAVX512SupportedBySystem() checks in KNNPlugin).
Plugin architecture, Mapping and analysis, Search execution, and Circuit breakers and memory — the core subsystems k-NN extends.

Now run it as a user. Continue to the k-NN Warm-Up.

Open-Source Engineer & Contributor