jvector

Introduction to approximate nearest neighbor search

Exact nearest neighbor search (k-nearest-neighbor or KNN) is prohibitively expensive at higher dimensions, because approaches to segment the search space that work in 2D or 3D like quadtree or k-d tree devolve to linear scans at higher dimensions. This is one aspect of what is called “the curse of dimensionality.”

With larger datasets, it is almost always more useful to get an approximate answer in logarithmic time, than the exact answer in linear time. This is abbreviated as ANN (approximate nearest neighbor) search.

There are two broad categories of ANN index:

• Partition-based indexes, like LSH or IVF or SCANN
• Graph indexes, like HNSW or DiskANN

Graph-based indexes tend to be simpler to implement and faster, but more importantly they can be constructed and updated incrementally. This makes them a much better fit for a general-purpose index than partitioning approaches that only work on static datasets that are completely specified up front. That is why all the major commercial vector indexes use graph approaches.

JVector is a graph index that merges the DiskANN and HNSW family trees. JVector borrows the hierarchical structure from HNSW, and uses Vamana (the algorithm behind DiskANN) within each layer.

JVector Architecture

JVector is a graph-based index that builds on the HNSW and DiskANN designs with composable extensions.

JVector implements a multi-layer graph with nonblocking concurrency control, allowing construction to scale linearly with the number of cores:

The upper layers of the hierarchy are represented by an in-memory adjacency list per node. This allows for quick navigation with no IOs. The bottom layer of the graph is represented by an on-disk adjacency list per node. JVector uses additional data stored inline to support two-pass searches, with the first pass powered by lossily compressed representations of the vectors kept in memory, and the second by a more accurate representation read from disk. The first pass can be performed with

• Product quantization (PQ), optionally with anisotropic weighting
• Binary quantization (BQ)
• Fused PQ, where PQ codebooks are written inline with the graph adjacency list

The second pass can be performed with

• Full resolution float32 vectors
• NVQ, which uses a non-uniform technique to quantize vectors with high-accuracy

This two-pass design reduces memory usage and reduces latency while preserving accuracy.

Additionally, JVector is unique in offering the ability to construct the index itself using two-pass searches, allowing larger-than-memory indexes to be built:

This is important because it allows you to take advantage of logarithmic search within a single index, instead of spilling over to linear-time merging of results from multiple indexes.

Getting started with JVector

Introductory tutorials for JVector are available in docs/tutorials. Start with the basic tutorial or review VectorIntro.java for a simple example using JVector.

For file-backed input datasets, see Batched input vector access, including runnable 100k-vector examples and optional integration with PQ and NVQ.

For writing completed graphs with FP or NVQ features, see Deferred batch serialization. It works with application vector sources and preloaded data; dataset prefetching is optional.

The older step-by-step guide for JV can be found here. New users should start with the tutorials mentioned earler, but the step-by-step guide contains useful commentary for advanced users.

SIMD provider selection

On Java 22+ with the Vector API enabled, JVector automatically prefers its Google Highway native provider when the native library is available. Native libraries are bundled for Linux x86-64 (SSE4.2 or newer) and Linux AArch64. Use these JVM options:

--add-modules jdk.incubator.vector
--enable-native-access=ALL-UNNAMED

Omitting native access can produce restricted-method warnings. If native access is denied by the runtime or JVM options, automatic selection falls back to Panama; an explicitly forced native provider fails during initialization. For named-module deployments, enable native access for the module containing JVector’s native code.

Operations implemented by Highway use native kernels; remaining operations use the inherited Panama/Java Vector API implementation with native vector storage. Native ISA selection remains automatic and respects the existing JVECTOR_MAX_ISA cap.

JVM property	Effect
No provider property	Prefer native on Java 22+; use Panama when native is unavailable
-Djvector.enable_native_vectorization=false	Disable automatic native selection
-Djvector.vectorization_provider=panama	Explicitly select Panama
-Djvector.vectorization_provider=native	Explicitly require native; fail if it cannot load
-Djvector.vectorization_provider=default	Explicitly select the scalar provider

The explicit jvector.vectorization_provider property takes precedence over the native-enable property. The former jvector.experimental.enable_native_vectorization name is a deprecated compatibility alias; the new jvector.enable_native_vectorization name takes precedence when both are supplied. Both names default to true when neither is supplied. Provider selection happens once per JVM. Java 20/21 uses Panama when available. Older runtimes, or runtimes without the Vector API module, retain the scalar fallback. Neither Panama nor native is enabled by these defaults when the existing runtime compatibility checks reject SIMD support.

Native selection also changes the concrete vector type to MemorySegmentVectorFloat. Use the VectorFloat interface instead of casting to ArrayVectorFloat or assuming that VectorFloat.get() returns a float[]. Both built-in float vector types support FloatArray.array() for array access and copyFrom between their storage types. Native vector operations still require vectors created by the selected provider; copy external vectors into provider-created vectors before passing them to those operations. To retain the previous provider and storage choice, disable automatic native selection with -Djvector.enable_native_vectorization=false.

The research behind the algorithms

• Foundational work: HNSW and DiskANN papers, and a higher level explainer
• Anisotropic PQ paper
• Quicker ADC paper
• NVQ paper

Developing and Testing

This project is organized as a multimodule Maven build. The intent is to produce a multirelease jar suitable for use as a dependency from any Java 11 code. When run on a Java 20+ JVM with the Vector module enabled, optimized vector providers will be used. In general, the project is structured to be built with JDK 20+, but when JAVA_HOME is set to Java 11 -> Java 19, certain build features will still be available.

Cloning

This repository uses a Git submodule for Google Highway, located at jvector-native/src/main/native/third_party/highway. After cloning, initialise it with:

git submodule update --init

Or clone with submodules in one step:

git clone --recurse-submodules <repo-url>

Building native libraries

The native SIMD library (libjvector.so) is built with Meson + Ninja and requires g++ 11+. Supported platforms: Linux x86-64 (SSE4.2, AVX2, AVX-512) and Linux AArch64 (NEON, SVE, SVE2). The entry-point script is jvector-native/src/main/native/src/build_native_lib.sh. Run it from that directory.

cd jvector-native/src/main/native/src
bash build_native_lib.sh

On a fresh Ubuntu machine you can install all required dependencies (g++, meson, ninja) and build in one step:

cd jvector-native/src/main/native/src
bash build_native_lib.sh --auto-install-deps

For other distributions the script will print the install commands needed. See jvector-native/src/main/native/README.md for full build options, Maven integration, ISA dispatch details, and how to add new kernels. The native ABI compatibility guide explains the supported ABI set, startup checks, and how to preserve compatibility when changing native entry points.

---

Base code is in jvector-base and will be built for Java 11 releases, restricting language features and APIs appropriately. Code in jvector-twenty will be compiled for Java 20 language features/APIs and included in the final multirelease jar targeting supported JVMs. jvector-multirelease packages jvector-base and jvector-twenty as a multirelease jar for release. jvector-examples is an additional sibling module that uses the reactor-representation of jvector-base/jvector-twenty to run example code. jvector-tests contains tests for the project, capable of running against both Java 11 and Java 20+ JVMs.

To run tests, use mvn test. To run tests against Java 20+, use mvn test. To run tests against Java 11, use mvn -Pjdk11 test. To run a single test class, use the Maven Surefire test filtering capability, e.g., mvn -Dsurefire.failIfNoSpecifiedTests=false -Dtest=TestNeighborArray test. You may also use method-level filtering and patterns, e.g., mvn -Dsurefire.failIfNoSpecifiedTests=false -Dtest=TestNeighborArray#testRetain* test. (The failIfNoSpecifiedTests option works around a quirk of surefire: it is happy to run test with submodules with empty test sets, but as soon as you supply a filter, it wants at least one match in every submodule.)

You can run SiftSmall and Bench directly to get an idea of what all is going on here. Bench will automatically download required datasets to the dataset_cache directory. The files used by SiftSmall can be found in the siftsmall directory in the project root.

To run either class, you can use the Maven exec-plugin via the following incantations:

mvn compile exec:exec@bench

or for Sift:

mvn compile exec:exec@sift

Bench takes an optional benchArgs argument that can be set to a list of whitespace-separated regexes. If any of the provided regexes match within a dataset name, that dataset will be included in the benchmark. For example, to run only the glove and nytimes datasets, you could use:

mvn compile exec:exec@bench -DbenchArgs="glove nytimes"

To run Sift/Bench without the JVM vector module available, you can use the following invocations:

mvn -Pjdk11 compile exec:exec@bench

mvn -Pjdk11 compile exec:exec@sift

The ... -Pjdk11 invocations will also work with JAVA_HOME pointing at a Java 11 installation.

For more information on running benchmarks, go through docs/benchmarking.md.

To release, configure ~/.m2/settings.xml to point to OSSRH and run mvn -Prelease clean deploy.

---