Project 4: An Upstream Apache Lucene HNSW Contribution

This is the one project whose deliverable does not land in an OpenSearch repository at all. It lands in apache/lucene — and then flows into OpenSearch on the next Lucene upgrade, because OpenSearch bundles a specific Lucene version and upgrading it is a recurring core task. A small, well-scoped HNSW / vector-format / VectorUtil improvement or bug fix in Lucene is the cleanest path in this entire portfolio to a real merged PR in a tier-1 open-source project, with the least cross-plugin surface area to learn.

It is also the project where you contribute under a different process: the Apache Software Foundation's. No DCO sign-off line, a CHANGES.txt entry instead of CHANGELOG.md, ASF PR conventions, and a community that lives partly in GitHub PRs and partly in JIRA history (LUCENE-NNNN). Project 4 is as much about learning that process as about the code.

Note: Read HNSW Vector Search in Lucene and SIMD and the Vector API first, and do Lab L4: Contribute to Apache Lucene — this brief assumes you can already build Lucene with ./gradlew check and run a single test with a seed. The lab is the on-ramp; this brief is the real thing.

Problem & motivation

Lucene's vector search is the foundation under OpenSearch's k-NN lucene engine and, indirectly, the bar everything else is measured against. It is also actively evolving — the scalar quantization formats, the SIMD-accelerated distance kernels, and the HNSW graph builder/merger are all areas with open work and reachable bugs. That makes it fertile ground for a scoped contribution:

• The scalar-quantized vector formats (int8 and the newer custom-bit variants) are a thin, well-tested, well-bounded subsystem. A correctness bug, an edge case in dequantization, or a missing validation is the kind of thing a careful newcomer can find and fix.
• VectorUtil and the Panama Vector API acceleration (dotProduct, squareDistance, cosine) are pure functions with exact scalar fallbacks — ideal for a small correctness or performance improvement that is easy to test against the fallback.
• The HNSW graph builder/merger is where the big wins live (faster merging gave ~25% indexing speedups in Lucene's nightly benchmarks) — too large to rewrite, but full of scoped, reachable improvements.

The motivation is leverage: a 50-line fix in Lucene's vector code is exercised by every Lucene user on earth and arrives in OpenSearch automatically at the next upgrade. That is the highest blast-radius-per-line in the whole portfolio — which is exactly why the bar for correctness and test rigor is highest here too.

Real-world grounding

Two real, verified Lucene vector-search issues anchor this — read both to understand the shape of the subsystem and the kind of work that lands:

• Scalar quantization codec — apache/lucene #12497: https://github.com/apache/lucene/issues/12497 — the introduction of a scalar-quantized vectors codec format. Reading this issue and its PR teaches you exactly how a vector format is structured, tested, and reviewed in Lucene.
• int8 scalar quantization origin — apache/lucene #11613 / LUCENE-10577: https://github.com/apache/lucene/issues/11613 — the original int8 scalar quantization work (tracked historically as LUCENE-10577). This is the design lineage of the Lucene99ScalarQuantizedVectorsFormat family and the later custom-bit Lucene104ScalarQuantizedVectorsFormat.

To find a current fixable issue, search live and filter for newcomer-sized work:

# in apache/lucene
is:issue is:open label:"module:vector" OR label:vector hnsw OR quantiz OR VectorUtil
is:issue is:open label:"good first issue" vector
is:issue is:open scalar quantiz dequantize OR edge case OR NaN OR overflow

Citation discipline: #12497 and #11613 / LUCENE-10577 are real — cite them. The specific issue you fix must be one you actually find open (or a bug you reproduce in a test); link it in your design note. Do not invent a LUCENE number.

Subsystems you'll touch

Cross-references: HNSW vector search in Lucene · SIMD and the Vector API · segments and codecs · the Lucene contribution lab · and the OpenSearch k-NN engines chapter where the lucene engine reuses all of this.

Phased plan

The ASF rule of thumb: reproduce the problem inside Lucene's own test framework first. Lucene reviewers expect a failing LuceneTestCase with a fixed seed before they look at a fix.

Phase 0 — Build Lucene and run the vector tests (½ day)

git clone https://github.com/apache/lucene.git && cd lucene
./gradlew :lucene:core:compileJava                     # JDK 21
./gradlew :lucene:core:test --tests "org.apache.lucene.util.hnsw.*"
./gradlew :lucene:core:test --tests "org.apache.lucene.util.TestVectorUtil"
./gradlew :lucene:luke:run                             # inspect a real index, optional

Reproduce a known-good run with a seed so you understand the randomized harness:

./gradlew :lucene:core:test --tests "*TestVectorUtil*" -Ptests.seed=DEADBEEF

Write capstone-work/lucene-vector-map.md: how a KnnFloatVectorField flows from IndexWriter → the KnnVectorsFormat → .vec/.vex/.vem files, and how a KnnFloatVectorQuery walks HnswGraphSearcher and scores with VectorUtil. Cite files.

Phase 1 — Pick and reproduce a scoped target (the real first deliverable)

Choose one target type. The whole project's mergeability depends on scoping this small:

Then reproduce it as a failing test:

// extends LuceneTestCase
public void testScalarQuantizedDequantizeEdgeCase() {
    float[] v = new float[] { 0f, 0f, 0f, 0f };   // degenerate input that triggers the bug
    // quantize -> dequantize -> assert the round-trip / similarity invariant the code claims
    assertEquals(expected, actual, DELTA);        // fails on main, passes with your fix
}

./gradlew :lucene:core:test --tests "*YourNewTest*"   # confirm it FAILS on main first

This failing test, with a seed, is the artifact that turns "I think there's a bug" into a Lucene PR reviewers will engage with.

Phase 2 — The fix (minimum diff)

Fix it where the bug lives, nothing more. Lucene reviewers are strict about scope and about not touching the on-disk format casually (format changes have versioning and BWC implications).

grep -rn "dequantize\|quantize\|scalar" lucene/core/src/java/org/apache/lucene/codecs/lucene99
grep -rn "squareDistance\|dotProduct\|cosine" lucene/core/src/java/org/apache/lucene/util/VectorUtil.java

If your change touches a kernel that has both a Panama and a scalar path, fix both and assert they agree — that parity test is often the most valuable thing in the PR.

public void testPanamaAndScalarAgree() {
    float[] a = randomVector(768), b = randomVector(768);
    assertEquals(VECTOR_UTIL_SCALAR.dotProduct(a, b),
                 VECTOR_UTIL_PANAMA.dotProduct(a, b), 1e-4f);
}

Phase 3 — Run the full gate

Lucene's check is broad and strict (forbidden APIs, formatting, the full test sweep). Run it, and run your area's tests with several seeds to flush randomized flakiness.

./gradlew check                                  # the full gate; expect it to be heavy
./gradlew :lucene:core:test --tests "org.apache.lucene.codecs.lucene99.*"
# repeat your test under a few seeds:
for s in CAFE D00D F00D; do \
  ./gradlew :lucene:core:test --tests "*YourNewTest*" -Ptests.seed=$s; done

If you touched a SIMD path, run with the Vector API module enabled to exercise the Panama provider, not just the fallback:

./gradlew :lucene:core:test --tests "*VectorUtil*" \
  -Dtests.jvmargs="--add-modules jdk.incubator.vector"

Phase 4 — The PR, the ASF way

This is where the process differs from OpenSearch. Do all of it:

• Add a CHANGES.txt entry under the right section (Bug Fixes / Improvements / Optimizations) for the next release — not a CHANGELOG.md, and no DCO sign-off line.
• Open a GitHub PR against apache/lucene:main following the PR template; reference the issue number (and the LUCENE-NNNN if one exists).
• Expect review from Lucene committers. Respond by changing code or explaining with a test, not by arguing. Randomized-test reviewers will ask "does this hold under any seed?" — have the answer.

  # CHANGES.txt (excerpt — section names/format vary by version, check the file)
  ## Bug Fixes
+ * GITHUB#NNNNN: Fix scalar-quantized dequantization of all-zero vectors so the
+   round-trip similarity matches the documented invariant. (Your Name)

Deliverables

• capstone-work/lucene-vector-map.md — field → format → files, query → graph → VectorUtil, cited
• capstone-work/design.md — the chosen target, why it's in scope, alternatives rejected
• A failing LuceneTestCase (with a seed) that is red on main
• The minimum-diff fix, green under multiple seeds
• A Panama/scalar parity test if a SIMD kernel was touched
• capstone-work/validation.md — ./gradlew check output, the seeds run, the JVM args used
• A CHANGES.txt entry (ASF-style, no DCO line)
• A GitHub PR against apache/lucene:main referencing the issue
• A 500–1000 word write-up: the bug/improvement, the reproduction, the ASF process you learned

Difficulty & time

Warning: Do not change the on-disk vector format for a first contribution. Format changes carry BWC and versioning weight and will not be reviewed as a newcomer PR. Stay in correctness, validation, VectorUtil, or an allocation/perf tweak that doesn't alter bytes on disk.

Stretch goals

• After a correctness fix lands, propose a small optimization in the same area (a reduced allocation in the graph merger hot path) with a luceneutil/JMH benchmark — this is the path toward the ~25%-indexing-speedup class of wins.
• Trace your merged change into OpenSearch: find the "Upgrade to Lucene X.Y" issue/PR in opensearch-project/OpenSearch and note where your fix arrives. Search: is:pr "Upgrade to Lucene" in the OpenSearch repo.
• Add a parity test for a kernel that currently lacks one, even without a bug — strengthening the test suite is a legitimate, welcome Lucene contribution.

Evaluation

Self-grade against the 100-point rubric:

A merged Lucene vector PR at 90+ is the single most transferable credential in this portfolio.

How to turn this into a real contribution

1. This is the real contribution — unlike the other projects, the upstream PR is the primary deliverable, not an optional stretch. Scope Phase 1 to land.
2. Reproduce in Lucene's tests first. A failing LuceneTestCase with a seed is the entry ticket. No reproducer, no review.
3. Learn the ASF process deliberately. CHANGES.txt (not CHANGELOG.md), no DCO line, ASF PR template, committer review. The Lab L4 walks the mechanics.
4. Respect the format boundary. Stay out of on-disk format changes for a first PR. Pick correctness, VectorUtil, validation, or a byte-stable optimization.
5. Close the loop into OpenSearch. When your fix ships in a Lucene release and OpenSearch upgrades, your code is running in OpenSearch — note that in your write-up. It is the cleanest demonstration in the whole curriculum of how the layers connect.

This is the project to choose when you want a real upstream merge with the least cross-plugin surface area and the clearest process — at the cost of the highest test-rigor bar in the portfolio.