Real Issues, RFCs, and Where to Contribute
This is the catalog — the page you bookmark. Everything else in Engineering at Scale explains how a hard change is made; this page tells you which changes are open, what each one teaches you, and how to take a first bite. Every issue, RFC, and PR below is real and verified, cited with a full GitHub URL so you can open it right now and read the actual thread.
The point is not to memorize numbers. The point is that contribution is a search problem, and this gives you the seed set: a curated map of the live frontier in vector search, sharding, aggregation acceleration, caching, and backpressure — grouped so that picking a theme tells you which subsystem you are about to learn. The threads themselves are the syllabus. Read them.
How to read each table. Description tells you what the issue is about. Exercises names the subsystem and skills it builds. How to start gives you what to read first and a good-first sub-task pattern — the shape of a bounded contribution you could carve out without boiling the ocean.
Theme 1 — Vector Search & GPU (the k-NN plugin)
The vector engine is the most active frontier in OpenSearch, and the most employable thing in this curriculum. Start at the meta issue, then pick a sub-task that matches your appetite for native code.
| Issue / RFC | Description | Exercises | How to start |
|---|---|---|---|
| k-NN #2605 — [META] Supporting New Vector Engine | The umbrella for evolving the k-NN engine abstraction. Your map of the whole vector-engine effort. | the KNNEngine abstraction, codec, faiss/lucene/nmslib boundaries | Read it top-to-bottom as a table of contents. Read the k-NN architecture and engines chapters alongside it. Pick a linked sub-issue that is good first issue. |
| k-NN #2293 — [RFC] Boosting Vector Engine Performance using GPUs | Offload graph construction to NVIDIA cuVS / the CAGRA algorithm (FP32). The cutting edge. | native build, faiss/cuVS, index-build offload, FP32 graphs | Read the RFC for the why (build is CPU-bound at scale). A bounded first task: improve the docs/benchmark methodology, or a CPU-side glue task in jni/. Pair with native JNI & memory. |
| k-NN #2294 — [RFC] Remote Vector Index Build | Build a segment's vector index on a remote fleet, not the indexing node. | remote build protocol, segment upload, the codec write path | Read with #2293 and #2391; they are one story. Start by tracing where the local build happens today (the KNN codec's flush/merge). |
| k-NN #2391 — [Meta] Remote Vector Index Build Component | The META that breaks #2294 into shippable pieces. | component decomposition, client/build-service split | This is your sub-task checklist for remote build. Find an unchecked box that is client-side (no GPU needed) and claim it. |
| k-NN #2595 — Benchmarking | Benchmark recall/latency/build-time for the new engines. | recall@k measurement, latency percentiles, datasets | The best first contribution in this theme: no production-code risk, high value. Pair with the benchmark-harness capstone and the recall/latency lab. |
| k-NN #1582 — Investigate rearchitecture of the native-memory circuit breaker | The faiss/nmslib graphs live in native memory off-heap; the breaker that caps them needs rework. | NativeMemoryCacheManager, the off-heap circuit breaker, guava cache | Read native JNI & memory first. Reproduce the current breaker behavior with GET /_plugins/_knn/stats under load, then propose a measurement or a small invariant fix. |
| k-NN #585 — a real circuit-breaker config bug | A concrete bug in the native-memory circuit-breaker configuration. | settings plumbing, the off-heap breaker, regression tests | A genuine bug-shaped task. Reproduce it, write the failing test first, then fix. This is the cleanest "real fix" entry point in the k-NN repo. |
Sub-task pattern for this theme: the safest first contribution is almost always measurement — a benchmark, a stat, a reproduction — because it has no BWC or correctness blast radius but forces you to learn the path. Graduate from measurement to a config/settings fix, then to the codec write path.
Theme 2 — Lucene Vectors (apache/lucene)
OpenSearch bundles a specific Apache Lucene version; the vector formats it inherits come from upstream Lucene, governed by the Apache Software Foundation. Contributing here is the deepest vector work available and a serious credential.
| Issue / PR | Description | Exercises | How to start |
|---|---|---|---|
| apache/lucene #12497 — scalar-quantization codec | Introduces a scalar-quantized vectors format (int8) into Lucene. | KnnVectorsFormat, scalar quantization, codec SPI | Read the PR diff as a worked example of how a new codec format lands in Lucene. Pair with HNSW in Lucene and the custom-codec lab. |
| apache/lucene #11613 — LUCENE-10577, int8 SQ origin | The original int8 scalar-quantization work (JIRA LUCENE-10577). | quantization math, recall/size trade-off, VectorSimilarityFunction | Read it for the origin story of byte-quantized vectors. The JIRA LUCENE-10577 has the design discussion the PR implements. |
Lucene99ScalarQuantizedVectorsFormat / Lucene99HnswScalarQuantizedVectorsFormat | The shipping SQ formats your bundled Lucene likely uses. | the on-disk .vec quantized layout, rescoring | grep -rn "ScalarQuantized" $LUCENE/lucene/core/src/java/org/apache/lucene/codecs/ to read the format. The SIMD chapter covers the scoring side. |
Lucene104ScalarQuantizedVectorsFormat / Lucene104HnswScalarQuantizedVectorsFormat | Newer formats (custom bits 1/2/4/7/8; Lucene 10.1.0). | configurable-bit quantization, format evolution, BWC of codecs | Diff a Lucene99* against a Lucene104* format to see how a codec evolves while staying back-compatible — a core skill for upgrading Lucene inside OpenSearch. |
Sub-task pattern for this theme: Lucene runs on Gradle + GitHub PRs (with some
history in JIRA LUCENE-NNNN). Build it (./gradlew check,
./gradlew :lucene:core:test, JDK 21), run a single test with -Ptests.seed=...,
and start with a test/documentation-labeled issue. The
contribute-to-Apache-Lucene lab
walks the whole flow. A recurring high-skill OpenSearch task is upgrading the
bundled Lucene version — search the OpenSearch repo for "Upgrade to Lucene".
Theme 3 — Sharding & Scaling (reader/writer separation)
The largest re-architecture in modern OpenSearch: scale reads independently of writes, and scale read capacity to zero when idle. Built on remote store. Read the full story in Sharding, Scaling & Reader/Writer Separation.
| Issue / PR | Description | Exercises | How to start |
|---|---|---|---|
| #7258 — [RFC] Reader and Writer Separation | The founding RFC: why separate read and write scaling at all. | the motivation, the constraints remote store imposes | Read this first — it frames the whole initiative. |
| #14596 — [RFC] Indexing and Search Separation | The refined separation proposal. | routing/allocation changes, search replicas | Read after #7258 as the evolved design. |
| #15237 — [Proposal] Design | The concrete design: interfaces and phasing. | ShardRouting roles, allocation deciders | Skim for the what: which classes change. Cross-check against shard allocation. |
| #15306 — [META] Reader/Writer Separation | The META: the checklist of sub-issues + PRs. | initiative decomposition | Your map. Find an unchecked, scoped sub-issue. |
| #16720 — Scale to Zero | Scale search replicas to zero when there is no read traffic. | search-replica lifecycle, cold-start | Read as the payoff feature of the whole effort. |
| #17299 — implementation PR | A concrete implementation PR for the separation work. | the real code: routing, allocation, recovery | Read the diff to see how the design becomes code. The best way to learn the change surface. |
| #17763 — multi-writer detection RFC | Detect/forbid two writers for one shard under separation. | safety invariants, primary terms, fencing | A distributed-correctness gem. Read with remote store & durability. |
Sub-task pattern for this theme: this is not a place to start with code. Start by being able to explain the change surface — write the summary the META is missing, or improve a test that exercises search replicas. Then find a scoped allocation/routing sub-issue. The vector-aware-allocation capstone builds on this terrain.
Theme 4 — Aggregation Acceleration (star-tree)
Precompute multi-field aggregations into a composite index for bounded-latency analytics. Full treatment in Star-Tree Indexes.
| Issue | Description | Exercises | How to start |
|---|---|---|---|
| #12498 — [RFC] Pre-Compute Aggregations with Star Tree | The founding RFC for the star-tree composite index. | the star-tree data structure, dimension/metric model | Read for the core idea: trade index-time work + space for query-time speed. |
| #13875 — [META] Star Tree Index issue list | The META: every star-tree sub-task. | initiative decomposition, supported aggregations | Your checklist. Many sub-issues add support for one more aggregation type — an ideal bounded task. |
| #14871 — [Star Tree][Search][RFC] resolve aggregation via star tree | How the search path decides to use the star-tree. | the agg → star-tree resolution, query rewriting | Read with aggregations. The resolution logic is where a new contributor can add a supported case. |
Sub-task pattern for this theme: the META (#13875) typically tracks "support aggregation X on the star-tree". Pick one unsupported aggregation, add the mapping/validation + the resolution case + a test. The extend-star-tree capstone is exactly this.
Theme 5 — Tiered Caching
Spill the request cache to an on-disk tier when heap is exhausted. Full treatment in Tiered Caching.
| Issue | Description | Exercises | How to start |
|---|---|---|---|
| #10024 — [Proposal] Tiered caching | The design: on-heap → disk spillover via TieredSpilloverCache. | the pluggable ICache, the cache hierarchy, serialization | Read the proposal, then read tiered caching and circuit breakers & memory. |
| #11464 — [META] perf benchmark | The performance benchmark META for tiered caching. | cache hit-ratio measurement, disk-tier latency | A measurement-first entry: help validate the disk-tier trade-off. No production-code risk. |
Sub-task pattern for this theme: caching is settings-heavy. A great first task is improving the observability — a stat or a clearer setting validation — for the disk tier, then a serialization edge case for a key/value type.
Theme 6 — Distributed Load / Backpressure
Smart, per-shard rejection to prevent cascading failure. Full treatment in Backpressure & Admission Control.
| Issue / PR | Description | Exercises | How to start |
|---|---|---|---|
| #1446 — [Meta] Indexing Backpressure | The umbrella for indexing backpressure. | outstanding-bytes tracking, rejection policy | Read as the map of the whole indexing-pressure effort. |
| #478 — [Meta] Shard level Indexing Back-Pressure | The shard-level smart-rejection design. | per-shard tracking, secondary-parameter rejection | Read for the core mechanism: reject the right shard, not the whole node. |
| #1336 — Shard Indexing Pressure (impl PR) | The implementation PR for shard indexing pressure. | the real tracking code, the rejection path | Read the diff. This is where the design above becomes ShardIndexingPressure. |
Sub-task pattern for this theme: reproduce a rejection under synthetic load, then improve the signal — a clearer rejection reason, a new stat, a test for a secondary-parameter case. The search-backpressure capstone adds a new signal on the search side.
Finding CURRENT Work (the durable skill)
Issue numbers go stale; search queries do not. The most important skill in this
chapter is finding live work yourself. Use GitHub's issue search (web UI or
gh issue list). These queries are real and current.
OpenSearch core (opensearch-project/OpenSearch)
# Good first issues, newest first.
gh issue list -R opensearch-project/OpenSearch \
--label "good first issue" --state open --limit 50
# Explicitly wants help.
gh issue list -R opensearch-project/OpenSearch --label "help wanted" --state open
# Open RFCs — the design frontier.
gh issue list -R opensearch-project/OpenSearch --label "RFC" --state open
# Roadmap-tracked and enhancements in a theme you care about.
gh issue list -R opensearch-project/OpenSearch --label "Roadmap" --state open
gh issue list -R opensearch-project/OpenSearch --label "enhancement" --search "star tree"
Web equivalents (paste into the GitHub search bar):
repo:opensearch-project/OpenSearch is:issue is:open label:"good first issue"
repo:opensearch-project/OpenSearch is:issue is:open label:"help wanted"
repo:opensearch-project/OpenSearch is:issue is:open label:"RFC"
repo:opensearch-project/OpenSearch is:issue is:open label:"enhancement" "concurrent segment search"
repo:opensearch-project/OpenSearch is:issue is:open "Upgrade to Lucene"
k-NN plugin (opensearch-project/k-NN)
gh issue list -R opensearch-project/k-NN --label "good first issue" --state open
gh issue list -R opensearch-project/k-NN --label "help wanted" --state open
gh issue list -R opensearch-project/k-NN --label "RFC" --state open
gh issue list -R opensearch-project/k-NN --search "GPU OR cuVS OR remote build"
Apache Lucene (apache/lucene)
gh issue list -R apache/lucene --label "good first issue" --state open
gh issue list -R apache/lucene --search "vector OR HNSW OR quantization is:open"
# Lucene also tracks design in JIRA (LUCENE-NNNN); the GitHub PR usually links it.
Note: Always check whether an issue is already claimed (someone commented "I am working on this") before you start, and comment to claim it yourself. The contributor mindset section covers this etiquette; jumping a claimed issue is the fastest way to waste your own work.
How to Turn an RFC Into Your Contribution
A real workflow, the way maintainers expect it:
- Pick a theme, not an issue. Choose from the six above based on what you want to learn. The theme decides the subsystem you will master.
- Read the META top-down, then the RFC for the why. Use the RFC-reading grammar. Find the rejected alternatives — they tell you the real constraints.
- Carve out a bounded sub-task. The whole RFC is not yours to land. Find the smallest piece that is independently shippable: a benchmark, a stat, one supported aggregation, one settings fix, one test. Match it to the "sub-task pattern" for that theme above.
- Reproduce before you build. For a bug, write the failing test first (capstone step 2). For a feature, write the benchmark that proves it is needed.
- Claim it in public. Comment on the issue with your plan and scope. This is the design-via-GitHub skill.
- Land it the maintainer way. A focused PR that leaves the system shippable, with tests and a clear description linking the issue. See PR quality and the capstone PR step.
- Defend the trade-offs. When a reviewer pushes back on latency, memory, or BWC, you should already have the numbers — because you started with a benchmark. See responding to feedback.
That sequence — theme → META → bounded sub-task → reproduce → claim → ship → defend — is the entire job at depth 3, compressed. Every chapter in this section is a worked example of it.
Next: Concurrent Segment Search — the cleanest example of an RFC that became a default.