Reading a Large OpenSearch Codebase

OpenSearch core is large. The server/ module alone is hundreds of thousands of lines of Java, and it sits on top of Apache Lucene (another large codebase you will read into), surrounded by libs/, dozens of bundled modules/ and plugins/, a test framework, and a qa/ BWC suite. No single human holds it all in their head — not the founders, not the most prolific maintainers. The skill is not memory; it is navigation.

This chapter gives you the strategies maintainers actually use to find their way through code they did not write and do not remember. Everything here assumes a clone at ~/os-src (git clone https://github.com/opensearch-project/OpenSearch.git ~/os-src).

Module Map First

Before reading any code, learn the module shape. OpenSearch is a Gradle multi-project build, so the authoritative module list comes from Gradle, not from guessing at directories:

cd ~/os-src
./gradlew projects | head -60

That prints the project tree (:server, :libs:core, :modules:transport-netty4, :plugins:repository-s3, :test:framework, :qa:..., and so on). Cross-reference it with the build files on disk:

find . -name build.gradle -maxdepth 3 | sort

The top-level directories that matter for ~90% of core work:

Pin a one-sentence-per-module note file. You will re-read it constantly:

mkdir -p ~/os-notes
$EDITOR ~/os-notes/module-map.md

Note: The big functional plugins — security, k-NN, sql, alerting, ml-commons, index-management — live in separate repos under opensearch-project/, not in this tree. If you grep core for a security filter or a k-NN field type and find nothing, that is why. Clone the relevant repo separately.

Strategy 1: Start From the Public Surface, Trace Inward

Every external interaction with OpenSearch enters through one of two surfaces: the REST layer (HTTP on :9200) for users, or the transport layer (internal RPC on :9300) for node-to-node traffic. That makes Rest*Action and Transport*Action classes the only mandatory entry points. Read inward from them.

The canonical reading order for a request:

RestXxxAction          (parses HTTP, builds a request, calls NodeClient)
   ↓
TransportXxxAction     (the registered handler; the business logic entry)
   ↓
XxxService             (e.g. SearchService, IndicesService — the subsystem)
   ↓
Lucene / Engine / ClusterState   (the actual mechanism)

Find the REST handlers and the actions that pair with them:

cd ~/os-src
# All REST handlers (the HTTP surface):
find server -name 'Rest*Action.java' | sort | head -40
# All transport actions (the internal surface):
find server -name 'Transport*Action.java' | sort | head -40

A Rest*Action is wired in ActionModule; so is the Transport*Action it dispatches to. When you do not know how a feature is reachable, grep ActionModule for the registration:

grep -n "SearchAction\|RestSearchAction" \
  server/src/main/java/org/opensearch/action/ActionModule.java

That registration line tells you the ActionType, the request/response classes, and the transport handler — the whole arc in one place. See the action-framework deep dive and the REST-layer deep dive for the machinery; here we only care that the surface is where you start reading.

Strategy 2: The Wire Surface Is the Real Contract

In a classic Apache project, protobuf .proto files are the source of truth for anything serialized. OpenSearch has no protobuf on the core hot path (historically — proto is appearing in some newer subsystems). The equivalent contract is twofold:

1. Wire serialization: classes implement Writeable and read/write themselves through StreamInput/StreamOutput. Polymorphic types resolve through NamedWriteableRegistry. Compatibility is enforced by Version gating inside those readFrom/writeTo methods (if (out.getVersion().onOrAfter(Version.V_3_0_0)) { ... }).
2. REST specs: the JSON files under rest-api-spec/ define every REST endpoint's path, parameters, and body — the user-facing contract.

To learn the wire shape of any serialized type, read its Writeable implementation, not a schema file:

# The serialization primitives:
find libs/core -name 'StreamInput.java' -o -name 'StreamOutput.java'
# Find a type's writeTo/readFrom and any Version gating in it:
grep -n "writeTo\|StreamInput\|getVersion().onOrAfter" \
  server/src/main/java/org/opensearch/action/search/SearchRequest.java

To learn the REST contract:

ls rest-api-spec/src/main/resources/rest-api-spec/api/ | grep search
sed -n '1,40p' rest-api-spec/src/main/resources/rest-api-spec/api/search.json

Practical rule: if you are changing a field that crosses a node boundary or changes a REST body, you are changing compatibility. Stop and read the compatibility chapter and the serialization-BWC deep dive before you write the change.

Strategy 3: IDE Call Hierarchy + git log -S / git log -G

Two tools, used together, replace most speculative reading.

Call hierarchy (IntelliJ Ctrl-Alt-H, VS Code "Show Call Hierarchy") answers "who calls this?". Use it on an entry point like SearchService.executeQueryPhase to find every caller in production code and tests. The test call sites are often the clearest documentation.

git log -S and git log -G answer "when and why did this code appear?". -S matches commits that changed the count of a string (added or removed it); -G matches commits whose diff touches lines matching a regex (better for following an existing identifier):

cd ~/os-src
# When was this method introduced / removed?
git log -S "executeQueryPhase" --oneline -- server/

# Which commits touched lines mentioning segment replication config?
git log -G "segment.replication" --oneline -- server/ | head

Pick the oldest relevant commit and read its full message. OpenSearch commits reference the PR number ((#1234)) and usually the issue:

git show <sha> --stat | head -30
# Look for "(#NNNN)" — that is the pull request.

That PR — its description, its review thread, the issue it closed — is the design discussion. It is often more valuable than the code. The next chapter, Design via GitHub, is entirely about turning that (#NNNN) into the "why."

Strategy 4: Tests Are Executable Spec

The OpenSearch test suite is the cheapest way to learn what a class does, because it exercises the class with assertions about expected behavior. There are three tiers, and you read them in this order:

For any class Foo.java, look for FooTests.java; the test method names alone form a behavior spec:

# Find the unit test for SearchRequest:
find server -name 'SearchRequestTests.java'
# Read the behaviors it asserts:
grep "public void test" $(find server -name 'SearchRequestTests.java')

For end-to-end behavior, the integration tests are the truth:

find server -name '*SearchIT.java' | head

And the REST-YAML tests show exactly what an HTTP client sees, including the skip blocks that document version-specific behavior:

ls rest-api-spec/src/main/resources/rest-api-spec/test/search/
sed -n '1,40p' rest-api-spec/src/main/resources/rest-api-spec/test/search/10_source_filtering.yml

To actually run a scoped slice while you read:

./gradlew :server:test --tests "org.opensearch.action.search.SearchRequestTests"

Read tests before guessing how a feature behaves. The test author already encoded the behavior you are about to reverse-engineer.

Strategy 5: Keep a Reading Log

Maintainers have working memory of the codebase because they wrote a lot of it. You don't. Compensate with notes. Keep one file and append a dated entry every time you trace a path:

cat >> ~/os-notes/reading-log.md <<'EOF'

## 2026-06-16 — Search request path
- RestSearchAction.prepareRequest (server/.../rest/action/search/) parses query, builds SearchRequest
- → client.execute(SearchAction.INSTANCE, searchRequest, listener)
- → TransportSearchAction.doExecute (server/.../action/search/) — coordinating node
- → fans out per-shard; SearchService.executeQueryPhase (QueryPhase) then executeFetchPhase
- → SearchPhaseController.reduce merges shard results on the coordinating node
EOF

Re-reading three months later, the log is gold. Without it, you re-trace the same path from zero and waste an afternoon. The best contributors treat the reading log as part of the work, not overhead.

Worked Exercise: RestSearchAction → TransportSearchAction → SearchService (90 minutes)

Goal: in 90 minutes, trace a _search request from the HTTP edge to per-shard execution, and record the path in your reading log. Use only the tools above. Do not read this chapter's hop names back — discover them.

Step 1 (15 min) — Find the REST entry

cd ~/os-src
find server -name 'RestSearchAction.java'
grep -n "prepareRequest\|registerHandlers\|routes()" \
  $(find server -name 'RestSearchAction.java')

Read prepareRequest. Note two things: it parses the query/body into a SearchRequest, and it ends by calling something like client.execute(SearchAction.INSTANCE, searchRequest, ...) (or a channel consumer that does). Write the file + method in your log.

Step 2 (15 min) — Find where the action is registered

grep -n "SearchAction\b" server/src/main/java/org/opensearch/action/ActionModule.java

This proves the mapping from SearchAction.INSTANCE to TransportSearchAction. Confirm the request and response classes named in the registration.

Step 3 (20 min) — Enter the transport action

find server -name 'TransportSearchAction.java'
grep -n "doExecute\|executeSearch\|class TransportSearchAction" \
  $(find server -name 'TransportSearchAction.java')

Read doExecute. This is the coordinating node logic: it resolves indices to shards, decides query-then-fetch vs DFS, and dispatches per-shard search requests. Note where it builds the shard iterator and where it hands off to the search phases.

Step 4 (20 min) — Drop into per-shard execution

find server -name 'SearchService.java'
grep -n "executeQueryPhase\|executeFetchPhase\|createContext" \
  $(find server -name 'SearchService.java')

executeQueryPhase runs on the node holding each shard. Trace how it builds a SearchContext/DefaultSearchContext and invokes QueryPhase. This is the boundary where OpenSearch hands off to Lucene. (Depth here belongs to the search-execution deep dive — do not go deeper now; just confirm the hop.)

Step 5 (10 min) — Verify with a test, then run it live

# A test that exercises the path end to end:
find server -name '*SearchIT.java' | head -1
# Optional: launch a node and hit it for real (separate terminal):
# ./gradlew run
# curl -s 'localhost:9200/_search?size=0' | head

Step 6 (10 min) — Record it

Append the five-hop trace to ~/os-notes/reading-log.md, citing the file and method for each hop. If you can reproduce the trace tomorrow without this page, you have the navigation skill.

Validation: Artifacts To Keep

After this chapter you should have produced and saved:

1. ~/os-notes/module-map.md — one sentence per top-level dir and per major server/ package, derived from ./gradlew projects and find . -name build.gradle.
2. ~/os-notes/reading-log.md — the _search trace from the exercise above, with file + method per hop.
3. One git log -S (or git log -G) command you ran and the PR number (#NNNN) it surfaced, saved to the log — your bridge into Design via GitHub.
4. The name of one unit test, one serialization test, and one *IT.java that exercise the search path — your "executable spec" set.

You have absorbed this chapter when you can find any feature in OpenSearch within 10 minutes, starting from a Rest*Action or Transport*Action, without re-reading these steps. The next chapter — Understanding Design Through Code and GitHub — tells you where the decisions behind that code actually lived.