Overview & Prerequisites

This section is the on-ramp. Before you read a single line of IndexShard.java or trace a search request through TransportSearchAction, you need a working build, a running local cluster, a GitHub identity wired for contribution, and a clear mental map of how the curriculum is structured. This page gets you there. Budget two to four hours for a cold setup; most of that is the first Gradle build downloading the world.

This curriculum will not hold your hand. It assumes you are a strong backend/distributed-systems engineer who can read unfamiliar Java without a guide. What it will do is point you at the exact parts of OpenSearch that matter, give you the right questions, and make you prove competence at each gate. The setup below is the first gate: if you cannot build OpenSearch and hit localhost:9200, nothing else in the curriculum will work.

What You Are Setting Up (the whole picture)

┌──────────────────────────────────────────────────────────────────────────┐
│  Your laptop                                                              │
│                                                                          │
│   JDK 21 (system)  ── runs ──►  IntelliJ / your editor                   │
│         │                                                                │
│         │  imports as a Gradle project                                   │
│         ▼                                                                │
│   ~/src/OpenSearch  ◄── git clone ── github.com/opensearch-project/...    │
│         │                                                                │
│         │  ./gradlew  (wrapper provisions its OWN bundled JDK 21)        │
│         ▼                                                                │
│   ./gradlew run  ──►  single-node cluster, REST on :9200, transport :9300 │
│         ▲                                                                │
│         │  curl / your tests hit this                                    │
│   git commit -s  ──►  DCO Signed-off-by  ──►  PR on GitHub               │
└──────────────────────────────────────────────────────────────────────────┘

Two facts shape everything:

1. OpenSearch builds with Gradle, not Maven. You drive it through the ./gradlew wrapper, which downloads the correct Gradle version and a bundled JDK matched to the branch. You do not install Gradle, and you do not need your system JDK to exactly match the build JDK.
2. Contribution happens on GitHub — issues and pull requests on github.com/opensearch-project/OpenSearch. There is no JIRA, no CLA. Every commit needs a DCO sign-off (git commit -s) and every PR needs a CHANGELOG.md entry.

Step 1 — Install the toolchain

# Verify your toolchain. JDK must report 21 (or compatible).
java -version      # openjdk version "21.x"
git --version      # git version 2.x
curl --version     # any recent curl

Note: You do not install Gradle. The repository ships ./gradlew and gradle/wrapper/. The wrapper also provisions a build JDK via the toolchain mechanism, so a slightly different system JDK is usually fine for running tools — but install JDK 21 to keep your IDE and the build aligned and avoid surprises. If ./gradlew --version runs, your wrapper is healthy.

On macOS the path of least resistance is a JDK distribution like Temurin/Corretto 21 via your package manager; on Linux use your distro's openjdk-21-jdk or a tarball; set JAVA_HOME accordingly.

Step 2 — Clone the repositories

You will spend ~90% of your time in the core engine repo. Clone Dashboards only when the cross-repo labs (Lab P1+) call for it.

mkdir -p ~/src && cd ~/src

# The core engine — your home for the whole curriculum.
git clone https://github.com/opensearch-project/OpenSearch.git
cd OpenSearch

# Confirm you are on the development line (main / 3.x).
git branch -a | head
git log --oneline -5

# OPTIONAL — only needed for the Dashboards cross-repo labs (Section: Plugin Labs).
cd ~/src
git clone https://github.com/opensearch-project/OpenSearch-Dashboards.git

Warning: The first ./gradlew invocation downloads Gradle, the bundled JDK, and a large dependency set. Do it on a good connection and expect 15–40 minutes the first time. Subsequent builds use the Gradle daemon and local caches and are far faster.

Read these four files in the repo root before you build anything — they are the project's own contract with you:

cd ~/src/OpenSearch
ls CONTRIBUTING.md DEVELOPER_GUIDE.md TESTING.md CHANGELOG.md MAINTAINERS.md

Step 3 — First build and first run

cd ~/src/OpenSearch

# Sanity-check the wrapper (provisions Gradle + bundled JDK on first call).
./gradlew --version

# Assemble a runnable local distribution under distribution/archives/.
./gradlew localDistro          # first run: slow; later runs: minutes

# Launch a single-node cluster straight from source — REST on :9200.
# Leave this running in one terminal.
./gradlew run

In a second terminal, prove the cluster is alive — this is the canonical health check you will run hundreds of times:

curl -s localhost:9200 | head -20
curl -s 'localhost:9200/_cluster/health?pretty'

You want "status" : "green" (or "yellow" for a single node with replicas unassigned — that is normal and expected, not a failure).

{
  "cluster_name" : "runTask",
  "status" : "green",
  "number_of_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "unassigned_shards" : 0
}

Note: ./gradlew run starts security-disabled core OpenSearch (the security plugin lives in a separate repo and is not part of ./gradlew run). That is exactly what you want for development — plain HTTP on :9200, no TLS, no auth.

To run a single test (your fast feedback loop for the entire curriculum):

# A whole test class:
./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests"

# A single method:
./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests.testToXContent"

If ./gradlew run serves :9200 and :server:test passes a known-good class, your environment is solid.

Step 4 — Import into IntelliJ (as a Gradle project)

OpenSearch is a Gradle project. Do not try to import it as a Maven or "from sources" project.

1. IntelliJ → File → Open → select the ~/src/OpenSearch directory (open the folder, not a single file).
2. When prompted, choose Open as Project and let IntelliJ detect the Gradle build. Accept "Trust Project".
3. Gradle JVM: set it to your JDK 21 in Settings → Build Tools → Gradle. If imports fail with toolchain errors, point IntelliJ's Gradle JVM at the same JDK 21 the wrapper uses.
4. Let the initial Gradle sync finish (it indexes server/, libs/, modules/, plugins/, …). This takes a while the first time.
5. Verify navigation works: press Go to Class and open RestSearchAction, TransportSearchAction, IndexShard. If "Go to Class" finds them, your index is healthy.

Tip: Run/debug a test directly from the IDE gutter once the Gradle import is done. Being able to set a breakpoint in IndexShard.applyIndexOperationOnPrimary(...) and step through it is the single highest-leverage skill in this curriculum. Eclipse users: import via File → Import → Gradle → Existing Gradle Project.

Step 5 — Configure Git for DCO sign-off

OpenSearch requires a Developer Certificate of Origin sign-off on every commit. This is not a CLA — it is a one-line Signed-off-by: trailer asserting you have the right to contribute the code. The PR check (DCO) fails if any commit is missing it.

# Set the identity that will appear in your Signed-off-by line.
# These MUST match the GitHub account/email you contribute from.
git config --global user.name  "Your Real Name"
git config --global user.email "you@example.com"

# Sign off every commit with -s. This appends:
#   Signed-off-by: Your Real Name <you@example.com>
git commit -s -m "Fix off-by-one in date_histogram bucket key"

# Forgot -s on the last commit? Amend it:
git commit --amend -s --no-edit

# Forgot it across several commits? Re-sign a range:
git rebase --signoff HEAD~3

A correctly signed commit message ends with:

Fix off-by-one in date_histogram bucket key

Signed-off-by: Your Real Name <you@example.com>

Warning: The name/email in Signed-off-by: must match your git identity exactly, and the email should be one GitHub recognizes for your account. Mismatches make the DCO check fail and force you to rewrite history.

You will also fork the repo on GitHub and push branches to your fork; that flow is covered in Level 2 and in the contributor-mindset chapter on PR quality. For now, just get your local identity and sign-off working.

Step 6 — Create accounts and join the community

OpenSearch is developed in the open. Get plugged into the channels now so you are not a stranger when you open your first issue or PR.

Large designs and roadmap items live as GitHub issues labeled RFC, meta, or proposal — not in a wiki. Bookmark the issues list: https://github.com/opensearch-project/OpenSearch/issues and learn the labels you will use constantly: good first issue, help wanted, bug, enhancement, flaky-test, untriaged, backport 2.x.

How the Curriculum Fits Together

The curriculum is 9 levels of core engineering, 5 supporting sections, and a capstone.

flowchart TD
    L1[Level 1: Lucene + OpenSearch Foundation] --> L2[Level 2: Contributor Onboarding]
    L2 --> L3[Level 3: Architecture / Request Path]
    L3 --> L4[Level 4: Cluster Coordination + State]
    L4 --> L5[Level 5: Testing + Debugging]
    L5 --> L6[Level 6: Indexing + Storage Engine]
    L6 --> L7[Level 7: Search + Aggregations]
    L7 --> L8[Level 8: Real Issue Contribution]
    L8 --> L9[Level 9: Advanced Maintainer / TSC]
    L9 --> CAP[Capstone: full contribution cycle]

    DD[Deep Dives x24] -.referenced by.-> L3
    DD -.-> L4
    DD -.-> L6
    DD -.-> L7
    PL[Plugin / Cross-Repo Labs] -.-> L8
    GOV[Release + Governance] -.-> L9

The deep dives are not optional reading — they are where the real depth lives. A level says "trace a search request"; the Search Execution deep dive is where you learn how QueryPhase and FetchPhase actually work. Treat the levels as the spine and the deep dives as the muscle.

How to Use the Labs

Every lab follows the same shape, so you always know where you are:

• Background and Why This Lab Matters for Contributors — the why before the how.
• Prerequisites — what must already work (usually a green ./gradlew run and a prior lab).
• Step-by-Step Tasks — numbered, with real ./gradlew/curl/git/grep commands.
• Implementation Requirements / Deliverables — checkboxes you must satisfy.
• Troubleshooting, Expected Output, Stretch Goals.
• Validation / Self-check — 5–7 questions or exercises that gate completion.

Rules for the labs:

1. Run every command. This is a hands-on apprenticeship; reading is not doing.
2. When a lab gives a grep/find, run it rather than trusting a line number — code moves between branches, so the curriculum points you at code with commands instead of fabricated lines.
3. Do not advance past a lab's Validation section until you can answer it without notes.
4. Keep ./gradlew run alive in a dedicated terminal for any lab that hits :9200.

You Are Ready When…

Run this and confirm every box before opening Level 1:

• java -version reports JDK 21; git --version is 2.x.
• ~/src/OpenSearch is cloned and you have read CONTRIBUTING.md, DEVELOPER_GUIDE.md, TESTING.md.
• ./gradlew --version runs and provisions cleanly.
• ./gradlew localDistro produced an archive under distribution/archives/.
• ./gradlew run serves curl -s localhost:9200 and _cluster/health returns green/yellow.
• ./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests" passes.
• IntelliJ imported the project as Gradle and Go to Class finds RestSearchAction.
• git config user.name/user.email are set and git commit -s adds a Signed-off-by: line.
• You have a GitHub account and have skimmed the OpenSearch issues list and its labels.
• You have joined (or bookmarked) the forum and Slack.

# A 5-minute "am I ready" smoke test (run from ~/src/OpenSearch):
java -version
./gradlew --version | head -3
./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests" 2>&1 | tail -5
# In another terminal, with ./gradlew run live:
curl -s 'localhost:9200/_cluster/health?pretty'

If any box is unchecked, fix it now. A broken baseline means every later ./gradlew test and every curl will produce confusing failures that hide the real work.

Where to Go Next

• OpenSearch Warm-Up: From User to Contributor — the most important page in this section. Run OpenSearch as a user across five real scenarios, then bridge each one to the org.opensearch.* source. Read this before Level 1.
• 16-Week Plan — a calendar that maps Levels 1–9 + capstone onto 16 weeks, with weekly reading, hands-on tasks, GitHub issue practice, and exit checkpoints.
• Milestones: M1–M9 — the competence gates. Each milestone has skills, self-check questions, and a 20-point rubric.

Continue to the Warm-Up, or jump straight to Level 1: Lucene and OpenSearch Foundation.