Lab 1.2: Run Unit and Integration Tests

Background

OpenSearch has one of the most thorough test suites of any open-source distributed system, and a PR is not credible unless it passes the relevant slice of it. The suite is built on Randomized Testing (the same RandomizedRunner Lucene uses): every run picks a random seed, and the test exercises randomized inputs (field types, document counts, cluster topologies, serialization round-trips). This finds bugs ordinary fixed-input tests never would — and it means a failure must be reproduced with the seed it failed under, not dismissed as flaky.

This lab teaches you to run unit and integration tests, scope them tightly, reproduce a randomized failure deterministically, run the precommit and full-check gates, and read the HTML reports under build/reports/tests.

Why This Lab Matters for Contributors

• CI's gradle-check runs a superset of what you will run here; passing locally first turns a five-round review into one.
• Knowing how to scope tests with --tests is the difference between a 5-second feedback loop and a 90-minute one.
• Reproducing a randomized failure from its seed is the core debugging skill for OpenSearch — you will use it constantly in Level 5 and Level 8.
• Reading the test reports tells you what failed and why, not just that something did.

Prerequisites

• Lab 1.1 complete: a clean ./gradlew assemble.
• Familiarity with the project paths printed by ./gradlew projects.

Step-by-Step Tasks

Step 1: The Test Types

OpenSearch tests come in tiers. Know which tier you are running before you run it.

The two you will touch most at Level 1 are OpenSearchTestCase (fast, no cluster) and OpenSearchIntegTestCase (slow, real cluster). The distinction is covered in depth in Level 5.

# See how many of each style live in :server (rough proxy via base class):
grep -rln "extends OpenSearchTestCase"            server/src/test | wc -l
grep -rln "extends OpenSearchSingleNodeTestCase"  server/src/test | wc -l
grep -rln "extends OpenSearchIntegTestCase"       server/src/internalClusterTest 2>/dev/null | wc -l

Step 2: Run a Single Unit-Test Class

Never start with the whole :server:test task — it runs thousands of classes. Scope with --tests.

# A small, fast, dependency-free class — good first run:
./gradlew :server:test --tests "org.opensearch.common.UUIDTests"

--tests accepts a fully-qualified class name. You can also use wildcards and pick a single method:

# All test classes in a package:
./gradlew :server:test --tests "org.opensearch.cluster.*"

# A single method on a class:
./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests.testToXContent"

# Every test class whose name ends in "Tests" under a subtree:
./gradlew :server:test --tests "org.opensearch.index.engine.*Tests"

Tip: ./gradlew :server:test --tests "X" will recompile changed sources first. If you only changed test code, the main-source compile is skipped — the loop stays fast.

Step 3: Run Integration (In-JVM Cluster) Tests

Integration tests spin up a real multi-node cluster inside the JVM via InternalTestCluster. They live under a separate source set and run via a separate task:

# One integration-test class (note the *IT naming convention):
./gradlew :server:internalClusterTest --tests "org.opensearch.cluster.SimpleClusterStateIT"

These are much slower (seconds-to-minutes per class) because each spins up nodes, allocates shards, and tears everything down. Run them scoped; never blanket-run :server:internalClusterTest.

Step 4: Randomized Testing and -Dtests.seed

Every test run uses a random seed. When a test fails, Gradle prints a reproduction line that pins the seed (and other randomization) so the failure is deterministic:

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.Foo.testBar" \
  -Dtests.seed=DEADBEEFCAFE -Dtests.locale=en-US -Dtests.timezone=UTC ...

To reproduce, copy that line verbatim:

./gradlew ':server:test' --tests "org.opensearch.SomeTest.testThing" -Dtests.seed=DEADBEEFCAFE

You can also force a seed to make a run deterministic, or iterate a flaky test to flush out the failing seed:

# Force a specific seed:
./gradlew :server:test --tests "org.opensearch.SomeTest" -Dtests.seed=DEADBEEFCAFE

# Run the same test many times with fresh random seeds to surface flakiness:
./gradlew :server:test --tests "org.opensearch.SomeTest" -Dtests.iters=50

Warning: A randomized failure is not automatically "flaky." If it reproduces under its seed, it is a real bug that only some inputs trigger. Only when it fails on one seed and passes on the same seed on re-run is it genuinely non-deterministic. Flaky-test handling — muting with @AwaitsFix(bugUrl=...), never @Ignore — is Level 5 material.

Step 5: Run the Precommit Gate

precommit is the static-analysis half of the gate: checkstyle, forbidden APIs, license/SPDX headers, dependency checks, loggerUsageCheck, and more. It is fast relative to the tests and CI runs it on every PR.

./gradlew precommit

Pair it with formatting:

./gradlew spotlessApply      # fix formatting
./gradlew spotlessJavaCheck  # verify formatting (what CI checks)

Step 6: The Full Check (Know What It Is, Run It Sparingly)

check is the everything gate: unit tests + precommit + integration tests for the project(s) you target. It is long-running. Scope it to a project, and only run it before a substantial PR.

# The full gate for the server project (long — minutes to tens of minutes):
./gradlew :server:check

# The whole repo (very long — reserve for pre-release / large changes):
# ./gradlew check

For most PRs you will run, in order: the affected --tests, then spotlessApply, then precommit. That mirrors what review will demand.

Step 7: Read the Test Reports

Whether a test passes or fails, Gradle writes an HTML report. For a failure you want the report, not just the console tail.

# Reports live under each project's build/reports/tests/<taskName>/index.html
find server/build/reports/tests -name index.html
# e.g. server/build/reports/tests/test/index.html
#      server/build/reports/tests/internalClusterTest/index.html

# Open it (macOS):
open server/build/reports/tests/test/index.html

The report gives you, per class and per method: pass/fail/skip counts, the full stack trace of each failure, and the captured stdout/stderr (where the test logged the reproduction line and any cluster diagnostics). The raw JUnit XML is alongside, under server/build/test-results/test/*.xml — useful for grepping or CI parsing.

# Quickly find which methods failed without opening a browser:
grep -rl 'testcase' server/build/test-results/test/*.xml >/dev/null 2>&1
grep -rE '<(failure|error)' server/build/test-results/test/*.xml | head

Implementation Requirements

This lab has no code to implement. Deliverables:

1. One OpenSearchTestCase-style unit class run green via --tests.
2. One *IT integration class run green via :server:internalClusterTest --tests.
3. A demonstrated seed reproduction: take any test's printed REPRODUCE WITH line and re-run it.
4. A clean ./gradlew precommit.
5. The path to your build/reports/tests/test/index.html and a one-line description of what it shows.

Troubleshooting

A test "passes locally but fails in CI" (or vice versa)

Almost always a seed difference. Reproduce CI's failure by copying the -Dtests.seed=... (and -Dtests.locale/-Dtests.timezone) from the CI log into your local command. Randomization includes locale and timezone — a locale-sensitive bug will only appear under certain -Dtests.locale values.

OutOfMemoryError during :server:test

Test JVMs are forked. Raise the daemon/test heap via org.gradle.jvmargs in gradle.properties (see Lab 1.1), and scope your run with --tests instead of running the whole suite.

Integration test hangs or leaks threads

OpenSearchIntegTestCase has aggressive leak detection; a hang at teardown usually means the test (or your change) left a thread/Closeable open. Read the report's captured output — the framework names the leaked resource. This is exactly the signal Level 5 teaches you to act on.

"Tests are UP-TO-DATE and won't re-run"

Gradle caches test results. Force a re-run:

./gradlew :server:test --tests "org.opensearch.common.UUIDTests" --rerun-tasks

precommit fails on a file you did not touch

Confirm it is pre-existing (some failures depend on branch state). Run on a clean checkout of main to establish the baseline:

git stash && ./gradlew precommit ; git stash pop

If main is clean and your change introduced it, the failure is yours to fix.

Expected Output

A passing scoped unit run:

> Task :server:test

org.opensearch.common.UUIDTests > testRandomUUID PASSED
org.opensearch.common.UUIDTests > testTimeBasedUUID PASSED

BUILD SUCCESSFUL in 38s

A failure prints the all-important reproduction line:

org.opensearch.cluster.SomeTest > testThing FAILED
    java.lang.AssertionError: expected:<3> but was:<2>
        at org.opensearch.cluster.SomeTest.testThing(SomeTest.java:91)

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.cluster.SomeTest.testThing" \
  -Dtests.seed=A1B2C3D4 -Dtests.locale=en-US -Dtests.timezone=UTC

BUILD FAILED in 41s

Stretch Goals

1. Find a serialization round-trip test and run it. These verify wire/XContent BWC and are the guardians of the protocol you must respect in Level 9:

   grep -rln "extends AbstractWireSerializingTestCase" server/src/test | head
   # pick one and run it with --tests
   

2. Stress a single test for flakiness. Run a chosen test 100 times with fresh seeds:

   ./gradlew :server:test --tests "org.opensearch.common.UUIDTests" -Dtests.iters=100
   

3. Run a module's REST-YAML tests and watch a black-box test of the REST API:

   ./gradlew :modules:reindex:yamlRestTest --tests "*" 2>&1 | tail -30
   

4. Diff two runs' reports. Run a test twice with two explicit different seeds and compare what the report captured — see the randomized inputs differ.

Validation / Self-check

You are done when you can answer these without notes:

1. What does --tests accept, and how do you scope to a single method? A whole package?
2. Why is :server:internalClusterTest separate from :server:test, and which base class backs it?
3. What is -Dtests.seed for, and where do you find the value to reproduce a given failure?
4. When is a randomized failure a real bug, and when is it genuinely non-deterministic?
5. What does precommit check that the test task does not?
6. Where do the HTML report and the raw JUnit XML for a :server:test run live?
7. Which three commands, in order, would you run before opening a PR that touches :server?

See the testing internals in depth in Level 5. Next: Lab 1.3 — Launch a Single-Node Cluster and Index Data.