Step 7: Validation

Your fix works and your tests are green on your machine. That is not the bar. The bar is: the full local gate is green before you push, so CI on the PR is green on the first run. A red CI run on your first push is not fatal, but it is a tell — it says you pushed before validating, and it costs a reviewer a round-trip.

This step runs the same checks CI will run, locally, first, in increasing order of cost, and produces a validation report you attach to the PR. The rule: you do not push until you have personally seen every gate go green.

What "Green" Means

OpenSearch CI ("gradle-check") is, in essence, ./gradlew check sharded across runners. "Green" is not "my one test passed." It means all of:

You run these from cheapest to most expensive so a fast failure stops you before you spend 40 minutes on check.

The Gate, in Order

1. Format (seconds)

./gradlew spotlessApply        # fix formatting
./gradlew spotlessJavaCheck    # verify; this is exactly what CI runs

If spotlessApply touched files you didn't mean to change, you have unrelated reformatting staged — strip it (see Step 5).

2. Precommit (a few minutes)

./gradlew precommit

precommit is where most first-time PRs fail CI, because it enforces rules you don't think about: an unregistered Setting, a forbidden API (System.currentTimeMillis() instead of the injected clock; raw java.util.Random), a missing SPDX header, a logger used without a guard. Read each failure literally — the task name tells you which rule (forbiddenApisMain, checkstyleMain, licenseHeaders, loggerUsageCheck).

3. Affected-module tests (minutes)

Don't run the whole world yet. Run the modules your diff touches. Find them from the diff:

git diff origin/main --stat        # which dirs changed?

Then scope to those Gradle projects:

# core search/agg change:
./gradlew :server:test --tests "org.opensearch.search.aggregations.*"

# if you touched a module/ or plugins/ project, run that project too, e.g.:
./gradlew :modules:reindex:test

Always include the class you changed and its whole package — your fix can break a sibling test that exercises the same code path.

4. Integration / REST tests (tens of minutes)

If your bug or fix touches cluster behavior or the REST contract:

./gradlew :server:internalClusterTest --tests "org.opensearch.cluster.*"
./gradlew :rest-api-spec:yamlRestTest

5. The full check scope (long — run once before push)

./gradlew check

This is the closest local mirror of CI. It is long (tens of minutes to over an hour depending on hardware) — run it once, after the cheaper gates are green, right before you push. Tips:

• Use --build-cache (and Gradle's local cache) so unchanged modules aren't rebuilt.
• If check fails in a module you never touched, suspect a flaky test before suspecting your change: re-run just that test with its printed -Dtests.seed=.... If it fails deterministically on main too (stash your fix and check), it's a pre-existing flake — note it; it is not yours to fix here.
• You can scope check to the heavy parts if hardware is limited: ./gradlew precommit :server:test :server:internalClusterTest is a strong proxy for most server-only fixes.

Note: OpenSearch CI on a PR is triggered by maintainers/automation (gradle-check) and reported as a status check on the PR. You cannot make CI green by wishing; you make it green by having run ./gradlew check locally first. The single best predictor of first-pass green CI is "I ran check and read every line of output."

Reading CI on the PR

Once the PR is up (Step 8), the gradle-check status appears on the PR's Checks tab. When it's red:

1. Open the failing check's logs (click "Details" next to gradle-check).

2. Find the failing task and the REPRODUCE WITH: line the test framework prints — it includes the seed, locale, and timezone:

   REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.x.YTests.testZ" \
     -Dtests.seed=ABC123 -Dtests.locale=fr-FR -Dtests.timezone=America/Sao_Paulo
   

3. Run that exact line locally. If it reproduces, it's yours — fix it. If it doesn't reproduce and the same test is flaky on main, it's a pre-existing flaky-test; comment on the PR linking the flaky-test issue and re-trigger CI (a maintainer can re-run, or a gradle-check retry).

Do not push speculative "maybe this fixes CI" commits. Reproduce locally with the seed first, then push a fix you've verified.

The Validation Report Artifact

Produce capstone-work/validation.md — a record of exactly what you ran and what you saw. This goes into (or is summarized in) the PR description; it is the single most reassuring thing a reviewer can read.

# Validation report: #NNNN

## Checkout
- Branch: fix/NNNN-terms-missing-min-doc-count
- Based on: origin/main @ <sha>

## Gates run (all green)
| Gate | Command | Result |
|---|---|---|
| Spotless | `./gradlew spotlessJavaCheck` | PASS |
| Precommit | `./gradlew precommit` | PASS |
| Unit (changed pkg) | `./gradlew :server:test --tests "...aggregations.bucket.terms.*"` | PASS (412 tests) |
| REST-YAML | `./gradlew :rest-api-spec:yamlRestTest` | PASS |
| Full check | `./gradlew check` | PASS (1h03m, --build-cache) |

## Reproducer status
- testMissingWithMinDocCountZeroOnEmptyShard: FAIL on main, PASS with fix
- Negative control testMissingWithMinDocCountOne...: PASS both (unchanged)

## Determinism
- 50 iters + 10 fresh seeds on the new tests: 0 flakes

## Diff scope
- `git diff origin/main --stat`: 2 files (TermsAggregator.java, TermsAggregatorTests.java) + 1 YAML
- No unrelated files touched.

## Notes
- One unrelated flake observed in qa:rolling-upgrade (issue #MMMM); not introduced by this PR.

Deliverable for Step 7

• spotlessJavaCheck, precommit, affected-module tests, and a full ./gradlew check all run locally and green.
• git diff origin/main --stat confirms only the bug's files changed.
• Any failure understood and either fixed or attributed (pre-existing flake, with the issue link).
• capstone-work/validation.md written with the exact commands and results.
• You have not pushed until every gate is green.

Validation / Self-check

Before advancing to Step 8:

1. You ran the gates in cost order and saw each one green — you did not skip precommit or check.
2. git diff origin/main --stat lists only files you intended to change.
3. You can explain every failing test you saw and whether it is yours or a pre-existing flake (with a link).
4. The new tests are deterministic across iters and seeds.
5. capstone-work/validation.md records the literal commands and results, not a vague "all tests passed."
6. You understand that local check green is what produces first-pass green CI; CI is not something you debug after pushing.
7. You did not push speculative fixes to chase CI without a local repro.

Then go to Step 8: Pull Request Preparation.