Level 8: Real Issue Contribution

Every level before this one was curated. You built Tez from source against a known-good master, you traced state machines someone chose for you, and you ran labs with a known answer. Level 8 is different. Here you work the way Tez contributors actually work: you mine issues.apache.org/jira/projects/TEZ for a real open issue, reproduce it deterministically, root-cause it, fix it with a minimal diff, write the test that fails-then-passes, and open a GitHub pull request that a committer will actually merge.

There is no answer key. The JIRA may be mis-triaged, five years stale, or already half-fixed. The "obvious" fix may be wrong. The repro may be intermittent. That is the job. This level teaches you the loop the rest of your contributor life runs on — the same loop the Capstone grades you on — and it hands you less scaffolding than any prior level.

Learning Objectives

By the end of Level 8 you must be able to:

1. Mine the TEZ JIRA with real JQL for tractable open issues and triage them by reproducibility signals (stack trace, version, component, reporter responsiveness).
2. Choose the right reproduction harness for a bug class: the tez-dag unit-test state-machine harness, a local-mode DAG, or a MiniTezCluster integration test.
3. Implement a minimal-diff fix that survives the build gates Tez actually enforces: Spotless (bound to the validate phase — it runs on every build), RAT, Checkstyle, and SpotBugs.
4. Write the fails-then-passes test in the same harness the module already uses.
5. Open a merge-quality PR against apache/tez titled TEZ-XXXX: <description>, link it to the JIRA, and read the Yetus precommit report it triggers.
6. Improve DAG failure diagnostics as a focused, low-risk, high-value contribution type.

The Real Tez Contribution Workflow

Tez is a JIRA-plus-GitHub project. The issue of record lives in Apache JIRA; the code review happens on a GitHub pull request. You need both halves.

flowchart LR
    J[JIRA: TEZ-XXXX<br/>issues.apache.org] --> R[Reproduce<br/>fails on master]
    R --> F[Fix + test<br/>minimal diff]
    F --> P["PR to apache/tez<br/>title: TEZ-XXXX: ..."]
    P --> CI[GitHub Actions build<br/>+ Jenkins/Yetus precommit]
    CI -->|comments posted to PR| REV[Committer review]
    REV -->|changes requested| F
    REV -->|approved| M["Squash-merge:<br/>TEZ-XXXX: ... (#PR) (Author reviewed by Reviewer)"]

Verify this yourself in your checkout — the git history is the process documentation:

cd ~/src/oss-repos/tez   # your real checkout
git log --oneline -5

330fdc8f1 TEZ-4711: Normalize ASF license header (#488) (Raghav Aggarwal reviewed by Laszlo Bodor)
b6b5d42fe TEZ-4718: Modernize Jenkins and Yetus integration to ensure full build and UT run on every PR (#498) ...
d0fa41151 [INFRA] Set up default rulesets for default and release branches (#499) (reviewed by Laszlo Bodor)

Read the pattern: TEZ-XXXX: <summary> (#<PR number>) (<Author> reviewed by <Reviewer>). The committer squash-merges your PR and appends the review credit. Your job is the first half of that line: a JIRA number, a one-line summary, and a PR.

Note: Older TEZ JIRAs are littered with attached .patch files and the "Patch Available" status — that was the pre-GitHub workflow, and you will still see it when you read old issues in Lab 8.1. Do not imitate it. The current process, verified from the repository's own Jenkinsfile and commit history, is a GitHub PR; the JIRA stays the discussion and tracking record.

What CI actually runs

Two independent systems examine your PR. Know them before you push:

Details worth knowing from dev-support/tez-personality.sh (read it — it is short): the personality forces compile and unit to run against the whole project, adds the -Ptest-patch profile (Xlint, 9999 max warnings) during the compile phase, and adds -Pspotbugs for the SpotBugs pass. The Jenkinsfile passes --tests-filter=checkstyle, meaning checkstyle findings are reported — a checkstyle nit will draw a reviewer comment even when it does not turn the build red.

The local gates

These run on your machine, driven by the root pom.xml (verified — read the <plugins> sections):

Because Spotless runs at validate, a stray trailing space or a missing license header fails mvn clean install before it compiles a single class. Run mvn spotless:apply before every commit and this class of CI failure disappears.

Required Reading

Before the labs, read these in your Tez checkout — none is long:

And from this book: patch quality, JIRA review, and the Capstone overview — Level 8 is the Capstone's dress rehearsal.

Source Areas You Will Touch

Choosing the Right Issue

Good first contributions, in rough order of acceptance likelihood:

Rule: start with Minor or Trivial priority JIRAs, and with bug classes that have a deterministic repro. Do not attempt Blocker/Critical issues until you have merged patches behind you — you would be occupying an issue a committer needs fixed now.

Labs

Deliverables

You must demonstrate all of the following before moving on:

• A triage note for a real open TEZ JIRA: symptom, suspected component, reproducibility signals, assignee check, plan.
• A deterministic reproducer that fails on current master, in the correct harness (unit / local-mode DAG / MiniTezCluster), with the master commit hash recorded (Lab 8.1).
• A minimal-diff fix plus a test that fails without the fix and passes with it (Lab 8.2).
• Local gates green: mvn spotless:apply then mvn clean install -DskipTests, mvn test -pl <module> -Dtest=<YourTest>, mvn apache-rat:check.
• A PR (or complete PR-ready branch) titled TEZ-XXXX: <description>, with the JIRA and PR cross-linked.
• One diagnostics improvement with a unit test asserting the new message (Lab 8.3).

Common Mistakes

How to Verify You Are Done

cd ~/src/oss-repos/tez
# 1. Your repro exists and fails on clean master
git stash && mvn test -pl <module> -Dtest=<YourReproTest> ; git stash pop   # expect FAIL
# 2. Fix applied: same test passes
mvn test -pl <module> -Dtest=<YourReproTest>                                # expect PASS
# 3. Gates
mvn spotless:apply && mvn clean install -DskipTests -Dmaven.javadoc.skip=true
mvn apache-rat:check
# 4. The PR exists, titled correctly, and the JIRA links to it

If all four hold, you have produced the same artifact set as every commit in git log --oneline.

PR Profile: Level 8 Graduate

A Level 8 graduate's first merged Tez PR looks like the small, real ones already in history — go read them (git show <hash> --stat):

• Scope: one JIRA, one logical change, a handful of files. TEZ-4308 changed one file (+6/−5) and was merged. TEZ-4699 changed one class and added one test class.
• Test discipline: the fix and its test land together — TEZ-4699's CSVResult.java change shipped with a new TestCSVResult.java; TEZ-4569's VertexImpl fix shipped with a 509-line recovery test.
• Process fluency: JIRA filed/claimed, PR titled TEZ-XXXX: ..., Yetus report read and acted on, review comments answered with commits, committer squash-merges with the (Author reviewed by Reviewer) credit.

That profile — small, proven, process-clean — is what earns you the benefit of the doubt on your second, larger PR.

Next: Level 9 takes you into harder, release-relevant work — scheduler test coverage and performance regressions. The full graded version of this level's loop is the Capstone.