Testing Framework

Tez has no product surface you can click. It is a library that turns a DAG into a running set of JVMs on YARN, and almost everything interesting happens inside asynchronous state machines that trade events. That shape dictates how Tez is tested: the fast tests drive those state machines by hand with a deterministic dispatcher and mocked collaborators, and the slow tests boot a real in-process YARN + HDFS cluster and run whole DAGs. If you cannot tell which tier a change belongs in, or you reach for Thread.sleep in a state-machine test, your patch stalls in review.

After this chapter you can:

• Write a tez-dag state-machine unit test the way TestVertexImpl and TestTaskImpl do — construct a real *Impl with a mocked AppContext, feed it events through a DrainDispatcher, call await(), and assert state.
• Read a MiniTezCluster integration test in tez-tests and know exactly what it boots (a MiniYARNCluster, usually a MiniDFSCluster, and a TezClient).
• Inject deterministic task/input failures with TestProcessor / TestInput and the tez.failing-* config keys, the way TestFaultTolerance does.
• Run a single module, class, or method with the real Maven invocations, and know what the surefire fork/timeout/argLine config gives you.
• Explain the two CI paths — the GitHub Actions build matrix and the Jenkins + Apache Yetus precommit — and what each actually runs on a PR.

Everything below is anchored to the checkout at /Users/s0x/src/oss-repos/tez (master, commit 330fdc8f1). Code moves between branches, so each section opens with a runnable rg/find command — run it yourself and read the class, never trust a line number.

The three tiers

find /Users/s0x/src/oss-repos/tez -name MiniTezCluster.java -not -path '*/target/*'

That returns exactly one file: tez-tests/src/test/java/org/apache/tez/test/MiniTezCluster.java. There is no separate failsafe/integration split — the root pom.xml configures only maven-surefire-plugin, so MiniTezCluster tests run under surefire alongside the pure unit tests. That is why mvn test at the reactor root is slow: it boots mini clusters inline.

Unit testing state machines (tez-dag)

The dominant pattern is arrange state, send event, drain dispatcher, assert. The canonical examples all live in one package:

find /Users/s0x/src/oss-repos/tez/tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl \
     -name 'Test*.java' | sort

You will see TestVertexImpl.java (~10k lines, 102 @Test methods), TestTaskImpl.java (37 tests), TestTaskAttempt.java (27 tests), TestDAGImpl.java (43 tests), plus TestEdge, TestVertexManager and others. These are the highest-value files to read before touching the AM.

The dispatcher: Tez's own DrainDispatcher

Tez does not use Hadoop's DrainDispatcher. It ships its own under test sources so it can be reused by every module:

find /Users/s0x/src/oss-repos/tez -name DrainDispatcher.java -not -path '*/target/*'
rg -n 'class DrainDispatcher|void await' \
   /Users/s0x/src/oss-repos/tez/tez-common/src/test/java/org/apache/tez/common/DrainDispatcher.java

From tez-common, org.apache.tez.common.DrainDispatcher — it extends the real AsyncDispatcher and adds a blocking await():

public class DrainDispatcher extends AsyncDispatcher {
  // ...
  /** Busy loop waiting for all queued events to drain. */
  public void await() {
    while (!drained) {
      Thread.yield();
    }
  }

drained is flipped to queue.isEmpty() inside the dispatch thread under a mutex. This is the single most important primitive in the whole test suite: you post events, call await(), and you are guaranteed every handler has run and every event those handlers posted has also run before the assertion. No timing, no sleeps.

The harness setup: TestVertexImpl

rg -n 'dispatcher = new DrainDispatcher|appContext = mock\(AppContext|dispatcher.register|dispatcher.start' \
   /Users/s0x/src/oss-repos/tez/tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestVertexImpl.java

The setup lives in setupPostDagCreation(...) (called from a @Before method). Trimmed to the load-bearing lines, from tez-dag / org.apache.tez.dag.app.dag.impl.TestVertexImpl:

dispatcher = new DrainDispatcher();
appContext = mock(AppContext.class);
when(appContext.getHadoopShim()).thenReturn(new DefaultHadoopShim());
// ...
thh = mock(TaskHeartbeatHandler.class);
historyEventHandler = mock(HistoryEventHandler.class);
TaskSchedulerManager taskScheduler = mock(TaskSchedulerManager.class);
DAG dag = mock(DAG.class);
// ...
doReturn(dispatcher.getEventHandler()).when(appContext).getEventHandler();
doReturn(clock).when(appContext).getClock();
doReturn(dag).when(appContext).getCurrentDAG();
// ...
dispatcher.register(CallableEventType.class, new CallableEventDispatcher());
taskAttemptEventDispatcher = new TaskAttemptEventDispatcher();
dispatcher.register(TaskAttemptEventType.class, taskAttemptEventDispatcher);
taskEventDispatcher = new TaskEventDispatcher();
dispatcher.register(TaskEventType.class, taskEventDispatcher);
vertexEventDispatcher = new VertexEventDispatcher();
dispatcher.register(VertexEventType.class, vertexEventDispatcher);
dagEventDispatcher = new DagEventDispatcher();
dispatcher.register(DAGEventType.class, dagEventDispatcher);
// ...
dispatcher.init(conf);
dispatcher.start();

Read that carefully — it is the whole idiom:

1. AppContext is mocked, aggressively. It is the AM's god-object (current DAG, clock, event handler, task scheduler, history handler, containers). Mocking it lets each test wire up exactly the collaborators it cares about and leave the rest as no-op mocks. Note clock is a MockClock (also in tez-dag test sources) so time-dependent transitions are driven explicitly, not by wall-clock.
2. The dispatcher's event handler is fed back into AppContext via doReturn(dispatcher.getEventHandler()).when(appContext).getEventHandler(), so when the code under test emits an event, it lands in the drainable queue.
3. Each event type is routed to a small in-test dispatcher class (VertexEventDispatcher, TaskEventDispatcher, …). These are inner classes that either forward to the real *Impl under test or capture the event for assertion. TestVertexImpl even routes CallableEventType through a mocked ListeningExecutorService whose submit synchronously re-dispatches the callable, so async initializer work stays deterministic.

The event-capture idiom

TestTaskImpl shows the minimal capture handler. From tez-dag / org.apache.tez.dag.app.dag.impl.TestTaskImpl:

class TestEventHandler implements EventHandler<Event> {
  List<Event> events = new ArrayList<Event>();
  @Override
  public void handle(Event event) {
    events.add(event);
  }
}

You install that as the event handler, drive the state machine, and then assert over eventHandler.events — count them, filter by type, inspect payloads. This is the assertion surface for "did this transition emit the right outgoing event," which is most of what the DAG/vertex/task machines do. TestTaskAttempt uses a variant, MockEventHandler, that watches for a DAGEventType.INTERNAL_ERROR so an illegal transition trips the test instead of being silently swallowed.

The factory-override idiom

State-machine *Impl classes create their children through protected factory methods, precisely so tests can override them and inject test doubles. TestTaskImpl overrides TaskImpl.createAttempt to return a MockTaskAttemptImpl instead of a real one:

rg -n 'class MockTaskImpl extends TaskImpl|protected TaskAttemptImpl createAttempt' \
   /Users/s0x/src/oss-repos/tez/tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestTaskImpl.java

From tez-dag / TestTaskImpl:

private class MockTaskImpl extends TaskImpl {
  // ...
  @Override
  protected TaskAttemptImpl createAttempt(int attemptNumber, TezTaskAttemptID schedCausalTA) {
    MockTaskAttemptImpl attempt = new MockTaskAttemptImpl(/* ... */);
    taskAttempts.add(attempt);
    return attempt;
  }
  @Override
  protected void internalError(TaskEventType type) {
    super.internalError(type);
    fail("Internal error: " + type);   // turn a swallowed bad transition into a test failure
  }

TestVertexImpl uses the same trick at a higher level: VertexImplWithControlledInitializerManager and VertexImplWithRunningInputInitializer both extend VertexImpl to control the root-input initializer so the test can step the vertex's INITIALIZING phase by hand. When you add a new transition, extend the real *Impl, override its factory method, and let the base class run — you are testing production transition logic, not a re-implementation of it.

The four-step recipe

@Test(timeout = 5000)
public void testVertexInit() throws Exception {
  // 1. ARRANGE: build the real VertexImpl (via setupPostDagCreation), mocks in place
  initVertex(v);

  // 2. ACT: post an event through the drainable dispatcher
  dispatcher.getEventHandler().handle(
      new VertexEventTaskAttemptCompleted(taId, TaskAttemptStateInternal.SUCCEEDED));

  // 3. DRAIN: block until the queue (and anything it spawns) is empty
  dispatcher.await();

  // 4. ASSERT: state + emitted events
  assertEquals(VertexState.RUNNING, v.getState());
}

The exact method above is illustrative; run rg -n 'public void testVertexInit\b' .../TestVertexImpl.java to read the real one. The four steps never change.

Warning: Never Thread.sleep to wait for a transition, and never spin on while (v.getState() != X). Both are flake generators — transition time depends on machine load. The correct wait is always dispatcher.await(), which is deterministic by construction.

flowchart LR
  A[Test thread] -->|handle event| Q[DrainDispatcher queue]
  Q --> D[Dispatch thread]
  D -->|route by type| R[Registered EventDispatcher]
  R --> I[Real VertexImpl / TaskImpl transition]
  I -->|emits events| Q
  I -->|captures| C[TestEventHandler.events]
  A -->|await blocks until| E{drained == queue.isEmpty}
  E -->|true| A2[Test thread asserts state + events]

Cross-reference state-machines.md for the transition tables these tests exercise, and task-attempt-lifecycle.md for what the events mean.

MiniTezCluster integration tests (tez-tests)

rg -n 'class MiniTezCluster' \
   /Users/s0x/src/oss-repos/tez/tez-tests/src/test/java/org/apache/tez/test/MiniTezCluster.java

The declaration answers the most important question directly. From tez-tests / org.apache.tez.test.MiniTezCluster:

public class MiniTezCluster extends MiniYARNCluster {

  public static final String APPJAR = JarFinder.getJar(DAGAppMaster.class);

  public MiniTezCluster(String testName, int noOfNMs) { this(testName, noOfNMs, 4, 4); }

  public MiniTezCluster(String testName, int noOfNMs, int numLocalDirs, int numLogDirs) {
    super(testName, noOfNMs, numLocalDirs, numLogDirs);
  }

  @Override
  public void serviceInit(Configuration conf) throws Exception {
    conf.set(MRConfig.FRAMEWORK_NAME, MRConfig.YARN_TEZ_FRAMEWORK_NAME);
    conf.setBoolean(TezConfiguration.TEZ_USE_CLUSTER_HADOOP_LIBS, true);
    conf.setBoolean(TezConfiguration.TEZ_AM_NODE_BLACKLISTING_ENABLED, false);
    // ...
    File appJarLocalFile = new File(MiniTezCluster.APPJAR);
    if (!appJarLocalFile.exists()) {
      throw new TezUncheckedException("TezAppJar " + MiniTezCluster.APPJAR + " not found. Exiting.");
    }
    // copy the AM jar into the mini cluster's DFS and point TEZ_LIB_URIS at it
    conf.set(TezConfiguration.TEZ_LIB_URIS, appRemoteJar.toUri().toString());

So MiniTezCluster is a MiniYARNCluster (in-process ResourceManager + N NodeManagers) with Tez-specific serviceInit: it locates the AM jar via JarFinder, disables node blacklisting (a single-node mini cluster can't afford to blacklist), forces the cluster Hadoop libs, and stages the AM jar so real containers can launch. Its init(...) also calls TezTestUtils.ensureHighDiskUtilizationLimit(conf) so the disk-health checker doesn't mark a nearly-full test box's NMs unhealthy. If you see "TezAppJar ... not found", you ran the test without building the assembly first — that's a build-order problem, not a test bug.

HDFS: MiniDFSCluster

Most MiniTezCluster tests also stand up a real in-process HDFS so paths, splits, and staging behave like production:

rg -l 'MiniDFSCluster' /Users/s0x/src/oss-repos/tez/tez-tests/src/test/java

That lists TestFaultTolerance, TestTezJobs, TestMRRJobsDAGApi, TestSecureShuffle, TestDAGRecovery, and more. The setup is uniform. From tez-tests / org.apache.tez.test.TestFaultTolerance, @BeforeClass:

conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, TEST_ROOT_DIR);
dfsCluster = new MiniDFSCluster.Builder(conf).numDataNodes(1)
    .format(true).racks(null).build();
remoteFs = dfsCluster.getFileSystem();
// ...
miniTezCluster = new MiniTezCluster(TestFaultTolerance.class.getName(), 3, 1, 1);
Configuration miniTezconf = new Configuration(conf);
miniTezconf.set("fs.defaultFS", remoteFs.getUri().toString()); // use HDFS
miniTezCluster.init(miniTezconf);
miniTezCluster.start();
// ...
tezSession = TezClient.create("TestFaultTolerance", tezConf, true);
tezSession.start();

TestTezJobs is the same shape with numDataNodes(2). The rest of that class runs real example jobs against the cluster — for instance testOrderedWordCount (@Test(timeout = 60000)) generates input on the mini DFS and invokes the actual OrderedWordCount example from tez-examples:

OrderedWordCount job = new OrderedWordCount();
Assert.assertTrue("OrderedWordCount failed",
    job.run(tezConf, new String[]{"-counter", inputDirStr, outputDirStr, "2"}, null) == 0);
verifyOutput(outputDir, remoteFs);

TestTezJobs also drives HashJoinExample and SimpleSessionExample, plus an OrderedWordCount variant that disables split grouping — it is the end-to-end proof that the example jobs and the client submission path actually work.

Note: There is a TestOrderedWordCount.java in the tree, but it lives under tez-tests/src/main/java/org/apache/tez/mapreduce/examples/ — it is an MR-compatibility example driver, not a JUnit test. The JUnit coverage of ordered word count is TestTezJobs#testOrderedWordCount. Grep before you cite; the name is a trap.

Lifecycle

flowchart TD
  BC[BeforeClass] --> DFS[MiniDFSCluster.Builder.numDataNodes.build]
  DFS --> MTC[new MiniTezCluster ; init ; start  = MiniYARNCluster + AM jar staged]
  MTC --> TC[TezClient.create session ; start]
  TC --> T1[Test: build DAG]
  T1 --> SUB[tezClient.submitDAG]
  SUB --> WAIT[DAGClient.waitForCompletion]
  WAIT --> ASSERT[assert DAGStatus.State + counters]
  AC[AfterClass] --> STOP[tezSession.stop ; miniTezCluster.stop ; dfsCluster.shutdown]

Every MiniTezCluster test carries a JUnit timeout — @Test(timeout=60000) is the common value, up to 600000 for heavy recovery tests. A mini-cluster test that hangs without a timeout blocks the entire surefire fork until the 900-second forkedProcessTimeoutInSeconds kills the whole JVM (see below), so the per-test timeout is not optional. See dag-app-master.md for what the AM these tests boot is actually doing.

Deterministic failure injection

tez-tests ships purpose-built IPOs whose whole reason to exist is to fail on command: TestProcessor, TestInput, TestOutput, all in org.apache.tez.test.

rg -n 'TEZ_FAILING_PROCESSOR_DO_FAIL|TEZ_FAILING_PROCESSOR_FAILING_TASK_INDEX|getVertexConfName' \
   /Users/s0x/src/oss-repos/tez/tez-tests/src/test/java/org/apache/tez/test/TestProcessor.java

The failure surface is a set of config keys, keyed per vertex (and optionally per task index) so one DAG can fail one vertex's task 0 attempt 0 while leaving everything else healthy. From tez-tests / TestProcessor:

TestInput mirrors this with TEZ_FAILING_INPUT_DO_FAIL (tez.failing-input.do-fail) and TEZ_FAILING_INPUT_DO_FAIL_AND_EXIT. Keys are namespaced onto a vertex with the helpers:

public static String getVertexConfName(String confName, String vertexName) {
  return confName + "." + vertexName;
}
public static String getVertexConfName(String confName, String vertexName, int taskIndex) {
  return confName + "." + vertexName + "." + String.valueOf(taskIndex);
}

TestFaultTolerance composes these into scenarios over SimpleTestDAG (a two-vertex v1 -> v2 DAG built by SimpleTestDAG.createDAG). From tez-tests / TestFaultTolerance#testBasicTaskFailure:

Configuration testConf = new Configuration(false);
testConf.setBoolean(TestProcessor.getVertexConfName(
    TestProcessor.TEZ_FAILING_PROCESSOR_DO_FAIL, "v1"), true);
testConf.set(TestProcessor.getVertexConfName(
    TestProcessor.TEZ_FAILING_PROCESSOR_FAILING_TASK_INDEX, "v1"), "0");
testConf.setInt(TestProcessor.getVertexConfName(
    TestProcessor.TEZ_FAILING_PROCESSOR_FAILING_UPTO_TASK_ATTEMPT, "v1"), 0);
// v2 verifies the arithmetic that only holds if the retry actually reran v1 task0
testConf.setInt(TestProcessor.getVertexConfName(
    TestProcessor.TEZ_FAILING_PROCESSOR_VERIFY_VALUE, "v2", 1), 4);
DAG dag = SimpleTestDAG.createDAG("testBasicTaskFailure", testConf);
runDAGAndVerify(dag, DAGStatus.State.SUCCEEDED, 1);

runDAGAndVerify(dag, expectedState, checkFailedAttempts) submits the DAG to the session, waits, asserts the final state, and asserts the number of failed attempts. The processors also carry VERIFY_VALUE assertions so the test proves the data is correct after a retry, not merely that the DAG went green. This is the template for any fault-tolerance change: encode the failure in config, encode the expected recovery arithmetic in VERIFY_VALUE, run it on the mini cluster. See failure-handling.md for the recovery semantics under test here.

Mocking conventions

Root pom.xml pins the versions — verify before you assume:

rg -n 'junit.version|mockito-core.version' /Users/s0x/src/oss-repos/tez/pom.xml

junit.version is 4.13.2 (JUnit 4 — org.junit.Test, @BeforeClass, @Before; there is no JUnit 5 usage in the test tree) and mockito-core.version is 4.8.1. Mockito is used pervasively: roughly 800 mock( and 760 verify( call sites across tez-dag test sources alone.

Idioms you will copy:

• mock(Foo.class) + when(...).thenReturn(...) / doReturn(...).when(...) for stubbing collaborators. TestTaskImpl builds its AppContext with mock(AppContext.class, RETURNS_DEEP_STUBS) so deep chains like appContext.getAllContainers().get(id) stub in one line.
• spy(new Real(...)) when you need mostly-real behavior with one method intercepted — TestVertexImpl spies a real ContainerLauncherManager and uses doCallRealMethod() to keep vertexComplete real while stubbing the rest. Grep it: rg -n 'spy\(' tez-dag/src/test/java.
• verify(mock, never()).method(...) to prove a transition did not emit something (e.g. no task was allocated in a given state).
• Factory-method overrides in *Impl subclasses (the MockTaskImpl / VertexImplWithControlledInitializerManager pattern above) rather than mocking the state machine itself — you test production transition code.

There is also a heavier fake, MockDAGAppMaster (with MockContainerLauncher), under tez-dag/src/test/java/org/apache/tez/dag/app/. It boots a near-complete AM with a fake container launcher so tests like TestMockDAGAppMaster and TestMemoryWithEvents can run a DAG through the real AM event flow without YARN. Reach for it when a mocked AppContext is too shallow but a full mini cluster is too slow.

Tip: The surefire argLine sets -Dnet.bytebuddy.experimental=true and -XX:+EnableDynamicAgentLoading plus several --add-opens flags. Mockito 4.8.1's inline mock maker uses ByteBuddy + a dynamic agent; on JDK 21/25 those flags are what keep mock creation from warning or failing. If you run a test outside Maven (say, from an IDE) and see ByteBuddy agent errors, copy those JVM args.

How to run tests

All test execution is maven-surefire-plugin. Inspect its real config:

rg -n -A20 'maven-surefire-plugin' /Users/s0x/src/oss-repos/tez/pom.xml | head -30

From root pom.xml, the surefire <configuration>:

<forkCount>1</forkCount>
<reuseForks>false</reuseForks>
<forkedProcessTimeoutInSeconds>900</forkedProcessTimeoutInSeconds>
<testFailureIgnore>true</testFailureIgnore>
<argLine>
  @{argLine}
  -XX:+HeapDumpOnOutOfMemoryError
  -XX:+EnableDynamicAgentLoading
  --add-opens=java.base/java.lang=ALL-UNNAMED
  --add-opens=java.base/java.util=ALL-UNNAMED
  --add-opens=java.base/java.io=ALL-UNNAMED
  -Dnet.bytebuddy.experimental=true
</argLine>

Read what this buys you: forkCount=1 + reuseForks=false means a fresh JVM per test class — deliberate isolation so static state can't leak between classes (a real hazard given how much AM state is process-global). The 900-second fork timeout is the backstop for a hung mini-cluster test. @{argLine} is a late-bound placeholder: the top of pom.xml defines an empty <argLine/> property so the build doesn't crash when the jacoco profile is inactive, and the JaCoCo agent prepends coverage args into it when that profile is on. testFailureIgnore=true means mvn test does not stop the reactor on the first failing module — check the surefire reports, not just the exit code.

Common invocations:

# one module
mvn test -pl tez-dag

# one class (add -am to also build upstream modules it depends on)
mvn test -pl tez-dag -Dtest=TestVertexImpl -am

# one method (verified real methods)
mvn test -pl tez-dag -Dtest='TestVertexImpl#testVertexInit'
mvn test -pl tez-tests -Dtest='TestFaultTolerance#testBasicTaskFailure' -am

# don't fail when a name filter matches nothing in a module
mvn test -pl tez-dag -Dtest=TestVertexImpl -DfailIfNoTests=false

Before a tez-tests run you generally need the assembly present so MiniTezCluster.APPJAR resolves — build once with mvn clean install -DskipTests (the same command CI uses, below), then run the targeted test.

Test-timeout convention is per-@Test: @Test(timeout = 60000) for mini-cluster tests, small values (or none) for unit tests. There are no TestNG groups or category profiles; the only profile that changes test behavior is jacoco (coverage).

CI: two paths

cat /Users/s0x/src/oss-repos/tez/.github/workflows/build.yml
rg -n 'YETUS|test-patch|DOCKERFILE|BUILDTOOL|add_test' \
   /Users/s0x/src/oss-repos/tez/Jenkinsfile \
   /Users/s0x/src/oss-repos/tez/dev-support/tez-personality.sh

GitHub Actions (.github/workflows/build.yml) runs on every push and PR to master. It is a build matrix, not a test run: Java [21, 25] × OS [ubuntu-latest, macos-latest], and the single step is:

run: >
  mvn --batch-mode --no-transfer-progress clean install
  -DskipTests -Dmaven.javadoc.skip=true

So Actions proves the code compiles and assembles across JDKs and OSes. -DskipTests means it does not run the suite — that is the Jenkins job's job.

Jenkins + Apache Yetus (Jenkinsfile) is the precommit that actually runs tests. It clones Yetus, then runs Yetus' precommit/src/main/shell/test-patch.sh against the PR, inside Docker using build-tools/docker/Dockerfile (base image eclipse-temurin:21-jdk-noble). The behavior is configured by dev-support/tez-personality.sh:

export PROJECT_NAME=tez
export BUILDTOOL=maven
export JIRA_ISSUE_RE='^TEZ-[0-9]+$'
export GITHUB_REPO="apache/tez"
export MAVEN_OPTS="${MAVEN_OPTS:-"-Xmx4g -XX:+UseG1GC"}"
export DOCKER_MEMORY="20g"
# ... add_test unit

Yetus runs the standard precommit plugin set on the changed modules only: compile, checkstyle, mvn install, unit tests (add_test unit), spotbugs, whitespace/license (ASF headers), etc., and posts a pass/fail table. Practical consequences for a contributor:

• Reference the JIRA as TEZ-1234 in the PR — the personality's regex (^TEZ-[0-9]+$) is how Yetus links the run to the issue.
• Keep ASF license headers correct; the most recent commit on this checkout is literally TEZ-4711: Normalize ASF license header, and Yetus fails on header violations.
• Yetus builds and tests only the modules your patch touches (plus dependents), which is why a change under tez-dag won't get tez-tests coverage unless you also touch it — run the mini-cluster tests locally when your change could affect end-to-end behavior.

Local mode for tests

Faster than MiniTezCluster: no YARN, no DFS, the AM and tasks run as threads in one JVM.

rg -n 'TEZ_LOCAL_MODE|TEZ_LOCAL_MODE_WITHOUT_NETWORK' \
   /Users/s0x/src/oss-repos/tez/tez-tests/src/test/java/org/apache/tez/test/TestLocalMode.java

TestLocalMode and TestTaskErrorsUsingLocalMode flip TezConfiguration.TEZ_LOCAL_MODE (and optionally TEZ_LOCAL_MODE_WITHOUT_NETWORK) to true and submit the same DAGs, exercising runtime IPO logic without cluster overhead. Use local mode when YARN scheduling and HDFS are irrelevant to what you're testing; use MiniTezCluster when container allocation, shuffle over the network, or recovery matter. See the local-mode.md deep dive for the execution model.

Reading exercise

Run each; read what it returns.

cd /Users/s0x/src/oss-repos/tez

# 1. The unit-test harness: read the whole setup + first real test.
sed -n '1,60p' tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestTaskImpl.java

# 2. Prove the drain idiom is universal, not incidental.
rg -c 'dispatcher.await\(\)' tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestVertexImpl.java

# 3. The factory-override pattern that lets tests use real transition code.
rg -n 'extends VertexImpl|extends TaskImpl|protected .*create' \
   tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestVertexImpl.java \
   tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestTaskImpl.java

# 4. A real mini-cluster test end to end.
sed -n '74,160p' tez-tests/src/test/java/org/apache/tez/test/TestFaultTolerance.java

# 5. The failure-injection vocabulary.
rg -n 'TEZ_FAILING_' tez-tests/src/test/java/org/apache/tez/test/TestProcessor.java \
                     tez-tests/src/test/java/org/apache/tez/test/TestInput.java

# 6. Run one and read the report structure.
mvn test -pl tez-dag -Dtest='TestTaskImpl#testInit' -DfailIfNoTests=false

Common bugs and symptoms

Validation: prove you understand this

1. Write the four-step recipe for a tez-dag state-machine test and give the exact call that guarantees all queued (and spawned) events have run before your assertions.
2. MiniTezCluster extends which Hadoop class, and what three things does a typical tez-tests @BeforeClass boot before submitting a DAG?
3. Explain the factory-override idiom (MockTaskImpl.createAttempt, VertexImplWithControlledInitializerManager). Why is overriding a protected factory better than mocking the state machine outright?
4. Show the config you'd set to make v1's task 0 fail on its first attempt and succeed on retry, using the real TestProcessor keys and getVertexConfName. How would you prove the output is still correct?
5. Give the exact Maven command to run only TestFaultTolerance#testBasicTaskFailure, and explain why you must build the assembly first.
6. From the surefire config, state what forkCount=1 + reuseForks=false and the @{argLine} placeholder each accomplish.
7. Describe the two CI paths. Which one runs the tests, which one only builds, and what does the Yetus personality's JIRA_ISSUE_RE control?