Key Classes by Module

This is the "where does X live" map. When you have a behavior and need the class that implements it — or a class name from a stack trace and need to know which module owns it — start here. Tez is a multi-module Maven build; the module boundary is the first thing to identify, because it tells you the layer (client API, AM control plane, or task-side data plane) and often the JIRA component.

The Maven modules, from the root pom.xml <modules> block:

Module	Layer	What it is
`tez-api`	Client + shared API	Public DAG-building API, config classes, runtime API interfaces, client
`tez-common`	Shared	ID classes, counters, dispatcher, utilities used everywhere
`tez-dag`	AM control plane	The ApplicationMaster: state machines, schedulers, launchers, communicators, history, recovery
`tez-runtime-library`	Task data plane	Inputs, outputs, sorters, shuffle, fetchers, partitioners, VM plugins
`tez-runtime-internals`	Task data plane	The container-side task runner and umbilical plumbing
`tez-mapreduce`	Compatibility	MR `InputFormat`/`OutputFormat` bridges, map/reduce processors
`tez-examples`	Examples	Runnable example DAGs (WordCount, joins, Cartesian product)
`hadoop-shim` / `hadoop-shim-impls`	Compatibility	Version-abstraction over Hadoop APIs
`tez-plugins`	Optional services	History-logging backends (ATS, Proto) and the shuffle aux-service
`tez-tools`	Tooling	Swimlanes, log-split, TFile parser, the config-doc doclet
`tez-tests`, `tez-ext-service-tests`	Tests	Integration tests / MiniTezCluster harness
`tez-ui`	UI	Ember.js history UI (not Java)
`tez-dist`, `tez-build-tools`, `docs`	Build	Assembly, checkstyle config, site docs

Warning: Class names get refactored across versions — a class may move packages, split, or be renamed. Treat every name here as a starting point and grep to confirm in your checkout:
# Confirm a class exists and find its module/package:
grep -rln "class TaskAttemptImpl" --include=*.java .
# Find an interface's implementations:
grep -rln "implements VertexManagerPlugin\|extends VertexManagerPlugin" --include=*.java .

Jump to: tez-api · tez-common · tez-dag: state machines · tez-dag: events · tez-dag: scheduling · tez-dag: launch & comms · tez-dag: recovery & history · tez-runtime-library: I/O · tez-runtime-library: sort · tez-runtime-library: shuffle · tez-runtime-library: VM plugins · tez-runtime-internals · tez-mapreduce · tez-examples · hadoop-shim · tez-plugins

tez-api

The public API a DAG author programs against, plus the config and runtime-API interfaces shared by both AM and tasks.

Class (`org.apache.tez.*`)	Role	Read
`client.TezClient`	Entry point: starts a session or submits a DAG, returns a `DAGClient`	`../deep-dives/tez-client.md`
`client.TezClientUtils`	Stages resources, builds the AM launch context, resolves `tez.lib.uris`	`../deep-dives/tez-client.md`
`dag.api.DAG`	The client-side DAG builder: add vertices, edges, vertex groups	`../deep-dives/dag-model.md`
`dag.api.Vertex`	A DAG stage: processor descriptor, parallelism, data sources/sinks	`../deep-dives/dag-model.md`
`dag.api.Edge` / `GroupInputEdge`	A connection between vertices / a group→vertex fan-in	`../deep-dives/logical-physical.md`
`dag.api.EdgeProperty`	Bundles data-movement, data-source, and scheduling types for an edge	`../deep-dives/logical-physical.md`
`dag.api.VertexGroup`	A named union of vertices for grouped input/commit	`../deep-dives/dag-model.md`
`dag.api.{Processor,Input,Output}Descriptor`	Serializable "class name + payload" that names a runtime component	`../deep-dives/ipo-abstractions.md`
`dag.api.VertexManagerPlugin` / `...PluginContext`	Base + context for runtime vertex control (slow-start, auto-parallelism)	`../deep-dives/scheduler.md`
`dag.api.TezConfiguration`	All AM/DAG/vertex config keys and defaults	config reference
`dag.api.client.DAGClient` / `DAGClientImpl`	Client handle to poll `DAGStatus` and wait for completion	`../deep-dives/dag-client.md`
`dag.api.client.DAGStatus` / `VertexStatus`	Snapshot of DAG/vertex state, progress, and counters	`../deep-dives/dag-client.md`
`runtime.api.LogicalIOProcessor` / `LogicalInput` / `LogicalOutput`	The IPO framework interfaces every task component implements	`../deep-dives/ipo-abstractions.md`
`runtime.api.OutputCommitter`	Commit/abort a data-sink output atomically	`../deep-dives/failure-handling.md`
`runtime.api.events.DataMovementEvent`	"Output partition N ready here" — the core routing event	`../deep-dives/event-routing.md`

tez-common

The shared substrate: identity, counters, event dispatch, and utilities that both the AM and the task side depend on.

Class (`org.apache.tez.*`)	Role	Read
`dag.records.TezDAGID` / `TezVertexID` / `TezTaskID` / `TezTaskAttemptID`	The hierarchical ID types that thread through every log line and event	`../deep-dives/dag-model.md`
`dag.records.TezID`	Base class for the ID hierarchy	`../deep-dives/dag-model.md`
`dag.records.TaskAttemptTerminationCause`	The enum classifying why an attempt ended (fetch failure, node fail, preempted, …)	`../deep-dives/failure-handling.md`
`common.counters.TezCounters`	The counter tree aggregated attempt→task→vertex→DAG	`../deep-dives/counters-diagnostics.md`
`common.counters.TaskCounter` / `DAGCounter`	Built-in counter enums (spilled records, shuffle bytes, etc.)	`../deep-dives/counters-diagnostics.md`
`common.AsyncDispatcher` / `AsyncDispatcherConcurrent`	The event bus that delivers each event type to its handler	`../deep-dives/event-routing.md`
`common.TezCommonUtils`	Staging-dir resolution, timeouts, path helpers	`../deep-dives/tez-client.md`
`common.TezUtils` / `ReflectionUtils`	Config↔payload conversion; instantiate descriptors by class name	`../deep-dives/ipo-abstractions.md`

Note: The ID classes live in tez-common under package org.apache.tez.dag.records even though "dag" is in the path — do not look for them in tez-dag. Grep: grep -rn "class TezTaskAttemptID" tez-common/.

tez-dag: state machines

The AM's control-plane entities. Each is a Hadoop StateMachineFactory machine — this is the state-machine trio+ you will read most often. See ../deep-dives/state-machines.md.

Class (`org.apache.tez.dag.app.*`)	Role	Read
`DAGAppMaster`	The AM god object: builds every service, owns the dispatcher, runs the DAG	`../deep-dives/dag-app-master.md`
`dag.impl.DAGImpl`	State machine for one DAG (`DAGState`)	`../deep-dives/state-machines.md`
`dag.impl.VertexImpl`	State machine for one vertex (`VertexState`); the largest class in the AM	`../deep-dives/vertex-lifecycle.md`
`dag.impl.TaskImpl`	State machine for one task (`TaskStateInternal`)	`../deep-dives/task-lifecycle.md`
`dag.impl.TaskAttemptImpl`	State machine for one attempt (`TaskAttemptStateInternal`)	`../deep-dives/task-attempt-lifecycle.md`
`dag.impl.Edge`	Runtime edge: holds the `EdgeManager` and routes events across it	`../deep-dives/logical-physical.md`
`dag.impl.VertexManager`	Wraps a `VertexManagerPlugin` and applies its parallelism/scheduling decisions	`../deep-dives/scheduler.md`
`dag.impl.{ScatterGather,Broadcast,OneToOne}EdgeManager`	Built-in routing implementations per data-movement type	`../deep-dives/logical-physical.md`
`DAGAppMasterState`	The AM's own lifecycle enum (`NEW`…`RECOVERING`…`RUNNING`…)	`../deep-dives/dag-app-master.md`
`AppContext`	The shared service registry passed to every AM component	`../deep-dives/dag-app-master.md`

tez-dag: events & dispatch

The internal event types and the handlers they wire to. See ../deep-dives/event-routing.md and the state-machine & event map.

Class (`org.apache.tez.dag.app.dag.event.*` unless noted)	Role	Read
`DAGEventType` / `DAGEvent` (+ subclasses)	Events driving `DAGImpl`	event map
`VertexEventType` / `VertexEvent` (+ subclasses)	Events driving `VertexImpl`	event map
`TaskEventType` / `TaskEvent` (+ subclasses)	Events driving `TaskImpl`	event map
`TaskAttemptEventType` / `TaskAttemptEvent` (+ subclasses)	Events driving `TaskAttemptImpl`	event map
`DAGAppMasterEventType` / `DAGAppMasterEvent`	AM-global events (reboot, DAG finished, service fatal error)	`../deep-dives/dag-app-master.md`
`SpeculatorEventType` / `SpeculatorEvent`	Feed the speculator attempt status updates	`../deep-dives/failure-handling.md`
`rm.AMSchedulerEventType` / `AMSchedulerEvent`	Task launch requests / ended / node-blacklist to the scheduler	`../deep-dives/scheduler.md`

tez-dag: scheduling & containers

Turning "this attempt needs to run" into a YARN container, and reusing containers. See ../deep-dives/scheduler.md and ../deep-dives/container-reuse.md.

Class (`org.apache.tez.dag.app.*`)	Role	Read
`rm.TaskSchedulerManager`	Dispatches `AMSchedulerEvent`s to the active task scheduler(s)	`../deep-dives/scheduler.md`
`rm.DagAwareYarnTaskScheduler`	Default YARN scheduler: DAG-priority-aware allocation and reuse	`../deep-dives/scheduler.md`
`rm.YarnTaskSchedulerService`	The older YARN scheduler implementation	`../deep-dives/scheduler.md`
`rm.LocalTaskSchedulerService`	Scheduler used in local mode (no RM)	`../deep-dives/local-mode.md`
`rm.container.AMContainerImpl`	Per-container state machine (`AMContainerState`) enabling reuse	`../deep-dives/container-reuse.md`
`rm.container.AMContainerMap`	Registry of all containers the AM holds	`../deep-dives/container-reuse.md`
`rm.node.AMNodeTracker` / `AMNodeImpl`	Per-node failure tracking and blacklisting	`../deep-dives/failure-handling.md`
`dag.impl.DAGSchedulerNaturalOrder`	Default cross-vertex scheduling order (topological)	`../deep-dives/scheduler.md`
`dag.impl.DAGSchedulerNaturalOrderControlled`	Throttled variant that gates downstream vertices	`../deep-dives/scheduler.md`

tez-dag: launch & communication

Launching container processes and the umbilical the tasks phone home on. See ../deep-dives/task-attempt-lifecycle.md.

Class (`org.apache.tez.dag.app.*`)	Role	Read
`launcher.ContainerLauncherManager`	Dispatches `ContainerLauncherEventType` to the active launcher	`../deep-dives/task-attempt-lifecycle.md`
`launcher.TezContainerLauncherImpl`	Default launcher: asks YARN NM to start container processes	`../deep-dives/yarn-integration.md`
`launcher.LocalContainerLauncher`	In-JVM launcher for local mode	`../deep-dives/local-mode.md`
`TaskCommunicatorManager`	Owns the umbilical RPC server; routes heartbeats to the state machines	`../deep-dives/task-attempt-lifecycle.md`
`TezTaskCommunicatorImpl`	Default umbilical implementation (`TezTaskUmbilicalProtocol` server)	`../deep-dives/task-attempt-lifecycle.md`
`TaskHeartbeatHandler` / `ContainerHeartbeatHandler`	Detect lost tasks/containers past their timeouts	`../deep-dives/failure-handling.md`
`serviceplugins.api.TaskCommunicator` / `ContainerLauncher`	The service-plugin base classes (pluggable comms/launch)	`../deep-dives/yarn-integration.md`

tez-dag: recovery & history

Surviving an AM crash, and emitting the lifecycle event stream. See ../deep-dives/failure-handling.md and ../deep-dives/counters-diagnostics.md.

Class (`org.apache.tez.dag.*`)	Role	Read
`app.RecoveryParser`	Replays the recovery log to rebuild DAG state on a new AM attempt	`../deep-dives/failure-handling.md`
`history.recovery.RecoveryService`	Writes the recovery event stream durably during a run	`../deep-dives/failure-handling.md`
`history.HistoryEventHandler`	Fans lifecycle events out to the configured logging service(s)	`../deep-dives/counters-diagnostics.md`
`history.HistoryEventType`	The enum of all loggable milestones	event map
`history.events.*` (e.g. `DAGSubmittedEvent`, `TaskAttemptFinishedEvent`)	The concrete history event records	`../deep-dives/counters-diagnostics.md`
`history.logging.impl.SimpleHistoryLoggingService`	Default file-based history backend	`../deep-dives/counters-diagnostics.md`
`history.logging.impl.DevNullHistoryLoggingService`	Discards history (disable logging)	`../deep-dives/counters-diagnostics.md`
`app.web.WebUIService`	The AM's embedded status web server	`../deep-dives/dag-app-master.md`

tez-runtime-library: inputs, outputs, processors

The concrete IPO components a DAG wires into vertices. See ../deep-dives/ipo-abstractions.md.

Class (`org.apache.tez.runtime.library.*`)	Role	Read
`input.OrderedGroupedKVInput`	Sorted-shuffle consumer input (the reduce-side input)	`../deep-dives/shuffle-sort.md`
`input.UnorderedKVInput`	Unsorted-shuffle consumer input	`../deep-dives/shuffle-sort.md`
`input.ConcatenatedMergedKeyValueInput`	Merges multiple physical inputs into one logical input	`../deep-dives/ipo-abstractions.md`
`output.OrderedPartitionedKVOutput`	Sort + partition producer output (the map-side output)	`../deep-dives/shuffle-sort.md`
`output.UnorderedKVOutput` / `UnorderedPartitionedKVOutput`	Unsorted producer outputs	`../deep-dives/shuffle-sort.md`
`processor.SimpleProcessor`	Base for a straightforward read-inputs/write-outputs processor	`../deep-dives/ipo-abstractions.md`
`partitioner.HashPartitioner` / `RoundRobinPartitioner`	Map keys to partitions for scatter-gather	`../deep-dives/shuffle-sort.md`

tez-runtime-library: sort & IFile

The producer-side sort/spill/merge machinery and the on-disk format. See ../deep-dives/shuffle-sort.md.

Class (`...runtime.library.common.sort.impl.*`)	Role	Read
`ExternalSorter`	Abstract base for the sorters (buffer, spill, merge)	`../deep-dives/shuffle-sort.md`
`PipelinedSorter`	Default sorter: block-based, multi-threaded, progressive spill	`../deep-dives/shuffle-sort.md`
`dflt.DefaultSorter`	The legacy quicksort-and-spill sorter	`../deep-dives/shuffle-sort.md`
`IFile`	Tez's compressed length-prefixed KV file format (spills, outputs, shuffle)	`../deep-dives/shuffle-sort.md`
`IFileInputStream` / `IFileOutputStream`	Checksummed stream wrappers around IFile data	`../deep-dives/shuffle-sort.md`
`TezMerger`	K-way merge of sorted IFile segments	`../deep-dives/shuffle-sort.md`
`TezSpillRecord` / `TezIndexRecord`	The per-partition offset index written alongside a sorted output	`../deep-dives/shuffle-sort.md`

tez-runtime-library: shuffle & fetch

The consumer-side transfer engines. Two parallel stacks: ordered (sorted) and unordered (unsorted). See ../deep-dives/shuffle-sort.md.

Class (`...runtime.library.common.shuffle.*`)	Role	Read
`orderedgrouped.Shuffle`	Orchestrates the ordered (sorted) shuffle for a consumer input	`../deep-dives/shuffle-sort.md`
`orderedgrouped.ShuffleScheduler`	Schedules ordered fetchers, tracks per-host progress, penalizes bad hosts	`../deep-dives/shuffle-sort.md`
`orderedgrouped.MergeManager`	In-memory + on-disk merge of fetched sorted map outputs	`../deep-dives/shuffle-sort.md`
`orderedgrouped.FetcherOrderedGrouped`	Fetches sorted output over HTTP / local disk	`../deep-dives/shuffle-sort.md`
`impl.ShuffleManager`	Orchestrates the unordered shuffle for a consumer input	`../deep-dives/shuffle-sort.md`
`Fetcher`	The unordered fetcher	`../deep-dives/shuffle-sort.md`
`impl.ShuffleInputEventHandlerImpl`	Turns incoming `DataMovementEvent`s into fetch work	`../deep-dives/event-routing.md`
`ShuffleUtils`	Shared helpers: event construction, connection setup, path resolution	`../deep-dives/shuffle-sort.md`

tez-runtime-library: vertex managers & edge managers

The runtime control plugins that live in the library, not the AM core. See ../deep-dives/scheduler.md.

Class (`org.apache.tez.dag.library.vertexmanager.*`)	Role	Read
`ShuffleVertexManager`	Slow-start + auto-parallelism for scatter-gather vertices	`../deep-dives/scheduler.md`
`ShuffleVertexManagerBase`	Shared base for shuffle-based vertex managers	`../deep-dives/scheduler.md`
`FairShuffleVertexManager` / `FairShuffleEdgeManager`	Auto-parallelism that balances partition sizes fairly	`../deep-dives/scheduler.md`
`InputReadyVertexManager`	Starts tasks as their inputs become ready (one-to-one/broadcast)	`../deep-dives/scheduler.md`
`RootInputVertexManager`	Manages parallelism for source (root-input) vertices	`../deep-dives/scheduler.md`
`VertexManagerWithConcurrentInput`	Handles vertices with concurrent input edges	`../deep-dives/scheduler.md`

tez-runtime-internals

The container-side runtime: what actually runs inside a task JVM. See ../deep-dives/tez-runtime.md.

Class (`org.apache.tez.runtime.*`)	Role	Read
`task.TezChild`	Container process entry point; heartbeats AM, runs assigned attempts	`../deep-dives/tez-runtime.md`
`task.TezTaskRunner2`	Runs one attempt: init inputs/outputs/processor, invoke `run`, report	`../deep-dives/tez-runtime.md`
`LogicalIOProcessorRuntimeTask`	The in-task object owning the processor and its I/O and memory	`../deep-dives/tez-runtime.md`
`task.TaskReporter`	Sends periodic heartbeats/counters and pulls events over the umbilical	`../deep-dives/task-attempt-lifecycle.md`
`common.TezTaskUmbilicalProtocol`	The RPC protocol tasks use to reach the AM	`../deep-dives/task-attempt-lifecycle.md`
`api.impl.TezUmbilical`	Task-side umbilical abstraction	`../deep-dives/task-attempt-lifecycle.md`
`InputReadyTracker`	Tracks which of a processor's inputs are ready to read	`../deep-dives/ipo-abstractions.md`

tez-mapreduce

The compatibility bridge so existing Hadoop MR formats and logic run under Tez. See ../deep-dives/tez-runtime.md.

Class (`org.apache.tez.mapreduce.*`)	Role	Read
`input.MRInput` / `MRInputLegacy`	Read via a Hadoop `InputFormat` inside a Tez task	`../deep-dives/tez-runtime.md`
`input.MultiMRInput`	Multiple MR inputs into one vertex	`../deep-dives/tez-runtime.md`
`output.MROutput` / `MROutputLegacy`	Write via a Hadoop `OutputFormat`	`../deep-dives/tez-runtime.md`
`common.MRInputAMSplitGenerator`	AM-side input initializer that groups MR splits	`../deep-dives/tez-runtime.md`
`committer.MROutputCommitter`	Adapts a Hadoop `OutputCommitter` to Tez's `OutputCommitter`	`../deep-dives/failure-handling.md`
`hadoop.DeprecatedKeys`	Maps old `mapred.*` keys to Tez runtime keys	config reference

tez-examples

Runnable, readable DAGs — the fastest way to see the API in action. See ../overview/index.md.

Class (`org.apache.tez.examples.*`)	Role
`WordCount`	The canonical two-vertex scatter-gather DAG
`OrderedWordCount`	Adds a sorting stage (three vertices)
`HashJoinExample` / `SortMergeJoinExample`	Broadcast (map) join vs sort-merge join DAGs
`CartesianProduct`	Custom-edge Cartesian product
`SimpleSessionExample`	Submitting multiple DAGs to one session
`TezExampleBase`	Shared client bootstrap for the examples

hadoop-shim

Isolates the Hadoop-version-specific API differences behind one interface. See ../contributor-mindset/compatibility.md.

Class (`org.apache.tez.hadoop.shim.*`)	Role
`HadoopShim`	The abstraction over version-specific Hadoop calls
`HadoopShimsLoader`	Selects the right shim at runtime via `HadoopShimProvider`
`DefaultHadoopShim`	The fallback implementation
`HadoopShim28` / `HadoopShim27` (in `hadoop-shim-impls`)	Version-specific implementations

tez-plugins

Optional services shipped as separate artifacts, selected by config. See ../deep-dives/counters-diagnostics.md.

Class / submodule	Role
`tez-aux-services` → `org.apache.tez.auxservices.ShuffleHandler`	NodeManager aux-service that serves shuffle output over HTTP
`tez-yarn-timeline-history` → `...logging.ats.ATSHistoryLoggingService`	Logs history to YARN ATS
`tez-protobuf-history-plugin` → `...logging.proto.ProtoHistoryLoggingService`	Logs history as protobuf files (Hive's preferred backend)
`tez-yarn-timeline-cache-plugin` → `TimelineCachePluginImpl`	ATS cache plugin backing the Tez UI
`tez-history-parser`	Offline parser turning ATS/proto history into analysis data

Note: History-logging backend is chosen by tez.history.logging.service.class; the default is SimpleHistoryLoggingService in tez-dag, and the ATS/Proto services above override it. See the config reference.

Open-Source Engineer & Contributor