State Machine & Event Map

Tez's application master is a set of event-driven finite state machines. Every control-plane entity — the DAG, each vertex, each task, each attempt, each container, each node — is a Hadoop StateMachineFactory machine. Nothing calls a method on another entity directly; instead it puts an event on the central dispatcher, which delivers it to the target entity's handler, which runs a transition. This file is the reference for those states and events: the real enums, and a routing table of who sends what to whom.

Read ../deep-dives/state-machines.md for the mechanism and ../deep-dives/event-routing.md for the dispatch. Every enum below was extracted from source; confirm with:

# A state enum:
grep -A15 "enum VertexState" tez-dag/src/main/java/org/apache/tez/dag/app/dag/VertexState.java
# An event-type enum:
grep -A20 "enum TaskAttemptEventType" \
  tez-dag/src/main/java/org/apache/tez/dag/app/dag/event/TaskAttemptEventType.java
# The dispatcher registrations that build the routing table:
grep -n "dispatcher.register" tez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java

The state enums

DAGState — ...dag.app.dag.DAGState

The internal state of one DAG (DAGImpl).

VertexState — ...dag.app.dag.VertexState

The internal state of one vertex (VertexImpl). Note the extra INITIALIZING state — a vertex can initialize asynchronously (input initializers, parallelism decisions) before it is INITED.

TaskStateInternal — ...dag.app.dag.TaskStateInternal

The internal state of one task (TaskImpl). Small, because a task is mostly a holder for attempts.

TaskAttemptStateInternal — ...dag.app.dag.TaskAttemptStateInternal

The internal state of one attempt (TaskAttemptImpl). The *_IN_PROGRESS states are the ones you will see in logs when an attempt is being torn down.

AMContainerState / AMNodeState / DAGAppMasterState

Public status enums (client-facing)

The internal states above are @Private. Clients see coarser enums over the wire. Do not confuse them with the internal ones.

Note: There are two VertexState enums. ...dag.app.dag.VertexState is the AM's internal state machine; ...dag.api.event.VertexState is the public notification enum a vertex manager gets via onVertexStateUpdated. Grep the package before assuming which one a symbol refers to.

The event-type enums

Each state machine is registered against exactly one event-type enum in DAGAppMaster.serviceInit (see the dispatcher.register(...) calls). These are the "control plane" events — distinct from the runtime org.apache.tez.runtime.api.events.* data-plane events.

DAGEventType — drives DAGImpl

VertexEventType — drives VertexImpl

TaskEventType — drives TaskImpl

TaskAttemptEventType — drives TaskAttemptImpl

Scheduler, launcher, container, node, and misc event types

HistoryEventType — the logged milestone stream

Not a state-machine driver, but the enum every history backend records. From ...dag.history.HistoryEventType (each value also carries a HistoryLogLevel):

APP_LAUNCHED, AM_LAUNCHED, AM_STARTED, DAG_SUBMITTED, DAG_INITIALIZED, DAG_STARTED, DAG_FINISHED, DAG_KILL_REQUEST, VERTEX_INITIALIZED, VERTEX_STARTED, VERTEX_CONFIGURE_DONE, VERTEX_FINISHED, TASK_STARTED, TASK_FINISHED, TASK_ATTEMPT_STARTED, TASK_ATTEMPT_FINISHED, CONTAINER_LAUNCHED, CONTAINER_STOPPED, DAG_COMMIT_STARTED, VERTEX_COMMIT_STARTED, VERTEX_GROUP_COMMIT_STARTED, VERTEX_GROUP_COMMIT_FINISHED, DAG_RECOVERED.

Dispatcher registrations (the backbone)

DAGAppMaster registers each event-type enum against one handler. This is the authoritative wiring — everything in the routing table below flows through it:

Note: TaskEventType and TaskAttemptEventType can share a concurrent dispatcher when tez.am.use.concurrent-dispatcher=true — the registration differs but the target *Impl is the same. Grep registerAndCreateDispatcher in DAGAppMaster to see it.

Who sends what to whom

The ~25 most important events on the happy path (schedule → launch → run → complete) plus the key failure edges. "Producer" is the entity that raises the event; "consumer" is the *Impl (via its registered dispatcher) that handles it. This is the flow you trace when debugging "why did nothing happen after X".

Warning: The producer/consumer pairs above describe the standard flow; a few events (e.g. TA_STATUS_UPDATE, TA_DONE, TA_FAILED) originate from the task over the umbilical and are injected by the TaskCommunicatorManager on the AM's behalf, not raised by an AM state machine. When tracing, confirm the actual producer with a grep for the event class's constructor call sites:

grep -rn "new TaskAttemptEventStatusUpdate" tez-dag/src/main/java/
grep -rn "new AMSchedulerEventTALaunchRequest" tez-dag/src/main/java/