The ClusterState Object

The ClusterState is the single most important data structure in OpenSearch. It is an immutable snapshot of everything the cluster has agreed upon: which indices exist and how they are mapped and configured (Metadata), where every shard copy lives (RoutingTable), who the nodes are (DiscoveryNodes), and what operations are currently blocked (ClusterBlocks). Every node holds a copy; the elected cluster manager produces new versions and publishes them to everyone (see cluster-state-publishing.md). If you understand this object, the rest of the cluster layer is bookkeeping around it.

After this chapter you should be able to: enumerate the top-level components of a cluster state; explain why immutability + the builder pattern + Diffable are load-bearing, not stylistic; read a live cluster state from a running node and map each JSON section back to a Java class; and find where any piece of metadata (a mapping, a setting, a shard assignment) lives in the object graph.

Note: Many accessor names still say master — e.g. DiscoveryNodes.getMasterNode(). Read "master" as "cluster manager" throughout; the role was renamed for inclusive language but the field names linger.

The top-level structure

find server/src/main/java/org/opensearch/cluster -name "ClusterState.java"
grep -n "private final\|public Metadata\|public RoutingTable\|public DiscoveryNodes\|\
public ClusterBlocks\|long version\|String stateUUID" \
  server/src/main/java/org/opensearch/cluster/ClusterState.java

A ClusterState is composed of:

flowchart TD
    CS[ClusterState] --> M[Metadata]
    CS --> RT[RoutingTable]
    CS --> DN[DiscoveryNodes]
    CS --> CB[ClusterBlocks]
    CS --> CU["customs: SnapshotsInProgress, ..."]
    M --> IM["IndexMetadata per index"]
    IM --> MAP[mappings]
    IM --> SET[settings]
    IM --> AL[aliases]
    M --> TPL[templates]
    M --> PCS[persistent cluster settings]
    RT --> IRT[IndexRoutingTable]
    IRT --> ISRT[IndexShardRoutingTable]
    ISRT --> SR["ShardRouting (primary/replica)"]

Metadata: the durable truth

Metadata holds everything that must survive a full cluster restart. Read its shape:

find server -name "Metadata.java" -path "*cluster/metadata*"
find server -name "IndexMetadata.java"
grep -n "ImmutableOpenMap\|indices\|templates\|persistentSettings\|customs\|indexGraveyard" \
  server/src/main/java/org/opensearch/cluster/metadata/Metadata.java

IndexMetadata is the per-index record. The fields you will touch most:

Note: number_of_shards is fixed at index creation and is part of the index's identity (it determines routing). number_of_replicas is dynamic. The asymmetry is encoded directly in IndexMetadata/IndexScopedSettings.

RoutingTable: where every shard copy lives

The RoutingTable is the plan for shard placement — the cluster manager's decision about which node holds each primary and replica. It is the output of the allocator (see shard-allocation.md).

find server -name "RoutingTable.java"
find server -name "IndexShardRoutingTable.java"
grep -n "primaryShard\|replicaShards\|allShards\|class IndexShardRoutingTable" \
  server/src/main/java/org/opensearch/cluster/routing/IndexShardRoutingTable.java

The hierarchy mirrors the data hierarchy from cluster-and-node-model.md:

RoutingTable
 └─ IndexRoutingTable (per index)
     └─ IndexShardRoutingTable (per shard id)
         ├─ ShardRouting (primary)
         └─ ShardRouting (replica…)

RoutingNodes is a derived, node-keyed view of the same data — "for node X, which shards are assigned here?" — that the allocator mutates in place during a single allocation round and then snapshots back into an immutable RoutingTable. That mutate-then-freeze pattern is the one exception to the immutability rule, and it is carefully contained.

grep -n "class RoutingNodes\|mutable\|unassigned()\|node(" \
  server/src/main/java/org/opensearch/cluster/routing/RoutingNodes.java

ClusterBlocks: what is forbidden right now

A ClusterBlocks is a set of ClusterBlocks that veto operations. Blocks are global or per-index, and each declares which operation levels it blocks (READ, WRITE, METADATA_READ, METADATA_WRITE).

find server -name "ClusterBlock.java" -o -name "ClusterBlocks.java"
grep -n "static final ClusterBlock\|NO_MASTER_BLOCK\|INDEX_READ_ONLY\|STATE_NOT_RECOVERED" \
  server/src/main/java/org/opensearch/cluster/block/ClusterBlocks.java \
  server/src/main/java/org/opensearch/cluster/coordination/NoClusterManagerBlockService.java 2>/dev/null

A request that hits a block fails fast with a ClusterBlockException. This is why "node has no cluster manager" surfaces to clients as a 503-style block rather than a hang.

Immutability, the builder pattern, and versioning

ClusterState and every component are immutable. You never mutate a state; you build a new one from an old one:

ClusterState newState = ClusterState.builder(currentState)
    .metadata(Metadata.builder(currentState.metadata()).put(newIndexMetadata, true))
    .routingTable(updatedRoutingTable)
    .build();

grep -n "public static Builder builder\|public ClusterState build\|incrementVersion" \
  server/src/main/java/org/opensearch/cluster/ClusterState.java

Why immutability is not optional here:

1. Safe sharing across threads. Dozens of components read the current state concurrently (the search layer, the indices layer, the REST layer). If state were mutable, every reader would need locking. Instead, each reader grabs a reference to an immutable snapshot and is guaranteed it will not change under them. See threadpools-concurrency.md.
2. Atomic transitions. A new state appears all-at-once. There is no window where, say, the routing table reflects a new index but the metadata does not.
3. Cheap diffing. Because old and new states are separate immutable objects that share most of their sub-structures, you can compute a compact Diff.

The version field is a monotonically increasing counter (per leader term). Combined with the coordination term, it gives a total order: a state with a higher (term, version) supersedes a lower one. Nodes reject older states.

Diff and Diffable: efficient publishing

Publishing a full multi-megabyte cluster state to every node on every tiny change would be wasteful. Most components implement Diffable<T>, which can emit a Diff<T> — a compact description of what changed from a previous version.

find server libs -name "Diffable.java" -o -name "Diff.java" -o -name "Diffs.java"
grep -rn "implements Diffable\|Diff<.*> diff(" \
  server/src/main/java/org/opensearch/cluster/ | head

flowchart LR
    Old["ClusterState v100"] --> D["Diff = changes only"]
    New["ClusterState v101"] --> D
    D -->|"small bytes"| Follower["follower applies Diff to its v100"]
    Follower --> Result["follower now at v101"]

The publishing layer sends a full state to nodes that are behind (or new), and a Diff to nodes that already hold the immediately previous version — a large bandwidth win on big clusters. The mechanism is detailed in cluster-state-publishing.md; the serialization contract (and its version gating) is in serialization-bwc.md.

Warning: Diffable correctness is subtle. If your component's diff and readDiffFrom disagree with its full read/write, followers will silently diverge from the leader. This is exactly the class of bug that the AbstractDiffableSerializationTestCase round-trip tests exist to catch. If you add a Custom to cluster state, you owe a diff test.

Customs: pluggable cluster state

Subsystems that need to track cluster-wide, transient or durable data attach a Custom to either the ClusterState (transient, e.g. SnapshotsInProgress, RestoreInProgress) or the Metadata (durable, e.g. persistent tasks, index-state-management metadata from a plugin).

grep -rn "implements ClusterState.Custom\|implements Metadata.Custom" \
  server/src/main/java/org/opensearch/ | head -20

This is the extension point plugins use to put their own state into the agreed cluster state. It is covered in plugin-architecture.md.

Reading the live state

The single most useful command for understanding this object:

# Full state (large). Scope it with filter_path.
curl -s "localhost:9200/_cluster/state?pretty" | less

# Just the routing table for one index.
curl -s "localhost:9200/_cluster/state/routing_table/orders?pretty"

# Just metadata of one index (settings + mappings).
curl -s "localhost:9200/_cluster/state/metadata/orders?pretty"

# Just the nodes and the elected cluster manager.
curl -s "localhost:9200/_cluster/state/master_node,nodes?pretty"

# Just the blocks.
curl -s "localhost:9200/_cluster/state/blocks?pretty"

# The version + UUID + term (proves monotonic versioning).
curl -s "localhost:9200/_cluster/state?filter_path=version,state_uuid,cluster_uuid,metadata.cluster_coordination"

Each top-level JSON key maps directly to a Java field:

Reading exercise

# 1. The class itself.
grep -n "private final\|Builder\|class ClusterState" \
  server/src/main/java/org/opensearch/cluster/ClusterState.java

# 2. How IndexMetadata is built and what it carries.
grep -n "Builder\|numberOfShards\|numberOfReplicas\|putMapping\|state(" \
  server/src/main/java/org/opensearch/cluster/metadata/IndexMetadata.java

# 3. Round-trip serialization tests (the BWC safety net).
./gradlew :server:test --tests "org.opensearch.cluster.ClusterStateTests"
./gradlew :server:test --tests "org.opensearch.cluster.metadata.MetadataTests"

# 4. Find every ClusterState.Custom.
grep -rn "implements ClusterState.Custom" server/src/main/java

Answer:

1. Name the five top-level components of a ClusterState and one thing each one holds.
2. number_of_shards cannot be changed after index creation, but number_of_replicas can. Where in IndexMetadata/IndexScopedSettings is that distinction enforced?
3. Explain why ClusterState is immutable in terms of concurrency. What would the search layer have to do differently if it were mutable?
4. What is the relationship between RoutingTable and RoutingNodes? Why does RoutingNodes get to be mutable when nothing else is?
5. Open the live blocks section after starting a node before any index exists. Which blocks are present during gateway recovery, and which class adds them?
6. You add a Custom to track per-cluster state for a plugin. Which two serialization paths must agree (full read/write vs diff), and which test base class verifies it?

Common bugs and symptoms

Validation: prove you understand this

1. Draw the ClusterState object graph from memory down to ShardRouting and IndexMetadata.mappings, labeling each edge with its Java type.
2. Explain the three concrete benefits of immutability (safe sharing, atomic transitions, cheap diffs) and give a bug that each one prevents.
3. Using a running node, fetch only the routing table for one index via filter_path and identify the primary's node and the replica's state.
4. Explain how (term, version) gives a total order over cluster states and why a node rejects a state with a lower pair.
5. Describe what Diffable buys the publishing layer on a 200-node cluster, and name the failure mode of a buggy diff().
6. List the four block levels (READ, WRITE, METADATA_READ, METADATA_WRITE) and give one real block for each level.