Deep Dives: Reading Order
This directory contains 24 deep-dive chapters. They are the reference material behind the Level curriculum. Each chapter is self-contained, but most chapters depend on a handful of earlier ones. Read in the order below the first time through; thereafter use the index as a lookup.
OpenSearch is a distributed search and analytics engine built on Apache Lucene,
forked from Elasticsearch 7.10.2. Almost everything you read lives under
server/src/main/java/org/opensearch/..., with low-level primitives in libs/
and bundled extensions in modules/. The chapters below mirror the request path
and the cluster lifecycle: you build up "what a cluster is" before "how requests
flow" before "how a shard stores and searches data."
The chapters are grouped by subsystem. For each chapter we list:
- Title — the file.
- One-line summary — what you should walk away knowing.
- Consumed by — which Levels/Labs depend on it.
Note: Throughout the book, "cluster manager" is the current term for what Elasticsearch (and older OpenSearch) called "master." OpenSearch renamed the role and most settings for inclusive language; the old
masternames survive as deprecated aliases. Each chapter notes both terms on first use.
Group 1 — Cluster and Node Model
These four chapters define "what is an OpenSearch cluster" — the nodes, the shared state object they agree on, and how that agreement is reached and distributed — before any request-handling machinery exists.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 1 | cluster-and-node-model.md | Node bootstrap, node roles, DiscoveryNode, index→shard→segment hierarchy, how services are wired in the Node constructor | Level 1 (all labs); Level 3 lab 3.2 |
| 2 | discovery-coordination.md | Coordinator, the Zen2/Raft-like consensus, elections, voting configuration, split-brain prevention | Level 4 lab 4.1 |
| 3 | cluster-state.md | The ClusterState object: Metadata, RoutingTable, DiscoveryNodes, ClusterBlocks; immutability, versioning, Diffable | Level 4 lab 4.2; Level 4 lab 4.3 |
| 4 | cluster-state-publishing.md | Update-task model, MasterService vs ClusterApplierService, two-phase publish→commit, appliers vs listeners | Level 4 lab 4.2; Level 4 lab 4.3 |
Start here. Without the cluster/node/state model in your head, every later chapter feels like trivia.
Group 2 — Transport and Actions
How a request becomes work. These chapters explain the layered request path: HTTP at the edge, an internal RPC fabric between nodes, an action registry that ties request types to handlers, and the thread pools that run all of it.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 5 | transport-layer.md | TransportService, Netty4Transport, request handlers, Writeable/StreamInput/StreamOutput, NamedWriteableRegistry, port 9300 | Level 3 lab 3.1; Level 3 lab 3.2 |
| 6 | rest-layer.md | RestController dispatch, BaseRestHandler, prepareRequest, NodeClient, XContent parsing, error rendering | Level 3 lab 3.1; Level 3 lab 3.3 |
| 7 | action-framework.md | ActionType, TransportAction base classes, ActionModule registration, ActionFilters, ActionListener, write vs read path | Level 3 lab 3.3; Level 4 lab 4.4 |
| 8 | threadpools-concurrency.md | ThreadPool and its named pools, pool types, single-writer-per-shard, rejections, ThreadContext, why blocking the applier thread is fatal | Level 3 (all labs); Level 4 (all labs) |
These chapters explain how a request is dispatched and executed. They must precede the storage and search chapters, which assume you know the action and thread-pool model.
Group 3 — Indexing and Storage
The write path, end to end, inside a single shard: the shard lifecycle wrapper, the Lucene-backed engine, the write-ahead log, the schema/analysis layer, and the three operations (refresh, flush, merge) that govern visibility, durability, and segment count.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 9 | index-shard-lifecycle.md | IndicesService→IndexService→IndexShard; shard states; Store/Directory; primary vs replica op application | Level 1 lab 1.4; Level 5 |
| 10 | engine-internals.md | InternalEngine and friends; Lucene IndexWriter; versioning, LiveVersionMap, sequence numbers, Engine.Searcher | Level 5; Level 6 |
| 11 | translog.md | Translog, generations, durability modes, fsync, the translog↔commit relationship, crash recovery | Level 5; the recovery deep dive |
| 12 | mapping-and-analysis.md | MapperService, FieldMapper/MappedFieldType, dynamic mapping, the analysis chain, analysis-common, _analyze | Level 5; Level 7 |
| 13 | refresh-flush-merge.md | The three background operations; near-real-time visibility; Lucene commits; merge policy and scheduler | Level 5; Level 6 |
If you skip chapter 9, every later storage chapter will reference a shard state machine you have not seen. Read it first.
Group 4 — Search and Aggregations
The read path: how a query fans out across shards, how QueryBuilders become
Lucene Querys, how aggregations build and reduce, and the columnar data
structures that sorting and aggregations actually read.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 14 | search-execution.md | TransportSearchAction, scatter/gather, query→fetch phases, SearchContext, SearchPhaseController reduce | Level 5; Level 8 |
| 15 | query-dsl-querybuilders.md | QueryBuilder/AbstractQueryBuilder→Lucene Query via QueryShardContext; parsing, serialization, registration | Level 5; Level 7 |
| 16 | aggregations.md | AggregatorFactory→Aggregator→InternalAggregation.reduce; bucket vs metric; the reduce pipeline | Level 5; Level 8 |
| 17 | docvalues-fielddata.md | Columnar DocValues, legacy fielddata, IndexFieldData, why heap pressure shows up here | Level 6; the circuit breaker deep dive |
Read 14 before 16 — an aggregation only makes sense once you understand the query/fetch phase split it runs inside.
Group 5 — Allocation, Recovery, Replication
How shards get placed on nodes, how a placed-but-empty shard is filled with data, and how primary and replica stay in sync after they are both running.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 18 | shard-allocation.md | AllocationService, BalancedShardsAllocator, AllocationDeciders, RoutingAllocation, the explain API | Level 4 lab 4.4; Level 6 |
| 19 | recovery.md | Peer recovery, RecoverySourceHandler, phase 1/2, sequence-number-based recovery, the translog hand-off | Level 6; the translog deep dive |
| 20 | replication.md | Document replication (TransportReplicationAction) vs segment replication; ReplicationTracker, global checkpoint | Level 6; Level 8 |
Group 6 — Cross-cutting
The subsystems that touch every other chapter: backups, memory safety, the extension model, and wire compatibility across versions.
| # | File | Summary | Consumed by |
|---|---|---|---|
| 21 | snapshots-repositories.md | SnapshotsService, RepositoriesService, BlobStoreRepository, incremental segment-level snapshots | Level 6; Level 9 |
| 22 | circuit-breakers-memory.md | HierarchyCircuitBreakerService, parent/fielddata/request/in-flight breakers, real-memory accounting | Level 6; Level 8 |
| 23 | plugin-architecture.md | Plugin and its extension interfaces, PluginsService, classloader isolation, plugin-descriptor.properties | the plugin labs; Level 7 |
| 24 | serialization-bwc.md | Version gating in StreamInput/StreamOutput, NamedWriteableRegistry, XContent BWC, the qa/ BWC suite | the compatibility mindset chapter; Level 9 |
A note on order vs index
The deep-dives are an index — they exist to be looked up later. The first read should follow the table above, top to bottom: cluster model, then request path, then storage, then search, then allocation/recovery/replication, then the cross-cutting subsystems. Each group assumes the previous ones.
But when you return to fix a bug, do not re-read linearly. Jump directly to
the chapter most relevant to the failing component and use the cross-references
inside it. A shard-stuck-UNASSIGNED bug starts in shard-allocation.md;
a "documents not visible after index" bug starts in refresh-flush-merge.md;
a "node won't join the cluster" bug starts in discovery-coordination.md.
Every chapter ends with a Validation: prove you understand this section.
Treat that as the gate before declaring the chapter "read." If you cannot answer
the validation questions from memory plus a grep, you have skimmed, not read.