Aggregations — Intensive

This masterclass extends the Aggregations deep-dive. That chapter gave you the four-stage lifecycle (build → create → collect → reduce), the terms approximation story, and the "register a custom agg" sketch. Read it first; this intensive assumes it. Here we go down to the data structures and the code: the exact aggregator tree (AggregatorFactories → AggregatorFactory → Aggregator/AggregatorBase), the per-segment collection lifecycle with the bucket-ordinal model that lets one physical sub-aggregator serve thousands of logical buckets, the GlobalOrdinalsStringTermsAggregator vs MapStringTermsAggregator split, the CompositeAggregator paging machine, the cardinality HyperLogLog++ sketch, and the three layers of memory safety that keep a terms agg over a billion docs from OOM-ing the node. Three labs then make you trace a live aggregation through the profiler and the source, build a custom metric aggregation as a plugin, and break things deliberately with composite paging, a pipeline agg, and search.max_buckets.

By the end you can: name every class on the collect → buildAggregations → reduce path and grep to it; explain owningBucketOrd and CardinalityUpperBound to a reviewer; reason about where the memory goes in a high-cardinality terms agg; and state precisely why an InternalAggregation's reduce must be associative and commutative now that there are two reduce levels (slice and coordinator).

Note: The collector machinery (Collector, LeafCollector, CollectorManager) is Lucene. The aggregation framework (Aggregator, AggregatorBase, InternalAggregation, the bucket/metric/ pipeline subclasses, MultiBucketConsumer, BigArrays) is OpenSearch, in server/src/main/java/org/opensearch/search/aggregations/. When you file a bug you must know which project owns the code — say "OpenSearch Aggregator" not "Lucene aggregation," because Lucene has no aggregations at all.

First principles: an aggregation is a streaming fold over doc values

Strip away the DSL and an aggregation is a fold. You have a stream of documents that matched the query. You visit each one exactly once, in segment order, reading a columnar value for it (a long ordinal, a double, a SortedNumericDocValues cursor) and updating some accumulator. At the end you emit the accumulator as an InternalAggregation. That is the whole model, and two consequences fall out of it immediately:

1. You never random-access documents. Aggregations read doc values / fielddata — the column-oriented, per-segment store — not the inverted index and not stored fields. A terms agg over a keyword field reads SortedSetDocValues; an avg over a long reads SortedNumericDocValues. This is why aggregating a text field is rejected by default: text has no doc values, and turning on fielddata: true builds an expensive in-heap structure instead.
2. The accumulator must be cheap per doc and mergeable across shards. Cheap per doc, because you call collect once per matched document and there can be billions. Mergeable across shards, because each shard folds independently and the coordinator has to combine the partials — which is why every InternalAggregation carries a reduce.

The art of the framework is making "one accumulator per bucket" not cost one object per bucket. A terms agg over a high-cardinality field has millions of buckets; a naive design allocates millions of avg accumulators. OpenSearch instead gives each bucket an integer ordinal and stores the accumulators in flat BigArrays-backed arrays indexed by that ordinal. That single idea — the bucket ordinal — is the backbone of the entire collection path, and the rest of this chapter keeps returning to it.

The aggregator tree: factories, factory, aggregator

There are three layers, and they map onto three lifecycle phases. Get the names exact — reviewers use them precisely.

The shapes nest identically: a terms builder with an avg child produces a TermsAggregatorFactory with an AggregatorFactories holding an AvgAggregatorFactory, which produces a *TermsAggregator whose subAggregators array holds an AvgAggregator. One tree per shard search context.

flowchart TD
    subgraph Parse["Coordinator: parse"]
      TB["TermsAggregationBuilder<br/>(child: AvgAggregationBuilder)"]
    end
    subgraph Shard["Each shard: AggregatorFactories.build"]
      TF["TermsAggregatorFactory"] --> AF["AvgAggregatorFactory"]
      TF -. createInternal .-> TA["GlobalOrdinalsStringTermsAggregator"]
      AF -. create .-> AA["AvgAggregator"]
      TA -- subAggregators[] --> AA
    end
    TB -- AggregatorFactories.build --> TF

cd ~/src/OpenSearch   # an opensearch-project/OpenSearch checkout
# The three layers, side by side:
grep -n "class AggregatorFactories\|public final Aggregator\[\] createSubAggregators\|public Aggregator create" \
  server/src/main/java/org/opensearch/search/aggregations/AggregatorFactories.java
grep -n "abstract class AggregatorFactory\|protected abstract Aggregator createInternal" \
  server/src/main/java/org/opensearch/search/aggregations/AggregatorFactory.java
grep -n "abstract class Aggregator\|public abstract InternalAggregation\[\] buildAggregations\|abstract class AggregatorBase" \
  server/src/main/java/org/opensearch/search/aggregations/Aggregator.java \
  server/src/main/java/org/opensearch/search/aggregations/AggregatorBase.java

AggregatorBase is where the shared plumbing lives: the subAggregators array, the link to the SearchContext, the addRequestCircuitBreakerBytes accounting, and the MultiBucketConsumer hook (next sections). Almost every real aggregator extends BucketsAggregator (for bucket aggs) or a metric base, both of which extend AggregatorBase.

The collection lifecycle: getLeafCollector → collect → buildAggregations

This is the hot loop. Lucene drives it once per segment, OpenSearch drives the folding inside. Three methods matter, in order.

1. getLeafCollector(LeafReaderContext) — bind to a segment

When the query phase reaches a new segment, it calls the aggregator's getLeafCollector(ctx). The aggregator opens the doc-values cursor for that segment here — a terms agg gets the segment's SortedSetDocValues, an avg agg gets SortedNumericDoubleValues — and returns a LeafBucketCollector, a per-segment closure that knows how to read this segment's column.

// shape of GlobalOrdinalsStringTermsAggregator.getLeafCollector (grep to confirm)
protected LeafBucketCollector getLeafCollector(LeafReaderContext ctx,
                                               LeafBucketCollector sub) throws IOException {
    SortedSetDocValues globalOrds = valuesSource.globalOrdinalsValues(ctx);
    return new LeafBucketCollectorBase(sub, globalOrds) {
        @Override
        public void collect(int doc, long owningBucketOrd) throws IOException {
            if (globalOrds.advanceExact(doc)) {
                for (long globalOrd = globalOrds.nextOrd();
                     globalOrd != NO_MORE_ORDS;
                     globalOrd = globalOrds.nextOrd()) {
                    long bucketOrd = bucketOrds.add(owningBucketOrd, globalOrd);
                    if (bucketOrd < 0) {                 // already seen
                        bucketOrd = -1 - bucketOrd;
                        collectExistingBucket(sub, doc, bucketOrd);
                    } else {                              // brand-new bucket
                        collectBucket(sub, doc, bucketOrd);
                    }
                }
            }
        }
    };
}

2. collect(int doc, long owningBucketOrd) — the per-doc fold

LeafBucketCollector.collect(doc, owningBucketOrd) is called once per matched doc in the segment. owningBucketOrd is the single most important parameter in the framework and the one people misread. It is the bucket ordinal of the parent aggregator — the bucket this document already landed in, one level up. The aggregator's job is to read its own value, map (owningBucketOrd, value) to its own bucket ordinal, and either collectBucket (new) or collectExistingBucket (seen). Both increment the bucket's doc count and then call the sub-aggregators' collect with the child bucket ordinal — that recursion is how nesting works without one object per logical bucket.

Warning: A top-level aggregator is always called with owningBucketOrd == 0. A sub-aggregator under a terms is called with owningBucketOrd ranging over every term-bucket ordinal that received this doc. If you write a custom aggregator and assume owningBucketOrd is always 0, your agg is correct as a top-level agg and silently wrong as a sub-agg. This is the classic "works alone, broken nested" bug.

3. buildAggregations(long[] owningBucketOrds) — materialize partials

After all segments are scanned, the framework calls buildAggregations(long[] owningBucketOrds). The argument is an array of owning bucket ordinals because the parent asks the child to build results for many parent buckets at once (one InternalAggregation[] per owning ord). The aggregator walks its bucket-ordinal arrays, reads the accumulators, recursively calls buildSubAggsForBuckets on its children, and emits an InternalAggregation (or one per owning ord). This is also where terms applies its top-shard_size trim — the shard only ships its best candidates.

flowchart TD
    QP["QueryPhase: per segment"] --> GLC["aggregator.getLeafCollector(ctx)"]
    GLC --> LBC["LeafBucketCollector for this segment"]
    LBC -->|"per matched doc"| COL["collect(doc, owningBucketOrd)"]
    COL --> MAP["read value -> bucketOrds.add(owningBucketOrd, value)"]
    MAP --> CB["collectBucket / collectExistingBucket<br/>(incr docCount, drive sub-aggs)"]
    CB -.recurse.-> COL
    QP -->|"after all segments"| BA["buildAggregations(long[] owningBucketOrds)"]
    BA --> IA["InternalAggregation[] (per owning ord)<br/>top-shard_size trim for terms"]
    IA --> WIRE["serialize -> coordinator"]

# The collect/build contract on the bucket base class:
grep -n "collectBucket\|collectExistingBucket\|incrementBucketDocCount\|buildAggregations\|buildSubAggsForBuckets\|bucketOrds" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/BucketsAggregator.java
# The leaf collector contract:
grep -n "abstract class LeafBucketCollector\|void collect(int\|class LeafBucketCollectorBase" \
  server/src/main/java/org/opensearch/search/aggregations/LeafBucketCollector.java

The bucket-ordinal model (the idea the whole framework rests on)

A bucket aggregator does not keep a HashMap<Object, Bucket>. It keeps a BucketsAggregator-owned LongKeyedBucketOrds (a BigArrays-backed hash) that maps (owningBucketOrd, value) → a dense long bucketOrd, plus flat arrays of accumulators indexed by that bucketOrd. The mapping is the crux of memory efficiency:

Concretely: a date_histogram with 50 time buckets and a child terms of 10k terms produces up to 500,000 logical leaf buckets. There is one *TermsAggregator object. Its LongKeyedBucketOrds packs all 500,000 (timeBucketOrd, termOrd) pairs into dense ordinals 0..499,999, and the child avg's sums live in one DoubleArray of length 500,000. The recursion in collect keeps the parent's bucketOrd flowing down as the child's owningBucketOrd, so the right slot is always hit without any per-bucket object.

grep -rn "class LongKeyedBucketOrds\|long add(long owningBucketOrd\|interface LongKeyedBucketOrds" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/terms/LongKeyedBucketOrds.java
grep -rn "CardinalityUpperBound\|class BucketsAggregator" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/BucketsAggregator.java

CardinalityUpperBound: the sizing hint that flows down the tree

When a parent builds its sub-aggregators it passes a CardinalityUpperBound — a hint of how many owning buckets the child might see. A top-level agg gets CardinalityUpperBound.ONE (exactly one owning bucket, ord 0). A terms agg passes CardinalityUpperBound.MANY to its children, telling them to size their BigArrays for many owning ordinals rather than one. Some aggregators specialize on this: a min/max under a single-bucket parent can keep a scalar, but under a MANY parent it must keep an array indexed by owning ord. Mis-handling CardinalityUpperBound is a real source of "correct at top level, OOM or wrong as a sub-agg" bugs.

grep -rn "CardinalityUpperBound\|cardinality.multiply\|cardinality == ONE" \
  server/src/main/java/org/opensearch/search/aggregations/ | head -20

terms: global ordinals vs the map, and how it picks

terms is the most-used and most-misunderstood bucket agg, and it has two execution engines selected per field per request.

Global-ordinals execution (GlobalOrdinalsStringTermsAggregator)

A keyword/SortedSet field has, per segment, segment ordinals: each distinct term gets a small integer 0..(unique-1) in that segment. But ordinals differ across segments — term "apple" might be ord 3 in segment _0 and ord 7 in segment _1. Lucene builds a global ordinal map that translates segment ordinals to a single shard-wide ordinal space. The aggregator then buckets by long ordinal — pure integer arithmetic, no string hashing, no UTF-8 comparison — and only resolves ordinals back to terms at buildAggregations time for the surviving top-shard_size. This is dramatically faster and is the default when global ordinals are cheap to load.

There is a further split inside it: dense (allocate a counter per global ordinal, good when most terms appear) vs remap (hash only the ordinals actually seen, good for sparse/filtered scans). Grep collectionStrategy/DenseGlobalOrds/ RemapGlobalOrds.

Map execution (MapStringTermsAggregator)

When global ordinals are not available or not worth building — numeric terms, scripted values, very high-cardinality fields under a tight filter — the agg falls back to hashing the raw BytesRef value into a BytesKeyedBucketOrds. Slower per doc (hash + compare) but no global-ordinal build cost.

find server/src/main/java/org/opensearch/search/aggregations/bucket/terms -name "*TermsAggregator*.java"
grep -n "GlobalOrdinalsStringTermsAggregator\|MapStringTermsAggregator\|collectionStrategy\|DenseGlobalOrds\|RemapGlobalOrds\|globalOrdinalsValues" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java \
  server/src/main/java/org/opensearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java

Note: The choice is in TermsAggregatorFactory via an ExecutionMode/SubAggCollectionMode decision plus the values-source type. When you debug "why is this terms agg slow," the first question is which engine ran — profile=true (Lab AG1) shows the concrete aggregator class.

date_histogram: rounding into time buckets

date_histogram is a numeric bucket agg whose bucket key is a rounded timestamp. Per doc it reads the long epoch-millis, applies a Rounding (calendar-aware: 1d, 1M, 1y respect months/DST; or fixed: 90m), and uses the rounded value as the bucket key via LongKeyedBucketOrds. The subtlety is calendar rounding: a 1M interval bucket is not a fixed number of milliseconds because months differ in length, so Rounding carries a timezone and a calendar unit. At reduce time InternalDateHistogram.reduce merges buckets by rounded key and, if min_doc_count/extended_bounds ask, fills empty buckets.

grep -rn "class DateHistogramAggregator\|Rounding\|prepareRounding\|class InternalDateHistogram" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/histogram/ | head
grep -n "round\|nextRoundingValue\|class Rounding" \
  server/src/main/java/org/opensearch/common/Rounding.java | head

composite: the only paginating aggregation

Every other bucket agg returns all its buckets (trimmed to size/shard_size) in one response. composite is different: it produces a sorted stream of composite keys (a tuple of several sources — terms, date_histogram, histogram, geotile_grid) and lets you page through them with after. This is how you exhaustively enumerate every combination without asking for a million buckets at once.

The mechanism is a bounded priority queue. CompositeAggregator builds a CompositeValuesCollectorQueue of capacity size. As it collects, it keeps only the size smallest composite keys greater than the after key. At buildAggregations it emits those size buckets plus an after_key — the last (largest) key in the page. The next request passes that after_key as after, and the agg resumes strictly after it. Because the keys are totally ordered and the queue is bounded, memory is O(size) regardless of total cardinality — that is the whole point.

flowchart LR
    R1["request 1<br/>(no after)"] --> Q1["collect: keep size<br/>smallest keys"]
    Q1 --> P1["page 1 buckets<br/>+ after_key = K1"]
    P1 --> R2["request 2<br/>after = K1"]
    R2 --> Q2["collect: keep size smallest<br/>keys > K1"]
    Q2 --> P2["page 2 buckets<br/>+ after_key = K2"]
    P2 --> R3["... until a page returns<br/>fewer than size buckets"]

grep -n "class CompositeAggregator\|CompositeValuesCollectorQueue\|afterKey\|class CompositeKey\|InternalComposite" \
  server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java \
  server/src/main/java/org/opensearch/search/aggregations/bucket/composite/InternalComposite.java

Warning: composite cannot sort by a sub-aggregation metric and cannot be a sub-aggregation itself (it must be top-level). It trades the rich ordering of terms for exhaustive, bounded-memory pagination. Use it for ETL-style "scroll every group," not for "top 10 categories by revenue." You build the full paging loop in Lab AG3.

cardinality: HyperLogLog++ in a fixed budget

Counting distinct values exactly costs memory proportional to the cardinality — unacceptable for a high-cardinality field over a billion docs. cardinality uses HyperLogLog++ (HLL++), a probabilistic sketch with a tunable precision_threshold. The idea: hash each value to 64 bits, use the first p bits to pick one of 2^p registers, and store in that register the maximum number of leading zeros seen in the remaining bits. Many leading zeros is rare, so a register holding "I saw 12 leading zeros" implies you've seen roughly 2^12 distinct values routed to it. Harmonic-mean across registers, correct for bias, and you get a cardinality estimate with standard error ≈ 1.04 / sqrt(2^p).

OpenSearch's HyperLogLogPlusPlus starts in a sparse linear-counting representation for small cardinalities (exact-ish, cheap) and switches to the dense register array past a threshold — that's the "++" refinement. precision_threshold (max 40000) trades memory for accuracy; at the default the sketch is a few KB per bucket regardless of how many distinct values flow through. reduce merges sketches register-by-register (take the max per register), which is associative and commutative — exactly the property the two-level reduce demands.

grep -rn "class CardinalityAggregator\|class HyperLogLogPlusPlus\|precision_threshold\|linearCounting\|leadingZeros\|merge" \
  server/src/main/java/org/opensearch/search/aggregations/metrics/CardinalityAggregator.java \
  server/src/main/java/org/opensearch/search/aggregations/metrics/HyperLogLogPlusPlus.java | head

Pipeline aggregations: post-processing on the reduce

Bucket and metric aggregators run during collection over docs. A pipeline aggregation (PipelineAggregator) runs after reduce, over the output buckets of a sibling or parent aggregation — it never sees a document. derivative differences consecutive date_histogram buckets; bucket_script evaluates a Painless expression over named sibling metrics; moving_fn/ cumulative_sum window over buckets. They are wired via a bucketsPath that names the metric to read (e.g. "sales>monthly_sum").

The load-bearing fact: pipeline logic runs in the final coordinator reduce (InternalAggregation.reducePipelines walks the reduced tree). It cannot run per-shard or per-slice because it needs the globally reduced bucket values. So a pipeline agg is invisible to the slice/shard reduce levels and only materializes once on the coordinator.

grep -rn "interface PipelineAggregator\|reducePipelines\|class DerivativePipelineAggregator\|bucketsPath\|class BucketScriptPipelineAggregator" \
  server/src/main/java/org/opensearch/search/aggregations/pipeline/ | head
grep -n "reducePipelines\|final InternalAggregation reduce" \
  server/src/main/java/org/opensearch/search/aggregations/InternalAggregation.java

Reduce: one level became two

The deep-dive described one reduce: each shard ships an InternalAggregation, the coordinator calls InternalAggregation.reduce(List, ReduceContext), done. With concurrent segment search default-on since 3.0 there are now two reduce levels, and a contributor must keep both correct:

flowchart TD
    subgraph Shard["Shard (data node, concurrent)"]
      s0["slice 0 aggregator tree"] --> sr["AggregationCollectorManager.reduce<br/>(slice reduce)"]
      s1["slice 1 aggregator tree"] --> sr
      sr --> sp["InternalAggregations partial<br/>(shard-local)"]
    end
    sp --> coord["SearchPhaseController.reducedQueryPhase<br/>+ InternalAggregation.reduce (coordinator)"]
    other["other shards"] --> coord
    coord --> pipe["reducePipelines (final only)"]
    pipe --> ans["final aggregations in SearchResponse"]

The discipline is one sentence: reduce must be associative and commutative. Slices finish in nondeterministic order, partial reduces batch arbitrarily (batched_reduce_size), and shard results arrive in arbitrary order. If your reduce depends on order, the answer is correct single-segment and wobbles under concurrency — invisible in a one-slice test, broken in production. This is the same invariant the concurrent segment search masterclass proves for CollectorManager; aggregations inherit it.

grep -rn "class AggregationCollectorManager\|implements CollectorManager\|InternalAggregations.reduce" \
  server/src/main/java/org/opensearch/search/aggregations/ | head
grep -n "public InternalAggregation reduce\|class ReduceContext\|isFinalReduce" \
  server/src/main/java/org/opensearch/search/aggregations/InternalAggregation.java

Three layers of memory safety

A terms agg over a billion-doc, high-cardinality field can allocate gigabytes — millions of bucket ordinals, the global-ordinal map, the sub-agg arrays. OpenSearch has three independent guards, and a contributor must know which one fires when.

1. MultiBucketConsumer / search.max_buckets — the bucket count cap

Every time a bucket aggregator creates a bucket it calls MultiBucketConsumer.accept(int). That consumer counts total buckets across the whole aggregation tree and throws TooManyBucketsException (too_many_buckets_exception) past search.max_buckets (default 65,536). This is a count guard, cheap and deterministic — it protects the coordinator from a response with millions of buckets, independent of how much heap is free.

grep -rn "class MultiBucketConsumer\|MAX_BUCKET_SETTING\|search.max_buckets\|TooManyBucketsException" \
  server/src/main/java/org/opensearch/search/aggregations/MultiBucketConsumerService.java

2. The request circuit breaker via BigArrays — the byte guard

Every BigArrays allocation an aggregator makes (bucket-ord hash, sum arrays, HLL registers) is accounted against the request circuit breaker. When the sum of in-flight request bytes crosses indices.breaker.request.limit (default ~60% of heap) the next allocation throws CircuitBreakingException. This is a byte guard — it doesn't care how many buckets, only how much memory. Bucket-count and byte guards are orthogonal: a few huge buckets trip the breaker; many tiny buckets trip max_buckets. Details in circuit breakers and memory.

grep -rn "addRequestCircuitBreakerBytes\|bigArrays\|REQUEST.*breaker\|CircuitBreakingException" \
  server/src/main/java/org/opensearch/search/aggregations/AggregatorBase.java | head

3. CardinalityUpperBound — pre-sizing the arrays

The sizing hint from earlier is also a safety mechanism: by telling a child how many owning buckets to expect, the parent lets the child allocate right-sized BigArrays (which grow geometrically) instead of thrashing. It does not stop an OOM by itself but keeps the byte guard from being hit by over-allocation.

You trip both of the first two deliberately in Lab AG3.

The star-tree fast path

Some aggregations never need to scan documents at all. The star-tree index precomputes aggregated metrics over chosen dimensions at index time; an eligible request (supported metrics over star-tree dimensions, no scripts, compatible filters) is answered by reading the tree instead of the per-doc fold. When it applies it is orders of magnitude faster because it skips collection entirely. The hook is a StarTreeQueryHelper/StarTreeAggregator-style path that the query phase chooses before falling back to the normal aggregator. Read the engineering chapter for eligibility rules and the tree layout.

grep -rn "StarTree\|star_tree\|StarTreeQueryHelper\|supportsStarTree" \
  server/src/main/java/org/opensearch/search/aggregations/ \
  server/src/main/java/org/opensearch/search/startree/ 2>/dev/null | head

Worked example: terms + sub-avg

"Average price per category, top 3 categories" — the canonical nested agg. Index six docs across three categories:

{ "category": "books",  "price": 10 }
{ "category": "books",  "price": 20 }
{ "category": "toys",   "price": 30 }
{ "category": "toys",   "price": 50 }
{ "category": "games",  "price": 40 }
{ "category": "books",  "price": 12 }

{
  "size": 0,
  "aggs": {
    "by_category": {
      "terms": { "field": "category", "size": 3 },
      "aggs": { "avg_price": { "avg": { "field": "price" } } }
    }
  }
}

What happens, in framework terms:

1. Build/create: a GlobalOrdinalsStringTermsAggregator with one child AvgAggregator. The terms agg passes CardinalityUpperBound.MANY to the avg.
2. Collect: for each doc, the terms agg reads category's global ordinal, maps (owningBucketOrd=0, ord) → a bucketOrd via LongKeyedBucketOrds (books→0, toys→1, games→2), increments that bucket's count, then calls the avg child's collect(doc, bucketOrd). The avg keeps sums[bucketOrd] and counts[bucketOrd] in two DoubleArray/LongArrays — so sums[0] accrues 10+20+12=42 over 3 docs, sums[1]=80 over 2, sums[2]=40 over 1.
3. buildAggregations: terms emits its top-3 buckets; for each it calls the avg child's buildAggregations with that bucketOrd, producing InternalAvg (books 14.0, toys 40.0, games 40.0). Result is one InternalTerms with three InternalAvg children.
4. Reduce: with one shard, the coordinator reduce is a pass-through; with many shards, InternalTerms.reduce merges buckets by key, sums counts, and recursively reduces each bucket's InternalAvg (sum the sums, sum the counts, divide once) before trimming to size.

The whole nested computation uses one terms aggregator and one avg aggregator, three bucket ordinals, and a handful of flat arrays — no per-bucket objects. That is the bucket-ordinal model paying off.

Real grep targets

cd ~/src/OpenSearch
# The tree
grep -n "createSubAggregators\|class AggregatorFactories" server/src/main/java/org/opensearch/search/aggregations/AggregatorFactories.java
grep -n "createInternal\|class AggregatorFactory" server/src/main/java/org/opensearch/search/aggregations/AggregatorFactory.java
# The collect path
grep -n "buildAggregations\|getLeafCollector\|class AggregatorBase" server/src/main/java/org/opensearch/search/aggregations/AggregatorBase.java
grep -n "collectBucket\|collectExistingBucket\|bucketOrds\|incrementBucketDocCount" server/src/main/java/org/opensearch/search/aggregations/bucket/BucketsAggregator.java
# terms engines
find server/src/main/java/org/opensearch/search/aggregations/bucket/terms -name "*.java" | sort
# composite paging
grep -n "afterKey\|CompositeValuesCollectorQueue" server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java
# cardinality sketch
grep -n "class HyperLogLogPlusPlus\|precision" server/src/main/java/org/opensearch/search/aggregations/metrics/HyperLogLogPlusPlus.java
# memory guards
grep -n "search.max_buckets\|MultiBucketConsumer" server/src/main/java/org/opensearch/search/aggregations/MultiBucketConsumerService.java
# registration SPI
grep -n "getAggregations\|AggregationSpec\|addResultReader" server/src/main/java/org/opensearch/plugins/SearchPlugin.java

Common bugs and symptoms

Validation: prove you understand this

1. Name the three tree layers (AggregatorFactories/AggregatorFactory/ Aggregator) and which lifecycle phase builds each, on which node.
2. Define owningBucketOrd precisely. Explain why a sub-aggregator of a terms agg sees many distinct owning ords and a top-level agg sees only 0.
3. Explain the bucket-ordinal model: how does one physical avg aggregator serve 500,000 logical buckets without 500,000 objects? Name the data structure.
4. Contrast GlobalOrdinalsStringTermsAggregator and MapStringTermsAggregator: the bucket key, the up-front cost, and when each wins.
5. Why is composite the only paginating agg? Describe the bounded queue and the after_key mechanism, and why memory is O(size).
6. Sketch HyperLogLog++: registers, leading-zeros, the linear-counting start, and why reduce (max per register) is associative and commutative.
7. List the three memory guards, the exception each throws, and whether it counts buckets or bytes. Which one does a high-cardinality, few-buckets agg trip?
8. There are now two reduce levels. Name them, the class that owns each, and give a reduce failure mode that only appears at the slice level.

Next: Lab AG1 — Trace an Aggregation, then Lab AG2 — Build a Custom Aggregation and Lab AG3 — Composite, Pipeline, and Memory. This intensive pairs with the concurrent segment search masterclass (the second reduce level) and the circuit breakers and memory deep-dive (the byte guard).