Plugin Architecture

Almost everything interesting about OpenSearch in production — security, k-NN vector search, SQL/PPL, alerting, ML — is a plugin. The core engine ships a deliberately small surface and exposes dozens of extension points; plugins implement them to add field types, queries, aggregations, REST endpoints, analyzers, repositories, scripts, and entire subsystems. As a contributor you will either write plugins or write the extension points plugins consume, and either way you must understand how PluginsService discovers, isolates, and wires them.

This chapter maps the Plugin base class and the extension interfaces, the PluginsService loading flow, the isolated-classloader model, the plugin-descriptor.properties contract, the Gradle build plugin, and the forward-looking Extensions SDK. It ties together the consumer-side deep dives: Query DSL (SearchPlugin), Aggregations (SearchPlugin), Snapshots and Repositories (RepositoryPlugin), The Action Framework (ActionPlugin), and Mapping and Analysis (MapperPlugin/AnalysisPlugin).

After this chapter you can:

• Name the major plugin extension interfaces and what each contributes.
• Trace plugin discovery and loading through PluginsService.
• Explain the isolated-classloader model and why it both protects and constrains plugins.
• Build an in-repo plugin skeleton with the opensearch.opensearchplugin Gradle plugin.

The base class and the extension interfaces

A plugin extends org.opensearch.plugins.Plugin (which provides lifecycle hooks and component creation) and implements zero or more extension interfaces, each of which the engine queries during startup for contributions.

ls server/src/main/java/org/opensearch/plugins/
grep -rln "interface .*Plugin" server/src/main/java/org/opensearch/plugins/
grep -n "interface SearchPlugin\|getQueries\|getAggregations\|getRescorers" \
  server/src/main/java/org/opensearch/plugins/SearchPlugin.java

The base Plugin also has general hooks: createComponents(...) (register singletons into the dependency-injection graph), getSettings() (declare custom Setting<T>), getNamedWriteables() / getNamedXContent() (register serialization — see Serialization and BWC), and getExecutorBuilders() (custom thread pools).

grep -n "createComponents\|getSettings\|getNamedWriteables\|getNamedXContent\|getExecutorBuilders" \
  server/src/main/java/org/opensearch/plugins/Plugin.java

PluginsService: discovery and loading

At node startup, PluginsService scans the plugins/ directory of the installed distribution. Each plugin is a subdirectory containing its jars and a plugin-descriptor.properties. PluginsService reads the descriptor, validates version compatibility, builds an isolated classloader per plugin, instantiates the Plugin subclass, and exposes the collection to the rest of the node.

flowchart TD
    N[Node startup] --> PS[PluginsService scans plugins/ dir]
    PS --> D[read plugin-descriptor.properties per plugin]
    D --> V[verify version + java version + dependencies]
    V --> CL[build isolated ClassLoader for the plugin jars]
    CL --> I[instantiate Plugin subclass]
    I --> R[Node queries each extension interface: getQueries/getActions/getMappers...]
    R --> W[contributions wired into registries, ActionModule, SearchModule, etc.]
    W --> C[createComponents -> DI graph; node starts]

grep -n "class PluginsService\|loadBundle\|loadPlugin\|PluginInfo\|verifyCompatibility\|getPluginsAndModules" \
  server/src/main/java/org/opensearch/plugins/PluginsService.java

Note: Modules (under modules/ in the source, e.g. analysis-common, aggs-matrix-stats, transport-netty4) use the same plugin mechanism but are bundled and loaded by default — you can't uninstall them. Optional plugins (under plugins/, or installed via bin/opensearch-plugin) are opt-in. Same machinery, different lifecycle.

Isolated classloaders — protection and constraint

Each plugin gets its own classloader, a child of the core classloader. This:

• Protects the core and other plugins: a plugin can bundle its own dependency versions without colliding with the engine's.
• Constrains plugins: a plugin can see core classes (parent) but not another plugin's classes — unless that other plugin is ExtensiblePlugin and deliberately exposes an SPI. This is why two unrelated plugins can't directly call each other.

flowchart TD
    SYS[System / core classloader: org.opensearch.* + Lucene] --> P1[Plugin A classloader]
    SYS --> P2[Plugin B classloader]
    SYS --> P3[Plugin C classloader]
    P1 -. cannot see .-> P2
    P3 -->|SPI via ExtensiblePlugin| P3X[Extension plugin loaded into C's loader]

grep -rn "ExtensionLoader\|loadExtensions\|reloadSPI\|ExtensiblePlugin\|URLClassLoader\|ExtendedPluginsClassLoader" \
  server/src/main/java/org/opensearch/plugins/

The SPI mechanism (ExtensiblePlugin + ExtensionLoader) is how, for example, an analysis plugin can be extended by another plugin's tokenizer — the extensible plugin's classloader is made visible to its extenders.

plugin-descriptor.properties — the contract

Every plugin ships a plugin-descriptor.properties that the engine validates before loading. The critical fields:

name=my-plugin
description=Adds a custom query and aggregation
version=3.0.0
opensearch.version=3.0.0
java.version=21
classname=org.example.MyPlugin
extended.plugins=                    # plugins this one extends via SPI (optional)
custom.foldername=                   # optional install dir name

opensearch.version must match the running node's version (plugins are version-locked — a 2.x plugin will not load on 3.x). classname names the Plugin subclass to instantiate. The opensearch.opensearchplugin Gradle plugin generates this file from your build.gradle.

grep -rn "opensearch.version\|classname\|java.version\|extended.plugins" \
  server/src/main/java/org/opensearch/plugins/PluginInfo.java
find . -name "plugin-descriptor.properties" 2>/dev/null | head

Warning: Version mismatch is the #1 plugin install failure. Building a plugin against opensearch 3.1.0 and installing it on a 3.0.0 node fails at load with a clear compatibility error. Plugin authors must rebuild per OpenSearch version; this is intentional, because the internal APIs plugins use are not guaranteed stable across versions (unlike the REST API).

Building and installing a plugin

The opensearch.opensearchplugin Gradle plugin packages a plugin zip with the descriptor, jars, and (optional) security.policy.

// build.gradle of an in-repo plugin
apply plugin: 'opensearch.opensearchplugin'

opensearchplugin {
  name 'my-plugin'
  description 'Adds a custom query and aggregation'
  classname 'org.example.MyPlugin'
  licenseFile rootProject.file('LICENSE.txt')
  noticeFile rootProject.file('NOTICE.txt')
}

# Build the plugin zip (in-repo)
./gradlew :plugins:my-plugin:assemble
ls plugins/my-plugin/build/distributions/

# Install into a distribution
bin/opensearch-plugin install file:///abs/path/my-plugin-3.0.0.zip
bin/opensearch-plugin list
bin/opensearch-plugin remove my-plugin

bin/opensearch-plugin verifies the descriptor, checks the version, optionally prompts for security policy grants the plugin requested, unpacks into plugins/, and the next node start loads it.

grep -rn "class InstallPluginCommand\|class ListPluginsCommand\|verify" \
  distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/ | head

In-repo vs out-of-repo plugins

The big functional plugins live in their own repos under opensearch-project/ precisely because they're large, independently-maintained subsystems. They consume the same extension interfaces you've seen here, built against the published artifacts.

Forward-looking: the Extensions SDK

Plugins run in-process with full access to internal APIs — powerful, but it means a buggy plugin can destabilize the node and plugin authors are coupled to unstable internals. OpenSearch 2.10+ ships an experimental Extensions SDK (opensearch-sdk-java) that runs extensions out-of-process, communicating over a defined protocol, for fault isolation and looser version coupling. Treat it as the strategic direction for some extension categories; in-process plugins remain the mainstream, fully-supported model you should learn first.

grep -rn "ExtensionsManager\|extension\b\|DiscoveryExtensionNode" \
  server/src/main/java/org/opensearch/extensions/ 2>/dev/null | head

Reading exercise

# 1. The extension interfaces and their methods
for f in SearchPlugin ActionPlugin MapperPlugin AnalysisPlugin RepositoryPlugin EnginePlugin ClusterPlugin; do
  echo "== $f =="; grep -n "interface $f\|default\|List<" \
    server/src/main/java/org/opensearch/plugins/$f.java | head
done

# 2. PluginsService loading
grep -n "loadBundle\|loadPlugin\|verifyCompatibility\|getPlugins\|onModule" \
  server/src/main/java/org/opensearch/plugins/PluginsService.java

# 3. Descriptor parsing + version lock
grep -n "opensearch.version\|isOfficialPlugin\|readFromProperties\|class PluginInfo" \
  server/src/main/java/org/opensearch/plugins/PluginInfo.java

# 4. A real in-repo plugin's Plugin class
find plugins -name "*Plugin.java" -path "*repository-s3*" -o -name "*Plugin.java" -path "*analysis-icu*" | head

Answer:

1. Name the extension interface for each of: a custom query, a custom field type, a custom REST endpoint, a snapshot repository, and a custom analyzer.
2. Walk PluginsService from "scan plugins dir" to "node has a live Plugin instance," naming the descriptor validation and the classloader step.
3. Why does each plugin get its own classloader? Give one thing it protects and one thing it prevents.
4. What does ExtensiblePlugin enable that a normal plugin can't, and how does the SPI/ExtensionLoader make one plugin's classes visible to another?
5. Why must a plugin be rebuilt for each OpenSearch version, and where is that version compatibility enforced?
6. Contrast in-repo and out-of-repo plugins on build dependency and release cadence, and name two examples of each.

Common bugs and symptoms

Validation: prove you understand this

1. From memory, list eight plugin extension interfaces and exactly one thing each contributes to the engine.
2. Draw the PluginsService loading flow from directory scan through descriptor validation, classloader creation, instantiation, and extension-point querying.
3. Explain the isolated-classloader model with a diagram showing why Plugin A can't see Plugin B's classes and how ExtensiblePlugin makes an exception.
4. Write a minimal plugin-descriptor.properties and explain which field locks the plugin to an engine version and which names the entrypoint class.
5. Show the Gradle config and the bin/opensearch-plugin commands to build, install, list, and remove an in-repo plugin.
6. Explain why the big functional plugins (security, k-NN, SQL) live in separate repos and how they still consume the same extension interfaces, then state how the Extensions SDK differs from in-process plugins.