Lab SE3: Build a Security Extension

Prerequisite: Lab SE1 and Lab SE2 (you understand the authc chain and the authz path). Concept reading: Security — Intensive and plugin-architecture.

Background

The Security plugin is itself extensible. The authc chain you configured in Lab SE1 is built from HTTP authenticators and authentication backends that implement small Java SPIs. New ones can be added — that is how JWT, OIDC, SAML, and Kerberos support all arrived. In this lab you will trace the real SPI for an HTTP authenticator, then implement a small one of your own — a header-based authenticator that reads a username from a trusted header — wire it into config.yml, and write a unit test.

Note: This is an illustrative extension to teach the SPI and the contribution mechanics. A "trust a header" authenticator is a real pattern (used behind an authenticating proxy) but is only safe when the header cannot be spoofed by clients — exactly the kind of caveat a good PR description must state. Treat the code as a faithful skeleton: grep the real interface in your checkout for the exact method signatures, which drift across versions.

Why this matters for contributors

Extension points are where most accepted security contributions land — a new authenticator, a principal extractor, an audit sink — because they add capability without touching the dangerous core authz path. Knowing how to find the SPI, implement against it, register it, and test it is the difference between "I have an idea" and "here is a mergeable PR." This lab also walks the contribution mechanics (DCO sign-off, CHANGELOG, tests) that opensearch-project/security enforces.

Prerequisites

• A clone of the security source: git clone https://github.com/opensearch-project/security && cd security.
• JDK 21+ and the ability to run ./gradlew in that repo.
• The secured demo node from Lab SE1 for an end-to-end test (optional but ideal).

Step-by-step tasks

Step 1 — Find the real SPI

The HTTP authenticator interface is the contract every chain link's http_authenticator implements. Grep for it:

# The HTTP authenticator SPI and an existing implementation to model on:
find . -name "HTTPAuthenticator.java"
grep -rn "interface HTTPAuthenticator\|extractCredentials\|reRequestAuthentication\|getType" \
  $(find . -name "HTTPAuthenticator.java")
# A concrete impl to copy the shape from:
find . -name "HTTPBasicAuthenticator.java"
grep -rn "implements HTTPAuthenticator\|extractCredentials\|AuthCredentials" \
  $(find . -name "HTTPBasicAuthenticator.java")

You are looking for (names approximate — confirm in your checkout):

The companion SPI is the backend:

find . -name "AuthenticationBackend.java"
grep -rn "interface AuthenticationBackend\|authenticate(\|exists(" \
  $(find . -name "AuthenticationBackend.java")

• Write down the exact signatures of HTTPAuthenticator and AuthCredentials from your checkout. Everything below targets that shape.

Step 2 — Implement a header-based authenticator

Create the class. It reads a configurable header (default x-proxy-user) and turns its value into an AuthCredentials for a username, leaving validation to a backend (or noop, when the proxy is trusted).

// src/main/java/org/opensearch/security/auth/http/proxy/HTTPHeaderUserAuthenticator.java
package org.opensearch.security.auth.http.proxy;

import org.opensearch.OpenSearchSecurityException;
import org.opensearch.common.settings.Settings;
import org.opensearch.security.auth.HTTPAuthenticator;
import org.opensearch.security.user.AuthCredentials;
import org.opensearch.security.filter.SecurityRequest;
import org.opensearch.security.filter.SecurityResponse;

import java.util.Optional;

/**
 * Trusts a username supplied in a configurable HTTP header (default x-proxy-user).
 * ONLY safe behind a proxy that strips/sets the header so clients cannot spoof it.
 *
 * config.yml:
 *   http_authenticator:
 *     type: header_user
 *     config:
 *       user_header: x-proxy-user
 */
public class HTTPHeaderUserAuthenticator implements HTTPAuthenticator {

    private final String userHeader;

    public HTTPHeaderUserAuthenticator(final Settings settings, final java.nio.file.Path configPath) {
        // settings is the per-domain `config:` block from config.yml
        this.userHeader = settings.get("user_header", "x-proxy-user");
    }

    @Override
    public String getType() {
        return "header_user"; // the `type:` you reference in config.yml
    }

    @Override
    public AuthCredentials extractCredentials(final SecurityRequest request, final org.opensearch.common.util.concurrent.ThreadContext context) {
        final String username = request.header(userHeader);
        if (username == null || username.isBlank()) {
            return null; // no credential here -> chain falls through to the next authc domain
        }
        // Mark complete: the username is fully established; no password to verify.
        return new AuthCredentials(username.trim()).markComplete();
    }

    @Override
    public Optional<SecurityResponse> reRequestAuthentication(final SecurityRequest request, final AuthCredentials creds) {
        // Header auth has no interactive challenge; do not send WWW-Authenticate.
        return Optional.empty();
    }
}

Warning: markComplete() (or the equivalent in your checkout) tells the chain "this credential needs no backend validation." That is only correct because we assume a trusted proxy set the header. If you returned an incomplete credential instead, BackendRegistry would route it to the configured authentication_backend for validation — which is what you want if a backend should still verify the user exists. State this design choice explicitly in your PR; reviewers will ask.

Step 3 — Register it / make it discoverable

Security instantiates authenticators reflectively by the type: string in config.yml, resolving it to a class. Find how an existing type maps to its class so your new one is discoverable:

grep -rn "newInstance\|getType\|\"basic\"\|\"jwt\"\|\"clientcert\"\|ReflectionHelper\|loadClass\|http_authenticator" \
  $(grep -rln "newInstance\|http_authenticator" src/main/java/org/opensearch/security/ | grep -i "registry\|config\|reflection" | head)

Two registration styles exist depending on version:

• Built-in type short name — a switch/map from "basic"/"jwt" → class. Add "header_user" → HTTPHeaderUserAuthenticator there, or
• Fully-qualified class name in config.yml — set type: org.opensearch.security.auth.http.proxy.HTTPHeaderUserAuthenticator, which the reflection helper loads directly (no core change needed).

The FQCN route is best for a first contribution: it needs no change to the registry and proves the SPI works.

# config/opensearch-security/config.yml — a new authc domain
    proxy_header_domain:
      http_enabled: true
      transport_enabled: false
      order: 2
      http_authenticator:
        type: org.opensearch.security.auth.http.proxy.HTTPHeaderUserAuthenticator
        challenge: false
        config:
          user_header: x-proxy-user
      authentication_backend:
        type: noop

• Decide and document which registration style you used and why.

Step 4 — Build the plugin against core

The security repo builds against a pinned OpenSearch core version. Compile and assemble:

./gradlew compileJava            # fast: just your class compiles
./gradlew assemble               # builds the plugin zip
# If you need it against a locally-built core, publish core to mavenLocal first
# (in the OpenSearch core checkout):  ./gradlew publishToMavenLocal

To try it end to end, install your built plugin zip into a distribution (replacing the bundled one) or drop the compiled classes onto a dev node, apply the config.yml above with securityadmin.sh, then:

curl -s --cacert config/root-ca.pem -H 'x-proxy-user: carol' \
  https://localhost:9200/_plugins/_security/authinfo?pretty | head
# user_name: carol  -- authenticated purely from the header, no -u, no token

Step 5 — Write a unit test

The security repo uses JUnit. Test the credential extraction directly — no cluster needed:

// src/test/java/org/opensearch/security/auth/http/proxy/HTTPHeaderUserAuthenticatorTest.java
package org.opensearch.security.auth.http.proxy;

import org.junit.Test;
import org.opensearch.common.settings.Settings;
import org.opensearch.security.user.AuthCredentials;
import org.opensearch.security.filter.SecurityRequest;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class HTTPHeaderUserAuthenticatorTest {

    private HTTPHeaderUserAuthenticator newAuth(String headerName) {
        Settings s = Settings.builder().put("user_header", headerName).build();
        return new HTTPHeaderUserAuthenticator(s, null);
    }

    @Test
    public void extractsUsernameFromConfiguredHeader() {
        SecurityRequest req = mock(SecurityRequest.class);
        when(req.header("x-proxy-user")).thenReturn("carol");

        AuthCredentials creds = newAuth("x-proxy-user").extractCredentials(req, null);

        assertNotNull(creds);
        assertEquals("carol", creds.getUsername());
        assertTrue("trusted-proxy creds must be complete", creds.isComplete());
    }

    @Test
    public void returnsNullWhenHeaderAbsent_soChainFallsThrough() {
        SecurityRequest req = mock(SecurityRequest.class);
        when(req.header("x-proxy-user")).thenReturn(null);

        assertNull(newAuth("x-proxy-user").extractCredentials(req, null));
    }

    @Test
    public void honorsCustomHeaderName() {
        SecurityRequest req = mock(SecurityRequest.class);
        when(req.header("x-sso-user")).thenReturn("dave");

        AuthCredentials creds = newAuth("x-sso-user").extractCredentials(req, null);

        assertEquals("dave", creds.getUsername());
    }
}

./gradlew test --tests "org.opensearch.security.auth.http.proxy.HTTPHeaderUserAuthenticatorTest"

• Get all three tests green. The middle test — null header ⇒ null creds ⇒ chain falls through — is the most important: it guarantees your authenticator plays nicely as one link in the chain rather than swallowing every request.

Deliverables

• The grep output capturing the real HTTPAuthenticator and AuthCredentials signatures from your checkout.
• HTTPHeaderUserAuthenticator.java compiling under ./gradlew compileJava.
• The config.yml authc domain registering it, and the authinfo response showing a user authenticated purely from the header.
• The passing unit test (three cases), and a written note on the markComplete() / trusted-proxy security caveat.

How to contribute this to opensearch-project/security

This is a real, mergeable shape of contribution. The mechanics (see pr-quality for the general bar):

1. Sign off every commit (DCO). The repo enforces the Developer Certificate of Origin; an unsigned commit fails the DCO check.

git commit -s -m "Add header-based HTTP authenticator for trusted-proxy auth"
# -s appends: Signed-off-by: Your Name <your@email>

2. Add a CHANGELOG entry. Security keeps a CHANGELOG.md; add a line under the unreleased/Added section referencing your PR.

grep -n "## \[Unreleased\]\|### Added" CHANGELOG.md | head

3. Tests + green CI. A new authenticator needs unit tests (above) and ideally an integration test that exercises the chain. Run the relevant suites locally:

./gradlew spotlessCheck    # formatting gate — fails CI if off
./gradlew test

4. A PR description that names the security trade-off. For a header authenticator the reviewer will immediately ask "how is the header protected from spoofing?" — answer it pre-emptively (trusted proxy strips/sets it; document the deployment requirement; default off).

5. Open against main, link any tracking issue, and respond to review (see responding-to-feedback).

git push origin add-header-authenticator
gh pr create --repo opensearch-project/security \
  --title "Add header-based HTTP authenticator (trusted-proxy)" \
  --body  "Implements HTTPAuthenticator reading a configurable user header.
Safe only behind a proxy that controls the header; default off; unit tests added.
DCO signed, CHANGELOG updated."

Note: Start by opening an issue or discussion for a new extension point before a large PR — see design-via-github. A small, well-tested authenticator with a clear security caveat is a textbook good first security contribution; a sweeping change to PrivilegesEvaluator is not — that touches the core authz decision and gets intense scrutiny.

Troubleshooting

Expected output

• ./gradlew compileJava and the three-case test pass.
• With the config.yml domain applied, curl -H 'x-proxy-user: carol' .../authinfo returns user_name: carol with no -u and no token.
• A removed/empty header falls through the chain (the next domain, or 401).

Stretch goals

• Add a custom principal extractor instead: implement the PrincipalExtractor SPI (grep -rn "interface PrincipalExtractor") to derive the username from a client-certificate field other than the default DN, and wire it via plugins.security.ssl.transport.principal_extractor_class.
• Make the authenticator return an incomplete credential and pair it with a real backend (internal users), so the header supplies the username but a backend still confirms the user exists — and write a test for the backend-validated path.
• Add an integration test that boots a security test cluster, applies a config.yml with your domain, and asserts the header → user end to end.

Coding Exercises

You implemented one authenticator and one unit test; these exercises grade it into a mergeable, well-tested security extension. All target your clone of opensearch-project/security. Confirm every SPI signature with rg first (rg -n "interface HTTPAuthenticator" $(find . -name HTTPAuthenticator.java)); the methods drift across versions, so never trust a stale skeleton.

1. (warm-up) Harden the unit test against header injection edge cases. Extend HTTPHeaderUserAuthenticatorTest with cases for: a blank header ("" ⇒ null, chain falls through), a header with surrounding whitespace (asserts it is trimmed), and a duplicate-header scenario if SecurityRequest.header(...) returns only the first. Keep them all green: ./gradlew test --tests "*HTTPHeaderUserAuthenticator*".

2. (core) Test the backend-validated (incomplete-credential) path. Right now you return markComplete(). Add a configuration toggle (validate_with_backend: true) that returns an incomplete AuthCredentials so BackendRegistry routes it to the configured authentication_backend. Write a unit test asserting that, when the toggle is on, the returned credential isComplete() is false. Read how an existing authenticator signals "needs backend validation" (rg -n "markComplete|isComplete|AuthCredentials" $(find . -name HTTPBasicAuthenticator.java)).

3. (core) Add a small security extension point + test: a PrincipalExtractor. Model the first Stretch goal as graded code. Implement the PrincipalExtractor SPI (rg -n "interface PrincipalExtractor" $(find . -name PrincipalExtractor.java)) to derive the username from a client-certificate field other than the default DN, and write a unit test that feeds it a mock X500Principal/cert and asserts the extracted principal. This is a second, independent SPI — proving you can find, implement, and test any security extension point, not just the one you copied.

4. (core) A security integration test for the chain. Model the second Deliverable in CI-runnable form. Find the integ-test harness (rg -l "LocalCluster|extends.*IntegrationTest|class.*IT" src/integrationTest src/test | head) and write a test that boots a secured node, applies a config.yml whose authc chain includes your header_user domain (via FQCN), sends a request with x-proxy-user: carol and asserts authinfo returns user_name: carol, and sends a request with no header and asserts it falls through to the next domain (or 401). This exercises the whole chain, not just extractCredentials.

5. (core) Prove the chain-fall-through contract under ordering changes. Add a test that places your header_user domain at order: 0 (before basic) and asserts a Basic-only request still authenticates (because your authenticator returns null for header-less requests and the chain advances). Then move it after basic and assert the same. This pins the most important property of any chain link: it must not swallow requests it cannot handle.

6. (advanced) Make header_user a first-class built-in type + full PR. Advanced challenge: instead of the FQCN route, register "header_user" as a short type: name in the authenticator registry (rg -n "newInstance|\"basic\"|\"jwt\"|ReflectionHelper|loadClass" $(rg -l "newInstance|http_authenticator" src/main/java/org/opensearch/security | grep -iE "registry|config|reflection" | head)), add the mapping, and prove it with: the unit tests (1–2), the integ test (4), a CHANGELOG.md entry under ### Added, a passing ./gradlew spotlessCheck, and a DCO-signed commit (git commit -s). The deliverable is the full mergeable bundle — code + both test levels + the four contribution gates green — exactly the shape the "How to contribute" section describes. State the trusted-proxy security caveat in your (draft) PR body.

Issues to Practice On

Extension points are where most accepted security contributions land. Find work on opensearch-project/security (labels move; confirm on the tracker):

gh issue list --repo opensearch-project/security --label "good first issue" --state open
gh issue list --repo opensearch-project/security --label "enhancement" --state open --search "authenticator OR SPI OR extension OR JWT OR OIDC OR SAML"
gh issue list --repo opensearch-project/security --label "help wanted" --state open
gh label list --repo opensearch-project/security | grep -iE "enhancement|good first|help wanted|auth"

Representative patterns. (1) "Support a new auth scheme / header / claim" — a new HTTPAuthenticator or backend: find the SPI via rg, implement against it, register, unit + integ test, and ship with the security trade-off stated up front. (2) "Make an existing authenticator configurable" (e.g. a custom header name or claim key) — locate the Settings.get(...) reads, add the option with a sane default, and add a test for the new path. Arc: reproduce/scope → locate via rg → implement → test (unit + integ) → PR with CHANGELOG + spotlessCheck + DCO. Prefer opening an issue/discussion for a new extension point first — see design-via-github.

Planted-bug drill. In your HTTPHeaderUserAuthenticator.extractCredentials, change the empty-header guard so it returns a credential for username == null instead of returning null. Run your unit tests and watch the "null header ⇒ chain falls through" case go red — that test is what stops your authenticator from hijacking every request in the chain. Revert, then keep that assertion as the regression guard, and add the ordering test from Exercise 5 so the fall-through contract is pinned at the chain level too.

Etiquette: claim or open the issue first, and every PR ships unit + integ tests, a CHANGELOG.md entry, a green spotlessCheck, and a DCO Signed-off-by (git commit -s). See pr-quality and responding-to-feedback.

Validation / self-check

• Name the two authc SPIs (HTTPAuthenticator, AuthenticationBackend), the method on each that does the real work, and where the type: string in config.yml resolves to your class.
• Explain why returning null from extractCredentials (not throwing) is the correct "no credential here" behavior for a chain link.
• State the security caveat of a header authenticator and how a PR should address it up front.
• List the four contribution gates this PR must pass: DCO sign-off, CHANGELOG, spotlessCheck, and tests — and the command for each.