Cluster tenant isolation

Cluster tenant isolation lets several customers share one Lucenia cluster while each addresses indices by its own names, without knowing that the others exist. A tenant writes to orders and the cluster stores org-acme-orders. Requests are rewritten on the way in and responses on the way out, so the prefix never reaches the customer in either direction.

This is an operator feature. It is configured by whoever runs the cluster, not by the customers on it, and nothing in it is visible to a tenant.

Isolation is about what a tenant can see. For what a tenant can consume — attributing load, bounding one tenant's share of the cluster, and keeping autoscaling from growing the tier on behalf of a tenant being throttled — see Tenant capacity and admission control.

note

This is not OpenSearch Dashboards multi-tenancy, which separates saved objects — visualizations, dashboards, index patterns — between groups of users sharing one set of indices. Cluster tenant isolation separates the indices themselves. The two are unrelated, can be used together, and unfortunately have similar names: the Dashboards setting is multitenancy_enabled in opensearch_dashboards.yml, and the one described here is multitenancy.enabled in opensearch.yml.

When to use it

Use it when one cluster serves customers who must not see each other's data and who should not have to think about namespacing. A shared tier where each customer believes the cluster is theirs is the case it was built for.

Do not use it as a substitute for a security boundary between untrusted parties who require physical separation. Tenants share nodes, memory, disk, and a circuit breaker; a tenant issuing an expensive query affects the others. Isolation here is logical.

How it works

Every tenant has a prefix, org-<tenant-id>-, derived from an attribute on the authenticated user. Two things then happen on every request:

1. Index names in the request are rewritten from logical to physical before privileges are evaluated. orders becomes org-acme-orders; * becomes org-acme-*; an exclusion -old becomes -org-acme-old; date math keeps its brackets, so <logs-{now/d}> becomes <org-acme-logs-{now/d}>.
2. The prefix is removed from the response, in search hits, mapping keys, write acknowledgements, and index names quoted inside error messages.

What actually enforces isolation

The tenant role's index pattern is the boundary. It is written in physical terms and substituted per user at authorization time:

org-${attr.internal.org_id}-*

The rewrite is defence in depth: even if it were removed entirely, a tenant would see prefixes but still could not read another tenant's data, because the role grants nothing outside its own namespace.

There is one exception worth knowing. Index State Management actions are cluster-level, and cluster permissions are evaluated without consulting index privileges. For ISM specifically, the rewrite is the boundary rather than a second line behind it.

Names a tenant cannot escape through

A tenant naming another tenant's physical index does not reach it. org-globex-orders is rewritten to org-acme-org-globex-orders, which matches nothing. The mapping is idempotent only within a tenant's own namespace, which is what makes this safe.

Enabling it

Set the following in opensearch.yml on every node. All of them are off or empty by default, so a single-tenant cluster is unaffected by upgrading.

multitenancy.enabled: true

Setting	Default	Description
multitenancy.enabled	false	Turns namespacing on. With it off, nothing in this feature runs.
multitenancy.tenant_attribute	org_id	The user attribute holding the tenant id.
multitenancy.shared_indices	(empty)	Index globs every tenant addresses by their real name. See Shared indices.
multitenancy.break_glass.required	false	Requires an administrator to state a reason before touching tenant data. See Break glass.
plugins.security.service_account.allowed_cluster_permissions	(empty)	Cluster actions a service account may hold. Required if customers authenticate with PATs. See Personal access tokens.

Enabling namespacing is not enough on its own — a user is only namespaced if it carries the tenant attribute. Users without it, such as operators and administrators, are never namespaced and see physical index names.

Setting up a tenant

1. Define the shared tenant role

One role serves every tenant, because the pattern is substituted per user.

lucenia_shared_tenant:
  cluster_permissions:
    - cluster_composite_ops
    - "cluster:monitor/main"
    - "cluster:monitor/health"
  index_permissions:
    - index_patterns:
        - "org-${attr.internal.org_id}-*"
      allowed_actions:
        - indices_all

warning

Those three cluster permissions are doing a great deal of work. They are what refuses a tenant the entire administrative surface — cluster settings, cluster state, tasks, nodes, ingest and search pipelines, and index templates. Widen them only deliberately.

2. Create the user with its tenant attribute

Store the attribute as org_id. The internal authentication backend exposes it to role substitution prefixed with attr.internal., which is why the role above refers to ${attr.internal.org_id}.

PUT _plugins/_security/api/internalusers/org-acme
{
  "password": "...",
  "attributes": { "org_id": "acme" },
  "opendistro_security_roles": ["lucenia_shared_tenant"]
}

A tenant id must be lowercase alphanumeric. Ids containing - or * are rejected rather than sanitized, because either would change what the resulting pattern matches.

3. Verify

Index as the tenant and confirm both directions:

PUT orders/_doc/1
{ "customer": "acme" }

GET orders/_search

The response names orders. An administrator listing indices sees org-acme-orders.

Personal access tokens

If your customers authenticate with PATs — as they do on Lucenia Cloud — this section is not optional.

A PAT is issued to a service account, and service accounts are refused every cluster-level permission before their roles are consulted. Several operations customers consider ordinary are classified cluster-level, so without configuration a PAT user cannot use _mget, _msearch, or read cluster health, even though its role grants them.

plugins.security.service_account.allowed_cluster_permissions lifts that blanket refusal for named actions:

plugins.security.service_account.allowed_cluster_permissions:
  - "indices:data/read/mget"
  - "indices:data/read/msearch"
  - "cluster:monitor/health"
  - "cluster:monitor/main"
  - "indices:data/write/bulk*"

note

Listing an action here grants nothing. It only allows evaluation to continue to the same role check every other user receives, so a role must still grant the permission. Naming an action a role does not grant leaves it refused.

What a tenant can do

Ordinary index and search operations work unchanged, under logical names. Beyond those:

Index State Management

Tenants can manage their own ISM policies. Policy ids are namespaced like index names, so two tenants can each own a policy called hot-delete without colliding or seeing each other's.

Grant cluster:admin/lucenia/ism/* in the role, and allowlist it for service accounts if customers use PATs.

ism_template patterns inside a policy body are namespaced too, so a policy claiming * claims only that tenant's indices — including ones created later.

Snapshot and restore

Tenants can snapshot and restore their own indices, with snapshot names namespaced so two tenants can each take a daily.

Repositories are operator-owned. Tenants must not be granted cluster:admin/repository/*: registering a filesystem repository names a path, which would let a tenant read every other tenant's backups. Create the repository as an administrator and grant tenants only the snapshot lifecycle actions.

rename_replacement on a restore is namespaced, so a renamed restore lands inside the caller's own namespace.

Compliance configuration

Tenants can choose which of their own indices and fields are audited, through:

GET  _plugins/_security/api/compliance
PUT  _plugins/_security/api/compliance

PUT _plugins/_security/api/compliance
{
  "write_watched_indices": ["orders"],
  "read_watched_fields": { "orders": ["ssn"] }
}

Only those two keys are accepted. Cluster-wide settings — whether auditing runs at all, which users are ignored, whether REST is audited — are refused rather than silently dropped, so a customer cannot believe they have changed something that did not change.

A tenant's write replaces only its own entries; every other tenant's are preserved. The endpoint answers 409 if a concurrent change landed first, and the caller should re-read and re-apply.

warning

Each write triggers a cluster-wide configuration update, and every tenant contends on one document. Rate limit this at whatever fronts the API if you expose it to customers directly.

What a tenant cannot do

These are refused deliberately:

• Point-in-time search. A PIT id names shards directly and skips index resolution, so a PIT created by one tenant could be replayed by another. PIT is refused for namespaced users.
• Cross-cluster search. Refused by the role's cluster permissions, by the rewrite, and by the remote cluster's own authorization.
• Registering or altering snapshot repositories.
• The administrative surface — cluster settings and state, tasks, nodes, ingest and search pipelines, index templates.
• Reading system indices, including the security index, the ISM configuration index, and the ML plugin's indices.

Shared indices

Some indices are genuinely cluster-wide and cannot be namespaced. The audit log is the important one: it is a single index, and prefixing it would produce a name matching nothing.

multitenancy.shared_indices exempts index globs from namespacing so every tenant addresses them by their real name:

multitenancy.shared_indices:
  - "security-auditlog*"

warning

Anything listed here is outside the isolation boundary. Two tenants naming a shared index reach the same index, so whatever separates its contents must be document-level security in the role. Never add an ordinary data index.

To let tenants read their own audit events and no one else's, add a second index-permission block with a DLS clause:

  index_permissions:
    - index_patterns:
        - "org-${attr.internal.org_id}-*"
      allowed_actions:
        - indices_all
    - index_patterns:
        - "security-auditlog*"
      dls: '{"prefix":{"audit_trace_resolved_indices.keyword":"org-${attr.internal.org_id}-"}}'
      allowed_actions:
        - read

The audit event records physical index names, because it is written after the request has been rewritten — which is what makes the tenant's own prefix a usable discriminator.

note

Use the .keyword subfield. audit_trace_resolved_indices is analyzed, so a prefix query against the bare field matches tokens (org, acme, orders) rather than the whole name, and silently returns nothing.

A tenant reading its compliance history sees its own prefix in the audit document's _source. That field is customer content and is never rewritten. It is the tenant's own prefix and never another's.

Bounding the audit log

Auditing writes a daily index and nothing removes the old ones. On a shared cluster carrying every tenant's traffic, this is the fastest-growing thing on disk.

Install a retention policy as an administrator, so that the template is not namespaced and claims the real audit indices:

PUT _plugins/_ism/policies/audit-retention
{
  "policy": {
    "description": "bound the audit log",
    "default_state": "hot",
    "states": [
      { "name": "hot", "actions": [], "transitions": [{ "state_name": "delete", "conditions": { "min_index_age": "30d" } }] },
      { "name": "delete", "actions": [{ "delete": {} }], "transitions": [] }
    ],
    "ism_template": [{ "index_patterns": ["security-auditlog-*"], "priority": 50 }]
  }
}

Choose the retention period to match your compliance obligations; audit evidence periods commonly run from three to twelve months.

note

The same policy written by a tenant would be namespaced to org-acme-security-auditlog-* and claim nothing. That asymmetry is deliberate: a tenant cannot bound, or unbound, the cluster's audit log.

ISM refuses two templates that conflict at the same priority, and that check is cluster-wide — template priority is a shared numeric space.

Operator and administrator access

An administrator carries no tenant attribute, is never namespaced, and addresses indices by their physical names. This is required: per-tenant attribution for autoscaling and billing depends on seeing org-acme-orders and org-globex-orders as distinct, and an operator who cannot reach the data cannot repair it.

Cluster health, node stats, allocation, and per-index sizes all remain available to non-tenant users.

Break glass

By default, any administrator can read and write any tenant's data at any time, and the only record is an ordinary audit entry. Where privileged access to customer data must be a deliberate, reviewable act, enable:

multitenancy.break_glass.required: true

An administrator touching tenant data must then supply a reason:

GET org-acme-orders/_search
X-Lucenia-Break-Glass: INC-4711 customer reported missing documents

Without the header the request is refused. With it, the access is logged separately from ordinary audit traffic, so the acts can be counted and reviewed.

The reason is required per request, not per session, so break glass cannot become a mode somebody enables and forgets. Operating the cluster — health, node stats, and anything else that touches no tenant index — never requires one, and a tenant reading its own data is never affected.

note

This is not a time-boxed elevation and not an approval workflow. A cluster cannot distinguish an incident from an ordinary day, so an expiry or a second approver belongs in whatever issues the credentials.

Limitations

• Logical, not physical, isolation. Tenants share nodes and resources.
• Point-in-time and cross-cluster search are unavailable to tenants.
• A tenant sees its own prefix inside audit _source, because _source is customer content that is never rewritten.
• Aggregation bucket keys are treated as customer data and are returned unchanged, except when the request explicitly names the _index field. A terms aggregation over _index shows logical names; other aggregations return the values a customer stored, prefix or not.
• Compliance writes are cluster-wide configuration updates and are not rate limited.
• Shared indices sit outside the boundary and depend entirely on document-level security in the role.