Deciders

A decider looks at one dimension of a tier and says how much capacity it needs. The controller runs every decider that applies to a policy's roles, and the most demanding one wins per resource dimension — scale-up beats scale-down beats steady, at equal priority.

A decider that has nothing to say abstains rather than voting for "steady". That distinction matters: if every decider abstains, the result is explicitly "all deciders abstained", not a confident recommendation of no change.

Point-in-time by design

Every decider evaluates a snapshot. There are no history buffers and no rolling windows inside the controller.

That is a deliberate division of labour. Sustained-window judgements — "has this been hot for ten minutes", "has this index been idle for 24 hours" — belong to whatever polls the controller across its cycles, because that component already has the time axis. Keeping deciders stateless makes them testable with a fixture instead of a clock, and means two controllers evaluating the same cluster reach the same answer.

The deciders

Storage

Combines reactive and proactive storage evaluation in one decider, rather than splitting them.

Setting	Default	Meaning
high_watermark	85	Percent used at which more storage is needed.
headroom_percent	20	Headroom kept above current usage when sizing.

Applies to: data.

Heap pressure

Setting	Default	Meaning
high_watermark	0.75	Fraction of heap held after GC that counts as pressure.
critical_watermark	0.90	Fraction that counts as critical.

critical_watermark is 0.90 because it is close enough to the parent circuit breaker — 0.95 of heap by default — that waiting any longer means the cluster starts rejecting requests before the controller reacts. The watermarks read what was still held after the last GC, which is the number that indicates real occupancy rather than garbage yet to be collected.

Applies to: data.

Search load

Setting	Default	Meaning
queue_threshold	100	Search queue depth that indicates pressure.
idle_threshold	0.3	Utilisation below which the tier is considered idle.
inference_queue_threshold	8	Queue depth for the inference pool, when present.

inference_queue_threshold is deliberately not queue_threshold's 100: the two pools are not the same size. Local inference runs on its own small pool with a queue of 16, so a threshold of 100 could never fire on it. Half the queue is the point at which waiting has become the normal case rather than a burst.

Applies to: data, search.

Ingest pressure

Setting	Default	Meaning
queue_threshold	200	Bulk queue depth that indicates pressure.
idle_threshold	0.2	Utilisation below which the tier is considered idle.

Indexing-pressure rejections short-circuit the evaluation: any rejection at all is a scale-up signal, because a rejection is a request that already failed, not a risk that one might.

The bulk queue threshold is twice the search one, and the idle threshold lower, because bulk work is batched — a deeper queue is normal, and the same depth means less about latency.

Applies to: data, ingest, write, bulk.

Shard balance

   min_nodes = ceil(total_shards / max_shards_per_node) + node_buffer

Setting	Default	Meaning
max_shards_per_node	cluster setting	Placement ceiling per node.
node_buffer	1	Spare nodes above the computed minimum.

Uses data already in the cluster view — no extra stats calls. node_buffer defaults to 1 so that losing a single node does not immediately make the cluster unplaceable.

max_shards_per_node has no default of its own. Left unset, the decider reads the cluster's cluster.max_shards_per_node — the same per-node cap the shard-limit validator enforces — so autoscaling adds nodes before the cluster starts refusing shard creation, rather than tracking a second number that can drift from it. Setting it here overrides that, for this policy only.

Applies to: data.

Idle index

The scale-down decider. Finds indices with no search and no indexing activity and recommends reducing replicas.

Setting	Default	Meaning
min_replicas	0	Floor below which replicas are never reduced.
protected_pattern	.lucenia-*,system-*	Comma-separated index name patterns never reduced.

protected_pattern is the safety valve. An index can look idle and still be one you must not touch — a disaster-recovery replica, a compliance archive. Idleness is evidence about traffic, not about importance.

Applies to: data.

Cluster-manager capacity

Sizes the cluster-manager tier from what it has to track, using ratios drawn from operational norms.

Setting	Default	Meaning
heap_per_indices	3000	Indices one GB of manager heap manages comfortably.
heap_per_shards	30000	Shards per GB (≈15K per 8 GB heap).
heap_per_nodes	30	Nodes per GB (≈30 per 8 GB heap).
heap_high_watermark	85	Percent of manager heap treated as full.
min_heap_gb	1	Floor.
max_heap_gb	31	Ceiling.
scale_down_buffer_percent	25	Hysteresis before scaling the tier down.

max_heap_gb is 31 rather than a round number because above roughly 32 GB the JVM loses compressed ordinary object pointers, and the usable heap actually shrinks. scale_down_buffer_percent exists so the tier does not oscillate around a threshold.

Applies to: cluster_manager.

Setting them

A policy is a document in the hidden index .octane-autoscale-policies, one per tier. Decider settings live under the decider's name:

PUT .octane-autoscale-policies/_doc/hot
{
  "name": "hot",
  "roles": ["data"],
  "min_nodes": 3,
  "max_nodes": 30,
  "scale_down_enabled": true,
  "deciders": {
    "storage": { "high_watermark": 80, "headroom_percent": 25 },
    "heap_pressure": { "high_watermark": 0.70 }
  }
}

Field	Default	Meaning
name	—	Policy name.
roles	—	Node roles this policy covers. A decider runs only if it applies to one of them.
min_nodes	1	Floor. The controller will not scale below it.
max_nodes	unbounded	Ceiling.
scale_down_enabled	true	Set false to let a tier grow but never shrink.
deciders	—	Per-decider settings, keyed by decider name.

A decider with no entry here uses every default above. Writes go through a sequence-number compare-and-set, so two controllers editing the same policy cannot silently overwrite each other.

The index is hidden on purpose: it is the controller's configuration, not user data, and it should not turn up in a wildcard search or an index listing someone is scanning for their own data.