Scaling

Octopus scales in two directions: up (more worker threads per process) and out (more replicas behind a load balancer). They're complementary — tune workers to use a node's cores, then add replicas for capacity and availability.

Vertical: gateway.workers01

Each Octopus process runs an async runtime with a pool of worker threads. gateway.workers controls the size of that pool:

gateway:
  listen: "0.0.0.0:8080"
  workers: 0          # 0 = auto: one worker per available CPU core

• 0 (default) — auto: the runtime uses one worker per CPU core it can see.

• A fixed number — pin the pool to exactly that many threads.

The Helm chart and bundled manifests ship workers: 0. See Gateway configuration for the field reference.

Horizontal: replicas02

Octopus instances are interchangeable — run several behind a load balancer (a Kubernetes Service, an external LB, or another ingress) and spread traffic across them. This is also how you get availability: a single replica is a single point of failure.

In Kubernetes, set the replica count on the Deployment. The Helm chart defaults to 2:

helm install octopus deploy/helm/octopus -n octopus --create-namespace \
  --set replicaCount=3

For availability across nodes and zones, the chart also exposes topologySpreadConstraints and a PodDisruptionBudget (enabled by default, minAvailable: 1). See the Helm chart reference.

Autoscaling (HPA)03

The Helm chart can render a HorizontalPodAutoscaler. It's off by default; enable it and the chart stops setting a static replicaCount (the HPA owns the replica count instead):

helm install octopus deploy/helm/octopus -n octopus --create-namespace \
  --set autoscaling.enabled=true \
  --set autoscaling.minReplicas=2 \
  --set autoscaling.maxReplicas=10 \
  --set autoscaling.targetCPUUtilizationPercentage=80

The chart's HPA scales on CPU utilization by default and optionally on memory (autoscaling.targetMemoryUtilizationPercentage). The HPA needs CPU/memory resource requests to compute utilization against — the chart sets those by default. Full values are in the Helm chart reference.

Shared state across replicas04

Once you run more than one replica, any feature that remembers things between requests must share that state — otherwise each replica counts and caches independently. The features that need it include:

• Rate limiting — counters must be shared so a limit means the same thing fleet-wide.

• Caching — to avoid each replica caching the same upstream response separately.

• Circuit breakers — shared failure counts so the breaker trips consistently.

• Sessions / auth caches — shared so a session isn't tied to one replica.

Octopus keeps this in a pluggable state store. The default in-memory backend is per-process and is fine for a single instance; for multiple replicas use a shared backend (Redis, Postgres, or hybrid).

See State for the backends and how to select one.

Related05

• Gateway configuration — workers and the rest of the gateway block.

• Helm chart — replicaCount, autoscaling, topologySpreadConstraints, PDB.

• State — shared state backends for multi-replica deployments.

• Graceful shutdown — draining cleanly as replicas come and go.