Core Concepts

Learn the fundamental concepts that power Octopus API Gateway.

Overview01

Octopus is built on several key concepts that work together to provide a powerful, flexible API gateway:

Request Flow02

Understanding how a request flows through Octopus. The live chain today is compression (optional) and the auth gateway (optional); see Request Lifecycle for the source-verified path and Middleware for which middleware are wired.

graph TD
    A[Client Request] --> B[Listener / TLS]
    B --> C[Protocol Detection]
    C --> D[Route Pre-match + Auth Context]
    D --> E[Middleware Chain]
    E --> F{Auth Gateway}
    F -->|Pass| G[Route Match + Prefix Rewrite]
    F -->|Fail| H[401 / 403 Response]
    G --> I[Load Balancer: select instance]
    I --> L[Upstream Service]
    L --> O[Response]
    O --> P[Compression on the way out]
    P --> Q[Client Response]

Key Components03

1. Server

The HTTP server that accepts incoming connections:

• Built on Tokio and Hyper

• Supports HTTP/1.1, HTTP/2, and HTTP/3 (experimental)

• Connection pooling and keep-alive

• Graceful shutdown

2. Router

Matches incoming requests to routes:

• Trie-based path matching for O(k) lookup time

• Supports path parameters (:id) and wildcards (*)

• Method-based routing

• Priority-based matching

3. Middleware

An ordered Rust chain that wraps the request/response path. The live chain today holds at most two entries — compression and the auth gateway — assembled in code (there is no middleware: config array). A larger catalogue of middleware exists in the octopus-middleware crate but is builder-only. See Middleware.

4. Service Discovery

Finds and tracks backend services:

• Discovery backends: Kubernetes (EndpointSlice), Consul, DNS, mDNS

• Static configuration via upstreams

• FARP protocol for schema-driven auto-routing

See Service Discovery and FARP.

5. Load Balancer

Selects an instance per request. Policies: round-robin, least-connections, weighted, random, and IP-hash. The policy and instance weights are honored on the Kubernetes/operator path; static-config upstreams currently balance round-robin regardless of lb_policy. See Load Balancing.

6. Health Tracker

Monitors upstream health with active checks (HTTP, TCP, gRPC). Active health checking and circuit breaking are wired on the operator-driven path; on the static-config path they are parsed and validated but not yet enforced. See Health Checks and Circuit Breaker.

7. Plugin System

A plugin SDK plus lifecycle runtime and a dynamic shared-library loader:

• Static plugins (compiled in) and dynamic plugins (.so/.dylib/.dll)

• Type-safe Rust trait system with lifecycle hooks

• An embedded Rhai scripting engine

The SDK and loader are implemented, but the plugins: config array is not yet consumed at request time. See Plugins and Plugins & Scripting.

Design Principles04

1. Performance First

• Zero-copy proxying where possible

• Async I/O with Tokio

• Connection pooling

• Minimal allocations

• Lock-free data structures

2. Stateless by Default

• Gateway instances are stateless

• Easy horizontal scaling

• Optional state via plugins (Redis, etc.)

3. Type Safety

• Leverages Rust's type system

• Compile-time guarantees

• No null pointer exceptions

• Memory safety without garbage collection

4. Extensibility

• Plugin system for custom logic

• Middleware pipeline for request processing

• Protocol handlers for custom protocols

• Admin dashboard extensions

5. Observability

• Prometheus metrics

• OpenTelemetry tracing

• Structured logging

• Built-in from day one

6. Production Ready

• Health checks

• Circuit breakers

• Graceful shutdown

• Error handling

• Timeouts and retries

Configuration Model05

Octopus uses a declarative configuration model:

# What you configure
server:
  http_port: 8080

routes:
  - path: /api/*
    upstream: my-service

# Octopus handles
- Connection management
- Request routing
- Load balancing
- Health checking
- Error handling
- Metrics collection

Next Steps06

Dive deeper into specific concepts:

• Architecture - System design and components

• Request Lifecycle - The path of a request

• Routing - How routing works

• Middleware - Request processing chain

• Plugins - Extending Octopus

• FARP Protocol - Automatic service discovery

Or explore practical guides:

• Configuration Guide - Configure Octopus

• Middleware Guide - Use middleware

• Writing Plugins - Create plugins