Nexus

#What it is

Nexus is an AI gateway you compile into your own binary. Route across providers, cache, guard and observe LLM traffic without running a separate SaaS in the path.

#What it does

• Multi-provider routing to OpenAI, Anthropic or any OpenAI-compatible API, with priority, cost-optimised, round-robin and latency-based strategies.
• Guardrails: PII detection, prompt injection blocking and content filtering, each with block, redact or warn actions.
• Caching with deterministic keys and optional semantic matching, over memory or Redis.
• Multi-tenant isolation: scoped requests, API keys, rate limits, usage tracking, budget enforcement and per-tenant model aliases.
• Model aliases mapping a virtual name to a provider target, with per-tenant overrides and weighted routing, so "fast" can mean something different for each customer.
• Transforms: system prompt injection, RAG context augmentation, anonymisation and output normalisation.
• Streaming: reasoning deltas, tool-call streaming, multi-modal chunks, and four wire formats: OpenAI-envelope SSE, nexus-native SSE with named events, NDJSON, and a bidirectional WebSocket at /v1/realtime. Record-and-replay caching is idempotent across all of them.
• OpenAI-compatible proxy, so existing SDKs point at your gateway unchanged.

#Why not a hosted gateway

Every hosted AI gateway puts a third party in the path of your prompts, which is a data-governance conversation before it is an architecture one. Nexus is fifteen lifecycle hooks and three storage backends in your own process. The trade is that you operate it, and you already operate the process it lives in.