The opencompat provider is a generic wrapper for any API that follows the OpenAI specification. Use this when a dedicated provider package is not available, or when connecting to custom/self-hosted endpoints like vLLM, text-generation-inference, or LocalAI.
Installation01
import "github.com/xraph/nexus/providers/opencompat"Quick Start02
provider := opencompat.New("my-provider",
"https://my-api.example.com/v1",
os.Getenv("MY_API_KEY"),
)
gw := nexus.New(
nexus.WithProvider(provider),
)Options03
| Option | Description |
opencompat.WithCapabilities(caps) | Override the default capabilities |
opencompat.WithModels(models) | Provide a list of available models |
Capabilities04
Default capabilities are Chat and Streaming. Override with WithCapabilities:
provider := opencompat.New("my-provider", baseURL, apiKey,
opencompat.WithCapabilities(provider.Capabilities{
Chat: true,
Streaming: true,
Embeddings: true,
}),
)Custom Models05
Since opencompat has no built-in model catalog, provide models explicitly:
provider := opencompat.New("vllm", baseURL, apiKey,
opencompat.WithModels([]provider.Model{
{
ID: "meta-llama/Llama-3.1-8B-Instruct",
Name: "Llama 3.1 8B",
Provider: "vllm",
ContextWindow: 131072,
},
}),
)