Skip to content

Fail over across providers

A single provider can rate-limit (HTTP 429), suffer an outage (5xx), or become unreachable. A fallback composite wraps an ordered list of clients and, on a retryable failure from the active one, transparently advances to the next. The composite is itself a ChatClient (and a StreamingChatClient iff every wrapped client streams), so callers are unchanged.

Build a composite

Three constructors, from lowest-level to highest:

// 1. Wrap already-constructed clients (first is primary, rest are fallbacks):
client, err := chat.NewFallback([]chat.ChatClient{primary, secondary})

// 2. Build from per-provider Settings — each provider self-resolves its own
//    credentials and model. The first Settings value is the primary.
client, err := chat.NewFallbackFromSettings(ctx, []chat.Settings{
    {Config: chat.Config{Provider: chat.ProviderClaude, Model: "claude-opus-4-8"}},
    {Config: chat.Config{Provider: chat.ProviderOpenAI}}, // OpenAI's default model
    {Config: chat.Config{Provider: chat.ProviderGemini}},
})

// 3. Build from Config values only (convenience over option 2):
client, err := chat.NewFallbackFromConfigs(ctx, []chat.Config{
    {Provider: chat.ProviderClaude, Model: "claude-opus-4-8"},
    {Provider: chat.ProviderOpenAI},
    {Provider: chat.ProviderGemini},
})

Activate every provider you list with its blank import (chat-anthropic, chat-openai, chat-gemini). When constructing from settings/configs, a non-primary provider that fails to construct (e.g. a missing credential) is dropped with a WARN — endpoint host only — so one missing fallback credential does not break the whole client. The primary's construction failure is fatal.

Options

Pass FallbackOption values to any constructor:

Option Effect
WithFailoverPolicy(p) Override the error-classification policy (default DefaultFailoverPolicy).
WithStrictToolContext() Fail fast instead of replaying a lossy text-only transcript once a tool call has executed.
WithOnFailover(fn) Observability hook func(from, to Provider), invoked on each transition (after the WARN log).
WithFallbackLogger(log) Logger for the one WARN line per transition (default: discard).

Which errors trigger failover

DefaultFailoverPolicy advances on transient/unavailable conditions and treats operator-fixable faults as fatal so they surface instead of being masked:

Advance (FailoverNext) Fatal (FailoverFatal)
HTTP 408, 429, 500, 502, 503, 504 HTTP 400, 401, 403, 404, 422
network errors (DNS, connection refused/reset, TLS) caller-cancelled context
a per-request timeout (the call's own deadline) claude-local non-zero exit (operator-fixable)

The policy classifies a provider's HTTP status through a status-extractor registry: each provider module registers an HTTPStatusExtractor in its init() (via RegisterStatusExtractor), so the core can read a status code out of a wrapped SDK error without importing any vendor SDK. The policy never inspects error messages. To register your own extractor for a custom provider, see Register a custom provider.

Behaviour across a failover boundary — limitations

  • Lossy transcript replay. The composite keeps a provider-neutral transcript of the user turns and replays them into a fallback provider on first use. Assistant turns and tool-call/tool-result interleaving cannot be reconstructed through the ChatClient interface, so a conversation that did heavy tool use before failover resumes with reduced context. Pass WithStrictToolContext to fail fast once a tool has executed instead.
  • Tools re-apply cleanly — handlers are provider-agnostic and are re-installed onto whichever provider becomes active.
  • Streaming fails over only before the first externally-visible event (EventTextDelta / EventToolCallStart) reaches your callback. Once a delta has been emitted it cannot be un-emitted, so a later error is terminal.
  • Usage is the sum across every provider the composite drove, so a failover's combined spend is visible from Usage().
  • Per-provider model. On the config/settings-driven paths each provider uses its own default model; a single global model name is not applied across Claude/OpenAI/Gemini. To pin a model per provider, set each Config.Model explicitly (options 2 and 3 above).

Logging & redaction

Each transition logs exactly one WARN line — chat provider failover with from/to (provider enum names) and a coarse reason (status or network). The triggering error's message is never logged verbatim, and any endpoint detail is reduced to the host only. See Provider-endpoint & credential security.