Fail over across providers¶

A single provider can rate-limit (HTTP 429), suffer an outage (5xx), or become unreachable. A fallback composite wraps an ordered list of clients and, on a retryable failure from the active one, transparently advances to the next. The composite is itself a ChatClient (and a StreamingChatClient iff every wrapped client streams), so callers are unchanged.

Build a composite¶

Three constructors, from lowest-level to highest:

// 1. Wrap already-constructed clients (first is primary, rest are fallbacks):
client, err := chat.NewFallback([]chat.ChatClient{primary, secondary})

// 2. Build from per-provider Settings; each provider self-resolves its own
//    credentials and model. The first Settings value is the primary.
client, err := chat.NewFallbackFromSettings(ctx, []chat.Settings{
    {Config: chat.Config{Provider: chat.ProviderClaude, Model: "claude-opus-5"}},
    {Config: chat.Config{Provider: chat.ProviderOpenAI}}, // OpenAI's default model
    {Config: chat.Config{Provider: chat.ProviderGemini}},
})

// 3. Build from Config values only (convenience over option 2):
client, err := chat.NewFallbackFromConfigs(ctx, []chat.Config{
    {Provider: chat.ProviderClaude, Model: "claude-opus-5"},
    {Provider: chat.ProviderOpenAI},
    {Provider: chat.ProviderGemini},
})

Activate every provider you list with its blank import (chat-anthropic, chat-openai, chat-gemini). When constructing from settings/configs, a non-primary provider that fails to construct (e.g. a missing credential) is dropped with a WARN (endpoint host only) so one missing fallback credential does not break the whole client. The primary's construction failure is fatal.

Credentials in a chain¶

A chain built from one Config (NewWithFallbackSettings) gives each member only the settings that mean the same thing to every provider. A token for one vendor is meaningless to another, so credentials do not travel: each member resolves its own.

The provider named in Config.Provider is the exception, and keeps the Token, Credentials, Model and BaseURL you supplied for it.

When each member needs its own key from somewhere only you can reach, supply a resolver:

client, err := chat.NewWithFallbackSettings(ctx, settings, fallbackCfg,
    chat.WithProviderCredentials(func(p chat.Provider) (chat.CredentialConfig, bool) {
        key, ok := vault[p]

        return chat.CredentialConfig{Key: key}, ok
    }),
)

Returning false leaves that provider to its own credential lookup.

Options¶

Pass FallbackOption values to any constructor:

Which errors trigger failover¶

DefaultFailoverPolicy advances on transient/unavailable conditions and treats operator-fixable faults as fatal so they surface instead of being masked:

The policy classifies a provider's HTTP status through a status-extractor registry: each provider module registers an HTTPStatusExtractor in its init() (via RegisterStatusExtractor), so the core can read a status code out of a wrapped SDK error without importing any vendor SDK. The policy never inspects error messages. To register your own extractor for a custom provider, see Register a custom provider.

Behaviour across a failover boundary: limitations¶

• Lossy transcript replay. The composite keeps a provider-neutral transcript of the user turns and replays them into a fallback provider on first use. Assistant turns and tool-call/tool-result interleaving cannot be reconstructed through the ChatClient interface, so a conversation that did heavy tool use before failover resumes with reduced context. Pass WithStrictToolContext to fail fast once a tool has executed instead.
• Stateless composites replay nothing, and so lose nothing: there is no cross-call context to carry. NewFallbackFromSettings and NewFallbackFromConfigs infer this from Config.Stateless. NewFallback is handed already-built clients and cannot see their config, so pass WithStateless() there, or the composite accumulates a transcript nobody will use and prepends it to the next one-shot call on failover.
• Tools re-apply cleanly, because handlers are provider-agnostic and are re-installed onto whichever provider becomes active.
• Streaming fails over only before the first externally-visible event (EventTextDelta / EventToolCallStart) reaches your callback. Once a delta has been emitted it cannot be un-emitted, so a later error is terminal.
• Usage is the sum across every provider the composite drove, so a failover's combined spend is visible from Usage().
• Per-provider model. On the config/settings-driven paths each provider uses its own default model; a single global model name is not applied across Claude/OpenAI/Gemini. To pin a model per provider, set each Config.Model explicitly (options 2 and 3 above).

Concurrency¶

A fallback composite is never safe for concurrent use. Including when the underlying clients are stateless and individually are. The composite advances its active provider mid-call during a failover, so concurrent calls would disagree about which provider is live. Guarding that with a lock would serialise every call through the composite, which costs more than it buys.

A caller wanting both fallback and a worker pool builds one composite per worker:

for range workers {
    composite, err := chat.NewFallbackFromConfigs(ctx, cfgs)
    if err != nil {
        return err
    }

    go consume(composite, docs) // each worker owns its own composite
}

Sharing a single stateless provider client across goroutines is fine; it is only the wrapper that is not. See Process a batch of documents.

Logging & redaction¶

Each transition logs exactly one WARN line, chat provider failover, with from/to (provider enum names) and a coarse reason (status or network). The triggering error's message is never logged verbatim, and any endpoint detail is reduced to the host only. See Provider-endpoint & credential security.

Related¶

• Choose & configure a provider: per-provider setup.
• Providers: the capability matrix.
• Process a batch of documents: stateless clients, and why the composite is not shareable.
• pkg.go.dev reference.