Fail over across providers¶

A single provider can rate-limit (HTTP 429), suffer an outage (5xx), or become unreachable. A fallback composite wraps an ordered list of clients and, on a retryable failure from the active one, transparently advances to the next. The composite is itself a ChatClient, so callers are unchanged. It is also a StreamingChatClient, a LogprobsCapable or a ToolCallProposer when every wrapped client is. The last two are one-shot calls: they fail over like Chat, but a replay after a later failover never includes them.

Build a composite¶

Three constructors, from lowest-level to highest:

// 1. Wrap already-constructed clients (first is primary, rest are fallbacks):
client, err := chat.NewFallback([]chat.ChatClient{primary, secondary})

// 2. Build from per-provider Settings; each provider self-resolves its own
//    credentials and model. The first Settings value is the primary.
client, err := chat.NewFallbackFromSettings(ctx, []chat.Settings{
    {Config: chat.Config{Provider: chat.ProviderClaude, Model: "claude-opus-5"}},
    {Config: chat.Config{Provider: chat.ProviderOpenAI}}, // OpenAI's default model
    {Config: chat.Config{Provider: chat.ProviderGemini}},
})

// 3. Build from Config values only (convenience over option 2):
client, err := chat.NewFallbackFromConfigs(ctx, []chat.Config{
    {Provider: chat.ProviderClaude, Model: "claude-opus-5"},
    {Provider: chat.ProviderOpenAI},
    {Provider: chat.ProviderGemini},
})

Activate every provider you list with its blank import (chat-anthropic, chat-openai, chat-gemini). When constructing from settings/configs, a non-primary provider that fails to construct (e.g. a missing credential) is dropped with a WARN (endpoint host only) so one missing fallback credential does not break the whole client. The primary's construction failure is fatal.

Credentials in a chain¶

A chain built from one Config (NewWithFallbackSettings) gives each member only the settings that mean the same thing to every provider. A token for one vendor is meaningless to another, so credentials do not travel: each member resolves its own.

The provider named in Config.Provider is the exception, and keeps the Token, Credentials, Model and BaseURL you supplied for it.

When each member needs its own key from somewhere only you can reach, supply a resolver:

client, err := chat.NewWithFallbackSettings(ctx, settings, fallbackCfg,
    chat.WithProviderCredentials(func(p chat.Provider) (chat.CredentialConfig, bool) {
        key, ok := vault[p]

        return chat.CredentialConfig{Key: key}, ok
    }),
)

Returning false leaves that provider to its own credential lookup.

Options¶

Pass FallbackOption values to any constructor:

Which errors trigger failover¶

DefaultFailoverPolicy advances on transient/unavailable conditions and treats operator-fixable faults as fatal so they surface instead of being masked:

The policy classifies a provider's HTTP status through a status-extractor registry: each provider module registers an HTTPStatusExtractor in its init() (via RegisterStatusExtractor), so the core can read a status code out of a wrapped SDK error without importing any vendor SDK. The policy never inspects error messages. To register your own extractor for a custom provider, see Register a custom provider.

Behaviour across a failover boundary: limitations¶

• Lossy transcript replay. The composite keeps a provider-neutral transcript of the user turns and replays them into a fallback provider on first use. Assistant turns and tool-call/tool-result interleaving cannot be reconstructed through the ChatClient interface, so a conversation that did heavy tool use before failover resumes with reduced context. Pass WithStrictToolContext to fail fast once a tool has executed instead.
• Stateless composites replay nothing, and so lose nothing: there is no cross-call context to carry. NewFallbackFromSettings and NewFallbackFromConfigs infer this from Config.Stateless. NewFallback is handed already-built clients and cannot see their config, so pass WithStateless() there, or the composite accumulates a transcript nobody will use and prepends it to the next one-shot call on failover.
• Tools re-apply cleanly, because handlers are provider-agnostic and are re-installed onto whichever provider becomes active.
• Streaming fails over only before the first externally-visible event (EventTextDelta / EventToolCallStart) reaches your callback. Once a delta has been emitted it cannot be un-emitted, so a later error is terminal.
• Usage is the sum across every provider the composite drove, so a failover's combined spend is visible from Usage().
• Per-provider model. On the config/settings-driven paths each provider uses its own default model; a single global model name is not applied across Claude/OpenAI/Gemini. To pin a model per provider, set each Config.Model explicitly (options 2 and 3 above).

Concurrency¶

From chat v0.29.0, a fallback composite is safe for concurrent use when every member is: stateless clients, and conversational clients from the provider versions listed under Concurrent use. One composite can serve a pool of workers (spec 0027):

• A burst of failures fails over once. Calls that fail together on one provider move the composite to the next provider, not one provider per call.
• A fallback provider is caught up before any call reaches it: the current tools, then every user turn it hasn't seen, in order. A catch-up that fails part-way resumes where it stopped on the next call.
• A call still running on the old provider isn't lost. If it succeeds after the failover, its prompt reaches the new provider before that provider's next call. Its reply stays behind, as every reply does on failover.
• The failover hook fires once per transition, from whichever call failed over.

The composite doesn't serialise calls itself. Each member already serves its calls in order, and combines the ones queued on it. A composite over a member that isn't safe for concurrent use, such as claude-local, is no safer than that member.

Logging & redaction¶

Each transition logs exactly one WARN line, chat provider failover, with from/to (provider enum names) and a coarse reason (status or network). The triggering error's message is never logged verbatim, and any endpoint detail is reduced to the host only. See Provider-endpoint & credential security.

Related¶

• Choose & configure a provider: per-provider setup.
• Providers: the capability matrix.
• Process a batch of documents: sharing one client across a worker pool.
• pkg.go.dev reference.