Skip to content

Register a custom provider

The provider registry is open for extension. You can add a provider from your own package — a house LLM gateway, a mock for tests, or a niche vendor — without modifying the core. This is the same mechanism the official chat-anthropic, chat-openai, and chat-gemini modules use.

Implement a ProviderFactory

A provider is a factory function matching chat.ProviderFactory:

type ProviderFactory func(ctx context.Context, settings chat.Settings) (chat.ChatClient, error)

The factory receives the fully-resolved Settings (its Config and a non-nil *slog.Logger) and returns anything implementing chat.ChatClient — the five methods Add, Ask, SetTools, Chat, and Usage. Embed chat.UsageTracker to satisfy Usage() and get per-round-trip accounting for free.

package mybackend

import (
    "context"

    "gitlab.com/phpboyscout/go/chat"
)

type client struct {
    chat.UsageTracker // provides Usage(); call RecordUsage per round-trip

    cfg chat.Config
    log *slog.Logger
}

func newMyBackend(ctx context.Context, settings chat.Settings) (chat.ChatClient, error) {
    // Resolve the API key through the shared five-step cascade, passing your
    // own well-known fallback env var.
    key := chat.ResolveAPIKey(ctx, settings.Config.Token, settings.Config.Credentials, "MYBACKEND_API_KEY")

    return &client{cfg: settings.Config, log: settings.Logger}, nil
}

// ... implement Add / Ask / SetTools / Chat on *client ...

Register it from init()

Register the factory under a Provider name in an init() function, so a consumer activates your provider with a blank import — the same pattern as database drivers:

func init() {
    chat.RegisterProvider(chat.Provider("my-backend"), newMyBackend)
}

A consumer then does:

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "example.com/mybackend" // registers "my-backend"
)

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.Provider("my-backend"),
}})

chat.New validates the BaseURL (if any), looks your factory up in the registry, calls it, and audit-logs the endpoint host — all before returning the client. An unregistered name yields unsupported provider: my-backend.

Support cross-provider fallback

If your provider surfaces HTTP errors and you want it to participate correctly in a fallback composite, also register an HTTPStatusExtractor so the failover policy can classify your errors without importing your SDK. Pull the status code out of your error type and return it:

func init() {
    chat.RegisterProvider(chat.Provider("my-backend"), newMyBackend)
    chat.RegisterStatusExtractor(myBackendHTTPStatus)
}

// myBackendHTTPStatus reports the HTTP status carried by one of this provider's
// errors, unwrapping any cockroachdb/errors layers. ok is false when the error
// is not this provider's status-bearing type.
func myBackendHTTPStatus(err error) (status int, ok bool) {
    var apiErr *MyBackendError
    if errors.As(err, &apiErr) {
        return apiErr.StatusCode, true
    }
    return 0, false
}

The default failover policy then treats your 408/429/5xx as retryable and your 4xx as fatal, exactly like the built-in providers. A nil extractor is ignored; registering none simply means your errors are classified by the generic network/timeout rules only.