Skip to content

Register a custom provider

The provider registry is open for extension. You can add a provider from your own package (a house LLM gateway, a mock for tests, or a niche vendor) without modifying the core. This is the same mechanism the official chat-anthropic, chat-openai, and chat-gemini modules use.

Implement a ProviderFactory

A provider is a factory function matching chat.ProviderFactory:

type ProviderFactory func(ctx context.Context, settings chat.Settings) (chat.ChatClient, error)

The factory receives the fully-resolved Settings (its Config and a non-nil *slog.Logger) and returns anything implementing chat.ChatClient: the five methods Add, Ask, SetTools, Chat, and Usage. Embed chat.UsageTracker to satisfy Usage() and get per-round-trip accounting for free.

package mybackend

import (
    "context"

    "gitlab.com/phpboyscout/go/chat"
)

type client struct {
    chat.UsageTracker // provides Usage(); call RecordUsage per round-trip

    cfg chat.Config
    log *slog.Logger
}

func newMyBackend(ctx context.Context, settings chat.Settings) (chat.ChatClient, error) {
    // Resolve the API key through the shared five-step cascade, passing your
    // own well-known fallback env var.
    key := chat.ResolveAPIKey(ctx, settings.Config.Token, settings.Config.Credentials, "MYBACKEND_API_KEY")

    return &client{cfg: settings.Config, log: settings.Logger}, nil
}

// ... implement Add / Ask / SetTools / Chat on *client ...

Register it from init()

Register the factory under a Provider name in an init() function, so a consumer activates your provider with a blank import, the same pattern as database drivers:

func init() {
    chat.RegisterProvider(chat.Provider("my-backend"), newMyBackend)
}

A consumer then does:

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "example.com/mybackend" // registers "my-backend"
)

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.Provider("my-backend"),
}})

chat.New validates the BaseURL (if any), looks your factory up in the registry, calls it, and audit-logs the endpoint host, all before returning the client. An unregistered name yields unsupported provider: my-backend.

Support cross-provider fallback

If your provider surfaces HTTP errors and you want it to participate correctly in a fallback composite, also register an HTTPStatusExtractor so the failover policy can classify your errors without importing your SDK. Pull the status code out of your error type and return it:

func init() {
    chat.RegisterProvider(chat.Provider("my-backend"), newMyBackend)
    chat.RegisterStatusExtractor(myBackendHTTPStatus)
}

// myBackendHTTPStatus reports the HTTP status carried by one of this provider's
// errors, unwrapping any annotation layers. ok is false when the error
// is not this provider's status-bearing type.
func myBackendHTTPStatus(err error) (status int, ok bool) {
    var apiErr *MyBackendError
    if errors.As(err, &apiErr) {
        return apiErr.StatusCode, true
    }
    return 0, false
}

The default failover policy then treats your 408/429/5xx as retryable and your 4xx as fatal, exactly like the built-in providers. A nil extractor is ignored; registering none simply means your errors are classified by the generic network/timeout rules only.

Support stateless one-shot calls

Config.Stateless asks a client to send no prior turns and retain none, so one client can serve many independent calls (see Process a batch of documents). A provider opts in by implementing chat.StatelessCapable, a marker method that is never called:

// SupportsStateless marks this provider as honouring chat.Config.Stateless.
func (c *client) SupportsStateless() {}

Implement the marker only if you actually honour the flag. chat.New type-asserts it and refuses to build a stateless client from a provider that does not:

provider my-backend does not support Config.Stateless

That guard exists because the alternative is the worst outcome available: a flag that compiles, is ignored, and bills the caller for the history they asked not to send. Declaring the marker without honouring it defeats it.

What honouring it means:

  • Across calls, nothing carries. Neither seed the request from prior turns nor retain this call's turns.
  • Within a call, everything carries. The ReAct loop still needs the assistant tool-call turn and the tool result answering it, or it cannot make progress. Build the turn list per call and discard it on return; do not simply skip the appends.
  • Add buffers rather than persists. Buffered turns go out with the next call and are then cleared. Keep Add working, because a fallback composite replays context through it, so an erroring Add breaks substitutability.
  • Configuration is not conversation. The system prompt still goes on every call.
  • Restore returns chat.ErrStatelessRestore if you implement PersistentChatClient; Save returns a snapshot with no conversation.

Stateless clients are also expected to be safe for concurrent use. Guard your mutable state with a mutex and release it before the round-trip. A lock held across the request serialises the batch and gives back exactly what sharing the client was meant to buy. Snapshot the tool registry at call entry so a SetTools racing an in-flight call is seen whole or not at all.

Apply the caller's history policy

Config.HistoryPolicy bounds a conversation before it is sent, and the provider is what applies it. The core cannot: a transcript is opaque to it by design, so only you can enumerate your turns and rewrite them.

Run three steps before every request you send, including each round-trip inside a ReAct loop, not once per Chat:

// 1. Describe the turns you have retained. The pending prompt is not one of
//    them: the budget counts history, not what is about to be sent.
turns := make([]chat.TurnInfo, 0, len(c.messages))
for i, m := range c.messages {
    turns = append(turns, chat.TurnInfo{
        Index:     i,
        Role:      roleOf(m),
        Text:      textOf(m),
        Pinned:    c.cachedPrefix > i, // turns added via AddCached
        ToolGroup: c.toolGroupOf(m),   // a tool call and its result share one
    })
}

// 2. Bound them. A nil policy keeps everything, so this needs no guard.
keep, replace, err := chat.BoundConversation(ctx, c.cfg.HistoryPolicy, turns)
if err != nil {
    return err // the call fails; it does not silently send unbounded
}

// 3. Rewrite your own transcript from the indices returned.
bounded := make([]message, 0, len(keep))
for _, i := range keep {
    m := c.messages[i]
    if text, ok := replace[i]; ok {
        m = c.replaceWithText(m, text)
    }
    bounded = append(bounded, m)
}
c.messages = bounded

Three things are easy to get wrong, and the first two are checked for you:

  • Describe pinned turns, never hide them. Mark turns added via AddCached with Pinned. BoundConversation refuses an edit that drops one, because caching bills a matching prefix, so dropping a cached turn quietly invalidates every later cache hit.
  • Group a tool call with its result. Give both turns the same non-zero ToolGroup. An edit that keeps one without the other is refused, because sending half a pair earns an API error rather than a shorter conversation.
  • Do not skip step 2 and act on the policy's edit directly. The invariants live in BoundConversation precisely so that every provider inherits them, including for policies written after your provider was.

If your client cannot enumerate its turns at all (the conversation lives somewhere you cannot see) report History.Known == false. The conformance case skips rather than fails, which is how claude-local is handled.

The case that proves this works is history_policy_applied, and it observes what your backend received rather than what your client says it did. See Verify a provider against the conformance suite.