Choose & configure a provider¶

This guide covers the chat.Config fields, how each provider is activated and credentialed, and the two providers that need extra setup: openai-compatible (a required BaseURL and Model) and claude-local (a local CLI binary).

The fields that choose a provider¶

chat.Config carries provider behaviour and is passed inside chat.Settings to chat.New. Only Provider is effectively required, and even that defaults; everything else has a working zero value.

Five fields decide which provider you get and how it authenticates:

Every other field (sampling, effort, caching, timeouts, the tool loop, the test seams) is in Configuration fields, with what each one defaults to and what happens when the value cannot be applied. Signatures are on pkg.go.dev.

Activate a provider by blank import¶

Each API provider lives in its own module. Blank-import it once (typically in main) to register it with the core; then reference it by its Provider constant.

The openai and openai-compatible providers are both registered by the single chat-openai import.

Resolve a credential¶

Every API provider resolves its key through one shared five-step cascade (chat.ResolveAPIKey). The first non-empty source wins; each step is trimmed so a half-configured value cannot mask a fully-configured one below it:

1. Direct token. Config.Token.
2. Env-var reference. Config.Credentials.Env names an env var; the value is read with os.Getenv. The secret stays out of any config file.
3. Keychain reference. Config.Credentials.Keychain is a "service/account" pair resolved through the host-injected Config.Credentials.Lookup func. With no lookup wired, this step is skipped.
4. Literal. Config.Credentials.Key, an inline value (legacy; least preferred).
5. Well-known env var. ANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY, the ecosystem fallback used by most SDKs and CI platforms.

The simplest path is to export the well-known env var and leave Token and Credentials empty:

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-anthropic"
)

// ANTHROPIC_API_KEY is set in the environment; nothing else needed.
client, err := chat.New(ctx, chat.Settings{
    Config: chat.Config{Provider: chat.ProviderClaude},
})

A host that keeps secrets in a config file or keychain populates Config.Credentials instead. See credential security.

Using go-tool-base? Its adapter maps the framework's ai.provider, AI_PROVIDER, and provider api config sections onto these fields for you; that adapter lives in go-tool-base, not this module.

OpenAI-compatible endpoints¶

ProviderOpenAICompatible targets any backend that speaks the OpenAI API: Ollama, xAI, Fireworks AI, Together AI, LM Studio, vLLM, and others. It is registered by the chat-openai import. Both BaseURL and Model are required (there is no default model, because names are backend-specific), and the BaseURL must pass endpoint validation.

import (
    "os"

    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-openai"
)

// Ollama (local)
ollama, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderOpenAICompatible,
    BaseURL:  "http://localhost:11434/v1",
    Model:    "llama3.2",
    // No Token: Ollama takes none, and none is sent.

    // A localhost HTTP endpoint is non-HTTPS; permitted only in tests.
    AllowInsecureBaseURL: true,
}})

// xAI (cloud, HTTPS)
grok, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderOpenAICompatible,
    BaseURL:  "https://api.x.ai/v1",
    Model:    "grok-4.3",
    Token:    os.Getenv("XAI_API_KEY"),
}})

The key comes from Config.Token or Config.Credentials alone. With neither set, no key is sent, which suits a local server started without one. OPENAI_API_KEY is never read, nor any other OpenAI account setting the SDK would take from the environment (OPENAI_ADMIN_KEY, OPENAI_ORG_ID, OPENAI_PROJECT_ID, OPENAI_CUSTOM_HEADERS), because those belong to api.openai.com and not to the server you've named. This holds from chat-openai v0.17.3; before it, an unset token fell back to OPENAI_API_KEY and sent it to BaseURL, and an empty one was refused.

Both of these have been run for real, not just against the test stub: Ollama and xAI each pass chat, streaming, the tool loop, structured output and schema-less Ask through this provider. The evidence, with dates and versions, is in snippet 6049842. The other names above are the same wire protocol and nothing more; treat them as plausible until one is run.

Non-HTTPS endpoints. ValidateBaseURL rejects http:// unless AllowInsecureBaseURL is set, which is for httptest.Server targets and for private endpoints on a network you control, such as an internal vLLM server. It can only be set in code, never from a config file. Anything reached over a network you do not control belongs behind HTTPS.

Token chunking falls back to the cl100k_base encoding for model names the tokenizer does not recognise, so Ollama and other non-OpenAI model names are handled gracefully.

Cloud-credentialled providers (Bedrock, Vertex)¶

Two providers do not take an API key at all. They authenticate through their cloud's credential chain, which is a resolution order rather than a value, so there is no Config.Token to set and supplying one is an error rather than an alternative.

AWS Bedrock¶

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderBedrock,
    Location: "eu-west-2", // or AWS_REGION, or anything the AWS chain resolves
}})

Credentials come from the standard AWS chain: environment, shared profile, instance role, SSO, or a web identity token in CI. Nothing needs configuring here if the chain already works.

Config.Location is the region. An explicit value wins; an empty one falls back to the chain. Construction fails when neither supplies one, rather than guessing.

Three things behave differently from the API-key providers:

1. Model access is granted per account and per region, so a model that works for one caller returns AccessDeniedException for another. That error has several unrelated causes, including an organisation policy rather than model access, so read the message rather than assuming which.
2. Capability varies per model, not per provider. Ask Capabilities(); a model that cannot honour ResponseSchema, tools or media refuses at construction rather than mid-request.
3. There is no BaseURL. Bedrock addresses by region, and Config.Model carries whichever form your region needs: a model ID, an inference profile ID, or an ARN.

Attaching a guardrail¶

A Bedrock guardrail is attached with a chat-bedrock option, so the client is built with chatbedrock.New rather than chat.New (spec 0031):

client, err := chatbedrock.New(ctx, chat.Settings{Config: cfg},
    chatbedrock.WithGuardrail("gr-abc123", "3"),
    chatbedrock.WithGuardrailObserver(func(o chatbedrock.GuardrailOutcome) {
        log.Info("guardrail", "blocked", o.Blocked, "masked", o.Masked)
    }))

• A block is a refusal. A reply the guardrail blocked, on the prompt or the answer, comes back as a *chat.RefusedError matching chat.ErrRefused, and is not recorded in history. Category names the policies, such as content:VIOLENCE + topic:investment-advice.
• A masked reply is an answer. The text carries the guardrail's placeholders, such as {EMAIL}, which can stop a schema-bound Ask from decoding. The observer is where masking shows.
• An intervention with no trace is a refusal, with Category: "untraced", because the client asked for a trace and can't tell a block from a mask without one.
• The observer runs once per model request, so once per tool-loop step. It reports which policies acted, never the text they matched.
• Streams are held while the guardrail evaluates (sync mode), so no unchecked text reaches your callback. That costs time to first token.
• A guardrail is charged per text unit evaluated, on top of the model.
• A block doesn't fail over under the default fallback policy, which doesn't advance on chat.ErrRefused.

A guardrail the account enforces on a client that attached none is still returned as a reply; see the limitations.

Gemini on Vertex AI¶

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderGeminiVertex,
    Project:  "acme-prod",    // or GOOGLE_CLOUD_PROJECT
    Location: "europe-west2", // or GOOGLE_CLOUD_LOCATION
}})

Authentication is Google application default credentials. Both project and location are required: an explicit field wins, the standard variable is the fallback, and construction fails when neither supplies one.

Setting GOOGLE_GENAI_USE_VERTEXAI while configured for ProviderGemini is refused at construction. The SDK reads that variable and would switch backend silently, which is a configuration the module could not report or validate.

Azure OpenAI¶

Azure takes a caller-owned endpoint, a dated api-version, and a deployment name in place of a model name.

import (
    "os"

    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-openai-azure"
)

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider:   chat.ProviderAzureOpenAI,
    BaseURL:    "https://my-resource.openai.azure.com",
    APIVersion: "2024-10-21",
    Model:      "my-deployment-name", // the deployment, not the model
    Token:      os.Getenv("AZURE_OPENAI_API_KEY"),
}})

All four fields are required, and each fails at construction rather than on the first request.

Config.Model is the deployment name. On Azure you deploy a model and give that deployment a name, and the API addresses the name. It is frequently not the model's name, so my-gpt4-prod is an ordinary deployment of gpt-4o. There is no default, because only you know what you called it.

Config.APIVersion has no default and never will. Azure's versions are dated, and which models exist varies between them, so a default would be correct today and quietly wrong later. Use the version your deployment expects.

Config.Token is the Azure api-key, sent as an api-key header rather than a bearer token. Unlike every other provider here, no environment variable is consulted: an OPENAI_API_KEY in your shell is neither read nor sent.

Entra ID instead of an api-key¶

Pass a credential to the module's own constructor:

import chatazure "gitlab.com/phpboyscout/go/chat-openai-azure"

client, err := chatazure.New(ctx, settings, chatazure.WithTokenCredential(cred))

cred is any azcore.TokenCredential. The module takes one rather than resolving one, which is what keeps azidentity out of its dependencies. For ambient discovery, resolve it with azureclient and pass the result in.

Set the api-key or the credential, not both. Both together is refused at construction, and so is neither.

What Azure reports about capability¶

Azure delegates capability reporting to openai-compatible, and from chat-openai v0.12.1 that answers SupportUnknown for everything model-dependent: tools, structured output, multimodal, sampling, effort and caching. A deployment name resolves to no known model, so nothing better is available.

Nothing is refused on that basis, since unknown proceeds untouched. What you lose is being warned in advance; what you gain is not being promised support nobody checked. See where capability answers come from.

Claude Local (the claude CLI)¶

ProviderClaudeLocal routes requests through a locally installed, signed-in claude CLI binary instead of the Anthropic API. It is valuable in secure or air-gapped environments where outbound HTTPS to api.anthropic.com is blocked but the binary is permitted. It is registered by the chat-anthropic import, alongside ProviderClaude, and takes no API key of its own.

Requirements:

• The claude CLI installed and on PATH.
• Signed in once (claude login).

npm install -g @anthropic-ai/claude-code
claude login

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-anthropic"
)

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider:     chat.ProviderClaudeLocal,
    Model:        "claude-sonnet-4-6", // optional; uses claude's default if empty
    SystemPrompt: "You are a helpful assistant.",
}})

Which account the CLI bills¶

The subprocess is given PATH and HOME and nothing else, so the CLI authenticates with the login you established with claude login.

That is deliberate. The CLI prefers an ANTHROPIC_API_KEY found in its environment over its own stored login, so inheriting the host's environment let an unrelated variable decide which account paid, silently. Around thirty of the CLI's environment variables route authentication or billing, and the set grows between versions, which is why the provider passes an allowlist rather than removing names from one.

If your CLI needs more, name what it needs:

import chatanthropic "gitlab.com/phpboyscout/go/chat-anthropic"

client, err := chatanthropic.NewClaudeLocal(ctx, settings,
    chatanthropic.WithClaudeEnvironment("HTTPS_PROXY", "ANTHROPIC_CONFIG_DIR"),
)

Multi-turn continuity is maintained via session IDs captured from the CLI's JSON output and passed back with --resume. Two limitations set claude-local apart: it does not stream, and its tool calling takes a different route (see below). It also reports no reliable token usage. See Providers.

Tools you register¶

Tools need the MCP tool bridge, which a program links with one blank import. It serves claude-local and codex-local alike:

import _ "gitlab.com/phpboyscout/go/chat-mcptools"

From chat-anthropic and chat-openai v0.17.0, neither module links the bridge, or the MCP SDK it brings, unless you import it (spec 0028). A program that never offers a local CLI tools stays about 3 MB lighter. Without the import, SetTools with tools returns an error naming it, clearing tools still works, and Capabilities reports CapTools: SupportNo. To give one client a server of its own, such as a fake in a test, pass chatanthropic.WithToolServer or chatopenai.WithToolServer.

With the bridge linked, SetTools works, and the handlers stay in your process. Each call publishes them on a loopback MCP server, points the CLI at it with --mcp-config, and takes it down again when the call returns, so nothing needs closing and no port is held between calls. Dispatch, panic recovery and result marshalling are the core's, so a tool behaves exactly as it would on an API provider.

The server is reachable only from 127.0.0.1, on an ephemeral port, and only with a per-call bearer token that lives in an 0600 file rather than on the command line where ps would show it. Each registered tool is named in --allowedTools, so the pre-authorisation is exactly as wide as what you registered.

It needs a claude CLI that supports --mcp-config; verified against 2.1.260.

Codex Local and Agy Local¶

Two more CLIs are driven the same way, each registered by the module that already carries its vendor's API:

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-openai" // registers codex-local
    _ "gitlab.com/phpboyscout/go/chat-gemini" // registers agy-local
)

codex, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderCodexLocal,
}})

agy, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderAgyLocal,
}})

Neither takes a model by default: each CLI owns its own and names models this module has no catalogue for. Set Config.Model to name one yourself.

codex-local accepts image attachments, which no other local-CLI provider does, and it carries tools you register, given the bridge import above. agy-local carries neither. agy has no per-invocation MCP configuration, so SetTools returns an error and Capabilities reports CapTools: SupportNo; use chat.ProviderGemini for tool calling against the same models. agy-local does report the fullest token usage of the three, and is the only one whose Usage.Known is true.

codex-local honours your own codex configuration, including the permissions you set through the CLI's /permissions command. Pass chatopenai.WithIgnoredUserConfig() for identical behaviour on every machine instead, understanding that what becomes reproducible is the absence of that machine's restrictions.

What the model on the far side can reach¶

The claude CLI is an agentic tool and ships its own tools, so this provider runs it with --restricted --strict-mcp-config --tools "". The model is offered none of the CLI's tools: no file reads or writes, no command execution, no WebFetch. It also gets no user, project or local settings files, and none of the MCP servers the machine has configured. What it is offered is your own tools from SetTools and, for Ask, the CLI's structured-output tool. Measured against claude 2.1.288, from chat-anthropic v0.16.2:

This matters because a chat prompt is often not trusted: every other provider sends text to somebody else's API, where the worst case is a bad answer. Before v0.16.2 the CLI's file tools could still read inside the working directory.

And claude is the only one of the three that can be told to give up its tools at all. codex-local and agy-local pass the strongest restraint their CLIs offer and a prompt can still read files and run commands as you. The comparison, with what each provider passes, is in what a local-CLI provider can and cannot promise.

To run the CLI as it behaves in a terminal, opt back in:

client, err := chatanthropic.NewClaudeLocal(ctx, settings,
    chatanthropic.WithUnrestrictedCLI(),
)

Related¶

• Architecture & the seams: how chat.New dispatches to a provider.
• Providers: the full capability matrix and per-provider notes.
• Provider-endpoint & credential security: BaseURL validation and credential handling.