Architecture & the seams¶

This page explains how chat is put together: the agentic ReAct loop at its core, the philosophy behind it, the split between the core module and the per-provider modules, and the standard-library seams that keep the whole thing light.

One interface, five methods¶

Everything a caller does goes through ChatClient:

• Add — append a user message to history without triggering a completion.
• Chat — send a message and get the text reply, running the ReAct loop when tools are registered.
• Ask — send a question and unmarshal a structured (JSON-schema-validated) response into a target.
• SetTools — register the callable tools (replaces, never merges).
• Usage — read cumulative, provider-neutral token accounting.

Streaming-capable providers additionally implement StreamingChatClient (StreamChat), and persistence-capable providers implement PersistentChatClient (Save/Restore). Both are discovered by type assertion, because not every provider offers them — claude-local, for example, does neither. History from Add persists across Chat and Ask on a single client instance; a new client via chat.New starts a fresh conversation.

Add, Ask, Chat, and StreamChat also accept a trailing variadic of chat.Media for multimodal input (see Providers).

Agentic vs. legacy workflows¶

It helps to contrast the "legacy" single-shot prompt with the "agentic" loop the client enables.

Legacy — the one-way prompt. The user sends a prompt, the model answers in a single pass, done. This is brittle for real tasks: if the model lacks a fact it must either hallucinate or fail, forcing the developer to front-load ("stuff") all conceivable context into the prompt — expensive, and often worse reasoning.

Agentic — the iterative loop. Instead of solving everything in one shot, the model is given a set of senses — your Go functions, exposed as tools. It then:

1. Reasons about the request and decides on a first step.
2. Acts by calling a local tool (read_file, get_config, …).
3. Observes the actual result from your system.
4. Corrects its plan based on that observation.
5. Repeats until it has enough verified information to answer.

The payoff is verification: an agentic workflow does not merely claim it fixed a bug — it can call a test tool to prove the fix before reporting success. That turns the model from a creative writer into a checkable collaborator.

The ReAct loop¶

Chat (and StreamChat) implement Reason → Act → Observe automatically. When the active provider returns one or more tool calls in a response step, the client:

1. Intercepts the response and identifies each requested tool by name.
2. Unmarshals the JSON arguments for each call.
3. Executes the handler(s) — sequentially, or concurrently when Config.ParallelTools is set and a step returns more than one call.
4. Injects each result back into the conversation history.
5. Resumes the conversation to get the model's next step.

The loop runs until the model returns a final text response, or Config.MaxSteps iterations elapse (default 20). A handler error — or a handler panic, which is recovered — becomes a tool-error string fed back to the model rather than aborting the loop or crashing the process. See Call tools.

The module layout¶

The project is one core module plus one module per vendor SDK:

gitlab.com/phpboyscout/go/chat            core: interfaces, ReAct loop, registry,
                                          config shapes, fallback, persistence,
                                          media validation, claude-local
├── chat-anthropic   registers "claude"                     (anthropic-sdk-go)
├── chat-openai      registers "openai" + "openai-compatible" (openai-go + tiktoken)
└── chat-gemini      registers "gemini"                      (google.golang.org/genai)

The core defines the Provider string constants, the ProviderFactory signature, and a registry (RegisterProvider). Each provider module imports its vendor SDK, implements a factory, and registers it from init() — so a consumer's binary links a provider's SDK only when it blank-imports that module. The SDK-free claude-local provider ships in the core as a zero-weight default. Why this matters is the subject of Dependency inversion.

chat.New is the one construction path: it defaults the provider (to claude, or the AI_PROVIDER env var), validates the endpoint, looks up the registered factory, calls it, and audit-logs the endpoint host.

The stdlib seams¶

Rather than invent its own abstractions, the core accepts standard-library types at every seam, each with a safe default so nothing is mandatory:

Each seam is a place a consumer can inject a hardened or instrumented implementation — a corporate HTTP transport, an OS-keychain backend, an in-memory test FS — without the core taking on a dependency to support it. The UsageObserver callback is a fifth, telemetry-shaped seam: it fires once per provider round-trip so a host can emit a metric or event, while the client keeps no telemetry dependency of its own.

Config on reload¶

Chat clients do not live-reconfigure themselves. Provider, model, timeout, and credential settings are read at construction; to pick up changed settings after a host reloads its config, construct a new client. This keeps provider clients independent of any config-observer machinery and avoids mutating conversation state mid-session.

Related¶

• Providers & the per-provider module pattern
• Dependency inversion
• Provider-endpoint & credential security