Skip to content

Stream responses

Streaming delivers partial response text as the model generates it, rather than waiting for the whole reply — cutting perceived latency for long answers and enabling live TUI/CLI output.

Streaming is offered by claude, openai, openai-compatible, and gemini. claude-local does not stream.

Discover support via type assertion

Streaming providers implement chat.StreamingChatClient (which embeds ChatClient and adds StreamChat). Because not every provider streams, discover support with a type assertion rather than assuming it:

client, err := chat.New(ctx, chat.Settings{Config: chat.Config{
    Provider: chat.ProviderClaude,
}})
if err != nil {
    return err
}

streamer, ok := client.(chat.StreamingChatClient)
if !ok {
    // e.g. ProviderClaudeLocal — fall back to Chat
    _, err := client.Chat(ctx, "Write a haiku about Go.")
    return err
}

Consume the event stream

StreamChat takes a StreamCallback invoked once per event, and returns the complete assembled text (the concatenation of every EventTextDelta fragment):

result, err := streamer.StreamChat(ctx, "Write a haiku about Go.", func(e chat.StreamEvent) error {
    switch e.Type {
    case chat.EventTextDelta:
        fmt.Print(e.Delta) // progressive output
    case chat.EventToolCallStart:
        fmt.Printf("\n[calling tool: %s]\n", e.ToolCall.Name)
    case chat.EventToolCallEnd:
        fmt.Printf("[tool result: %s]\n", e.ToolCall.Result)
    case chat.EventComplete:
        fmt.Println() // newline after the stream ends
    case chat.EventError:
        return e.Error
    }
    return nil
})
if err != nil {
    return err
}
_ = result // full assembled text

The event types are EventTextDelta, EventToolCallStart, EventToolCallEnd, EventComplete, and EventError.

Callback contract

  • The callback runs synchronously for each event; it blocks the stream while executing, so keep it fast.
  • Return a non-nil error from the callback to cancel the stream — that error is returned by StreamChat.
  • StreamChat returns the assembled response text (all EventTextDelta fragments concatenated) regardless of whether it exited early on a callback error.

Tool calls during streaming

Tool calls are handled transparently inside the same ReAct loop as Chat. The callback receives EventToolCallStart when a call begins and EventToolCallEnd (with ToolCall.Result populated) when it completes. Config.ParallelTools and Config.MaxParallelTools are respected — see Call tools.

Prefer streaming, fall back gracefully

For components that benefit from progressive output (TUI widgets, answer commands, doc generators), prefer StreamChat and degrade to Chat when the active provider does not stream:

func queryAI(ctx context.Context, client chat.ChatClient, prompt string, deltaFn func(string)) (string, error) {
    if streamer, ok := client.(chat.StreamingChatClient); ok {
        return streamer.StreamChat(ctx, prompt, func(e chat.StreamEvent) error {
            if e.Type == chat.EventTextDelta && deltaFn != nil {
                deltaFn(e.Delta)
            }
            return nil
        })
    }

    return client.Chat(ctx, prompt)
}

This one helper works across all four streaming providers without the caller knowing which is active — and still functions when claude-local is configured.