Skip to content

Handle a partially-applied configuration

chat.New returns the best usable client it can build, along with an error describing anything it could not apply. Both return values can be meaningful at the same time.

client, err := chat.New(ctx, chat.Settings{Config: cfg})
What happened client err
Everything applied usable nil
Some settings could not be applied usable non-nil, lists them
Nothing could be built nil non-nil, marked ErrUnableToConstruct

If you do nothing, nothing breaks

client, err := chat.New(ctx, settings)
if err != nil {
    return err
}

That is the conservative reading and it behaves exactly as it always has. You bail whenever anything is wrong. You only need to read further if you want the more forgiving behaviour.

Telling the two apart

One check separates "I got nothing" from "I got a client that is missing a setting":

client, err := chat.New(ctx, settings)
if errors.Is(err, chat.ErrUnableToConstruct) {
    return err // nothing was built; there is no client to use
}
if err != nil {
    log.Warn("chat client built with settings unapplied", "err", err)
}
// client is usable here in both remaining cases

Deciding whether the shortfall matters

Whether a dropped setting is acceptable is your call, not the module's: Temperature going unapplied is irrelevant to one job and unacceptable to another. So the drop list is structured, rather than something to grep out of a message:

var cfgErr *chat.ConfigErrors
if errors.As(err, &cfgErr) {
    for _, d := range cfgErr.Dropped() {
        if d.Capability == chat.CapSampling {
            return fmt.Errorf("this job needs deterministic sampling: %w", err)
        }
        log.Info("continuing without", "fields", d.Fields, "why", d.Reason)
    }
}

Read the hints

Errors from this module carry hints saying what to do, not just what is wrong.

import "gitlab.com/phpboyscout/go/errors"

for _, h := range errors.GetAllHints(err) {
    log.Info("hint", "advice", h)
}
// e.g. "the setting was not applied and the provider default is used instead;
//       set Config.Model to one that accepts it, or use Config.Effort"

GetAllHints and FlattenHints both descend into the aggregate chat.New returns, so a hint attached below it is still found.

chat.Hints was removed in v0.9.0

Earlier versions shipped a chat.Hints helper because cockroachdb/errors walked single unwraps only, so hints below an aggregate vanished silently, with no error and no panic. go/errors traverses the whole chain including multi-errors, so the standard readers now return the same thing and the helper was removed rather than kept as a second way to do it.

If you were calling chat.Hints, switch to errors.GetAllHints.

What degrades, and what never does

The line is ownership, not severity.

The vendor's features degrade. Temperature, TopP, Effort and CacheTTL are parameters forwarded to a provider. A model declining one is the vendor's decision, so the setting is dropped, the provider's own default applies, and you are told.

A dropped setting is not applied at all, never clamped or approximated. Temperature: 3.0 against a provider whose range stops at 2 samples at the provider's default, not at 2.0. Delivering a value you never asked for while reporting success would be worse than making you read the range.

This module's own features never degrade. Stateless is the module's own buffering behaviour, not a parameter forwarded to a vendor. If it cannot be honoured, that is the module failing to deliver what it promises rather than a vendor limitation being reported, and there is no honest client to hand back, so it is fatal, never dropped.

ResponseSchema is the exception, and it is not checked

Structured output is the module's guarantee too, but there is no marker interface for it and chat.New performs no check. Every provider module shipped here honours Config.ResponseSchema; a custom provider that ignores it is not detected, and you get a client that quietly returns unstructured text. See What this module does not do.

Fallback composites

A composite is built from every provider that could be built, and is fatal only when none could:

fb, err := chat.NewFallbackFromConfigs(ctx, cfgs)
if errors.Is(err, chat.ErrUnableToConstruct) {
    return err // no provider at all
}
if err != nil {
    // some members are missing or degraded; the error names which
    log.Warn("fallback built with gaps", "err", err)
}

A failing primary no longer sinks the composite. If two secondaries build, you get a working client and an error naming the primary. Refusing a usable client because the preferred provider was unavailable is the failure this design exists to avoid.