llm

type API ¶

type API string

API identifies a provider wire protocol. Unknown nonempty values are valid so applications can describe protocols implemented outside this module.

const (
	APIAnthropicMessages     API = "anthropic-messages"
	APIOpenAIChatCompletions API = "openai-chat-completions"
	APIOpenAIResponses       API = "openai-responses"
	APIAzureOpenAIResponses  API = "azure-openai-responses"
	APIOpenAICodexResponses  API = "openai-codex-responses"
	APIGeminiGenerateContent API = "gemini-generate-content"
	APIMistralConversations  API = "mistral-conversations"
	APIGoogleVertex          API = "google-vertex"
	APIBedrockConverseStream API = "bedrock-converse-stream"
)

type AnthropicCompatibility ¶

type AnthropicCompatibility struct {
	EagerToolInputStreaming CompatibilityToggle
	LongCacheRetention      CompatibilityToggle
	SessionAffinity         CompatibilityToggle
	CacheControlOnTools     CompatibilityToggle
	Temperature             CompatibilityToggle
	AdaptiveThinking        CompatibilityToggle
	EmptyThinkingSignature  CompatibilityToggle
	StrictTools             CompatibilityToggle
	// ToolReferences enables defer_loading definitions and tool_reference
	// result content.
	ToolReferences CompatibilityToggle
}

AnthropicCompatibility describes model-specific differences among Anthropic Messages-compatible APIs.

type Attempt ¶

type Attempt struct {
	Operation   string
	Number      int
	MaxAttempts int
	Model       ModelInfo
}

Attempt describes one call made by a retrying Generator. Number is one-based. Model and its nested storage are a fresh copy owned by the hook and may be retained or mutated without affecting the generator.

type AttemptHook ¶

type AttemptHook func(context.Context, Attempt) (http.Header, error)

AttemptHook is called after Request validation and immediately before each provider attempt. It may observe the attempt and return request headers, but cannot inspect, replace, or mutate the validated Request payload. Hooks may be called concurrently and therefore must be safe for concurrent use. A hook must honor ctx cancellation and return promptly so Stream.Close can unblock retry opening attempts. Returned headers are borrowed only until the hook returns and are cloned before use.

type BackgroundGenerator ¶ added in v0.2.0

type BackgroundGenerator interface {
	Generator
	StartBackground(context.Context, Request) (*BackgroundResult, error)
	FetchBackground(context.Context, BackgroundHandle) (*BackgroundResult, error)
	CancelBackground(context.Context, BackgroundHandle) (*BackgroundResult, error)
}

BackgroundGenerator starts, fetches, and cancels durable provider-side generations. A handle passed to FetchBackground or CancelBackground must match the provider, model, and API reported by Info. Implementations validate requests and handles before I/O and are safe for concurrent use. A failed provider generation returns a result with BackgroundFailed together with its provider error. Malformed output or cancellation that races with an observed provider response may also return its handle and status with an error; transport errors before a response is observed return a nil result.

type BackgroundHandle ¶ added in v0.2.0

type BackgroundHandle struct {
	Provider       string
	Model          string
	API            API
	ID             string
	Tools          []Tool
	ResponseFormat ResponseFormat
}

BackgroundHandle contains the durable identity and output-validation contract for a provider-side generation. Tools and their schemas are retained for caller-owned validation of fetched tool calls. Handles returned by implementations are caller-owned and may be persisted. Callers must not modify a handle concurrently with FetchBackground or CancelBackground.

func (h BackgroundHandle) Validate() error

type BackgroundResult ¶ added in v0.2.0

type BackgroundResult struct {
	Handle   BackgroundHandle
	Status   BackgroundStatus
	Response *Response
}

BackgroundResult is one observed state of a background generation. Response is non-nil only when Status is BackgroundCompleted or BackgroundIncomplete and output conversion succeeds. Handle and all nested response storage are caller-owned. Callers must retain a non-nil result even when the accompanying error is non-nil, because StartBackground may have created a durable job.

type BackgroundStatus ¶ added in v0.2.0

type BackgroundStatus string

BackgroundStatus describes the lifecycle of a durable provider-side generation. The zero value is invalid.

const (
	BackgroundQueued     BackgroundStatus = "queued"
	BackgroundInProgress BackgroundStatus = "in_progress"
	BackgroundCompleted  BackgroundStatus = "completed"
	BackgroundIncomplete BackgroundStatus = "incomplete"
	BackgroundCancelled  BackgroundStatus = "cancelled"
	BackgroundFailed     BackgroundStatus = "failed"
)

type CacheControlFormat ¶

type CacheControlFormat string

CacheControlFormat identifies a prompt-cache marker convention. Empty means the protocol default. Metadata support does not by itself add cache markers to a request that contains none.

const (
	CacheControlDefault   CacheControlFormat = ""
	CacheControlAnthropic CacheControlFormat = "anthropic"
)

type CacheRetention ¶

type CacheRetention string

CacheRetention controls provider prompt-cache policy. Default leaves the provider's policy unchanged; None explicitly disables request cache markers; Short and Long request the provider's short and longest portable retention.

const (
	CacheRetentionDefault CacheRetention = ""
	CacheRetentionNone    CacheRetention = "none"
	CacheRetentionShort   CacheRetention = "short"
	CacheRetentionLong    CacheRetention = "long"
)

type Capability ¶

type Capability string

Capability identifies an operation supported by a model.

const (
	// CapabilityGeneration indicates text or multimodal generation support.
	CapabilityGeneration Capability = "generation"
	// CapabilityStreaming indicates incremental generation support.
	CapabilityStreaming Capability = "streaming"
	// CapabilityTools indicates tool-calling support.
	CapabilityTools Capability = "tools"
	// CapabilityJSON indicates structured JSON output support.
	CapabilityJSON Capability = "json"
	// CapabilityVision indicates image input support.
	CapabilityVision Capability = "vision"
	// CapabilityAudio indicates audio input support.
	CapabilityAudio Capability = "audio"
)

type Chunk ¶

type Chunk struct {
	ID               string
	Model            string
	Content          []Part
	ToolCalls        []ToolCall
	ReasoningSummary string
	ProviderData     json.RawMessage
	FinishReason     FinishReason
	Usage            *Usage
	Events           []StreamEvent
}

Chunk contains new output from a generation stream. To assemble a Response, append Content, ToolCalls, and ReasoningSummary in chunk order, retain any nonempty ID, Model, and FinishReason, use the single nonempty ProviderData, and use the last non-nil Usage. Events describe the same output at a finer granularity and must not be appended to the response. The assembled message has RoleAssistant.

Text parts may be fragments; other parts and ToolCalls must be complete values. Events may expose incomplete tool argument fragments, but ToolCalls never do. Nonempty ID and Model values must not change between chunks. A nonzero FinishReason must not change between chunks. ProviderData must be a complete value and appear in at most one chunk. Usage, when present, is cumulative for the request.

type CompatibilityToggle ¶

type CompatibilityToggle string

CompatibilityToggle is a tri-state model compatibility override. Default leaves protocol behavior unchanged; Enabled and Disabled explicitly opt in or out. The zero value is Default.

const (
	CompatibilityDefault  CompatibilityToggle = ""
	CompatibilityEnabled  CompatibilityToggle = "enabled"
	CompatibilityDisabled CompatibilityToggle = "disabled"
)

type Error ¶

type Error struct {
	Kind       ErrorKind
	Op         string
	Provider   string
	HTTPStatus int
	RetryAfter time.Duration
	Retryable  *bool
	Err        error
}

Error describes a provider-neutral model error. Provider and Op identify its source when known. HTTPStatus is zero when unavailable; provider-native codes remain available through Err. RetryAfter is positive when a provider advised a delay. Err retains the underlying cause, including context cancellation and deadline errors. Retryable is nil when the provider supplied no explicit retry hint; non-nil true or false takes precedence over generic classification.

func (e *Error) Error() string

func (e *Error) Unwrap() error

type ErrorKind ¶

type ErrorKind uint8

ErrorKind classifies an error independently of a provider's wire format. The constants below are the complete set of valid kinds. Implementations must use the most specific applicable kind. The zero value is KindUnknown.

const (
	// KindUnknown identifies an error that has no more specific classification.
	KindUnknown ErrorKind = iota
	// KindInvalidRequest identifies an invalid request without a more specific
	// classification, whether detected locally or rejected by a provider.
	KindInvalidRequest
	// KindAuthentication identifies missing or invalid credentials.
	KindAuthentication
	// KindPermission identifies a request forbidden for valid credentials.
	KindPermission
	// KindRateLimit identifies provider throttling.
	KindRateLimit
	// KindContextLimit identifies a request that does not fit within a model's
	// context limit.
	KindContextLimit
	// KindUnsupported identifies a capability the implementation cannot honor.
	KindUnsupported
	// KindTransport identifies a failure communicating with a provider.
	KindTransport
	// KindProvider identifies another provider-side failure.
	KindProvider
	// KindMalformedResponse identifies a response that cannot be decoded or
	// violates the provider protocol.
	KindMalformedResponse
)

func (k ErrorKind) String() string

type FinishReason ¶

type FinishReason string

FinishReason describes why generation stopped.

const (
	// FinishReasonStop indicates a natural or requested stop sequence.
	FinishReasonStop FinishReason = "stop"
	// FinishReasonLength indicates that a token limit was reached.
	FinishReasonLength FinishReason = "length"
	// FinishReasonToolCall indicates that the model requested a tool call.
	FinishReasonToolCall FinishReason = "tool_call"
	// FinishReasonContentFilter indicates that content was withheld by a safety
	// system.
	FinishReasonContentFilter FinishReason = "content_filter"
)

type GeminiCompatibility ¶ added in v0.3.3

type GeminiCompatibility struct {
	// ThinkingLevelsOnly restricts explicit reasoning controls to low, medium,
	// and high effort. Token budgets, disabled reasoning, and other effort
	// levels are rejected before provider I/O.
	ThinkingLevelsOnly CompatibilityToggle
}

GeminiCompatibility describes model-specific differences among Gemini GenerateContent-compatible APIs.

type Generator ¶

type Generator interface {
	Info() ModelInfo
	Generate(ctx context.Context, request Request) (*Response, error)
	Stream(ctx context.Context, request Request) (Stream, error)
}

Generator produces model responses. A Generator is configured for the model reported by Info and may be used concurrently by multiple goroutines. Implementations must first call Request.Validate and return its error unchanged or wrapped so errors.As preserves its Error fields. They must next return an error matching ctx.Err if the context is already done, then apply capability checks, then model-specific checks, all before provider I/O. Implementations must continue to honor cancellation until Generate returns or a stream commits its terminal state. They return a nil response when Generate returns an error and a nil Stream when Stream returns an error. Before returning a Stream error, an implementation must release all storage borrowed from the request. Info must include CapabilityGeneration. If it does not include CapabilityStreaming, Stream returns an error classified KindUnsupported. Callers should give operations a context deadline appropriate to their workload when an unbounded request is not acceptable.

Returned tool calls have valid structure and strict JSON, but their arguments may violate the declared schemas. Execution owners must use ValidateToolCalls before invoking tools; a schema mismatch is not a provider/stream failure.

func WithRetry(generator Generator, policy RetryPolicy) (Generator, error)

type InstructionRole ¶

type InstructionRole string

InstructionRole selects the Chat Completions role used for canonical system instructions. The zero value uses system.

const (
	InstructionRoleDefault   InstructionRole = ""
	InstructionRoleSystem    InstructionRole = "system"
	InstructionRoleDeveloper InstructionRole = "developer"
)

type MaxTokensField ¶

type MaxTokensField string

MaxTokensField identifies the Chat Completions request field used for an output-token limit. The zero value uses the protocol default.

const (
	MaxTokensFieldDefault    MaxTokensField = ""
	MaxTokensFieldCompletion MaxTokensField = "max_completion_tokens"
	MaxTokensFieldLegacy     MaxTokensField = "max_tokens"
)

type Message ¶

type Message struct {
	Role         Role
	Content      []Part
	ToolCalls    []ToolCall
	ToolResults  []ToolResult
	ProviderData json.RawMessage
}

Message is one turn in a model conversation. ToolCalls are valid only on an assistant message and may accompany Content. ToolResults are valid only on a tool message; its Content must be empty because each result carries its own content. Providers translate this canonical form to their wire format.

ProviderData is opaque state returned by a provider. Callers may pass it back unchanged on a later turn. A provider must ignore data it does not recognize.

func (m Message) Text() string

type Model ¶

type Model struct {
	Provider        string
	ID              string
	Name            string
	API             API
	Capabilities    []Capability
	ContextWindow   int
	MaxOutputTokens int
	Reasoning       bool
	Cost            *ModelCost
	Compatibility   *ModelCompatibility
}

Model describes one catalog entry. Name is a human-readable display name and defaults conceptually to ID when empty. Reasoning reports whether the model supports explicit reasoning controls. ContextWindow and MaxOutputTokens are zero when unknown.

func (m Model) Info() ModelInfo

type ModelCompatibility ¶

type ModelCompatibility struct {
	OpenAIChat      *OpenAIChatCompatibility
	OpenAIResponses *OpenAIResponsesCompatibility
	Gemini          *GeminiCompatibility
	Anthropic       *AnthropicCompatibility
}

ModelCompatibility contains at most one protocol-specific compatibility block. Catalog validation requires that block to match the model API.

func (c *ModelCompatibility) Validate(api API) error

type ModelCost ¶

type ModelCost struct {
	Input      float64
	Output     float64
	CacheRead  float64
	CacheWrite float64
	Tiers      []ModelCostTier
}

ModelCost contains prices in US dollars per million tokens. Tiers apply the highest threshold exceeded by total input usage to the entire request.

func (c ModelCost) Validate() error

type ModelCostTier ¶

type ModelCostTier struct {
	InputTokensAbove int
	Input            float64
	Output           float64
	CacheRead        float64
	CacheWrite       float64
}

ModelCostTier overrides all rates when total input usage is greater than InputTokensAbove.

type ModelInfo ¶

type ModelInfo struct {
	Provider        string
	Model           string
	API             API
	Capabilities    []Capability
	ContextWindow   int
	MaxOutputTokens int
	Reasoning       bool
	Cost            *ModelCost
	Compatibility   *ModelCompatibility
}

ModelInfo describes a configured model. API identifies its wire protocol. ContextWindow and MaxOutputTokens are zero when unknown. Reasoning reports support for explicit reasoning controls. Cost and Compatibility are nil when pricing or protocol compatibility metadata is unknown.

type OpenAIChatCompatibility ¶

type OpenAIChatCompatibility struct {
	MaxTokensField           MaxTokensField
	InstructionRole          InstructionRole
	ThinkingFormat           ThinkingFormat
	ReasoningEffort          CompatibilityToggle
	StreamingUsage           CompatibilityToggle
	FinishReason             CompatibilityToggle
	ToolResultName           CompatibilityToggle
	ToolResultImageFallback  CompatibilityToggle
	AssistantAfterToolResult CompatibilityToggle
	ReasoningContentReplay   CompatibilityToggle
	StrictTools              CompatibilityToggle
	CacheControlFormat       CacheControlFormat
	LongCacheRetention       CompatibilityToggle
	// SessionAffinity enables SessionID headers for this compatible endpoint.
	SessionAffinity CompatibilityToggle
	// SessionAffinityFormat selects the header convention when enabled.
	SessionAffinityFormat SessionAffinityFormat
}

OpenAIChatCompatibility describes model-specific differences among OpenAI Chat Completions-compatible APIs.

type OpenAIResponsesCompatibility ¶

type OpenAIResponsesCompatibility struct {
	DeveloperRole      CompatibilityToggle
	EncryptedReasoning CompatibilityToggle
	StrictTools        CompatibilityToggle
	// AdditionalTools enables replaying deferred tool definitions as
	// additional_tools input items.
	AdditionalTools CompatibilityToggle
	// ToolSearch enables client tool-search call/output replay when
	// AdditionalTools is not enabled.
	ToolSearch              CompatibilityToggle
	LongCacheRetention      CompatibilityToggle
	ExplicitPromptCacheMode CompatibilityToggle
	// SessionAffinityFormat selects the SessionID header convention. Its zero
	// value auto-detects native OpenAI or OpenRouter from provider routing.
	SessionAffinityFormat SessionAffinityFormat
}

OpenAIResponsesCompatibility describes model-specific differences among OpenAI Responses-compatible APIs.

type Part ¶

type Part struct {
	Kind            PartKind
	Text            string
	Data            []byte
	MediaType       string
	CacheBreakpoint bool `json:",omitzero"`
}

Part is one piece of message content. Text parts use Text and leave Data and MediaType empty. Image and audio parts use Data and MediaType and leave Text empty. Other combinations are invalid. CacheBreakpoint marks the exact end of a reusable prompt prefix. It requires short or long cache retention and is rejected by adapters that do not support explicit breakpoint placement.

type PartKind ¶

type PartKind uint8

PartKind identifies the representation of a message part. The zero value is PartText.

const (
	// PartText contains UTF-8 text in Part.Text.
	PartText PartKind = iota
	// PartImage contains encoded image bytes in Part.Data.
	PartImage
	// PartAudio contains encoded audio bytes in Part.Data.
	PartAudio
)

type ReasoningEffort ¶

type ReasoningEffort string

ReasoningEffort controls how much internal reasoning a capable model uses. The zero value leaves the provider's default unchanged. Providers may map an effort to the nearest level supported by the configured model.

const (
	ReasoningEffortDefault ReasoningEffort = ""
	ReasoningEffortNone    ReasoningEffort = "none"
	ReasoningEffortMinimal ReasoningEffort = "minimal"
	ReasoningEffortLow     ReasoningEffort = "low"
	ReasoningEffortMedium  ReasoningEffort = "medium"
	ReasoningEffortHigh    ReasoningEffort = "high"
	ReasoningEffortXHigh   ReasoningEffort = "xhigh"
	ReasoningEffortMax     ReasoningEffort = "max"
)

type Request ¶

type Request struct {
	Messages        []Message
	Tools           []Tool
	ToolChoice      ToolChoice
	ResponseFormat  ResponseFormat
	ReasoningEffort ReasoningEffort
	// ReasoningBudgetTokens requests an explicit provider reasoning-token
	// budget. Zero leaves budgeting to the provider or ReasoningEffort.
	ReasoningBudgetTokens int
	CacheRetention        CacheRetention
	// CacheKey partitions provider prompt-cache identity. It is not a session
	// affinity identifier and unsupported mappings are rejected.
	CacheKey string
	// SessionID supplies provider session-affinity identity independently of
	// CacheRetention. When caching is enabled, it may also be used as the fallback
	// prompt-cache key if CacheKey is empty.
	SessionID        string
	MaxOutputTokens  int
	Temperature      *float64
	TopP             *float64
	PresencePenalty  *float64
	FrequencyPenalty *float64
	Stop             []string
}

Request describes one generation operation. Messages must not be empty. MaxOutputTokens set to zero and nil sampling or penalty fields ask the provider to use its default. An empty Tools or Stop list means none. Implementations must treat Request as read-only.

func (r Request) HasCacheBreakpoints() bool

func (r Request) Validate() error

type Response ¶

type Response struct {
	ID      string
	Model   string
	Message Message
	// ReasoningSummary is provider-generated explanatory text about the model's
	// reasoning. It is empty when the provider omits a summary and is separate
	// from the assistant message returned by Text.
	ReasoningSummary string
	FinishReason     FinishReason
	Usage            *Usage
}

Response is the result of a generation operation.

func Collect(stream Stream, tools []Tool) (*Response, error)

func CollectStructural(stream Stream) (*Response, error)

func (r Response) Text() string

type ResponseFormat ¶

type ResponseFormat uint8

ResponseFormat identifies the requested response representation. The zero value is ResponseFormatText.

const (
	// ResponseFormatText requests ordinary model output.
	ResponseFormatText ResponseFormat = iota
	// ResponseFormatJSON requests a valid JSON value.
	ResponseFormatJSON
)

type RetryPolicy ¶

type RetryPolicy struct {
	MaxAttempts    int
	InitialBackoff time.Duration
	MaxBackoff     time.Duration
	Multiplier     float64
	Jitter         float64
	ShouldRetry    func(error) bool
	Hook           AttemptHook
}

RetryPolicy configures an opt-in retrying Generator. MaxAttempts includes the first attempt and must be positive. Zero backoff fields use conservative defaults. MaxBackoff also bounds an acceptable provider RetryAfter; a larger requested minimum stops retrying instead of sleeping less than requested. Ambiguous transport failures are not retried by default because a POST may already have been accepted; use an explicit provider Retryable hint or ShouldRetry when the caller can accept that risk. Jitter is a fractional range from zero through one. ShouldRetry, if non-nil, replaces the default retry classification and must be concurrency-safe.

type Role ¶

type Role string

Role identifies a message participant. The constants below are the complete set of valid roles; the zero value is invalid.

const (
	// RoleSystem carries instructions that apply to the conversation.
	RoleSystem Role = "system"
	// RoleUser carries input from the end user.
	RoleUser Role = "user"
	// RoleAssistant carries model output.
	RoleAssistant Role = "assistant"
	// RoleTool carries results produced by tools.
	RoleTool Role = "tool"
)

type SessionAffinityFormat ¶

type SessionAffinityFormat string

SessionAffinityFormat identifies provider session-affinity header conventions. The zero value is resolved from the configured provider endpoint.

const (
	SessionAffinityDefault         SessionAffinityFormat = ""
	SessionAffinityOpenAI          SessionAffinityFormat = "openai"
	SessionAffinityOpenAINoSession SessionAffinityFormat = "openai-nosession"
	SessionAffinityOpenRouter      SessionAffinityFormat = "openrouter"
)

type Stream ¶

type Stream interface {
	// Recv returns the next chunk. It returns io.EOF on clean completion. When
	// it returns an error, the chunk is zero and later calls return the same
	// error. Context cancellation and deadline errors remain discoverable with
	// errors.Is.
	Recv() (Chunk, error)

	// Close stops the stream and releases its resources. It is idempotent and
	// must unblock a pending Recv. It commits io.EOF unless another terminal
	// state won first or the context was already done. Close does not return
	// until the implementation has released all storage borrowed from the
	// request. Repeated calls return the same error; a Close error does not
	// replace the stream's terminal state.
	Close() error
}

Stream is a sequence of generation chunks. Callers must close every Stream, including after Recv returns an error. Only one goroutine may call Recv at a time. Close is safe to call concurrently with any Stream method.

A stream commits an ordered sequence of chunks followed by exactly one terminal state: io.EOF or a non-EOF error. Recv delivers every committed chunk exactly once, in order, before the terminal state; Close does not discard committed chunks. Natural completion and Close commit io.EOF, while provider and context failures commit their error. The first terminal transition wins. However, if the context passed to Generator.Stream is already done when Close is called, its error precedes Close. If cancellation and Close occur concurrently, either may win.

func ValidateToolCallStream(stream Stream, tools []Tool, provider string) (Stream, error)

func ValidateToolCallStructureStream(stream Stream, provider string) (Stream, error)

type StreamEvent ¶

type StreamEvent struct {
	Kind         StreamEventKind
	Index        int
	Subindex     int
	Delta        string
	Content      string
	ToolCallID   string
	ToolName     string
	ToolCall     *ToolCall
	FinishReason FinishReason
}

StreamEvent describes a semantic transition represented by a Chunk. Index identifies a provider output item and Subindex identifies a part within it; both are stable for events belonging to that item. Providers without nested output parts leave Subindex zero. Delta is newly received text, reasoning text, or raw tool argument fragment; a tool argument delta need not be valid JSON by itself. Content is the completed text for an end event when the provider makes it available. ToolCallID and ToolName contain the identity known at a tool-call start or delta. ToolCall is set on a successfully completed ToolCallEnd and contains complete strict-JSON arguments, which may violate the tool schema; execution owners must use ValidateToolCalls before invoking tools. It is nil when an incomplete response interrupts a started call. FinishReason is set only for Done.

Streams report failures through Recv rather than as events. Event storage is owned by the caller after Recv returns.

type StreamEventKind ¶

type StreamEventKind string

StreamEventKind identifies one semantic event in a generation stream.

const (
	StreamEventStart          StreamEventKind = "start"
	StreamEventTextStart      StreamEventKind = "text_start"
	StreamEventTextDelta      StreamEventKind = "text_delta"
	StreamEventTextEnd        StreamEventKind = "text_end"
	StreamEventReasoningStart StreamEventKind = "reasoning_start"
	StreamEventReasoningDelta StreamEventKind = "reasoning_delta"
	StreamEventReasoningEnd   StreamEventKind = "reasoning_end"
	StreamEventToolCallStart  StreamEventKind = "tool_call_start"
	StreamEventToolCallDelta  StreamEventKind = "tool_call_delta"
	StreamEventToolCallEnd    StreamEventKind = "tool_call_end"
	StreamEventDone           StreamEventKind = "done"
)

type ThinkingFormat ¶

type ThinkingFormat string

ThinkingFormat identifies an OpenAI-compatible reasoning request convention. The zero value uses reasoning_effort.

const (
	ThinkingFormatDefault    ThinkingFormat = ""
	ThinkingFormatOpenAI     ThinkingFormat = "openai"
	ThinkingFormatOpenRouter ThinkingFormat = "openrouter"
	ThinkingFormatDeepSeek   ThinkingFormat = "deepseek"
	ThinkingFormatTogether   ThinkingFormat = "together"
	ThinkingFormatZAI        ThinkingFormat = "zai"
	ThinkingFormatQwen       ThinkingFormat = "qwen"
	ThinkingFormatString     ThinkingFormat = "string-thinking"
)

type TokenBudget ¶

type TokenBudget struct {
	EstimatedInputTokens  int
	ReserveTokens         int
	AvailableOutputTokens int
	MaxOutputTokens       int
	Clamped               bool
}

TokenBudget reports how BudgetRequest derived the output limit.

type TokenEstimator ¶

type TokenEstimator interface {
	EstimateInputTokens(context.Context, Request) (int, error)
}

TokenEstimator estimates input tokens for a request. Implementations must treat Request as read-only, be safe for the caller's intended concurrency, and honor ctx. Estimates are explicitly approximate unless the implementation uses the target model's exact tokenizer.

type TokenEstimatorFunc ¶

type TokenEstimatorFunc func(context.Context, Request) (int, error)

TokenEstimatorFunc adapts a function to TokenEstimator.

func (f TokenEstimatorFunc) EstimateInputTokens(ctx context.Context, request Request) (int, error)

type Tool ¶

type Tool struct {
	Name        string
	Description string
	InputSchema json.RawMessage
	Strict      bool
	Strictness  ToolStrictness
}

Tool describes a function that a model may call. InputSchema is a JSON Schema describing the function arguments. Providers check completed calls for structure and strict JSON, not argument-schema conformance. Execution owners must use ValidateToolCalls before invoking tools. Schemas default to draft 2020-12, may declare any supported standard draft, and may use only references resolved within that schema. Strict is the legacy require-strict flag. Strictness provides an explicit prefer/require policy; setting both Strict and Strictness is invalid.

func (t Tool) RequiresStrict() bool

func (t Tool) StrictEnabled(supported bool) bool

type ToolCall ¶

type ToolCall struct {
	ID        string
	Name      string
	Arguments json.RawMessage
}

ToolCall is a tool invocation requested by a model. Arguments contains one JSON value, normally an object. ID may be empty when a provider supplies only a function name. The order of ID-less calls is significant. Arguments are returned unchanged, not schema-validated; use ValidateToolCalls before execution.

type ToolChoice ¶

type ToolChoice struct {
	Mode ToolChoiceMode
	Name string
}

ToolChoice is a closed provider-neutral tool selection. Name is required only for ToolChoiceNamed and must identify a declared tool.

type ToolChoiceMode ¶

type ToolChoiceMode string

ToolChoiceMode controls whether and which tools the model may call. The zero value is automatic selection.

const (
	ToolChoiceAuto     ToolChoiceMode = ""
	ToolChoiceNone     ToolChoiceMode = "none"
	ToolChoiceRequired ToolChoiceMode = "required"
	ToolChoiceNamed    ToolChoiceMode = "named"
)

type ToolResult ¶

type ToolResult struct {
	CallID         string
	Name           string
	Content        []Part
	IsError        bool
	AddedToolNames []string
}

ToolResult is the output of one tool invocation. CallID must match the call when its ID is available. Otherwise, Name must match the call and results are paired with preceding unmatched ID-less calls of that name in order. When both CallID and Name are set, both must identify the same call. AddedToolNames records tool definitions introduced after this result while replaying a transcript. Names absent from the current Request.Tools are ignored. Providers that support deferred tool loading may serialize available definitions at this position; other providers ignore the marker and send all request tools normally.

type ToolStrictness ¶

type ToolStrictness string

ToolStrictness controls provider-side JSON Schema constrained generation. The zero value preserves legacy Tool.Strict behavior. Prefer enables provider strictness when supported, while Require fails before I/O when unsupported.

const (
	ToolStrictDefault ToolStrictness = ""
	ToolStrictPrefer  ToolStrictness = "prefer"
	ToolStrictRequire ToolStrictness = "require"
)

type TransformWarning ¶

type TransformWarning struct {
	Code    TransformWarningCode
	Path    string
	Message string
}

TransformWarning describes one transformed or repaired history item.

type TransformWarningCode ¶

type TransformWarningCode string

TransformWarningCode identifies one explicit history transformation or repair.

const (
	TransformRemovedProviderData     TransformWarningCode = "removed_provider_data"
	TransformReplacedImage           TransformWarningCode = "replaced_image"
	TransformReplacedAudio           TransformWarningCode = "replaced_audio"
	TransformNormalizedToolCallID    TransformWarningCode = "normalized_tool_call_id"
	TransformAddedMissingToolResult  TransformWarningCode = "added_missing_tool_result"
	TransformRemovedOrphanToolResult TransformWarningCode = "removed_orphan_tool_result"
	TransformCorrectedToolResultName TransformWarningCode = "corrected_tool_result_name"
)

type Usage ¶

type Usage struct {
	InputTokens        int
	OutputTokens       int
	CacheReadTokens    int
	CacheWriteTokens   int
	CacheWrite1hTokens int
	ReasoningTokens    int
	TotalTokens        int
	Cost               *UsageCost
}

Usage reports token counts when a provider supplies them. InputTokens excludes cache reads and writes; TotalTokens is the sum of input, output, cache-read, and cache-write tokens. ReasoningTokens is a subset of OutputTokens. CacheWrite1hTokens is a subset of CacheWriteTokens. Cost is nil when no pricing metadata was configured.

type UsageCost ¶

type UsageCost struct {
	Input      float64
	Output     float64
	CacheRead  float64
	CacheWrite float64
	Total      float64
}

UsageCost reports cost in US dollars for each token category.

func CalculateCost(cost ModelCost, usage Usage) (UsageCost, error)