Documentation
¶
Overview ¶
Package avelin is an unofficial Go client for the AVELIN AI API (https://api.avelin.ai).
It covers the OpenAI-compatible chat completions endpoint, the Anthropic-compatible messages endpoint, model listing, embeddings and the public model catalog. It uses only the standard library.
Index ¶
- Constants
- Variables
- func Ptr[T any](v T) *T
- type APIError
- type Catalog
- type CatalogModel
- type CatalogPricing
- type ChatChoice
- type ChatChunkChoice
- type ChatCompletion
- type ChatCompletionChunk
- type ChatCompletionRequest
- type ChatMessage
- type Client
- func (c *Client) CreateChatCompletion(ctx context.Context, req ChatCompletionRequest) (*ChatCompletion, error)
- func (c *Client) CreateChatCompletionStream(ctx context.Context, req ChatCompletionRequest) (*Stream[ChatCompletionChunk], error)
- func (c *Client) CreateEmbeddings(ctx context.Context, req EmbeddingRequest) (*EmbeddingList, error)
- func (c *Client) CreateMessage(ctx context.Context, req MessageRequest) (*Message, error)
- func (c *Client) CreateMessageStream(ctx context.Context, req MessageRequest) (*Stream[MessageStreamEvent], error)
- func (c *Client) FetchCatalog(ctx context.Context) (*Catalog, error)
- func (c *Client) ListModels(ctx context.Context) (*ModelList, error)
- type CompletionTokensDetails
- type ContentBlock
- type Embedding
- type EmbeddingList
- type EmbeddingRequest
- type FunctionCall
- type FunctionDefinition
- type Message
- type MessageParam
- type MessageRequest
- type MessageStreamDelta
- type MessageStreamEvent
- type MessageTool
- type MessageToolChoice
- type MessageUsage
- type Meta
- type Model
- type ModelID
- type ModelList
- type Option
- type PromptTokensDetails
- type Stream
- type StreamOptions
- type Thinking
- type Tool
- type ToolCall
- type ToolChoiceFunction
- type Usage
Constants ¶
const ( // DefaultBaseURL is the API root. Endpoint paths such as /v1/models are // appended to it, so it must not end in /v1. DefaultBaseURL = "https://api.avelin.ai" // DefaultMaxRetries is how many times a request is retried after a 429 // or 5xx response. DefaultMaxRetries = 2 // DefaultTimeout bounds each HTTP attempt. See WithTimeout. DefaultTimeout = 10 * time.Minute )
Variables ¶
var ErrMaxTokens = errors.New("avelin: MessageRequest.MaxTokens must be positive")
ErrMaxTokens is returned, without sending a request, when MessageRequest.MaxTokens is not positive. The API would answer 400, or 500 when the field is missing.
var ErrMissingAPIKey = errors.New("avelin: no API key: set AVELIN_API_KEY or use WithAPIKey")
ErrMissingAPIKey is returned, without sending a request, when an endpoint that needs a key is called and no key is configured.
Functions ¶
Types ¶
type APIError ¶
type APIError struct {
// StatusCode is the HTTP status. For errors reported inside a stream it
// is the status of the stream response, usually 200.
StatusCode int
// Type is error.type, such as "rate_limit_error". It may be empty.
Type string
// Code is error.code as text. AVELIN documents numeric codes that repeat
// the HTTP status. It may be empty.
Code string
// Message is error.message, or a fallback built from the body or status.
Message string
// RequestID comes from the X-Avelin-Request-Id header (see
// Meta.RequestID). AVELIN omits it on some errors, such as 401 and 403.
RequestID string
// Header holds the response headers.
Header http.Header
// Body is the raw response body (at most 1 MiB), or the stream event data.
Body []byte
}
APIError is returned when the API answers with a non-2xx status or reports an error inside a stream. Use errors.As to get it from a returned error.
type Catalog ¶
type Catalog struct {
Object string `json:"object"`
Data []CatalogModel `json:"data"`
// Meta is the HTTP status and headers of the response.
Meta Meta `json:"-"`
// Raw is the full response body, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
Catalog is the public model catalog served at /public/models.json. AVELIN does not document this file; the fields mirror its observed content.
type CatalogModel ¶
type CatalogModel struct {
ID string `json:"id"`
Name string `json:"name"`
Created int64 `json:"created"`
InputModalities []string `json:"input_modalities"`
OutputModalities []string `json:"output_modalities"`
Pricing CatalogPricing `json:"pricing"`
// SupportedSamplingParameters lists request fields such as
// "reasoning_effort" and "temperature".
SupportedSamplingParameters []string `json:"supported_sampling_parameters"`
ContextLength int `json:"context_length"`
MaxOutputLength int `json:"max_output_length"`
}
CatalogModel is one model in the public catalog.
type CatalogPricing ¶
CatalogPricing holds prices in US dollars per token as decimal strings, for example "0.0000014" ($1.40 per million tokens).
type ChatChoice ¶
type ChatChoice struct {
Index int `json:"index"`
Message ChatMessage `json:"message"`
// FinishReason is "stop", "length", "tool_calls" or "content_filter".
// A forced tool call reports "stop", so check Message.ToolCalls rather
// than this field.
FinishReason string `json:"finish_reason"`
}
ChatChoice is one generated message.
type ChatChunkChoice ¶
type ChatChunkChoice struct {
Index int `json:"index"`
Delta ChatMessage `json:"delta"`
FinishReason string `json:"finish_reason"`
}
ChatChunkChoice is the delta for one choice. FinishReason is empty until the choice's last chunk.
type ChatCompletion ¶
type ChatCompletion struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []ChatChoice `json:"choices"`
Usage Usage `json:"usage"`
// Meta is the HTTP status and headers of the response.
Meta Meta `json:"-"`
// Raw is the full response body, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
ChatCompletion is the response of CreateChatCompletion.
func (*ChatCompletion) Accumulate ¶ added in v0.2.0
func (c *ChatCompletion) Accumulate(chunk ChatCompletionChunk) error
Accumulate adds a stream chunk to c, so that after the last chunk c holds the whole completion. Content, reasoning and tool call arguments are concatenated per choice, and the merged tool calls have a nil Index. Usage is copied from the final chunk when StreamOptions.IncludeUsage was set. It returns an error if a chunk refers to a choice or tool call more than one past the last one started.
type ChatCompletionChunk ¶
type ChatCompletionChunk struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []ChatChunkChoice `json:"choices"`
// Usage is set on the final chunk when StreamOptions.IncludeUsage is
// true.
Usage *Usage `json:"usage,omitempty"`
// Raw is the event's JSON data, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
ChatCompletionChunk is one event of a chat completion stream.
type ChatCompletionRequest ¶
type ChatCompletionRequest struct {
Model ModelID `json:"model"`
Messages []ChatMessage `json:"messages"`
// MaxTokens caps the generated tokens. Zero leaves it to the server.
MaxTokens int `json:"max_tokens,omitempty"`
// Temperature is between 0 and 2. Use Ptr to set it.
Temperature *float64 `json:"temperature,omitempty"`
// TopP is between 0 and 1. Use Ptr to set it.
TopP *float64 `json:"top_p,omitempty"`
// N is the number of choices to generate. Zero means the default, 1.
N int `json:"n,omitempty"`
// Stop holds up to 4 stop sequences.
Stop []string `json:"stop,omitempty"`
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
Tools []Tool `json:"tools,omitempty"`
// ToolChoice controls tool use: "auto", "none", "required" (call at
// least one tool), or a ToolChoiceFunction to force one function; nil or
// "" leaves it to the server. A bare function name, which AVELIN's
// reference shows, is accepted but ignored.
ToolChoice any `json:"tool_choice,omitempty"`
// ReasoningEffort is "low", "medium" or "high" on models that reason.
// AVELIN documents only "high", but all three are accepted. Reasoning
// tokens count toward MaxTokens.
ReasoningEffort string `json:"reasoning_effort,omitempty"`
// StreamOptions applies to CreateChatCompletionStream only;
// CreateChatCompletion leaves it out of the request.
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
}
ChatCompletionRequest is the body of POST /v1/chat/completions (OpenAI-compatible).
type ChatMessage ¶
type ChatMessage struct {
// Role is "system", "user", "assistant" or "tool".
Role string `json:"role"`
Content string `json:"content"`
// ReasoningContent is the model's reasoning, returned separately from
// Content by reasoning-capable models.
ReasoningContent string `json:"reasoning_content,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ToolCallID identifies the call a "tool" message answers.
ToolCallID string `json:"tool_call_id,omitempty"`
}
ChatMessage is one message of a conversation. In a response it is the generated message; in a stream chunk it is the incremental delta. A response message can be sent back as history as it is: AVELIN accepts its ReasoningContent and the tool calls' Index.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client calls the AVELIN API. It is safe for concurrent use.
func NewClient ¶
NewClient returns a client. The API key defaults to the AVELIN_API_KEY environment variable and the base URL to AVELIN_BASE_URL, falling back to DefaultBaseURL.
func (*Client) CreateChatCompletion ¶
func (c *Client) CreateChatCompletion(ctx context.Context, req ChatCompletionRequest) (*ChatCompletion, error)
CreateChatCompletion creates a chat completion.
func (*Client) CreateChatCompletionStream ¶
func (c *Client) CreateChatCompletionStream(ctx context.Context, req ChatCompletionRequest) (*Stream[ChatCompletionChunk], error)
CreateChatCompletionStream creates a chat completion streamed as server-sent events. The caller must Close the stream.
func (*Client) CreateEmbeddings ¶
func (c *Client) CreateEmbeddings(ctx context.Context, req EmbeddingRequest) (*EmbeddingList, error)
CreateEmbeddings embeds the input texts.
func (*Client) CreateMessage ¶
CreateMessage creates a message.
func (*Client) CreateMessageStream ¶
func (c *Client) CreateMessageStream(ctx context.Context, req MessageRequest) (*Stream[MessageStreamEvent], error)
CreateMessageStream creates a message streamed as server-sent events. The caller must Close the stream.
func (*Client) FetchCatalog ¶
FetchCatalog fetches the public model catalog. It needs no API key and never sends one.
type CompletionTokensDetails ¶ added in v0.2.0
type CompletionTokensDetails struct {
ReasoningTokens int `json:"reasoning_tokens"`
TextTokens int `json:"text_tokens"`
}
CompletionTokensDetails splits completion tokens by kind.
type ContentBlock ¶
type ContentBlock struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
Thinking string `json:"thinking,omitempty"`
Signature string `json:"signature,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
ToolUseID string `json:"tool_use_id,omitempty"`
// Content is a tool result: a string, or any value that marshals to an
// array of content blocks.
Content any `json:"content,omitempty"`
IsError bool `json:"is_error,omitempty"`
}
ContentBlock is one block of message content. Type selects the fields that apply:
- "text": Text
- "thinking": Thinking, Signature
- "tool_use": ID, Name, Input
- "tool_result" (input only): ToolUseID, Content, IsError
type Embedding ¶
type Embedding struct {
Object string `json:"object"`
Index int `json:"index"`
Embedding []float64 `json:"embedding"`
}
Embedding is the vector for the input at Index.
type EmbeddingList ¶
type EmbeddingList struct {
Object string `json:"object"`
Data []Embedding `json:"data"`
Model string `json:"model"`
// Usage has PromptTokens and TotalTokens set.
Usage Usage `json:"usage"`
// Meta is the HTTP status and headers of the response.
Meta Meta `json:"-"`
// Raw is the full response body, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
EmbeddingList is the response of CreateEmbeddings.
type EmbeddingRequest ¶
type EmbeddingRequest struct {
Model ModelID `json:"model"`
// Input holds the texts to embed, one embedding per text.
Input []string `json:"input"`
}
EmbeddingRequest is the body of POST /v1/embeddings.
type FunctionCall ¶
type FunctionCall struct {
Name string `json:"name,omitempty"`
Arguments string `json:"arguments"`
}
FunctionCall is a called function's name and its JSON-encoded arguments. In stream chunks the arguments arrive in fragments to be concatenated.
type FunctionDefinition ¶
type FunctionDefinition struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters any `json:"parameters,omitempty"`
}
FunctionDefinition describes a function. Parameters is a JSON Schema object, given as any value that marshals to one, such as a map or a json.RawMessage.
type Message ¶
type Message struct {
ID string `json:"id"`
Type string `json:"type"`
Role string `json:"role"`
Model string `json:"model"`
// Content holds the blocks in order. With thinking on, a "thinking"
// block comes before the "text" block, so use Text for the answer.
Content []ContentBlock `json:"content"`
// StopReason is, for example, "end_turn" or "tool_use". A forced tool
// call reports "end_turn", so look for "tool_use" blocks rather than
// relying on this field.
StopReason string `json:"stop_reason"`
Usage MessageUsage `json:"usage"`
// Meta is the HTTP status and headers of the response.
Meta Meta `json:"-"`
// Raw is the full response body, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
Message is the response of CreateMessage.
func (*Message) Accumulate ¶ added in v0.2.0
func (m *Message) Accumulate(ev MessageStreamEvent) error
Accumulate adds a stream event to m, so that after message_stop m holds the whole message, as CreateMessage would return it. Text, thinking and signatures are concatenated per block, a tool_use block's Input is assembled from its input_json_delta fragments (valid JSON once the block's content_block_stop has arrived), and non-zero usage counts in message_delta replace the estimates from message_start. It returns an error for a delta or stop event whose block has not started.
type MessageParam ¶
type MessageParam struct {
// Role is "user" or "assistant".
Role string
Content string
Blocks []ContentBlock
}
MessageParam is an input message. Set Content for plain text, or Blocks for content blocks such as tool results. A non-nil Blocks is sent instead of Content.
func (MessageParam) MarshalJSON ¶
func (m MessageParam) MarshalJSON() ([]byte, error)
MarshalJSON encodes content as a string, or as an array when Blocks is set.
func (*MessageParam) UnmarshalJSON ¶
func (m *MessageParam) UnmarshalJSON(data []byte) error
UnmarshalJSON accepts content as a string or an array of blocks, so saved conversations can be read back.
type MessageRequest ¶
type MessageRequest struct {
Model ModelID `json:"model"`
// MaxTokens is required and must be positive; CreateMessage and
// CreateMessageStream return an error before sending otherwise.
MaxTokens int `json:"max_tokens"`
Messages []MessageParam `json:"messages"`
System string `json:"system,omitempty"`
// Thinking controls extended thinking. AVELIN documents it as on by
// default, but live responses carry an empty thinking block unless it is
// set to &Thinking{Type: "enabled", BudgetTokens: n}. Use
// &Thinking{Type: "disabled"} to drop the block.
Thinking *Thinking `json:"thinking,omitempty"`
Tools []MessageTool `json:"tools,omitempty"`
// ToolChoice controls tool use; nil leaves it to the model.
ToolChoice *MessageToolChoice `json:"tool_choice,omitempty"`
// Temperature and TopP are optional. Use Ptr to set them.
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
TopK int `json:"top_k,omitempty"`
StopSequences []string `json:"stop_sequences,omitempty"`
}
MessageRequest is the body of POST /v1/messages (Anthropic-compatible).
type MessageStreamDelta ¶
type MessageStreamDelta struct {
Type string `json:"type"`
Text string `json:"text"`
Thinking string `json:"thinking"`
PartialJSON string `json:"partial_json"`
Signature string `json:"signature"`
StopReason string `json:"stop_reason"`
}
MessageStreamDelta is the delta of a content_block_delta event (Type "text_delta", "thinking_delta", "input_json_delta" or "signature_delta") or of a message_delta event (StopReason).
type MessageStreamEvent ¶
type MessageStreamEvent struct {
Type string `json:"type"`
Message *Message `json:"message,omitempty"`
Index int `json:"index"`
ContentBlock *ContentBlock `json:"content_block,omitempty"`
Delta *MessageStreamDelta `json:"delta,omitempty"`
Usage *MessageUsage `json:"usage,omitempty"`
// Raw is the event's JSON data, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
MessageStreamEvent is one event of a messages stream. Type selects the fields that are set:
- "message_start": Message
- "content_block_start": Index, ContentBlock
- "content_block_delta": Index, Delta
- "content_block_stop": Index
- "message_delta": Delta.StopReason, Usage
- "message_stop", "ping": none
Other event types are passed through with Type and Raw set. An "error" event ends the stream with an *APIError.
type MessageTool ¶
type MessageTool struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
InputSchema any `json:"input_schema"`
}
MessageTool is a tool definition. InputSchema is a JSON Schema object, as any value that marshals to one.
type MessageToolChoice ¶ added in v0.2.0
type MessageToolChoice struct {
// Type is "auto", "any" (call at least one tool), "tool" (call Name) or
// "none".
Type string `json:"type"`
// Name is the tool to call when Type is "tool".
Name string `json:"name,omitempty"`
}
MessageToolChoice controls tool use on the messages endpoint. AVELIN's docs do not list it; the live API accepts it. Clear a forcing choice ("any" or "tool") before sending the tool result, or the model is forced to call a tool again.
type MessageUsage ¶
type MessageUsage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
// TotalTokens is in AVELIN's docs but missing from live responses, so it
// is usually zero; add InputTokens and OutputTokens instead.
TotalTokens int `json:"total_tokens"`
CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
CacheReadInputTokens int `json:"cache_read_input_tokens"`
}
MessageUsage reports token counts.
type Meta ¶
Meta is the HTTP metadata of the response that produced a result, such as the X-Avelin-Request-Id header AVELIN sends on most responses.
type Model ¶
type Model struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
OwnedBy string `json:"owned_by"`
}
Model is a model available to the API key.
type ModelID ¶
type ModelID string
ModelID names a model, such as "avelin-pro". Constants for the public catalog are generated into models_gen.go; the legacy names are below.
const ( ModelCoding ModelID = "avelin-coding" // canonical: ModelCodingFast ModelCodingPlus ModelID = "avelin-coding-plus" // canonical: ModelCodingPro ModelCodingArchitect ModelID = "avelin-coding-architect" // canonical: ModelCodingUltra ModelAgentic ModelID = "avelin-agentic" // canonical: ModelAgenticPro ModelAgenticHigh ModelID = "avelin-agentic-high" // canonical: ModelAgenticUltra )
Legacy model names. AVELIN's docs call them aliases of the canonical names, but keys created in AVELIN's dashboard are scoped by these names: such a key accepts ModelCoding and ModelCodingArchitect and rejects ModelCodingFast and ModelCodingUltra with 403. GET /v1/models lists the names a key can use.
const ( // ModelAgenticFast is Avelin Agentic Fast: 256000-token context, input: text. ModelAgenticFast ModelID = "avelin-agentic-fast" // ModelAgenticPro is Avelin Agentic Pro: 256000-token context, input: text. ModelAgenticPro ModelID = "avelin-agentic-pro" // ModelAgenticUltra is Avelin Agentic Ultra: 256000-token context, input: text. ModelAgenticUltra ModelID = "avelin-agentic-ultra" // ModelCodingFast is Avelin Coding Fast: 1000000-token context, input: text. ModelCodingFast ModelID = "avelin-coding-fast" // ModelCodingPro is Avelin Coding Pro: 1000000-token context, input: text. ModelCodingPro ModelID = "avelin-coding-pro" // ModelCodingUltra is Avelin Coding Ultra: 1000000-token context, input: text. ModelCodingUltra ModelID = "avelin-coding-ultra" // ModelFast is Avelin Fast: 256000-token context, input: text, image. ModelFast ModelID = "avelin-fast" // ModelPro is Avelin Pro: 256000-token context, input: text, image. ModelPro ModelID = "avelin-pro" // ModelUltra is Avelin Ultra: 256000-token context, input: text, image. ModelUltra ModelID = "avelin-ultra" )
Models in the public catalog (https://api.avelin.ai/public/models.json).
const ModelBGEM3 ModelID = "bge-m3"
ModelBGEM3 is the embeddings model AVELIN documents: 1024 dimensions, up to 8192 input tokens. It is not listed in the public catalog.
type ModelList ¶
type ModelList struct {
Object string `json:"object"`
Data []Model `json:"data"`
// Meta is the HTTP status and headers of the response.
Meta Meta `json:"-"`
// Raw is the full response body, for fields this package does not model.
Raw json.RawMessage `json:"-"`
}
ModelList is the response of ListModels.
type Option ¶
type Option func(*Client)
Option configures a Client.
func WithAPIKey ¶
WithAPIKey sets the API key. It overrides the AVELIN_API_KEY environment variable.
func WithBaseURL ¶
WithBaseURL sets the API root, for example a local mock server. It overrides the AVELIN_BASE_URL environment variable. Do not include /v1.
func WithHTTPClient ¶
WithHTTPClient sets the HTTP client used for requests. Its Timeout, if set, also cuts off long streams; prefer WithTimeout.
func WithMaxRetries ¶
WithMaxRetries sets how many times a request is retried after a 429 or 5xx response. Zero disables retries.
func WithTimeout ¶
WithTimeout bounds each HTTP attempt. For regular calls it covers the whole response. For streams it stops once a successful response starts, so a long stream is limited by its context alone. Zero disables the timeout.
type PromptTokensDetails ¶ added in v0.2.0
type PromptTokensDetails struct {
CachedTokens int `json:"cached_tokens"`
TextTokens int `json:"text_tokens"`
}
PromptTokensDetails splits prompt tokens by kind.
type Stream ¶
type Stream[T any] struct { // contains filtered or unexported fields }
Stream iterates over a streamed response. It is not safe for concurrent use.
defer stream.Close()
for stream.Next() {
event := stream.Current()
}
if err := stream.Err(); err != nil {
// handle err
}
A stream that ends without its terminating event ([DONE] for chat, message_stop for messages), or in the middle of an event, reports an error wrapping io.ErrUnexpectedEOF. Events with empty data, such as keep-alives, are skipped. Streams are never retried once the response has started.
func (*Stream[T]) Current ¶
func (s *Stream[T]) Current() T
Current returns the event read by the last successful call to Next.
func (*Stream[T]) Err ¶
Err returns the error that stopped the stream, or nil if it completed or was closed by the caller.
type StreamOptions ¶ added in v0.2.0
type StreamOptions struct {
// IncludeUsage asks for a final chunk that carries Usage. It is not in
// AVELIN's docs but works on the live API.
IncludeUsage bool `json:"include_usage"`
}
StreamOptions configures a chat completion stream.
type Thinking ¶
type Thinking struct {
// Type is "enabled" or "disabled". AVELIN documents only "disabled".
Type string `json:"type"`
// BudgetTokens caps thinking tokens for Type "enabled". It comes from the
// Anthropic format and works on the live API, though AVELIN's reference
// does not document it.
BudgetTokens int `json:"budget_tokens,omitempty"`
}
Thinking configures extended thinking.
type Tool ¶
type Tool struct {
// Type is "function".
Type string `json:"type"`
Function FunctionDefinition `json:"function"`
}
Tool is a function the model may call.
type ToolCall ¶
type ToolCall struct {
// Index is the call's position in the response; in streams it ties
// fragments of one call together. AVELIN sends it in plain responses too.
Index *int `json:"index,omitempty"`
ID string `json:"id,omitempty"`
Type string `json:"type,omitempty"`
Function FunctionCall `json:"function"`
}
ToolCall is a function call requested by the model.
type ToolChoiceFunction ¶ added in v0.2.0
type ToolChoiceFunction struct {
Name string
}
ToolChoiceFunction forces a call to the named function when used as ChatCompletionRequest.ToolChoice. AVELIN reports such a call with FinishReason "stop", not "tool_calls". Set ToolChoice back to nil before sending the tool result, or the model is forced to call the function again.
func (ToolChoiceFunction) MarshalJSON ¶ added in v0.2.0
func (f ToolChoiceFunction) MarshalJSON() ([]byte, error)
MarshalJSON encodes {"type":"function","function":{"name":...}}.
type Usage ¶
type Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
// CompletionTokensDetails splits CompletionTokens into reasoning and
// text.
CompletionTokensDetails CompletionTokensDetails `json:"completion_tokens_details"`
// PromptTokensDetails reports how many prompt tokens came from the
// prompt cache.
PromptTokensDetails PromptTokensDetails `json:"prompt_tokens_details"`
CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
CacheReadInputTokens int `json:"cache_read_input_tokens"`
}
Usage reports token counts. Only the first three fields are documented by AVELIN; the rest appear in live chat responses and are zero for embeddings.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
avelin-models
command
Command avelin-models prints AVELIN's public model catalog as a table.
|
Command avelin-models prints AVELIN's public model catalog as a table. |
|
mockserver
command
Command mockserver serves canned AVELIN API responses from testdata so the examples and cmd/avelin-models run without an API key or network.
|
Command mockserver serves canned AVELIN API responses from testdata so the examples and cmd/avelin-models run without an API key or network. |
|
examples
|
|
|
chat
command
Command chat sends one chat completion request and prints the reply.
|
Command chat sends one chat completion request and prints the reply. |
|
chat-tools
command
Command chat-tools runs a tool-calling round trip on the chat completions endpoint: the model asks for get_weather, the program runs it and sends the result back, and the model answers.
|
Command chat-tools runs a tool-calling round trip on the chat completions endpoint: the model asks for get_weather, the program runs it and sends the result back, and the model answers. |
|
messages
command
Command messages calls the Anthropic-compatible messages endpoint and prints the model's thinking to stderr and its answer to stdout.
|
Command messages calls the Anthropic-compatible messages endpoint and prints the model's thinking to stderr and its answer to stdout. |
|
messages-tools
command
Command messages-tools runs a tool-use round trip on the Anthropic-style messages endpoint: the model asks for get_weather, the program runs it and sends a tool_result back, and the model answers.
|
Command messages-tools runs a tool-use round trip on the Anthropic-style messages endpoint: the model asks for get_weather, the program runs it and sends a tool_result back, and the model answers. |
|
stream
command
Command stream streams a chat completion, printing the answer as it arrives and collecting the whole completion with Accumulate.
|
Command stream streams a chat completion, printing the answer as it arrives and collecting the whole completion with Accumulate. |
|
internal
|
|
|
genmodels
command
Command genmodels generates models_gen.go: one avelin.ModelID constant per model in the public catalog at https://api.avelin.ai/public/models.json.
|
Command genmodels generates models_gen.go: one avelin.ModelID constant per model in the public catalog at https://api.avelin.ai/public/models.json. |
|
mockapi
Package mockapi serves canned AVELIN API responses from fixture files.
|
Package mockapi serves canned AVELIN API responses from fixture files. |