go-llm/docs/guide.md

go-llm guide

How to use the library. Symbol signatures and doc comments are in API. Words for the domain are in CONTEXT.md.

Adapters

Purpose

An adapter is the wire protocol a client speaks. There are two: Ollama native and OpenAI-compatible. One client is bound to one adapter, one base URL, and an optional API key.

When to use it

Choose the adapter when constructing the client. Use the OpenAI adapter for any server that implements /v1/chat/completions and /v1/models. Use the Ollama adapter for Ollama's /api/chat and /api/tags.

Call sequence

1. Pick AdapterOpenAI or AdapterOllama.
2. Call NewClient with the adapter, the base URL, and the API key.
3. Use that client for Chat, Stream, and List Models. The client adds the path for the adapter.

A trailing slash on the base URL is removed. An empty API key sends no Authorization header. A non-empty key is sent as Bearer on every request.

OpenAI paths are POST /v1/chat/completions and GET /v1/models. Ollama paths are POST /api/chat and GET /api/tags. On Ollama, max tokens are sent as num_predict.

Errors

NewClient returns an error when the adapter is not one of the two values, or when the base URL is empty or only whitespace. It does not contact the server.

Example

client, err := gollm.NewClient(gollm.AdapterOpenAI, "http://localhost:8000", "")
if err != nil {
    return err
}
_ = client

Chat

Purpose

Chat sends a list of messages and returns the assistant's text in one response.

When to use it

Use Chat when the caller wants the full reply before continuing. Use Stream when the caller wants to show text as it arrives.

Call sequence

1. Build a []Message. Each message has a Role (RoleSystem, RoleUser, or RoleAssistant) and Content.
2. Set Options.Model to the server-side model name.
3. Set Options.Temperature or Options.MaxTokens only when the call should send them. A nil pointer is omitted. On the Ollama adapter, MaxTokens is sent as num_predict.
4. Call Client.Chat with a context, the messages, and the options.
5. Use the returned string as the assistant text.

The client does not check that Model is set. The server rejects an empty model.

Errors

Chat returns an error when the context is canceled, the HTTP status is not 2xx, or the body cannot be decoded. An OpenAI response with no choices is also an error. An HTTP failure includes a short prefix of the response body.

Example

temp := 0.2
text, err := client.Chat(ctx, []gollm.Message{{
    Role:    gollm.RoleUser,
    Content: "Hello",
}}, gollm.Options{Model: "llama", Temperature: &temp})
if err != nil {
    return err
}
_ = text

Stream

Purpose

Stream sends the same request as Chat and delivers the assistant text as non-empty chunks, in order.

When to use it

Use Stream when the caller wants incremental text. The request body matches Chat, with streaming turned on.

Call sequence

1. Build messages and options the same way as Chat.
2. Call Client.Stream with a yield function. yield receives each non-empty chunk.
3. Return nil from yield to keep reading. Return an error from yield to stop the stream.

The OpenAI adapter ends the stream when a data line is the token DONE. The Ollama adapter reads one JSON object per line and yields message.content.

Errors

Stream returns the error from yield. It also returns an error when the context is canceled, the HTTP status is not 2xx, or a stream line cannot be decoded.

Example

var reply strings.Builder
err := client.Stream(ctx, []gollm.Message{{
    Role:    gollm.RoleUser,
    Content: "Hello",
}}, gollm.Options{Model: "llama"}, func(chunk string) error {
    reply.WriteString(chunk)
    return nil
})
if err != nil {
    return err
}

List Models

Purpose

List Models returns the model names the server reports.

When to use it

Use it to discover names to put in Options.Model. The client does not cache the list.

Call sequence

1. Use the same client as Chat.
2. Call Client.ListModels with a context.
3. Use the returned names. Empty names from the server are omitted.

The OpenAI adapter reads data[].id from GET /v1/models. The Ollama adapter reads models[].name from GET /api/tags.

Errors

ListModels returns an error when the context is canceled, the HTTP status is not 2xx, or the body cannot be decoded.

Example

names, err := client.ListModels(ctx)
if err != nil {
    return err
}
_ = names