llumpy.providers.OllamaClient

class OllamaClient(model_name, model_tag=None, server_url=None)

Bases: _OllamaClientMixin, OpenAIClient

Interface for using Ollama API

Parameters:

• model_name (str)

• model_tag (str | None)

• server_url (str | None)

__init__(model_name, model_tag=None, server_url=None)

Create new Ollama Client Server URL is resolved from the param, then the OLLAMA_SERVER_URL env var, then the default

Parameters:

• model_name (str) – Name of model to use

• model_tag (str | None) – Optional model tag (Default: latest)

• server_url (str | None) – Optional URL of the ollama server (Default: http://localhost:11434)

Methods

__init__(model_name[, model_tag, server_url])	Create new Ollama Client Server URL is resolved from the param, then the OLLAMA_SERVER_URL env var, then the default
download_model([force_download])	Download a model locally from Ollama library See https://ollama.com/search for all available models
extract_text(response)	Extract LLM response text from OpenAI object
prompt_many(conversation, *[, handler, retries])	Prompt a model with a full conversation
prompt_one(message, *[, handler, retries])	Prompt a model with a single user message
prompt_stream(conversation, **prompt_kwargs)	Prompt a model and stream the response
system([content, file])	Init conversation with a system message
user([content, file])	Init conversation with a user message
validate()	Ensure the ollama server is available and model is available
vendor_prompt(conversation, **prompt_kwargs)	Prompt a model for OpenAI chat completion
vendor_prompt_stream(conversation, ...)	Prompt a model for a streamed OpenAI chat completion
wakeup()	Send a simple wakeup prompt to warm up the LLM.

Attributes

model	Name of the model
model_name	Name of model
model_tag	Tag of model

download_model(force_download=False)

Download a model locally from Ollama library See https://ollama.com/search for all available models

Parameters:

force_download (bool) – Force download even if model already downloaded (Default: False)

Raises:

• ModelNotFoundError – If could not find requested model in Ollama library

• HTTPError – If fail to download model

Return type:

None

validate()

Ensure the ollama server is available and model is available

Raises:

• InvalidOllamaServerError – If could not connect to the Ollama server

• ModelNotFoundError – If could not find requested model in Ollama library

Return type:

None

wakeup()

Send a simple wakeup prompt to warm up the LLM. Useful before actually user prompting to prevent delay on he first prompt

Raises:

ModelNotDownloadedError – If the request model has not been downloaded or available

Return type:

None