Provider Clients

Anthropic, OpenAI, and Ollama clients have been developed and are ready to use out of the box. For custom clients, see here.

Warning

Clients will fail to be initialized if API keys are invalid, models do not exist, or the key does not have access to the requested model.

The ProviderFactory is the main means of generating clients and handles any validation or additional setup. Clients instantiated without the factory, but do not have the same guarantees as ones created with the factory.

Anthropic

Factory

from llumpy.providers import ProviderFactory

# API key env var: ANTHROPIC_API_KEY
claude = await ProviderFactory.async_anthropic('claude-sonnet-4-6')

Standalone

from llumpy.providers import AsyncAnthropicClient

claude = AsyncAnthropicClient('claude-sonnet-4-6')
await claude.validate()

The ANTHROPIC_API_KEY env variable MUST be set. Anthropic models can be found here.

OpenAI

Factory

from llumpy.providers import ProviderFactory

# API key env var: OPENAI_API_KEY
gpt = await ProviderFactory.async_openai('gpt-5.4')
await gpt.validate()

Standalone

from llumpy.providers import AsyncOpenAIClient

gpt = AsyncOpenAIClient('gpt-5.4')
await gpt.validate()

The OPENAI_API_KEY env variable MUST be set. OpenAI models can be found here.

Ollama

Factory

from llumpy.providers import ProviderFactory

# Ollama server url env var: OLLAMA_SERVER_URL (Default: http://localhost:11434)
llama3_latest = await ProviderFactory.async_ollama('llama3')  # default ':latest'
llama3_8b = await ProviderFactory.async_ollama('llama3', '8b')

The ProviderFactory also handles downloading the model if it has not been download locally. To skip this step, use the defer_download param:

from llumpy.providers import ProviderFactory

llama3_latest = await ProviderFactory.async_ollama('llama3', defer_download=True)

If the model is not downloaded, the download_model() method MUST be called before prompting to ensure the model is downloaded. Deferring is useful since models can take a while to download, so the download task can be done in the background while other code is running.

The AsyncOllamaClient also supports a wake_up() method that sends a short message to “warm up” the LLM so the first actual prompt does not take a long time if this is a fresh running Ollama instance. To disable this, use the skip_wakeup param:

from llumpy.providers import ProviderFactory

llama3_latest = await ProviderFactory.async_ollama('llama3', skip_wakeup=True)

This is often used with the defer_download param since if the model has not been downloaded, a ModelNotDownloadedError will be thrown.

Standalone

from llumpy.providers import AsyncOllamaClient

llama3_8b = AsyncOllamaClient('llama3', '8b')
await llama3_8b.validate()  # ensure the model exists
await llama3_8b.download_model()  # download the model if it hasn't been downloaded already

The Ollama server url will be set using the following precedence:

1. server_url param

2. OLLAMA_SERVER_URL env variable

3. Default (http://localhost:11434)

Ollama models can be found here.