llumpy.providers.AsyncOpenAIClient

class AsyncOpenAIClient(model, api_key=None, base_url=None)

Bases: AsyncModelClient

Async client interface for interacting with OpenAI API

Parameters:

• model (str)

• api_key (str | None)

• base_url (str | None)

__init__(model, api_key=None, base_url=None)

Initialize connection to OpenAI compatible server

Parameters:

• model (str) – LLM to use

• api_key (str | None) – API key to use (Default: None)

• base_url (str | None) – URL of api server (Default: OpenAI)

Raises:

EnvironmentError – api_key is none and the ‘OPENAI_API_KEY’ env var is not defined

Methods

__init__(model[, api_key, base_url])	Initialize connection to OpenAI compatible server
extract_text(response)	Extract LLM response text from OpenAI object
prompt_many(conversation, *[, handler, retries])	Prompt a model with a full conversation
prompt_one(message, *[, handler, retries])	Prompt a model with a single user message
prompt_stream(conversation, **prompt_kwargs)	Prompt a model and stream the response
system([content, file])	Init conversation with a system message
user([content, file])	Init conversation with a user message
validate()	Verify OpenAI key is valid and has access to the requested model
vendor_prompt(conversation, **prompt_kwargs)	Prompt a model for OpenAI chat completion
vendor_prompt_stream(conversation, ...)	Prompt a model for a streamed OpenAI chat completion

Attributes

model

Name of the model

extract_text(response)

Extract LLM response text from OpenAI object

Parameters:

response (ChatCompletion | ChatCompletionChunk) – OpenAI chat response

Returns:

Text message if present, else None

Return type:

str | None

async validate()

Verify OpenAI key is valid and has access to the requested model

Raises:

• InvalidAPIKeyError – If the OpenAI key is invalid

• ModelNotFoundError – If requested model does not exist

• PermissionDeniedError – If key does not have access to requested model

Return type:

None

async vendor_prompt(conversation, **prompt_kwargs)

Prompt a model for OpenAI chat completion

Parameters:

• conversation (Conversation) – Messages to send to llm

• prompt_kwargs (Any) – kwargs for chat

Returns:

Chat completion

Return type:

ChatCompletion

async vendor_prompt_stream(conversation, **prompt_kwargs)

Prompt a model for a streamed OpenAI chat completion

Parameters:

• conversation (Conversation) – Messages to send to llm

• prompt_kwargs (Any) – kwargs for chat

Returns:

Chat stream

Return type:

AsyncStream[ChatCompletionChunk]