llumpy.core.AsyncModelClient

class AsyncModelClient(model, model_client)[source]

Bases: _SingleUseConversationMixIn, ABC

Abstract base class for asynchronous model clients

Parameters:

  • model (str)

  • model_client (Any)

__init__(model, model_client)[source]

Create new client

Parameters:

  • model (str) – Name of model to use

  • model_client (Any) – Async client interface to LLM

Methods

__init__(model, model_client)

Create new client

extract_text(response)

Extract text from vendor-specific response object

prompt_many(conversation, *[, handler, retries])

Prompt a model with a full conversation

prompt_one(message, *[, handler, retries])

Prompt a model with a single user message

prompt_stream(conversation, **prompt_kwargs)

Prompt a model and stream the response

system([content, file])

Init conversation with a system message

user([content, file])

Init conversation with a user message

validate()

Validate the client is ready to use

vendor_prompt(conversation, **prompt_kwargs)

Raw API call, returns vendor-specific response object

vendor_prompt_stream(conversation, ...)

Raw API call, returns vendor-specific response stream object

Attributes

model

Name of the model
abstractmethod extract_text(response)[source]

Extract text from vendor-specific response object

Parameters:

response (Any)

Return type:

str | None

property model: str

Name of the model

Type:

return

async prompt_many(conversation, *, handler=None, retries=5, **prompt_kwargs)[source]

Prompt a model with a full conversation

Parameters:

  • conversation (Conversation) – Conversation with prompt to send to LLM

  • handler (AsyncRetryHandler | None) – Optional handler to ensure the response is valid (Default: None)

  • retries (int) – Number of retries allowed (Default: 5)

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Completed chat response text or parsed object from retry handler

Return type:

Any

async prompt_one(message, *, handler=None, retries=5, **prompt_kwargs)[source]

Prompt a model with a single user message

Parameters:

  • message (str) – Message to send to LLM

  • handler (AsyncRetryHandler | None) – Optional handler to ensure the response is valid (Default: None)

  • retries (int) – Number of retries allowed (Default: 5)

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Completed chat response text or parsed object from retry handler

Return type:

Any

async prompt_stream(conversation, **prompt_kwargs)[source]

Prompt a model and stream the response

Parameters:

  • conversation (Conversation) – Messages to send to LLM

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Chat stream

Return type:

Any

abstractmethod async validate()[source]

Validate the client is ready to use

Return type:

None

abstractmethod async vendor_prompt(conversation, **prompt_kwargs)[source]

Raw API call, returns vendor-specific response object

Parameters:
Return type:

Any

abstractmethod async vendor_prompt_stream(conversation, **prompt_kwargs)[source]

Raw API call, returns vendor-specific response stream object

Parameters:
Return type:

Any