llumpy.core.ModelClient

class ModelClient(model, vendor_client)[source]

Bases: _SingleUseConversationMixIn, ABC

Abstract base class for synchronous model clients

Parameters:

  • model (str)

  • vendor_client (Any)

__init__(model, vendor_client)[source]

Create new client

Parameters:

  • model (str) – Name of model to use

  • vendor_client (Any) – Vendor specific client interface to LLM

Methods

__init__(model, vendor_client)

Create new client

extract_text(response)

Extract text from vendor-specific response object

prompt_many(conversation, *[, handler, retries])

Prompt a model with a full conversation

prompt_one(message, *[, handler, retries])

Prompt a model with a single user message

prompt_stream(conversation, **prompt_kwargs)

Prompt a model and stream the response

system([content, file])

Init conversation with a system message

user([content, file])

Init conversation with a user message

validate()

Validate the client is ready to use

vendor_prompt(conversation, **prompt_kwargs)

Raw API call, returns vendor-specific response object

vendor_prompt_stream(conversation, ...)

Raw API call, returns vendor-specific response stream object

Attributes

model

Name of the model
abstractmethod extract_text(response)[source]

Extract text from vendor-specific response object

Parameters:

response (Any)

Return type:

str | None

property model: str

Name of the model

Type:

return

prompt_many(conversation, *, handler=None, retries=5, **prompt_kwargs)[source]

Prompt a model with a full conversation

Parameters:

  • conversation (Conversation) – Conversation with prompt to send to LLM

  • handler (RetryHandler | None) – Optional handler to ensure the response is valid (Default: None)

  • retries (int) – Number of retries allowed (Default: 5)

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Completed chat response text or parsed object from retry handler

Return type:

Any

prompt_one(message, *, handler=None, retries=5, **prompt_kwargs)[source]

Prompt a model with a single user message

Parameters:

  • message (str) – Message to send to LLM

  • handler (RetryHandler | None) – Optional handler to ensure the response is valid (Default: None)

  • retries (int) – Number of retries allowed (Default: 5)

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Completed chat response text or parsed object from retry handler

Return type:

Any

prompt_stream(conversation, **prompt_kwargs)[source]

Prompt a model and stream the response

Parameters:

  • conversation (Conversation) – Messages to send to LLM

  • prompt_kwargs (Any) – kwargs for chat

Returns:

Chat stream

Return type:

Any

abstractmethod validate()[source]

Validate the client is ready to use

Return type:

None

abstractmethod vendor_prompt(conversation, **prompt_kwargs)[source]

Raw API call, returns vendor-specific response object

Parameters:
Return type:

Any

abstractmethod vendor_prompt_stream(conversation, **prompt_kwargs)[source]

Raw API call, returns vendor-specific response stream object

Parameters:
Return type:

Any