Chat

A chat for a continuous conversation with an LLM.

from llumpy.core import AsyncChat
from llumpy.providers import AsyncOllamaClient

chat = AsyncChat(AsyncOllamaClient('llama3', '8b'))

print((await chat.message("Hello World!")))
print((await chat.message("What was my last message")))
# 'Your last message was: "Hello World!"'

To get the chat history, use the history property:

from llumpy.core import AsyncChat
from llumpy.providers import AsyncOllamaClient

chat = AsyncChat(AsyncOllamaClient('llama3', '8b'))
print(chat.history)

Streaming Response

Stream token responses from LLM instead of waiting for complete response.

from llumpy.core import AsyncChat
from llumpy.providers import AsyncOllamaClient

llama3_8b = AsyncOllamaClient('llama3', '8b')
chat = AsyncChat(llama3_8b)
async for chunk in chat.stream("Hello World"):
    print(llama3_8b.extract_text(chunk) , end="_", flush=True)

See Chat Options for advanced usage.