Configuration for chat models.
Usage
import marimo as mo
# Create a basic configuration
config = mo.ai.ChatModelConfig(
max_tokens=1000,
temperature=0.7
)
# Create a configuration with all parameters
config = mo.ai.ChatModelConfig(
max_tokens=2000,
temperature=0.8,
top_p=0.9,
top_k=50,
frequency_penalty=0.5,
presence_penalty=0.5
)
Signature
mo.ai.ChatModelConfig(
max_tokens: Optional[int] = None,
temperature: Optional[float] = None,
top_p: Optional[float] = None,
top_k: Optional[int] = None,
frequency_penalty: Optional[float] = None,
presence_penalty: Optional[float] = None
)
Parameters
max_tokens
Optional[int]
default:"None"
Maximum number of tokens to generate.
temperature
Optional[float]
default:"None"
Temperature for the model (randomness). Higher values make the output more random, lower values make it more deterministic.
top_p
Optional[float]
default:"None"
Restriction on the cumulative probability of prediction candidates. Also known as nucleus sampling.
top_k
Optional[int]
default:"None"
Number of top prediction candidates to consider.
frequency_penalty
Optional[float]
default:"None"
Penalty for tokens which appear frequently. Positive values decrease the likelihood of repeating tokens based on their frequency in the text so far.
presence_penalty
Optional[float]
default:"None"
Penalty for tokens which already appeared at least once. Positive values encourage the model to use new tokens.
Usage with LLM Providers
This configuration is typically used with LLM provider functions:
import marimo as mo
config = mo.ai.ChatModelConfig(
max_tokens=1000,
temperature=0.7
)
messages = [
mo.ai.ChatMessage(role="user", content="Hello!", id="1")
]
# Use with a provider (example)
# response = mo.ai.llm.openai(messages, config)