Skip to main content

​
Overview

BaseChatModel is the abstract base class for all Large Language Model implementations in Qwen-Agent. It provides a unified interface for chat completions with support for function calling, streaming, and multimodal inputs.

​
Class Signature

from qwen_agent.llm.base import BaseChatModel

class BaseChatModel(ABC):
    def __init__(self, cfg: Optional[Dict] = None)

​
Constructor Parameters

​
cfg
Dict
Configuration dictionary with the following options:
  • model (str): Model name/identifier
  • api_key (str): API key for the model service
  • model_server (str): Model server endpoint
  • model_type (str): Type of model (e.g., ‘qwen_dashscope’, ‘oai’)
  • generate_cfg (dict): Generation configuration
    • max_retries (int): Maximum number of retries on failure
    • max_input_tokens (int): Maximum input context length
    • seed (int): Random seed for generation
    • temperature (float): Sampling temperature
    • top_p (float): Nucleus sampling parameter
  • cache_dir (str): Directory for caching responses

​
Properties

​
support_multimodal_input
bool
default:"False"
Whether the model supports multimodal input (images, audio, video)
​
support_multimodal_output
bool
default:"False"
Whether the model generates multimodal outputs beyond text
​
support_audio_input
bool
default:"False"
Whether the model supports audio input

​
Methods

​
chat

def chat(
    self,
    messages: List[Union[Message, Dict]],
    functions: Optional[List[Dict]] = None,
    stream: bool = True,
    delta_stream: bool = False,
    extra_generate_cfg: Optional[Dict] = None,
) -> Union[List[Message], Iterator[List[Message]]]
Main LLM chat interface.
​
messages
List[Union[Message, Dict]]
required
Input messages for the conversation
​
functions
List[Dict]
List of functions available for function calling (OpenAI format)
​
stream
bool
default:"True"
Whether to use streaming generation
​
delta_stream
bool
default:"False"
Whether to stream chunked responses (deprecated)
  • False (recommended): Stream the full response every iteration
  • True: Stream delta responses
​
extra_generate_cfg
Dict
Extra generation hyperparameters:
  • temperature (float): Sampling temperature
  • top_p (float): Nucleus sampling
  • max_tokens (int): Maximum tokens to generate
  • stop (List[str]): Stop sequences
  • function_choice (str): ‘auto’, ‘none’, or function name
​
return
Union[List[Message], Iterator[List[Message]]]
Generated message list or iterator of message lists

​
quick_chat

def quick_chat(self, prompt: str) -> str
Quick chat interface for simple text-in, text-out interactions.
​
prompt
str
required
User prompt
​
return
str
Model’s text response

​
Abstract Methods

Subclasses must implement:

​
_chat_stream

@abstractmethod
def _chat_stream(
    self,
    messages: List[Message],
    delta_stream: bool,
    generate_cfg: dict,
) -> Iterator[List[Message]]
Implements streaming chat generation.

​
_chat_no_stream

@abstractmethod
def _chat_no_stream(
    self,
    messages: List[Message],
    generate_cfg: dict,
) -> List[Message]
Implements non-streaming chat generation.

​
_chat_with_functions

@abstractmethod
def _chat_with_functions(
    self,
    messages: List[Message],
    functions: List[Dict],
    stream: bool,
    delta_stream: bool,
    generate_cfg: dict,
    lang: Literal['en', 'zh'],
) -> Union[List[Message], Iterator[List[Message]]]
Implements chat with function calling.

​
Usage Example

from qwen_agent.llm import get_chat_model
from qwen_agent.llm.schema import Message

# Initialize model
llm = get_chat_model({
    'model': 'qwen-max',
    'api_key': 'your-api-key',
    'model_type': 'qwen_dashscope'
})

# Simple chat
messages = [
    Message(role='system', content='You are a helpful assistant.'),
    Message(role='user', content='What is the capital of France?')
]

# Streaming generation
for response in llm.chat(messages=messages, stream=True):
    print(response[-1].content)

# Non-streaming
response = llm.chat(messages=messages, stream=False)
print(response[-1].content)

# Quick chat
response = llm.quick_chat('Hello!')
print(response)

​
Function Calling Example

functions = [
    {
        'name': 'get_weather',
        'description': 'Get weather for a location',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {
                    'type': 'string',
                    'description': 'City name'
                }
            },
            'required': ['location']
        }
    }
]

messages = [Message(role='user', content='What is the weather in Paris?')]

for response in llm.chat(messages=messages, functions=functions):
    if response[-1].function_call:
        print(f"Function: {response[-1].function_call.name}")
        print(f"Arguments: {response[-1].function_call.arguments}")

​
Error Handling

from qwen_agent.llm.base import ModelServiceError

try:
    response = llm.chat(messages=messages)
except ModelServiceError as e:
    print(f"Error code: {e.code}")
    print(f"Error message: {e.message}")

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us