Skip to main content

​
Overview

TypeAgent uses pydantic-ai for provider-agnostic model configuration. Support for 25+ providers including OpenAI, Anthropic, Google, Azure, AWS Bedrock, Groq, Mistral, Ollama, and more. Spec Format: provider:model (e.g., "openai:gpt-4o", "anthropic:claude-sonnet-4-20250514")

​
Quick Start

from typeagent.aitools.model_adapters import (
    create_chat_model,
    create_embedding_model,
    configure_models
)

# Create individual models
chat = create_chat_model("openai:gpt-4o")
embedder = create_embedding_model("openai:text-embedding-3-small")

# Or configure both at once
chat, embedder = configure_models(
    "openai:gpt-4o",
    "openai:text-embedding-3-small"
)

​
create_chat_model

def create_chat_model(
    model_spec: str | None = None,
) -> PydanticAIChatModel
Create a chat model from a provider:model specification.
​
model_spec
str | None
default:"None"
Model specification in format "provider:model". If None, uses OPENAI_MODEL environment variable or defaults to "openai:gpt-4o".Examples:
  • "openai:gpt-4o"
  • "anthropic:claude-sonnet-4-20250514"
  • "google:gemini-2.0-flash"
  • "groq:llama-3.3-70b-versatile"
​
model
PydanticAIChatModel
Chat model adapter implementing TypeChat’s TypeChatLanguageModel interface.

​
Azure OpenAI Auto-Detection

If model_spec uses "openai:" as the provider but OPENAI_API_KEY is not set and AZURE_OPENAI_API_KEY is available, automatically switches to Azure OpenAI. 
# With AZURE_OPENAI_API_KEY set and no OPENAI_API_KEY:
model = create_chat_model("openai:gpt-4o")
# Uses Azure OpenAI automatically

​
Examples

# Use environment variable or default
model = create_chat_model()  # Uses OPENAI_MODEL or "openai:gpt-4o"

# Explicit model selection
openai_model = create_chat_model("openai:gpt-4o")
claude_model = create_chat_model("anthropic:claude-sonnet-4-20250514")
gemini_model = create_chat_model("google:gemini-2.0-flash")
ollama_model = create_chat_model("ollama:llama3.2")

​
create_embedding_model

def create_embedding_model(
    model_spec: str | None = None,
) -> CachingEmbeddingModel
Create an embedding model from a provider:model specification.
​
model_spec
str | None
default:"None"
Embedding model specification. If None, uses OPENAI_EMBEDDING_MODEL environment variable or defaults to "openai:text-embedding-ada-002".Examples:
  • "openai:text-embedding-3-small"
  • "openai:text-embedding-3-large"
  • "cohere:embed-english-v3.0"
  • "google:text-embedding-004"
​
model
CachingEmbeddingModel
Embedding model with automatic caching wrapping a PydanticAIEmbedder.

​
Azure OpenAI Auto-Detection

Same auto-detection as chat models. Additionally supports model-specific Azure endpoints: 
# Fallback endpoint (any model)
export AZURE_OPENAI_ENDPOINT_EMBEDDING="https://...openai.azure.com/openai/deployments/.../embeddings?..."

# Model-specific endpoints (preferred)
export AZURE_OPENAI_ENDPOINT_EMBEDDING_3_SMALL="https://..."
export AZURE_OPENAI_ENDPOINT_EMBEDDING_3_LARGE="https://..."

​
Examples

# Use environment variable or default
model = create_embedding_model()  # Uses OPENAI_EMBEDDING_MODEL or ada-002

# Explicit model selection
small_model = create_embedding_model("openai:text-embedding-3-small")
large_model = create_embedding_model("openai:text-embedding-3-large")
cohere_model = create_embedding_model("cohere:embed-english-v3.0")
gemini_model = create_embedding_model("google:text-embedding-004")

​
configure_models

def configure_models(
    chat_model_spec: str,
    embedding_model_spec: str,
) -> tuple[PydanticAIChatModel, CachingEmbeddingModel]
Configure both a chat model and an embedding model at once.
​
chat_model_spec
str
required
Chat model specification ("provider:model").
​
embedding_model_spec
str
required
Embedding model specification ("provider:model").
​
models
tuple[PydanticAIChatModel, CachingEmbeddingModel]
Tuple of (chat_model, embedding_model).

​
Example

from typeagent.aitools.model_adapters import configure_models
from typeagent.knowpro.convsettings import ConversationSettings
from typeagent.knowpro.convknowledge import KnowledgeExtractor

# Configure both models
chat, embedder = configure_models(
    "openai:gpt-4o",
    "openai:text-embedding-3-small"
)

# Use in conversation settings
settings = ConversationSettings(model=embedder)

# Use in knowledge extractor
extractor = KnowledgeExtractor(model=chat)

​
PydanticAIChatModel

class PydanticAIChatModel(TypeChatLanguageModel)
Adapter from pydantic-ai’s Model to TypeChat’s TypeChatLanguageModel. Enables any pydantic-ai chat model to work with TypeChat’s structured output system.

​
Constructor

def __init__(self, model: Model) -> None
​
model
Model
required
A pydantic-ai Model instance.

​
Methods

​
complete

async def complete(
    self,
    prompt: str | list[PromptSection]
) -> Result[str]
Generate a completion from the model.
​
prompt
str | list[PromptSection]
required
Either a simple string prompt or a list of TypeChat prompt sections with roles.
​
result
Result[str]
TypeChat result (Success or Failure) containing the completion text.
Example: 
from typeagent.aitools.model_adapters import create_chat_model

model = create_chat_model("openai:gpt-4o")

# Simple prompt
result = await model.complete("What is TypeAgent?")
if isinstance(result, Success):
    print(result.value)

# Multi-turn prompt
prompt = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain semantic search."}
]
result = await model.complete(prompt)

​
PydanticAIEmbedder

class PydanticAIEmbedder
Adapter from pydantic-ai’s Embedder to TypeAgent’s IEmbedder protocol.

​
Constructor

def __init__(
    self,
    embedder: _PydanticAIEmbedder,
    model_name: str,
) -> None
​
embedder
_PydanticAIEmbedder
required
A pydantic-ai Embedder instance.
​
model_name
str
required
The embedding model name.

​
Methods

Implements IEmbedder protocol:
  • get_embedding_nocache(input: str) - Compute single embedding
  • get_embeddings_nocache(input: list[str]) - Compute batch embeddings
All embeddings are automatically L2-normalized.

​
Test Utilities

​
create_test_embedding_model

def create_test_embedding_model(
    embedding_size: int = 3,
) -> CachingEmbeddingModel
Create a fake embedding model for testing. Returns deterministic embeddings without network calls.
​
embedding_size
int
default:"3"
Dimension of fake embeddings (default: 3 for easy debugging).
​
model
CachingEmbeddingModel
Test embedding model with model_name = "test".
Example: 
from typeagent.aitools.model_adapters import create_test_embedding_model
import numpy as np

# Create test model
model = create_test_embedding_model(embedding_size=8)

# Get deterministic embeddings
emb1 = await model.get_embedding("hello")
emb2 = await model.get_embedding("hello")
assert np.array_equal(emb1, emb2)  # Same input = same output

# Different inputs = different embeddings
emb3 = await model.get_embedding("world")
assert not np.array_equal(emb1, emb3)

print(f"Model name: {model.model_name}")  # "test"
Never use test embeddings in production. They are not semantically meaningful.

​
Supported Providers

Provider: openaiChat Models:
  • gpt-4o
  • gpt-4o-mini
  • gpt-4-turbo
  • gpt-3.5-turbo
Embedding Models:
  • text-embedding-3-small (1536 dims)
  • text-embedding-3-large (3072 dims)
  • text-embedding-ada-002 (1536 dims, legacy)
Required: OPENAI_API_KEY
Provider: openai (auto-detected) or azureAuto-detection: When OPENAI_API_KEY is not set but AZURE_OPENAI_API_KEY is.Required:
  • AZURE_OPENAI_API_KEY (or "identity" for managed identity)
  • AZURE_OPENAI_ENDPOINT - Full deployment URL for chat
  • AZURE_OPENAI_ENDPOINT_EMBEDDING - Full deployment URL for embeddings
See Environment Variables for URL format.
Provider: anthropicChat Models:
  • claude-sonnet-4-20250514
  • claude-opus-4-20250514
  • claude-3-5-sonnet-20241022
Required: ANTHROPIC_API_KEY
Provider: googleChat Models:
  • gemini-2.0-flash
  • gemini-1.5-pro
  • gemini-1.5-flash
Embedding Models:
  • text-embedding-004
Required: GOOGLE_API_KEY or GOOGLE_APPLICATION_CREDENTIALS
Provider: bedrockChat Models:
  • anthropic.claude-3-5-sonnet-20241022-v2:0
  • meta.llama3-3-70b-instruct-v1:0
Required: AWS credentials (boto3 configuration)
Provider: groqChat Models:
  • llama-3.3-70b-versatile
  • mixtral-8x7b-32768
Required: GROQ_API_KEY
Provider: ollamaChat Models: Any locally installed model
  • llama3.2
  • mistral
  • codellama
Embedding Models:
  • nomic-embed-text
  • mxbai-embed-large
Required: Ollama running locally (default: http://localhost:11434)
Additional providers supported by pydantic-ai:
  • Cohere - cohere:command-r-plus, cohere:embed-english-v3.0
  • Mistral AI - mistral:mistral-large-latest
  • Databricks - databricks:...
  • Vertex AI - vertexai:gemini-pro
  • And more…
See pydantic-ai models for the complete list.

​
Environment Variables

​
OpenAI

export OPENAI_API_KEY="sk-..."
export OPENAI_MODEL="gpt-4o"  # Optional, for chat
export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"  # Optional, for embeddings

​
Azure OpenAI

export AZURE_OPENAI_API_KEY="..."  # Or "identity" for managed identity
export AZURE_OPENAI_ENDPOINT="https://RESOURCE.openai.azure.com/openai/deployments/DEPLOYMENT/chat/completions?api-version=2023-05-15"
export AZURE_OPENAI_ENDPOINT_EMBEDDING="https://RESOURCE.openai.azure.com/openai/deployments/EMBEDDING_DEPLOYMENT/embeddings?api-version=2024-08-01-preview"

​
Other Providers

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# Google
export GOOGLE_API_KEY="..."

# Groq
export GROQ_API_KEY="gsk_..."

# Cohere
export COHERE_API_KEY="..."
See Environment Variables for complete reference.

Build docs developers (and LLMs) love

Get started for freeTalk to us