Skip to main content

Azure OpenAI Provider

Azure OpenAI Service provides REST API access to OpenAI’s powerful language models including GPT-4, GPT-4o, and GPT-3.5-Turbo, with enterprise-grade security, compliance, and regional availability. The Microsoft Agent Framework provides multiple ways to work with Azure OpenAI depending on your use case.

Provider Options

Azure OpenAI integration offers three main approaches:

Responses Client

Direct chat completions with streaming and tools

Chat Client

Chat-based interactions with conversation management

Azure AI Project

Persistent agents with Azure AI Foundry projects

Installation

pip install agent-framework --pre
# Azure OpenAI support is included in the core package

Authentication

Azure OpenAI supports multiple authentication methods:
from azure.identity import AzureCliCredential
from agent_framework.azure import AzureOpenAIResponsesClient

# Run 'az login' first
credential = AzureCliCredential()
client = AzureOpenAIResponsesClient(credential=credential)
from azure.identity import ManagedIdentityCredential
from agent_framework.azure import AzureOpenAIResponsesClient

credential = ManagedIdentityCredential()
client = AzureOpenAIResponsesClient(credential=credential)
from azure.core.credentials import AzureKeyCredential
from agent_framework.azure import AzureOpenAIResponsesClient

credential = AzureKeyCredential(api_key)
client = AzureOpenAIResponsesClient(credential=credential)
API keys should only be used for local development. Always use managed identities in production for better security.

Azure OpenAI Responses Client

The Responses Client provides direct access to Azure OpenAI chat completions with support for streaming, function calling, and structured outputs.

Basic Usage

import asyncio
from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential

async def main():
    # Create agent with Azure OpenAI Responses Client
    agent = AzureOpenAIResponsesClient(
        credential=AzureCliCredential()
    ).as_agent(
        instructions="You are a helpful assistant.",
    )
    
    # Non-streaming response
    result = await agent.run("What is the capital of France?")
    print(result)

asyncio.run(main())

Configuration

AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=gpt-4o
AZURE_OPENAI_API_VERSION=2024-10-01-preview

Streaming Responses

import asyncio
from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential

async def main():
    agent = AzureOpenAIResponsesClient(
        credential=AzureCliCredential()
    ).as_agent(
        instructions="You are a helpful assistant.",
    )
    
    # Streaming response
    query = "Write a short story about AI agents."
    print("Agent: ", end="", flush=True)
    async for chunk in agent.run(query, stream=True):
        if chunk.text:
            print(chunk.text, end="", flush=True)
    print()

asyncio.run(main())

Function Calling

import asyncio
from typing import Annotated
from agent_framework import tool
from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential
from pydantic import Field

@tool(approval_mode="never_require")  # Use "always_require" in production
def get_weather(
    location: Annotated[str, Field(description="City name")],
) -> str:
    """Get the weather for a location."""
    return f"Weather in {location}: Sunny, 72°F"

async def main():
    agent = AzureOpenAIResponsesClient(
        credential=AzureCliCredential()
    ).as_agent(
        instructions="You are a weather assistant.",
        tools=[get_weather],
    )
    
    result = await agent.run("What's the weather in Seattle?")
    print(result)

asyncio.run(main())

Azure OpenAI Chat Client

The Chat Client provides chat-based interactions with conversation management.
import asyncio
from agent_framework.azure import AzureOpenAIChatClient
from azure.identity import AzureCliCredential

async def main():
    agent = AzureOpenAIChatClient(
        credential=AzureCliCredential()
    ).as_agent(
        instructions="You are a helpful assistant.",
    )
    
    result = await agent.run("Hello, how are you?")
    print(result)

asyncio.run(main())

Azure AI Project Agent Provider

For persistent agents managed through Azure AI Foundry projects, use the AzureAIProjectAgentProvider.

Setup

  1. Create an Azure AI Foundry project at ai.azure.com
  2. Get your project endpoint from the project settings
  3. Deploy a model to your project

Basic Usage

import asyncio
from agent_framework.azure import AzureAIProjectAgentProvider
from azure.identity.aio import AzureCliCredential

async def main():
    async with (
        AzureCliCredential() as credential,
        AzureAIProjectAgentProvider(credential=credential) as provider
    ):
        # Create a persistent agent
        agent = await provider.create_agent(
            name="WeatherAgent",
            instructions="You are a weather assistant.",
            tools=[...],
        )
        
        # Use the agent
        result = await agent.run("What's the weather?")
        print(result)

asyncio.run(main())

Configuration

AZURE_AI_PROJECT_ENDPOINT=https://your-project.services.ai.azure.com
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4o

Working with Existing Agents

import asyncio
from agent_framework.azure import AzureAIProjectAgentProvider
from azure.identity.aio import AzureCliCredential

async def main():
    async with (
        AzureCliCredential() as credential,
        AzureAIProjectAgentProvider(credential=credential) as provider
    ):
        # Get an existing agent by ID
        agent = await provider.get_agent(agent_id="your-agent-id")
        
        # Use the agent
        result = await agent.run("Hello!")
        print(result)

asyncio.run(main())

Available Models

Azure OpenAI supports various model families:
Model FamilyModelsBest For
GPT-4ogpt-4o, gpt-4o-miniLatest multimodal model, vision, function calling
GPT-4gpt-4, gpt-4-32kComplex reasoning, high-quality outputs
GPT-3.5gpt-35-turboFast, cost-effective chat
o1/o3o1-preview, o1-mini, o3-miniAdvanced reasoning tasks
Model availability varies by region. Check the Azure OpenAI model availability page for details.

Advanced Features

Code Interpreter

Azure OpenAI Assistants support code interpreter for Python code execution:
from agent_framework.azure import code_interpreter_tool

agent = await provider.create_agent(
    name="DataAnalyst",
    instructions="You are a data analyst.",
    tools=[code_interpreter_tool()],
)
Enable file search for RAG capabilities:
from agent_framework.azure import file_search_tool

agent = await provider.create_agent(
    name="DocumentAssistant",
    instructions="You help with documents.",
    tools=[file_search_tool()],
)

Structured Outputs

from pydantic import BaseModel

class WeatherResponse(BaseModel):
    location: str
    temperature: float
    condition: str

agent = client.as_agent(
    instructions="You are a weather assistant.",
    response_format=WeatherResponse,
)

Best Practices

Always use Azure managed identities for authentication in production environments. This eliminates the need to store API keys and provides better security through Azure AD.
from azure.identity import ManagedIdentityCredential
credential = ManagedIdentityCredential()
  • Use Responses Client for simple chat completions and streaming
  • Use Chat Client when you need conversation management
  • Use Azure AI Project Provider for persistent agents with managed state
Azure OpenAI has rate limits. Implement retry logic with exponential backoff for production applications.
Different models have different pricing. Use GPT-4o-mini for development and testing, and reserve GPT-4o for production workloads that require maximum quality.
Deploy models in regions close to your users for better latency. Azure OpenAI is available in multiple regions worldwide.

Troubleshooting

If you see authentication errors:
  1. Run az login to authenticate with Azure CLI
  2. Ensure your account has access to the Azure OpenAI resource
  3. Check that the correct subscription is selected: az account show
  4. Verify RBAC roles: You need “Cognitive Services OpenAI User” or higher
Azure OpenAI has rate limits based on your deployment:
  1. Check your quota in the Azure Portal
  2. Implement exponential backoff retry logic
  3. Consider requesting a quota increase
  4. Use multiple deployments to distribute load
If you get model not found errors:
  1. Verify the deployment name matches exactly
  2. Check that the model is deployed in your resource
  3. Ensure you’re using the correct endpoint
  4. Verify the API version is compatible with the model

Next Steps

Function Tools

Learn how to add function calling to your agents

Sessions & Memory

Manage multi-turn conversations and sessions

Workflows

Build complex multi-agent workflows

Deploy to Azure

Deploy your agents to Azure Functions

Build docs developers (and LLMs) love