Other LLM Providers

Beyond Azure OpenAI, OpenAI, and Anthropic, Microsoft Agent Framework supports several additional providers including AWS Bedrock, Ollama for local models, GitHub Copilot, and more.

Supported Providers

AWS Bedrock

Access models via Amazon Bedrock

Ollama

Run models locally with Ollama

GitHub Copilot

Use GitHub Copilot models

Azure AI Foundry Local

Local model inference via Foundry

AWS Bedrock

AWS Bedrock provides access to foundation models from multiple providers through a single API.

Installation

pip install agent-framework --pre
pip install agent-framework-bedrock

Authentication

Bedrock uses AWS credentials:

Environment Variables
AWS Profiles

AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_SESSION_TOKEN=your-session-token  # Optional
BEDROCK_REGION=us-east-1
BEDROCK_CHAT_MODEL_ID=anthropic.claude-3-sonnet-20240229-v1:0

AWS_PROFILE=your-profile-name
BEDROCK_REGION=us-east-1

Basic Usage

import asyncio
from agent_framework import Agent
from agent_framework.amazon import BedrockChatClient

async def main():
    # Create agent with Bedrock
    agent = Agent(
        client=BedrockChatClient(),
        instructions="You are a helpful assistant.",
        name="BedrockAgent",
    )
    
    result = await agent.run("What is the capital of France?")
    print(result.text)

asyncio.run(main())

Available Models

Bedrock provides access to models from multiple providers:

Provider	Model ID	Best For
Anthropic	anthropic.claude-3-sonnet-20240229-v1:0	General purpose
Anthropic	anthropic.claude-3-haiku-20240307-v1:0	Speed and cost
Anthropic	anthropic.claude-3-opus-20240229-v1:0	Maximum capability
Meta	meta.llama3-70b-instruct-v1:0	Open source, reasoning
Amazon	amazon.titan-text-premier-v1:0	AWS-native
AI21 Labs	ai21.jamba-instruct-v1:0	Long context
Cohere	cohere.command-r-plus-v1:0	Retrieval, summarization
Mistral	mistral.mistral-large-2407-v1:0	Multilingual

Model availability varies by AWS region. Check the Bedrock documentation for details.

Configuration

from agent_framework.amazon import BedrockChatClient

client = BedrockChatClient(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    region="us-east-1",
    max_tokens=4096,
)

Function Calling

import asyncio
from typing import Annotated
from agent_framework import Agent, tool
from agent_framework.amazon import BedrockChatClient
from pydantic import Field

@tool(approval_mode="never_require")
def get_weather(city: Annotated[str, Field(description="City name")]) -> dict:
    """Get the weather for a city."""
    return {"city": city, "forecast": "72F and sunny"}

async def main():
    agent = Agent(
        client=BedrockChatClient(),
        instructions="You are a weather assistant.",
        name="WeatherAgent",
        tools=[get_weather],
    )
    
    result = await agent.run("What's the weather in Seattle?")
    print(result.text)

asyncio.run(main())

Not all Bedrock models support function calling. Claude 3 models have excellent function calling support.

Ollama

Ollama enables running large language models locally on your machine.

Installation

Install Ollama from ollama.com
Pull a model: ollama pull llama3.2
Install the framework package:

pip install agent-framework --pre
pip install agent-framework-ollama

Basic Usage

import asyncio
from agent_framework.ollama import OllamaChatClient

async def main():
    # Ollama must be running locally (ollama serve)
    client = OllamaChatClient()
    agent = client.as_agent(
        instructions="You are a helpful assistant.",
    )
    
    result = await agent.run("What is the capital of France?")
    print(result)

asyncio.run(main())

Configuration

Environment Variables
Explicit Configuration

OLLAMA_ENDPOINT=http://localhost:11434
OLLAMA_MODEL_ID=llama3.2

from agent_framework.ollama import OllamaChatClient

client = OllamaChatClient(
    endpoint="http://localhost:11434",
    model_id="llama3.2",
)

Available Models

Popular models available via Ollama:

Model	Size	Best For	Function Calling
llama3.2	3B	Fast, general purpose	✅ Limited
llama3.1	8B/70B	Reasoning, coding	✅ Limited
mistral	7B	Instruction following	⚠️ Limited
codellama	7B/13B/34B	Code generation	❌
phi3	3.8B	Small, efficient	⚠️ Limited
gemma2	9B/27B	Google’s model	⚠️ Limited
qwen2.5	0.5B-72B	Multilingual	✅ Good
deepseek-coder	6.7B/33B	Code understanding	❌

Install models with ollama pull <model-name>. Not all models support function calling - check model capabilities before using tools.

Multimodal Models

Some Ollama models support vision:

import asyncio
from agent_framework import Message
from agent_framework.ollama import OllamaChatClient

async def main():
    # Use a multimodal model like llava
    client = OllamaChatClient(model_id="llava")
    agent = client.as_agent(
        instructions="You analyze images.",
    )
    
    message = Message(
        role="user",
        text="What's in this image?",
        images=["path/to/image.jpg"],
    )
    
    result = await agent.run(message)
    print(result)

asyncio.run(main())

Multimodal models like llava and llava-phi3 support image inputs. Pull them with ollama pull llava.

GitHub Copilot

Use GitHub Copilot models through the Copilot CLI.

Installation

Install GitHub Copilot CLI
Install the framework package:

pip install agent-framework --pre
pip install agent-framework-github-copilot

Basic Usage

import asyncio
from agent_framework.github import GitHubCopilotAgent

async def main():
    agent = GitHubCopilotAgent(
        instructions="You are a helpful assistant.",
    )
    
    async with agent:
        result = await agent.run("What is the capital of France?")
        print(result)

asyncio.run(main())

Configuration

Environment Variables
Explicit Configuration

GITHUB_COPILOT_CLI_PATH=/path/to/copilot-cli
GITHUB_COPILOT_MODEL=gpt-5
GITHUB_COPILOT_TIMEOUT=30
GITHUB_COPILOT_LOG_LEVEL=info

Python

from agent_framework.github import GitHubCopilotAgent

agent = GitHubCopilotAgent(
    instructions="You are a helpful assistant.",
    model="claude-sonnet-4",
    timeout=30,
)

Available Models

GitHub Copilot provides access to multiple models:

gpt-5 - Latest GPT model
claude-sonnet-4 - Anthropic Claude
o1-preview - OpenAI reasoning model
o3-mini - Compact reasoning model

Model availability depends on your GitHub Copilot subscription and organization settings.

Azure AI Foundry Local

Run models locally via Azure AI Foundry for development and testing.

Installation

pip install agent-framework --pre
pip install agent-framework-foundry-local

Basic Usage

import asyncio
from agent_framework.foundry_local import FoundryLocalAgent

async def main():
    agent = FoundryLocalAgent(
        instructions="You are a helpful assistant.",
    )
    
    result = await agent.run("What is the capital of France?")
    print(result)

asyncio.run(main())

Choosing a Provider

Here’s guidance on when to use each provider:

Use AWS Bedrock when...

You’re already using AWS infrastructure
You need access to multiple model providers
You want managed scaling and availability
You require AWS compliance features
You need region-specific deployments

Use Ollama when...

You want to run models locally
You need offline operation
You’re concerned about data privacy
You want to avoid API costs
You’re doing local development
You need fast iteration without rate limits

Use GitHub Copilot when...

You have a GitHub Copilot subscription
You want access to multiple models through one API
You’re building developer tools
You want model selection flexibility

Use Foundry Local when...

You’re developing Azure AI Foundry applications
You need local testing before cloud deployment
You want to prototype without cloud costs
You’re working offline or in restricted environments

Provider Comparison

Feature	Bedrock	Ollama	GitHub Copilot	Foundry Local
Cost	$$	Free (local)	$ (subscription)	Free (local)
Internet Required	✅	❌	✅	❌
Setup Complexity	Medium	Low	Low	Medium
Model Selection	Multiple providers	Large catalog	Multiple	Limited
Function Calling	✅ Model dependent	⚠️ Limited	✅	⚠️ Limited
Streaming	✅	✅	✅	✅
Production Ready	✅	⚠️ Depends	✅	❌ Dev only

Best Practices

AWS Bedrock

Use IAM roles for authentication in production
Enable CloudWatch logging for debugging
Choose region based on data residency requirements
Monitor costs - different models have different pricing
Test model availability in your target region

Ollama

Ensure sufficient RAM for your chosen model
Use GPU acceleration when available
Keep Ollama updated for latest models
Test model capabilities before production use
Not all models support function calling
Consider model size vs. quality tradeoffs

GitHub Copilot

Verify your organization allows Copilot use
Check model availability for your subscription
Monitor token usage
Implement retry logic for rate limits
Test fallback to other providers

Foundry Local

Only use for development and testing
Transition to cloud for production
Test with same models as production
Monitor resource usage
Keep dependencies updated

Troubleshooting

Bedrock Connection Issues

Verify AWS credentials are configured correctly
Check IAM permissions for Bedrock access
Ensure the model is available in your region
Verify network connectivity to AWS
Check CloudWatch logs for detailed errors

Ollama Not Responding

Verify Ollama is running: ollama serve
Check if the model is pulled: ollama list
Verify endpoint URL (default: http://localhost:11434)
Check system resources (RAM, GPU)
Review Ollama logs for errors

GitHub Copilot Errors

Verify Copilot CLI is installed
Check authentication: gh auth status
Verify subscription is active
Check model availability
Review CLI logs for details

Next Steps

Provider Comparison

Compare all available providers

Function Tools

Add function calling capabilities

Workflows

Build multi-agent workflows

Hosting & Deployment

Deploy agents to production

Get Started

Core Concepts

Workflows

Providers

Hosting & Deployment

Migration Guides

​Other LLM Providers

​Supported Providers

AWS Bedrock

Ollama

GitHub Copilot

Azure AI Foundry Local

​AWS Bedrock

​Installation

​Authentication

​Basic Usage

​Available Models

​Configuration

​Function Calling

​Ollama

​Installation

​Basic Usage

​Configuration

​Available Models

​Multimodal Models

​GitHub Copilot

​Installation

​Basic Usage

​Configuration

​Available Models

​Azure AI Foundry Local

​Installation

​Basic Usage

​Choosing a Provider

​Provider Comparison

​Best Practices

​Troubleshooting

​Next Steps

Provider Comparison

Function Tools

Workflows

Hosting & Deployment

Build docs developers (and LLMs) love

Other LLM Providers

Supported Providers

AWS Bedrock

Installation

Authentication

Basic Usage

Available Models

Configuration

Function Calling

Ollama

Installation

Basic Usage

Configuration

Available Models

Multimodal Models

GitHub Copilot

Installation

Basic Usage

Configuration

Available Models

Azure AI Foundry Local

Installation

Basic Usage

Choosing a Provider

Provider Comparison

Best Practices

Troubleshooting

Next Steps