OpenAI Provider
The OpenAI provider gives you direct access to OpenAI’s latest models including GPT-4o, GPT-4, o1, o3, and GPT-3.5-Turbo. Use this provider when you want to work directly with OpenAI’s API rather than through Azure.Provider Options
OpenAI integration offers multiple approaches:Chat Client
Direct chat completions with streaming
Responses Client
Structured responses with tools
Assistants API
Persistent agents with code interpreter and file search
Installation
Authentication
OpenAI uses API keys for authentication:Environment Variable (Recommended)
.env
Explicit API Key
OpenAI Chat Client
The Chat Client provides direct access to OpenAI chat completions.Basic Usage
Configuration
- Environment Variables
- Explicit Configuration
Streaming Responses
Function Calling
OpenAI Responses Client
The Responses Client provides structured response generation:OpenAI Assistants API
The Assistants API provides persistent agents with managed state, code interpreter, and file search capabilities.Creating an Assistant
Using an Existing Assistant
Code Interpreter
Enable code interpreter for Python code execution:File Search
Enable file search for RAG capabilities:Available Models
OpenAI offers several model families:| Model | Context Window | Best For | Function Calling |
|---|---|---|---|
| gpt-4o | 128k | Latest flagship, multimodal, vision | ✅ |
| gpt-4o-mini | 128k | Fast, cost-effective, intelligent | ✅ |
| gpt-4-turbo | 128k | High performance, vision | ✅ |
| gpt-4 | 8k | Complex reasoning | ✅ |
| gpt-3.5-turbo | 16k | Fast, economical | ✅ |
| o1 | 200k | Advanced reasoning | ❌ |
| o3-mini | 200k | Cost-effective reasoning | ❌ |
Reasoning models (o1, o3) do not support function calling or streaming. They are optimized for complex reasoning tasks.
Reasoning Models
OpenAI’s o1 and o3 models provide advanced reasoning capabilities:Vision Capabilities
GPT-4o and GPT-4-turbo support image inputs:Structured Outputs
Force the model to return structured JSON outputs:Best Practices
Secure API Key Management
Secure API Key Management
Never hardcode API keys in your source code:
- Use environment variables for API keys
- Use
.envfiles locally (add to.gitignore) - Use Azure Key Vault or similar for production
- Rotate API keys regularly
- Use project-scoped keys when possible
Choose the Right Model
Choose the Right Model
- Use gpt-4o-mini for development and testing
- Use gpt-4o for production workloads requiring maximum quality
- Use gpt-3.5-turbo for simple tasks where cost is critical
- Use o1/o3 for complex reasoning tasks that don’t need function calling
Handle Rate Limits
Handle Rate Limits
OpenAI has rate limits based on your tier:
- Implement exponential backoff retry logic
- Monitor usage in the OpenAI dashboard
- Consider upgrading your tier for higher limits
- Use batch processing for high-volume workloads
Optimize Token Usage
Optimize Token Usage
Tokens directly impact cost:
- Keep system prompts concise but clear
- Use smaller models when appropriate
- Implement conversation pruning for long sessions
- Monitor token usage in responses
- Use structured outputs to reduce verbose responses
Clean Up Resources
Clean Up Resources
When using the Assistants API:
- Delete assistants when no longer needed
- Clean up uploaded files
- Delete old threads to manage costs
- Use try-finally blocks to ensure cleanup
Troubleshooting
Invalid API Key
Invalid API Key
If you see API key errors:
- Verify the API key is correct and active
- Check if the key has expired
- Ensure the key has proper permissions
- Verify organization ID if using organization keys
Rate Limit Errors
Rate Limit Errors
If you’re hitting rate limits:
- Check your usage tier at platform.openai.com
- Implement exponential backoff
- Reduce request frequency
- Consider upgrading to a higher tier
- Use batch API for non-urgent requests
Model Not Available
Model Not Available
If the model is not available:
- Check if you have access to the model
- Verify the model name is correct
- Some models require waitlist access
- Check if your account tier supports the model
Function Calling Not Working
Function Calling Not Working
If function calling isn’t working:
- Verify the model supports function calling (o1/o3 don’t)
- Check function schema is valid JSON Schema
- Ensure function descriptions are clear
- Verify parameter types match the schema
Cost Optimization
OpenAI pricing varies by model. Here are tips to optimize costs:- Use appropriate models: Don’t use GPT-4o when GPT-4o-mini or GPT-3.5-turbo will suffice
- Monitor token usage: Track input and output tokens to identify optimization opportunities
- Optimize prompts: Shorter, clearer prompts use fewer tokens
- Cache common responses: Cache responses for frequently asked questions
- Use batch API: For non-time-sensitive workloads, batch API is cheaper
Check current pricing at openai.com/pricing
Next Steps
Function Tools
Add function calling capabilities to your agents
Sessions & Memory
Manage multi-turn conversations with memory
Tools
Add function calling to your agents
Workflows
Build multi-agent workflows