Skip to main content
This guide walks you through configuring ElevenLabs Conversational AI to enable outbound phone calling functionality in the AI Voice Testing Platform.

​
Overview

The platform uses ElevenLabs’ hosted outbound-call endpoint: 
POST /v1/convai/twilio/outbound-call
This endpoint allows the application to initiate phone calls that use the same AI caller persona configured in your web chat, maintaining consistency across both testing modes.
Phone calls are made through ElevenLabs’ Twilio integration. You need both an ElevenLabs account and a Twilio-connected phone number.

​
Prerequisites

Before starting this setup, ensure you have:

​
Step-by-Step Setup

1

Create or Select a Conversational AI Agent

Log into your ElevenLabs dashboard and navigate to the Conversational AI section.Option A: Create a New Agent
  1. Click Create New Agent
  2. Give your agent a name (e.g., “Voice Testing Caller”)
  3. Configure basic settings (voice selection, language, etc.)
Option B: Use an Existing Agent
  1. Select an agent from your list
  2. Note that the application will override the agent’s prompt at call time
The agent’s default prompt will be replaced by the application, so you can use any existing agent without worrying about its base configuration.
2

Enable Prompt Overrides

This is a critical step that allows the application to inject custom business descriptions and scenarios at call time.
  1. Open your agent’s settings
  2. Navigate to the Advanced or Developer settings section
  3. Enable Allow Prompt Overrides (or similar option)
  4. Save your changes
Without prompt overrides enabled, the application cannot customize the caller’s behavior for each business scenario. Calls will use the agent’s default prompt instead.
How the Application Uses This:
app.py:73-83
def build_conversation_initiation_data(business_description: str, scenario: str):
    return {
        "conversation_config_override": {
            "agent": {
                "prompt": {
                    "prompt": build_caller_prompt(business_description, scenario),
                },
                "first_message": build_first_message(scenario),
            }
        }
    }
The application sends this override data with every outbound call to ensure the AI stays in character.
3

Copy the Agent ID

After creating or configuring your agent:
  1. Navigate to the agent’s settings or details page
  2. Find the Agent ID (usually displayed near the top)
  3. Copy this ID to your clipboard
Example Agent ID format:
agent_abc123def456xyz789
  1. Add it to your .env file:
ELEVENLABS_AGENT_ID=agent_abc123def456xyz789
4

Connect a Twilio Phone Number

ElevenLabs needs a Twilio-backed phone number to make outbound calls.In ElevenLabs Dashboard:
  1. Navigate to Phone Numbers section
  2. Click Add Phone Number or Import from Twilio
  3. Follow the prompts to connect your Twilio account
  4. Select a phone number from your Twilio account
  5. Complete the import/connection process
  1. Sign up at twilio.com
  2. Navigate to Phone Numbers → Manage → Buy a number
  3. Purchase a phone number (costs vary by region)
  4. Return to ElevenLabs and connect this number
Most Twilio accounts start in trial mode, which allows calls but with some limitations. Upgrade to a paid account for full functionality.
5

Copy the Phone Number ID

After connecting your Twilio phone number in ElevenLabs:
  1. Navigate to the Phone Numbers section
  2. Click on your connected phone number
  3. Find the Phone Number ID in the settings
  4. Copy this ID to your clipboard
Example Phone Number ID format:
phone_xyz789abc123def456
  1. Add it to your .env file:
ELEVENLABS_AGENT_PHONE_NUMBER_ID=phone_xyz789abc123def456
6

Verify Your Configuration

Ensure all three ElevenLabs variables are set in your .env file:
ELEVENLABS_API_KEY=sk_1234567890abcdef1234567890abcdef
ELEVENLABS_AGENT_ID=agent_abc123def456xyz789
ELEVENLABS_AGENT_PHONE_NUMBER_ID=phone_xyz789abc123def456
The application validates all three variables before allowing outbound calls:
app.py:239-245
missing_env = [
    name
    for name in ("ELEVENLABS_API_KEY", "ELEVENLABS_AGENT_ID", "ELEVENLABS_AGENT_PHONE_NUMBER_ID")
    if not os.getenv(name)
]
if missing_env:
    return jsonify({"error": f"Missing required environment variables: {', '.join(missing_env)}"}), 500
7

Test Your Setup

Now test the outbound calling functionality:
  1. Start your Flask application:
python app.py
  1. Open http://localhost:5000
  2. Fill in:
    • Business description (e.g., “Bob’s Pizza - We deliver pizzas in under 30 minutes”)
    • Scenario (e.g., “check availability and order a large pepperoni pizza”)
    • Phone number in E.164 format (e.g., +15551234567)
  3. Click Start Call
Use a real phone number you control for testing. The AI will actually call this number!
Expected Response:
{
  "ok": true,
  "status": "initiated",
  "message": "Call initiated.",
  "to_number": "+15551234567",
  "scenario": "check availability and order a large pepperoni pizza",
  "call_sid": "CA1234567890abcdef",
  "conversation_id": "conv_abc123"
}

​
How Outbound Calls Work

When you initiate a call through the platform, here’s what happens:

​
1. Application Builds the Request

app.py:247-255
payload = {
    "agent_id": os.getenv("ELEVENLABS_AGENT_ID"),
    "agent_phone_number_id": os.getenv("ELEVENLABS_AGENT_PHONE_NUMBER_ID"),
    "to_number": to_number,
    "conversation_initiation_client_data": build_conversation_initiation_data(
        business_description,
        scenario,
    ),
}

​
2. Caller Prompt is Generated

The application creates a dynamic prompt that makes the AI behave like a customer:
app.py:57-66
def build_caller_prompt(business_description: str, scenario: str):
    return (
        "You are simulating a real customer contacting the business below. "
        "The other side is the company, an operator, or an IVR. "
        "Stay in character as the caller/customer, try to complete the task, "
        "and avoid escalating to a human unless the flow requires it.\n\n"
        f"Business description:\n{business_description}\n\n"
        f"Caller goal:\n{scenario}\n\n"
        "Speak naturally, ask one thing at a time, and keep each response concise."
    )

​
3. First Message is Crafted

app.py:69-70
def build_first_message(scenario: str):
    return f"Hi, I'm calling because I'd like to {scenario.rstrip('.')}."
Example: If scenario is “check appointment availability”, the first message becomes: 
"Hi, I'm calling because I'd like to check appointment availability."

​
4. ElevenLabs Makes the Call

The application posts to ElevenLabs:
app.py:258
call_response = elevenlabs_post("/v1/convai/twilio/outbound-call", payload)
ElevenLabs then:
  1. Uses your configured Twilio number as the caller ID
  2. Calls the target phone number
  3. Applies the custom prompt and first message
  4. Handles the conversation using Conversational AI

​
Troubleshooting

Problem: You see this error when trying to make a call:
{"error": "Missing required environment variables: ELEVENLABS_AGENT_ID, ELEVENLABS_AGENT_PHONE_NUMBER_ID"}
Solution:
  1. Verify your .env file contains all three variables
  2. Restart your Flask application after updating .env
  3. Check for typos in variable names
Problem: Your phone number is rejected:
{"error": "Phone number must be in E.164 format, for example +15551234567."}
Solution: Phone numbers must:
  • Start with +
  • Include country code
  • Contain only digits (no spaces, dashes, or parentheses)
  • Be 8-15 digits long after the +
Valid examples:
  • +15551234567 ✓
  • +442071234567 ✓
Invalid examples:
  • 5551234567 ✗ (missing + and country code)
  • +1 555 123 4567 ✗ (contains spaces)
Problem: Authentication failedSolution:
  1. Verify your ELEVENLABS_API_KEY is correct
  2. Check that the key hasn’t expired
  3. Ensure you copied the entire key without trailing/leading spaces
Problem: The agent ID or phone number ID is invalidSolution:
  1. Double-check your ELEVENLABS_AGENT_ID in the ElevenLabs dashboard
  2. Verify your ELEVENLABS_AGENT_PHONE_NUMBER_ID is correct
  3. Ensure the agent and phone number belong to the same ElevenLabs account as your API key
Problem: The AI uses its default prompt instead of your business descriptionSolution:
  1. Ensure prompt overrides are enabled on your ElevenLabs agent
  2. Check the agent settings in the ElevenLabs dashboard
  3. If recently enabled, try creating a test call again
Problem: Network or connectivity issueSolution:
  1. Check your internet connection
  2. Verify ELEVENLABS_API_BASE if you set a custom value
  3. Check if ElevenLabs API is experiencing downtime (status.elevenlabs.io)

​
Advanced Configuration

​
Custom API Base URL

If you need to use a custom ElevenLabs endpoint (for proxies, testing, etc.): 
ELEVENLABS_API_BASE=https://custom-proxy.example.com
The application automatically removes trailing slashes:
app.py:20
ELEVENLABS_API_BASE = os.getenv("ELEVENLABS_API_BASE", "https://api.elevenlabs.io").rstrip("/")

​
API Request Details

The application makes requests with these headers:
app.py:101-108
api_request = Request(
    f"{ELEVENLABS_API_BASE}{path}",
    data=request_body,
    headers={
        "Content-Type": "application/json",
        "xi-api-key": api_key,
    },
    method="POST",
)
Timeout is set to 30 seconds:
app.py:112
with urlopen(api_request, timeout=30) as response:

​
Call Response Data

When a call is successfully initiated, the application returns: 
{
  "ok": true,
  "status": "initiated",
  "message": "Call initiated.",
  "to_number": "+15551234567",
  "scenario": "check availability and order a large pepperoni pizza",
  "call_sid": "CA1234567890abcdef",
  "conversation_id": "conv_abc123",
  "raw": {
    // Full ElevenLabs API response
  }
}
Field Descriptions:
  • call_sid: Twilio’s unique identifier for the call (can be used to track call status in Twilio)
  • conversation_id: ElevenLabs conversation ID (can be used to retrieve call transcripts)
  • raw: Complete response from ElevenLabs API for debugging
The application handles different response field formats:
app.py:269-270
"call_sid": call_response.get("call_sid") or call_response.get("callSid"),
"conversation_id": call_response.get("conversation_id") or call_response.get("conversationId"),

​
Next Steps

Environment Variables

Complete reference for all configuration options

API Reference

Explore the API endpoints for making calls

Build docs developers (and LLMs) love

Get started for freeTalk to us