Receiving Incoming Calls

Set up Agentic AI to automatically answer incoming calls to your Twilio phone number.

How It Works

When someone calls your Twilio number:

Twilio receives the call and requests TwiML instructions from your webhook
Server detects incoming call and creates a session automatically
AI answers immediately with a default or custom greeting
Natural conversation begins powered by OpenAI Realtime API
Telegram notifications (optional) send live transcripts

Reference: src/agenticai/server/app.py:86-159 (incoming call detection)

Prerequisites

Step 1: Get a Twilio Phone Number

Purchase a phone number from Twilio Console:

Go to Phone Numbers → Buy a number

Search for numbers in your region

Choose a number with Voice capability

Click Buy

Step 2: Start the Server

The server must be running to handle incoming calls:

agenticai server

Or run as a background service:

agenticai service start

Step 3: Expose via Tunnel

Twilio needs a public URL to reach your server:

agenticai tunnel start

Copy the public URL that appears (e.g., https://abc123.ngrok.io).

Configure Twilio Webhook

Point your Twilio number to the Agentic AI webhook:

Step 1: Open Twilio Console

Navigate to Phone Numbers → Manage → Active Numbers

Step 2: Select Your Number

Click on the phone number you want to configure

Step 3: Configure Voice Settings

Under Voice & Fax section:

A call comes in: Select Webhook

URL: Enter your webhook URL:

https://your-subdomain.ngrok.io/twilio/voice

HTTP Method: Select POST

Step 4: Save Configuration

Click Save at the bottom of the page

Make sure to include /twilio/voice at the end of your webhook URL.

Test Incoming Calls

Now test by calling your Twilio number:

# From your phone, dial your Twilio number
# The AI should answer immediately!

What you’ll hear:

The AI will greet you with the default prompt
It will listen and respond naturally
Transcripts appear in real-time (if Telegram is configured)

Customize the Greeting

The default greeting comes from config.yaml:

config.yaml

gemini:
  system_instruction: |
    You are Alchemy, an AI agent created by Istiqlal.
    Be helpful, friendly, and assist the caller with whatever they need.
    You can send messages, search the web, make notes, and more.

Custom Greeting Example

config.yaml

gemini:
  system_instruction: |
    You are the support assistant for Acme Corp.
    Greet callers warmly and ask how you can help them today.
    You can:
    - Answer product questions
    - Take support tickets
    - Schedule callbacks
    - Transfer to a human agent if needed
    
    Always be polite and professional.

The system_instruction is used for ALL incoming calls unless overridden programmatically.

Advanced Configuration

Programmatic Greeting Override

You can detect incoming calls and customize behavior in the webhook handler. Reference implementation in src/agenticai/server/app.py:130-145:

# When incoming call is detected:
if not call_info:
    # INCOMING CALL
    logger.info("Incoming call detected", from_number=from_number)
    
    # Register this as an incoming call
    call_id = await call_manager.register_incoming_call(
        call_sid=call_sid,
        from_number=from_number,
        to_number=to_number,
    )
    
    # Use custom prompt based on caller or time of day
    prompt = config.gemini.system_instruction or default_prompt

Caller-Specific Greetings

You can customize greetings based on the caller’s number:

# Example: VIP caller detection
VIP_NUMBERS = ["+15551234567", "+15559876543"]

if from_number in VIP_NUMBERS:
    prompt = "Welcome back! How can I assist you today?"
else:
    prompt = "Thank you for calling. How may I help you?"

Time-Based Greetings

from datetime import datetime

hour = datetime.now().hour
if 5 <= hour < 12:
    greeting = "Good morning!"
elif 12 <= hour < 17:
    greeting = "Good afternoon!"
else:
    greeting = "Good evening!"

prompt = f"{greeting} Thank you for calling. How can I help?"

Incoming Call Flow

Here’s what happens when a call arrives:

Step 1: Twilio Receives Call

Twilio receives the incoming call and immediately makes a POST request to your webhook:

POST https://your-url.ngrok.io/twilio/voice

Form Data:
  CallSid: CA1234567890abcdef
  From: +15551234567
  To: +15559876543
  Direction: inbound

Step 2: Server Detects Incoming

The server checks if this is an outbound call we initiated or an incoming call:

call_info = call_manager.get_pending_call_info(call_sid)

if not call_info:
    # This is an incoming call!
    call_id = await call_manager.register_incoming_call(...)

Step 3: Session Created

A new call session is created with:

Unique call ID

Caller’s phone number

Default or custom prompt

Metadata (direction: “inbound”)

Step 4: TwiML Returned

Server responds with TwiML to connect the media stream:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://your-url.ngrok.io/twilio/media-stream">
      <Parameter name="prompt" value="..." />
      <Parameter name="call_sid" value="CA123..." />
    </Stream>
  </Connect>
</Response>

Step 5: WebSocket Opens

Twilio opens a WebSocket connection for bidirectional audio streaming.

Step 6: AI Conversation Begins

OpenAI Realtime API:

Listens to the caller via Whisper STT

Generates responses

Speaks back using the configured voice

Telegram Integration

Get notified of incoming calls and see live transcripts:

Enable Telegram Notifications

config.yaml

telegram:
  enabled: true
  bot_token: ${TELEGRAM_BOT_TOKEN}
  chat_id: ${TELEGRAM_CHAT_ID}

What You’ll Receive

When an incoming call arrives:

📞 Call Started

Call ID: `incoming_CA123...`
To: +15559876543
Prompt: You are Alchemy, an AI agent...

Listening for transcripts...

During the call:

👤 User
Hello, I need help with my order

🤖 Assistant
Of course! I'd be happy to help. Can you provide your order number?

When call ends:

✅ Call Ended

Call ID: `incoming_CA123...`
Duration: 142.3s
Transcripts: 12
Outcome: completed

Reference: src/agenticai/telegram/direct_client.py

Status Callbacks

Track incoming call status changes:

Configure Status Callback

In Twilio Console under your number’s Voice settings:

Status Callback URL:

https://your-url.ngrok.io/twilio/status

Status Events: Select all that apply:
- initiated
- ringing
- answered
- completed

Handle Status Updates

The server receives status updates throughout the call:

# POST /twilio/status
{
  "CallSid": "CA1234567890abcdef",
  "CallStatus": "completed",
  "From": "+15551234567",
  "To": "+15559876543"
}

Reference: src/agenticai/server/app.py:162-174

Troubleshooting

Incoming calls not connecting

Symptoms: Caller hears ringing but no answerSolutions:

Verify webhook URL is correct:

curl https://your-url.ngrok.io/twilio/voice
# Should return 405 Method Not Allowed (GET not supported)

Check server is running:
```
agenticai status
```

Test webhook URL accessibility:

curl https://your-url.ngrok.io/health
# Should return: {"status":"healthy","active_calls":0}

Verify Twilio webhook configuration:
- Go to Twilio Console → Phone Numbers
- Check Voice & Fax settings
- Ensure URL ends with /twilio/voice
- Confirm HTTP method is POST

AI not speaking to caller

Symptoms: Call connects but AI is silentSolutions:

Check server logs for WebSocket errors:
```
agenticai service logs -f
```
Verify OpenAI API key is valid:
```
agenticai test-connection
```
Confirm OpenAI Realtime API access:
- Realtime API may require waitlist approval
- Check your OpenAI account dashboard

Test with different voice:

openai_realtime:
  voice: "alloy"  # Try: alloy, echo, fable, onyx, nova, shimmer

Call drops immediately

Symptoms: Call connects then drops after 1-2 secondsSolutions:

WebSocket URL might be incorrect:
- Server logs should show “WebSocket connection accepted”
- Verify WSS (secure WebSocket) is used with HTTPS/ngrok
Check firewall/security groups:
- Port 8080 must be accessible
- WebSocket upgrades must be allowed

Increase timeout in uvicorn config:

# src/agenticai/server/app.py
uvicorn.run(
    app,
    timeout_keep_alive=120,  # Add this
)

No Telegram notifications

Symptoms: Calls work but no Telegram messagesSolutions:

Verify Telegram is enabled:

telegram:
  enabled: true  # Make sure this is true

Check bot token and chat ID:

# Test manually:
curl https://api.telegram.org/bot<YOUR_TOKEN>/getMe

Ensure you’ve messaged the bot:
- Open Telegram and search for your bot
- Send any message (e.g., “hi”)
- Then get your chat ID

Get your chat ID:

curl https://api.telegram.org/bot<YOUR_TOKEN>/getUpdates
# Find "chat":{"id":123456789}

Reference: src/agenticai/telegram/direct_client.py:159-173

API Endpoints

List Active Calls

See all incoming and outbound calls:

curl http://localhost:8080/api/calls

Response:

{
  "calls": [
    {
      "call_id": "incoming_CA123...",
      "to_number": "+15559876543",
      "status": "in-progress",
      "direction": "inbound"
    }
  ],
  "count": 1
}

End an Active Call

Programmatically end an incoming call:

curl -X POST http://localhost:8080/api/calls/incoming_CA123.../end

Reference: src/agenticai/server/app.py:236-264

Next Steps

Making Calls

Initiate outbound calls

Telegram Integration

Set up live transcripts

Service Management

Run as a daemon

Scheduling

Automate recurring calls

Get Started

Core Concepts

Configuration

Usage Guides

Integrations

Troubleshooting

Receiving Incoming Calls

How It Works

Prerequisites

Configure Twilio Webhook

Test Incoming Calls

Customize the Greeting

Custom Greeting Example

Advanced Configuration

Programmatic Greeting Override

Caller-Specific Greetings

Time-Based Greetings

Incoming Call Flow

Telegram Integration

Enable Telegram Notifications

What You’ll Receive

Status Callbacks

Configure Status Callback

Handle Status Updates

Troubleshooting

API Endpoints

List Active Calls

End an Active Call

Next Steps

Making Calls

Telegram Integration

Service Management

Scheduling

Build docs developers (and LLMs) love

Get Started

Core Concepts

Configuration

Usage Guides

Integrations

Troubleshooting

​How It Works

​Prerequisites

​Configure Twilio Webhook

​Test Incoming Calls

​Customize the Greeting

​Custom Greeting Example

​Advanced Configuration

​Programmatic Greeting Override

​Caller-Specific Greetings

​Time-Based Greetings

​Incoming Call Flow

​Telegram Integration

​Enable Telegram Notifications

​What You’ll Receive

​Status Callbacks

​Configure Status Callback

​Handle Status Updates

​Troubleshooting

​API Endpoints

​List Active Calls

​End an Active Call

​Next Steps

Making Calls

Telegram Integration

Service Management

Scheduling

Build docs developers (and LLMs) love

How It Works

Prerequisites

Configure Twilio Webhook

Test Incoming Calls

Customize the Greeting

Custom Greeting Example

Advanced Configuration

Programmatic Greeting Override

Caller-Specific Greetings

Time-Based Greetings

Incoming Call Flow

Telegram Integration

Enable Telegram Notifications

What You’ll Receive

Status Callbacks

Configure Status Callback

Handle Status Updates

Troubleshooting

API Endpoints

List Active Calls

End an Active Call

Next Steps