Telephony Integration

This guide covers Telnyx integration, webhook configuration, and audio streaming setup for DispatchAI.

Overview

DispatchAI uses Telnyx for telephony infrastructure. The integration handles:

Incoming call webhooks - Call lifecycle events (initiated, answered, hangup)
Audio streaming - Real-time PCM audio via WebSocket
Call control - Answer, TTS greeting, and streaming commands

Call flow:

Incoming call → Telnyx webhook → /api/v1/call/incoming
       ↓
   Answer call + TTS greeting
       ↓
   Start audio stream → /ws (WebSocket)
       ↓
   Live transcription + analysis
       ↓
   Hangup → Final triage packet

Telnyx Setup

Create a Telnyx account

Purchase a phone number

Navigate to Numbers → Buy Numbers
Search for a number in your desired area code
Select a number and complete purchase

Choose a local number for the region you’re deploying in to reduce caller confusion.

Create a Call Control Application

Go to Call Control → Applications
Click Add New Application
Configure:
- Name: DispatchAI
- Webhook URL: https://<your-ngrok-url>/api/v1/call/incoming
- Webhook HTTP Method: POST
- Send Webhook: Enable
Save the application

In development, get your ngrok URL by running ./scripts/dev_start.sh and copying the printed URL.

Assign number to application

Go to Numbers → My Numbers
Click your purchased number
Under Call Settings, select your DispatchAI application
Save changes

Get API credentials

Navigate to API Keys under Settings
Create a new API key with Call Control permissions
Copy the API key to your .env:

.env

TELNYX_API_KEY=KEY...

Set webhook secret (optional but recommended)

In your Call Control Application settings
Generate a Webhook Signing Secret
Add to .env:

.env

TELNYX_WEBHOOK_SECRET=your_secret_here

This secret validates webhook authenticity. Required for production deployments.

Webhook Events

DispatchAI handles three primary webhook events:

1. call.initiated

Triggered when: An incoming call is received Handler: app/main.py:363-403 Actions:

Maps call_control_id to call_session_id for tracking
Initiates call answering + audio streaming (background task)
Seeds LIVE_CALLS state with initial metadata

Example payload:

{
  "data": {
    "event_type": "call.initiated",
    "payload": {
      "call_control_id": "v3:T02llNDk1...",
      "call_session_id": "428c31b6-abf1...",
      "from": "+15551234567",
      "to": "+15559876543",
      "start_time": "2026-03-03T19:52:31.123456Z"
    }
  }
}

State created:

LIVE_CALLS[call_id] = {
    "call_id": "428c31b6-abf1...",
    "call_control_id": "v3:T02llNDk1...",
    "from_masked": "•567",
    "to": "+15559876543",
    "started_at": "2026-03-03T19:52:31Z",
    "status": "RINGING",
    "transcript_live": "",
    "risk_level": "UNKNOWN",
    "risk_score": 0.0
}

2. call.answered

Triggered when: The call is answered (after TTS greeting) Handler: app/main.py:405-435 Actions:

Updates call status to ACTIVE
Seeds LIVE_QUEUE for dispatcher dashboard
Begins live signal tracking

State update:

LIVE_QUEUE[call_id] = {
    "id": call_id,
    "status": "LIVE",
    "risk_level": "UNKNOWN",
    "summary": "Listening…"
}

3. call.hangup

Triggered when: The call ends Handler: app/main.py:438-625 Actions:

Transcribes recorded audio via Deepgram batch API
Analyzes emotion and distress
Classifies service category (EMS/FIRE/POLICE/OTHER)
Generates summary
Computes risk score and priority ranking
Creates CallPacket and adds to dispatch queue
Removes from LIVE_QUEUE and LIVE_CALLS

Final packet structure:

{
  "call_id": "428c31b6-abf1...",
  "duration_seconds": 45,
  "audio": {
    "distress_score": 0.82,
    "distress_max": 0.91,
    "voiced_seconds": 38.4
  },
  "nlp": {
    "transcript": "Someone's been shot, we need help now...",
    "emotion": {
      "label": "HIGHLY_DISTRESSED",
      "intensity": 0.87,
      "sentiment": "negative"
    },
    "category": "EMS",
    "tags": ["ACTIVE_SHOOTER", "TRAUMA", "VIOLENCE"]
  },
  "risk": {
    "level": "CRITICAL",
    "score": 0.95
  },
  "ranking": {
    "priority": 1,
    "weight": 950,
    "created_at": "2026-03-03T19:53:16Z"
  }
}

Call Control Flow

When a call is initiated, DispatchAI executes the following sequence:

Answer and Stream

Function: _answer_and_stream() in app/main.py:175-214

Wait for call ready

3-second delay to ensure Telnyx call is fully established:

await asyncio.sleep(3)

Answer the call

POST to Telnyx Call Control API:

POST https://api.telnyx.com/v2/calls/{call_control_id}/actions/answer

Response: 200 OK with call details

Play TTS greeting

Speak greeting to caller:

POST https://api.telnyx.com/v2/calls/{call_control_id}/actions/speak
{
  "voice": "female",
  "language": "en-US",
  "payload": "This is Dispatch AI. I'm listening. Please describe your emergency.",
  "overlay": false  # Block until finished
}

overlay: false ensures the greeting completes before audio streaming starts.

Start audio streaming

Begin real-time audio feed to WebSocket:

POST https://api.telnyx.com/v2/calls/{call_control_id}/actions/streaming_start
{
  "stream_url": "wss://your-domain.com/ws",
  "audio": {
    "direction": "inbound",
    "format": "pcm16",
    "sample_rate": 8000
  }
}

Audio format:

Codec: PCM16 (linear 16-bit)
Sample rate: 8 kHz
Channels: Mono
Endianness: Big-endian (converted to little-endian in handler)

WebSocket Audio Handler

The /ws endpoint receives audio frames from Telnyx: Implementation: app/api/ws/handler.py:344-549

Connection Lifecycle

@router.websocket("/ws")
async def telnyx_ws(ws: WebSocket):
    await ws.accept()
    # Initialize buffers and STT client
    stt = STTClient(on_partial=_on_partial, on_final=_on_final)
    
    while True:
        text = await ws.receive_text()
        evt = json.loads(text)
        
        if evt["event"] == "start":
            # Extract call_control_id → resolve call_id
            # Start Deepgram streaming
            await stt.start(sample_rate=8000)
        
        elif evt["event"] == "media":
            # Decode base64 audio
            raw = base64.b64decode(evt["media"]["payload"])
            
            # Convert µ-law or swap endianness
            le16 = mulaw_to_pcm16le(raw) if len(raw) in (80, 160) else swap_endian_16(raw)
            
            # Feed to STT
            await stt.feed(le16)
            
            # Compute distress proxy (RMS-based VAD)
            rms = rms_norm_pcm16le(le16)
            voiced = rms >= 0.02
            
            # Update live signals
            LIVE_SIGNALS[call_id]["distress"] = compute_distress(rms, ema)
            LIVE_SIGNALS[call_id]["transcript_live"] = stt.partial_text
        
        elif evt["event"] == "stop":
            # Finalize transcription
            result = await stt.finalize()
            LIVE_SIGNALS[call_id]["transcript"] = result["transcript"]
            break

Audio Processing Pipeline

Receive - Base64-encoded audio chunks (20ms each)
Decode - Convert µ-law to PCM16 little-endian
Stream - Feed to Deepgram WebSocket for live transcription
Analyze - Compute RMS for voice activity + distress proxy
Buffer - Save to WAV file for batch transcription
Finalize - On hangup, flush STT and save final transcript

Distress calculation:

# Exponential moving average of loudness
ema = alpha * rms + (1 - alpha) * ema
diff = max(0.0, rms - ema)
score = min(1.0, diff * 8.0)  # Amplify relative spikes

LIVE_SIGNALS[call_id]["distress"] = score
LIVE_SIGNALS[call_id]["max_distress"] = max(max_distress, score)

Testing Your Integration

Start the development server

./scripts/dev_start.sh

Note the ngrok URL (e.g., https://abc123.ngrok.io)

Update Telnyx webhook URL

In Telnyx Portal:

Go to your Call Control Application
Update Webhook URL to https://abc123.ngrok.io/api/v1/call/incoming
Save

Call your Telnyx number

Use your mobile phone to call the number you configured.Expected behavior:

Call connects
TTS greeting plays: “This is Dispatch AI…”
Speak your emergency scenario
Hang up

Check the live dashboard

Open http://localhost:8000/debug/live_calls/ in your browser.You should see:

Call status (ACTIVE during call, ENDED after)
Live transcript updates
Real-time distress score
Risk level and emotion

View the dispatch queue

After hangup, check http://localhost:8000/debug/list_queue/Your call should appear with:

Final transcript
Service category (EMS/FIRE/POLICE)
Risk level and priority
Recommended tags

Production Deployment

Using a Fixed Domain

Replace ngrok with a production domain:

.env

HTTP_PUBLIC_URL=https://api.yourdomain.com
WS_PUBLIC_URL=wss://api.yourdomain.com/ws

Nginx configuration:

server {
    listen 443 ssl;
    server_name api.yourdomain.com;

    # HTTP endpoints
    location /api/ {
        proxy_pass http://localhost:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # WebSocket endpoint
    location /ws {
        proxy_pass http://localhost:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_read_timeout 3600s;  # Long timeout for call duration
    }
}

Webhook Security

Validate webhook signatures in production:

import hmac
import hashlib

def verify_telnyx_signature(payload: bytes, signature: str, secret: str) -> bool:
    expected = hmac.new(
        secret.encode(),
        payload,
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, signature)

@app.post("/api/v1/call/incoming")
async def incoming_call(req: Request):
    body = await req.body()
    signature = req.headers.get("telnyx-signature-sha256")
    
    if not verify_telnyx_signature(body, signature, TELNYX_WEBHOOK_SECRET):
        raise HTTPException(status_code=401, detail="Invalid signature")
    
    # Process webhook...

Troubleshooting

Webhook Not Received

Check ngrok tunnel status

Visit http://127.0.0.1:4040 to see incoming requests.If empty, your ngrok URL may have changed - update Telnyx webhook URL.

Verify Telnyx configuration

In Telnyx Portal:

Check Webhook URL is correct
Ensure Send Webhook is enabled
Review Webhook Logs for delivery failures

Test webhook manually

curl -X POST http://localhost:8000/api/v1/call/incoming \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "event_type": "call.initiated",
      "payload": {
        "call_control_id": "test_ccid",
        "call_session_id": "test_call_id",
        "from": "+15551234567",
        "to": "+15559876543"
      }
    }
  }'

WebSocket Connection Fails

Check WS_PUBLIC_URL

Ensure it points to your WebSocket endpoint:

echo $WS_PUBLIC_URL
# Should be wss://... (not http://)

Verify WebSocket route

Test locally:

import asyncio
import websockets

async def test():
    async with websockets.connect('ws://localhost:8000/ws') as ws:
        await ws.send('{"event": "connected"}')
        print(await ws.recv())

asyncio.run(test())

Check firewall/proxy settings

WebSocket requires:

HTTP/1.1 protocol
Upgrade: websocket header
Long timeout (calls can last minutes)

No Audio Received

Verify streaming_start response

Check server logs for:

[telephony] streaming_start -> 200 {"result":"ok"}

If not 200, check call_control_id validity.

Check audio format

Ensure Telnyx is sending correct format:

{
  "audio": {
    "direction": "inbound",
    "format": "pcm16",
    "sample_rate": 8000
  }
}

Test with recorded call

Replay a saved WAV file:

python scripts/replay_call.py data/calls/1234567890.wav

Get Started

Core Concepts

Guides

Telephony Integration

Overview

Telnyx Setup

Webhook Events

1. call.initiated

2. call.answered

3. call.hangup

Call Control Flow

Answer and Stream

WebSocket Audio Handler

Connection Lifecycle

Audio Processing Pipeline

Testing Your Integration

Production Deployment

Using a Fixed Domain

Webhook Security

Troubleshooting

Webhook Not Received

WebSocket Connection Fails

No Audio Received

Next Steps

Agent Configuration

API Reference

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

​Overview

​Telnyx Setup

​Webhook Events

​1. call.initiated

​2. call.answered

​3. call.hangup

​Call Control Flow

​Answer and Stream

​WebSocket Audio Handler

​Connection Lifecycle

​Audio Processing Pipeline

​Testing Your Integration

​Production Deployment

​Using a Fixed Domain

​Webhook Security

​Troubleshooting

​Webhook Not Received

​WebSocket Connection Fails

​No Audio Received

​Next Steps

Agent Configuration

API Reference

Build docs developers (and LLMs) love

Overview

Telnyx Setup

Webhook Events

1. call.initiated

2. call.answered

3. call.hangup

Call Control Flow

Answer and Stream

WebSocket Audio Handler

Connection Lifecycle

Audio Processing Pipeline

Testing Your Integration

Production Deployment

Using a Fixed Domain

Webhook Security

Troubleshooting

Webhook Not Received

WebSocket Connection Fails

No Audio Received

Next Steps