Skip to main content

Overview

Haggle integrates with multiple AI and communication services to power its marketplace and voice negotiation features.

Grok LLM

Task inference and question generation

Grok Realtime

AI voice agent for negotiations

OpenAI Search

Web search for finding providers

Twilio Voice

Phone call infrastructure

Supabase

Managed PostgreSQL database

xAI SDK

Python client for Grok API

Grok LLM (xAI)

Overview

Grok LLM powers the intelligent task inference and context generation features using xAI’s official Python SDK. Model: grok-3-fast
API Endpoint: https://api.x.ai/v1/chat
Documentation: https://docs.x.ai/docs/guides/chat

Use Cases

Converts free-text user queries into structured service types.Function: infer_task(query: str) -> strExample:
task = await infer_task("my toilet is broken and leaking")
# Returns: "plumber"
System Prompt:
You are a service task classifier. Given a user's request, 
identify the type of service professional needed.

Respond with ONLY a single word or short phrase for the service type, such as:
- plumber
- electrician
- house cleaner
- painter
- handyman
- HVAC technician
- locksmith
Generates 3-5 context-specific questions to understand job requirements.Function: generate_clarifying_questions(...) -> List[Dict[str, str]]Example:
questions = await generate_clarifying_questions(
    task="plumber",
    query="fix my toilet",
    zip_code="95126",
    date_needed="2025-12-10",
    price_limit=250
)
Key rules enforced in system prompt:
  • Do NOT ask about location (already provided)
  • Do NOT ask about timing (already provided)
  • Do NOT ask about budget (already provided)
  • Maximum 5 questions
  • Focus on work specifics
Converts first-person queries into second-person problem descriptions for AI agent.Function: format_problem_statement(original_query: str, task: str) -> strExample:
problem = await format_problem_statement("my lawn is too long", "landscaper")
# Returns: "your lawn needs to be mowed"
Analyzes call transcripts to extract negotiated prices.Function: extract_negotiated_price(transcript: List[Dict]) -> Optional[float]Example:
transcript = [
    {"role": "assistant", "text": "How much would you charge?"},
    {"role": "user", "text": "I can do it for $175"},
    {"role": "assistant", "text": "That sounds good, let's go with $175"}
]

price = await extract_negotiated_price(transcript)
# Returns: 175.0

Configuration

Environment Variables
XAI_API_KEY=your_xai_api_key_here
Fallback behavior: If no API key is set, uses rule-based fallbacks:
  • Task inference: keyword matching (“toilet” → “plumber”)
  • Questions: predefined question sets per task type
  • Problem formatting: simple string replacement
  • Price extraction: regex pattern matching

SDK Usage

pip install xai-sdk
Location: /workspace/source/services/grok_llm.py

Grok Realtime API (xAI)

Overview

Grok Realtime API provides low-latency voice conversations via WebSocket for the AI negotiation agent. Endpoint: wss://api.x.ai/v1/realtime
Protocol: WebSocket
Audio Format: PCM 24kHz (bidirectional)

Voice Agent Flow

1

Connect WebSocket

import websockets

GROK_URL = "wss://api.x.ai/v1/realtime"
API_KEY = os.getenv("XAI_API_KEY")

async with websockets.connect(
    GROK_URL, 
    additional_headers={"Authorization": f"Bearer {API_KEY}"}
) as grok_ws:
    # Connection established
2

Configure Session

{
    "type": "session.update",
    "session": {
        "voice": "Rex",
        "instructions": "YOU ARE a homeowner calling for service...",
        "turn_detection": {"type": "server_vad"},
        "audio": {
            "input": {"format": {"type": "audio/pcm", "rate": 24000}},
            "output": {"format": {"type": "audio/pcm", "rate": 24000}}
        }
    }
}
3

Trigger Initial Greeting

{"type": "response.create"}
This tells Grok to generate its first message immediately.
4

Stream Audio (Twilio → Grok)

# Convert Twilio μ-law to Grok PCM 24kHz
mulaw = base64.b64decode(twilio_payload)
pcm_8k = audioop.ulaw2lin(mulaw, 2)
pcm_24k, _ = audioop.ratecv(pcm_8k, 2, 1, 8000, 24000, None)

await grok_ws.send(json.dumps({
    "type": "input_audio_buffer.append",
    "audio": base64.b64encode(pcm_24k).decode('utf-8')
}))
5

Stream Audio (Grok → Twilio)

# Convert Grok PCM 24kHz to Twilio μ-law
event = json.loads(msg)
if event['type'] == 'response.output_audio.delta':
    pcm_24k = base64.b64decode(event['delta'])
    pcm_8k, _ = audioop.ratecv(pcm_24k, 2, 1, 24000, 8000, None)
    mulaw = audioop.lin2ulaw(pcm_8k, 2)
    
    await twilio_ws.send_json({
        "event": "media",
        "streamSid": stream_sid,
        "media": {"payload": base64.b64encode(mulaw).decode()}
    })
6

Capture Transcript

# User speech
if event['type'] == 'conversation.item.input_audio_transcription.completed':
    transcript.append({"role": "user", "text": event['transcript']})

# AI speech
if event['type'] == 'response.audio_transcript.done':
    transcript.append({"role": "assistant", "text": event['transcript']})

Event Types

Event TypeDescription
session.updateConfigure voice, instructions, VAD
response.createTrigger AI response generation
input_audio_buffer.appendSend audio chunk
conversation.item.createAdd text message
Event TypeDescription
response.output_audio.deltaAudio chunk from AI
response.audio_transcript.doneFull AI transcript
conversation.item.input_audio_transcription.completedUser transcript
response.doneAI response complete
errorError occurred

Negotiation Prompt

def generate_prompt(provider: dict) -> str:
    return f"""
YOU ARE a homeowner in zip code {provider['zip_code']} calling for service.
{provider['problem']}.
YOU ARE NOT an AI. You are a real person speaking to {provider['service_provider']}.
Your tone should be casual, direct, and slightly cost-conscious.
Refer to the following context if needed: {provider['context_answers']}

1. You must begin with: "Hi, is this {provider['service_provider']}?"
2. After confirming, ask for a price estimate.
3. Negotiate to secure the lowest price, using **${provider['max_price']}** as target.
4. Agreeing to a price up to ${provider['max_price']} is acceptable.

End the call based on outcome:
    - No Agreement: "Thank you for the info. I need to think about it."
    - Price Agreed: "Thank you! I will reach out to you again shortly."
"""
Location: /workspace/source/backend/app.py:41

Overview

OpenAI’s Responses API with web search tool finds local service providers. Model: gpt-4o
Tool: web_search_preview
API Endpoint: https://api.openai.com/v1/responses

Implementation

from openai import OpenAI

client = OpenAI(
    api_key=OPENAI_API_KEY, 
    organization=OPENAI_ORG_API_KEY
)

search_prompt = f"""Find {task} services near zip code {zip_code}.

Search the web for local {task}s and provide a list with:
1. Business name
2. Phone number

Format each result as: NAME | PHONE

Find up to 5 providers near {zip_code}."""

response = client.responses.create(
    model="gpt-4o",
    tools=[{"type": "web_search_preview"}],
    input=search_prompt
)

full_response = response.output_text

Configuration

Environment Variables
OPENAI_API_KEY=sk-proj-...
OPENAI_ORG_API_KEY=org-...  # Optional
Fallback providers: If API key is missing or request fails, returns mock data:
fallback_providers = {
    "plumber": [
        ("Reliable Plumbing Services", "(408) 555-0101"),
        ("Quick Drain Solutions", "(408) 555-0102"),
        ("Bay Area Master Plumbers", "(408) 555-0103"),
        ("24/7 Emergency Plumbing", "(408) 555-0104"),
        ("Budget Plumbing Co.", "(408) 555-0105"),
    ],
    "electrician": [...],
    "house cleaner": [...],
    # etc.
}
Location: /workspace/source/services/grok_search.py

Twilio Voice API

Overview

Twilio provides phone call infrastructure for connecting to service providers. API Version: Twilio REST API
Media Streams: WebSocket-based audio streaming

Call Flow

1

Initiate Call

from twilio.rest import Client

client = Client(TWILIO_SID, TWILIO_TOKEN)

twiml_url = f"https://{DOMAIN}/twiml?provider_id={provider['id']}"
client.calls.create(
    to=provider['phone_number'],
    from_=FROM_NUMBER,
    url=twiml_url
)
2

Return TwiML

<Response>
    <Connect>
        <Stream url="wss://your-domain.com/media-stream">
            <Parameter name="provider_id" value="123"/>
        </Stream>
    </Connect>
</Response>
Generated via FastAPI endpoint:
@app.post("/twiml")
async def get_twiml(provider_id: str):
    response = VoiceResponse()
    connect = Connect()
    stream = connect.stream(url=f"wss://{DOMAIN}/media-stream")
    stream.parameter(name="provider_id", value=provider_id)
    response.append(connect)
    return HTMLResponse(content=str(response), media_type="application/xml")
3

WebSocket Connection

Twilio connects to /media-stream WebSocket with audio events:
{"event": "start", "start": {"streamSid": "MZ...", "customParameters": {...}}}
{"event": "media", "media": {"payload": "base64_mulaw_audio"}}
{"event": "stop"}
4

Stream Audio

Send audio back to Twilio:
await websocket.send_json({
    "event": "media",
    "streamSid": stream_sid,
    "media": {"payload": base64_mulaw}
})

Audio Format

  • Codec: μ-law (G.711)
  • Sample Rate: 8kHz
  • Channels: Mono
  • Encoding: Base64
  • Chunk Size: 20ms frames
Conversion to Grok format:
import audioop

# Decode from base64
mulaw = base64.b64decode(payload)

# μ-law to PCM 8kHz
pcm_8k = audioop.ulaw2lin(mulaw, 2)

# Resample to 24kHz for Grok
pcm_24k, _ = audioop.ratecv(pcm_8k, 2, 1, 8000, 24000, None)

# Encode to base64 for Grok
base64.b64encode(pcm_24k).decode('utf-8')

Configuration

Environment Variables
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=your_auth_token
TWILIO_PHONE_NUMBER=+1234567890
DOMAIN=your-ngrok-or-production-domain.com
Twilio requires a public HTTPS endpoint for webhooks. Use ngrok for local development.
Location: /workspace/source/backend/app.py

Supabase

Overview

Managed PostgreSQL database with Python client library. Service: Supabase (PostgreSQL 15+)
Client: supabase-py

Setup

1

Install Client

pip install supabase
2

Initialize Client

from supabase import create_client, Client

SUPABASE_URL = os.getenv("SUPABASE_URL")
SUPABASE_KEY = os.getenv("SUPABASE_KEY")

supabase: Client = create_client(SUPABASE_URL, SUPABASE_KEY)
3

Run Migration

Copy SQL from supabase_migration.sql and run in Supabase SQL Editor.

Operations

data = {
    "job_id": "550e8400-...",
    "service_provider": "Reliable Plumbing",
    "phone_number": "(408) 555-0101",
    "zip_code": "95126",
    "max_price": 250.0,
    "call_status": "pending"
}

response = supabase.table("providers").insert(data).execute()
provider_id = response.data[0]['id']
Location: /workspace/source/db/models.py

API Key Management

Never commit API keys to Git. Use environment variables.

Environment Variables

# AI Services
XAI_API_KEY=xai-...
OPENAI_API_KEY=sk-proj-...
OPENAI_ORG_API_KEY=org-...  # Optional

# Database
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

# Twilio
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_PHONE_NUMBER=+1234567890

# Deployment
DOMAIN=your-domain.com
CALL_BACKEND_URL=http://localhost:6000

Security Best Practices

1

Use Secret Management

  • Development: .env file (not in Git)
  • Production: AWS Secrets Manager, Railway secrets, Vercel env vars
2

Rotate Keys Periodically

  • Twilio: Rotate auth tokens every 90 days
  • OpenAI: Generate new keys, deprecate old ones
  • Supabase: Use service_role key only on backend
3

Separate Keys by Environment

# .env.development
XAI_API_KEY=xai-dev-key

# .env.production
XAI_API_KEY=xai-prod-key
4

Monitor Usage

  • OpenAI Dashboard: Track API usage and costs
  • Twilio Console: Monitor call volumes
  • Supabase: Check database queries and bandwidth

Error Handling

Common errors:
  • Invalid API key
  • Rate limit exceeded
  • Network timeout
Handling:
try:
    task = await infer_task(query)
except Exception as e:
    print(f"Grok API exception: {e}")
    task = _fallback_infer_task(query)  # Use rule-based fallback
Common errors:
  • API key missing
  • Model not available
  • Search timeout
Handling:
if not OPENAI_API_KEY:
    print("⚠️  No OPENAI_API_KEY set - using fallback providers")
    return _fallback_providers(job)

try:
    response = client.responses.create(...)
except Exception as e:
    print(f"OpenAI Search API exception: {e}")
    return _fallback_providers(job)
Common errors:
  • Invalid phone number format
  • Call failed (busy, no answer, etc.)
  • WebSocket disconnection
Handling:
try:
    client.calls.create(to=phone, from_=FROM_NUMBER, url=twiml_url)
except Exception as e:
    print(f"❌ Failed to dial {provider['service_provider']}: {e}")
    update_provider_call_status(provider_id, "failed")
Common errors:
  • Connection timeout
  • Invalid credentials
  • Row Level Security policy violation
Handling:
try:
    response = supabase.table("providers").insert(data).execute()
    if not response.data:
        raise Exception("Failed to create provider in Supabase")
except Exception as e:
    print(f"Database error: {e}")
    # Log error, retry, or return error to user

Rate Limits

ServiceFree TierRate LimitNotes
Grok LLMUnknownUnknownContact xAI for details
OpenAI API$5 credit500 RPMTier 1 (pay-as-you-go)
TwilioTrial credit1 call/secUpgrade for higher limits
SupabaseFree tier500 MB DBUnlimited API requests
Production deployments should use paid tiers with higher rate limits.

Cost Estimation

Per job (1 user request):
ServiceOperationCost
Grok LLMTask inference~$0.001
Grok LLM5 questions~$0.002
OpenAIWeb search~$0.01
Grok Realtime2-min call × 5 providers~$0.50
Twilio2-min call × 5 providers~$0.20
SupabaseDB operationsFree
Total~$0.73
Costs are estimates. Monitor actual usage via provider dashboards.

Build docs developers (and LLMs) love