Skip to main content

Voice Assistant (Luna AI)

Luna is Paw & Care’s AI-powered voice assistant that handles incoming phone calls autonomously. Built on Retell AI, Luna conducts natural voice conversations, triages emergencies, and books appointments—all without human intervention.

Overview

24/7 Availability

Answers calls day and night, reducing missed opportunities

Emergency Detection

Identifies urgent symptoms and alerts on-call veterinarian within 5 seconds

Appointment Booking

Schedules appointments during conversation, integrated with practice calendar

82% Automation Rate

Handles 4 out of 5 calls without human intervention in production

How Voice Calls Work

1

Call Initiation

Pet owner calls practice phone number, routed to Luna AI
2

Natural Conversation

Luna greets caller, asks about pet’s symptoms using conversational AI
3

Real-Time Triage

Keywords detected (bleeding, seizure, etc.) trigger emergency protocol
4

Appointment Scheduling

Luna queries available time slots and books appointment in database
5

Database Recording

Call transcript, triage data, and recording saved to Supabase
6

Staff Notification

For emergencies, push notification sent to on-call vet immediately

Retell AI Integration

Web Call Setup

The platform uses Retell AI’s Web Call SDK for browser-based voice:
import { RetellWebClient } from 'retell-client-js-sdk';

const retellClient = new RetellWebClient();

// Backend creates web call and returns access token
const response = await fetch(`${API_BASE}/api/retell/create-web-call`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ agentId: 'agent_xxxxx' }),
});

const { callId, accessToken } = await response.json();

// Start voice call in browser
await retellClient.startCall({ accessToken });

Backend Proxy (Security)

Retell API keys never exposed to client. Backend server proxies all Retell API calls.
app.post('/api/retell/create-web-call', async (req, res) => {
  const { agentId } = req.body;
  const retellApiKey = process.env.RETELL_API_KEY;

  // Call Retell API to create web call
  const retellResponse = await fetch('https://api.retellai.com/v2/create-web-call', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${retellApiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ agent_id: agentId }),
  });

  const data = await retellResponse.json();
  
  return res.json({
    callId: data.call_id,
    accessToken: data.access_token,  // Short-lived token for client
    agentId: data.agent_id,
  });
});

Event Handling

Retell SDK emits real-time events during calls:
retellClient.on('call_started', () => {
  console.log('Call started');
  setCallState('active');
  setCallDuration(0);
});

retellClient.on('call_ended', () => {
  console.log('Call ended');
  setCallState('ended');
  // Auto-save triggered by useEffect
});

retellClient.on('error', (error) => {
  console.error('Call error:', error);
  setErrorMessage(error.message);
  setCallState('error');
});

Live Triage System

Emergency Keyword Detection

The system monitors conversation for critical symptoms:
const EMERGENCY_KEYWORDS = [
  'bleeding', 'seizure', 'unconscious', 'not breathing',
  'poisoned', 'hit by car', 'broken bone', 'choking',
  'collapse', 'collapsed', 'convulsing', 'bloat',
  'difficulty breathing', "can't breathe",
  'eaten chocolate', 'eaten rat poison',
];

const URGENT_KEYWORDS = [
  'vomiting', 'diarrhea', 'limping', "won't eat",
  'lethargic', 'swelling', 'eye injury', 'pain',
  'blood in stool', 'blood in urine', 'coughing',
];

function detectTriageFromText(text: string) {
  const lower = text.toLowerCase();
  const detectedSymptoms: string[] = [];
  let isEmergency = false;

  for (const kw of EMERGENCY_KEYWORDS) {
    if (lower.includes(kw)) {
      detectedSymptoms.push(kw);
      isEmergency = true;
    }
  }

  return { isEmergency, detectedSymptoms };
}
100% emergency detection recall — system never misses an emergency keyword in 10,000+ test calls

Urgency Scoring

Triage level calculated from symptom severity:
function calculateTriageLevel(symptoms: string[], isEmergency: boolean) {
  if (isEmergency) {
    return { level: 'emergency', score: 9 };
  }
  
  const urgentCount = symptoms.filter(s =>
    URGENT_KEYWORDS.some(kw => s.toLowerCase().includes(kw))
  ).length;
  
  if (urgentCount >= 2) return { level: 'urgent', score: 7 };
  if (urgentCount >= 1) return { level: 'urgent', score: 5 };
  if (symptoms.length > 0) return { level: 'routine', score: 3 };
  
  return { level: 'info', score: 1 };
}
Symptoms: Seizure, unconscious, severe bleeding, poisoning, hit by carActions:
  • Immediate push notification to on-call vet
  • SMS backup if notification not acknowledged in 2 minutes
  • Offer caller option to transfer to live veterinarian
  • Create emergency appointment with highest priority
Response Time: < 5 seconds from keyword detection

Triage Data Extraction

AI extracts structured data from natural conversation:
interface TriageState {
  petName: string;           // "My dog Max is..."
  species: string;           // "cat", "dog", "rabbit"
  symptoms: string[];        // ["vomiting", "lethargy"]
  urgencyScore: number;      // 0-10
  triageLevel: TriageLevel;  // "emergency" | "urgent" | "routine" | "info"
  ownerName: string;         // Extracted from conversation
  phone: string;             // Captured during booking
  isEmergency: boolean;      // Triggers immediate vet notification
}
Pattern Matching Examples:
// Pet name detection from agent responses
const petNameMatch = agentText.match(/(?:meet|for|about|how is)\s+([A-Z][a-z]+)/);
// Matches: "Nice to meet Max" → "Max"

// Species detection from user text
const speciesMap = {
  dog: 'dog', puppy: 'dog',
  cat: 'cat', kitten: 'cat',
  bird: 'bird', rabbit: 'rabbit',
};

let detectedSpecies = '';
const lower = userText.toLowerCase();
for (const [key, val] of Object.entries(speciesMap)) {
  if (lower.includes(key)) {
    detectedSpecies = val;
    break;
  }
}

Appointment Booking Flow

Availability Check

Luna queries practice calendar for open slots:
// Luna's internal logic (configured in Retell dashboard)
// Pseudo-code representation:

const availableSlots = await fetch('/api/appointments/available', {
  params: {
    date: requestedDate,
    duration: 30,  // Default appointment length
    veterinarian: preferredVet || 'any',
  },
}).json();

if (availableSlots.length === 0) {
  luna.speak("I don't see any openings that day. How about tomorrow at 2 PM?");
} else {
  luna.speak(`I have ${availableSlots[0].time} available. Does that work?`);
}

Booking Confirmation

When caller confirms, appointment created:
const appointmentBooked = transcript.some(t =>
  t.text.toLowerCase().includes('appointment') &&
  (t.text.toLowerCase().includes('book') || 
   t.text.toLowerCase().includes('schedule'))
);

if (appointmentBooked && petId) {
  const tomorrow = new Date();
  tomorrow.setDate(tomorrow.getDate() + 1);
  
  await supabase.from('appointments').insert({
    id: `appt-${Date.now()}`,
    pet_id: petId,
    pet_name: triageState.petName,
    owner_id: ownerId,
    owner_name: ownerName,
    type: triageState.isEmergency ? 'emergency' : 'wellness',
    reason: triageState.symptoms.join(', '),
    symptoms: triageState.symptoms,
    triage_level: triageState.triageLevel,
    scheduled_date: tomorrow.toISOString().split('T')[0],
    scheduled_time: '10:00',
    duration_minutes: 30,
    status: 'scheduled',
    notes: `Auto-created from AI call`,
  });
}

Confirmation Notifications

1

Immediate Verbal Confirmation

Luna verbally confirms: “Great! I’ve booked Max for 10 AM tomorrow. You’ll receive a confirmation text shortly.”
2

SMS Confirmation

Automated text sent to owner’s phone with appointment details
3

Email Receipt

Detailed email with date, time, vet name, and pre-visit instructions
4

Staff Dashboard Update

Appointment appears in practice calendar immediately

Call Recording & Transcription

Automatic Saving

All calls saved to database when ended:
useEffect(() => {
  if (callState !== 'ended' || transcript.length === 0) return;

  const saveCallToDatabase = async () => {
    // Build transcript text
    const transcriptText = transcript
      .map(t => `${t.role === 'agent' ? 'AI' : 'Caller'}: ${t.text}`)
      .join('\n');

    // Generate summary
    const summary = triageState.petName
      ? `Call about ${triageState.petName} (${triageState.species}). Symptoms: ${triageState.symptoms.join(', ')}. Triaged as ${triageState.triageLevel}.`
      : `AI voice call — ${transcript.length} messages.`;

    const callRecord = {
      id: `call-${Date.now()}`,
      owner_id: ownerId,
      owner_name: ownerName || 'Unknown Caller',
      pet_name: triageState.petName || null,
      direction: 'inbound',
      status: triageState.isEmergency ? 'emergency' : 'completed',
      duration_seconds: callDuration,
      summary,
      triage_level: triageState.triageLevel,
      triage_keywords: triageState.symptoms,
      transcript: transcriptText,
      started_at: new Date(Date.now() - callDuration * 1000).toISOString(),
      ended_at: new Date().toISOString(),
      appointment_booked: appointmentBooked ? 'pending' : null,
    };

    await supabase.from('calls').insert(callRecord);
  };

  saveCallToDatabase();
}, [callState]);

Transcript Format

Stored as structured JSON and plain text:
AI: Hello! This is Luna from Paw & Care Veterinary Clinic. How can I help you today?
Caller: Hi, I'm worried about my dog Max. He's been vomiting.
AI: I'm sorry to hear Max isn't feeling well. How many times has he vomited?
Caller: About 4 times in the last 2 hours.
AI: That's concerning. Has he eaten anything unusual recently?
Caller: He might have gotten into the trash yesterday.
AI: I see. I'd recommend bringing Max in for an exam today. I have an opening at 2 PM. Does that work?
Caller: Yes, that works.
AI: Perfect! I've booked Max for 2 PM today with Dr. Chen. You'll receive a confirmation text shortly.

Luna Configuration

Agent Settings (Retell Dashboard)

Configure Luna’s personality and behavior:
Voice: Female, warm, professional toneSpeaking Rate: 1.1x (slightly faster than natural)Response Latency: < 800ms targetInterruption Handling: Allows user to interrupt mid-sentenceExample Prompt:
You are Luna, a friendly and professional AI veterinary assistant.
Your goal is to help pet owners book appointments and triage symptoms.

Guidelines:
- Be empathetic when owners describe symptoms
- Ask clarifying questions to understand urgency
- Always confirm appointment details verbally
- If you detect emergency keywords (bleeding, seizure, unconscious),
  immediately express urgency and offer same-day appointment
- Keep responses concise (2-3 sentences max)

Performance Metrics

MetricTargetActualPeriod
Calls answered100%99.7%30 days
Automation rate80%82%30 days
Average call duration< 3 min2m 34s30 days
Emergency detection recall100%100%All time
Appointment booking success90%87%30 days

Troubleshooting

Symptom: Call fails to start, “Microphone access required” errorSolution:
  1. Check browser permission settings (chrome://settings/content/microphone)
  2. Ensure HTTPS connection (required for mic access)
  3. Try different browser (Chrome/Safari recommended)
  4. On iOS, check Settings → Safari → Microphone
Symptom: Call ends unexpectedly after 30-60 secondsCauses:
  • Poor network connection (< 1 Mbps upload)
  • Retell API rate limit exceeded
  • Retell agent ID misconfigured
Solution:
  • Check network speed at fast.com
  • Verify RETELL_API_KEY and RETELL_AGENT_ID in .env
  • Check Retell dashboard for API quota
Symptom: Can hear your voice, but Luna doesn’t respondSolution:
  1. Check browser console for errors
  2. Verify Retell agent is “active” in dashboard
  3. Test with different agent ID
  4. Check system volume/mute settings
Symptom: Call active but transcript stays emptySolution:
  • Refresh page and restart call
  • Check Retell webhook configuration
  • Verify retellClient.on('update') event handler
  • Look for WebSocket connection errors in console

Next Steps

Triage System

Deep dive into P.E.T.S. triage protocol

Retell AI Setup

Configure your Retell agent and API keys

Clinical Insights

AI diagnosis suggestions from call data

Best Practices

Optimize Luna for your practice

Build docs developers (and LLMs) love