Skip to main content

​
POST /api/capture/identify

Identify a person by providing their name and a photo URL. This endpoint downloads the image, runs the full face detection and identification pipeline, and returns comprehensive results.

​
Authentication

No authentication required (for hackathon demo).

​
Request

​
name
string
required
Person’s full name (minimum 1 character).
​
image_url
string
required
Publicly accessible URL of a photo of the person. Must be a valid HTTP/HTTPS URL.

​
Response

​
capture_id
string
Unique identifier for this identification session (format: identify_xxxxxxxxxxxx).
​
total_frames
integer
Number of frames processed from the image (typically 1 for still images).
​
faces_detected
integer
Number of faces detected in the image.
​
persons_created
array
Array of person IDs created from the detected faces.
​
persons_enriched
integer
Number of persons that were successfully enriched with research data.
​
success
boolean
true if the identification completed without errors, false otherwise.
​
error
string
Error message if success is false, otherwise null.

​
Example Request

curl -X POST https://api.jarvis.local/api/capture/identify \
  -H "Content-Type: application/json" \
  -d '{
    "name": "John Smith",
    "image_url": "https://example.com/photos/john-smith.jpg"
  }'

​
Example Response (Success)

{
  "capture_id": "identify_a1b2c3d4e5f6",
  "total_frames": 1,
  "faces_detected": 1,
  "persons_created": ["person_xyz123abc"],
  "persons_enriched": 1,
  "success": true,
  "error": null
}

​
Example Response (Image Download Failed)

{
  "capture_id": "identify_a1b2c3d4e5f6",
  "total_frames": 0,
  "faces_detected": 0,
  "persons_created": [],
  "persons_enriched": 0,
  "success": false,
  "error": "Failed to download image: HTTP 404"
}

​
Status Codes

  • 200 - Identification completed (check success field for pipeline status)
  • 400 - Invalid request (image download failed, invalid URL, or missing fields)
  • 500 - Server error during processing

​
Error Codes

When success is false, the error field contains:
  • Failed to download image: HTTP - Image URL returned non-200 status
  • Failed to download image: - Network error or timeout
  • No faces detected - Image processed but no faces found
  • Face detection failed: - MediaPipe detection error
  • Embedding generation failed: - ArcFace embedding error
  • Enrichment timeout - Research agents exceeded timeout (partial results may exist)

​
Processing Pipeline

  1. Download - Fetch image from provided URL (30s timeout)
  2. Detection - Extract face bounding boxes using MediaPipe
  3. Embedding - Generate 512-dim face embeddings with ArcFace
  4. Person Creation - Store person record in database (Convex/MongoDB)
  5. Enrichment - Trigger research agents:
    • PimEyes face search for additional photos
    • Reverse image search via SerpAPI/Google Vision
    • Exa API fast-pass research
    • Browser Use deep research (LinkedIn, Twitter, Google, Crunchbase)
  6. Synthesis - Generate comprehensive dossier via Claude/Gemini

​
Best Practices

Use high-quality frontal face photos for best results
Ensure image URLs are publicly accessible (no auth required)
Person name should match the face in the photo
Large images (>5MB) may experience longer download times
Images with multiple faces will create multiple person records

​
Synchronous vs. Asynchronous

This endpoint is synchronous - it waits for the full pipeline to complete before returning. For large images or complex scenes, this can take 10-30 seconds. For asynchronous processing:
  • Use /api/capture with file upload
  • Subscribe to real-time updates via /api/research/{person_name}/stream
  • Use Convex subscriptions to watch person record updates

​
Integration Example (JavaScript)

const identifyPerson = async (name, imageUrl) => {
  const response = await fetch('https://api.jarvis.local/api/capture/identify', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ name, image_url: imageUrl })
  });
  
  const result = await response.json();
  
  if (result.success && result.persons_created.length > 0) {
    console.log(`Identified ${name}: ${result.persons_created[0]}`);
    // Fetch full person data
    const person = await fetch(`https://api.jarvis.local/api/person/${result.persons_created[0]}`);
    return await person.json();
  } else {
    console.error(`Identification failed: ${result.error}`);
    return null;
  }
};

// Usage
const person = await identifyPerson('Jane Doe', 'https://example.com/jane.jpg');

Build docs developers (and LLMs) love

Get started for freeTalk to us