Backend Deployment

Overview

The JARVIS backend is a Python FastAPI application that runs in Daytona sandboxes. It handles:

Image capture and face detection
PimEyes identification via Browser Use agents
Agent swarm orchestration for profile research
Report synthesis using Gemini/OpenAI
Real-time updates to Convex database

Prerequisites

Daytona Account

Docker Installed

Required for local testing and container builds

Dockerfile Configuration

The backend uses the official Python 3.11 slim image:

FROM python:3.11-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app

COPY . .
RUN pip install --no-cache-dir .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Environment Variables

The Docker container requires environment variables to be set. See the complete list in Environment Variables.

Build Arguments

PYTHONDONTWRITEBYTECODE

string

default:"1"

Prevents Python from writing .pyc files

PYTHONUNBUFFERED

string

default:"1"

Forces stdout/stderr streams to be unbuffered for real-time logging

Deployment Steps

Prepare Environment

Create a .env file in the backend directory with all required environment variables:

cp .env.example .env
# Edit .env with your actual API keys and configuration

See Environment Variables for the complete list.

Build Docker Image

Build the Docker image locally to verify it works:

cd backend
docker build -t jarvis-backend:latest .

Test locally:

docker run -p 8000:8000 --env-file .env jarvis-backend:latest

Verify the health endpoint:

curl http://localhost:8000/api/health

Expected response:

{"status": "ok", "service": "specter"}

Deploy to Daytona Sandbox

Create a new Daytona sandbox:

daytona create --name jarvis-backend --image python:3.11

Upload your code:

daytona upload jarvis-backend ./backend

Set environment variables:

daytona env set jarvis-backend --env-file .env

Start the application:

daytona exec jarvis-backend -- pip install .
daytona exec jarvis-backend -- uvicorn main:app --host 0.0.0.0 --port 8000

Configure External Access

Expose the sandbox to the internet:

daytona expose jarvis-backend 8000

This will provide a public URL like:

https://jarvis-backend-abc123.daytona.app

Update your frontend environment variables with this URL.

Verify Deployment

Test the deployed API:

curl https://jarvis-backend-abc123.daytona.app/api/health

Check service flags:

curl https://jarvis-backend-abc123.daytona.app/api/status

This will show which services are properly configured.

Configuration Details

Application Settings

The backend uses Pydantic settings management defined in backend/config.py:

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        extra="ignore",
    )

    app_name: str = "JARVIS API"
    environment: str = Field(default="development", alias="SPECTER_ENV")
    log_level: str = Field(default="INFO", alias="SPECTER_LOG_LEVEL")
    frontend_origin: str = Field(default="http://localhost:3000", alias="SPECTER_FRONTEND_ORIGIN")
    api_port: int = Field(default=8000, alias="SPECTER_API_PORT")

Service Integration Flags

The backend automatically detects which services are configured:

def service_flags(self) -> dict[str, bool]:
    return {
        "convex": bool(self.convex_url),
        "mongodb": bool(self.mongodb_uri),
        "exa": bool(self.exa_api_key),
        "browser_use": bool(self.browser_use_api_key),
        "openai": bool(self.openai_api_key),
        "gemini": bool(self.gemini_api_key),
        "laminar": bool(self.laminar_api_key),
        # ... more services
    }

Access this at /api/status to verify your configuration.

Dependencies

The backend requires these Python packages:

fastapi==0.115.0
uvicorn==0.30.0
python-telegram-bot==21.0
browser-use==0.2.0
google-generativeai==0.8.0
exa-py==1.0.0
mediapipe==0.10.14
pymongo==4.8.0
motor==3.5.0
convex==0.7.0
twscrape==0.12.0
laminar-ai==0.5.0
python-dotenv==1.0.0
Pillow==10.4.0
numpy==1.26.0

These are installed automatically during the Docker build via pip install .

API Endpoints

POST /api/capture

Receive photo/video for processing:

curl -X POST https://jarvis-backend-abc123.daytona.app/api/capture \
  -F "[email protected]"

Response:

{
  "capture_ids": ["uuid1", "uuid2"],
  "status": "processing"
}

GET /api/person/:id

Retrieve complete person data:

curl https://jarvis-backend-abc123.daytona.app/api/person/uuid

Response:

{
  "person_id": "uuid",
  "name": "John Smith",
  "confidence": 0.92,
  "status": "complete",
  "dossier": {
    "summary": "CTO at Acme Corp...",
    "workHistory": [...],
    "socialProfiles": {...}
  }
}

GET /api/health

Health check endpoint:

curl https://jarvis-backend-abc123.daytona.app/api/health

Response:

{"status": "ok", "service": "specter"}

GET /api/status

Service configuration status:

curl https://jarvis-backend-abc123.daytona.app/api/status

Response:

{
  "services": {
    "convex": true,
    "mongodb": true,
    "exa": true,
    "browser_use": true,
    "openai": false,
    "gemini": true
  }
}

Logging Configuration

The backend uses Python’s logging module with configurable levels:

# Set via environment variable
SPECTER_LOG_LEVEL=DEBUG

Available levels:

DEBUG: Verbose logging for development
INFO: Standard operational logging (default)
WARNING: Warning messages only
ERROR: Error messages only
CRITICAL: Critical errors only

Viewing Logs

View logs in Daytona sandbox:

daytona logs jarvis-backend --follow

Troubleshooting

Port Already in Use

If port 8000 is already in use, change it:

SPECTER_API_PORT=8001
uvicorn main:app --host 0.0.0.0 --port 8001

CORS Errors

Ensure SPECTER_FRONTEND_ORIGIN matches your frontend URL:

SPECTER_FRONTEND_ORIGIN=https://your-frontend.vercel.app

Service Connection Failures

Check service flags:

curl https://jarvis-backend-abc123.daytona.app/api/status

Verify all required environment variables are set correctly.

Browser Use Agent Failures

Ensure you have:

Valid BROWSER_USE_API_KEY
Sufficient API credits
Proper network connectivity from Daytona sandbox

Check Browser Use dashboard at cloud.browser-use.com

Performance Optimization

Worker Configuration

For production, use multiple workers:

uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

Browser Session Pre-warming

Pre-warm browser sessions for faster agent startup:

# Add to startup event in main.py
@app.on_event("startup")
async def warmup_browsers():
    # Pre-initialize browser sessions
    pass

Connection Pooling

MongoDB connection pooling is configured automatically:

client = MongoClient(
    mongodb_uri,
    maxPoolSize=50,
    minPoolSize=10
)

Get Started

Core Concepts

Hardware Integration

Backend Services

Agent System

Frontend

Data & Storage

Deployment

​Overview

​Prerequisites

Daytona Account

Docker Installed

​Dockerfile Configuration

​Environment Variables

​Build Arguments

​Deployment Steps

​Configuration Details

​Application Settings

​Service Integration Flags

​Dependencies

​API Endpoints

​POST /api/capture

​GET /api/person/:id

​GET /api/health

​GET /api/status

​Logging Configuration

​Viewing Logs

​Troubleshooting

​Port Already in Use

​CORS Errors

​Service Connection Failures

​Browser Use Agent Failures

​Performance Optimization

​Worker Configuration

​Browser Session Pre-warming

​Connection Pooling

​Next Steps

Frontend Deployment

Environment Variables

Build docs developers (and LLMs) love

Overview

Prerequisites

Dockerfile Configuration

Environment Variables

Build Arguments

Deployment Steps

Configuration Details

Application Settings

Service Integration Flags

Dependencies

API Endpoints

POST /api/capture

GET /api/person/:id

GET /api/health

GET /api/status

Logging Configuration

Viewing Logs

Troubleshooting

Port Already in Use

CORS Errors

Service Connection Failures

Browser Use Agent Failures

Performance Optimization

Worker Configuration

Browser Session Pre-warming

Connection Pooling

Next Steps