Skip to main content

​
Prerequisites

Before deploying serverless functions, ensure you have:
  • CVAT self-hosted installation running
  • Docker installed and running
  • Nuclio CLI (nuctl) installed
  • Access to the CVAT serverless functions repository
  • Sufficient system resources (4GB+ RAM recommended for model loading)

​
Installation

​
Install Nuclio CLI

curl -s https://api.github.com/repos/nuclio/nuclio/releases/latest \
  | grep -i "browser_download_url.*nuctl.*$(uname)" \
  | cut -d : -f 2,3 \
  | tr -d \" \
  | wget -O nuctl -qi - && chmod +x nuctl

sudo mv nuctl /usr/local/bin/

​
Clone Serverless Functions

The serverless functions are included in the CVAT repository: 
git clone https://github.com/cvat-ai/cvat.git
cd cvat/serverless

​
Deployment Options

​
Option 1: Deploy All CPU Functions

Deploy all available functions optimized for CPU: 
./deploy_cpu.sh
This script:
  1. Builds the OpenVINO base image
  2. Creates a cvat project in Nuclio
  3. Discovers all function.yaml files
  4. Builds Docker images for each function
  5. Deploys functions to the local Nuclio platform

​
Option 2: Deploy All GPU Functions

Deploy GPU-optimized versions (requires NVIDIA GPU and drivers): 
./deploy_gpu.sh
This uses function-gpu.yaml files instead of standard function.yaml files.

​
Option 3: Deploy Individual Functions

Deploy specific functions manually: 
# Create Nuclio project
nuctl create project cvat --platform local

# Build base image (for OpenVINO functions)
docker build -t cvat.openvino.base ./openvino/base

# Deploy a specific function
nuctl deploy --project-name cvat \
  --path ./pytorch/facebookresearch/sam/nuclio \
  --file ./pytorch/facebookresearch/sam/nuclio/function.yaml \
  --platform local

​
Configuration

​
Function YAML Structure

Each function is defined by a function.yaml file: 
metadata:
  name: pth-facebookresearch-sam-vit-h
  namespace: cvat
  annotations:
    name: Segment Anything
    version: 2
    type: interactor
    spec: |
      [
        { "name": "object", "type": "mask" }
      ]
    min_pos_points: 0
    min_neg_points: 0
    startswith_box_optional: true

spec:
  description: Interactive object segmentation with Segment-Anything
  runtime: 'python:3.10'
  handler: main:handler
  eventTimeout: 30s
  
  build:
    image: cvat.pth.facebookresearch.sam.vit_h
    baseImage: ubuntu:22.04
    directives:
      preCopy:
        - kind: RUN
          value: apt-get update && apt-get install -y python3-pip
        - kind: RUN
          value: pip install torch torchvision segment-anything

  triggers:
    myHttpTrigger:
      numWorkers: 2
      kind: 'http'
      attributes:
        maxRequestBodySize: 33554432 # 32MB

​
Key Configuration Parameters

​
Metadata Annotations

  • name: Display name in CVAT UI
  • type: Function type (detector, interactor, tracker, reid)
  • spec: Label schema with supported output types
  • version: Function version for compatibility

​
Spec Settings

  • runtime: Python version for the function
  • handler: Entry point (typically main:handler)
  • eventTimeout: Maximum execution time per request
  • numWorkers: Number of concurrent workers
  • maxRequestBodySize: Maximum request payload size

​
Environment Variables

Configure CVAT connection when deploying: 
nuctl deploy --project-name cvat \
  --path ./function/path \
  --file ./function/path/function.yaml \
  --platform local \
  --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk \
  --env CVAT_FUNCTIONS_REDIS_PORT=6666 \
  --platform-config '{"attributes": {"network": "cvat_cvat"}}'

​
Connecting to CVAT

​
Configure Nuclio Settings

In your CVAT deployment, configure Nuclio connection in docker-compose.yml: 
services:
  cvat_server:
    environment:
      CVAT_SERVERLESS: 1
      NUCLIO_SCHEME: http
      NUCLIO_HOST: nuclio
      NUCLIO_PORT: 8070
      NUCLIO_FUNCTION_NAMESPACE: nuclio
      NUCLIO_DEFAULT_TIMEOUT: 30
      NUCLIO_INVOKE_METHOD: direct  # or 'dashboard'

​
Invoke Methods

Direct Invocation (default):
  • Calls functions directly via HTTP port
  • Lower latency, fewer network hops
  • Requires functions accessible from CVAT container
Dashboard Invocation:
  • Routes through Nuclio dashboard API
  • Better for complex networking scenarios
  • Slightly higher latency

​
Network Configuration

Ensure CVAT and Nuclio containers share a network: 
docker network create cvat_cvat

# When deploying functions, specify network:
nuctl deploy --project-name cvat \
  --platform-config '{"attributes": {"network": "cvat_cvat"}}' \
  ...

​
Verification

​
List Deployed Functions

nuctl get function --platform local
Expected output: 
NAMESPACE | NAME                              | PROJECT | STATE | REPLICAS
nuclio    | pth-facebookresearch-sam-vit-h    | cvat    | ready | 2/2
nuclio    | onnx-wongkinyiu-yolov7            | cvat    | ready | 2/2
...

​
Test Function Invocation

Test a function directly: 
echo '{"image": "base64_encoded_image_data"}' | \
  nuctl invoke pth-facebookresearch-sam-vit-h --method POST

​
Check CVAT Integration

  1. Open CVAT UI
  2. Create or open a task
  3. Navigate to annotation view
  4. Check “Tools” menu for available AI models
  5. Verify functions appear under “Magic” or “Automatic annotation”

​
Troubleshooting

​
Function Not Appearing in CVAT

Check Nuclio function status: 
nuctl get function --platform local
Verify CVAT environment variables: 
docker exec cvat_server env | grep NUCLIO
Check CVAT logs: 
docker logs cvat_server | grep lambda

​
Function Build Failures

Increase Docker memory:
  • Some models (especially Mask R-CNN) require 4GB+ RAM
  • Adjust Docker Desktop settings or system resources
Check base image: 
docker images | grep cvat.openvino.base
Rebuild base image: 
docker build -t cvat.openvino.base ./openvino/base

​
Function Timeout Issues

Increase timeout in function.yaml: 
spec:
  eventTimeout: 60s  # Increase from 30s
Update CVAT timeout: 
environment:
  NUCLIO_DEFAULT_TIMEOUT: 60

​
Network Connection Errors

Verify network: 
docker network inspect cvat_cvat
Check function accessibility: 
docker exec cvat_server curl http://nuclio:8070/api/functions
Update invoke method: 
environment:
  NUCLIO_INVOKE_METHOD: dashboard  # Try alternative method

​
Resource Management

​
Memory Requirements

Typical memory usage per function:
  • YOLO v7 (ONNX): ~500MB
  • Mask R-CNN (OpenVINO): ~2GB
  • SAM (PyTorch): ~2.5GB
  • Detectron2 models: ~1-3GB

​
GPU Support

For GPU acceleration:
  1. Install NVIDIA Container Toolkit:
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
  1. Deploy GPU functions:
./deploy_gpu.sh
  1. Verify GPU access:
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

​
Updating Functions

​
Update Existing Function

# Delete old function
nuctl delete function pth-facebookresearch-sam-vit-h --platform local

# Redeploy with updated configuration
nuctl deploy --project-name cvat \
  --path ./pytorch/facebookresearch/sam/nuclio \
  --file ./pytorch/facebookresearch/sam/nuclio/function.yaml \
  --platform local

​
Update All Functions

# Remove all functions
nuctl delete project cvat --platform local

# Redeploy
./deploy_cpu.sh

​
Best Practices

  1. Start Small: Deploy only the functions you need initially
  2. Monitor Resources: Track memory and CPU usage during operation
  3. Use CPU for Batch: CPU functions work well for background annotation jobs
  4. Use GPU for Interactive: GPU accelerates real-time interactor tools
  5. Version Control: Track function.yaml changes for reproducibility
  6. Test Locally: Verify functions work before deploying to production
  7. Network Isolation: Use dedicated networks for security
  8. Regular Updates: Keep base images and dependencies updated

​
Next Steps

Custom Models

Create your own serverless functions

Overview

Learn more about serverless function types

Build docs developers (and LLMs) love

Get started for freeTalk to us