System Architecture

​

High-Level Architecture

┌─────────────┐
│   Browser   │
└──────┬──────┘
       │ HTTPS
       ▼
┌─────────────┐
│   Traefik   │  ← Reverse Proxy
└──────┬──────┘
       │
       ├────────────────┬────────────────┬───────────────┐
       ▼                ▼                ▼               ▼
┌────────────┐   ┌────────────┐   ┌──────────┐   ┌─────────┐
│  Frontend  │   │  Backend   │   │   OPA    │   │ Vector  │
│   (UI)     │   │  (Django)  │   │ (AuthZ)  │   │ (Logs)  │
└────────────┘   └──────┬─────┘   └──────────┘   └─────────┘
                        │
        ┌───────────────┼───────────────┬──────────────┐
        ▼               ▼               ▼              ▼
   ┌─────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
   │PostgreSQL│   │  Redis   │   │ClickHouse│   │ Kvrocks  │
   │   (DB)   │   │ (Cache)  │   │(Analytics)│   │ (Cache)  │
   └─────────┘   └──────────┘   └──────────┘   └──────────┘
                        │
                        ▼
                ┌───────────────┐
                │  RQ Workers   │
                │ (Background)  │
                └───────────────┘

​

Core Components

​

Frontend (cvat-ui)

• React 18
• TypeScript
• Redux for state management
• Ant Design for UI components
• Webpack for bundling

​

cvat-ui

• Task and project management interfaces
• Annotation workspace
• User management
• Analytics dashboards
• Settings and configuration

​

cvat-core

• REST API communication
• Data models (Task, Job, Project, Annotation)
• State management
• Authentication handling

​

cvat-canvas

• Drawing tools (rectangle, polygon, polyline, points, ellipse, mask)
• Interaction handling (drag, resize, rotate)
• Zoom and pan
• SVG-based rendering

​

cvat-canvas3d

• Point cloud rendering
• 3D cuboid annotations
• Camera views (perspective, top, side, front)
• Three.js-based rendering

​

cvat-data

• Frame providers
• Data chunking
• Video/image processing
• Compression utilities

​

Backend (cvat)

• Python 3.10+
• Django 4.2
• Django REST Framework
• Uvicorn (ASGI server)
• RQ (Redis Queue) for background tasks

​

engine

• Models: Task, Job, Project, Label, Annotation
• API endpoints for CRUD operations
• Annotation management
• Frame caching and serving

​

dataset_manager

• Format converters (COCO, YOLO, Pascal VOC, etc.)
• Annotation transformations
• Dataset validation

​

organizations

• Organization management
• Membership handling
• Resource isolation

​

iam (Identity and Access Management)

• User management
• Permission system
• Role-based access control
• Integration with OPA

​

quality_control

• Quality reports
• Conflict detection
• Inter-annotator agreement
• Honeypot frames

​

consensus

• Replica job management
• Annotation merging
• Agreement calculation

​

lambda_manager

• Auto-annotation with AI models
• Function management
• Request handling

​

webhooks

• Event notifications
• Webhook configuration
• Delivery management

​

events

• User action logging
• Event aggregation
• Analytics data export

​

Background Workers (RQ)

​

cvat_worker_import

• Parses uploaded files
• Validates annotations
• Inserts data into database

​

cvat_worker_export

• Converts annotations to target format
• Packages data
• Generates download archives

​

cvat_worker_annotation

• Calls serverless functions
• Processes model predictions
• Creates annotations from results

​

cvat_worker_quality_reports

• Compares annotations
• Calculates metrics
• Generates reports

​

cvat_worker_consensus

• Aggregates replica annotations
• Calculates agreement scores
• Merges annotations

​

Data Storage

​

PostgreSQL

• User accounts and permissions
• Tasks, jobs, and projects
• Labels and annotations
• Organizations and memberships
• Audit logs

​

Redis (In-Memory)

• Session data
• Cached API responses
• RQ job queue
• Temporary data

​

Kvrocks (On-Disk)

• Compressed image chunks
• Video frames
• Cached media files

​

ClickHouse

• User events
• Action logs
• Performance metrics
• Analytics aggregations

​

Supporting Services

​

Traefik

• HTTPS termination
• Request routing
• Access logging
• Rate limiting

​

OPA (Open Policy Agent)

• Fine-grained access control
• Policy evaluation
• Rule-based permissions

​

Vector

• Collects container logs
• Transforms log data
• Forwards to analytics systems

​

Data Flow

​

Annotation Creation Flow

1. User draws annotation in UI (cvat-canvas)
2. Canvas dispatches Redux action
3. cvat-core serializes annotation
4. API request sent to backend
5. Django view validates data
6. OPA checks permissions
7. Data saved to PostgreSQL
8. Event logged to ClickHouse
9. Response returned to frontend
10. Redux state updated
11. UI re-renders

​

Task Creation Flow

1. User submits task form
2. Files uploaded via TUS protocol
3. Backend creates Task record
4. Job creation queued in RQ
5. Worker processes media:
   • Extracts frames from video
   • Generates thumbnails
   • Creates chunks
   • Compresses and caches
6. Jobs created and assigned
7. Task marked as ready
8. User notified

​

Export Flow

1. User requests dataset export
2. Backend queues export job
3. Worker fetches annotations
4. Format converter transforms data
5. Files packaged into archive
6. Archive stored temporarily
7. Download link provided
8. User downloads file

​

API Design

​

Endpoints Structure

/api/
  ├── auth/          # Authentication
  ├── users/         # User management
  ├── projects/      # Projects
  ├── tasks/         # Tasks
  ├── jobs/          # Jobs
  ├── labels/        # Labels
  ├── issues/        # Review issues
  ├── comments/      # Comments
  ├── organizations/ # Organizations
  ├── cloudstorages/ # Cloud storage
  ├── lambda/        # Serverless functions
  ├── webhooks/      # Webhook management
  ├── events/        # Event logs
  └── schema/        # OpenAPI schema

​

Authentication

• Session authentication: For browser-based access
• Token authentication: For API/SDK access (being deprecated)
• Access tokens: New token system with expiration

​

Frontend Architecture

​

State Management

Store
├── auth          # Authentication state
├── tasks         # Task list and details
├── jobs          # Job list and details
├── projects      # Project list and details
├── annotations   # Current annotation state
├── canvas        # Canvas state (zoom, mode, etc.)
├── settings      # User preferences
├── notifications # UI notifications
└── plugins       # Plugin system state

​

Component Structure

cvat-ui/src/
├── components/   # Presentational components
├── containers/   # Connected containers
├── actions/      # Redux actions
├── reducers/     # Redux reducers
├── utils/        # Utility functions
└── assets/       # Static assets

​

Security

​

Authentication Flow

1. User logs in with credentials
2. Django creates session
3. Session ID stored in cookie
4. Cookie sent with each request
5. Backend validates session
6. OPA evaluates permissions
7. Request allowed or denied

​

Authorization (OPA)

allow {
    input.auth.user.is_staff
}

allow {
    input.auth.user.id == input.resource.owner.id
}

allow {
    input.resource.organization.id in input.auth.user.organizations
    input.auth.user.has_perm["view_task"]
}

​

Scalability

• Frontend: Served statically, can use CDN
• Backend: Multiple server instances behind load balancer
• Workers: Scale worker count based on queue depth
• Database: PostgreSQL replication and read replicas
• Cache: Redis Cluster or Kvrocks sharding

​

Development Workflow

1. Clone repository
2. Start development environment
3. Make changes to code
4. Frontend: Hot-reload automatically
5. Backend: Restart container or use debugger
6. Run tests
7. Submit pull request

​

Deployment

• Docker Compose: Simple single-server deployment
• Kubernetes/Helm: Production-grade orchestration
• Cloud: AWS, GCP, Azure with managed services

​

Next Steps

• Review the code style guidelines
• Learn about testing
• Read the pull request guide
• Explore the API documentation