UVC Camera Setup

Overview

This guide covers the complete setup and configuration of UVC (USB Video Class) cameras for video streaming on ESP32-S2/S3. The library supports MJPEG format video streaming with configurable resolution and frame rates.

Buffer Configuration

Allocate Transfer Buffers

The library uses double buffering for USB transfer. Both buffers must be allocated in DMA-capable memory.

#define XFER_BUFFER_SIZE (55 * 1024)  // Adjust based on resolution

uint8_t *xfer_buffer_a = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE, 
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);
uint8_t *xfer_buffer_b = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE,
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);

Transfer buffers must be larger than one complete frame size. For higher resolutions, increase the buffer size accordingly.

Allocate Frame Buffer

A separate buffer stores the assembled frame for processing.

uint8_t *frame_buffer = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE,
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);

All buffers must be successfully allocated before configuration. Always check for NULL.

Verify Allocation

if (!xfer_buffer_a || !xfer_buffer_b || !frame_buffer) {
    ESP_LOGE(TAG, "Failed to allocate buffers");
    // Free any allocated buffers and handle error
    return;
}

Resolution Configuration

Using Any Resolution

To automatically select the camera’s default resolution:

USB_STREAM *usb = new USB_STREAM();

usb->uvcConfiguration(
    FRAME_RESOLUTION_ANY,      // width - accept any
    FRAME_RESOLUTION_ANY,      // height - accept any
    FRAME_INTERVAL_FPS_15,     // frame interval (15 fps)
    XFER_BUFFER_SIZE,          // transfer buffer size
    xfer_buffer_a,             // transfer buffer A
    xfer_buffer_b,             // transfer buffer B
    XFER_BUFFER_SIZE,          // frame buffer size
    frame_buffer               // frame buffer
);

FRAME_RESOLUTION_ANY is defined as __UINT16_MAX__ and tells the driver to use the camera’s first available resolution.

Using Specific Resolution

To request a specific resolution:

#define FRAME_WIDTH 480
#define FRAME_HEIGHT 320

usb->uvcConfiguration(
    FRAME_WIDTH,
    FRAME_HEIGHT,
    FRAME_INTERVAL_FPS_15,
    XFER_BUFFER_SIZE,
    xfer_buffer_a,
    xfer_buffer_b,
    XFER_BUFFER_SIZE,
    frame_buffer
);

Frame Rate Configuration

Frame rate is specified using frame intervals (in 100-ns units):

// Predefined intervals
FRAME_INTERVAL_FPS_5   // 5 fps
FRAME_INTERVAL_FPS_10  // 10 fps
FRAME_INTERVAL_FPS_15  // 15 fps
FRAME_INTERVAL_FPS_20  // 20 fps
FRAME_INTERVAL_FPS_30  // 30 fps (actually 25 fps)

// Custom frame rate
#define FPS2INTERVAL(fps) (10000000ul / fps)
uint32_t custom_interval = FPS2INTERVAL(12); // 12 fps

Registering Frame Callbacks

Define Callback Function

The callback receives each assembled frame:

void frame_callback(uvc_frame_t *frame, void *user_ptr) {
    // Process frame data
    ESP_LOGI(TAG, "Frame: %dx%d, format=%d, bytes=%u, seq=%u",
             frame->width,
             frame->height,
             frame->frame_format,
             frame->data_bytes,
             frame->sequence);
    
    // Frame data available in frame->data
    // Do NOT block in this callback!
}

Never block in the frame callback! The callback runs in the USB task context. Queue frames for processing in another task if needed.

void *callback_arg = NULL;  // Optional user data
usb->uvcCamRegisterCb(&frame_callback, callback_arg);

Starting and Stopping Streaming

Basic Start/Stop

// Start streaming
usb->start();

// Wait for camera connection
usb->connectWait(10000);  // 10 second timeout

// Camera is now streaming...

// Stop streaming
usb->stop();

With State Callback

Monitor connection state changes:

void state_callback(usb_stream_state_t event, void *arg) {
    switch (event) {
        case STREAM_CONNECTED:
            ESP_LOGI(TAG, "Camera connected");
            break;
        case STREAM_DISCONNECTED:
            ESP_LOGI(TAG, "Camera disconnected");
            break;
    }
}

// Register state callback before starting
usb->registerState(&state_callback, NULL);
usb->start();

Suspend and Resume Workflow

Suspend Streaming

Temporarily pause video streaming without disconnecting:

usb->uvcCamSuspend(NULL);

Suspending is useful before changing configuration or to save power.

Modify Configuration (Optional)

While suspended, you can change resolution or frame rate:

// Change to 640x480 @ 15 fps
usb->uvcCamFrameReset(640, 480, FRAME_INTERVAL_FPS_15);

Resume Streaming

usb->uvcCamResume(NULL);

New configuration takes effect after resume.

Suspend After Start

Automatically suspend after starting (useful for initialization):

src/original/include/usb_stream.h

// Using C API
uvc_config_t uvc_config = {
    .frame_width = FRAME_RESOLUTION_ANY,
    .frame_height = FRAME_RESOLUTION_ANY,
    .frame_interval = FRAME_INTERVAL_FPS_15,
    // ... buffer configuration ...
    .flags = FLAG_UVC_SUSPEND_AFTER_START  // Suspend on start
};

Changing Resolution at Runtime

Get Available Resolutions

Query supported resolutions from the connected camera:

size_t frame_size = 0;
size_t frame_index = 0;

// Get list size
usb->uvcCamGetFrameListSize(&frame_size, &frame_index);

// Allocate and retrieve list
uvc_frame_size_t *frame_list = 
    (uvc_frame_size_t *)malloc(frame_size * sizeof(uvc_frame_size_t));
usb->uvcCamGetFrameSize(frame_list);

// Print available resolutions
for (size_t i = 0; i < frame_size; i++) {
    ESP_LOGI(TAG, "Resolution[%u]: %ux%u, interval: %u-%u (step: %u)",
             i,
             frame_list[i].width,
             frame_list[i].height,
             frame_list[i].interval_min,
             frame_list[i].interval_max,
             frame_list[i].interval_step);
}

free(frame_list);

Suspend Streaming

usb->uvcCamSuspend(NULL);

Reset Frame Parameters

// Change to specific resolution
usb->uvcCamFrameReset(640, 480, FRAME_INTERVAL_FPS_15);

// Or keep current resolution, only change frame rate
usb->uvcCamFrameReset(0, 0, FRAME_INTERVAL_FPS_30);

Setting both width and height to 0 means no change to resolution, only frame interval is updated.

Resume Streaming

usb->uvcCamResume(NULL);

The camera will now stream with the new configuration.

Complete Example

#include "USB_STREAM.h"

static const char *TAG = "UVC_Demo";
USB_STREAM *usb_stream = nullptr;

void frame_callback(uvc_frame_t *frame, void *ptr) {
    ESP_LOGI(TAG, "Frame %u: %ux%u, %u bytes",
             frame->sequence,
             frame->width,
             frame->height,
             frame->data_bytes);
    
    // Process MJPEG frame data here
}

void state_callback(usb_stream_state_t event, void *arg) {
    if (event == STREAM_CONNECTED) {
        ESP_LOGI(TAG, "Camera connected");
    } else {
        ESP_LOGI(TAG, "Camera disconnected");
    }
}

void setup() {
    Serial.begin(115200);
    
    // Allocate buffers
    uint8_t *xfer_a = (uint8_t *)heap_caps_malloc(55 * 1024,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    uint8_t *xfer_b = (uint8_t *)heap_caps_malloc(55 * 1024,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    uint8_t *frame_buf = (uint8_t *)heap_caps_malloc(55 * 1024,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    
    if (!xfer_a || !xfer_b || !frame_buf) {
        ESP_LOGE(TAG, "Buffer allocation failed!");
        return;
    }
    
    // Create and configure
    usb_stream = new USB_STREAM();
    usb_stream->uvcConfiguration(
        FRAME_RESOLUTION_ANY,
        FRAME_RESOLUTION_ANY,
        FRAME_INTERVAL_FPS_15,
        55 * 1024,
        xfer_a,
        xfer_b,
        55 * 1024,
        frame_buf
    );
    
    // Register callbacks
    usb_stream->uvcCamRegisterCb(&frame_callback, NULL);
    usb_stream->registerState(&state_callback, NULL);
    
    // Start streaming
    usb_stream->start();
    usb_stream->connectWait(10000);
    
    ESP_LOGI(TAG, "UVC streaming started");
}

void loop() {
    delay(1000);
}

Buffer Size Guidelines

Resolution-Based Buffer Sizing

Resolution	Typical MJPEG Frame Size	Recommended Buffer
320x240	8-15 KB	20 KB
480x320	15-25 KB	35 KB
640x480	25-40 KB	55 KB
800x600	40-60 KB	75 KB
1024x768	60-100 KB	120 KB

Actual frame sizes vary based on image complexity and compression. Always test with your specific camera.

ESP32-S2 vs ESP32-S3 Considerations

ESP32-S2: Limited RAM, use 45 KB buffers for resolutions up to 640x480
ESP32-S3: More RAM available, can use 55 KB+ buffers for higher resolutions
Both require DMA-capable memory (MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL)

Next Steps

Learn about UAC Audio Setup for microphone and speaker
Optimize Memory Management for your application
Review Troubleshooting for common issues

Get Started

Core Concepts

Guides

Examples

Overview

Buffer Configuration

Resolution Configuration

Using Any Resolution

Using Specific Resolution

Frame Rate Configuration

Registering Frame Callbacks

Starting and Stopping Streaming

Basic Start/Stop

With State Callback

Suspend and Resume Workflow

Suspend After Start

Changing Resolution at Runtime

Complete Example

Buffer Size Guidelines

Next Steps

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Examples

​Overview

​Buffer Configuration

​Resolution Configuration

​Using Any Resolution

​Using Specific Resolution

​Frame Rate Configuration

​Registering Frame Callbacks

​Starting and Stopping Streaming

​Basic Start/Stop

​With State Callback

​Suspend and Resume Workflow

​Suspend After Start

​Changing Resolution at Runtime

​Complete Example

​Buffer Size Guidelines

​Next Steps

Build docs developers (and LLMs) love

Overview

Buffer Configuration

Resolution Configuration

Using Any Resolution

Using Specific Resolution

Frame Rate Configuration

Registering Frame Callbacks

Starting and Stopping Streaming

Basic Start/Stop

With State Callback

Suspend and Resume Workflow

Suspend After Start

Changing Resolution at Runtime

Complete Example

Buffer Size Guidelines

Next Steps