Skip to main content

​
Overview

Proper memory management is critical for reliable USB streaming on ESP32-S2/S3. This guide covers buffer allocation strategies, memory requirements, and optimization techniques based on the library’s implementation.

​
Memory Allocation Requirements

​
DMA-Capable Memory

All USB transfer buffers must be allocated in DMA-capable memory: 
void *malloc_dma(size_t size) {
    return heap_caps_malloc(
        size,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
    );
}

void free_dma(void *ptr) {
    heap_caps_free(ptr);
}
Regular malloc() may not allocate DMA-capable memory. Always use heap_caps_malloc() with appropriate flags.

​
Required Capabilities

  • MALLOC_CAP_DMA: Memory accessible by DMA controller
  • MALLOC_CAP_INTERNAL: Internal RAM (not external PSRAM)
  • MALLOC_CAP_8BIT: Byte-addressable memory
PSRAM cannot be used for USB buffers as it’s not DMA-capable.

​
UVC Video Buffer Strategy

​
Double-Buffering Pattern

The library uses double-buffering for USB transfers: 
#define XFER_BUFFER_SIZE (55 * 1024)

// Allocate transfer buffers (double buffer)
uint8_t *xfer_buffer_a = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE,
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);

uint8_t *xfer_buffer_b = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE,
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);

// Allocate frame assembly buffer
uint8_t *frame_buffer = (uint8_t *)heap_caps_malloc(
    XFER_BUFFER_SIZE,
    MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT
);

​
Why Double-Buffering?

1

Buffer A receives USB data

While Buffer A is being filled with USB payload data, Buffer B is being processed.
2

Buffers swap

When Buffer A is full, the library switches to Buffer B for incoming data.
3

Buffer A is processed

While Buffer B receives new data, Buffer A’s content is assembled into a complete frame.
This pattern prevents data loss and ensures smooth streaming without blocking the USB pipe.

​
Memory Requirements by Resolution

​
UVC Video Buffers

ResolutionMJPEG Frame SizeBuffer ABuffer BFrame BufferTotal
320x2408-15 KB20 KB20 KB20 KB60 KB
480x32015-25 KB35 KB35 KB35 KB105 KB
640x48025-40 KB55 KB55 KB55 KB165 KB
800x60040-60 KB75 KB75 KB75 KB225 KB
1024x76860-100 KB120 KB120 KB120 KB360 KB
Buffer sizes must be larger than the maximum expected frame size. MJPEG compression varies based on image complexity.

​
Example from Source Code

test_apps/main/test_usb_stream.c
#ifdef CONFIG_IDF_TARGET_ESP32S2
#define DEMO_XFER_BUFFER_SIZE (45 * 1024)  // ESP32-S2: 45 KB
#else
#define DEMO_XFER_BUFFER_SIZE (55 * 1024)  // ESP32-S3: 55 KB
#endif

// Three buffers needed
uint8_t *xfer_buffer_a = (uint8_t *)_malloc(DEMO_XFER_BUFFER_SIZE);
uint8_t *xfer_buffer_b = (uint8_t *)_malloc(DEMO_XFER_BUFFER_SIZE);
uint8_t *frame_buffer = (uint8_t *)_malloc(DEMO_XFER_BUFFER_SIZE);

// Total memory: 135 KB (S2) or 165 KB (S3)

​
UAC Audio Buffer Strategy

​
Microphone Buffers

Microphone uses an internal ring buffer: 
// Calculate buffer size for 200ms of audio
#define SAMPLE_RATE 16000
#define BIT_DEPTH 16
#define CHANNELS 1
#define DURATION_MS 200

size_t mic_buffer_size = 
    (SAMPLE_RATE / 1000) * (BIT_DEPTH / 8) * CHANNELS * DURATION_MS;

// mic_buffer_size = 6,400 bytes for 16kHz, 16-bit, mono, 200ms

usb->uacConfiguration(
    CHANNELS,
    BIT_DEPTH,
    SAMPLE_RATE,
    mic_buffer_size,  // Internal ring buffer
    0, 0, 0, 0
);

​
Speaker Buffers

Speaker buffer should be larger to prevent underruns: 
// Speaker buffer for 400ms
size_t spk_buffer_size = 
    (SAMPLE_RATE / 1000) * (BIT_DEPTH / 8) * CHANNELS * 400;

// spk_buffer_size = 12,800 bytes for 16kHz, 16-bit, mono, 400ms

usb->uacConfiguration(
    0, 0, 0, 0,
    CHANNELS,
    BIT_DEPTH,
    SAMPLE_RATE,
    spk_buffer_size  // Must be multiple of endpoint MPS
);
Speaker buffer size should be a multiple of the endpoint’s max packet size (MPS) for optimal performance.

​
Memory Allocation Patterns

​
Pattern 1: Static Allocation

Allocate once at initialization: 
class VideoStreamer {
private:
    uint8_t *xfer_a;
    uint8_t *xfer_b;
    uint8_t *frame_buf;
    USB_STREAM *stream;
    
public:
    bool init() {
        // Allocate buffers
        xfer_a = (uint8_t *)heap_caps_malloc(55 * 1024,
            MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
        xfer_b = (uint8_t *)heap_caps_malloc(55 * 1024,
            MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
        frame_buf = (uint8_t *)heap_caps_malloc(55 * 1024,
            MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
        
        if (!xfer_a || !xfer_b || !frame_buf) {
            cleanup();
            return false;
        }
        
        stream = new USB_STREAM();
        stream->uvcConfiguration(
            FRAME_RESOLUTION_ANY, FRAME_RESOLUTION_ANY,
            FRAME_INTERVAL_FPS_15,
            55 * 1024, xfer_a, xfer_b,
            55 * 1024, frame_buf
        );
        
        return true;
    }
    
    void cleanup() {
        if (stream) {
            stream->stop();
            delete stream;
            stream = nullptr;
        }
        heap_caps_free(xfer_a);
        heap_caps_free(xfer_b);
        heap_caps_free(frame_buf);
    }
    
    ~VideoStreamer() {
        cleanup();
    }
};

​
Pattern 2: Dynamic Reallocation

Adjust buffer size based on detected resolution: 
bool reallocate_for_resolution(uint16_t width, uint16_t height) {
    // Stop streaming
    usb_stream->stop();
    
    // Calculate new buffer size
    size_t new_size = calculate_buffer_size(width, height);
    
    // Free old buffers
    heap_caps_free(xfer_a);
    heap_caps_free(xfer_b);
    heap_caps_free(frame_buf);
    
    // Allocate new buffers
    xfer_a = (uint8_t *)heap_caps_malloc(new_size,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    xfer_b = (uint8_t *)heap_caps_malloc(new_size,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    frame_buf = (uint8_t *)heap_caps_malloc(new_size,
        MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
    
    if (!xfer_a || !xfer_b || !frame_buf) {
        return false;
    }
    
    // Reconfigure and restart
    usb_stream->uvcConfiguration(
        width, height, FRAME_INTERVAL_FPS_15,
        new_size, xfer_a, xfer_b,
        new_size, frame_buf
    );
    
    usb_stream->start();
    return true;
}

​
Memory Optimization Techniques

​
1. Use Appropriate Buffer Sizes

// Bad: Oversized buffers waste memory
#define BUFFER_SIZE (200 * 1024)  // 200 KB - wasteful for 640x480

// Good: Size based on actual needs
#define BUFFER_SIZE (55 * 1024)   // 55 KB - adequate for 640x480 MJPEG

​
2. Share Buffers When Possible

Do NOT share buffers between transfer and frame assembly. The library requires separate buffers.
But you can share application-level buffers: 
// Single buffer for frame processing
uint8_t *processing_buffer = (uint8_t *)malloc(100 * 1024);

void frame_callback(uvc_frame_t *frame, void *ptr) {
    // Copy to processing buffer for async work
    memcpy(processing_buffer, frame->data, frame->data_bytes);
    // Queue for processing in another task
}

​
3. Monitor Memory Usage

void print_memory_info() {
    ESP_LOGI(TAG, "Free heap: %u bytes", esp_get_free_heap_size());
    ESP_LOGI(TAG, "Free DMA heap: %u bytes",
             heap_caps_get_free_size(MALLOC_CAP_DMA));
    ESP_LOGI(TAG, "Largest free block: %u bytes",
             heap_caps_get_largest_free_block(MALLOC_CAP_DMA));
    ESP_LOGI(TAG, "Minimum free heap: %u bytes",
             esp_get_minimum_free_heap_size());
}

// Call before and after allocation
print_memory_info();
allocate_buffers();
print_memory_info();

​
4. ESP32-S2 Specific Optimizations

ESP32-S2 has less RAM than S3: 
#ifdef CONFIG_IDF_TARGET_ESP32S2
    // Use smaller buffers
    #define XFER_BUFFER_SIZE (45 * 1024)
    // Limit resolution
    #define MAX_WIDTH 640
    #define MAX_HEIGHT 480
#else  // ESP32-S3
    // Can use larger buffers
    #define XFER_BUFFER_SIZE (55 * 1024)
    #define MAX_WIDTH 800
    #define MAX_HEIGHT 600
#endif

​
Memory Cleanup and Deallocation

​
Proper Cleanup Sequence

1

Stop Streaming

usb_stream->stop();
This ensures no active transfers are using the buffers.
2

Delete USB_STREAM Object

delete usb_stream;
usb_stream = nullptr;
3

Free Buffers

heap_caps_free(xfer_buffer_a);
heap_caps_free(xfer_buffer_b);
heap_caps_free(frame_buffer);

xfer_buffer_a = nullptr;
xfer_buffer_b = nullptr;
frame_buffer = nullptr;

​
Complete Cleanup Example

void cleanup_usb_streaming() {
    if (usb_stream) {
        ESP_LOGI(TAG, "Stopping USB stream...");
        usb_stream->stop();
        
        ESP_LOGI(TAG, "Deleting stream object...");
        delete usb_stream;
        usb_stream = nullptr;
    }
    
    if (xfer_buffer_a) {
        heap_caps_free(xfer_buffer_a);
        xfer_buffer_a = nullptr;
    }
    
    if (xfer_buffer_b) {
        heap_caps_free(xfer_buffer_b);
        xfer_buffer_b = nullptr;
    }
    
    if (frame_buffer) {
        heap_caps_free(frame_buffer);
        frame_buffer = nullptr;
    }
    
    ESP_LOGI(TAG, "Cleanup complete. Free DMA heap: %u bytes",
             heap_caps_get_free_size(MALLOC_CAP_DMA));
}

​
Memory Leak Detection

From the test suite:
test_apps/main/test_usb_stream.c
static size_t before_free_8bit;
static size_t before_free_32bit;

void setUp(void) {
    before_free_8bit = heap_caps_get_free_size(MALLOC_CAP_8BIT);
    before_free_32bit = heap_caps_get_free_size(MALLOC_CAP_32BIT);
}

void tearDown(void) {
    size_t after_free_8bit = heap_caps_get_free_size(MALLOC_CAP_8BIT);
    size_t after_free_32bit = heap_caps_get_free_size(MALLOC_CAP_32BIT);
    
    ssize_t delta_8bit = after_free_8bit - before_free_8bit;
    ssize_t delta_32bit = after_free_32bit - before_free_32bit;
    
    ESP_LOGI(TAG, "8BIT: Before %u, After %u (delta %d)",
             before_free_8bit, after_free_8bit, delta_8bit);
    ESP_LOGI(TAG, "32BIT: Before %u, After %u (delta %d)",
             before_free_32bit, after_free_32bit, delta_32bit);
    
    // Threshold allows for small variations
    #define TEST_MEMORY_LEAK_THRESHOLD (-400)
    assert(delta_8bit >= TEST_MEMORY_LEAK_THRESHOLD);
    assert(delta_32bit >= TEST_MEMORY_LEAK_THRESHOLD);
}

​
Best Practices

Always Use DMA Memory

Use heap_caps_malloc() with MALLOC_CAP_DMA flag for all USB buffers.

Check Allocations

Always verify that malloc returns non-NULL before use.

Right-Size Buffers

Calculate buffer sizes based on actual resolution and format needs.

Clean Up Properly

Stop streaming before freeing buffers. Free in reverse order of allocation.

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us