Skip to main content
Stream is used to attach a video stream to a Filament Texture, enabling real-time rendering of camera feeds, video playback, and other external image sources.

​
Overview

The Stream class supports two different configurations for integrating external image sources:
  • ACQUIRED - Connects to an Android AHardwareBuffer with explicit acquire/release semantics
  • NATIVE - Connects to an Android SurfaceTexture (deprecated)
The Stream class is Android-centric but the concepts apply to platform-specific external image sources.

​
Stream Synchronization

Consider a typical AR or video application structure: 
while (true) {
    // Application work:
    // - Writing image data for a video frame into a Stream
    // - Moving the Filament Camera
    
    if (renderer->beginFrame(swapChain)) {
        renderer->render(view);
        renderer->endFrame();
    }
}
Let’s say video image data at the time of beginFrame() becomes visible to users at time A, and the 3D scene state (including camera) becomes visible at time B.
  • If time A matches time B, the stream is synchronized
  • The thread that calls beginFrame is the main thread
  • Filament invokes low-level graphics commands on the driver thread

​
ACQUIRED Streams

For ACQUIRED streams, Filament explicitly acquires the image and triggers a callback when done with it. This configuration:
  • Provides synchronization guarantees
  • Works especially well with the Vulkan backend
  • Requires the user to manage buffer lifecycle via callbacks

​
NATIVE Streams (Deprecated)

NATIVE streams are deprecated because they:
  • Are backend-specific
  • Do not make synchronization guarantees
  • Have platform-dependent behavior

​
Creation and Usage

A Stream is created using the Stream::Builder and destroyed by calling Engine::destroyStream().

​
Creating an ACQUIRED Stream

filament::Stream* stream = filament::Stream::Builder()
    .width(1920)
    .height(1080)
    .build(*engine);

// Attach to a texture
filament::Texture* texture = filament::Texture::Builder()
    .width(1920)
    .height(1080)
    .sampler(filament::Texture::Sampler::SAMPLER_EXTERNAL)
    .format(filament::Texture::InternalFormat::RGB8)
    .build(*engine);
    
texture->setExternalStream(engine, stream);

engine->destroyStream(stream);

​
Enumerations

​
StreamType

Alias for backend::StreamType.
  • ACQUIRED - Explicit acquire/release with callbacks
  • NATIVE - Platform-specific native stream (deprecated)

​
Builder Methods

​
stream() (Deprecated)

Builder& stream(void* stream) noexcept;
This method is deprecated. Use setAcquiredImage() instead.
Creates a NATIVE stream from an opaque platform object (e.g., Android SurfaceTexture JNI jobject). Parameters:
  • stream - Opaque native stream handle
Returns: Reference to this Builder for chaining calls

​
width()

Builder& width(uint32_t width) noexcept;
Sets the initial width of the incoming stream. This value is stream-dependent and must be set when using external texture IDs on Android. Parameters:
  • width - Initial width of the stream
Returns: Reference to this Builder for chaining calls

​
height()

Builder& height(uint32_t height) noexcept;
Sets the initial height of the incoming stream. This value is stream-dependent and must be set when using external texture IDs on Android. Parameters:
  • height - Initial height of the stream
Returns: Reference to this Builder for chaining calls

​
name()

Builder& name(utils::StaticString const& name) noexcept;
Associates an optional name with this Stream for debugging purposes. The name will appear in error messages. Parameters:
  • name - A string literal to identify this Stream
Returns: Reference to this Builder for chaining calls Example: 
Stream::Builder()
    .width(1920)
    .height(1080)
    .name("CameraStream")
    .build(*engine);

​
build()

Stream* build(Engine& engine);
Creates the Stream object and returns a pointer to it. Parameters:
  • engine - Reference to the Filament Engine
Returns: Pointer to the newly created Stream

​
Instance Methods

​
getStreamType()

StreamType getStreamType() const noexcept;
Indicates whether this stream is a NATIVE stream or ACQUIRED stream. Returns: The stream type

​
setAcquiredImage()

void setAcquiredImage(void* image,
        Callback callback, 
        void* userdata, 
        math::mat3f const& transform = math::mat3f()) noexcept;
Updates an ACQUIRED stream with an image that is guaranteed to be used in the next frame. This method tells Filament to immediately “acquire” the image and trigger a callback when done with it. This should be called:
  • Outside of beginFrame / endFrame
  • Only once per frame
  • On the same thread that calls Renderer::beginFrame
If multiple images are pushed to the same stream in a single frame, only the final image is honored, but all callbacks are invoked.
Parameters:
  • image - Pointer to AHardwareBuffer (casted to void*)
  • callback - Triggered when Filament releases the image. Takes two arguments: the AHardwareBuffer and userdata
  • userdata - Optional closure data passed to the callback
  • transform - Optional transform matrix to apply to the image (default: identity)
Example: 
void releaseCallback(void* image, void* userdata) {
    AHardwareBuffer* buffer = static_cast<AHardwareBuffer*>(image);
    // Release or return buffer to pool
    AHardwareBuffer_release(buffer);
}

// In your render loop (outside beginFrame/endFrame):
stream->setAcquiredImage(
    hardwareBuffer,
    releaseCallback,
    nullptr  // userdata
);

​
setAcquiredImage() with Handler

void setAcquiredImage(void* image,
        backend::CallbackHandler* handler,
        Callback callback, 
        void* userdata,
        math::mat3f const& transform = math::mat3f()) noexcept;
Same as above, but allows specifying a custom callback handler for dispatching the callback. Parameters:
  • image - Pointer to AHardwareBuffer
  • handler - Handler to dispatch the callback, or nullptr for the default handler
  • callback - Release callback function
  • userdata - Optional closure data
  • transform - Optional transform matrix

​
setDimensions()

void setDimensions(uint32_t width, uint32_t height) noexcept;
Updates the size of the incoming stream. Whether this value is used is stream-dependent. On Android, it must be set when using external texture IDs. Parameters:
  • width - New width of the incoming stream
  • height - New height of the incoming stream
Example: 
stream->setDimensions(3840, 2160);  // Update to 4K resolution

​
getTimestamp()

int64_t getTimestamp() const noexcept;
Returns the presentation time of the currently displayed frame in nanoseconds. This value can change at any time. Returns: Timestamp in nanoseconds

​
Complete Example: Camera Stream

Here’s a complete example of using a Stream with a camera on Android: 
#include <filament/Engine.h>
#include <filament/Stream.h>
#include <filament/Texture.h>
#include <filament/Material.h>
#include <android/hardware_buffer.h>

class CameraStreamManager {
public:
    CameraStreamManager(filament::Engine* engine) : mEngine(engine) {
        // Create stream
        mStream = filament::Stream::Builder()
            .width(1920)
            .height(1080)
            .name("ARCameraStream")
            .build(*engine);
        
        // Create external texture
        mTexture = filament::Texture::Builder()
            .width(1920)
            .height(1080)
            .sampler(filament::Texture::Sampler::SAMPLER_EXTERNAL)
            .format(filament::Texture::InternalFormat::RGB8)
            .build(*engine);
        
        mTexture->setExternalStream(engine, mStream);
    }
    
    ~CameraStreamManager() {
        mEngine->destroy(mTexture);
        mEngine->destroy(mStream);
    }
    
    void updateFrame(AHardwareBuffer* buffer) {
        // This should be called outside beginFrame/endFrame
        mStream->setAcquiredImage(
            buffer,
            [](void* image, void* userdata) {
                auto* mgr = static_cast<CameraStreamManager*>(userdata);
                AHardwareBuffer* buf = static_cast<AHardwareBuffer*>(image);
                mgr->releaseBuffer(buf);
            },
            this  // userdata
        );
    }
    
    filament::Texture* getTexture() const { return mTexture; }
    
private:
    void releaseBuffer(AHardwareBuffer* buffer) {
        // Return buffer to camera or release it
        AHardwareBuffer_release(buffer);
    }
    
    filament::Engine* mEngine;
    filament::Stream* mStream;
    filament::Texture* mTexture;
};

// Usage in render loop:
void renderFrame(CameraStreamManager& cameraMgr, 
                filament::Renderer* renderer,
                filament::SwapChain* swapChain,
                filament::View* view) {
    // Update camera stream BEFORE beginFrame
    AHardwareBuffer* latestFrame = getCameraFrame();
    if (latestFrame) {
        cameraMgr.updateFrame(latestFrame);
    }
    
    // Now render
    if (renderer->beginFrame(swapChain)) {
        renderer->render(view);
        renderer->endFrame();
    }
}

​
Best Practices

​
Timing

  1. Call setAcquiredImage() before beginFrame() to ensure synchronization
  2. Call it once per frame to avoid callback overhead
  3. Use the same thread for both setAcquiredImage() and beginFrame()

​
Memory Management

  1. Always implement a release callback to manage buffer lifecycle
  2. Consider using a buffer pool to avoid allocation overhead
  3. Release buffers promptly in the callback to avoid blocking the camera

​
Transform Matrix

The optional transform matrix can be used to:
  • Flip the image (common for camera feeds)
  • Apply rotation
  • Adjust for aspect ratio
// Example: flip vertically
math::mat3f flipY(
    1.0f,  0.0f, 0.0f,
    0.0f, -1.0f, 1.0f,
    0.0f,  0.0f, 1.0f
);

stream->setAcquiredImage(buffer, callback, userdata, flipY);

​
See Also

  • Texture - Create textures and attach streams with setExternalStream()
  • Engine - Create and destroy streams with destroyStream()
  • Sample applications: sample-stream-test, sample-hello-camera

Build docs developers (and LLMs) love

Get started for freeTalk to us