Skip to main content

​
Overview

The AudioDecoder class provides hardware-accelerated AAC audio decoding using Android’s MediaCodec API and plays the decoded PCM audio through AudioTrack. It runs in a separate thread for continuous audio playback. Package: org.client.scrcpy.decoder Audio Codec: audio/mp4a-latm (AAC-LC) Sample Rate: 48000 Hz Channels: 2 (Stereo) Encoding: PCM 16-bit

​
Initialization

​
start()

Starts the decoder worker thread. 
public void start()
Example: 
AudioDecoder audioDecoder = new AudioDecoder();
audioDecoder.start();

​
stop()

Stops the decoder, AudioTrack, and releases resources. 
public void stop()
Example: 
@Override
protected void onDestroy() {
    if (audioDecoder != null) {
        audioDecoder.stop();
    }
    super.onDestroy();
}

​
Configuration

​
configure()

Configures the MediaCodec decoder with AAC codec-specific data. 
public void configure(byte[] data)
​
data
byte[]
required
Codec-specific data (csd-0) containing AAC configuration
Example: 
// Receive CONFIG packet
AudioPacket configPacket = AudioPacket.readHead(packet);
if (configPacket.flag == AudioPacket.Flag.CONFIG) {
    byte[] configData = new byte[dataLength];
    System.arraycopy(packet, AudioPacket.getHeadLen(), 
                    configData, 0, dataLength);
    audioDecoder.configure(configData);
}
Internal Implementation: 
// From source (AudioDecoder.java:90-109)
MediaFormat format = MediaFormat.createAudioFormat(
    MIMETYPE_AUDIO_AAC, // "audio/mp4a-latm"
    SAMPLE_RATE,        // 48000
    2                   // stereo channels
);

// Set bit rate
format.setInteger(MediaFormat.KEY_BIT_RATE, 128000);

// Set codec-specific data
format.setByteBuffer("csd-0", ByteBuffer.wrap(data));

mCodec = MediaCodec.createDecoderByType(MIMETYPE_AUDIO_AAC);
mCodec.configure(format, null, null, 0);
mCodec.start();

// Initialize and start AudioTrack
initAudioTrack();
audioTrack.play();
Calling configure() while already configured will stop the current decoder and AudioTrack, then create new instances. This allows dynamic reconfiguration.

​
Decoding

​
decodeSample()

Queues an encoded audio sample for decoding and playback. 
public void decodeSample(byte[] data, int offset, int size, 
                         long presentationTimeUs, int flags)
​
data
byte[]
required
Byte array containing the encoded audio frame
​
offset
int
required
Starting position in the data array
​
size
int
required
Number of bytes to decode
​
presentationTimeUs
long
required
Presentation timestamp in microseconds
​
flags
int
required
MediaCodec flags
Example: 
AudioPacket audioPacket = AudioPacket.readHead(packet);
if (audioPacket.flag == AudioPacket.Flag.FRAME || 
    audioPacket.flag == AudioPacket.Flag.KEY_FRAME) {
    
    audioDecoder.decodeSample(
        packet,
        AudioPacket.getHeadLen(),
        packet.length - AudioPacket.getHeadLen(),
        audioPacket.presentationTimeStamp,
        audioPacket.flag.getFlag()
    );
}
Frame Processing: 
// From source (AudioDecoder.java:116-129)
int index = mCodec.dequeueInputBuffer(-1);
if (index >= 0) {
    ByteBuffer buffer;
    
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {
        buffer = mCodec.getInputBuffers()[index];
        buffer.clear();
    } else {
        buffer = mCodec.getInputBuffer(index);
    }
    
    if (buffer != null) {
        buffer.put(data, offset, size);
        mCodec.queueInputBuffer(index, 0, size, presentationTimeUs, flags);
    }
}

​
AudioTrack Integration

The decoder automatically manages an AudioTrack instance for audio playback.

​
AudioTrack Configuration

// From source (AudioDecoder.java:28-32)
private void initAudioTrack() {
    int bufferSizeInBytes = AudioTrack.getMinBufferSize(
        SAMPLE_RATE,                      // 48000 Hz
        AudioFormat.CHANNEL_OUT_STEREO,   // 2 channels
        AudioFormat.ENCODING_PCM_16BIT    // 16-bit PCM
    );
    
    audioTrack = new AudioTrack(
        AudioManager.STREAM_MUSIC,
        SAMPLE_RATE,
        AudioFormat.CHANNEL_OUT_STEREO,
        AudioFormat.ENCODING_PCM_16BIT,
        bufferSizeInBytes,
        AudioTrack.MODE_STREAM
    );
}

​
Audio Playback Loop

// From source (AudioDecoder.java:140-157)
int index = mCodec.dequeueOutputBuffer(info, 0);
if (index >= 0) {
    if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) 
        == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
        break;
    }
    
    // Read decoded PCM data
    ByteBuffer outputBuffer = mCodec.getOutputBuffer(index);
    if (outputBuffer != null) {
        byte[] data = new byte[info.size];
        outputBuffer.get(data);
        outputBuffer.clear();
        
        // Write to AudioTrack for playback
        audioTrack.write(data, 0, info.size);
    }
    
    // Release output buffer
    mCodec.releaseOutputBuffer(index, true);
}

​
AudioPacket Structure

Audio data is transmitted using the AudioPacket structure.

​
Packet Format

Header (10 bytes): 
Byte 0:    Type (1 = AUDIO)
Byte 1:    Flag (frame type)
Bytes 2-9: Presentation timestamp (long)
Total packet structure: 
[Type][Flag][Timestamp][Data...]
  1B    1B      8B       Variable

​
Flag Types

​
FRAME
byte
Value: 0 - Regular audio frame
​
KEY_FRAME
byte
Value: 1 - Key audio frame
​
CONFIG
byte
Value: 2 - Configuration data (codec-specific)
​
END
byte
Value: 4 - End of audio stream

​
Reading Packets

// Read packet header only
AudioPacket audioPacket = AudioPacket.readHead(packet);

// Access packet properties
MediaPacket.Type type = audioPacket.type;              // AUDIO
AudioPacket.Flag flag = audioPacket.flag;             // FRAME, CONFIG, etc.
long timestamp = audioPacket.presentationTimeStamp;    // microseconds

// Extract data from full packet
int dataLength = packet.length - AudioPacket.getHeadLen();
byte[] audioData = new byte[dataLength];
System.arraycopy(packet, AudioPacket.getHeadLen(), audioData, 0, dataLength);

​
Usage Example

Complete example of setting up and using the audio decoder: 
public class AudioPlayer {
    private AudioDecoder audioDecoder;
    
    public void startAudioPlayback() {
        // 1. Initialize decoder
        audioDecoder = new AudioDecoder();
        audioDecoder.start();
        
        // 2. Wait for configuration packet
        byte[] packet = receiveAudioPacket();
        AudioPacket configPacket = AudioPacket.readHead(packet);
        
        if (configPacket.flag == AudioPacket.Flag.CONFIG) {
            // Extract configuration data
            int dataLength = packet.length - AudioPacket.getHeadLen();
            byte[] configData = new byte[dataLength];
            System.arraycopy(packet, AudioPacket.getHeadLen(), 
                           configData, 0, dataLength);
            
            // Configure decoder
            audioDecoder.configure(configData);
        }
        
        // 3. Decode and play audio frames
        while (isPlaying) {
            packet = receiveAudioPacket();
            AudioPacket audioPacket = AudioPacket.readHead(packet);
            
            if (audioPacket.flag == AudioPacket.Flag.FRAME || 
                audioPacket.flag == AudioPacket.Flag.KEY_FRAME) {
                
                audioDecoder.decodeSample(
                    packet,
                    AudioPacket.getHeadLen(),
                    packet.length - AudioPacket.getHeadLen(),
                    audioPacket.presentationTimeStamp,
                    audioPacket.flag.getFlag()
                );
            }
            
            if (audioPacket.flag == AudioPacket.Flag.END) {
                break;
            }
        }
        
        // 4. Cleanup
        audioDecoder.stop();
    }
}

​
Threading Model

The AudioDecoder uses an internal Worker thread: 
// From source (AudioDecoder.java:134-167)
@Override
public void run() {
    try {
        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        while (mIsRunning.get()) {
            if (mIsConfigured.get()) {
                int index = mCodec.dequeueOutputBuffer(info, 0);
                if (index >= 0) {
                    // Check for end of stream
                    if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) 
                        == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
                        break;
                    }
                    
                    // Decode and play PCM audio
                    ByteBuffer outputBuffer = mCodec.getOutputBuffer(index);
                    if (outputBuffer != null) {
                        byte[] data = new byte[info.size];
                        outputBuffer.get(data);
                        outputBuffer.clear();
                        audioTrack.write(data, 0, info.size);
                    }
                    
                    mCodec.releaseOutputBuffer(index, true);
                }
            } else {
                // Wait for configuration
                Thread.sleep(5);
            }
        }
    } catch (IllegalStateException e) {
        // Handle codec errors
    }
}

​
Audio Specifications

​
MIMETYPE_AUDIO_AAC
String
Value: "audio/mp4a-latm" - AAC Low Complexity profile
​
SAMPLE_RATE
int
Value: 48000 - 48 kHz sampling rate
​
CHANNELS
int
Value: 2 - Stereo output
​
BIT_RATE
int
Value: 128000 - 128 kbps encoding bitrate
​
PCM_ENCODING
int
Value: AudioFormat.ENCODING_PCM_16BIT - 16-bit PCM output

​
Performance Considerations

The decoder uses hardware acceleration and real-time audio playback. Ensure that:
  • The device supports AAC hardware decoding
  • Audio focus is properly managed
  • The AudioTrack stream type is appropriate for your use case
  • Frame processing keeps up with the sample rate to avoid buffer underruns
Key Points:
  • Decoded PCM data is written directly to AudioTrack
  • Input buffer dequeue timeout is set to -1 (wait indefinitely)
  • Output buffer dequeue timeout is 0 (non-blocking)
  • AudioTrack uses MODE_STREAM for continuous playback
  • Minimum buffer size is calculated automatically

​
Common Issues

​
Buffer Underruns

If you experience audio stuttering: 
// Increase AudioTrack buffer size
int minBufferSize = AudioTrack.getMinBufferSize(...);
int bufferSize = minBufferSize * 2; // Double the minimum size

audioTrack = new AudioTrack(
    AudioManager.STREAM_MUSIC,
    SAMPLE_RATE,
    AudioFormat.CHANNEL_OUT_STEREO,
    AudioFormat.ENCODING_PCM_16BIT,
    bufferSize, // Larger buffer
    AudioTrack.MODE_STREAM
);

​
Audio Focus

Manage audio focus properly: 
AudioManager audioManager = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
int result = audioManager.requestAudioFocus(
    focusChangeListener,
    AudioManager.STREAM_MUSIC,
    AudioManager.AUDIOFOCUS_GAIN
);

if (result == AudioManager.AUDIOFOCUS_REQUEST_GRANTED) {
    audioDecoder.start();
}

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us