LibAudio

LibAudio provides comprehensive audio file loading, decoding, and encoding capabilities for SerenityOS. It supports multiple audio formats and offers a unified interface for audio stream processing.

Overview

LibAudio handles audio data through a plugin-based architecture that supports various codecs and formats:

Loader System: Format-agnostic audio file loading
Codec Support: FLAC, MP3, WAV, and QOA formats
Sample Processing: Multi-channel audio with various sample formats
Metadata Extraction: ID3 tags and embedded pictures
Server Connection: Integration with AudioServer for playback

Core Components

Loader

The main interface for loading audio files.

Loader

class

High-level audio file loader supporting multiple formats.

class Loader : public RefCounted<Loader> {
    static ErrorOr<NonnullRefPtr<Loader>, LoaderError> create(StringView path);
    static ErrorOr<NonnullRefPtr<Loader>, LoaderError> create(ReadonlyBytes buffer);
    
    LoaderSamples get_more_samples(size_t samples_to_read_from_input = 128 * KiB);
    MaybeLoaderError reset();
    MaybeLoaderError seek(int const position);
};

Factory Methods:

create(StringView path): Load from file path
create(ReadonlyBytes buffer): Load from memory buffer

Audio Properties

sample_rate

method

Get the sample rate in Hz.

u32 sample_rate() const;

Common values: 44100, 48000, 96000 Hz

num_channels

method

Get the number of audio channels.

u16 num_channels() const;

Typical values: 1 (mono), 2 (stereo), 6 (5.1 surround)

total_samples

method

Get total sample count (per channel).

int total_samples() const;

For a 3-second stereo file at 44.1kHz: returns 132,300

loaded_samples

method

Get number of samples loaded so far.

int loaded_samples() const;

Sample Formats

Supported PCM Formats

LibAudio supports various PCM sample formats:

enum class PcmSampleFormat {
    Uint8,
    Int16,
    Int24,
    Int32,
    Float32,
    Float64
};

u16 bits_per_sample() const;
PcmSampleFormat pcm_format() const;

The Sample type represents a normalized floating-point sample value.

Supported Formats

FLAC (Free Lossless Audio Codec)

FlacLoader

class

Lossless audio compression format loader.Features:

Lossless compression
Seek table support
Embedded metadata and pictures
Streaming decoding

Location: LibAudio/FlacLoader.h

MP3 (MPEG Audio Layer III)

MP3Loader

class

Lossy audio compression format loader.Features:

Frame-based decoding
Variable and constant bitrate support
ID3v2 tag parsing
Huffman table decoding

Location: LibAudio/MP3Loader.h

WAV (Waveform Audio)

Uncompressed PCM audio format with RIFF container.

QOA (Quite OK Audio)

Lightweight lossy audio compression format.

Metadata Support

Metadata

struct

Audio file metadata including tags and embedded data.

Metadata const& metadata() const;

Contains:

Artist, album, title information
Track numbers and dates
Genre and comment fields
Custom metadata tags

Picture Data

Embedded Album Art

LibAudio can extract embedded pictures from audio files:

struct PictureData {
    ID3PictureType type;        // FrontCover, BackCover, etc.
    String mime_string;          // "image/jpeg", "image/png"
    String description_string;
    
    u32 width;
    u32 height;
    u32 color_depth;
    u32 colors;
    
    Vector<u8> data;             // Image data
};

Vector<PictureData> const& pictures() const;

Picture Types:

FrontCover: Album front cover
BackCover: Album back cover
LeadArtist: Lead artist photo
Media: Media (e.g., CD) image
And 17 other types defined in ID3v2 spec

Seek Points

SeekTable

class

Efficient seeking within audio streams.

struct SeekPoint {
    u64 sample_index;
    u64 byte_offset;
};

class SeekTable {
    Optional<SeekPoint const&> seek_point_before(u64 sample_index) const;
    ErrorOr<void> insert_seek_point(SeekPoint);
    size_t size() const;
};

Seek tables enable fast random access within compressed audio files.

Seeking Precision:

Maximum seek point distance: 1000ms
Seek tolerance: ±5000ms
Loaders may provide frame-accurate seeking

Audio Server Integration

Connection Management

Server Connections

LibAudio provides connections to AudioServer and AudioManagerServer:

class ConnectionToServer {
    // Main audio playback connection
};

class ConnectionToManagerServer {
    // Audio device and configuration management
};

Location:

ConnectionToServer.h: Audio playback
ConnectionToManagerServer.h: Device management

Plugin Architecture

LoaderPlugin

class

Base class for format-specific audio decoders.

class LoaderPlugin {
    virtual ErrorOr<Vector<FixedArray<Sample>>, LoaderError> 
        load_chunks(size_t samples_to_read_from_input) = 0;
    
    virtual MaybeLoaderError reset() = 0;
    virtual MaybeLoaderError seek(int const sample_index) = 0;
    
    virtual int loaded_samples() = 0;
    virtual int total_samples() = 0;
    virtual u32 sample_rate() = 0;
    virtual u16 num_channels() = 0;
    
    virtual ByteString format_name() = 0;
    virtual PcmSampleFormat pcm_format() = 0;
};

Chunk-Based Loading

Audio is loaded in chunks for efficient streaming:

Chunks are format-specific (FLAC frames, MP3 frames, etc.)
Default buffer size: 8 KiB
Loaders return multiple chunks to satisfy sample requests
May overshoot requested sample count by small amounts

Error Handling

LoaderError

enum

Audio loading and decoding errors.

using LoaderSamples = ErrorOr<FixedArray<Sample>, LoaderError>;
using MaybeLoaderError = ErrorOr<void, LoaderError>;

Location: LibAudio/LoaderError.h

Generic Types

Sample Representation

LibAudio uses normalized floating-point samples:

// Defined in GenericTypes.h
struct Sample {
    float left;
    float right;
    // ... additional channels
};

Samples are normalized to the range [-1.0, 1.0] regardless of source format.

Encoding Support

FlacWriter

class

FLAC encoding capabilities.

class FlacWriter {
    // Encode audio to FLAC format
};

Location: LibAudio/FlacWriter.h

Usage Example

// Load audio file
auto loader = TRY(Audio::Loader::create("/path/to/audio.flac"));

// Get audio properties
dbgln("Sample rate: {} Hz", loader->sample_rate());
dbgln("Channels: {}", loader->num_channels());
dbgln("Duration: {} samples", loader->total_samples());
dbgln("Format: {}", loader->format_name());

// Read samples
while (loader->loaded_samples() < loader->total_samples()) {
    auto samples = TRY(loader->get_more_samples());
    // Process samples...
}

// Access metadata
auto const& metadata = loader->metadata();
dbgln("Title: {}", metadata.title);

// Extract album art
for (auto const& picture : loader->pictures()) {
    if (picture.type == Audio::ID3PictureType::FrontCover) {
        // Use picture.data
    }
}

Source Location

Directory: Userland/Libraries/LibAudio/ Key Files:

Loader.h: Main loader interface
GenericTypes.h: Sample and common types
FlacLoader.h: FLAC decoder
MP3Loader.h: MP3 decoder
LoaderError.h: Error definitions

Core Libraries

System Libraries

Overview

Core Components

Loader

Audio Properties

Sample Formats

Supported Formats

FLAC (Free Lossless Audio Codec)

MP3 (MPEG Audio Layer III)

WAV (Waveform Audio)

QOA (Quite OK Audio)

Metadata Support

Picture Data

Seeking and Navigation

Seek Points

Audio Server Integration

Connection Management

Plugin Architecture

Chunk-Based Loading

Error Handling

Generic Types

Encoding Support

Usage Example

Source Location

Build docs developers (and LLMs) love

Core Libraries

System Libraries

​Overview

​Core Components

​Loader

​Audio Properties

​Sample Formats

​Supported Formats

​FLAC (Free Lossless Audio Codec)

​MP3 (MPEG Audio Layer III)

​WAV (Waveform Audio)

​QOA (Quite OK Audio)

​Metadata Support

​Picture Data

​Seeking and Navigation

​Seek Points

​Audio Server Integration

​Connection Management

​Plugin Architecture

​Chunk-Based Loading

​Error Handling

​Generic Types

​Encoding Support

​Usage Example

​Source Location

Build docs developers (and LLMs) love

Overview

Core Components

Loader

Audio Properties

Sample Formats

Supported Formats

FLAC (Free Lossless Audio Codec)

MP3 (MPEG Audio Layer III)

WAV (Waveform Audio)

QOA (Quite OK Audio)

Metadata Support

Picture Data

Seeking and Navigation

Seek Points

Audio Server Integration

Connection Management

Plugin Architecture

Chunk-Based Loading

Error Handling

Generic Types

Encoding Support

Usage Example

Source Location