xmtp_db

Overview

The xmtp_db crate provides a durable, encrypted SQLite-based storage layer for XMTP. It handles persistence of groups, messages, identity information, and MLS protocol state using Diesel ORM with encrypted storage via SQLCipher.

Installation

[dependencies]
xmtp_db = "*"

Key Exports

DefaultStore

EncryptedMessageStore<DefaultDatabase>

The default platform-specific encrypted message store implementation.

DefaultDbConnection

DbConnection

Default database connection type for the store.

DefaultMlsStore

SqlKeyStore

Default MLS key store implementation backed by SQL.

EncryptedMessageStore

struct

Main encrypted storage implementation parameterized by database type.

DbConnection

struct

Wrapper around database connections with query execution methods.

Core Modules

encrypted_store

Main encrypted storage implementation:

Database initialization
Migration management
Encrypted operations
Transaction handling

sql_key_store

OpenMLS key store implementation:

MLS group state storage
Key package storage
Cryptographic key management
Credential storage

group

Group conversation storage:

Group metadata
Membership information
Group state tracking
Conversation types (group/DM)

group_message

Message persistence:

Message storage and retrieval
Delivery status tracking
Message sequencing
Reactions and replies

identity

Identity information storage:

Installation identity
Inbox associations
Key rotation state

consent_record

Consent state management:

Per-contact consent
Per-group consent
Allow/deny lists

Main Types and Traits

XmtpDb

trait

Core trait for XMTP database operations.Associated Types:

DbQuery - Query type for database operations

Key Methods:

raw_query_read() - Execute read-only queries
raw_query_write() - Execute write queries
transaction() - Execute transactional operations

EncryptedMessageStore<D>

struct

Encrypted storage implementation.Type Parameters:

D: Database - Underlying database type

Methods:

new() - Create new store with encryption key
new_unencrypted() - Create unencrypted store (testing only)
conn() - Get database connection

StoredGroup

struct

Represents a stored group conversation.Fields:

id: Vec<u8> - Group ID
created_at_ns: i64 - Creation timestamp
membership_state: GroupMembershipState - Current membership
conversation_type: ConversationType - Group or DM
added_by_inbox_id: String - Who added this installation

StoredGroupMessage

struct

Represents a stored message.Fields:

id: Vec<u8> - Message ID
group_id: Vec<u8> - Parent group ID
decrypted_message_bytes: Vec<u8> - Message content
sent_at_ns: i64 - Send timestamp
sender_inbox_id: String - Sender’s inbox ID
kind: GroupMessageKind - Message type
delivery_status: DeliveryStatus - Delivery state
content_type: ContentType - Content encoding

StoredIdentity

struct

Stored installation identity.Fields:

inbox_id: String - Associated inbox ID
installation_keys: Vec<u8> - Installation key bytes
credential_bytes: Vec<u8> - MLS credential
next_key_package_rotation_ns: i64 - Next rotation time

Consent state for a contact or group.Fields:

entity_type: ConsentType - Address, Group, or Inbox
state: ConsentState - Allowed, Denied, or Unknown
entity: String - Entity identifier

Database Traits

Store

trait

Trait for storing entities in the database.

pub trait Store {
    fn store(&self, conn: &impl XmtpDb) -> Result<(), StorageError>;
}

Fetch

trait

Trait for fetching entities from the database.

pub trait Fetch<Key> {
    fn fetch(conn: &impl XmtpDb, key: &Key) -> Result<Self, StorageError>;
}

Delete

trait

Trait for deleting entities.

pub trait Delete {
    fn delete(self, conn: &impl XmtpDb) -> Result<(), StorageError>;
}

Usage Examples

Creating an Encrypted Store

use xmtp_db::{EncryptedMessageStore, EncryptionKey};

// Create encryption key (32 bytes)
let encryption_key = EncryptionKey::try_from(key_bytes)?;

// Create encrypted store
let store = EncryptedMessageStore::new(
    "path/to/database.db",
    encryption_key,
)?;

Storing and Fetching Groups

use xmtp_db::{
    Store, Fetch,
    group::StoredGroup,
    group::ConversationType,
};

// Create and store a group
let group = StoredGroup {
    id: group_id.clone(),
    created_at_ns: now_ns(),
    membership_state: GroupMembershipState::Allowed,
    conversation_type: ConversationType::Group,
    added_by_inbox_id: inbox_id.clone(),
    ..Default::default()
};

group.store(&store)?;

// Fetch the group
let fetched = StoredGroup::fetch(&store, &group_id)?;

Storing Messages

use xmtp_db::group_message::{
    StoredGroupMessage,
    GroupMessageKind,
    DeliveryStatus,
};

let message = StoredGroupMessage {
    id: message_id,
    group_id: group_id.clone(),
    decrypted_message_bytes: content.into(),
    sent_at_ns: timestamp,
    sender_inbox_id: sender_id.clone(),
    kind: GroupMessageKind::Application,
    delivery_status: DeliveryStatus::Published,
    content_type: ContentType::Text,
    ..Default::default()
};

message.store(&store)?;

Querying Messages

use xmtp_db::group_message::{
    QueryGroupMessage,
    MsgQueryArgs,
};

// Query with filters
let messages = store.conn().query_group_messages(
    MsgQueryArgs::default()
        .group_id(group_id)
        .limit(50)
        .sent_after_ns(start_time),
)?;

for msg in messages {
    println!("Message: {:?}", msg.decrypted_message_bytes);
}

use xmtp_db::consent_record::{
    StoredConsentRecord,
    ConsentType,
    ConsentState,
};

// Store consent record
let consent = StoredConsentRecord {
    entity_type: ConsentType::Address,
    state: ConsentState::Allowed,
    entity: wallet_address.clone(),
};

consent.store(&store)?;

// Query consent
let is_allowed = store.conn().is_allowed(
    ConsentType::Address,
    &wallet_address,
)?;

Transactions

use xmtp_db::traits::Transaction;

// Execute multiple operations in a transaction
store.transaction(|conn| {
    group.store(conn)?;
    message.store(conn)?;
    consent.store(conn)?;
    Ok(())
})?;

Using the Prelude

use xmtp_db::prelude::*;

// Import all common query traits and types
let groups = conn.query_groups(
    GroupQueryArgs::default()
        .allowed_states(vec![GroupMembershipState::Allowed])
)?;

Storage Modules

Each storage module provides query traits and types:

QueryGroup

trait

Query operations for groups.Methods:

query_groups() - List groups with filters
find_groups() - Find specific groups

QueryGroupMessage

trait

Query operations for messages.Methods:

query_group_messages() - List messages
get_latest_message() - Get most recent message

QueryIdentity

trait

Query operations for identity.Methods:

fetch_identity() - Get installation identity
store_identity() - Store identity

Query operations for consent.Methods:

is_allowed() - Check if entity is allowed
get_consent_record() - Get consent state

Migrations

Migrations are located in crates/xmtp_db/migrations/ and run automatically:

// Migrations run on store creation
let store = EncryptedMessageStore::new(path, key)?;
// Database is now at latest schema version

Error Types

use xmtp_db::StorageError;

pub enum StorageError {
    Diesel(diesel::result::Error),
    NotFound(NotFound),
    DuplicateItem(DuplicateItem),
    Serialization(String),
    Deserialization(String),
    PoolNeedsConnection,
    DbInit(String),
}

Features

test-utils

feature

Testing utilities including in-memory databases and test helpers.

update-schema

feature

Tools for updating database schema definitions.

Platform Support

Native

Uses SQLCipher with bundled vendored OpenSSL
Connection pooling via r2d2
Full encryption support

WASM

Uses sqlite-wasm-rs
Single-threaded connections
Limited encryption (browser security model)

Testing Utilities

With test-utils feature:

use xmtp_db::test_utils::*;

// Create in-memory test database
let store = TestStore::new()?;

// Register test triggers for tracking
store.conn().register_triggers();

// Check test metadata
let intents_count = store.conn().intents_published();

Performance Considerations

Indexing

The schema includes indexes on commonly queried fields:

Group ID and creation time
Message sent_at and sender
Consent entity lookups

Connection Pooling

On native platforms, use connection pools for concurrent access:

// Connections are pooled automatically
let conn = store.conn();

Batch Operations

Use transactions for batch inserts:

store.transaction(|conn| {
    for message in messages {
        message.store(conn)?;
    }
    Ok(())
})?;

Core Crates

API Clients

Client

Groups

Messages

Identity

Overview

Installation

Key Exports

Core Modules

encrypted_store

sql_key_store

group

group_message

identity

consent_record

Main Types and Traits

Database Traits

Usage Examples

Creating an Encrypted Store

Storing and Fetching Groups

Storing Messages

Querying Messages

Transactions

Using the Prelude

Storage Modules

Migrations

Error Types

Features

Platform Support

Native

WASM

Testing Utilities

Performance Considerations

Indexing

Connection Pooling

Batch Operations

Build docs developers (and LLMs) love

Core Crates

API Clients

Client

Groups

Messages

Identity

​Overview

​Installation

​Key Exports

​Core Modules

​encrypted_store

​sql_key_store

​group

​group_message

​identity

​consent_record

​Main Types and Traits

​Database Traits

​Usage Examples

​Creating an Encrypted Store

​Storing and Fetching Groups

​Storing Messages

​Querying Messages

​Managing Consent

​Transactions

​Using the Prelude

​Storage Modules

​Migrations

​Error Types

​Features

​Platform Support

​Native

​WASM

​Testing Utilities

​Performance Considerations

​Indexing

​Connection Pooling

​Batch Operations

Build docs developers (and LLMs) love

Overview

Installation

Key Exports

Core Modules

encrypted_store

sql_key_store

group

group_message

identity

consent_record

Main Types and Traits

Database Traits

Usage Examples

Creating an Encrypted Store

Storing and Fetching Groups

Storing Messages

Querying Messages

Managing Consent

Transactions

Using the Prelude

Storage Modules

Migrations

Error Types

Features

Platform Support

Native

WASM

Testing Utilities

Performance Considerations

Indexing

Connection Pooling

Batch Operations