Database Integration

TAPLE Core provides a flexible database abstraction layer that allows you to integrate custom storage backends. This guide explains how to implement and use database managers and collections.

Database Architecture

TAPLE Core’s database system consists of two main traits:

DatabaseManager: Creates and manages database collections
DatabaseCollection: Provides CRUD operations for stored data

The database module is defined in core/src/database/mod.rs with the main DB wrapper in core/src/database/db.rs:28.

Database Structure

pub struct DB<C: DatabaseCollection> {
    signature_db: SignatureDb<C>,
    subject_db: SubjectDb<C>,
    event_db: EventDb<C>,
    prevalidated_event_db: PrevalidatedEventDb<C>,
    event_request_db: EventRequestDb<C>,
    request_db: RequestDb<C>,
    controller_id_db: ControllerIdDb<C>,
    validation_db: ValidationDb<C>,
    contract_db: ContractDb<C>,
    witness_signatures_db: WitnessSignaturesDb<C>,
    subject_by_governance_db: SubjectByGovernanceDb<C>,
    keys_db: KeysDb<C>,
    preauthorized_subjects_and_providers_db: PreauthorizedSbujectsAndProovidersDb<C>,
    lce_validation_proofs_db: LceValidationProofs<C>,
    approvals_db: ApprovalsDb<C>,
}

Defined in core/src/database/db.rs:28-60.

DatabaseCollection Trait

Implement this trait for your storage backend:

pub trait DatabaseCollection {
    fn get(&self, key: &str) -> Result<Vec<u8>, Error>;
    fn put(&self, key: &str, data: Vec<u8>) -> Result<(), Error>;
    fn del(&self, key: &str) -> Result<(), Error>;
    fn iter<'a>(
        &'a self,
        reverse: bool,
        prefix: String,
    ) -> Box<dyn Iterator<Item = (String, Vec<u8>)> + 'a>;
}

Methods

get: Retrieve value by key
put: Store key-value pair
del: Delete entry by key
iter: Iterate over entries with optional prefix filter

DatabaseManager Trait

Implement this trait to manage collections:

pub trait DatabaseManager<C: DatabaseCollection> {
    fn default() -> Self;
    fn create_collection(&self, identifier: &str) -> C;
}

In-Memory Implementation

TAPLE Core includes a reference implementation using in-memory storage.

MemoryManager

pub struct MemoryManager {
    data: RwLock<HashMap<String, Arc<DataStore>>>,
}

impl MemoryManager {
    pub fn new() -> Self {
        Self {
            data: RwLock::new(HashMap::new()),
        }
    }
}

Defined in core/src/database/memory.rs:40.

Implementing DatabaseManager

impl DatabaseManager<MemoryCollection> for MemoryManager {
    fn default() -> Self {
        Self {
            data: RwLock::new(HashMap::new()),
        }
    }

    fn create_collection(&self, _identifier: &str) -> MemoryCollection {
        let mut lock = self.data.write().unwrap();
        let db: Arc<DataStore> = match lock.get("") {
            Some(map) => map.clone(),
            None => {
                let db: Arc<DataStore> = Arc::new(DataStore::new());
                lock.insert("".to_string(), db.clone());
                db
            }
        };
        MemoryCollection { data: db }
    }
}

See core/src/database/memory.rs:52.

MemoryCollection

pub struct MemoryCollection {
    data: Arc<DataStore>,
}

impl DatabaseCollection for MemoryCollection {
    fn get(&self, key: &str) -> Result<Vec<u8>, Error> {
        let lock = self.data._get_inner_read_lock();
        let Some(data) = lock.get(key) else {
            return Err(Error::EntryNotFound);
        };
        Ok(data.clone())
    }

    fn put(&self, key: &str, data: Vec<u8>) -> Result<(), Error> {
        let mut lock = self.data._get_inner_write_lock();
        lock.insert(key.to_string(), data);
        Ok()
    }

    fn del(&self, key: &str) -> Result<(), Error> {
        let mut lock = self.data._get_inner_write_lock();
        lock.remove(key);
        Ok()
    }

    fn iter<'a>(
        &'a self,
        reverse: bool,
        prefix: String,
    ) -> Box<dyn Iterator<Item = (String, Vec<u8>)> + 'a> {
        if reverse {
            Box::new(self.data.rev_iter(prefix))
        } else {
            Box::new(self.data.iter(prefix))
        }
    }
}

Implemented in core/src/database/memory.rs:74-110.

Creating Custom Database Implementation

1. Define Your Collection Type

pub struct MyDatabaseCollection {
    connection: MyDatabaseConnection,
    collection_name: String,
}

2. Implement DatabaseCollection

impl DatabaseCollection for MyDatabaseCollection {
    fn get(&self, key: &str) -> Result<Vec<u8>, Error> {
        // Implement retrieval logic
        self.connection
            .get(&self.collection_name, key)
            .map_err(|_| Error::EntryNotFound)
    }

    fn put(&self, key: &str, data: Vec<u8>) -> Result<(), Error> {
        // Implement storage logic
        self.connection
            .put(&self.collection_name, key, data)
            .map_err(|_| Error::CustomError)
    }

    fn del(&self, key: &str) -> Result<(), Error> {
        // Implement deletion logic
        self.connection
            .delete(&self.collection_name, key)
            .map_err(|_| Error::CustomError)
    }

    fn iter<'a>(
        &'a self,
        reverse: bool,
        prefix: String,
    ) -> Box<dyn Iterator<Item = (String, Vec<u8>)> + 'a> {
        // Implement iteration logic
        Box::new(MyIterator::new(
            &self.connection,
            &self.collection_name,
            reverse,
            prefix,
        ))
    }
}

3. Implement DatabaseManager

pub struct MyDatabaseManager {
    connection_pool: Arc<MyConnectionPool>,
}

impl DatabaseManager<MyDatabaseCollection> for MyDatabaseManager {
    fn default() -> Self {
        Self {
            connection_pool: Arc::new(MyConnectionPool::new()),
        }
    }

    fn create_collection(&self, identifier: &str) -> MyDatabaseCollection {
        let connection = self.connection_pool.get_connection();
        MyDatabaseCollection {
            connection,
            collection_name: identifier.to_string(),
        }
    }
}

4. Initialize TAPLE Node

let database = MyDatabaseManager::default();
let (node, api) = Node::build(settings, database)?;

Using the Database

Initialize DB Wrapper

Create the DB instance:

let db = DB::new(Arc::new(database_manager));

Implemented in core/src/database/db.rs:63.

Subject Operations

// Store a subject
db.set_subject(&subject_id, subject)?;

// Retrieve a subject
let subject = db.get_subject(&subject_id)?;

// Get all subjects
let subjects = db.get_all_subjects();

// Get subjects with pagination
let subjects = db.get_subjects(Some("start_key".to_string()), 10)?;

// Delete a subject
db.del_subject(&subject_id)?;

Defined in core/src/database/db.rs:129-155.

Event Operations

// Store an event
db.set_event(&subject_id, signed_event)?;

// Get a specific event
let event = db.get_event(&subject_id, sn)?;

// Get event range
let events = db.get_events_by_range(
    &subject_id,
    Some(0),  // from sequence number
    10        // quantity
)?;

// Delete an event
db.del_event(&subject_id, sn)?;

See core/src/database/db.rs:157-185.

Validation Operations

// Store validation proof
db.set_validation_register(&subject_id, &validation_proof)?;

// Retrieve validation proof
let proof = db.get_validation_register(&subject_id)?;

// Store signatures with proof
db.set_signatures(&subject_id, sn, signatures, validation_proof)?;

// Get signatures and proof
let (signatures, proof) = db.get_signatures(&subject_id, sn)?;

// Get validation proof (latest)
let (signatures, proof) = db.get_validation_proof(&subject_id)?;

Implemented in core/src/database/db.rs:99-127 and core/src/database/db.rs:260-274.

Contract Operations

// Store contract
db.put_contract(
    &governance_id,
    "schema_id",
    contract_bytes,
    contract_hash,
    gov_version
)?;

// Retrieve contract
let (contract, hash, version) = db.get_contract(
    &governance_id,
    "schema_id"
)?;

Defined in core/src/database/db.rs:276-294.

Request Operations

// Store TAPLE request
db.set_taple_request(&request_id, &taple_request)?;

// Get TAPLE request
let request = db.get_taple_request(&request_id)?;

// Get all requests
let requests = db.get_taple_all_request();

// Delete request
db.del_taple_request(&request_id)?;

See core/src/database/db.rs:232-250.

Key Management

// Store keys
db.set_keys(&public_key, keypair)?;

// Retrieve keys
let keypair = db.get_keys(&public_key)?;

// Get all keys
let all_keys = db.get_all_keys()?;

// Delete keys
db.del_keys(&public_key)?;

Implemented in core/src/database/db.rs:361-375.

Approval Operations

// Store approval
db.set_approval(&request_id, approval_entity)?;

// Get approval
let approval = db.get_approval(&request_id)?;

// Get approvals with filter
let approvals = db.get_approvals(
    Some(ApprovalState::Pending),
    None,
    20
)?;

// Delete approval
db.del_approval(&request_id)?;

Defined in core/src/database/db.rs:425-448.

Indexing

Subject-Governance Index

// Create index
db.set_governance_index(&subject_id, &governance_id)?;

// Get subjects by governance
let subject_ids = db.get_subjects_by_governance(&governance_id)?;

// Get governance subjects with pagination
let subjects = db.get_governance_subjects(
    &governance_id,
    None,
    Some(10)
)?;

Approval Indexes

// Index by subject
db.set_subject_approval_index(&subject_id, &request_id)?;
let approval_ids = db.get_approvals_by_subject(&subject_id)?;

// Index by governance
db.set_governance_approval_index(&governance_id, &request_id)?;
let approval_ids = db.get_approvals_by_governance(&governance_id)?;

Implemented in core/src/database/db.rs:450-498.

Building a Node with Custom Database

Complete example:

use taple_core::*;

// Initialize your database manager
let database = MyDatabaseManager::new(connection_string)?;

// Configure node settings
let mut settings = Settings::default();
settings.node.secret_key = hex::encode(node_key_pair.secret_key_bytes());

// Build node with custom database
let (mut node, api) = Node::build(settings, database)
    .expect("TAPLE node built");

// Node is now using your custom database

Node initialization is shown in examples/basic_usage/src/main.rs:26.

Error Handling

Database errors:

pub enum Error {
    EntryNotFound,
    CustomError,
    SerializationError,
    DeserializationError,
}

Handle database errors appropriately:

match db.get_subject(&subject_id) {
    Ok(subject) => {
        // Process subject
    },
    Err(Error::EntryNotFound) => {
        // Handle missing subject
    },
    Err(e) => {
        eprintln!("Database error: {:?}", e);
    }
}

Best Practices

Connection Pooling: Use connection pools for database managers
Error Handling: Implement proper error mapping from your database
Transactions: Use transactions for multi-operation atomicity if supported
Indexing: Create appropriate indexes for frequent queries
Serialization: Use consistent serialization (Borsh is recommended)
Performance: Implement efficient iteration with proper buffering
Cleanup: Implement proper resource cleanup in drop handlers
Testing: Test your implementation thoroughly with the provided test suite

Iterator Implementation

Implement efficient iterators:

pub struct MyIterator<'a> {
    cursor: MyCursor<'a>,
    prefix: String,
}

impl<'a> Iterator for MyIterator<'a> {
    type Item = (String, Vec<u8>);

    fn next(&mut self) -> Option<Self::Item> {
        while let Some((key, value)) = self.cursor.next() {
            if key.starts_with(&self.prefix) {
                return Some((key, value));
            }
        }
        None
    }
}

Reference implementation in core/src/database/memory.rs:114-155.

The in-memory implementation in MemoryManager is for testing only. Production deployments should use persistent storage.

Ensure your database implementation is thread-safe. TAPLE Core accesses the database from multiple async tasks.

Overview

Core Concepts

Getting Started

Guides

Advanced

Database Integration

Database Architecture

Database Structure

DatabaseCollection Trait

Methods

DatabaseManager Trait

In-Memory Implementation

MemoryManager

Implementing DatabaseManager

MemoryCollection

Creating Custom Database Implementation

Using the Database

Initialize DB Wrapper

Subject Operations

Event Operations

Validation Operations

Contract Operations

Request Operations

Key Management

Approval Operations

Indexing

Subject-Governance Index

Approval Indexes

Building a Node with Custom Database

Error Handling

Best Practices

Iterator Implementation

Build docs developers (and LLMs) love

Overview

Core Concepts

Getting Started

Guides

Advanced

​Database Architecture

​Database Structure

​DatabaseCollection Trait

​Methods

​DatabaseManager Trait

​In-Memory Implementation

​MemoryManager

​Implementing DatabaseManager

​MemoryCollection

​Creating Custom Database Implementation

​Using the Database

​Initialize DB Wrapper

​Subject Operations

​Event Operations

​Validation Operations

​Contract Operations

​Request Operations

​Key Management

​Approval Operations

​Indexing

​Subject-Governance Index

​Approval Indexes

​Building a Node with Custom Database

​Error Handling

​Best Practices

​Iterator Implementation

Build docs developers (and LLMs) love

Database Architecture

Database Structure

DatabaseCollection Trait

Methods

DatabaseManager Trait

In-Memory Implementation

MemoryManager

Implementing DatabaseManager

MemoryCollection

Creating Custom Database Implementation

Using the Database

Initialize DB Wrapper

Subject Operations

Event Operations

Validation Operations

Contract Operations

Request Operations

Key Management

Approval Operations

Indexing

Subject-Governance Index

Approval Indexes

Building a Node with Custom Database

Error Handling

Best Practices

Iterator Implementation