Skip to main content
This guide covers best practices for using BinaryDB in production applications, including performance optimization, error handling, and data organization strategies.

​
Performance Optimization

​
When to Commit

BinaryDB uses a dirty flag to track whether changes need to be persisted. Understanding when to commit is crucial for performance.
Only commit when you need to persist changes to disk. Excessive commits can impact performance.
from binarydb.database import Database

db = Database("./data/users")
db.load()

# BAD: Committing after every operation
db.set("user:1", {"name": "Alice"})
db.commit()  # Slow!
db.set("user:2", {"name": "Bob"})
db.commit()  # Slow!

# GOOD: Batch operations, then commit once
db.set("user:1", {"name": "Alice"})
db.set("user:2", {"name": "Bob"})
db.set("user:3", {"name": "Charlie"})
db.commit()  # Fast!

db.close()

​
Batch Operations

Group related operations together to minimize disk I/O: 
db = Database("./data/products")
db.load()

# Batch insert multiple records
products = [
    ("product:1", {"name": "Laptop", "price": 999}),
    ("product:2", {"name": "Mouse", "price": 29}),
    ("product:3", {"name": "Keyboard", "price": 79}),
]

for key, value in products:
    db.set(key, value)

db.commit()  # Single commit for all records
db.close()

​
Memory Considerations

BinaryDB stores ALL data in memory. Monitor your dataset size to avoid excessive memory usage.
BinaryDB loads the entire database into memory. Consider these guidelines:
  • Small datasets (< 10 MB): Ideal for BinaryDB
  • Medium datasets (10-100 MB): Acceptable, monitor memory usage
  • Large datasets (> 100 MB): Consider alternatives like SQLite
db = Database("./data/mydb")
db.load()

# Check database size
print(f"Total records: {len(db)}")
print(f"Database state: {db}")

db.close()

​
Error Handling Patterns

​
Always Use Try/Except

Wrap database operations in try/except blocks to handle errors gracefully: 
from binarydb.database import Database
from binarydb.errors import (
    DatabaseIOError,
    DatabaseCorruptedError,
    KeyValidationError,
    RecordTypeError,
)

db = Database("./data/mydb")

try:
    db.load()
except DatabaseCorruptedError:
    print("Database file is corrupted, starting fresh")
except DatabaseIOError as e:
    print(f"Failed to load database: {e}")
    raise

try:
    db.set("user:1", {"name": "Alice", "age": 30})
    db.commit()
except DatabaseIOError as e:
    print(f"Failed to commit changes: {e}")
    raise
finally:
    db.close()

​
Check Exists Before Operations

Use exists() to verify keys before performing operations that expect existing records.
db = Database("./data/users")
db.load()

user_key = "user:123"

# GOOD: Check before updating
if db.exists(user_key):
    try:
        db.update(user_key, {"last_login": "2026-03-04"})
        db.commit()
    except RecordTypeError:
        print("User record is not a dictionary")
else:
    print(f"User {user_key} not found")

db.close()

​
Handle Type Errors

The update() method only works on dictionary records: 
db = Database("./data/config")
db.load()

# This will raise RecordTypeError if value is not a dict
try:
    db.update("config:app", {"debug": True})
    db.commit()
except RecordTypeError:
    print("Cannot update non-dictionary record")
    # Handle appropriately - maybe use set() instead
    db.set("config:app", {"debug": True})
    db.commit()

db.close()

​
Data Organization

​
Key Naming Conventions

Use consistent, hierarchical key naming patterns for better organization: 
db = Database("./data/app")
db.load()

# GOOD: Namespaced, hierarchical keys
db.set("user:1", {"name": "Alice"})
db.set("user:2", {"name": "Bob"})
db.set("post:1:author", "user:1")
db.set("post:1:title", "Hello World")
db.set("config:app:version", "1.0.0")
db.set("config:app:debug", False)

# BAD: Flat, inconsistent keys
db.set("Alice", {"type": "user"})
db.set("post1author", "Alice")
db.set("version", "1.0.0")

db.commit()
db.close()
Common patterns:
  • type:id - Simple entity reference (e.g., user:123)
  • type:id:field - Specific field (e.g., user:123:email)
  • namespace:key - Configuration (e.g., config:db:host)

​
Structuring Nested Data

BinaryDB supports any pickle-serializable Python object, including nested structures.
db = Database("./data/users")
db.load()

# Store complex nested structures
db.set("user:1", {
    "name": "Alice",
    "email": "[email protected]",
    "profile": {
        "bio": "Software engineer",
        "location": "San Francisco",
        "social": {
            "twitter": "@alice",
            "github": "alice-dev"
        }
    },
    "preferences": {
        "theme": "dark",
        "notifications": True
    }
})

db.commit()
db.close()

​
Transaction Usage

BinaryDB provides basic transaction support with begin(), rollback(), and end().

​
When to Use Transactions

Use transactions when you need to ensure a group of operations either all succeed or all fail: 
from binarydb.database import Database
from binarydb.errors import TransactionError

db = Database("./data/bank")
db.load()

try:
    db.begin()
    
    # Transfer money between accounts
    from_account = db.get("account:1")
    to_account = db.get("account:2")
    
    if from_account["balance"] < 100:
        raise ValueError("Insufficient funds")
    
    from_account["balance"] -= 100
    to_account["balance"] += 100
    
    db.set("account:1", from_account)
    db.set("account:2", to_account)
    
    db.end()  # Commit transaction
    
except (ValueError, TransactionError) as e:
    print(f"Transaction failed: {e}")
    db.rollback()  # Undo all changes

db.close()

​
Keep Transactions Short

Transactions create a memory snapshot of all data. Keep them short to minimize memory usage.
db = Database("./data/inventory")
db.load()

# GOOD: Short transaction
db.begin()
db.set("product:1", {"stock": 50})
db.set("product:2", {"stock": 30})
db.end()

# BAD: Long-running transaction with complex logic
db.begin()
# ... many operations ...
# ... complex calculations ...
# ... API calls ...
db.end()  # Memory snapshot held for too long

db.close()

​
Resource Management

​
Always Close Databases

Always call close() to ensure pending changes are committed: 
db = Database("./data/mydb")
db.load()

try:
    db.set("key", "value")
    db.commit()
finally:
    db.close()  # Always close, even if errors occur

​
Context Manager Pattern

BinaryDB does not currently implement context manager protocol (__enter__/__exit__), but you can wrap it if needed.
Since BinaryDB doesn’t support with statements natively, use try/finally: 
from binarydb.database import Database

def process_data():
    db = Database("./data/mydb")
    try:
        db.load()
        db.set("processed", True)
        db.commit()
    finally:
        db.close()

process_data()
Or create a simple wrapper: 
from contextlib import contextmanager

@contextmanager
def database_context(path):
    db = Database(path)
    try:
        db.load()
        yield db
    finally:
        db.close()

# Now you can use with statements
with database_context("./data/mydb") as db:
    db.set("key", "value")
    db.commit()

​
When to Use BinaryDB

​
Ideal Use Cases

BinaryDB is perfect for:

Configuration Storage

Store application configuration that needs to persist across restarts.

Small Datasets

Datasets under 100 MB where simplicity is more important than raw performance.

Prototyping

Rapid prototyping and development where you need quick persistence without setup.

Embedded Applications

Single-user applications that need simple, embedded storage without external dependencies.

​
When to Use Alternatives

Consider other solutions when: 
# If you need:
# - Large datasets (> 100 MB) → Use SQLite, PostgreSQL
# - Multi-process access → Use Redis, PostgreSQL
# - Complex queries → Use SQL databases
# - Production-grade performance → Use specialized databases
# - Untrusted data sources → Use JSON, SQLite (NOT pickle)

​
Quick Reference

  • Batch operations before committing
  • Wrap load() and commit() in try/except
  • Check exists() before update() operations
  • Keep transactions short and focused
  • Always close() databases in finally blocks
  • Use namespaced key naming conventions
  • Monitor memory usage for large datasets
  • Never load databases from untrusted sources

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us