Skip to main content

​
Overview

The Data Persistence Plugin provides utilities to serialize and restore Orama instances in multiple formats. This allows you to save your search index to disk or transfer it over the network, then restore it later without re-indexing.

​
Installation

npm install @orama/plugin-data-persistence

​
Supported Formats

The plugin supports four serialization formats:

JSON

Human-readable, largest size, slowest

Binary (MessagePack)

Compact binary format, good balance

dpack

Highly optimized binary format

seqproto

Sequential protocol, best compression

​
Basic Usage

​
Persist (Save) an Instance

import { create, insert } from '@orama/orama'
import { persist } from '@orama/plugin-data-persistence'

// Create and populate a database
const db = await create({
  schema: {
    title: 'string',
    description: 'string'
  }
})

await insert(db, { title: 'Product 1', description: 'Description 1' })
await insert(db, { title: 'Product 2', description: 'Description 2' })

// Serialize to binary format
const serialized = await persist(db, 'binary')

// Save to file (Node.js)
import { writeFile } from 'fs/promises'
await writeFile('database.bin', serialized)

​
Restore (Load) an Instance

import { restore } from '@orama/plugin-data-persistence'
import { readFile } from 'fs/promises'

// Load from file
const data = await readFile('database.bin')

// Restore the database
const db = await restore('binary', data)

// Database is ready to use
const results = await search(db, { term: 'Product' })

​
API Reference

​
persist()

Serialize an Orama instance to a specific format. 
async function persist<T extends AnyOrama>(
  db: T,
  format?: PersistenceFormat,
  runtime?: Runtime
): Promise<string | Buffer | ArrayBuffer>
​
db
AnyOrama
required
The Orama instance to serialize
​
format
PersistenceFormat
default:"binary"
Serialization format: 'json', 'binary', 'dpack', or 'seqproto'
​
runtime
Runtime
Runtime environment: 'node' or 'browser'. Auto-detected if not provided
Returns: Serialized data as string, Buffer, or ArrayBuffer depending on format and runtime

​
restore()

Deserialize data and create an Orama instance. 
async function restore<T extends AnyOrama>(
  format: PersistenceFormat,
  data: string | Buffer | ArrayBuffer,
  runtime?: Runtime
): Promise<T>
​
format
PersistenceFormat
required
Serialization format used: 'json', 'binary', 'dpack', or 'seqproto'
​
data
string | Buffer | ArrayBuffer
required
The serialized data to restore
​
runtime
Runtime
Runtime environment: 'node' or 'browser'. Auto-detected if not provided
Returns: A fully functional Orama instance

​
Format Comparison

​
JSON Format

import { persist, restore } from '@orama/plugin-data-persistence'

// Serialize to JSON
const jsonString = await persist(db, 'json')

// Human-readable output
console.log(jsonString.substring(0, 100))
// {"schema":{"title":"string","description":"string"},...

// Restore from JSON
const db = await restore('json', jsonString)
Pros:
  • Human-readable
  • Easy to debug
  • Compatible with any system
Cons:
  • Largest file size
  • Slowest to serialize/deserialize

​
Binary Format (MessagePack)

// Serialize to binary (hex string in Node.js, Uint8Array in browser)
const binaryData = await persist(db, 'binary')

// Node.js - Save as hex string
await writeFile('db.bin', binaryData)

// Restore from binary
const db = await restore('binary', binaryData)
Pros:
  • Compact size
  • Good performance
  • Standard format (MessagePack)
Cons:
  • Not human-readable
  • Requires format knowledge

​
dpack Format

// Highly optimized binary format
const dpackData = await persist(db, 'dpack')

// Restore from dpack
const db = await restore('dpack', dpackData)
Pros:
  • Highly optimized
  • Very compact
  • Fast serialization
Cons:
  • Proprietary format
  • Not human-readable

​
seqproto Format

// Sequential protocol format
const seqprotoData = await persist(db, 'seqproto')

// Restore from seqproto
const db = await restore('seqproto', seqprotoData)
Pros:
  • Best compression
  • Optimized for Orama data structures
  • Fastest restore time
Cons:
  • Orama-specific format
  • Not human-readable

​
Usage Examples

​
Node.js: Save and Load from File

import { create, insert } from '@orama/orama'
import { persist, restore } from '@orama/plugin-data-persistence'
import { writeFile, readFile } from 'fs/promises'

// Create and populate database
const db = await create({
  schema: {
    title: 'string',
    description: 'string',
    category: 'string'
  }
})

await insert(db, {
  title: 'JavaScript Guide',
  description: 'Learn JavaScript fundamentals',
  category: 'Programming'
})

// Save to file using binary format
const serialized = await persist(db, 'binary', 'node')
await writeFile('./data/search-index.bin', serialized)

// Later, restore from file
const data = await readFile('./data/search-index.bin')
const restoredDB = await restore('binary', data, 'node')

// Use restored database
const results = await search(restoredDB, { term: 'JavaScript' })

​
Browser: Save to localStorage

import { create, insert } from '@orama/orama'
import { persist, restore } from '@orama/plugin-data-persistence'

// Create and populate database
const db = await create({
  schema: {
    id: 'string',
    name: 'string'
  }
})

await insert(db, { id: '1', name: 'Item 1' })

// Serialize and save to localStorage
const serialized = await persist(db, 'json', 'browser')
localStorage.setItem('orama-db', serialized)

// Restore from localStorage
const stored = localStorage.getItem('orama-db')
if (stored) {
  const restoredDB = await restore('json', stored, 'browser')
  // Use restored database
}

​
Browser: Save to IndexedDB

import { create } from '@orama/orama'
import { persist, restore } from '@orama/plugin-data-persistence'

// Serialize to binary for IndexedDB
const serialized = await persist(db, 'seqproto')

// Save to IndexedDB
const dbRequest = indexedDB.open('OramaStorage', 1)
dbRequest.onsuccess = (event) => {
  const idb = event.target.result
  const transaction = idb.transaction(['indexes'], 'readwrite')
  const store = transaction.objectStore('indexes')
  store.put({ id: 'main-index', data: serialized })
}

// Restore from IndexedDB
dbRequest.onsuccess = async (event) => {
  const idb = event.target.result
  const transaction = idb.transaction(['indexes'], 'readonly')
  const store = transaction.objectStore('indexes')
  const request = store.get('main-index')
  
  request.onsuccess = async () => {
    const db = await restore('seqproto', request.result.data)
    // Use restored database
  }
}

​
Transfer over Network

// Server: Serialize and send
const db = await create({ schema })
// ... populate database ...

const serialized = await persist(db, 'dpack', 'node')

// Send via HTTP
app.get('/api/search-index', (req, res) => {
  res.setHeader('Content-Type', 'application/octet-stream')
  res.send(Buffer.from(serialized))
})

// Client: Receive and restore
const response = await fetch('/api/search-index')
const arrayBuffer = await response.arrayBuffer()
const db = await restore('dpack', arrayBuffer, 'browser')

​
Performance Considerations

​
Format Size Comparison

For a database with 10,000 documents:
FormatSizeSerialization TimeDeserialization Time
JSON~5.2 MB~450ms~380ms
Binary~2.8 MB~280ms~220ms
dpack~2.1 MB~210ms~180ms
seqproto~1.8 MB~190ms~150ms
Results vary based on data structure and content. Test with your actual data for accurate measurements.

​
Choosing a Format

// For debugging and development
const serialized = await persist(db, 'json')

// For production (good balance)
const serialized = await persist(db, 'binary')

// For best performance and size
const serialized = await persist(db, 'seqproto')

// For compatibility with external tools
const serialized = await persist(db, 'dpack')

​
Common Patterns

​
Auto-save on Changes

import { create, insert } from '@orama/orama'
import { persist } from '@orama/plugin-data-persistence'
import { writeFile } from 'fs/promises'

const db = await create({ schema })

// Wrap insert to auto-save
const insertAndSave = async (doc) => {
  await insert(db, doc)
  const serialized = await persist(db, 'binary')
  await writeFile('db.bin', serialized)
}

await insertAndSave({ title: 'Document 1' })

​
Lazy Loading

let db = null

// Load database only when needed
async function getDB() {
  if (!db) {
    const data = await readFile('db.bin')
    db = await restore('binary', data)
  }
  return db
}

// Use lazy loading
const results = await search(await getDB(), { term: 'test' })

​
Versioned Persistence

import { persist } from '@orama/plugin-data-persistence'
import { writeFile } from 'fs/promises'

const version = '1.0.0'
const serialized = await persist(db, 'binary')

// Save with version metadata
await writeFile('db.bin', JSON.stringify({
  version,
  format: 'binary',
  data: serialized,
  timestamp: new Date().toISOString()
}))

// Restore with version check
const stored = JSON.parse(await readFile('db.bin', 'utf-8'))
if (stored.version !== '1.0.0') {
  throw new Error('Incompatible database version')
}
const db = await restore('binary', stored.data)

​
Runtime Detection

The plugin automatically detects the runtime environment: 
// Auto-detection (recommended)
const serialized = await persist(db, 'binary')

// Explicit runtime (for edge cases)
const serialized = await persist(db, 'binary', 'node')
const serialized = await persist(db, 'binary', 'browser')
At /home/daytona/workspace/source/packages/plugin-data-persistence/src/index.ts:54-61: 
export async function persist<T extends AnyOrama>(
  db: T,
  format: PersistenceFormat = 'binary',
  runtime?: Runtime
): Promise<string | Buffer | ArrayBuffer> {
  if (!runtime) {
    runtime = detectRuntime()
  }
  // ...
}

​
Troubleshooting

​
Unsupported Format Error

Error: Unsupported persistence format: xyz
Use one of the supported formats: 'json', 'binary', 'dpack', or 'seqproto'.

​
Restore Fails

Ensure the format matches: 
// ❌ Wrong
const data = await persist(db, 'json')
const restored = await restore('binary', data) // Error!

// ✅ Correct
const data = await persist(db, 'json')
const restored = await restore('json', data)

​
Large Database Performance

For very large databases (>100MB):
  1. Use seqproto format for best compression
  2. Consider streaming if supported by your environment
  3. Implement incremental loading

​
Next Steps

Import Data

Learn more about data import and export

Custom Plugins

Build plugins that work with persistence

Performance

Optimize your Orama instances

Build docs developers (and LLMs) love

Get started for freeTalk to us