Skip to main content
The SyncService provides JSONL-based import/export operations for syncing Stoneforge data between SQLite and git-tracked files. It implements the dual storage model where JSONL is the source of truth and SQLite is the cache.

​
Overview

The SyncService is implemented in packages/quarry/src/sync/service.ts and provides:
  • Full and incremental export to JSONL
  • Import with automatic merge conflict resolution
  • Dirty tracking for efficient incremental exports
  • Dependency import/export
  • Cycle-safe element ordering
See also: Storage Model for sync concepts and dual storage architecture.

​
Installation

import { SyncService } from '@stoneforge/quarry';
import { createStorageBackend } from '@stoneforge/storage';

const backend = await createStorageBackend({ dbPath: '.stoneforge/stoneforge.db' });
const syncService = new SyncService(backend);

​
Export Operations

​
export()

Exports elements to JSONL files (async).
​
options
ExportOptions
required
​
result
ExportResult
// Full export
const result = await syncService.export({
  outputDir: '.stoneforge',
  full: true,
  includeEphemeral: false,
});

console.log(`Exported ${result.elementsExported} elements`);
console.log(`Exported ${result.dependenciesExported} dependencies`);

// Incremental export (only dirty elements)
const incrementalResult = await syncService.export({
  outputDir: '.stoneforge',
  full: false,
});

console.log(`Incremental: ${incrementalResult.elementsExported} elements`);

​
exportSync()

Synchronous version of export (useful for CLI).
​
options
ExportOptions
required
Same as export()
​
result
ExportResult
Export result
// Synchronous export
const result = syncService.exportSync({
  outputDir: '.stoneforge',
  full: true,
});

​
exportToString()

Exports to in-memory strings (for API use).
​
options
object
​
result
object
const { elements, dependencies } = syncService.exportToString({
  includeEphemeral: false,
  includeDependencies: true,
});

// Send over HTTP or save to custom location
response.setHeader('Content-Type', 'application/x-ndjson');
response.send(elements);

​
Import Operations

​
import()

Imports elements from JSONL files with merge conflict resolution (async).
​
options
ImportOptions
required
​
result
ImportResult
// Preview import
const dryRun = await syncService.import({
  inputDir: '.stoneforge',
  dryRun: true,
});

console.log(`Would import ${dryRun.elementsImported} elements`);
if (dryRun.conflicts.length > 0) {
  console.log('Conflicts detected:');
  dryRun.conflicts.forEach(c => {
    console.log(`  ${c.elementId}: ${c.field} (${c.resolution})`);
  });
}

// Actual import with automatic merge
const result = await syncService.import({
  inputDir: '.stoneforge',
  dryRun: false,
});

console.log(`Imported ${result.elementsImported} elements`);
console.log(`Skipped ${result.elementsSkipped} (no changes)`);

if (result.errors.length > 0) {
  console.error(`${result.errors.length} errors:`);
  result.errors.forEach(e => {
    console.error(`  Line ${e.line}: ${e.message}`);
  });
}

// Force import (overwrite local changes)
const forced = await syncService.import({
  inputDir: '.stoneforge',
  force: true,
});

​
importSync()

Synchronous version of import (useful for CLI).
​
options
ImportOptions
required
Same as import()
​
result
ImportResult
Import result
// Synchronous import
const result = syncService.importSync({
  inputDir: '.stoneforge',
});

​
importFromStrings()

Imports from in-memory JSONL strings (for API use).
​
elementsContent
string
required
Elements JSONL content
​
dependenciesContent
string
required
Dependencies JSONL content
​
options
Partial<ImportOptions>
Import options (dryRun, force)
​
result
ImportResult
Import result
// Import from HTTP request body
const elementsContent = await request.text();
const dependenciesContent = '';

const result = syncService.importFromStrings(
  elementsContent,
  dependenciesContent,
  { dryRun: false }
);

console.log(`Imported ${result.elementsImported} elements from API`);

​
Merge Conflict Resolution

The SyncService automatically resolves merge conflicts using these rules:

​
Element Merge Strategy

  1. Closed/Tombstone Always Wins: If either version is closed or tombstoned, that status takes precedence
  2. Latest Timestamp Wins: For other conflicts, the element with the later updatedAt timestamp is used
  3. Field-Level Merging: Some fields are merged intelligently:
    • Tags: Union of both tag sets
    • Metadata: Merged with remote values taking precedence
// Example conflict scenario:
// Local:  { title: 'Fix bug', status: 'open', updatedAt: '2026-03-01T10:00:00Z' }
// Remote: { title: 'Fix bug', status: 'closed', updatedAt: '2026-03-01T09:00:00Z' }
// Result: status='closed' (closed always wins), other fields from local (newer)

const result = await syncService.import({
  inputDir: '.stoneforge',
});

result.conflicts.forEach(conflict => {
  console.log(`Conflict in ${conflict.elementId}.${conflict.field}`);
  console.log(`  Local: ${JSON.stringify(conflict.localValue)}`);
  console.log(`  Remote: ${JSON.stringify(conflict.remoteValue)}`);
  console.log(`  Resolution: ${conflict.resolution}`);
});

​
Dependency Merge Strategy

  1. Additive: Remote dependencies not in local are added
  2. Removal Detection: Local dependencies not in remote are removed
  3. Referential Integrity: Dependencies with dangling references (missing elements) are skipped
// Dependencies are compared by (blockedId, blockerId, type) composite key
const result = await syncService.import({
  inputDir: '.stoneforge',
});

console.log(`Added ${result.dependenciesImported} dependencies`);
if (result.dependencyConflicts.length > 0) {
  console.log('Dependency conflicts:');
  result.dependencyConflicts.forEach(c => {
    console.log(`  ${c.blockedId} → ${c.blockerId} (${c.type})`);
  });
}

​
JSONL Format

​
Elements File Format

Elements are serialized with entities first (for referential integrity), then sorted by creation time: 
{"id":"el-0000","type":"entity","name":"operator","createdAt":"2026-03-01T00:00:00.000Z","updatedAt":"2026-03-01T00:00:00.000Z","createdBy":"el-0000","tags":[],"metadata":{}}
{"id":"el-1234","type":"task","title":"Fix bug","status":"open","priority":3,"createdAt":"2026-03-01T10:00:00.000Z","updatedAt":"2026-03-01T10:00:00.000Z","createdBy":"el-0000","tags":["bug"],"metadata":{}}

​
Dependencies File Format

Dependencies are sorted by creation time: 
{"blockedId":"el-1234","blockerId":"el-5678","type":"blocks","createdAt":"2026-03-01T10:05:00.000Z","createdBy":"el-0000","metadata":{}}

​
Dirty Tracking

The SQLite backend tracks which elements have been modified since the last export: 
// Make some changes
await api.update(taskId, { status: 'in_progress' });
await api.create(newTask);

// Incremental export only exports changed elements
const result = await syncService.export({
  outputDir: '.stoneforge',
  full: false, // Only dirty elements
});

console.log(`Exported ${result.elementsExported} changed elements`);

// Dirty tracking is cleared after successful export

​
Element Ordering

Elements are exported in a specific order to maintain referential integrity during import:
  1. Entities - Must exist before they’re referenced in other elements
  2. Other Elements - Sorted by createdAt timestamp
This ensures that when importing, entities exist before tasks/documents that reference them as createdBy or assignee. 
// Export order example:
// 1. el-0000 (entity: operator)
// 2. el-0001 (entity: director)
// 3. el-1234 (task created by el-0000)
// 4. el-5678 (document created by el-0001)

​
Error Handling

​
Parse Errors

Invalid JSONL lines are reported but don’t stop the import: 
const result = await syncService.import({
  inputDir: '.stoneforge',
});

if (result.errors.length > 0) {
  result.errors.forEach(error => {
    console.error(`Line ${error.line} in ${error.file}: ${error.message}`);
    console.error(`Content: ${error.content}`);
  });
}

​
Referential Integrity

Dependencies with missing elements are automatically skipped: 
// If elements.jsonl is missing el-5678, but dependencies.jsonl has:
// {"blockedId":"el-1234","blockerId":"el-5678","type":"blocks",...}
// This dependency will be skipped with a warning

const result = await syncService.import({
  inputDir: '.stoneforge',
});

console.log(`Skipped ${result.dependenciesSkipped} dependencies (missing elements)`);

​
CLI Usage

The SyncService powers the sf export and sf import CLI commands: 
# Full export
sf export --full

# Incremental export
sf export

# Import with dry run
sf import --dry-run

# Import with conflict preview
sf import

# Force import (overwrite local)
sf import --force

Build docs developers (and LLMs) love

Get started for freeTalk to us