Synchronization - nteract Desktop

nteract Desktop uses Conflict-free Replicated Data Types (CRDTs) via Automerge to enable real-time synchronization of notebook state across multiple windows and the daemon.

Why CRDTs?

Traditional approaches to multi-user editing require:

Central server making decisions about conflicts
Locking preventing concurrent edits
Last-write-wins losing data on conflicts

CRDTs enable:

Automatic merge of concurrent edits
No central authority needed for conflict resolution
Eventual consistency all replicas converge to the same state
Offline-first edits work without connectivity

Automerge is a CRDT library that provides JSON-like documents with automatic merge semantics.

Sync Architecture

Two Sync Channels

nteract Desktop maintains two separate Automerge documents:

1. Settings Document

A single shared document for user preferences:

ROOT/
  theme: "system"
  default_runtime: "python"
  default_python_env: "uv"
  uv/
    default_packages: ["numpy", "pandas"]
  conda/
    default_packages: ["scipy"]

Synced across: All notebook windows Persistence: ~/.cache/runt/settings.automerge with JSON mirror at ~/.config/nteract/settings.json Migration: Backward-compatible migration from flat keys to nested structures

2. Notebook Documents

Each open notebook gets its own Automerge document in a “room”:

ROOT/
  notebook_id: Str
  cells/
    [i]/
      id: Str
      cell_type: Str
      source: Text              # Automerge Text CRDT
      execution_count: Str
      outputs/
        [j]: Str                # Output manifest hash
  metadata/
    runtime: Str

Synced across: All windows viewing the same notebook Persistence: ~/.cache/runt/notebook-docs/{sha256(notebook_id)}.automerge

Room-Based Architecture

The daemon manages notebook sync through rooms:

pub struct NotebookRoom {
    pub doc: Arc<RwLock<NotebookDoc>>,
    pub changed_tx: broadcast::Sender<()>,
    pub persist_path: PathBuf,
    pub active_peers: AtomicUsize,
}

Room lifecycle:

First window opens → Daemon creates/loads room
Client handshake → Handshake::NotebookSync { notebook_id }
Initial sync → Exchange Automerge sync messages until convergence
Watch loop → Listen for local edits and peer changes
Peer changes → Apply, persist, broadcast to other peers
Last window closes → Room evicted, document persisted

Rooms are identified by notebook_id. Multiple windows opening the same notebook join the same room automatically.

Sync Protocol

Wire Format

Length-prefixed binary frames over Unix socket:

[4 bytes: length (big-endian u32)] [Automerge sync message]

Automerge sync messages are binary-encoded change sets, not JSON.

Sync Flow

Initial Sync

Server sends first: Daemon initiates with its current state
Client responds: Window sends its local changes (if any)
Exchange until convergence: Both sides send sync messages with 100ms timeout
Sync complete: Both replicas have identical state

Change Propagation

After initial sync, changes propagate immediately:

Local edit: Window modifies Automerge doc
Generate sync message: Automerge creates minimal change representation
Send to daemon: Sync message over socket
Daemon applies: Under write lock, update canonical doc
Persist: Serialize and write to disk (outside lock)
Broadcast: Notify all other peers
Peers apply: Other windows receive and apply change

Latency target: Sub-200ms from edit in Window A to display in Window B for local connections.

Conflict Resolution

Concurrent Edits

Two windows editing the same cell simultaneously: Window A: Types hello in cell source Window B: Types world in same cell source Automerge’s Text CRDT merges these character-level:

Both edits applied to document
Automerge determines character insertion order
Result might be helloworld or worldhello (deterministic based on Lamport clocks)
Both windows converge to same final text

Key property: No data is lost, edits are merged automatically.

Deterministic Merge

Automerge uses logical clocks to establish a total order on concurrent operations:

Operation from Actor A at time T1
Operation from Actor B at time T2

If T1 < T2: A's change ordered before B's
If T1 == T2: Tie-break by actor ID (lexicographic)

All replicas apply the same ordering, guaranteeing convergence.

Write-Once Data

Outputs are write-once from a single actor (the kernel), so they don’t need CRDT semantics:

cell/
  outputs/
    [0]: "manifest-hash-1"    # Written once by kernel
    [1]: "manifest-hash-2"    # Appended, never modified

No concurrent editing of outputs, so simple list append works.

Text Editing Semantics

Cell source uses Automerge’s Text type for proper concurrent editing:

Character-Level Merging

Window A: "hello" → "hello world" Window B: "hello" → "hello there" Automerge represents this as:

['h', 'e', 'l', 'l', 'o']
  → Window A inserts [' ', 'w', 'o', 'r', 'l', 'd'] at position 5
  → Window B inserts [' ', 't', 'h', 'e', 'r', 'e'] at position 5

Merged result: Both insertions applied, order determined by Lamport clocks.

Update Operation

When you type in a cell, the frontend sends the full new text:

sync_client.update_source(cell_id, new_source)

Internally, Automerge:

Runs Myers diff algorithm
Generates minimal character-level patch
Applies patch operations (insert/delete)
Broadcasts patch, not full text

This keeps sync messages small even for large cells.

Persistence

Automerge Binary Format

Documents are saved as compact binary (not JSON):

~/.cache/runt/settings.automerge        # Settings doc
~/.cache/runt/notebook-docs/{hash}.automerge  # Notebook docs

Format: Automerge’s native binary serialization including full CRDT history. Size: Grows with edit history. Occasional compaction removes old history.

Persistence Strategy

After every sync message:

Serialize: Inside write lock, call doc.save()
Write to disk: Outside write lock, async I/O
Atomic write: Temp file + rename for crash safety

I/O happens outside the lock so it doesn’t block other peers.

Corrupt Document Recovery

If a persisted .automerge file can’t be loaded:

Rename to .automerge.corrupt
Create fresh document
Log warning
Continue operation

This preserves corrupt data for debugging without blocking the user.

Settings Sync

JSON Mirror

Settings maintain a JSON mirror for external tool compatibility: Automerge doc: ~/.cache/runt/settings.automerge (canonical) JSON mirror: ~/.config/nteract/settings.json (read-only view) The daemon watches the JSON file with a debounced file watcher (500ms):

External tool edits JSON
Daemon detects change
Parse JSON, apply to Automerge doc
Persist Automerge binary (not back to JSON)
Broadcast to all peers

The JSON file is overwritten on first run. Manual edits are preserved via Automerge after being imported.

Migration

Backward-compatible migration from flat keys: Old format:

{
  "default_uv_packages": "numpy, pandas"
}

New format:

{
  "uv": {
    "default_packages": ["numpy", "pandas"]
  }
}

Migration runs on load, converting flat keys to nested structure.

Multi-Window Benefits

Real-Time Collaboration

Multiple windows editing the same notebook see changes in real-time.

Late-Joiner Sync

Open a second window and it catches up instantly with current state.

Output Sharing

Execute in one window, outputs appear in all windows viewing that notebook.

No Save Conflicts

Automatic merge means no “file changed on disk” dialogs.

Performance

Latency Measurements

Operation	Typical Latency
Local edit → sync message	<5ms
Sync message round-trip	1-5ms (local daemon)
Daemon apply + persist	5-20ms
Total edit propagation	<50ms

Optimization: Dual Channel

For execution outputs, nteract uses a dual-channel design: Channel 1 (Automerge): Durable, synced state Channel 2 (Broadcasts): Ephemeral, real-time events When a cell executes:

Kernel outputs → daemon writes to Automerge (persisted)
Daemon also sends broadcast event to all peers (fast)
Executing window shows output from broadcast (<50ms)
Other windows apply Automerge change (synced state)

This gives low-latency display while maintaining consistency.

Batching

Rapid consecutive changes can be batched:

Edit 1 → Edit 2 → Edit 3 → ... → Edit N
  ↓
Batched sync message (after debounce)

Debounce period: 50-100ms. Balances responsiveness with message overhead.

Known Limitations

Widgets only render in the window that created them. Secondary windows show “Loading widget” because they miss the initial comm_open message. Root cause: The Jupyter comm protocol establishes widget models via messages. Late joiners don’t receive historical messages. Workaround: Use single window for widget-heavy notebooks. Future fix: Sync widget/comm state via Automerge for late-joiner reconstruction.

Output Sync Race

During cell execution, there’s a brief window where daemon sync updates may conflict with local output updates:

Frontend clears outputs, marks cell executing
Kernel outputs arrive, frontend updates local state
Frontend calls sync_append_output (async to daemon)
Daemon may send notebook:updated before append arrives

The frontend tracks “executing cells” and preserves local outputs for those cells during sync. Proper fix: Store parent_header.msg_id in cell metadata to correlate execution requests with outputs.

Troubleshooting

Changes not syncing

Check:

Daemon running: runt daemon status
Same notebook_id: Check daemon logs for room join
Socket permissions: ls -l ~/.cache/runt/runtimed.sock

Debug:

runt daemon logs -f | grep notebook-sync

Look for sync messages and errors.

Sync latency high

Causes:

Large Automerge document (many historical changes)
Disk I/O bottleneck (spinning disk)
Many peers (broadcast overhead)

Solutions:

Compact Automerge doc (future feature)
Use SSD for cache directory
Limit number of concurrent windows

Document corrupted

Symptoms: Error loading notebook, sync failsCheck:

ls -lh ~/.cache/runt/notebook-docs/

Look for .automerge.corrupt files.Recovery:

Delete corrupted file
Reopen notebook (loads from .ipynb)
Fresh Automerge doc created

Advanced: Inspecting Sync State

List Active Rooms

runt daemon status

Shows active notebook rooms with peer counts.

Inspect Notebook State

runt notebooks

Shows all open notebooks with:

Notebook ID
Number of active peers
Kernel status
Environment source

Daemon Logs

runt daemon logs -f

Sync-related logs:

[notebook-sync] Room lifecycle
[automerge] Sync protocol messages
[persist] Document save operations

Next Steps

Daemon

Learn about room management

Notebooks

Understand notebook format

Architecture

View system overview

Kernels

Explore kernel execution

Get Started

Core Concepts

Guides

Development

​Why CRDTs?

​Sync Architecture

​Two Sync Channels

​1. Settings Document

​2. Notebook Documents

​Room-Based Architecture

​Sync Protocol

​Wire Format

​Sync Flow

​Initial Sync

​Change Propagation

​Conflict Resolution

​Concurrent Edits

​Deterministic Merge

​Write-Once Data

​Text Editing Semantics

​Character-Level Merging

​Update Operation

​Persistence

​Automerge Binary Format

​Persistence Strategy

​Corrupt Document Recovery

​Settings Sync

​JSON Mirror

​Migration

​Multi-Window Benefits

Real-Time Collaboration

Late-Joiner Sync

Output Sharing

No Save Conflicts

​Performance

​Latency Measurements

​Optimization: Dual Channel

​Batching

​Known Limitations

​Widget Multi-Window Sync

​Output Sync Race

​Troubleshooting

​Advanced: Inspecting Sync State

​List Active Rooms

​Inspect Notebook State

​Daemon Logs

​Next Steps

Daemon

Notebooks

Architecture

Kernels

Build docs developers (and LLMs) love

Why CRDTs?

Sync Architecture

Two Sync Channels

1. Settings Document

2. Notebook Documents

Room-Based Architecture

Sync Protocol

Wire Format

Sync Flow

Initial Sync

Change Propagation

Conflict Resolution

Concurrent Edits

Deterministic Merge

Write-Once Data

Text Editing Semantics

Character-Level Merging

Update Operation

Persistence

Automerge Binary Format

Persistence Strategy

Corrupt Document Recovery

Settings Sync

JSON Mirror

Migration

Multi-Window Benefits

Performance

Latency Measurements

Optimization: Dual Channel

Batching

Known Limitations

Widget Multi-Window Sync

Output Sync Race

Troubleshooting

Advanced: Inspecting Sync State

List Active Rooms

Inspect Notebook State

Daemon Logs

Next Steps