LiveSync Setup and Configuration

Prerequisites

Before setting up LiveSync, ensure you have:

An active Ably account with an API key
A supported database (Postgres 10+ or MongoDB 4.0+)
Database credentials with appropriate permissions
Basic understanding of your database architecture

Setup Overview

Setting up LiveSync involves three main steps:

Configure your database - Set up required tables/collections
Create an integration rule - Connect your database to Ably
Integrate client SDKs - Subscribe to changes in your applications

Postgres Setup

Step 1: Create Required Tables

The Postgres database connector requires two tables in your database: an outbox table and a nodes table.

-- Create nodes table for cluster coordination
CREATE TABLE IF NOT EXISTS nodes (
  id TEXT PRIMARY KEY,
  expiry TIMESTAMP WITHOUT TIME ZONE NOT NULL
);

-- Create outbox table for change events
CREATE TABLE IF NOT EXISTS outbox (
  sequence_id  SERIAL PRIMARY KEY,
  mutation_id  TEXT NOT NULL,
  channel      TEXT NOT NULL,
  name         TEXT NOT NULL,
  rejected     BOOLEAN NOT NULL DEFAULT false,
  data         JSONB,
  headers      JSONB,
  locked_by    TEXT,
  lock_expiry  TIMESTAMP WITHOUT TIME ZONE,
  processed    BOOLEAN NOT NULL DEFAULT false
);

Step 2: Create Trigger Function

Create a trigger to notify the connector when new records are added to the outbox:

CREATE OR REPLACE FUNCTION public.outbox_notify()
RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify('ably_adbc'::text, ''::text);
  RETURN NULL;
EXCEPTION
  WHEN others THEN
    RAISE WARNING 'unexpected error in %s: %%', SQLERRM;
    RETURN NULL;
END;
$$ LANGUAGE plpgsql;

-- Attach trigger to outbox table
CREATE TRIGGER public_outbox_trigger
AFTER INSERT ON public.outbox
FOR EACH STATEMENT EXECUTE PROCEDURE public.outbox_notify();

Step 3: Configure Database Permissions

Create a dedicated user with minimal required permissions:

CREATE ROLE livesync_user LOGIN PASSWORD 'secure_password' VALID UNTIL 'infinity';

-- Grant connection permission
GRANT CONNECT ON DATABASE your_database TO livesync_user;

-- Grant permissions on outbox table
GRANT SELECT, UPDATE, INSERT, DELETE ON public.outbox TO livesync_user;
GRANT USAGE, SELECT ON outbox_sequence_id_seq TO livesync_user;

-- Grant permissions on nodes table
GRANT SELECT, UPDATE, INSERT, DELETE ON public.nodes TO livesync_user;

Step 4: Create Integration Rule

Create a Postgres integration rule in the Ably dashboard:

Navigate to your Ably Dashboard
Select your application
Click the Integrations tab
Click New Integration Rule
Choose Postgres from the list

Configure the rule with the following settings:

Field	Description	Example
URL	Postgres connection string	`postgres://user:pass@host:5432/dbname`
Outbox table schema	Schema name for outbox table	`public`
Outbox table name	Name of the outbox table	`outbox`
Nodes table schema	Schema name for nodes table	`public`
Nodes table name	Name of the nodes table	`nodes`
SSL mode	SSL connection level	`prefer` / `require` / `verify-ca` / `verify-full`
SSL root certificate	CA certificate (if required)	Certificate content
Primary site	Data center location	`us-east-1-A`

Step 5: Test the Connection

Verify your setup by inserting a test record:

INSERT INTO outbox (
  mutation_id,
  channel,
  name,
  data
) VALUES (
  gen_random_uuid()::text,
  'test-channel',
  'test-event',
  '{"message": "Hello LiveSync!"}'
);

Monitor your Ably dashboard to confirm the message was published.

MongoDB Setup

Step 1: Enable Change Streams

Ensure your MongoDB deployment supports Change Streams:

MongoDB Atlas: Change Streams are enabled by default
Self-hosted: Requires a replica set or sharded cluster (MongoDB 4.0+)

Step 2: Configure Collection

Optionally enable pre and post images for full document access:

db.runCommand({
  collMod: 'your_collection',
  changeStreamPreAndPostImages: { enabled: true }
});

Step 3: Create Integration Rule

Create a MongoDB integration rule in the Ably dashboard:

Navigate to your Ably Dashboard
Select your application
Click the Integrations tab
Click New Integration Rule
Choose MongoDB from the list

Configure the rule with the following settings:

Field	Description	Example
URL	MongoDB connection string	`mongodb://user:pass@host/db`
Watch	What to watch	`collection`
Database name	Target database	`myapp`
Collection name	Target collection	`users`
Full Document	Include full document	`updateLookup` / `whenAvailable` / `off`
Full Document before change	Include previous state	`whenAvailable` / `off`
Pipeline	Change stream pipeline	See below
Primary site	Data center location	`us-east-1-A`

Step 4: Configure Pipeline

Define a MongoDB aggregation pipeline to route events to channels:

[
  {
    "$match": {
      "operationType": { "$in": ["insert", "update", "delete"] }
    }
  },
  {
    "$set": {
      "_ablyChannel": "users:updates"
    }
  }
]

Dynamic channel routing based on document fields:

[
  {
    "$set": {
      "_ablyChannel": {
        "$concat": [
          "users:",
          "$fullDocument.userId"
        ]
      }
    }
  }
]

Step 5: Configure Database Permissions

Create a user with Change Stream access:

db.createUser({
  user: 'livesync_user',
  pwd: 'secure_password',
  roles: [
    {
      role: 'read',
      db: 'your_database'
    }
  ]
});

// Grant Change Stream permissions
db.grantRolesToUser('livesync_user', [
  { role: 'changeStream', db: 'your_database' }
]);

Client SDK Integration

Install Ably SDK

Install the Ably SDK for your platform:

# JavaScript/Node.js
npm install ably

# Python
pip install ably

# Java
# Add to pom.xml or build.gradle

# Swift
# Add to Package.swift or use CocoaPods

Implement realtime subscriptions in your client application:

import { Realtime } from 'ably';

// Initialize with token auth (recommended for production)
const ably = new Realtime({
  authUrl: '/api/ably/auth' // Your token endpoint
});

// Get the channel
const channel = ably.channels.get('users:updates');

// Subscribe to all messages
await channel.subscribe((message) => {
  console.log('Change event:', message.data);
});

// Subscribe to specific event names
await channel.subscribe('user.created', (message) => {
  console.log('New user:', message.data);
});

await channel.subscribe('user.updated', (message) => {
  console.log('User updated:', message.data);
});

Authentication & Security

API Keys and Capabilities

Configure API keys with appropriate capabilities:

{
  "users:*": ["subscribe"],
  "public:*": ["subscribe", "history"]
}

Token Authentication

Implement token authentication for client applications:

// Backend token endpoint
app.get('/api/ably/auth', async (req, res) => {
  const tokenRequest = await ably.auth.createTokenRequest({
    clientId: req.user.id,
    capability: {
      [`users:${req.user.id}`]: ['subscribe']
    }
  });
  
  res.json(tokenRequest);
});

// Frontend client
const ably = new Realtime({
  authUrl: '/api/ably/auth',
  authMethod: 'GET'
});

Configuration Best Practices

Channel Naming

Use consistent channel naming conventions:

// Resource-based naming
'users:123'
'posts:456'
'comments:789'

// Hierarchical naming
'workspace:abc:tasks'
'workspace:abc:users'

// Event-type based
'orders:created'
'orders:updated'
'orders:cancelled'

Message Structure

Design consistent message payloads:

{
  "eventType": "user.updated",
  "timestamp": 1234567890,
  "data": {
    "id": "user123",
    "changes": {
      "email": "[email protected]"
    }
  },
  "metadata": {
    "source": "api",
    "userId": "admin123"
  }
}

Error Handling

Implement robust error handling:

const channel = ably.channels.get('users:updates');

// Handle connection errors
ably.connection.on('failed', (stateChange) => {
  console.error('Connection failed:', stateChange.reason);
});

// Handle channel errors
channel.on('failed', (stateChange) => {
  console.error('Channel error:', stateChange.reason);
});

// Handle subscription errors
try {
  await channel.subscribe((message) => {
    console.log('Message:', message.data);
  });
} catch (error) {
  console.error('Subscription error:', error);
}

Monitoring and Debugging

Dashboard Metrics

Monitor your LiveSync integration in the Ably dashboard:

Message throughput
Connection count
Error rates
Integration rule status

Logging

Enable debug logging for troubleshooting:

const ably = new Realtime({
  key: 'YOUR_API_KEY',
  logLevel: 4 // 0=none, 1=errors, 2=warnings, 3=info, 4=debug
});

Testing

Test your integration with curl:

# Test outbox insertion (Postgres)
curl -X POST http://your-api/test-livesync \
  -H "Content-Type: application/json" \
  -d '{
    "channel": "test-channel",
    "name": "test-event",
    "data": {"message": "test"}
  }'

Next Steps

Database Synchronization

Learn database sync patterns and best practices

Conflict Resolution

Handle conflicts in distributed systems

Postgres Connector

Deep dive into Postgres-specific features

MongoDB Connector

Explore MongoDB Change Streams integration

Overview

Features

LiveSync Setup and Configuration

Prerequisites

Setup Overview

Postgres Setup

Step 1: Create Required Tables

Step 2: Create Trigger Function

Step 3: Configure Database Permissions

Step 4: Create Integration Rule

Step 5: Test the Connection

MongoDB Setup

Step 1: Enable Change Streams

Step 2: Configure Collection

Step 3: Create Integration Rule

Step 4: Configure Pipeline

Step 5: Configure Database Permissions

Client SDK Integration

Install Ably SDK

Authentication & Security

API Keys and Capabilities

Token Authentication

Configuration Best Practices

Channel Naming

Message Structure

Error Handling

Monitoring and Debugging

Dashboard Metrics

Logging

Testing

Next Steps

Database Synchronization

Conflict Resolution

Postgres Connector

MongoDB Connector

Build docs developers (and LLMs) love

Overview

Features

​Prerequisites

​Setup Overview

​Postgres Setup

​Step 1: Create Required Tables

​Step 2: Create Trigger Function

​Step 3: Configure Database Permissions

​Step 4: Create Integration Rule

​Step 5: Test the Connection

​MongoDB Setup

​Step 1: Enable Change Streams

​Step 2: Configure Collection

​Step 3: Create Integration Rule

​Step 4: Configure Pipeline

​Step 5: Configure Database Permissions

​Client SDK Integration

​Install Ably SDK

​Subscribe to Changes

​Authentication & Security

​API Keys and Capabilities

​Token Authentication

​Configuration Best Practices

​Channel Naming

​Message Structure

​Error Handling

​Monitoring and Debugging

​Dashboard Metrics

​Logging

​Testing

​Next Steps

Database Synchronization

Conflict Resolution

Postgres Connector

MongoDB Connector

Build docs developers (and LLMs) love

Prerequisites

Setup Overview

Postgres Setup

Step 1: Create Required Tables

Step 2: Create Trigger Function

Step 3: Configure Database Permissions

Step 4: Create Integration Rule

Step 5: Test the Connection

MongoDB Setup

Step 1: Enable Change Streams

Step 2: Configure Collection

Step 3: Create Integration Rule

Step 4: Configure Pipeline

Step 5: Configure Database Permissions

Client SDK Integration

Install Ably SDK

Subscribe to Changes

Authentication & Security

API Keys and Capabilities

Token Authentication

Configuration Best Practices

Channel Naming

Message Structure

Error Handling

Monitoring and Debugging

Dashboard Metrics

Logging

Testing

Next Steps