Skip to main content
This quickstart guide will help you set up Metadb, configure your first data source, and run your first queries.

​
Prerequisites

Before you begin, ensure you have the following installed:

PostgreSQL 15+

Metadb stores data in PostgreSQL 15 or later

Go 1.21+

Required to build Metadb from source

GCC 9+

C compiler for building dependencies

Ragel 6

State machine compiler for parsers
You’ll also need a Kafka data source with Debezium configured. See the Kafka Data Sources guide for details.

​
Installation

1

Set up Go workspace

Configure your Go environment:
export GOPATH=$HOME/go
2

Clone and build Metadb

Download the source code and compile:
git clone https://github.com/metadb-project/metadb.git -b v1.4.0
cd metadb
./build
The compiled binary will be in bin/metadb.
3

Prepare PostgreSQL database

Create a database and users for Metadb:
CREATE DATABASE metadb;
CREATE USER mdbadmin WITH PASSWORD 'your-password-here';
CREATE USER postgres; -- superuser if not exists
4

Initialize Metadb

Create the data directory and configuration:
./bin/metadb init -D data
This creates data/metadb.conf which you’ll configure in the next step.
5

Configure database connection

Edit data/metadb.conf with your PostgreSQL connection details:
[main]
host = localhost
port = 5432
database = metadb
superuser = postgres
superuser_password = postgres-password
systemuser = mdbadmin
systemuser_password = mdbadmin-password
sslmode = require
6

Start the Metadb server

Launch the server:
nohup ./bin/metadb start -D data -l metadb.log &
The server listens on port 8550 by default.

​
Create Your First Data Source

Connect to Metadb using psql: 
psql -X -h localhost -d metadb -p 8550
Create a Kafka data source: 
create data source sensor type kafka options (
    brokers 'kafka:29092',
    topics '^metadb_sensor_1\.',
    consumer_group 'metadb_sensor_1_1',
    add_schema_prefix 'sensor_',
    schema_stop_filter 'admin'
);
After creating a data source, Metadb enters synchronizing mode. When the initial snapshot completes, you’ll see “source snapshot complete (deadline exceeded)” in the logs.

​
Complete Initial Synchronization

After the snapshot finishes streaming:
1

Stop the server

./bin/metadb stop -D data
2

Run endsync

./bin/metadb endsync -D data --source sensor
3

Restart the server

nohup ./bin/metadb start -D data -l metadb.log &

​
Query Your Data

Reconnect to Metadb and explore the synchronized tables: 
-- List all tables in the sensor schema
SELECT table_name FROM information_schema.tables 
WHERE table_schema = 'sensor_';

-- Query current data from a table
SELECT * FROM sensor_patrongroup LIMIT 10;

-- Query historical data with all changes
SELECT __start, __end, __current, id, groupname, description 
FROM sensor_patrongroup__ 
WHERE id = 15;

​
Understanding Table Metadata

Every Metadb table includes these metadata columns:
  • __id - Unique row identifier (Metadb-managed)
  • __start - When this version of the row became active
  • __end - When this version was superseded (or 9999-12-31 if current)
  • __current - Boolean indicating if this is the current version
  • __origin - Optional identifier for grouping related data
Use current tables (without __ suffix) for simple queries. Use main tables (with __ suffix) for historical analysis.

​
Configure JSON Transformation

If your data includes JSON columns, you can automatically flatten them: 
create data mapping for json
    from table sensor_inventory__ column jsondata path '$'
    to 't';
This creates transformed tables (sensor_inventory__t and sensor_inventory_t) with JSON fields as columns.

​
Common Commands

Here are essential commands for working with Metadb: 
# Check Metadb version
./bin/metadb version

# View help for any command
./bin/metadb help start

# Upgrade to a new version
./bin/metadb upgrade -D data

​
Next Steps

Core Concepts

Learn about data sources, table types, and transformations

Administration Guide

Configure Metadb for production use

User Guide

Master querying and reporting techniques

CLI Reference

Explore all available CLI commands

​
Troubleshooting

Check the log file (metadb.log) for errors. Common issues:
  • PostgreSQL connection failed (verify credentials in metadb.conf)
  • Port 8550 already in use (use --port flag to change)
  • Database permissions insufficient (ensure superuser and systemuser exist)
Verify:
  • Data source is created: list data_sources;
  • Kafka topics are streaming: Check Kafka connector status
  • Schema filters aren’t blocking tables: Review schema_stop_filter and table_stop_filter
This is normal for large datasets. Monitor progress:
SELECT * FROM metadb.log ORDER BY log_time DESC LIMIT 10;
Look for “source snapshot complete” message in logs.

Build docs developers (and LLMs) love

Get started for freeTalk to us