Snuba is a service that provides a rich data model on top of ClickHouse together with a fast ingestion consumer and a query optimizer.
What is Snuba?
Snuba is a time-series oriented data platform backed by ClickHouse, designed to power analytics and metrics for Sentry. It provides a sophisticated query layer with custom query languages (SnQL and MQL), optimized data ingestion from Kafka, and advanced features like subscriptions and migrations. Originally developed to replace a combination of Postgres and Redis for searching and aggregating Sentry error data, Snuba has evolved to support multiple datasets and power most time-series related features across the Sentry platform.Key Features
ClickHouse Integration
Database access layer to the ClickHouse distributed data store with optimized storage engines
Query Languages
SnQL and MQL provide graph logical data model queries with SQL-like functionality
Multi-Dataset Support
Support multiple independent datasets in a single installation (events, transactions, metrics, etc.)
Query Optimizer
Rule-based query optimizer for efficient ClickHouse query execution
Kafka Ingestion
Direct data ingestion from Kafka with a flexible consumer framework
Migration System
DDL migration system for single-node and distributed ClickHouse environments
Subscriptions
Streaming queries for real-time alerts and continuous data monitoring
Admin Interface
Web-based admin interface for managing queries, migrations, and system state
Quick Start
Get Started
Set up Snuba and run your first query in minutes
Installation
Install and configure Snuba for your environment
Architecture Overview
Understand how Snuba works under the hood
CLI Reference
Explore available CLI commands for operations
Main Components
Data Model
Snuba organizes data into Datasets, which represent independent partitions of the data model. Each dataset contains one or more Entities (query-level abstractions) mapped to Storages (physical ClickHouse tables).Query Processing
Queries are expressed in SnQL or MQL and go through a sophisticated processing pipeline that includes parsing, validation, optimization, and translation to ClickHouse SQL.Ingestion Pipeline
Data flows from Kafka topics through specialized consumers that process messages and write batches to ClickHouse tables, supporting both real-time ingestion and eventual consistency.Use Cases
Error and Event Analytics
Error and Event Analytics
The
events dataset powers issue search, filtering, and aggregations in Sentry’s error tracking features.Performance Monitoring
Performance Monitoring
The
transactions dataset supports all performance monitoring features with distributed tracing data.Metrics and Aggregations
Metrics and Aggregations
The
metrics and generic_metrics datasets handle high-volume pre-aggregated metrics data.Usage Statistics
Usage Statistics
The
outcomes dataset powers the Sentry stats page with billing and usage information.Resources
GitHub Repository
View source code and contribute to Snuba
API Reference
Explore HTTP endpoints and Python API