Query Exporter automatically provides several built-in metrics to help monitor query execution and database health. These metrics are always available and don’t need to be defined in the configuration.
Available Built-in Metrics
database_errors
Type: Counter
Labels: database
Tracks the number of database errors that occur per database.
database_errors{database="mydb"} 5
This counter increments whenever:
- A database connection fails
- A query execution fails
- A database operation times out
This metric uses increment: true behavior, meaning each error adds 1 to the counter.
queries
Type: Counter
Labels: database, query, status
Tracks the number of executed queries per database, query name, and execution status.
Status values:
success: Query completed successfully
error: Query failed with an error
timeout: Query execution timed out
queries{database="postgres",query="user_count",status="success"} 1523
queries{database="postgres",query="user_count",status="error"} 2
queries{database="postgres",query="user_count",status="timeout"} 0
This metric uses increment: true behavior, meaning each query execution adds 1 to the counter.
query_interval
Type: Gauge
Labels: query
Reports the configured execution interval in seconds for each query that has an interval set.
query_interval{query="user_count"} 60
query_interval{query="daily_stats"} 86400
This metric is only present for queries that have an interval configured. Queries with schedule or on-demand queries will not appear in this metric.
query_latency
Type: Histogram
Labels: database, query
Tracks query execution latency (duration) in seconds as a histogram, allowing you to analyze the distribution of query execution times.
query_latency_bucket{database="postgres",query="user_count",le="0.1"} 145
query_latency_bucket{database="postgres",query="user_count",le="0.5"} 189
query_latency_bucket{database="postgres",query="user_count",le="1.0"} 195
query_latency_bucket{database="postgres",query="user_count",le="+Inf"} 200
query_latency_sum{database="postgres",query="user_count"} 45.3
query_latency_count{database="postgres",query="user_count"} 200
Default buckets: [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0, +Inf]
query_timestamp
Type: Gauge
Labels: database, query
Reports the Unix timestamp of the last successful query execution per database and query.
query_timestamp{database="postgres",query="user_count"} 1709481234.567
This is useful for:
- Detecting when queries last ran
- Creating alerts for stale data
- Monitoring query scheduling
Configuration
Most built-in metrics don’t require configuration, but you can customize certain aspects in the builtin-metrics section.
Customizing query_latency Buckets
You can configure custom histogram buckets for the query_latency metric:
builtin-metrics:
query_latency:
buckets: [0.1, 0.5, 1.0, 5.0, 10.0, 30.0]
builtin-metrics.query_latency.buckets
Custom histogram buckets for query latency measurements.
- Must contain at least 1 value
- Values must be sorted in ascending order
- Values must be unique
- Values represent seconds
Complete Configuration Example
databases:
postgres:
dsn: postgresql://user:pass@localhost/db
mysql:
dsn: mysql://user:pass@localhost/db
metrics:
user_count:
type: gauge
description: Number of users
queries:
count_users:
databases: [postgres, mysql]
metrics: [user_count]
interval: 60
timeout: 5.0
sql: SELECT COUNT(*) AS user_count FROM users
# Customize built-in metrics
builtin-metrics:
query_latency:
buckets: [0.1, 0.5, 1.0, 5.0, 10.0]
This configuration will produce the following built-in metrics:
# Query execution counts
queries{database="postgres",query="count_users",status="success"} 150
queries{database="mysql",query="count_users",status="success"} 148
queries{database="mysql",query="count_users",status="timeout"} 2
# Query execution interval
query_interval{query="count_users"} 60
# Query latency with custom buckets
query_latency_bucket{database="postgres",query="count_users",le="0.1"} 145
query_latency_bucket{database="postgres",query="count_users",le="0.5"} 150
query_latency_bucket{database="postgres",query="count_users",le="1.0"} 150
query_latency_bucket{database="postgres",query="count_users",le="5.0"} 150
query_latency_bucket{database="postgres",query="count_users",le="10.0"} 150
query_latency_bucket{database="postgres",query="count_users",le="+Inf"} 150
query_latency_sum{database="postgres",query="count_users"} 12.4
query_latency_count{database="postgres",query="count_users"} 150
# Last query execution timestamp
query_timestamp{database="postgres",query="count_users"} 1709481234.567
query_timestamp{database="mysql",query="count_users"} 1709481233.891
# Database errors
database_errors{database="postgres"} 0
database_errors{database="mysql"} 2
Using Built-in Metrics
Alerting on Query Failures
# Prometheus alerting rule
- alert: HighQueryFailureRate
expr: |
rate(queries{status="error"}[5m]) > 0.1
annotations:
summary: "High query failure rate for {{ $labels.query }} on {{ $labels.database }}"
Detecting Stale Queries
# Alert when query hasn't run in expected interval
- alert: QueryNotRunning
expr: |
time() - query_timestamp > 2 * query_interval
annotations:
summary: "Query {{ $labels.query }} hasn't run in 2x its interval"
# Alert on slow queries
- alert: SlowQueries
expr: |
histogram_quantile(0.95, rate(query_latency_bucket[5m])) > 5
annotations:
summary: "95th percentile query latency over 5s for {{ $labels.query }}"
Dashboard Queries
Query success rate:
sum(rate(queries{status="success"}[5m])) by (query)
/
sum(rate(queries[5m])) by (query)
Average query latency:
rate(query_latency_sum[5m])
/
rate(query_latency_count[5m])
Database error rate:
rate(database_errors[5m])
Reserved Metric Names
The following metric names are reserved for built-in metrics and cannot be used in your custom metric definitions:
database_errors
queries
query_interval
query_latency
query_timestamp
Attempting to define a metric with a reserved name will result in a configuration error.
Additional Labels
Built-in metrics automatically include the same extra labels as your custom metrics:
- Database label: All built-in metrics (except
query_interval) include the database label
- Custom database labels: If you configure labels on databases, those labels are also added to built-in metrics
databases:
prod_db:
dsn: postgresql://localhost/db
labels:
environment: production
region: us-east-1
With this configuration, built-in metrics will include:
queries{database="prod_db",environment="production",region="us-east-1",query="...",status="..."}