Query Exporter supports custom YAML tags to make configuration more flexible, modular, and secure. These tags help you:
- Split configuration across multiple files
- Load external content like SQL queries
- Reference environment variables for secrets
Query Exporter provides three YAML tags:
!include - Include another YAML file
!file - Load text content from a file
!env - Read environment variables
!include Tag
Purpose
Include the content of another YAML file to modularize your configuration.
Syntax
Path Resolution
- Absolute paths: Used as-is
- Relative paths: Resolved relative to the including file
Examples
Split databases and queries:
# config.yaml
databases: !include databases.yaml
metrics: !include metrics.yaml
queries: !include queries.yaml
# databases.yaml
prod_db:
dsn: postgresql://user:pass@prod-host/db
labels:
environment: production
staging_db:
dsn: postgresql://user:pass@staging-host/db
labels:
environment: staging
Modular query configuration:
# queries.yaml
query1: !include queries/user-stats.yaml
query2: !include queries/order-metrics.yaml
Include database-specific configs:
# config.yaml
databases:
postgres: !include databases/postgres.yaml
mysql: !include databases/mysql.yaml
oracle: !include databases/oracle.yaml
metrics: !include metrics.yaml
queries: !include queries.yaml
Use Cases
- Team collaboration: Different team members can work on separate configuration files
- Environment separation: Maintain separate configs for dev, staging, and production
- Organization: Group related queries, metrics, or databases
- Reusability: Share common configurations across multiple deployments
!file Tag
Purpose
Include text content from a file as a string. Primarily used for loading SQL queries from external files.
Syntax
Path Resolution
- Absolute paths: Used as-is
- Relative paths: Resolved relative to the including file
Examples
Load SQL from external file:
# config.yaml
queries:
database_stats:
databases: [pg]
metrics: [pg_db_numbackends, pg_db_xact_commit]
sql: !file sql/database-stats.sql
-- sql/database-stats.sql
SELECT
datname,
numbackends AS pg_db_numbackends,
xact_commit AS pg_db_xact_commit
FROM pg_stat_database
WHERE datname NOT IN ('template0', 'template1');
Organize queries by directory:
queries:
process_stats:
databases: [pg]
metrics: [pg_process]
sql: !file queries/postgresql/process.sql
table_stats:
databases: [pg]
metrics: [pg_table_seq_scan, pg_table_idx_scan]
sql: !file queries/postgresql/table.sql
user_activity:
databases: [mysql]
metrics: [active_users]
sql: !file queries/mysql/users.sql
Complex query structure:
queries:
oracle_sessions:
databases: [oracle]
metrics: [oracle_sessions]
sql: !file queries/oracle/session.sql
oracle_tablespace:
databases: [oracle]
metrics: [oracle_tablespace_bytes, oracle_tablespace_free]
sql: !file queries/oracle/tablespace.sql
Benefits
- Version control: Track SQL changes separately from configuration
- Editor support: Use SQL syntax highlighting and linting
- Readability: Keep YAML configuration clean and focused
- Reusability: Share SQL queries across configurations
- Testing: Test SQL queries independently
!env Tag
Purpose
Expand to the value of an environment variable. Essential for managing secrets and environment-specific configuration.
Syntax
Type Handling
The value is interpreted as YAML/JSON, allowing for different types:
- Strings:
"value" or value
- Numbers:
123 or 45.67
- Booleans:
true or false
- Null:
null
Error Handling
The specified environment variable must be set. If undefined, Query Exporter will fail to start with an error.
Examples
Database credentials:
# config.yaml
databases:
prod_db:
dsn: !env DATABASE_DSN
# .env file or environment
export DATABASE_DSN="postgresql://user:secret@prod-host:5432/mydb"
Multiple environment-specific databases:
databases:
postgres:
dsn: !env POSTGRES_DSN
mysql:
dsn: !env MYSQL_DSN
oracle:
dsn: !env ORACLE_DSN
Partial DSN with environment variable:
databases:
app_db:
dsn:
dialect: postgresql
user: !env DB_USER
password: !env DB_PASSWORD
host: !env DB_HOST
port: !env DB_PORT
database: !env DB_NAME
Using with .env file:
# .env
DB_USER=myuser
DB_PASSWORD=secret123
DB_HOST=db.example.com
DB_PORT=5432
DB_NAME=production
Numeric values:
queries:
user_count:
interval: !env QUERY_INTERVAL # e.g., 30
timeout: !env QUERY_TIMEOUT # e.g., 5.0
databases: [db]
metrics: [user_count]
sql: SELECT COUNT(*) as user_count FROM users
Using .env Files
Query Exporter automatically loads environment variables from a .env file.
Default location:
# Looks for .env in current directory
query-exporter --config config.yaml
Custom .env location:
# Set via environment variable
export QE_DOTENV=/path/to/.env
query-exporter --config config.yaml
Example .env file:
# Database connections
POSTGRES_DSN=postgresql://user:pass@localhost:5432/db
MYSQL_DSN=mysql://user:pass@localhost:3306/db
# API keys
API_KEY=secret-key-123
# Query intervals
FAST_QUERY_INTERVAL=30
SLOW_QUERY_INTERVAL=300
You can combine multiple tags for powerful configurations:
# config.yaml
databases: !include databases.yaml
metrics: !include metrics.yaml
queries: !include queries.yaml
# databases.yaml
postgres:
dsn: !env POSTGRES_DSN
connect-sql:
- !file sql/postgres-init.sql
# queries.yaml
user_stats:
databases: [postgres]
metrics: [active_users]
sql: !file sql/user-stats.sql
interval: !env USER_STATS_INTERVAL
Best Practices
For !include
- Use relative paths for portability
- Organize by logical groupings (databases, queries, metrics)
- Keep a main
config.yaml as the entry point
For !file
- Store SQL files in a dedicated directory (e.g.,
sql/ or queries/)
- Name SQL files descriptively (e.g.,
user-activity.sql, not query1.sql)
- Include comments in SQL files for documentation
For !env
- Never commit secrets to version control
- Use
.env for local development
- Use system environment variables for production
- Document required environment variables in your README
- Provide
.env.example with placeholder values
Example .env.example:
# Database connections
POSTGRES_DSN=postgresql://user:password@host:5432/database
MYSQL_DSN=mysql://user:password@host:3306/database
# Query configuration
QUERY_INTERVAL=60
QUERY_TIMEOUT=5.0
Complete Example
Here’s a complete example using all three tags:
# config.yaml
databases: !include config/databases.yaml
metrics: !include config/metrics.yaml
queries: !include config/queries.yaml
# config/databases.yaml
postgres_prod:
dsn: !env POSTGRES_PROD_DSN
labels:
environment: production
region: us-east
postgres_staging:
dsn: !env POSTGRES_STAGING_DSN
labels:
environment: staging
region: us-west
# config/queries.yaml
database_stats:
interval: 30
databases: [postgres_prod, postgres_staging]
metrics: [db_connections, db_transactions]
sql: !file sql/database-stats.sql
table_stats:
interval: 60
databases: [postgres_prod]
metrics: [table_size, table_rows]
sql: !file sql/table-stats.sql
Further Reading