Production Deployment

Deploying TrailBase to production requires careful consideration of security, reliability, and performance. This guide covers best practices and essential configuration for production environments.

Production Checklist

Before deploying TrailBase to production, ensure you’ve completed these essential steps:

Set a public URL

Configure the public URL for OAuth redirects and email links:

trail --public-url https://yourdomain.com run

Or via environment variable:

export PUBLIC_URL=https://yourdomain.com
trail run

Bind to the correct address

For production behind a reverse proxy:

trail run --address 0.0.0.0:4000

For direct internet exposure, bind to localhost and use a reverse proxy:

trail run --address localhost:4000

Configure CORS

Restrict CORS origins to your domains:

trail run --cors-allowed-origins "https://yourdomain.com,https://www.yourdomain.com"

Set up SMTP for emails

Configure email delivery for auth verification and password resets. Edit traildepot/config.textproto:

email: {
  smtp_host: "smtp.example.com"
  smtp_port: 587
  smtp_encryption: STARTTLS
  smtp_username: "[email protected]"
  smtp_password: "<REDACTED>"  # Actual password goes in secrets.textproto
  sender_address: "[email protected]"
  sender_name: "Your App"
}

Review authentication settings

Adjust token TTLs in config.textproto:

auth: {
  # Auth token expires after 1 hour (in seconds)
  auth_token_ttl_sec: 3600
  
  # Refresh token expires after 30 days
  refresh_token_ttl_sec: 2592000
}

Set up SSL/TLS

Use a reverse proxy (nginx, Caddy, Traefik) for SSL/TLS termination. Never expose TrailBase directly to the internet without HTTPS.

Configure backups

Implement automated backups of the data directory (see Backup Strategies below).

Set up monitoring

Implement health checks and monitoring (see Monitoring section below).

Never disable or weaken security features for convenience. Always use HTTPS, strong passwords, and proper CORS settings in production.

Security Best Practices

SSL/TLS Configuration

Always run TrailBase behind a reverse proxy with SSL/TLS:

server {
    listen 443 ssl http2;
    server_name yourdomain.com;

    ssl_certificate /etc/ssl/certs/yourdomain.crt;
    ssl_certificate_key /etc/ssl/private/yourdomain.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://localhost:4000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

File Permissions

Secure the data directory with proper permissions:

# Set ownership to the TrailBase user
chown -R trailbase:trailbase /var/lib/trailbase

# Restrict access to owner only
chmod 700 /var/lib/trailbase

# Protect sensitive files
chmod 600 /var/lib/trailbase/secrets.textproto
chmod 600 /var/lib/trailbase/*.db

Secrets Management

TrailBase separates configuration into two files:

config.textproto - Non-sensitive configuration
secrets.textproto - Sensitive values (OAuth secrets, SMTP passwords)

Secrets marked with [(secret) = true] in the protobuf schema are automatically redacted to <REDACTED> in config.textproto and stored in secrets.textproto.

You can also override any configuration value using environment variables prefixed with TRAIL_, for example: TRAIL_EMAIL_SMTP_PASSWORD.

Admin Access

For production, consider separating admin UI access:

trail run \
  --address 0.0.0.0:4000 \
  --admin-address 127.0.0.1:4001

This binds:

Public API to 0.0.0.0:4000 (accessible externally)
Admin UI to 127.0.0.1:4001 (localhost only)

Access the admin UI through an SSH tunnel:

ssh -L 4001:localhost:4001 user@yourserver
# Then browse to http://localhost:4001/_/admin

Rate Limiting

Implement rate limiting at the reverse proxy level:

http {
    limit_req_zone $binary_remote_addr zone=trailbase:10m rate=10r/s;
    
    server {
        location / {
            limit_req zone=trailbase burst=20 nodelay;
            proxy_pass http://localhost:4000;
        }
    }
}

Firewall Configuration

# Only allow HTTP/HTTPS and SSH
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw default deny incoming
sudo ufw enable

Backup Strategies

Regular backups are essential for production deployments.

Full Data Directory Backup

The simplest approach is backing up the entire data directory:

#!/bin/bash
# backup-trailbase.sh

DATA_DIR="/var/lib/trailbase/traildepot"
BACKUP_DIR="/backups/trailbase"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p "$BACKUP_DIR"

# Create tarball of data directory
tar -czf "$BACKUP_DIR/trailbase_$TIMESTAMP.tar.gz" -C "$(dirname $DATA_DIR)" "$(basename $DATA_DIR)"

# Keep only last 7 days of backups
find "$BACKUP_DIR" -name "trailbase_*.tar.gz" -mtime +7 -delete

echo "Backup completed: trailbase_$TIMESTAMP.tar.gz"

Schedule with cron:

# Run backup daily at 2 AM
0 2 * * * /usr/local/bin/backup-trailbase.sh

SQLite Backup

For online backups without stopping the server, use SQLite’s backup API:

#!/bin/bash
# sqlite-backup.sh

DATA_DIR="/var/lib/trailbase/traildepot"
BACKUP_DIR="/backups/trailbase/sqlite"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR/$TIMESTAMP"

# Backup main database
sqlite3 "$DATA_DIR/main.db" ".backup '$BACKUP_DIR/$TIMESTAMP/main.db'"

# Backup logs database
sqlite3 "$DATA_DIR/logs.db" ".backup '$BACKUP_DIR/$TIMESTAMP/logs.db'"

# Copy configuration files
cp "$DATA_DIR/config.textproto" "$BACKUP_DIR/$TIMESTAMP/"
cp "$DATA_DIR/secrets.textproto" "$BACKUP_DIR/$TIMESTAMP/"

echo "SQLite backup completed: $TIMESTAMP"

Cloud Backup

Sync backups to cloud storage:

#!/bin/bash
aws s3 sync /backups/trailbase s3://your-bucket/trailbase/ \
  --storage-class STANDARD_IA \
  --exclude "*" \
  --include "*.tar.gz"

Restore from Backup

To restore from a backup:

# Stop TrailBase
sudo systemctl stop trailbase

# Extract backup
tar -xzf trailbase_20240115_020000.tar.gz -C /var/lib/trailbase/

# Restore permissions
chown -R trailbase:trailbase /var/lib/trailbase/traildepot
chmod 700 /var/lib/trailbase/traildepot

# Start TrailBase
sudo systemctl start trailbase

Always test your backup and restore procedures before you need them. Perform regular restore drills to ensure backups are valid.

Monitoring

Health Check Endpoint

TrailBase provides a health check endpoint:

curl http://localhost:4000/api/healthcheck

Returns 200 OK if the server is healthy.

Uptime Monitoring

Use external monitoring services:

UptimeRobot: Simple HTTP monitoring
Pingdom: Advanced monitoring with alerting
Better Uptime: Modern uptime monitoring
StatusCake: HTTP and performance monitoring

Example configuration:

Monitor URL: https://yourdomain.com/api/healthcheck
Check interval: 60 seconds
Alert after: 2 consecutive failures

Log Monitoring

TrailBase stores access and error logs in the logs.db SQLite database:

-- Query recent errors
SELECT * FROM _logs 
WHERE level = 'ERROR' 
ORDER BY timestamp DESC 
LIMIT 100;

-- Query access patterns
SELECT 
  COUNT(*) as requests,
  path,
  method
FROM _logs
WHERE timestamp > datetime('now', '-1 hour')
GROUP BY path, method
ORDER BY requests DESC;

For stdout logging (e.g., in Docker):

trail run --stderr-logging

Log Retention

Configure log retention in config.textproto:

server: {
  # Keep logs for 7 days (in seconds)
  logs_retention_sec: 604800
}

Resource Monitoring

Monitor system resources:

# CPU and memory usage
top -b -n 1 | grep trail

# Database size
du -sh /var/lib/trailbase/traildepot/*.db

# Disk usage
df -h /var/lib/trailbase

For automated monitoring, use:

Prometheus + Grafana: Metrics and dashboards
Netdata: Real-time performance monitoring
Datadog: Full-stack monitoring

Performance Metrics

Key metrics to monitor:

Request latency: P50, P95, P99 response times
Request rate: Requests per second
Error rate: 4xx and 5xx responses
Database size: Growth rate
CPU usage: Average and peak
Memory usage: RSS and growth
Disk I/O: Read/write operations

Scaling

Vertical Scaling

For most workloads, vertical scaling is the simplest approach:

Small: 1 CPU, 512 MB RAM - Up to 100 req/s
Medium: 2 CPU, 2 GB RAM - Up to 500 req/s
Large: 4 CPU, 4 GB RAM - Up to 2000 req/s
X-Large: 8 CPU, 8 GB RAM - Up to 5000+ req/s

TrailBase is extremely efficient. SQLite’s sub-millisecond latencies mean most applications won’t need horizontal scaling.

Horizontal Scaling (Read Replicas)

For read-heavy workloads:

Primary instance

Run a single write instance:

trail --data-dir /var/lib/trailbase/primary run

Create read replicas

Periodically copy the database to read-only instances:

# On primary
sqlite3 /var/lib/trailbase/primary/main.db ".backup '/tmp/main.db'"

# Transfer to replica
scp /tmp/main.db replica:/var/lib/trailbase/replica/main.db

Route traffic

Use a load balancer to route:

Writes → Primary instance
Reads → Read replicas (round-robin)

SQLite does not have built-in replication. For true high-availability and write scaling, consider commercial solutions like Turso or LiteFS.

CDN for Static Assets

If serving static files:

trail run --public-dir /var/www/static

Place a CDN (CloudFlare, Fastly, CloudFront) in front to:

Cache static assets globally
Reduce load on TrailBase
Improve latency for users worldwide

Database Optimization

Optimize SQLite performance:

-- Analyze query patterns
ANALYZE;

-- Rebuild indexes
REINDEX;

-- Vacuum to reclaim space
VACUUM;

Run maintenance periodically:

#!/bin/bash
sqlite3 /var/lib/trailbase/traildepot/main.db "ANALYZE; VACUUM;"

Systemd Service

Run TrailBase as a systemd service:

/etc/systemd/system/trailbase.service

[Unit]
Description=TrailBase Server
After=network.target

[Service]
Type=simple
User=trailbase
Group=trailbase
WorkingDirectory=/var/lib/trailbase
Environment="DATA_DIR=/var/lib/trailbase/traildepot"
Environment="PUBLIC_URL=https://yourdomain.com"
ExecStart=/usr/local/bin/trail run --address 0.0.0.0:4000
Restart=always
RestartSec=10

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/trailbase

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable trailbase
sudo systemctl start trailbase
sudo systemctl status trailbase

Troubleshooting

High CPU Usage

Check for:

Expensive queries (add indexes)
Too many realtime subscriptions
Insufficient runtime-threads for WASM workloads

Database Locked

SQLite allows one writer at a time:

Enable WAL mode (enabled by default in TrailBase)
Reduce write transaction duration
Consider read replicas for read-heavy loads

Memory Leaks

Monitor memory over time:

watch -n 60 'ps aux | grep trail'

If memory grows unbounded, file a bug report with:

TrailBase version
Memory growth rate
Request patterns
Configuration

Next Steps

Configuration

Detailed configuration options and environment variables

Self-Hosting

Learn about different deployment strategies

Getting Started

Core Concepts

Guides

Deployment

Production Deployment

Production Checklist

Security Best Practices

SSL/TLS Configuration

File Permissions

Secrets Management

Admin Access

Rate Limiting

Firewall Configuration

Backup Strategies

Full Data Directory Backup

SQLite Backup

Cloud Backup

Restore from Backup

Monitoring

Health Check Endpoint

Uptime Monitoring

Log Monitoring

Log Retention

Resource Monitoring

Performance Metrics

Scaling

Vertical Scaling

Horizontal Scaling (Read Replicas)

CDN for Static Assets

Database Optimization

Systemd Service

Troubleshooting

High CPU Usage

Database Locked

Memory Leaks

Next Steps

Configuration

Self-Hosting

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Guides

Deployment

​Production Checklist

​Security Best Practices

​SSL/TLS Configuration

​File Permissions

​Secrets Management

​Admin Access

​Rate Limiting

​Firewall Configuration

​Backup Strategies

​Full Data Directory Backup

​SQLite Backup

​Cloud Backup

​Restore from Backup

​Monitoring

​Health Check Endpoint

​Uptime Monitoring

​Log Monitoring

​Log Retention

​Resource Monitoring

​Performance Metrics

​Scaling

​Vertical Scaling

​Horizontal Scaling (Read Replicas)

​CDN for Static Assets

​Database Optimization

​Systemd Service

​Troubleshooting

​High CPU Usage

​Database Locked

​Memory Leaks

​Next Steps

Configuration

Self-Hosting

Build docs developers (and LLMs) love

Production Checklist

Security Best Practices

SSL/TLS Configuration

File Permissions

Secrets Management

Admin Access

Rate Limiting

Firewall Configuration

Backup Strategies

Full Data Directory Backup

SQLite Backup

Cloud Backup

Restore from Backup

Monitoring

Health Check Endpoint

Uptime Monitoring

Log Monitoring

Log Retention

Resource Monitoring

Performance Metrics

Scaling

Vertical Scaling

Horizontal Scaling (Read Replicas)

CDN for Static Assets

Database Optimization

Systemd Service

Troubleshooting

High CPU Usage

Database Locked

Memory Leaks

Next Steps