Skip to main content

Overview

Regular backups are essential for protecting your Headscale deployment. This guide covers both SQLite and PostgreSQL backup strategies, automated scheduling, and disaster recovery procedures.

Backup Components

A complete backup includes:
  • Database: PostgreSQL or SQLite database containing all Headscale state
  • Configuration files: config/ directory with config.yaml and policy.json
  • Environment variables: .env file with credentials and settings
  • Headscale data: data/ directory with keys and state files
  • Headplane configuration: headplane/config.yaml
Never commit backup files, .env, or the data/ directory to version control. These contain sensitive credentials and private keys.

SQLite Backup

Manual Backup

For deployments using SQLite (default configuration):
# Stop Headscale to ensure consistent backup
docker compose stop headscale

# Create compressed backup with timestamp
tar -czf headscale-backup-$(date +%Y%m%d).tar.gz config/ data/ headplane/

# Restart Headscale
docker compose start headscale
Stopping Headscale ensures database consistency. The brief downtime (10-30 seconds) prevents write conflicts during backup.

Automated SQLite Backup

Create a cron job for daily backups:
cron
# Daily backup at 2 AM
0 2 * * * cd /path/to/headscale && docker compose stop headscale && tar -czf backups/backup-$(date +\%Y\%m\%d).tar.gz config/ data/ headplane/ && docker compose start headscale

# Weekly backup retention (keep last 4 weeks)
0 3 * * 0 find /path/to/headscale/backups -name "*.tar.gz" -mtime +28 -delete

PostgreSQL Backup

Manual PostgreSQL Backup

For production deployments using PostgreSQL:
# Create backup directory
mkdir -p ./backups

# Backup PostgreSQL database
docker exec headscale-db pg_dump -U headscale headscale > "./backups/database_$(date +%Y%m%d_%H%M%S).sql"

# Backup configuration files
tar -czf "./backups/config_$(date +%Y%m%d_%H%M%S).tar.gz" config/ .env

# Backup Headscale data directory
tar -czf "./backups/data_$(date +%Y%m%d_%H%M%S).tar.gz" data/

Using the Backup Script

The included backup script automates the entire process:
# Run the automated backup script
./scripts/backup.sh
The script performs:
  1. PostgreSQL database dump
  2. Configuration file backup
  3. Headscale data backup
  4. Automatic cleanup of backups older than 7 days
Starting backup...
Backing up PostgreSQL database...
Database backed up to: ./backups/database_20260304_143022.sql
Backing up configuration files...
Configuration backed up to: ./backups/config_20260304_143022.tar.gz
Backing up Headscale data...
Data backed up to: ./backups/data_20260304_143022.tar.gz

Backup completed successfully!
Backup location: ./backups

Cleaning up old backups...
Old backups cleaned up

Automated PostgreSQL Backup

cron
# Daily PostgreSQL backup at 2 AM
0 2 * * * cd /path/to/headscale && ./scripts/backup.sh >> /var/log/headscale-backup.log 2>&1

Restore Procedures

SQLite Restore

# Stop all services
docker compose down

# Restore from backup archive
tar -xzf headscale-backup-YYYYMMDD.tar.gz

# Verify restored files
ls -lh config/ data/

# Restart services
docker compose up -d

# Verify health
curl http://localhost:8000/health

PostgreSQL Restore

# Stop Headscale service
docker compose stop headscale

# Restore database
cat ./backups/database_TIMESTAMP.sql | docker exec -i headscale-db psql -U headscale

# Restore configuration
tar -xzf ./backups/config_TIMESTAMP.tar.gz

# Restore data directory
tar -xzf ./backups/data_TIMESTAMP.tar.gz

# Restart all services
docker compose up -d

# Verify restoration
docker exec headscale headscale nodes list
Always verify database credentials match between .env and config/config.yaml after restoration:
  • POSTGRES_PASSWORD in .env
  • database.postgres.pass in config/config.yaml

Backup Storage Strategies

Local Backup Retention

Implement a rotation policy:
# Keep daily backups for 7 days
find ./backups -name "*.sql" -mtime +7 -delete
find ./backups -name "*.tar.gz" -mtime +7 -delete

# Keep weekly backups for 4 weeks
find ./backups/weekly -name "*.tar.gz" -mtime +28 -delete

# Keep monthly backups for 1 year
find ./backups/monthly -name "*.tar.gz" -mtime +365 -delete

Remote Backup Storage

Sync backups to remote storage for disaster recovery:
# Upload to Amazon S3
aws s3 sync ./backups s3://your-bucket/headscale-backups/ \
  --exclude "*" \
  --include "*.sql" \
  --include "*.tar.gz" \
  --storage-class STANDARD_IA

Encrypted Backups

For sensitive environments, encrypt backups before storage:
# Encrypt backup with GPG
tar -czf - config/ data/ | gpg --symmetric --cipher-algo AES256 > backup-$(date +%Y%m%d).tar.gz.gpg

# Decrypt and restore
gpg --decrypt backup-YYYYMMDD.tar.gz.gpg | tar -xz

Disaster Recovery

Full System Recovery

Recover from complete system failure:
  1. Install Prerequisites
    # Install Docker and Docker Compose
    curl -fsSL https://get.docker.com | sh
    
  2. Clone Repository
    git clone https://github.com/your-org/headscale-docker.git
    cd headscale-docker
    
  3. Restore Backups
    # Extract configuration and data
    tar -xzf backup-YYYYMMDD.tar.gz
    
    # Or restore from remote storage
    aws s3 sync s3://your-bucket/headscale-backups/latest ./
    
  4. Verify Configuration
    # Check database credentials
    grep POSTGRES_PASSWORD .env
    grep "pass:" config/config.yaml
    
    # Update server URL if changed
    grep server_url config/config.yaml
    
  5. Start Services
    docker compose up -d
    
    # Watch logs for errors
    docker compose logs -f
    
  6. Verify Restoration
    # Check health
    curl http://localhost:8000/health
    
    # Verify nodes
    docker exec headscale headscale nodes list
    
    # Verify users
    docker exec headscale headscale users list
    

Partial Recovery

Recover specific components:
# Stop Headscale
docker compose stop headscale

# Restore database
cat backup.sql | docker exec -i headscale-db psql -U headscale

# Restart
docker compose start headscale
# Extract configuration
tar -xzf config_backup.tar.gz config/

# Restart to apply changes
docker compose restart headscale
# Extract keys from data backup
tar -xzf data_backup.tar.gz data/private.key data/noise_private.key

# Restart Headscale
docker compose restart headscale

Migration Between Hosts

Move your Headscale deployment to a new server:
  1. On Source Server
    # Create complete backup
    docker compose stop headscale
    tar -czf headscale-migration.tar.gz config/ data/ .env headplane/
    docker compose start headscale
    
    # Transfer to new server
    scp headscale-migration.tar.gz user@new-server:/tmp/
    
  2. On Target Server
    # Extract backup
    tar -xzf /tmp/headscale-migration.tar.gz
    
    # Update server URL if changed
    nano config/config.yaml  # Update server_url
    
    # Start services
    docker compose up -d
    
  3. Update DNS
    • Point domain to new server IP
    • Wait for DNS propagation
    • Verify clients reconnect automatically
Clients will automatically reconnect to the new server once DNS propagates. No client-side configuration changes are needed.

Testing Backups

Regularly verify backup integrity:
# Create test environment
mkdir -p /tmp/headscale-test
cd /tmp/headscale-test

# Extract backup
tar -xzf /path/to/backup.tar.gz

# Verify files exist
test -f config/config.yaml && echo "Config OK"
test -f data/db.sqlite && echo "Database OK"
test -f .env && echo "Environment OK"

# Test database integrity (SQLite)
sqlite3 data/db.sqlite "PRAGMA integrity_check;"

# Test PostgreSQL dump
grep -q "PostgreSQL database dump complete" backup.sql && echo "SQL dump OK"

Best Practices

Backup Frequency

  • Daily automated backups for production
  • Pre-update manual backups
  • Before major configuration changes

Storage Security

  • Encrypt backups containing sensitive data
  • Store in multiple geographic locations
  • Restrict access with proper permissions

Retention Policy

  • Keep 7 daily backups
  • Keep 4 weekly backups
  • Keep 12 monthly backups

Validation

  • Test restores quarterly
  • Verify backup integrity automatically
  • Document recovery procedures

Troubleshooting

# Fix file permissions
sudo chown -R $USER:$USER config/ data/ backups/

# Verify Docker socket access
ls -l /var/run/docker.sock
# Check PostgreSQL connection
docker exec -it headscale-db psql -U headscale -d headscale

# Verify credentials match
grep POSTGRES_PASSWORD .env
grep "pass:" config/config.yaml

# Clear database before restore
docker exec -i headscale-db psql -U headscale -c "DROP DATABASE headscale;"
docker exec -i headscale-db psql -U headscale -c "CREATE DATABASE headscale;"
# Check logs for errors
docker compose logs headscale

# Verify server_url matches deployment
grep server_url config/config.yaml

# Check file ownership
ls -la data/

Security

Learn about securing your backups and credentials

Monitoring

Set up backup monitoring and alerting

Build docs developers (and LLMs) love