Scheduling & Automation

Overview

Dockhand’s unified scheduler service manages all automated tasks using cron expressions with timezone support. The scheduler handles container auto-updates, Git stack synchronization, system cleanup, and custom maintenance tasks.

Scheduler Architecture

The scheduler is built on Croner, a robust cron implementation with timezone support:

/**
 * Unified Scheduler Service
 *
 * Manages all scheduled tasks using croner with automatic job lifecycle:
 * - System cleanup jobs (static cron schedules)
 * - Container auto-updates (dynamic schedules from database)
 * - Git stack auto-sync (dynamic schedules from database)
 *
 * All execution logic is in separate task files for clean architecture.
 */
import { Cron } from 'croner';

const activeJobs: Map<string, Cron> = new Map();

export async function startScheduler(): Promise<void> {
  console.log('[Scheduler] Starting scheduler service...');
  
  // Get default timezone from database
  const defaultTimezone = await getDefaultTimezone();
  
  // Start system cleanup jobs
  cleanupJob = new Cron(scheduleCleanupCron, { timezone: defaultTimezone }, async () => {
    await runScheduleCleanupJob();
  });
  
  // Register all dynamic schedules from database
  await refreshAllSchedules();
  
  console.log('[Scheduler] Service started');
}

Cron Expression Format

Dockhand uses standard cron expressions with optional seconds:

# Format: [second] minute hour day month weekday
# Ranges: 0-59   0-59  0-23 1-31 1-12  0-7

# Examples:
"0 3 * * *"        # Daily at 3:00 AM
"*/15 * * * *"     # Every 15 minutes
"0 */6 * * *"      # Every 6 hours
"0 0 * * 0"        # Weekly on Sunday at midnight
"0 2 1 * *"        # Monthly on the 1st at 2:00 AM
"0 0 1 1 *"        # Yearly on January 1st at midnight

Advanced Expressions

# Multiple values
"0 0 8,12,16 * * *"     # At 8 AM, 12 PM, and 4 PM

# Ranges
"0 0 9-17 * * 1-5"      # Every hour from 9 AM to 5 PM, Monday-Friday

# Steps
"*/30 9-17 * * 1-5"     # Every 30 min during work hours on weekdays

# Day of week names
"0 9 * * MON-FRI"       # 9 AM Monday through Friday

Timezone Support

Every schedule can have its own timezone:

// Environment-specific timezone
{
  "cronExpression": "0 2 * * *",
  "timezone": "America/New_York"  // 2 AM Eastern Time
}

// Default timezone for system jobs
{
  "defaultTimezone": "Europe/Warsaw"  // Stored in settings
}

Supported Timezones

Any IANA timezone is supported:

UTC
America/New_York
Europe/London
Asia/Tokyo
Australia/Sydney

Container Auto-Updates

Configuration

Configure automatic updates for any container:

POST /api/auto-update/settings

{
  "containerName": "myapp",
  "environmentId": 1,
  "enabled": true,
  "cronExpression": "0 3 * * *",
  "vulnerabilityCriteria": "critical_high"
}

Vulnerability Criteria

Block updates based on vulnerability scans:

type VulnerabilityCriteria = 
  | 'never'              // Never block updates (no scanning)
  | 'any'                // Block if any vulnerabilities found
  | 'critical_high'      // Block if critical or high vulnerabilities
  | 'critical'           // Block only on critical vulnerabilities
  | 'more_than_current'; // Block if worse than current image

Update Process

The auto-update task performs intelligent updates:

export async function runContainerUpdate(
  settingId: number,
  containerName: string,
  environmentId: number | null | undefined,
  triggeredBy: ScheduleTrigger
): Promise<void> {
  const log = (message: string) => {
    console.log(`[Auto-update] ${message}`);
    appendScheduleExecutionLog(execution.id, `[${new Date().toISOString()}] ${message}`);
  };

  log(`Checking container: ${containerName}`);
  
  // Check registry for updates
  const registryCheck = await checkImageUpdateAvailable(imageNameFromConfig, currentImageId, envId);
  
  if (!registryCheck.hasUpdate) {
    log(`Already up-to-date: ${containerName}`);
    return;
  }

  log(`Update available! Registry digest: ${registryCheck.registryDigest}`);
  
  // Pull new image
  await pullImage(imageNameFromConfig, undefined, envId);
  
  // Scan for vulnerabilities if enabled
  if (shouldScan) {
    const scanOutcome = await scanAndCheckBlock({
      newImageId,
      currentImageId,
      envId,
      vulnerabilityCriteria,
      log
    });
    
    if (scanOutcome.blocked) {
      log(`UPDATE BLOCKED: ${scanOutcome.reason}`);
      await sendEventNotification('auto_update_blocked', {
        title: 'Auto-update blocked',
        message: `Container "${containerName}" update blocked: ${scanOutcome.reason}`,
        type: 'warning'
      }, envId);
      return;
    }
  }
  
  // Recreate container with new image
  log(`Recreating container with full config passthrough...`);
  await recreateContainer(containerName, envId, log, imageNameFromConfig);
  
  log(`Successfully updated container: ${containerName}`);
}

Skip Conditions

Auto-updates are automatically skipped for:

Digest-pinned images: nginx@sha256:abc123...
Local images: Images not available in a registry
System containers: Dockhand and Hawser agents
Already up-to-date: Current image matches registry
Failed vulnerability scan: When criteria not met

Git Stack Sync

Scheduled Sync

Automatically sync Git stacks on a schedule:

{
  "stackName": "production-app",
  "autoUpdate": true,
  "autoUpdateCron": "0 */6 * * *"  // Every 6 hours
}

Smart Sync Behavior

The scheduler only redeploys when changes are detected:

export async function runGitStackSync(
  stackId: number,
  stackName: string,
  environmentId: number | null | undefined,
  triggeredBy: ScheduleTrigger
): Promise<void> {
  log(`Starting sync for stack: ${stackName}`);

  // Deploy the git stack (only if there are changes)
  const result = await deployGitStack(stackId, { force: false });

  if (result.success) {
    if (result.skipped) {
      log(`No changes detected for stack: ${stackName}, skipping redeploy`);
      await sendEventNotification('git_sync_skipped', {
        title: 'Git sync skipped',
        message: `Stack "${stackName}" sync skipped: no changes detected`,
        type: 'info'
      }, envId);
    } else {
      log(`Successfully deployed stack: ${stackName}`);
      await sendEventNotification('git_sync_success', {
        title: 'Git stack deployed',
        message: `Stack "${stackName}" was synced and deployed successfully`,
        type: 'success'
      }, envId);
    }
  }
}

System Cleanup Jobs

Schedule Execution Cleanup

Automatically remove old execution logs:

{
  "scheduleCleanupEnabled": true,
  "scheduleCleanupCron": "0 2 * * *",      // Daily at 2 AM
  "scheduleRetentionDays": 30               // Keep 30 days
}

Event Cleanup

Clean up container event logs:

{
  "eventCleanupEnabled": true,
  "eventCleanupCron": "0 3 * * *",         // Daily at 3 AM
  "eventRetentionDays": 7                   // Keep 7 days
}

Volume Helper Cleanup

Automatic cleanup of temporary volume browser containers:

// Runs every 30 minutes automatically
volume HelperCleanupJob = new Cron('*/30 * * * *', { timezone: defaultTimezone }, async () => {
  await runVolumeHelperCleanupJob('cron', volumeCleanupFns);
});

Environment Update Checks

Schedule periodic checks for available updates across an entire environment:

{
  "environmentId": 1,
  "enabled": true,
  "cron": "0 8 * * *",  // Daily at 8 AM
  "notifyOnAvailable": true
}

Image Pruning

Automate Docker image cleanup:

{
  "environmentId": 1,
  "enabled": true,
  "cronExpression": "0 4 * * 0",  // Weekly on Sunday at 4 AM
  "pruneAll": false,
  "pruneFilters": {
    "dangling": true,
    "until": "720h"  // 30 days
  }
}

Schedule Management

Register a Schedule

export async function registerSchedule(
  scheduleId: number,
  type: 'container_update' | 'git_stack_sync' | 'env_update_check' | 'image_prune',
  environmentId: number | null
): Promise<boolean> {
  const key = `${type}-${scheduleId}`;

  // Get timezone for this environment
  const timezone = environmentId ? await getEnvironmentTimezone(environmentId) : 'UTC';

  // Create new Cron instance with timezone
  const job = new Cron(cronExpression, { timezone }, async () => {
    // Execute the task
    if (type === 'container_update') {
      await runContainerUpdate(scheduleId, containerName, environmentId, 'cron');
    } else if (type === 'git_stack_sync') {
      await runGitStackSync(scheduleId, stackName, environmentId, 'cron');
    }
  });

  // Store in active jobs map
  activeJobs.set(key, job);
  console.log(`[Scheduler] Registered ${type} schedule ${scheduleId}: ${cronExpression} [${timezone}]`);
  return true;
}

Unregister a Schedule

export function unregisterSchedule(
  scheduleId: number,
  type: 'container_update' | 'git_stack_sync' | 'env_update_check' | 'image_prune'
): void {
  const key = `${type}-${scheduleId}`;
  const job = activeJobs.get(key);

  if (job) {
    job.stop();
    activeJobs.delete(key);
    console.log(`[Scheduler] Unregistered ${type} schedule ${scheduleId}`);
  }
}

Refresh Schedules

Reload all schedules from database:

POST /api/schedules/refresh

Execution Tracking

All scheduled tasks create execution records:

interface ScheduleExecution {
  id: number;
  scheduleType: 'container_update' | 'git_stack_sync' | 'system_cleanup';
  scheduleId: number;
  environmentId: number | null;
  entityName: string;
  triggeredBy: 'cron' | 'manual' | 'webhook';
  status: 'running' | 'success' | 'failed' | 'skipped';
  startedAt: string;
  completedAt: string | null;
  duration: number | null;
  logs: string;
  errorMessage: string | null;
  details: any;
}

View Execution History

GET /api/schedules/executions?scheduleType=container_update&limit=50

Response:

{
  "executions": [
    {
      "id": 123,
      "scheduleType": "container_update",
      "entityName": "nginx",
      "status": "success",
      "triggeredBy": "cron",
      "startedAt": "2024-03-04T03:00:00Z",
      "completedAt": "2024-03-04T03:02:15Z",
      "duration": 135000,
      "logs": "[2024-03-04T03:00:00Z] Checking container: nginx\n..."
    }
  ]
}

Manual Triggers

Trigger any scheduled task manually:

Trigger Container Update

POST /api/auto-update/settings/{id}/trigger

Trigger Git Stack Sync

POST /api/git/stacks/{id}/sync

Trigger System Job

POST /api/schedules/system/{jobId}/trigger

Available system jobs:

schedule-cleanup
event-cleanup
volume-helper-cleanup

Best Practices

Cron Expression Tips

Avoid peak hours for intensive operations

"0 3 * * *"  # Good: 3 AM
"0 14 * * *" # Bad: 2 PM during work hours

Distribute load across the hour

# Instead of all at the top of the hour
"0 0 * * *"   # All jobs at :00

# Spread them out
"0 2 * * *"   # Schedule cleanup at 2 AM
"0 3 * * *"   # Container updates at 3 AM
"0 4 * * *"   # Git syncs at 4 AM

Consider timezones for multi-region deployments

// US East Coast production
{ "timezone": "America/New_York", "cron": "0 2 * * *" }

// European production
{ "timezone": "Europe/London", "cron": "0 2 * * *" }

Update Strategy

// Conservative: Daily updates with critical vulnerability blocking
{
  "cronExpression": "0 3 * * *",
  "vulnerabilityCriteria": "critical_high"
}

// Aggressive: Frequent updates, no blocking
{
  "cronExpression": "0 */6 * * *",
  "vulnerabilityCriteria": "never"
}

// Paranoid: Only update if new image is better
{
  "cronExpression": "0 3 * * 0",  // Weekly
  "vulnerabilityCriteria": "more_than_current"
}

Troubleshooting

Schedule Not Running

Check if scheduler is running:
```
GET /api/health
```

Verify cron expression:

POST /api/schedules/validate
{"cronExpression": "0 3 * * *"}

Check execution logs:

GET /api/schedules/executions?scheduleId={id}

Timezone Issues

// Get next run time in specific timezone
GET /api/schedules/next-run?cron=0 3 * * *&timezone=America/New_York

Response:
{
  "nextRun": "2024-03-05T03:00:00-05:00",
  "localTime": "2024-03-05T08:00:00Z"
}

Performance Issues

If too many schedules are impacting performance:

Consolidate schedules: Group similar tasks
Increase intervals: Use longer cron periods
Distribute timing: Spread jobs across different times
Monitor execution times: Check logs for slow operations

API Reference

# List all schedules
GET /api/schedules

# Get schedule details
GET /api/schedules/{type}/{id}

# Create/update schedule
POST /api/auto-update/settings
POST /api/git/stacks

# Trigger manually
POST /api/schedules/{type}/{id}/trigger

# Get execution history
GET /api/schedules/executions

# Validate cron expression
POST /api/schedules/validate

# Get next run time
GET /api/schedules/next-run

# Refresh all schedules
POST /api/schedules/refresh

Overview

Getting Started

Core Features

Advanced Features

Authentication & Security

Deployment

Integrations

​Overview

​Scheduler Architecture

​Cron Expression Format

​Advanced Expressions

​Timezone Support

​Supported Timezones

​Container Auto-Updates

​Configuration

​Vulnerability Criteria

​Update Process

​Skip Conditions

​Git Stack Sync

​Scheduled Sync

​Smart Sync Behavior

​System Cleanup Jobs

​Schedule Execution Cleanup

​Event Cleanup

​Volume Helper Cleanup

​Environment Update Checks

​Image Pruning

​Schedule Management

​Register a Schedule

​Unregister a Schedule

​Refresh Schedules

​Execution Tracking

​View Execution History

​Manual Triggers

​Trigger Container Update

​Trigger Git Stack Sync

​Trigger System Job

​Best Practices

​Cron Expression Tips

​Update Strategy

​Troubleshooting

​Schedule Not Running

​Timezone Issues

​Performance Issues

​API Reference

Build docs developers (and LLMs) love

Overview

Scheduler Architecture

Cron Expression Format

Advanced Expressions

Timezone Support

Supported Timezones

Container Auto-Updates

Configuration

Vulnerability Criteria

Update Process

Skip Conditions

Git Stack Sync

Scheduled Sync

Smart Sync Behavior

System Cleanup Jobs

Schedule Execution Cleanup

Event Cleanup

Volume Helper Cleanup

Environment Update Checks

Image Pruning

Schedule Management

Register a Schedule

Unregister a Schedule

Refresh Schedules

Execution Tracking

View Execution History

Manual Triggers

Trigger Container Update

Trigger Git Stack Sync

Trigger System Job

Best Practices

Cron Expression Tips

Update Strategy

Troubleshooting

Schedule Not Running

Timezone Issues

Performance Issues

API Reference