Queue Worker & Scheduler Setup - Discord Webhook Manager

Queue workers and the Laravel scheduler are essential for Discord Webhook Manager to function properly. Without them, scheduled messages will not be sent.

This configuration is CRITICAL. Skipping this step will break scheduled message functionality.

Architecture Overview

The scheduled message system works as follows:

Cron executes php artisan schedule:run every minute
Laravel Scheduler triggers the scheduled-messages:process command
Command finds messages where next_send_at <= now()
Job dispatches SendScheduledMessage to the Redis queue
Worker (managed by Supervisor):
- Sends the payload to Discord API
- For recurring messages: calculates next send time
- For one-time messages: marks as completed
- Deletes attachment files after sending

Queue Worker Setup (Supervisor)

1. Install Supervisor

If not already installed:

sudo apt update
sudo apt install -y supervisor

# Start and enable Supervisor
sudo systemctl start supervisor
sudo systemctl enable supervisor

2. Create Worker Configuration

Create a Supervisor configuration file:

sudo nano /etc/supervisor/conf.d/discord-worker.conf

Add the following configuration:

[program:discord-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/discord-webhook-manager/app/artisan queue:work redis --queue=default --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/var/www/discord-webhook-manager/app/storage/logs/worker.log
stopwaitsecs=3600

Configuration Breakdown

process_name

Names each worker process uniquely (discord-worker_00, discord-worker_01, etc.)

command

The queue worker command with options:

queue:work redis: Process jobs from Redis queue
--queue=default: Listen to the default queue
--sleep=3: Sleep 3 seconds when no jobs available
--tries=3: Retry failed jobs up to 3 times
--max-time=3600: Restart worker after 1 hour (prevents memory leaks)

autostart & autorestart

Worker starts automatically on boot and restarts if it crashes

user

Run as www-data user (same as web server)

numprocs

Number of worker processes (2 recommended for production)

Increase to 4-8 workers for high-volume deployments.

stdout_logfile

Log file location for worker output and errors

stopwaitsecs

Wait up to 1 hour for jobs to finish before killing worker during shutdown

3. Update Supervisor

Load the new configuration and start workers:

# Reload Supervisor configuration
sudo supervisorctl reread

# Update Supervisor to apply changes
sudo supervisorctl update

# Start the workers
sudo supervisorctl start discord-worker:*

4. Verify Workers Are Running

# Check worker status
sudo supervisorctl status discord-worker:*

You should see output like:

discord-worker:discord-worker_00   RUNNING   pid 12345, uptime 0:00:10
discord-worker:discord-worker_01   RUNNING   pid 12346, uptime 0:00:10

If workers show FATAL or EXITED status, check the worker log for errors:

tail -f /var/www/discord-webhook-manager/app/storage/logs/worker.log

Laravel Scheduler Setup (Cron)

1. Understanding the Scheduler

Laravel’s scheduler is a single cron entry that runs every minute. It then internally manages all scheduled tasks defined in the application.

2. Configure Cron

Edit the crontab for the www-data user:

sudo crontab -u www-data -e

Add this line at the end:

* * * * * cd /var/www/discord-webhook-manager/app && php artisan schedule:run >> /dev/null 2>&1

Cron Entry Breakdown

* * * * *: Run every minute
cd /var/www/discord-webhook-manager/app: Change to application directory
php artisan schedule:run: Execute Laravel scheduler
>> /dev/null 2>&1: Suppress output (logged by Laravel instead)

Make sure the path matches your actual installation directory. The cron job must run as the www-data user to have proper permissions.

3. Verify Cron is Active

Check the crontab was saved:

sudo crontab -u www-data -l

You should see your scheduler entry.

4. Test the Scheduler

Manually run the scheduler to test:

cd /var/www/discord-webhook-manager/app
sudo -u www-data php artisan schedule:run

Check the output for any errors.

Worker Management Commands

Supervisor Commands

sudo supervisorctl status discord-worker:*

Laravel Queue Commands

php artisan queue:monitor

Monitoring and Maintenance

1. Check Worker Logs

Monitor worker activity:

tail -f /var/www/discord-webhook-manager/app/storage/logs/worker.log

2. Check Laravel Logs

View application logs:

tail -f /var/www/discord-webhook-manager/app/storage/logs/laravel.log

3. Monitor Queue Size

Check pending jobs in Redis:

redis-cli
LLEN queues:default
exit

A continuously growing queue indicates workers can’t keep up.

4. Monitor System Resources

Check resource usage:

# CPU and memory usage
top -u www-data

# Worker processes
ps aux | grep "queue:work"

Updating Worker Configuration

When you change the Supervisor configuration:

Edit configuration

sudo nano /etc/supervisor/conf.d/discord-worker.conf

Reload and restart

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl restart discord-worker:*

Verify status

sudo supervisorctl status discord-worker:*

Deployment Updates

After deploying code updates that affect queue jobs:

# Restart workers to load new code
sudo supervisorctl restart discord-worker:*

# Or use artisan command
php artisan queue:restart

The queue:restart command signals workers to restart gracefully after finishing current jobs.

Alternative: Database Queue

If Redis is unavailable, you can use database-backed queues:

1. Update Environment

# In .env
QUEUE_CONNECTION=database

2. Update Supervisor Configuration

[program:discord-worker]
command=php /var/www/discord-webhook-manager/app/artisan queue:work database --queue=default --sleep=3 --tries=3 --max-time=3600
# ... rest of configuration

3. Create Jobs Table

php artisan queue:table
php artisan migrate

Database queues are significantly slower than Redis. Use Redis for production deployments.

Troubleshooting

Workers Not Processing Jobs

Check workers are running:

sudo supervisorctl status discord-worker:*

Check worker logs:
```
tail -f storage/logs/worker.log
```
Verify queue connection:
```
php artisan queue:monitor
```
Test Redis connection:
```
redis-cli ping
```

Scheduler Not Running

Verify cron is configured:
```
sudo crontab -u www-data -l
```
Check cron logs:
```
sudo grep CRON /var/log/syslog
```

Manually test scheduler:

sudo -u www-data php artisan schedule:run -v

High Memory Usage

If workers consume too much memory:

Reduce max-time:

# In supervisor config
command=php artisan queue:work redis --max-time=1800

Add memory limit:

command=php artisan queue:work redis --memory=128

Restart workers more frequently:
```
php artisan queue:restart
```

Failed Jobs Accumulating

View failed jobs:
```
php artisan queue:failed
```
Retry specific job:
```
php artisan queue:retry {job-id}
```
Retry all failed jobs:
```
php artisan queue:retry all
```
Clear failed jobs:
```
php artisan queue:flush
```

Performance Tuning

Increase Workers for High Volume

For high-traffic deployments:

[program:discord-worker]
# ... other config
numprocs=8  # Increase from 2 to 8

Multiple Queue Priorities

Create separate workers for priority queues:

[program:discord-worker-high]
command=php artisan queue:work redis --queue=high,default --sleep=1 --tries=3
numprocs=2

[program:discord-worker-default]
command=php artisan queue:work redis --queue=default --sleep=3 --tries=3
numprocs=4

Optimize Redis

Tune Redis for queue performance:

# In /etc/redis/redis.conf
maxmemory 512mb
maxmemory-policy allkeys-lru

Production Checklist

Before going live, verify:

Supervisor is installed and enabled
Workers are running (supervisorctl status)
Cron is configured for www-data user
Scheduler runs successfully (schedule:run)
Worker logs are being written
Redis connection is working
Queue jobs process correctly
Failed jobs retry mechanism works
Workers restart after code deploys

Next Steps

Monitor worker logs regularly
Set up alerting for worker failures
Configure log rotation for worker logs
Plan for scaling workers based on load
Review Redis migration guide for optimization

Get Started

Core Features

Collaboration

Advanced

Deployment

​Architecture Overview

​Queue Worker Setup (Supervisor)

​1. Install Supervisor

​2. Create Worker Configuration

​Configuration Breakdown

​3. Update Supervisor

​4. Verify Workers Are Running

​Laravel Scheduler Setup (Cron)

​1. Understanding the Scheduler

​2. Configure Cron

​Cron Entry Breakdown

​3. Verify Cron is Active

​4. Test the Scheduler

​Worker Management Commands

​Supervisor Commands

​Laravel Queue Commands

​Monitoring and Maintenance

​1. Check Worker Logs

​2. Check Laravel Logs

​3. Monitor Queue Size

​4. Monitor System Resources

​Updating Worker Configuration

​Deployment Updates

​Alternative: Database Queue

​1. Update Environment

​2. Update Supervisor Configuration

​3. Create Jobs Table

​Troubleshooting

​Workers Not Processing Jobs

​Scheduler Not Running

​High Memory Usage

​Failed Jobs Accumulating

​Performance Tuning

​Increase Workers for High Volume

​Multiple Queue Priorities

​Optimize Redis

​Production Checklist

​Next Steps

Build docs developers (and LLMs) love

Architecture Overview

Queue Worker Setup (Supervisor)

1. Install Supervisor

2. Create Worker Configuration

Configuration Breakdown

3. Update Supervisor

4. Verify Workers Are Running

Laravel Scheduler Setup (Cron)

1. Understanding the Scheduler

2. Configure Cron

Cron Entry Breakdown

3. Verify Cron is Active

4. Test the Scheduler

Worker Management Commands

Supervisor Commands

Laravel Queue Commands

Monitoring and Maintenance

1. Check Worker Logs

2. Check Laravel Logs

3. Monitor Queue Size

4. Monitor System Resources

Updating Worker Configuration

Deployment Updates

Alternative: Database Queue

1. Update Environment

2. Update Supervisor Configuration

3. Create Jobs Table

Troubleshooting

Workers Not Processing Jobs

Scheduler Not Running

High Memory Usage

Failed Jobs Accumulating

Performance Tuning

Increase Workers for High Volume

Multiple Queue Priorities

Optimize Redis

Production Checklist

Next Steps