Skip to main content
The queue API wraps the BullMQ job system that powers GenieHelper’s media processing, platform scraping, and onboarding pipelines. Jobs are consumed by the media-worker process.
All three BullMQ queues run at concurrency:1. This is a deliberate constraint — the server has 16GB RAM, and parallel media or LLM jobs would cause RAM exhaustion. Jobs execute sequentially per queue.

​
Endpoints

MethodPathDescription
GET/api/queue/statsQueue depths, job counts, and taxonomy cache status
POST/api/queue/mediaEnqueue a job that has already been created in Directus
POST/api/queue/retry/:jobIdRe-enqueue a failed job at elevated priority
POST/api/queue/taxonomy/invalidateBust the taxonomy DuckDB cache

​
GET /api/queue/stats

Returns the current depth and job counts for all BullMQ queues, plus the taxonomy cache stats. 
GET /api/queue/stats
Authorization: Bearer <directus_jwt>

​
Response

{
  "ok": true,
  "stats": {
    "media-jobs": {
      "waiting": 2,
      "active": 1,
      "completed": 147,
      "failed": 3,
      "delayed": 0
    },
    "scrape-jobs": {
      "waiting": 0,
      "active": 0,
      "completed": 89,
      "failed": 1,
      "delayed": 0
    },
    "onboarding-jobs": {
      "waiting": 0,
      "active": 0,
      "completed": 12,
      "failed": 0,
      "delayed": 0
    }
  },
  "taxonomy_cache": {
    "hits": 231,
    "misses": 14,
    "size": 191
  }
}

​
POST /api/queue/media

Enqueues a media processing job. The caller is responsible for creating the Directus media_jobs record first, then calling this endpoint to push the job ID into BullMQ so the worker picks it up. 
POST /api/queue/media
Authorization: Bearer <directus_jwt>
Content-Type: application/json
Create the Directus record before calling this endpoint. The worker reads job details from Directus using the provided jobId. If the record does not exist, the worker will fail the job.

​
Body

​
jobId
string
required
The UUID of the existing media_jobs record in Directus.
​
operation
string
required
The job type to execute. See the operations table below for valid values.
​
input_file_id
string
Directus file ID for the input asset. Required for operations that process an existing uploaded file.
​
input_url
string
URL of the input asset. Used for URL-based operations (e.g., scraping, remote downloads).
​
params
object
Operation-specific parameters. Structure varies by job type (see operations table).
​
creator_profile_id
string
Directus creator_profiles record ID. Used to scope output storage and watermark identity.
​
priority
number
BullMQ priority. Lower numbers = higher priority. Default: 10. Retried jobs use 5.
{
  "jobId": "e4b0f1c2-3d45-6789-abcd-ef0123456789",
  "operation": "apply_watermark",
  "input_file_id": "a1b2c3d4-...",
  "params": {
    "watermark_text": "@creator",
    "opacity": 0.4
  },
  "creator_profile_id": "f9e8d7c6-..."
}

​
Response

​
ok
boolean
true when the job was successfully enqueued.
​
bullJobId
string
The BullMQ-assigned job ID for polling status directly against the Redis queue.
​
jobId
string
The Directus media_jobs record ID (echoed from the request).
{
  "ok": true,
  "bullJobId": "bull:media-jobs:1042",
  "jobId": "e4b0f1c2-3d45-6789-abcd-ef0123456789"
}

​
Subscription quota check

Before enqueuing, the endpoint calls canPerform(userId, operation) from subscriptionValidator.js. If the user has exhausted their tier quota for the given operation, the job is rejected with a 403: 
{
  "error": "tier_limit_reached",
  "reason": "limit_reached",
  "message": "You have reached your plan limit for this feature. Upgrade to continue."
}
The validator is fail-open for unexpected errors (i.e., a quota check failure does not block the job). Only explicit limit_reached or not_in_tier results reject the request.

​
POST /api/queue/retry/:jobId

Re-enqueues a failed job. The endpoint fetches the original job record from Directus, validates it is in failed or error status, and re-submits it to BullMQ with elevated priority (5). 
POST /api/queue/retry/e4b0f1c2-3d45-6789-abcd-ef0123456789
Authorization: Bearer <directus_jwt>
StatusMeaning
200Job re-enqueued successfully
404Job not found in Directus
409Job is not in a failed state — cannot retry

​
Queues

media-jobs

Image and video processing operations. FFmpeg/ImageMagick workers. concurrency:1.

scrape-jobs

Browser automation jobs via Stagehand/PinchTab. Profile scraping, post performance, session management. concurrency:1.

onboarding-jobs

LLM-heavy onboarding pipeline jobs. ETL from scraped platform data into user_nodes. concurrency:1.

​
Job types

​
Media operations (media-jobs)

OperationInputDescription
convert_imageinput_file_idConvert image to a different format
resize_imageinput_file_idResize image to target dimensions
crop_imageinput_file_idCrop image to bounding box
resize_videoinput_file_idResize video to target resolution
crop_videoinput_file_idCrop video to bounding box
compress_videoinput_file_idRe-encode video at lower bitrate
apply_watermarkinput_file_idApply visible watermark overlay
apply_steganographic_watermarkinput_file_idEmbed invisible fan-specific watermark
create_teaserinput_file_idGenerate a short preview clip
strip_metadatainput_file_idRemove EXIF and metadata from file

​
Scrape operations (scrape-jobs)

OperationInputDescription
scrape_profilecreator_profile_idScrape platform profile stats
scrape_post_performancecreator_profile_idScrape per-post engagement metrics
publish_postcreator_profile_idAutomated post submission to platform

​
Onboarding operations (onboarding-jobs)

Onboarding jobs follow the pattern onboarding:<phase> (e.g., onboarding:extract, onboarding:classify, onboarding:embed). These are triggered internally by the onboarding state machine, not directly by API consumers.
The media-worker implementation was split into an operations/ module structure on 2026-03-15. Handlers live in operations/media.js, operations/scrape.js, operations/publish.js, and operations/onboarding.js, with shared helpers in operations/helpers.js.

Build docs developers (and LLMs) love

Get started for freeTalk to us