Polling Pattern for Async Jobs

The CGIAR Risk Intelligence Tool uses an asynchronous job pattern for long-running AI operations like prompt previews, document parsing, gap detection, risk analysis, and report generation.

Architecture

The async processing flow works as follows:

Implementation Pattern

Step 1: Initiate Job

Call an endpoint that creates an async job (e.g. POST /api/admin/prompts/preview). The endpoint returns immediately with:

202 Accepted status
jobId (UUID) to poll
status: "PROCESSING"

const response = await fetch('https://api.example.com/api/admin/prompts/preview', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    systemPrompt: 'You are an expert agricultural risk analyst...',
    userPrompt: 'Identify gaps in market risk coverage.',
    variables: { category_1: 'Market Risk' }
  })
});

const { data } = await response.json();
// data = { jobId: '550e8400-e29b-41d4-a716-446655440000', status: 'PROCESSING' }

Step 2: Poll Job Status

Poll GET /api/jobs/:id every 3 seconds until status is COMPLETED or FAILED.

function pollJobStatus(jobId: string, token: string): Promise<Job> {
  return new Promise((resolve, reject) => {
    const interval = setInterval(async () => {
      try {
        const response = await fetch(`https://api.example.com/api/jobs/${jobId}`, {
          headers: { 'Authorization': `Bearer ${token}` }
        });
        
        const job = await response.json();
        
        if (job.status === 'COMPLETED') {
          clearInterval(interval);
          resolve(job);
        } else if (job.status === 'FAILED') {
          clearInterval(interval);
          reject(new Error(job.error || 'Job failed'));
        }
        // If PENDING or PROCESSING, continue polling
      } catch (error) {
        clearInterval(interval);
        reject(error);
      }
    }, 3000); // Poll every 3 seconds
  });
}

// Usage
try {
  const job = await pollJobStatus(data.jobId, token);
  console.log('Job completed:', job.result);
} catch (error) {
  console.error('Job failed:', error);
}

Step 3: Handle Results

Once the job status is COMPLETED, extract the result field:

if (job.status === 'COMPLETED') {
  // For AI_PREVIEW jobs:
  const aiOutput = job.result.output;
  
  // For PARSE_DOCUMENT jobs:
  const extractedText = job.result.text;
  const metadata = job.result.metadata;
  
  // For GAP_DETECTION jobs:
  const gaps = job.result.gaps;
  
  // For RISK_ANALYSIS jobs:
  const riskScores = job.result.scores;
  
  // For REPORT_GENERATION jobs:
  const reportUrl = job.result.reportUrl;
}

Retry Logic

Jobs automatically retry up to 3 times on failure:

attempts field tracks retry count
maxAttempts is set to 3 by default
After 3 failed attempts, status becomes FAILED

Timeout Recommendations

Client-Side Timeout

Implement a maximum polling duration to avoid infinite loops:

function pollJobStatus(
  jobId: string, 
  token: string,
  maxDuration: number = 120000 // 2 minutes
): Promise<Job> {
  return new Promise((resolve, reject) => {
    const startTime = Date.now();
    
    const interval = setInterval(async () => {
      if (Date.now() - startTime > maxDuration) {
        clearInterval(interval);
        reject(new Error('Polling timeout exceeded'));
        return;
      }
      
      try {
        const response = await fetch(`https://api.example.com/api/jobs/${jobId}`, {
          headers: { 'Authorization': `Bearer ${token}` }
        });
        
        const job = await response.json();
        
        if (job.status === 'COMPLETED') {
          clearInterval(interval);
          resolve(job);
        } else if (job.status === 'FAILED') {
          clearInterval(interval);
          reject(new Error(job.error || 'Job failed'));
        }
      } catch (error) {
        clearInterval(interval);
        reject(error);
      }
    }, 3000);
  });
}

Expected Durations

Job Type	Typical Duration
`AI_PREVIEW`	3-10 seconds
`PARSE_DOCUMENT`	5-30 seconds (depends on file size)
`GAP_DETECTION`	10-30 seconds
`RISK_ANALYSIS`	15-60 seconds (7 categories analyzed)
`REPORT_GENERATION`	20-90 seconds

Best Practices

1. Exponential Backoff (Optional)

For production use, consider exponential backoff if the job is taking longer than expected:

let pollInterval = 3000; // Start at 3 seconds
let attempt = 0;

const interval = setInterval(async () => {
  attempt++;
  
  // After 10 attempts (30 seconds), slow down polling
  if (attempt > 10) {
    clearInterval(interval);
    // Switch to slower polling
    pollInterval = 10000; // 10 seconds
  }
  
  // ... fetch job status
}, pollInterval);

2. User Feedback

Show progress indicators while polling:

const statuses = {
  PENDING: { message: 'Job queued...', icon: '⏳' },
  PROCESSING: { message: 'AI model processing...', icon: '🤖' },
  COMPLETED: { message: 'Complete!', icon: '✅' },
  FAILED: { message: 'Job failed', icon: '❌' }
};

// Update UI with current status
const { message, icon } = statuses[job.status];

3. Error Handling

Handle different failure scenarios:

if (job.status === 'FAILED') {
  if (job.error?.includes('ThrottlingException')) {
    // Bedrock rate limit hit — suggest retry later
  } else if (job.error?.includes('ValidationException')) {
    // Invalid input — don't retry
  } else {
    // Unknown error — allow manual retry
  }
}

4. Cancellation

Allow users to cancel polling (jobs continue running in background):

let pollController: AbortController | null = null;

function startPolling(jobId: string) {
  pollController = new AbortController();
  
  const interval = setInterval(async () => {
    if (pollController?.signal.aborted) {
      clearInterval(interval);
      return;
    }
    // ... fetch job status
  }, 3000);
}

function cancelPolling() {
  pollController?.abort();
}

React Hook Example

import { useState, useEffect } from 'react';

type Job = {
  id: string;
  status: 'PENDING' | 'PROCESSING' | 'COMPLETED' | 'FAILED';
  result?: any;
  error?: string;
};

export function useJobPolling(jobId: string | null) {
  const [job, setJob] = useState<Job | null>(null);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  
  useEffect(() => {
    if (!jobId) return;
    
    setLoading(true);
    setError(null);
    
    const interval = setInterval(async () => {
      try {
        const response = await fetch(`/api/jobs/${jobId}`);
        const data = await response.json();
        
        setJob(data);
        
        if (data.status === 'COMPLETED' || data.status === 'FAILED') {
          clearInterval(interval);
          setLoading(false);
          
          if (data.status === 'FAILED') {
            setError(data.error || 'Job failed');
          }
        }
      } catch (err) {
        clearInterval(interval);
        setLoading(false);
        setError('Failed to fetch job status');
      }
    }, 3000);
    
    return () => clearInterval(interval);
  }, [jobId]);
  
  return { job, loading, error };
}

// Usage in component
function MyComponent() {
  const [jobId, setJobId] = useState<string | null>(null);
  const { job, loading, error } = useJobPolling(jobId);
  
  const handlePreview = async () => {
    const response = await fetch('/api/admin/prompts/preview', {
      method: 'POST',
      body: JSON.stringify({ /* ... */ })
    });
    const { data } = await response.json();
    setJobId(data.jobId);
  };
  
  return (
    <div>
      <button onClick={handlePreview}>Preview Prompt</button>
      
      {loading && <p>Processing... ({job?.status})</p>}
      {error && <p className="error">{error}</p>}
      {job?.status === 'COMPLETED' && (
        <pre>{JSON.stringify(job.result, null, 2)}</pre>
      )}
    </div>
  );
}

Job Scoping

Jobs are scoped to the user who created them:

Job records include createdById field
GET /api/jobs/:id only returns jobs created by the authenticated user
Attempting to access another user’s job returns 404

Database Schema

Jobs are stored in the jobs table with the following fields:

model Job {
  id          String    @id @default(uuid())
  type        JobType   // AI_PREVIEW, PARSE_DOCUMENT, etc.
  status      JobStatus @default(PENDING)
  input       Json      // serialized request payload
  result      Json?     // output from worker
  error       String?   // error message if FAILED
  attempts    Int       @default(0)
  maxAttempts Int       @default(3)
  createdById String
  createdAt   DateTime  @default(now())
  startedAt   DateTime?
  completedAt DateTime?
  updatedAt   DateTime  @updatedAt
}

Authentication

Assessments

Jobs

Prompts

Admin

Architecture

Implementation Pattern

Step 1: Initiate Job

Step 2: Poll Job Status

Step 3: Handle Results

Retry Logic

Timeout Recommendations

Client-Side Timeout

Expected Durations

Best Practices

1. Exponential Backoff (Optional)

2. User Feedback

3. Error Handling

4. Cancellation

React Hook Example

Job Scoping

Database Schema

See Also

Build docs developers (and LLMs) love

Authentication

Assessments

Jobs

Prompts

Admin

​Architecture

​Implementation Pattern

​Step 1: Initiate Job

​Step 2: Poll Job Status

​Step 3: Handle Results

​Retry Logic

​Timeout Recommendations

​Client-Side Timeout

​Expected Durations

​Best Practices

​1. Exponential Backoff (Optional)

​2. User Feedback

​3. Error Handling

​4. Cancellation

​React Hook Example

​Job Scoping

​Database Schema

​See Also

Build docs developers (and LLMs) love

Architecture

Implementation Pattern

Step 1: Initiate Job

Step 2: Poll Job Status

Step 3: Handle Results

Retry Logic

Timeout Recommendations

Client-Side Timeout

Expected Durations

Best Practices

1. Exponential Backoff (Optional)

2. User Feedback

3. Error Handling

4. Cancellation

React Hook Example

Job Scoping

Database Schema

See Also