Skip to main content

​
POST /v1/batches

Create a batch of API requests to be processed asynchronously. Batches are useful for processing large volumes of requests that don’t require immediate responses.

​
Authentication

Requires provider authentication headers: 
x-portkey-provider: openai
Authorization: Bearer YOUR_OPENAI_API_KEY

​
Request

​
Headers

​
x-portkey-provider
string
required
The provider to route the request to (e.g., openai)
​
Authorization
string
required
Bearer token for the provider API

​
Body Parameters

​
input_file_id
string
required
The ID of an uploaded file that contains requests for the batch. The file must be a JSONL file with each line containing a request object.
​
endpoint
string
required
The endpoint to be used for all requests in the batch. Currently supports:
  • /v1/chat/completions
  • /v1/completions
  • /v1/embeddings
​
completion_window
string
required
The time frame within which the batch should be processed. Currently only 24h is supported.
​
metadata
object
Optional metadata to attach to the batch

​
Response

​
id
string
The batch identifier
​
object
string
The object type, always “batch”
​
endpoint
string
The endpoint used for the batch
​
errors
object
Error information if any requests failed
​
input_file_id
string
The ID of the input file
​
completion_window
string
The completion time frame
​
status
string
The status of the batch: validating, in_progress, finalizing, completed, failed, or cancelled
​
output_file_id
string
The ID of the file containing the outputs (available when status is completed)
​
error_file_id
string
The ID of the file containing errors (if any)
​
created_at
integer
Unix timestamp of when the batch was created
​
in_progress_at
integer
Unix timestamp of when the batch started processing
​
completed_at
integer
Unix timestamp of when the batch completed
​
metadata
object
Custom metadata attached to the batch

​
Example

​
Input File Format

First, create a JSONL file with your requests: 
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is 2+2?"}]}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}}

# First, upload the batch file
curl https://localhost:8787/v1/files \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="batch" \
  -F file="@batch_requests.jsonl"

# Then create the batch
curl https://localhost:8787/v1/batches \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

​
Response Example

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "validating",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1713894800,
  "in_progress_at": null,
  "completed_at": null,
  "metadata": {
    "description": "Daily batch job"
  }
}

​
Best Practices

  • Keep batch sizes reasonable (1,000 - 50,000 requests)
  • Monitor processing times and adjust batch sizes accordingly
  • Split very large jobs into multiple batches
  • Always check the error_file_id after batch completion
  • Implement retry logic for failed requests
  • Use the custom_id field to track individual requests
  • Batch API typically offers 50% cost savings compared to real-time API
  • Use batches for non-time-sensitive workloads
  • Combine similar requests to maximize efficiency

Retrieve Batch

Check batch status and results

Cancel Batch

Cancel a running batch

List Batches

View all batches

Build docs developers (and LLMs) love

Get started for freeTalk to us