Endpoint
Authentication
This endpoint requires API key authentication. Include your API key in the request headers:
Authorization: Bearer YOUR_API_KEY
Request Body
No specific parameters required. Send an empty object.
Response
Array of evaluator objectsUnique identifier for the evaluator
Type of scoring: “LLM-BOOLEAN”, “LLM-CHOICE”, “LLM-RANGE”, “PYTHON”, or “LASTMILE”
LLM configuration for LLM-based evaluators
Python code for code-based evaluators
Configuration for LastMile evaluators
Organization that owns this evaluator
Error message if the request failed, null otherwise
Example Request
curl -X POST https://api.helicone.ai/v1/evaluator/query \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{}'
Example Response
{
"data": [
{
"id": "eval_abc123",
"name": "Hallucination Detector",
"scoring_type": "LLM-BOOLEAN",
"llm_template": {
"model": "gpt-4",
"prompt": "Does this response contain hallucinations?"
},
"code_template": null,
"last_mile_config": null,
"organization_id": "org_xyz789",
"created_at": "2024-01-10T09:00:00Z",
"updated_at": "2024-01-15T14:30:00Z"
},
{
"id": "eval_def456",
"name": "Relevance Score",
"scoring_type": "LLM-RANGE",
"llm_template": {
"model": "gpt-4",
"prompt": "Rate the relevance from 1-10",
"min": 1,
"max": 10
},
"code_template": null,
"last_mile_config": null,
"organization_id": "org_xyz789",
"created_at": "2024-01-12T11:00:00Z",
"updated_at": "2024-01-12T11:00:00Z"
}
],
"error": null
}
Evaluator Types
LLM-BOOLEAN
Uses an LLM to return a true/false score
LLM-CHOICE
Uses an LLM to select from predefined choices
LLM-RANGE
Uses an LLM to score within a numeric range
PYTHON
Executes custom Python code for evaluation
LASTMILE
Integration with LastMile evaluation framework