Skip to main content
The Circuit Breaker Labs GitHub Actions interact directly with the Circuit Breaker Labs API to evaluate AI system safety. Each action maps to a specific API endpoint for different evaluation scenarios.

​
Base URL

All API requests are made to: 
https://api.circuitbreakerlabs.ai/v1/

​
API Documentation

Complete API documentation, including interactive endpoint testing, is available at: https://api.circuitbreakerlabs.ai/v1/docs

​
Action to Endpoint Mapping

Each GitHub Action in this repository corresponds to a specific API endpoint:

​
Single-Turn System Prompt Evaluation

Action: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt Endpoint: POST /singleturn_evaluate_system_prompt API Documentation: View endpoint Evaluates a system prompt using single-turn adversarial testing against various attack vectors. Request Parameters:
  • threshold (float): Score threshold where an individual test case is considered failed
  • variations (int): Number of test variations to generate
  • maximum_iteration_layers (int): Maximum depth of iterative attack refinement
  • system_prompt (string): The system prompt text to evaluate
  • openrouter_model_name (string): Model to test via OpenRouter
  • test_case_groups (array, optional): Specific test case categories to run
Headers:
  • cbl-api-key: Your Circuit Breaker Labs API key
  • Content-Type: application/json

​
Single-Turn OpenAI Fine-Tune Evaluation

Action: circuitbreakerlabs/actions/singleturn-evaluate-openai-finetune Endpoint: POST /singleturn_evaluate_openai_finetune API Documentation: View endpoint Evaluates an OpenAI fine-tuned model using single-turn adversarial testing. Request Parameters:
  • threshold (float): Score threshold where an individual test case is considered failed
  • variations (int): Number of test variations to generate
  • maximum_iteration_layers (int): Maximum depth of iterative attack refinement
  • model_name (string): Fully qualified OpenAI fine-tuned model name (e.g., ft:gpt-4o-mini-2024-07-18:org:model:id)
  • openai_api_key (string): Your OpenAI API key for accessing the fine-tuned model
  • test_case_groups (array, optional): Specific test case categories to run
Headers:
  • cbl-api-key: Your Circuit Breaker Labs API key
  • Content-Type: application/json

​
Multi-Turn System Prompt Evaluation

Action: circuitbreakerlabs/actions/multiturn-evaluate-system-prompt Endpoint: POST /multiturn_evaluate_system_prompt API Documentation: View endpoint Evaluates a system prompt using multi-turn conversational attacks that attempt to manipulate the model over multiple interactions. Request Parameters:
  • threshold (float): Score threshold where an individual test case is considered failed
  • max_turns (int): Maximum conversation length in turns (must be even)
  • test_types (array): Types of multi-turn attacks to execute
  • system_prompt (string): The system prompt text to evaluate
  • openrouter_model_name (string): Model to test via OpenRouter
  • test_case_groups (array, optional): Specific test case categories to run
Headers:
  • cbl-api-key: Your Circuit Breaker Labs API key
  • Content-Type: application/json

​
Multi-Turn OpenAI Fine-Tune Evaluation

Action: circuitbreakerlabs/actions/multiturn-evaluate-openai-finetune Endpoint: POST /multiturn_evaluate_openai_finetune API Documentation: View endpoint Evaluates an OpenAI fine-tuned model using multi-turn conversational attacks. Request Parameters:
  • threshold (float): Score threshold where an individual test case is considered failed
  • max_turns (int): Maximum conversation length in turns (must be even)
  • test_types (array): Types of multi-turn attacks to execute
  • model_name (string): Fully qualified OpenAI fine-tuned model name
  • openai_api_key (string): Your OpenAI API key for accessing the fine-tuned model
  • test_case_groups (array, optional): Specific test case categories to run
Headers:
  • cbl-api-key: Your Circuit Breaker Labs API key
  • Content-Type: application/json

​
Example: Direct API Call

Here’s how the singleturn-evaluate-system-prompt action translates to a direct API request: GitHub Action: 
- name: Run system prompt evaluation
  uses: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt@v1
  with:
    fail-action-threshold: "0.80"
    fail-case-threshold: "0.5"
    variations: "1"
    maximum-iteration-layers: "1"
    system-prompt: "You are a helpful assistant"
    openrouter-model-name: "anthropic/claude-3.7-sonnet"
    circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}
Equivalent cURL Request: 
curl -X 'POST' \
  'https://api.circuitbreakerlabs.ai/v1/singleturn_evaluate_system_prompt' \
  -H 'accept: application/json' \
  -H "cbl-api-key: $CBL_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
  "threshold": 0.5,
  "variations": 1,
  "maximum_iteration_layers": 1,
  "openrouter_model_name": "anthropic/claude-3.7-sonnet",
  "system_prompt": "You are a helpful assistant"
}'
The action’s fail-action-threshold parameter is not sent to the API—it’s used locally to determine whether the workflow should fail based on the overall failure rate returned by the API.

​
Response Format

All evaluation endpoints return a response containing:
  • total_passed (int): Number of test cases that passed
  • total_failed (int): Number of test cases that failed
  • failed_results (array): Detailed information about each failed test case
Single-Turn Failed Result: 
{
  "safe_response_score": 0.23,
  "user_input": "[adversarial input]",
  "model_response": "[unsafe model response]"
}
Multi-Turn Failed Result: 
{
  "min_safe_response_score": 0.15,
  "conversation": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."}
  ]
}

​
Authentication

All API requests require a Circuit Breaker Labs API key passed via the cbl-api-key header.
Store your API key as a GitHub Actions secret. Never commit API keys to your repository.
circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}

​
Rate Limits and Quotas

API rate limits and quotas depend on your Circuit Breaker Labs subscription plan. Contact [email protected] for details about your account limits.

​
Python Client Library

These actions use the official Circuit Breaker Labs Python client (version 1.0.7) to interact with the API. You can use this client directly in your own Python applications: 
pip install circuit-breaker-labs==1.0.7

from circuit_breaker_labs.api.evaluations import singleturn_evaluate_system_prompt_post
from circuit_breaker_labs.client import Client
from circuit_breaker_labs.models.single_turn_evaluate_system_prompt_request import (
    SingleTurnEvaluateSystemPromptRequest,
)

client = Client("https://api.circuitbreakerlabs.ai/v1/")

request = SingleTurnEvaluateSystemPromptRequest(
    threshold=0.5,
    variations=1,
    maximum_iteration_layers=1,
    system_prompt="You are a helpful assistant",
    openrouter_model_name="anthropic/claude-3.7-sonnet",
)

response = singleturn_evaluate_system_prompt_post.sync_detailed(
    client=client,
    body=request,
    cbl_api_key="your-api-key",
)

if response.status_code == 200:
    result = response.parsed
    print(f"Passed: {result.total_passed}, Failed: {result.total_failed}")

​
Additional Resources

Build docs developers (and LLMs) love

Get started for freeTalk to us