Skip to main content
All Circuit Breaker Labs actions share a common set of core parameters, with some action-specific inputs depending on whether you’re evaluating a system prompt or fine-tuned model.

​
Common Parameters

These parameters are available across all Circuit Breaker Labs actions.

​
Threshold Configuration

​
fail-action-threshold
number
required
The failure rate threshold that determines whether the GitHub Action workflow fails. If the test failure rate exceeds this value, the action exits with a non-zero status code.Value range: 0.0 to 1.0 (representing 0% to 100%)Example: 0.80 means the action fails if more than 80% of test cases fail.See Thresholds for detailed examples.
​
fail-case-threshold
number
required
The safety score threshold for determining whether an individual test case passes or fails. Test cases with safety scores below this threshold are considered failures.Value range: 0.0 to 1.0Example: 0.5 means any test case with a safety score below 0.5 is marked as failed.See Thresholds for detailed examples.

​
Authentication

​
circuit-breaker-labs-api-key
string
required
Your Circuit Breaker Labs API key for authenticating requests to the evaluation API.
Always store this as a GitHub secret. Never hardcode API keys in your workflow files.
circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}

​
Test Filtering

​
test-case-groups
string
Optional space-separated list of test case groups to run. When omitted, all available test groups are executed.This parameter allows you to target specific categories of security tests, enabling focused evaluation during development or CI/CD.See Test Case Groups for available groups and usage examples.

​
Single-Turn Evaluation Parameters

These parameters apply to singleturn-evaluate-system-prompt and singleturn-evaluate-openai-finetune actions.

​
Evaluation Configuration

​
variations
integer
required
Number of test variations to generate and run for each test case. Higher values provide more comprehensive coverage but increase evaluation time and API usage.Minimum: 1Example: 3 generates three variations of each test case.
Start with 1 variation during development, then increase to 3-5 for production evaluations.
​
maximum-iteration-layers
integer
required
Maximum number of iteration layers for adversarial testing. Each layer represents a level of increasingly sophisticated attack attempts.Minimum: 1Example: 2 runs tests across two iteration layers.
Higher iteration layers test your model’s resilience against more advanced jailbreak attempts.

​
Model Configuration (System Prompt)

​
system-prompt
string
required
The system prompt text to evaluate. This is the instruction text that guides your model’s behavior.Available in: singleturn-evaluate-system-prompt, multiturn-evaluate-system-prompt
system-prompt: "You are a helpful AI assistant. Always prioritize user safety and refuse harmful requests."
For multi-line prompts, use YAML multiline syntax or read from a file using GitHub Actions expressions.
​
openrouter-model-name
string
required
The OpenRouter model identifier to use for evaluation. Must be a valid model name supported by OpenRouter.Available in: singleturn-evaluate-system-prompt, multiturn-evaluate-system-promptExamples:
  • anthropic/claude-3.7-sonnet
  • openai/gpt-4
  • meta-llama/llama-3.1-70b-instruct
See OpenRouter models for the full list.

​
Model Configuration (Fine-Tune)

​
model-name
string
required
The fully qualified name of your OpenAI fine-tuned model to evaluate.Available in: singleturn-evaluate-openai-finetune, multiturn-evaluate-openai-finetuneFormat: ft:gpt-4-0125-preview:organization:model-name:identifier
model-name: "ft:gpt-4-0125-preview:acme-corp:customer-support:abc123"
​
openai-api-key
string
required
Your OpenAI API key for accessing fine-tuned models.Available in: singleturn-evaluate-openai-finetune, multiturn-evaluate-openai-finetune
Store this as a GitHub secret alongside your Circuit Breaker Labs API key.
openai-api-key: ${{ secrets.OPENAI_API_KEY }}

​
Multi-Turn Evaluation Parameters

These parameters apply to multiturn-evaluate-system-prompt and multiturn-evaluate-openai-finetune actions.
​
max-turns
integer
required
Maximum number of conversation turns to simulate in multi-turn evaluations. Must be an even number.Constraint: Must be an even integerExample: 4 simulates conversations with up to 4 turns (2 user messages, 2 assistant responses)
Multi-turn evaluations test whether adversarial prompts can exploit your model across extended conversations.
​
test-types
string
required
Space-separated list of multi-turn test types to execute. Each test type represents a different attack strategy.Format: Multiple values separated by spaces
test-types: "crescendo context_switching"
Refer to the Circuit Breaker Labs API documentation for available multi-turn test types.

​
Complete Examples

​
Single-Turn System Prompt Evaluation

- name: Evaluate system prompt
  uses: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt@v1
  with:
    fail-action-threshold: "0.80"
    fail-case-threshold: "0.5"
    variations: "3"
    maximum-iteration-layers: "2"
    system-prompt: "You are a helpful assistant"
    openrouter-model-name: "anthropic/claude-3.7-sonnet"
    circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}
    test-case-groups: "prompt_injection data_exfiltration"

​
Multi-Turn Fine-Tune Evaluation

- name: Evaluate fine-tuned model
  uses: circuitbreakerlabs/actions/multiturn-evaluate-openai-finetune@v1
  with:
    fail-action-threshold: "0.90"
    fail-case-threshold: "0.6"
    max-turns: "4"
    test-types: "crescendo context_switching"
    model-name: "ft:gpt-4-0125-preview:org:model:id"
    circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}
    openai-api-key: ${{ secrets.OPENAI_API_KEY }}
    test-case-groups: "jailbreak"

​
Parameter Validation

The actions perform validation on input parameters:
  • Numeric ranges: Threshold values must be between 0.0 and 1.0
  • Integer constraints: max-turns must be an even number
  • Required combinations: Fine-tune actions require both model-name and openai-api-key
  • Format validation: test-types and test-case-groups accept space-separated values
Invalid parameter values will cause the action to fail during initialization, before any API calls are made.

Build docs developers (and LLMs) love

Get started for freeTalk to us