Skip to main content
The Circuit Breaker Labs CLI can be configured through environment variables, command-line flags, and provider-specific options.

​
Environment Variables

​
Required Variables

​
CBL_API_KEY

Your Circuit Breaker Labs API key for authenticating with the evaluation service. 
export CBL_API_KEY="cbl_your_api_key_here"
Get your API key by contacting [email protected]

​
Optional Variables

​
CBL_API_BASE_URL

Custom base URL for the Circuit Breaker Labs API. Defaults to wss://api.circuitbreakerlabs.ai/v1. 
export CBL_API_BASE_URL="wss://custom-endpoint.example.com/v1"

​
OPENAI_API_KEY

Required when using the OpenAI provider. 
export OPENAI_API_KEY="sk-your_openai_key_here"

​
OPENAI_BASE_URL

Custom base URL for OpenAI-compatible endpoints. Defaults to https://api.openai.com/v1. 
export OPENAI_BASE_URL="https://api.openai.com/v1"

​
OPENAI_ORG_ID

Optional OpenAI organization ID. 
export OPENAI_ORG_ID="org-your_org_id"

​
OLLAMA_BASE_URL

Base URL for Ollama server. Defaults to http://localhost:11434. 
export OLLAMA_BASE_URL="http://localhost:11434"

​
Global Options

These flags can be used with any evaluation command and must be specified before the evaluation type.

​
—log-level

Set the logging verbosity level. 
cbl --log-level debug single-turn ...
Values: error, warn, info (default), debug, trace

​
—log-mode

Enable log mode to disable the TUI and output logs to stdout instead. Useful for CI/CD pipelines and debugging. 
cbl --log-mode single-turn ...
In log mode, the interactive TUI is disabled. Progress will be logged as text output instead.

​
—output-file

Specify a custom output file path for evaluation results. By default, results are saved to auto-generated files with timestamps:
  • Single-turn: circuit_breaker_labs_single_turn_evaluation_YYYYMMDD_HHMMSS.json
  • Multi-turn: circuit_breaker_labs_multi_turn_evaluation_YYYYMMDD_HHMMSS.json
cbl --output-file results/my-evaluation.json single-turn ...

​
—add-header

Add custom HTTP headers to provider requests. Can be specified multiple times for multiple headers. 
cbl --add-header "X-Custom-Header:value1" --add-header "Authorization:Bearer token" single-turn ...
Format: "Key:Value"

​
Provider Configuration

​
OpenAI Provider

​
Required Options

cbl single-turn openai \
  --api-key "sk-..." \        # Or use OPENAI_API_KEY env var
  --model "gpt-4o"

​
Optional Parameters

—temperature <float>
Sampling temperature between 0 and 2. Higher values make output more creative.
--temperature 0.7
—top-p <float>
Nucleus sampling parameter. Alternative to temperature.
--top-p 0.9
—frequency-penalty <float>
Number between -2.0 and 2.0. Positive values penalize repeated tokens based on frequency.
--frequency-penalty 0.5
—presence-penalty <float>
Number between -2.0 and 2.0. Positive values penalize repeated tokens based on presence.
--presence-penalty 0.5
—max-completion-tokens <integer>
Maximum number of tokens to generate in the completion.
--max-completion-tokens 1000
—stop <string,string,...>
Up to 4 sequences where the API will stop generating tokens.
--stop "END,STOP"
—logit-bias <token_id:bias,...>
Modify likelihood of specified tokens appearing. Bias values between -100 and 100.
--logit-bias "50256:-100,13:5"
—n <integer>
Number of chat completion choices to generate for each input.
--n 3
—logprobs <bool>
Return log probabilities of output tokens.
--logprobs true
—top-logprobs <integer>
Number between 0 and 20 specifying most likely tokens to return.
--top-logprobs 5
—service-tier <auto|default|flex|scale|priority>
Specifies the processing tier for the request.
--service-tier scale
—reasoning-effort <none|minimal|low|medium|high|xhigh>
Constrains effort on reasoning for reasoning models.
--reasoning-effort high
—store <bool>
Whether to store the output of this chat completion request.
--store true

​
Ollama Provider

​
Required Options

cbl single-turn ollama \
  --model "llama2"

​
Optional Parameters

—temperature <float>
Model temperature. Higher values = more creative (default: 0.8).
--temperature 0.7
—top-k <integer>
Reduces probability of generating nonsense. Higher = more diverse (default: 40).
--top-k 50
—top-p <float>
Works with top-k. Higher values = more diverse text (default: 0.9).
--top-p 0.95
—num-predict <integer>
Maximum tokens to predict (default: 128, -1 = infinite, -2 = fill context).
--num-predict 256
—num-ctx <integer>
Size of the context window (default: 2048).
--num-ctx 4096
—mirostat <0|1|2>
Enable Mirostat sampling (0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0).
--mirostat 2
—mirostat-eta <float>
Mirostat learning rate (default: 0.1).
--mirostat-eta 0.15
—mirostat-tau <float>
Controls balance between coherence and diversity (default: 5.0).
--mirostat-tau 6.0
—repeat-penalty <float>
How strongly to penalize repetitions (default: 1.1).
--repeat-penalty 1.2
—repeat-last-n <integer>
How far back to look to prevent repetition (default: 64, 0 = disabled, -1 = num_ctx).
--repeat-last-n 128
—tfs-z <float>
Tail free sampling - reduces impact of less probable tokens (default: 1).
--tfs-z 0.95
—num-gpu <integer>
Number of layers to send to GPU(s).
--num-gpu 32
—num-thread <integer>
Number of threads to use during computation.
--num-thread 8
—num-gqa <integer>
Number of GQA groups in transformer layer.
--num-gqa 8
—seed <integer>
Random number seed for generation (default: 0).
--seed 42
—stop <string>
Stop sequences (can be specified multiple times).
--stop "END"
—logprobs <bool>
Return log probabilities for each token.
--logprobs true

​
Custom Provider

For APIs that aren’t OpenAI-compatible, use Rhai scripts to translate between schemas. 
cbl single-turn custom \
  --url "https://api.example.com/v1/chat" \
  --script ./my-provider.rhai
—url <string> (required)
Endpoint URL to POST requests to. —script <path> (required)
Path to the Rhai script file that handles request/response translation.
See the examples/providers/ directory for example Rhai scripts.

​
Complete Example

export CBL_API_KEY="cbl_your_key"
export OPENAI_API_KEY="sk-your_key"

cbl \
  --log-level info \
  --output-file evaluations/test-run.json \
  --add-header "X-Request-ID:test-123" \
  single-turn \
  --threshold 0.5 \
  --variations 3 \
  --maximum-iteration-layers 2 \
  --test-case-groups suicidal_ideation \
  openai \
  --model gpt-4o \
  --temperature 0.7 \
  --max-completion-tokens 500

Build docs developers (and LLMs) love

Get started for freeTalk to us