Changelog

​

Version History

​

Latest (HEAD)

• Updated to Circuit Breaker Labs Python client v1.0.7
• Enhanced API compatibility with latest endpoint specifications
• Improved error handling and response parsing

​

v1.0.2

• Multi-turn evaluation support - Added two new actions for evaluating conversational safety:
  • multiturn-evaluate-system-prompt - Test system prompts with multi-turn conversations
  • multiturn-evaluate-openai-finetune - Test fine-tuned models with multi-turn conversations
• Multi-turn evaluations detect sophisticated attacks that unfold over multiple conversation turns
• Support for configurable max-turns parameter (must be even)
• Support for test-types parameter to specify attack strategies

• Enhanced output formatting for multi-turn conversation display
• Added conversation turn numbering for easier debugging
• Improved safety score calculation across conversation history

• Multi-Turn System Prompt Action
• Multi-Turn OpenAI Fine-Tune Action

# New multi-turn action
- name: Run multi-turn evaluation
  uses: circuitbreakerlabs/actions/[email protected]
  with:
    fail-action-threshold: "0.80"
    fail-case-threshold: "0.5"
    max-turns: "4"
    test-types: "jailbreak context_manipulation"
    system-prompt: ${{ env.SYSTEM_PROMPT }}
    openrouter-model-name: "anthropic/claude-3.7-sonnet"
    circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}

​

v1.0.1

• Code formatting standardization across all action implementations
• Improved YAML structure and consistency in action.yml files
• Enhanced code comments and documentation strings
• Standardized naming conventions throughout the codebase

• Applied Ruff formatting to all Python source files
• Updated type hints for better static analysis
• Improved error message clarity

​

v1.0.0

• System Prompt Evaluation - singleturn-evaluate-system-prompt action
  • Test system prompts against adversarial attacks
  • Configurable safety thresholds
  • Iterative attack refinement with multiple layers
  • Support for OpenRouter models
  • Optional test case group filtering
• OpenAI Fine-Tune Evaluation - singleturn-evaluate-openai-finetune action
  • Test OpenAI fine-tuned models
  • Direct integration with OpenAI API
  • Same threshold and variation controls as system prompt evaluation
  • Support for all OpenAI fine-tuning base models

• Automated safety evaluation in CI/CD pipelines
• Configurable failure thresholds at both case and action levels
• Detailed failure output with adversarial inputs and model responses
• GitHub Actions secret integration for API keys
• Zero-configuration setup using composite actions
• Automatic dependency management via uv

• Direct mapping to Circuit Breaker Labs API v1 endpoints
• Authentication via cbl-api-key header
• Support for custom test case groups
• Comprehensive error handling and reporting

name: Evaluate System Prompt

on:
  push:
    branches: [main]
  pull_request:

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      
      - name: Run safety evaluation
        uses: circuitbreakerlabs/actions/[email protected]
        with:
          fail-action-threshold: "0.80"
          fail-case-threshold: "0.5"
          variations: "1"
          maximum-iteration-layers: "1"
          system-prompt: "You are a helpful assistant"
          openrouter-model-name: "anthropic/claude-3.7-sonnet"
          circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}

• Python 3.13+
• circuit-breaker-labs Python client v1.0.7
• uv package manager
• astral-sh/setup-uv action

​

Pre-Release Development

​

December 5, 2025

• Initial documentation improvements
• Added README badges for zizmor, Ruff, and MyPy
• Improved usage examples in README
• Added Python client repository link

​

December 5, 2025

• Added CI/CD workflows for linting and type checking
• Integrated zizmor for GitHub Actions security scanning
• Added Ruff linter configuration
• Added MyPy for static type checking

​

November 26, 2025

• Initial repository setup
• Project structure and tooling configuration
• Base implementation of evaluation actions

​

Versioning Policy

• Major version (X.0.0): Breaking changes that require workflow updates
• Minor version (0.X.0): New features, backward compatible
• Patch version (0.0.X): Bug fixes, backward compatible

​

Recommended Version Pinning

# Recommended: Pin to major version (gets latest minor/patch updates)
uses: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt@v1

# More strict: Pin to specific version (no automatic updates)
uses: circuitbreakerlabs/actions/[email protected]

# Not recommended for production: Use main branch (unstable)
uses: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt@main

​

Upgrade Guides

​

Upgrading to v1.0.2 (Multi-Turn Support)

jobs:
  # Existing single-turn tests continue to work
  singleturn-test:
    runs-on: ubuntu-latest
    steps:
      - uses: circuitbreakerlabs/actions/singleturn-evaluate-system-prompt@v1
        with:
          # ... existing configuration ...

  # Add new multi-turn tests
  multiturn-test:
    runs-on: ubuntu-latest
    steps:
      - uses: circuitbreakerlabs/actions/multiturn-evaluate-system-prompt@v1
        with:
          fail-action-threshold: "0.80"
          fail-case-threshold: "0.5"
          max-turns: "4"
          test-types: "jailbreak context_manipulation"
          system-prompt: ${{ env.SYSTEM_PROMPT }}
          openrouter-model-name: "anthropic/claude-3.7-sonnet"
          circuit-breaker-labs-api-key: ${{ secrets.CBL_API_KEY }}

• max-turns: Maximum conversation length (must be even)
• test-types: Space-separated list of attack types

• variations - Not applicable to multi-turn tests
• maximum-iteration-layers - Not applicable to multi-turn tests

​

Upgrading to v1.0.0 (Initial Release)

​

Breaking Changes

​

Deprecation Notices

​

Python Client Dependency

• Current version: 1.0.7
• Repository: circuitbreakerlabs/circuitbreakerlabs-python
• PyPI: circuit-breaker-labs

​

Continuous Integration

• Linting: Ruff checks code formatting and style
• Type Checking: MyPy validates type annotations
• Security Scanning: zizmor analyzes GitHub Actions for security issues

• Linting Workflow
• Type Checking Workflow

​

Stay Updated

• GitHub Releases: Watch the repository for release notifications
• GitHub Discussions: Join conversations about upcoming features
• Email Updates: Contact [email protected] to subscribe to release announcements

​

Feedback and Contributions

• Report bugs: GitHub Issues
• Request features: GitHub Discussions
• Contribute code: Contributing Guidelines