Overview
RAPTOR includes a comprehensive test suite to validate all commands, workflows, and user scenarios. Tests cover command structure, security analysis modes, package architecture, and real vulnerability detection.Tests use real vulnerable code samples to ensure RAPTOR’s detection capabilities work in practice.
Quick Start
Expected duration: 2-3 minutes for full suite
Test Structure
RAPTOR’s test suite is organized into multiple categories:Test Categories
1. Command Structure Tests
1. Command Structure Tests
Validates that all RAPTOR commands are properly registered and accessible.Tests:
- Main launcher recognizes all modes (scan, fuzz, web, agentic, codeql)
- Core execution scripts exist (raptor_agentic.py, raptor_fuzzing.py, etc.)
- Main scripts have valid Python syntax
- Help information is accessible
2. Scan Mode Tests
2. Scan Mode Tests
Validates static analysis capabilities.Tests:
- Scan mode help available
- Scan requires —repo argument
- Scan supports policy groups
3. Agentic Mode Tests
3. Agentic Mode Tests
Validates autonomous workflow capabilities.Tests:
- Agentic script exists
- Agentic accepts arguments
- CodeQL integration available
- Exploit/patch generation controls present
4. Fuzzing Mode Tests
4. Fuzzing Mode Tests
Validates binary fuzzing capabilities.Tests:
- Fuzzing script exists
- Requires —binary argument
- Supports —duration option
- Supports —parallel option
- Supports autonomous mode
5. CodeQL Mode Tests
5. CodeQL Mode Tests
Validates deep semantic analysis.Tests:
- CodeQL script exists
- CodeQL accepts arguments
- Language detection supported
6. Web Mode Tests
6. Web Mode Tests
Validates web application security testing (alpha).Tests:
- Web mode script exists
- Web mode listed in help
7. Package Architecture Tests
7. Package Architecture Tests
Validates internal package structure.Tests:
- Core modules directory exists
- Packages directory exists
- LLM analysis package exists
- Static analysis package exists
- Fuzzing utilities exist
8. Test Fixture Tests
8. Test Fixture Tests
Validates that test data samples are present and contain real vulnerabilities.Tests:
- Test data directory exists
- Sample Python vulnerable code present
- Sample JavaScript vulnerable code present
- Code samples contain vulnerability patterns
9. Workflow Availability Tests
9. Workflow Availability Tests
Validates that all user-facing workflows are accessible.Tests:
- scan —help accessible
- agentic —help accessible
- fuzz —help accessible
- codeql —help accessible
Test Data Examples
RAPTOR includes realistic vulnerable code samples for testing:Python SQL Injection
- SQL injection (string concatenation)
- Hardcoded credentials
- Weak cryptography (MD5)
- Command injection (shell=True)
- Path traversal (unsanitized file path)
JavaScript XSS
- DOM-based XSS (innerHTML)
- Reflected XSS (document.write)
- Arbitrary code execution (eval)
- Hardcoded secrets (API keys)
- Insecure storage (localStorage)
- Weak random generation (Math.random)
- Prototype pollution
- ReDoS (inefficient regex)
Running Tests
Comprehensive Test Suite
Run all tests with detailed output:Test Result Indicators
- ✓ PASS - Test succeeded
- ✗ FAIL - Test failed (with error message)
- ⊘ SKIP - Test skipped (with reason)
Test Workflows (Claude Code)
RAPTOR includes a/test-workflows command for Claude Code:
- Basic scan (findings only, no exploits)
- Full agentic workflow (scan + exploit + patch)
- Binary fuzzing
- Manual crash validation
- Tool routing sanity checks
Integration Tests
Integration tests verify RAPTOR works with external tools:- Semgrep installed
- CodeQL installed (optional)
- AFL++ installed (optional)
- Test data present
- Semgrep scanning works
- CodeQL database creation (if available)
- AFL++ fuzzing setup (if available)
- Python package imports
User Stories
Tests are designed around real user scenarios:User Story 1: Quick Scan
As a developer, I want to quickly scan my code for vulnerabilities.Tested by: Scan mode tests, fixture tests
User Story 2: Autonomous Analysis
As a security researcher, I want RAPTOR to autonomously analyze, generate exploits, and propose patches.Tested by: Agentic mode tests, workflow tests
User Story 3: Binary Fuzzing
As a bug hunter, I want to fuzz a binary and analyze crashes.Tested by: Fuzzing mode tests, integration tests
User Story 4: Deep Analysis
As an analyst, I want to use CodeQL for deep semantic analysis.Tested by: CodeQL mode tests, integration tests
Writing New Tests
To add tests to the suite:Choose Test Category
Determine which test file to modify:
comprehensive_test.sh- Core functionalityintegration_tests.sh- External tool integrationtest_workflows.sh- End-to-end workflows
Continuous Integration
RAPTOR uses GitHub Actions for CI. Tests run automatically on pull requests.
- Install dependencies
- Run syntax validation
- Run comprehensive test suite
- Run integration tests (if tools available)
- Report results
Troubleshooting Tests
Tests fail with 'command not found'
Tests fail with 'command not found'
Problem: Required tools not installed.Solution:
- Use the devcontainer (all tools pre-installed)
- Or install missing tools:
Tests skip with 'May require valid parameters'
Tests skip with 'May require valid parameters'
Problem: Test requires actual execution, skipped for safety.Solution: This is normal. Skipped tests indicate optional features or safety guards.
Integration tests fail
Integration tests fail
Problem: External tools not working.Solution:
- Verify tool installation:
- Check test/data directory exists
- Ensure PYTHONPATH is set correctly
Fixture tests fail with 'vulnerability patterns not found'
Fixture tests fail with 'vulnerability patterns not found'
Problem: Test data files modified or missing.Solution:
- Restore test/data files from git:
- Ensure files contain documented vulnerabilities
Test Coverage
RAPTOR’s test suite covers:Command Coverage
All major commands:
- /scan
- /agentic
- /fuzz
- /codeql
- /web
- /analyze
Vulnerability Coverage
Multiple vulnerability types:
- SQL injection
- XSS (DOM, reflected)
- Command injection
- Path traversal
- Secrets exposure
- Weak cryptography
Language Coverage
Multiple languages:
- Python
- JavaScript
- C/C++ (via binaries)
Package Coverage
All packages:
- llm_analysis
- static-analysis
- fuzzing
- codeql
- web
Next Steps
DevContainer
Set up pre-configured environment
Contributing
Add your own tests
Quick Start
Start using RAPTOR
FAQ
Common questions