Overview
The Codenames AI Benchmark supports custom board configurations through theGameConfig class. Modify board size, word distributions, and game rules to test agents under different conditions.
Game Configuration
All game settings are defined inconfig.py:
config.py
Standard Configuration
The default 25-word board matches official Codenames rules:Creating Custom Variants
Small Board (9 words)
Faster games for rapid testing:Large Board (49 words)
More complex, longer games:Multiple Bombs
Increased difficulty and risk:Balanced Board
Equal words for both teams (no first-mover advantage):Using the Custom Config Helper
TheGameConfig.custom() method automatically calculates word distributions:
config.py
Example Usage
Running Games with Custom Config
Use custom configurations with the orchestrator:run_custom_game.py
Configuration with Config Class
Use the unifiedConfig class for complete configuration:
config.py
Example
Validation Rules
Thevalidate() method enforces configuration constraints:
config.py
Advanced Configuration Options
Limit Guesses Per Turn
Adjust Hint Count Limits
Red Team Starts First
Benchmark Suite with Variants
Test agents across multiple board configurations:benchmark_variants.py
Board Configuration Best Practices
Test Variants
Run agents on multiple board sizes to ensure robustness
Odd Board Sizes
Use odd numbers for fair play (starting team gets +1 word)
Scale MAX_TURNS
Increase turn limits proportionally with board size
Validate Early
Call
validate() immediately after creating custom configsCommon Variant Use Cases
- Rapid Testing
- Standard Benchmarks
- Stress Testing
- High Stakes
Small Board (9-15 words)
- Fast iteration during development
- Quick agent validation
- Reduced API costs
- Shorter runtime for CI/CD
Next Steps
Custom Agents
Build agents that adapt to different board sizes
Analysis Metrics
Compare agent performance across board variants