Config Utilities
Utilities for validating required environment variables, particularly API keys.
Overview
The verifiers.utils.config_utils module provides:
- Environment variable validation
- Clear error messages for missing keys
- Guidance on setting secrets in different contexts
Functions
ensure_keys
def ensure_keys(keys: list[str]) -> None
List of environment variable names to check (e.g., ["OPENAI_API_KEY", "ANTHROPIC_API_KEY"]).
MissingKeyError if any keys are not set in the environment.
Example:
import verifiers as vf
def load_environment():
# Validate required API keys
vf.ensure_keys(["OPENAI_API_KEY"])
return vf.SingleTurnEnv(
dataset=dataset,
rubric=rubric,
)
Exceptions
MissingKeyError
class MissingKeyError(ValueError):
def __init__(self, keys: list[str])
List of missing environment variable names.
Missing required environment variable(s): OPENAI_API_KEY, ANTHROPIC_API_KEY
To set these variables:
- Environments Hub CI: Add secrets on the environment's Settings page
- Hosted Training: Set env_file in your config (e.g. env_file = ["secrets.env"])
- Local: Export in your shell (e.g. export OPENAI_API_KEY=...)
import verifiers as vf
try:
vf.ensure_keys(["OPENAI_API_KEY", "ANTHROPIC_API_KEY"])
except vf.MissingKeyError as e:
print(f"Missing {len(e.keys)} keys: {e.keys}")
# Missing 2 keys: ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']
raise
Example Usage
Basic Validation
import verifiers as vf
from datasets import load_dataset
def load_environment():
# Ensure OpenAI key is set
vf.ensure_keys(["OPENAI_API_KEY"])
dataset = load_dataset("gsm8k", "main", split="test")
return vf.SingleTurnEnv(
dataset=dataset,
rubric=vf.Rubric(lambda **kw: 1.0),
)
Multiple Keys
import verifiers as vf
def load_environment():
# Check multiple API keys
vf.ensure_keys([
"OPENAI_API_KEY",
"ANTHROPIC_API_KEY",
"BROWSERBASE_API_KEY",
])
# ... environment setup ...
Custom Services
import verifiers as vf
def load_environment():
# Validate custom service keys
vf.ensure_keys([
"GITHUB_TOKEN",
"SLACK_WEBHOOK_URL",
"DATABASE_URL",
])
# ... use these in your environment ...
Error Handling
import verifiers as vf
def load_environment():
try:
vf.ensure_keys(["OPENAI_API_KEY"])
except vf.MissingKeyError as e:
print(f"Missing keys: {', '.join(e.keys)}")
print("Please set environment variables before continuing.")
raise
# Safe to use API key here
return create_environment()
Setting Environment Variables
Local Development
# Export in shell
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
# Or use .env file with python-dotenv
pip install python-dotenv
# load_dotenv in your script
from dotenv import load_dotenv
load_dotenv()
import verifiers as vf
vf.ensure_keys(["OPENAI_API_KEY"]) # Now works
Environments Hub CI
- Go to your environment’s page on Environments Hub
- Click Settings
- Add secrets in the Environment Variables section
- Secrets are automatically injected during CI runs
Hosted Training
Create a secrets.env file:
# secrets.env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
# config.py
from verifiers.trainer import TrainConfig
config = TrainConfig(
env_id="my_env",
env_file=["secrets.env"], # Load secrets
# ... other config ...
)
Best Practices
1. Validate Early
Call ensure_keys() in load_environment() to fail fast:
def load_environment():
# Fail immediately if keys missing
vf.ensure_keys(["OPENAI_API_KEY"])
# Expensive operations only run if keys present
dataset = expensive_data_loading()
return create_env(dataset)
2. Document Required Keys
List required keys in your environment’s README:
# My Environment
## Required Environment Variables
- `OPENAI_API_KEY`: OpenAI API key for model inference
- `BROWSERBASE_API_KEY`: Browserbase API key for browser automation
## Setup
```bash
export OPENAI_API_KEY=sk-...
export BROWSERBASE_API_KEY=...
### 3. Validate Only What You Use
Don't validate keys you don't need:
```python
# Bad: Requires unnecessary keys
vf.ensure_keys(["OPENAI_API_KEY", "ANTHROPIC_API_KEY"])
# ... but only uses OpenAI
# Good: Only check what's needed
vf.ensure_keys(["OPENAI_API_KEY"])
4. Use Consistent Key Names
Follow common conventions:
*_API_KEY for API keys*_URL for endpoints*_TOKEN for auth tokens
vf.ensure_keys([
"OPENAI_API_KEY", # API key
"DATABASE_URL", # Connection string
"GITHUB_TOKEN", # Auth token
])
Common API Key Names
OpenAI API key (starts with sk-)
Anthropic API key (starts with sk-ant-)
Prime API key for inference
Browserbase API key for browser environments
GitHub personal access token
Error Message Customization
The default error message provides context-specific guidance. You can catch and customize:
import verifiers as vf
def load_environment():
try:
vf.ensure_keys(["CUSTOM_API_KEY"])
except vf.MissingKeyError as e:
print("\nCustom instructions:")
print("Get your API key from: https://custom-service.com/api-keys")
print("Then set: export CUSTOM_API_KEY=<your-key>\n")
raise
See Also