Skip to main content
The Environments Hub is a platform for publishing, discovering, and using Verifiers environments. It serves as a central registry for environments that can be used for evaluation and training.

​
Features

  • Discover environments - Browse and search public environments
  • Publish your own - Share environments with the community
  • Install from Hub - Use Hub environments in your projects with a single command
  • Version control - Track environment versions and updates
  • Automatic CI - Validate environments on push
  • Usage analytics - See how many people are using your environment

​
Installing Hub Environments

Install any environment from the Hub: 
prime env install primeintellect/math-python
This installs the environment into your project, making it available for evaluation and training.

​
Running Hub Environments

Evaluate without installing: 
prime eval run primeintellect/math-python -m openai/gpt-4.1-mini -n 10
Train with Hosted Training: 
[[env]]
id = "primeintellect/math-python"
args = { max_turns = 10 }

​
Publishing to the Hub

1

Prepare your environment

Create an environment following the standard structure:
prime env init my-env
Ensure your environment has:
  • A load_environment() function
  • A pyproject.toml with metadata and dependencies
  • A README.md with documentation
2

Test locally

Validate your environment works:
prime env install my-env
prime eval run my-env -m openai/gpt-4.1-mini -n 5
Check that:
  • Environment loads without errors
  • Rollouts complete successfully
  • Rewards are computed correctly
3

Push to Hub

Publish your environment:
prime env push my-env
This uploads your environment to the Hub under your username or organization.
4

Set visibility

By default, environments are private. To make your environment public:
  1. Go to app.primeintellect.ai/dashboard/environments
  2. Click on your environment
  3. Go to Settings > Visibility
  4. Set to “Public”

​
Environment Metadata

The Hub reads metadata from your pyproject.toml: 
[project]
name = "my-env"
description = "A short description of what this environment does"
tags = ["reasoning", "multi-turn", "tools", "train", "eval"]
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "verifiers>=0.1.8",
    "some-other-package>=1.0.0",
]
Key fields:
  • name - Environment ID (used in prime env install)
  • description - Shown in Hub search results
  • tags - Categorize your environment for discoverability
  • version - Semantic version for your environment
  • dependencies - Packages required to run your environment
Task type:
  • single-turn - Single response tasks
  • multi-turn - Multi-step interactions
  • tools - Environments with tool use
  • code - Programming tasks
  • math - Mathematical reasoning
  • reasoning - General reasoning tasks
  • qa - Question answering
Use case:
  • train - Suitable for RL training
  • eval - Suitable for evaluation
  • benchmark - Standardized benchmark
Data type:
  • synthetic - Procedurally generated
  • human - Human-annotated
  • curated - Hand-selected examples

​
README Documentation

Include a comprehensive README with:

​
Overview Section

### Overview
- **Environment ID**: `my-env`
- **Short description**: One sentence describing the task
- **Tags**: List relevant tags

​
Datasets Section

### Datasets
- **Primary dataset(s)**: GSM8K, MATH, etc.
- **Source links**: Links to original datasets
- **Split sizes**: 1000 train / 200 eval

​
Task Section

### Task
- **Type**: single-turn | multi-turn | tool use
- **Rubric overview**: How responses are scored

​
Quickstart Section

### Quickstart

Run an evaluation:
```bash
prime eval run my-env -m openai/gpt-4.1-mini -n 20
With custom arguments: 
prime eval run my-env -a '{"difficulty": "hard", "max_turns": 10}'


### Environment Arguments Section
```markdown
### Environment Arguments

| Arg | Type | Default | Description |
|-----|------|---------|-------------|
| `difficulty` | str | "medium" | Task difficulty level |
| `max_turns` | int | 10 | Maximum turns per rollout |

​
Metrics Section

### Metrics

| Metric | Meaning |
|--------|----------|
| `reward` | Main task score (0-1) |
| `format_reward` | XML format compliance |
| `num_turns` | Average turns per rollout |

​
Required API Keys

If your environment requires API keys, document them clearly: 
### Required Environment Variables

- `OPENAI_API_KEY` - For judge model
- `BROWSERBASE_API_KEY` - For browser automation

Set these in:
- **Local**: `export OPENAI_API_KEY=...`
- **Hosted Training**: Add to environment variables in dashboard
- **Hub CI**: Add secrets in Settings > Environment Variables
In your code, validate keys early: 
import verifiers as vf

def load_environment():
    vf.ensure_keys(["OPENAI_API_KEY", "BROWSERBASE_API_KEY"])
    # ... rest of environment setup

​
Continuous Integration

The Hub automatically runs CI on your environment when you push:
  1. Linting - Checks code style with ruff
  2. Loading - Validates load_environment() works
  3. Evaluation - Runs a small evaluation to ensure rollouts complete
CI results are shown in your environment’s Hub page.

​
Configuring CI

Customize CI behavior in pyproject.toml: 
[tool.verifiers.ci]
test_model = "openai/gpt-4.1-mini"
test_examples = 5
test_rollouts = 2
test_args = { difficulty = "easy" }

​
Versioning

Update your environment version in pyproject.toml: 
version = "0.2.0"
Then push: 
prime env push my-env
Users can install specific versions: 
prime env install my-env==0.1.0
prime env install my-env  # latest version

​
Private Environments

By default, environments are private. Private environments:
  • Only visible to you and your organization
  • Can be used in your own training/evaluation
  • Don’t appear in Hub search
To share privately, invite collaborators via Settings > Access.

​
Discovering Environments

Browse environments at app.primeintellect.ai/dashboard/environments. Filter by:
  • Tags (single-turn, multi-turn, tools, etc.)
  • Author (username or organization)
  • Stars (most popular)
  • Recent (newest first)
Search by:
  • Environment name
  • Description keywords
  • Tags

​
Best Practices

Good documentation is key to adoption. Include:
  • Clear description of the task
  • Example inputs and expected outputs
  • Explanation of rubric/scoring
  • Any special requirements or gotchas

​
Environment Design

  • Start simple - Begin with a minimal working environment
  • Test thoroughly - Run multiple evaluations before publishing
  • Document well - Explain the task, rubric, and any quirks
  • Use standard patterns - Follow conventions from existing environments
  • Minimize dependencies - Only include essential packages

​
README Quality

  • Keep it concise but complete
  • Include copy-pasteable code examples
  • Document all arguments with types and defaults
  • Explain metrics and what they measure
  • Link to relevant papers or external resources

​
Maintenance

  • Respond to issues and questions
  • Update documentation as you learn from users
  • Fix bugs promptly
  • Consider feedback for new features
  • Keep dependencies up to date

​
Examples

Popular environments on the Hub: See these for examples of good documentation and structure.

Build docs developers (and LLMs) love

Get started for freeTalk to us