Skip to main content

​
Prerequisites

Before running tests, ensure you have:
  • Metlo CLI installed
  • Test files written in YAML format
  • Metlo backend configured (if using cloud features)

​
Basic Usage

​
Run a Single Test

Execute a specific test file: 
metlo test run path/to/test.yaml

​
Run Multiple Tests

Run multiple test files at once: 
metlo test run tests/auth.yaml tests/bola.yaml tests/sqli.yaml

​
Run Tests from a Directory

Run all tests in a directory: 
metlo test run tests/*.yaml

​
Command Options

​
Verbose Output

Get detailed information about test execution: 
metlo test run test.yaml --verbose
Verbose mode shows:
  • Full request and response details
  • Context variables at each step
  • Complete error stack traces

​
Environment Variables

Provide environment variables from a file: 
metlo test run test.yaml --envfile .env
Example .env file: 
API_URL=https://api.staging.example.com
AUTH_TOKEN=your-test-token
USER_ID=12345
Override specific environment variables: 
metlo test run test.yaml --env API_URL=https://api.prod.example.com

​
Filter by Endpoint

Run tests for specific endpoints (when using Metlo backend): 
metlo test run --endpoint /api/users --method GET --host api.example.com

​
Test Output

​
Successful Test

When all assertions pass: 
$ metlo test run auth-test.yaml
Running test at path "auth-test.yaml":
✓ Done running test...
All Tests Succeeded!

​
Failed Test

When assertions fail: 
$ metlo test run auth-test.yaml
Running test at path "auth-test.yaml":
✓ Done running test...
Some Tests Failed.
Request 2 Assertion 1 Failed:
{
    "type": "EQ",
    "key": "resp.status",
    "value": [401, 403]
}
Use the --verbose flag for more information.

​
Failed Request

When a request fails: 
Request 1 Failed With Error "Network Error":
{
    "url": "https://api.example.com/users",
    "method": "GET",
    "headers": {
        "Authorization": "Bearer token123"
    }
}

​
Understanding Test Results

1

Test Execution

Each test step is executed sequentially. The test runner:
  • Makes the HTTP request
  • Extracts variables from the response
  • Runs all assertions
2

Assertion Validation

For each assertion, the test runner:
  • Evaluates the assertion condition
  • Records pass/fail status
  • Continues to next assertion or step
3

Result Reporting

After all steps complete:
  • Success: All assertions passed
  • Failure: One or more assertions failed
  • Error: A request failed to execute

​
Running Auto-Generated Tests

​
Generate Tests

Generate a test from a template: 
metlo test generate \
  --test BROKEN_AUTHENTICATION \
  --host api.example.com \
  --endpoint /api/users/billing \
  --method POST \
  --path output-test.yaml
Available Templates:
  • BOLA - Broken Object Level Authorization
  • BROKEN_AUTHENTICATION - Authentication bypass testing
  • BOLA_ADMIN - Admin-specific BOLA testing
  • BOLA_MULTI_TENANT - Multi-tenant BOLA testing
  • SQLI_TIME - Time-based SQL injection
  • HSTS - HTTP Strict Transport Security validation
  • CSP - Content Security Policy validation
  • GENERIC - Generic security test

​
Run Auto-Generated Auth Tests

Automatically generate and run authentication tests: 
metlo test run-auth
This command:
  1. Fetches endpoints from the Metlo backend
  2. Generates authentication tests for each endpoint
  3. Runs all tests
  4. Reports results

​
Working with Test Estimates

For tests with many steps or payload permutations, Metlo estimates the number of requests: 
$ metlo test run complex-test.yaml
Running test at path "complex-test.yaml":
? Estimated request count is high (450). Would you like to continue? (y/N)
The estimate prevents accidentally running tests that make excessive requests.

​
Exit Codes

The CLI returns standard exit codes:
  • 0 - All tests passed
  • 1 - One or more tests failed
Use exit codes in CI/CD pipelines: 
metlo test run tests/*.yaml
if [ $? -ne 0 ]; then
  echo "Tests failed!"
  exit 1
fi

​
Debugging Failed Tests

Ensure you’re using the correct assertion type:
# Wrong - comparing string to number
assert:
  - key: resp.status
    value: "200"

# Correct
assert:
  - key: resp.status
    value: 200
Use verbose mode to see the actual response:
metlo test run test.yaml --verbose
Check that your assertion keys match the response structure:
# If response is: {"user": {"name": "John"}}
assert:
  - key: resp.data.user.name
    value: John
Ensure all required variables are defined:
# List variables in .env file
cat .env

# Run with specific env file
metlo test run test.yaml --envfile .env.staging
Check that headers and data are formatted correctly:
# Ensure Content-Type matches data format
headers:
  - name: Content-Type
    value: application/json
data: |-
  {"key": "value"}
Verify the API is reachable:
curl https://api.example.com/health

​
Advanced Usage

​
Custom Test Templates

Run tests generated from custom templates: 
metlo test generate \
  --test ./templates/custom-auth.ts \
  --host api.example.com \
  --endpoint /api/data \
  --method GET \
  --path output.yaml

​
Fuzzing

Run fuzz testing on endpoints: 
metlo test fuzz \
  --host api.example.com \
  --endpoint /api/search \
  --method POST

​
Global Environment Variables

When connected to Metlo backend, tests automatically have access to global environment variables defined in the Metlo UI. These are available under the global namespace: 
env:
  - name: API_KEY
    value: {{global.PROD_API_KEY}}

​
Performance Considerations

Request Limits

Tests with more than 300 estimated requests will prompt for confirmation. Consider:
  • Breaking large tests into smaller, focused tests
  • Using stopOnFailure: true to halt on first failure
  • Running subsets of tests in parallel in CI/CD

​
Next Steps

CI/CD Integration

Integrate tests into your deployment pipeline

Custom Templates

Create custom test templates for your needs

Build docs developers (and LLMs) love

Get started for freeTalk to us