Configuration Validation - GitHub Webhook Server

Overview

The GitHub Webhook Server includes comprehensive configuration validation using JSON Schema to ensure your configuration files are correct before deployment. This prevents runtime errors and provides IDE autocompletion for faster configuration.

Schema Validation System

Schema Location

The configuration schema is defined in:

webhook_server/config/schema.yaml

Public URL for IDE support:

https://raw.githubusercontent.com/myk-org/github-webhook-server/refs/heads/main/webhook_server/config/schema.yaml

What Gets Validated

Required Fields

github-app-id - GitHub App ID
webhook-ip - Webhook URL with full path
github-tokens - At least one GitHub token
repositories - Repository configurations
Repository name - Full repository name (org/repo)

Type Checking

Strings: log-level, log-file, webhook-secret
Integers: github-app-id, port, max-workers, minimum-lgtm
Booleans: verify-github-ips, pre-commit, verified-job
Arrays: github-tokens, protected-branches, enabled-labels
Objects: docker, pypi, container, tox, labels

Enum Validation

log-level: INFO, DEBUG
ai-provider: claude, gemini, cursor
conventional-title: true, false, fix
Label categories: verified, hold, wip, needs-rebase, has-conflicts, can-be-merged, size, branch, cherry-pick, automerge

Structural Validation

Repository configurations nested correctly
PR size thresholds properly formatted
Protected branch rules structure
Test oracle configuration
AI features configuration
Container build arguments

Cross-Field Validation

Label categories in enabled-labels must be valid
Repository names must match pattern org/repo
Webhook URL must be valid URI
Color names for labels must be CSS3 colors

IDE Integration

Enabling Schema Validation in Your Editor

Add the schema reference to the first line of your config.yaml:

config.yaml

# yaml-language-server: $schema=https://raw.githubusercontent.com/myk-org/github-webhook-server/refs/heads/main/webhook_server/config/schema.yaml

github-app-id: 123456
webhook-ip: https://your-domain.com/webhook_server
github-tokens:
  - ghp_your_github_token

repositories:
  my-repository:
    name: my-org/my-repository
    protected-branches:
      main: []

Supported Editors

VS Code

Install YAML extension by Red Hat
Schema validation automatic
Hover documentation
Autocomplete

IntelliJ IDEA

Built-in YAML support
Schema validation automatic
Quick documentation
Code completion

Vim/Neovim

Install yaml-language-server
Configure with coc.nvim or LSP
Schema validation via LSP

IDE Features with Schema

Autocompletion
Validation Errors
Hover Documentation
Enum Suggestions

Type to see available fields:

repositories:
  my-repo:
    name: org/repo
    min  # ← Press Ctrl+Space
    # Suggestions:
    # - minimum-lgtm
    # - mask-sensitive-data

Red squiggles under invalid values:

log-level: TRACE  # ❌ Error: Must be INFO or DEBUG
port: "5000"      # ❌ Error: Must be integer, not string

Hover over fields to see descriptions:

minimum-lgtm: 2
# Hover shows:
# "Minimum number of approvals from approvers required to merge PR"

Autocomplete for restricted values:

ai-features:
  ai-provider:  # ← Suggests: claude, gemini, cursor
  conventional-title:  # ← Suggests: "true", "false", "fix"

Validation Commands

Validate Configuration Files

Use the built-in validator script to check your configuration:

# Validate your config.yaml
uv run webhook_server/tests/test_schema_validator.py config.yaml

# Validate example configuration
uv run webhook_server/tests/test_schema_validator.py examples/config.yaml

# Validate repository-specific config
uv run webhook_server/tests/test_schema_validator.py .github-webhook-server.yaml

Output on success:

✅ config.yaml is valid

Output on error:

❌ config.yaml validation failed:
Error at repositories.my-repo.minimum-lgtm: '5' is not of type 'integer'
Error at log-level: 'TRACE' is not one of ['INFO', 'DEBUG']

Running Configuration Tests

Run the full test suite for configuration validation:

# Run all schema validation tests
uv run pytest webhook_server/tests/test_config_schema.py -v

# Run specific test
uv run pytest webhook_server/tests/test_config_schema.py::TestConfigSchema::test_valid_full_config_loads -v

# Run with coverage
uv run pytest webhook_server/tests/test_config_schema.py --cov=webhook_server.libs.config

Test categories:

test_valid_minimal_config()        # Minimal required fields
test_valid_full_config_loads()     # Full config with all options
test_valid_pr_size_thresholds()    # PR size label validation
test_valid_labels_config()         # Label category validation

Validation Features

Required Fields Validation

github-app-id

integer

required

GitHub App ID for webhook authentication

webhook-ip

string (uri)

required

Full webhook URL including path (e.g., https://your-domain.com/webhook_server)

github-tokens

array<string>

required

At least one GitHub Personal Access Token with admin permissions

repositories

object

required

Map of repository short names to configuration objects, each containing:

name (required): Full repository name in org/repo format
Other repository-level settings (optional)

Example minimal valid config:

github-app-id: 123456
webhook-ip: https://example.com/webhook_server
github-tokens:
  - ghp_token123
repositories:
  my-repo:
    name: org/my-repo

Type Checking Examples

port: 5000                    # integer ✅
max-workers: 10               # integer ✅
verify-github-ips: true       # boolean ✅
log-level: INFO               # string enum ✅
minimum-lgtm: 2               # integer ✅

Enum Validation Examples

log-level: INFO               # ✅
log-level: DEBUG              # ✅

ai-features:
  ai-provider: claude         # ✅
  ai-provider: gemini         # ✅
  ai-provider: cursor         # ✅
  conventional-title: "true"  # ✅
  conventional-title: "false" # ✅
  conventional-title: "fix"   # ✅

Label Categories Validation

The schema validates enabled-labels against allowed categories:

labels:
  enabled-labels:
    - verified          # ✅
    - hold              # ✅
    - wip               # ✅
    - needs-rebase      # ✅
    - has-conflicts     # ✅
    - can-be-merged     # ✅
    - size              # ✅
    - branch            # ✅
    - cherry-pick       # ✅
    - automerge         # ✅

Review labels (approved-*, lgtm-*, changes-requested-*) are always enabled and cannot be disabled.

Structural Validation

The schema enforces correct nesting and structure:

repositories:
  my-repo:
    name: org/my-repo
    container:
      username: user
      password: pass
      repository: quay.io/org/repo
      tag: latest
    pr-size-thresholds:
      Small:
        threshold: 50
        color: green

Common Validation Errors

Error: Missing Required Field

❌ Error: 'github-app-id' is a required property

Fix: Add the required field:

github-app-id: 123456  # Add this

Error: Invalid Type

❌ Error at port: '5000' is not of type 'integer'

Fix: Remove quotes from numeric values:

port: 5000  # Not "5000"

Error: Invalid Enum Value

❌ Error at log-level: 'WARNING' is not one of ['INFO', 'DEBUG']

Fix: Use a valid enum value:

log-level: INFO  # Or DEBUG

Error: Invalid Label Category

❌ Invalid label categories in enabled-labels: ['custom-label']. 
Valid categories: ['automerge', 'branch', 'can-be-merged', 'cherry-pick', 
'has-conflicts', 'hold', 'needs-rebase', 'size', 'verified', 'wip']

Fix: Use only valid categories:

labels:
  enabled-labels:
    - verified
    - hold
    - wip
    # Remove 'custom-label'

Error: Malformed YAML

❌ yaml.scanner.ScannerError: mapping values are not allowed here

Fix: Check YAML syntax (indentation, colons, quotes):

# ❌ Wrong
repositories:
  my-repo
    name: org/repo

# ✅ Correct
repositories:
  my-repo:
    name: org/repo

Validation Best Practices

Enable Schema in IDE

Add schema reference to first line of config files for real-time validation

Validate Before Commit

Run uv run webhook_server/tests/test_schema_validator.py config.yaml before committing

Test in Staging First

Deploy configuration changes to staging environment before production

Use Version Control

Commit config files to git and use pull requests for review

Document Custom Settings

Add comments explaining non-obvious configuration choices

Run Full Test Suite

Run uv run pytest webhook_server/tests/test_config_schema.py after schema changes

Pre-Commit Hook Integration

Validate configuration automatically before commits:

.pre-commit-config.yaml

repos:
  - repo: local
    hooks:
      - id: validate-webhook-config
        name: Validate webhook server config
        entry: uv run webhook_server/tests/test_schema_validator.py
        language: system
        files: ^(config\.yaml|.*\.github-webhook-server\.yaml)$
        pass_filenames: true

Install pre-commit:

pip install pre-commit
pre-commit install

Now config files are validated automatically:

git commit -m "Update config"

# Hook runs:
Validate webhook server config..................................✅ Passed

Configuration Reference

Root Level Options

Server

object

ip-bind (string): IP address to bind server (default: 0.0.0.0)
port (integer): Port number (default: 5000)
max-workers (integer): Worker count (default: 10)
log-level (enum): INFO or DEBUG
log-file (string): Log file path

Security

object

webhook-secret (string): HMAC signature secret
verify-github-ips (boolean): IP allowlist validation
verify-cloudflare-ips (boolean): Cloudflare IP validation
disable-ssl-warnings (boolean): Suppress SSL warnings

GitHub

object

github-app-id (integer, required): GitHub App ID
github-tokens (array, required): Personal access tokens
webhook-ip (string, required): Full webhook URL

Repository Level Options

Basic

object

name (string, required): Full repository name (org/repo)
log-level (enum): INFO or DEBUG
log-file (string): Repository-specific log file
slack-webhook-url (string): Slack notifications

Features

object

verified-job (boolean): Enable verified label automation
pre-commit (boolean): Run pre-commit hooks
pypi (object): PyPI publishing configuration
tox (object): Tox test configuration
container (object): Container build/push configuration

Pull Requests

object

minimum-lgtm (integer): Required approvals count
conventional-title (string): Conventional commit validation
can-be-merged-required-labels (array): Labels required to merge
create-issue-for-new-pr (boolean): Auto-create tracking issues

Labels

object

enabled-labels (array): Label categories to enable
colors (object): Custom label colors
pr-size-thresholds (object): PR size label configuration

Configuration Guide

Complete configuration options and examples

Schema Reference

Full JSON Schema definition on GitHub

Examples

Example configuration files

Testing

Running tests and validation

Getting Started

Core Concepts

Features

Deployment

Security

Advanced

​Overview

​Schema Validation System

​Schema Location

​What Gets Validated

​IDE Integration

​Enabling Schema Validation in Your Editor

​Supported Editors

VS Code

IntelliJ IDEA

Vim/Neovim

​IDE Features with Schema

​Validation Commands

​Validate Configuration Files

​Running Configuration Tests

​Validation Features

​Required Fields Validation

​Type Checking Examples

​Enum Validation Examples

​Label Categories Validation

​Structural Validation

​Common Validation Errors

​Error: Missing Required Field

​Error: Invalid Type

​Error: Invalid Enum Value

​Error: Invalid Label Category

​Error: Malformed YAML

​Validation Best Practices

​Pre-Commit Hook Integration

​Configuration Reference

​Root Level Options

​Repository Level Options

​Related Documentation

Configuration Guide

Schema Reference

Examples

Testing

Build docs developers (and LLMs) love

Overview

Schema Validation System

Schema Location

What Gets Validated

IDE Integration

Enabling Schema Validation in Your Editor

Supported Editors

IDE Features with Schema

Validation Commands

Validate Configuration Files

Running Configuration Tests

Validation Features

Required Fields Validation

Type Checking Examples

Enum Validation Examples

Label Categories Validation

Structural Validation

Common Validation Errors

Error: Missing Required Field

Error: Invalid Type

Error: Invalid Enum Value

Error: Invalid Label Category

Error: Malformed YAML

Validation Best Practices

Pre-Commit Hook Integration

Configuration Reference

Root Level Options

Repository Level Options

Related Documentation