LLM Checker can act as a policy gate in any CI/CD pipeline. The check command with --policy enforces governance rules; audit export writes machine-readable compliance artifacts that downstream jobs, SIEM systems, and security scanners can consume.
In enforce mode, a blocking policy violation causes a non-zero exit code. If your CI system treats non-zero exits as failures (GitHub Actions, GitLab CI, Jenkins, etc.), ensure your policy is validated and correct before enabling enforce mode in a required pipeline stage. Use audit mode first to identify violations without breaking builds.
GitHub Actions Policy Gate
Copy this workflow directly into .github/workflows/policy-gate.yml. It runs the policy check on every pull request, exports all report formats, and uploads them as a build artifact.
name: Policy Gate
on: [pull_request]
jobs:
policy-gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: node bin/enhanced_cli.js check --policy ./policy.yaml --runtime ollama --no-verbose
- if: always()
run: node bin/enhanced_cli.js audit export --policy ./policy.yaml --command check --format all --runtime ollama --no-verbose --out-dir ./policy-reports
- if: always()
uses: actions/upload-artifact@v4
with:
name: policy-audit-reports
path: ./policy-reports
The --no-verbose flag suppresses progress output, keeping CI logs clean. The if: always() on the export and upload steps ensures audit artifacts are written even when the policy check step fails.
Add the policy file to your repository
Generate a policy template and commit it alongside your code:llm-checker policy init
# edit policy.yaml, then:
git add policy.yaml
git commit -m "chore: add llm-checker policy"
Choose enforcement mode
Set enforcement.mode in policy.yaml:
audit — logs violations, exits 0. Use this while iterating on policy rules.enforce — exits non-zero on blocking violations. Use this once your policy is stable.
enforcement:
mode: enforce
exit_code: 1
Add the workflow file
Copy the GitHub Actions YAML above into .github/workflows/policy-gate.yml and push. The gate runs automatically on every pull request.
Review audit artifacts
After each run, download the policy-audit-reports artifact from the Actions summary. It contains json, csv, and sarif reports for the policy evaluation.
audit export to produce compliance evidence in any of three formats. All formats are written before a non-zero exit in enforce mode.
# CI artifact for post-processing in pipeline jobs
llm-checker audit export \
--policy ./policy.yaml \
--command check \
--format json \
--out ./reports/policy-report.json
llm-checker audit export \
--policy ./policy.yaml \
--command check \
--format all \
--out-dir ./reports
--format all honours reporting.formats in your policy file and falls back to json,csv,sarif if not configured.
SIEM Integration
The CSV format is designed for direct ingestion into log management and observability platforms:
# Write CSV and ingest with Splunk Universal Forwarder
llm-checker audit export \
--policy ./policy.yaml \
--command check \
--format csv \
--out /var/log/llm-checker/policy-report.csv
# Configure inputs.conf to monitor /var/log/llm-checker/
SARIF for Security Scanning
The SARIF format integrates with GitHub Advanced Security, GitLab SAST, and any tool that accepts the Static Analysis Results Interchange Format:
llm-checker audit export \
--policy ./policy.yaml \
--command check \
--format sarif \
--out ./reports/policy-report.sarif
github/codeql-action/upload-sarif action:
- name: Upload SARIF
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ./reports/policy-report.sarif
JSON for Pipeline Post-Processing
The JSON format is suitable for any pipeline job that needs to parse, filter, or forward compliance data programmatically:
llm-checker audit export \
--policy ./policy.yaml \
--command check \
--format json \
--out ./reports/check-policy.json
# Example: extract violation count with jq
jq '.violations | length' ./reports/check-policy.json
--no-verbose for Clean CI Output
Pass --no-verbose to any check, recommend, or audit export invocation to suppress progress banners, spinners, and decorative output. This keeps CI logs readable and avoids noise in artifact uploads:
llm-checker check --policy ./policy.yaml --no-verbose
llm-checker audit export --policy ./policy.yaml --command check --format all --no-verbose --out-dir ./reports
