Skip to main content
Anubis is designed as a modular reverse proxy with pluggable components for storage, policy evaluation, and external integrations.

System Components

Core Components

Reverse Proxy

Location: lib/anubis.go:Server The main HTTP handler that intercepts all requests:
type Server struct {
    next        http.Handler      // Upstream application
    store       store.Interface   // Storage backend
    mux         *http.ServeMux    // Internal routes
    policy      *policy.ParsedConfig
    OGTags      *ogtags.OGTagCache
    logger      *slog.Logger
    opts        Options
    ed25519Priv ed25519.PrivateKey
    hs512Secret []byte
}
Responsibilities:
  • Cookie validation and JWT parsing
  • Request routing to policy engine
  • Static asset serving (JS/CSS for challenges)
  • OpenGraph tag caching
  • Proxying validated requests to upstream
Key Methods:
  • ServeHTTP(): Main request handler
  • maybeReverseProxy(): Core request flow logic
  • check(): Policy evaluation orchestration

Policy Engine

Location: lib/policy/ Evaluates incoming requests against configured rules:
type ParsedConfig struct {
    Store             store.Interface
    Bots              []Bot
    Thresholds        []*Threshold
    DefaultDifficulty int
    DNSBL             bool
    DnsCache          *dns.DnsCache
    Dns               *dns.Dns
    Logger            *slog.Logger
}
Checker Implementations:

RemoteAddrChecker

CIDR-based IP matching using gaissmai/bart prefix tables.Location: lib/policy/checker.go:20

HeaderMatchesChecker

Regex matching against HTTP headers.Location: lib/policy/checker.go:66

PathChecker

Regex matching against request paths.Location: lib/policy/checker.go:96

CELChecker

Common Expression Language for complex conditions.Location: lib/policy/celchecker.go
Flow:
  1. Load and parse YAML policy configuration
  2. Compile regex patterns and CEL expressions
  3. Build checker chain for each bot rule
  4. Evaluate rules sequentially on each request
  5. Return CheckResult with action and weight

Challenge Engine

Location: lib/challenge/ Pluggable challenge system with multiple implementations:
type Impl interface {
    Setup(mux *http.ServeMux)
    Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in *IssueInput) (templ.Component, error)
    Validate(r *http.Request, lg *slog.Logger, in *ValidateInput) error
}
Registered Implementations:
AlgorithmFileDescription
fastlib/challenge/proofofwork/SHA256-based proof-of-work
slowlib/challenge/proofofwork/Deprecated alias for fast
metarefreshlib/challenge/metarefresh/HTML meta refresh redirect
preactlib/challenge/preact/Interactive Preact component
Challenge Lifecycle:
  1. issueChallenge(): Generate UUID, random data, store metadata
  2. Issue(): Render challenge page with embedded solver
  3. Client solves challenge and submits solution
  4. Validate(): Verify solution, mark as spent
  5. signJWT(): Generate authentication token

Storage Backends

Location: lib/store/ Unified interface for data persistence:
type Interface interface {
    Delete(ctx context.Context, key string) error
    Get(ctx context.Context, key string) ([]byte, error)
    Set(ctx context.Context, key string, value []byte, expiry time.Duration) error
    IsPersistent() bool
}
Implementations:
Location: lib/store/memory/In-memory map with TTL support using decaymap.Pros: Zero dependencies, instant accessCons: Not persistent, single-instance onlyUse case: Development, testing, single-server deployments
Location: lib/store/bbolt/Embedded key-value database.Pros: Persistent, ACID transactions, no networkCons: File locking (single process), slower than memoryUse case: Single-instance production with persistence
store:
  backend: bbolt
  parameters:
    path: /var/lib/anubis/anubis.db
Location: lib/store/valkey/Network-based key-value store.Pros: Shared state across instances, high performance, native TTLCons: External dependency, network latencyUse case: Multi-instance production deployments
store:
  backend: valkey
  parameters:
    addr: redis.example.com:6379
    password: ${REDIS_PASSWORD}
    db: 0
Location: lib/store/s3api/S3-compatible object storage.Pros: Unlimited capacity, durable, shared stateCons: High latency, not optimized for small objectsUse case: Very high-volume or compliance-driven deployments
store:
  backend: s3api
  parameters:
    endpoint: https://s3.amazonaws.com
    bucket: anubis-challenges
    region: us-east-1
Stored Data:
Key PatternTypeTTLPurpose
challenge:<uuid>challenge.Challenge30mChallenge metadata and spent status
dronebl:<ip>dnsbl.DroneBLResponse24hDNSBL lookup results
ogtags:allow:<host><path>boolVariableOpenGraph tag asset allowlist
dns:forward:<ip>[]stringConfigurableForward DNS cache
dns:reverse:<hostname>[]stringConfigurableReverse DNS cache

JWT Validator

Location: lib/anubis.go:getTokenKeyfunc() Supports two signing algorithms:
func (s *Server) getTokenKeyfunc() jwt.Keyfunc {
    if len(s.hs512Secret) == 0 {
        return func(token *jwt.Token) (interface{}, error) {
            return s.ed25519Priv.Public().(ed25519.PublicKey), nil
        }
    }
    // ...
}
JWT Claims:
{
  "challenge": "01JXXXXXXXXXXXXXXXXXXXXXX",
  "method": "fast",
  "policyRule": "abc123...def",
  "action": "CHALLENGE",
  "restriction": "sha256(header_value)",
  "difficulty": 3,
  "iat": 1234567890,
  "nbf": 1234567830,
  "exp": 1234654290
}
Validation Checks:
  • Signature integrity
  • Expiration time (exp)
  • Not-before time (nbf)
  • Policy rule hash match (detects config changes)
  • Optional restriction header binding

Thoth Integration

Location: lib/thoth/ Optional gRPC service for enriched IP intelligence:
type Client struct {
    conn    *grpc.ClientConn
    health  healthv1.HealthClient
    IPToASN iptoasnv1.IpToASNServiceClient
}
Features:
  • ASN lookup and matching
  • GeoIP country detection
  • IP reputation data
  • Cached responses (configurable TTL)
Configuration:
ANUBIS_THOTH_URL=thoth.example.com:443
ANUBIS_THOTH_API_TOKEN=your-token-here
ANUBIS_THOTH_PLAINTEXT=false
Policy Integration:
bots:
  - name: cloud-providers
    asns:
      match: [16509, 15169, 8075]
    action: CHALLENGE
  
  - name: high-risk-countries
    geoip:
      countries: [CN, RU, KP]
    action: DENY
Thoth is optional. ASN/GeoIP rules are skipped (with warnings) if Thoth is not configured.

Deployment Patterns

Single Instance

Configuration:
store:
  backend: bbolt
  parameters:
    path: /var/lib/anubis/anubis.db
Pros: Simple, no external dependencies Cons: Single point of failure, limited scalability

Multi-Instance (Shared State)

Configuration:
store:
  backend: valkey
  parameters:
    addr: redis.prod.internal:6379
    password: ${REDIS_PASSWORD}
Pros: High availability, horizontal scaling Cons: Redis as dependency, network latency

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: anubis
spec:
  replicas: 3
  selector:
    matchLabels:
      app: anubis
  template:
    metadata:
      labels:
        app: anubis
    spec:
      containers:
      - name: anubis
        image: ghcr.io/techarohq/anubis:latest
        env:
        - name: ANUBIS_UPSTREAM
          value: http://backend-service:8080
        - name: ANUBIS_STORE_BACKEND
          value: valkey
        - name: ANUBIS_STORE_ADDR
          value: redis-service:6379
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: anubis
spec:
  selector:
    app: anubis
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

Nginx Integration

Nginx as TLS terminator:
upstream anubis {
    server localhost:8080;
}

server {
    listen 443 ssl http2;
    server_name example.com;

    ssl_certificate /etc/ssl/certs/example.com.crt;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        proxy_pass http://anubis;
        proxy_set_header Host $host;
        proxy_set_header X-Real-Ip $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Uri $request_uri;
    }
}
Required Headers: Anubis depends on X-Real-Ip being set. Without it, all requests will fail with misconfiguration errors.

Caddy Integration

example.com {
    reverse_proxy localhost:8080 {
        header_up X-Real-Ip {remote_host}
        header_up X-Forwarded-For {remote_host}
        header_up X-Forwarded-Proto {scheme}
        header_up X-Forwarded-Host {host}
    }
}

Request Headers

Anubis requires from upstream proxy:
HeaderRequiredPurpose
X-Real-IpYesClient IP for policy evaluation
X-Forwarded-ForRecommendedIP chain for logging
X-Forwarded-ProtoFor redirectsProtocol for redirect URLs
X-Forwarded-HostFor redirectsHost for redirect URLs
X-Forwarded-UriFor redirectsOriginal URI for redirects
Anubis adds to upstream requests:
HeaderValuePurpose
X-Anubis-Rulebot/rule-nameWhich rule matched
X-Anubis-ActionALLOW/CHALLENGE/etcWhat action was taken
X-Anubis-StatusPASSChallenge validation status

Performance Characteristics

Latency Impact

Cold path (no cookie):
  • Policy evaluation: ~1-5ms
  • Challenge rendering: ~10-50ms
  • Total: 11-55ms overhead
Warm path (valid cookie):
  • JWT validation: ~0.5-2ms
  • Total: less than 2ms overhead
Challenge solving:
  • Difficulty 3: ~500ms client-side
  • Difficulty 5: ~60s client-side

Memory Usage

Baseline:
  • Anubis process: ~50-100 MB
  • Per challenge (memory store): ~2 KB
  • Per JWT: 0 bytes (stateless)
With 10,000 active challenges:
  • Memory store: ~120 MB total
  • BBolt: ~30 MB on disk
  • Valkey: Negligible (external)

Throughput

Benchmark results (Intel i7, 8 cores):
ScenarioRequests/secLatency p50Latency p99
Valid JWT (proxy)25,0001.2ms4.5ms
Policy eval (ALLOW)18,0002.1ms8.2ms
Challenge render5,00012ms35ms
Use ANUBIS_LOG_LEVEL=warn in production to reduce logging overhead.

Monitoring and Observability

Prometheus Metrics

Anubis exposes metrics at /.within.website/metrics:
# Challenge lifecycle
anubis_challenges_issued{method="embedded"}
anubis_challenges_validated{method="fast"}
anubis_failed_validations{method="fast"}
anubis_challenge_time_taken_seconds{algorithm="fast"}

# Policy decisions
anubis_policy_results{rule="bot/name",action="ALLOW"}

# Proxy traffic
anubis_proxied_requests_total{host="example.com"}

# DroneBL
anubis_dronebl_hits{status="all_good"}

Structured Logging

Anubis uses log/slog for structured logging:
{
  "time": "2026-03-03T10:15:30Z",
  "level": "INFO",
  "msg": "new challenge issued",
  "challenge": "01JXXXXXXXXXXXXXXXXXXXXXX",
  "rule": "bot/suspicious-crawler",
  "difficulty": 3,
  "remote_addr": "203.0.113.42"
}
Log Levels:
  • DEBUG: Policy evaluation details, JWT validation
  • INFO: Challenge issuance, successful validations
  • WARN: Configuration warnings, deprecated features
  • ERROR: Validation failures, store errors

Health Checks

Anubis doesn’t expose a dedicated health endpoint, but you can:
  1. Check metrics endpoint: GET /.within.website/metrics
  2. Use upstream health (after ALLOW rule)
  3. Monitor store connectivity

Security Considerations

JWT signature validation and challenge hash comparison use crypto/subtle.ConstantTimeCompare to prevent timing attacks.
Challenges marked as Spent=true after first successful validation. Store atomicity prevents reuse.
JWT contains policy rule hash. Config changes automatically invalidate all existing tokens.
Redirect URL validation prevents javascript:, data:, and other dangerous schemes.

Troubleshooting

X-Real-Ip not set

Error: [misconfiguration] X-Real-Ip header is not setFix: Configure your reverse proxy to set this header from client IP.

Policy doesn't match

Issue: Requests not triggering expected rulesDebug: Enable ANUBIS_LOG_LEVEL=debug and check check_result logs.

Cookies not persisting

Issue: Challenges loop indefinitelyCauses:
  • Cookie domain mismatch
  • Third-party cookie blocking
  • SameSite=None without Secure

Store connectivity

Issue: can't fetch challenge errorsCheck: Store backend health, network connectivity, credentials.

Next Steps

Configuration Guide

Learn about all configuration options and environment variables

Deployment Guide

Production deployment best practices and examples

Build docs developers (and LLMs) love