AlertChannel CRD

Overview

The AlertChannel custom resource defines how and where to send alerts. AlertChannels are cluster-scoped resources that can be referenced by any CronJobMonitor.

// From api/v1alpha1/alertchannel_types.go:7-36
type AlertChannelSpec struct {
    Type          string               // slack, pagerduty, webhook, email
    Slack         *SlackConfig
    PagerDuty     *PagerDutyConfig
    Webhook       *WebhookConfig
    Email         *EmailConfig
    RateLimiting  *RateLimitConfig
    TestOnSave    bool
}

Channel Types

Slack

Send rich formatted alerts to Slack channels via webhooks

PagerDuty

Create incidents in PagerDuty for on-call escalation

Email

Deliver alerts via SMTP email with custom templates

Webhook

POST alerts to any HTTP endpoint with custom payloads

Slack

Send alerts to Slack channels using incoming webhooks.

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: slack-alerts
spec:
  type: slack
  slack:
    webhookSecretRef:
      name: slack-webhook
      namespace: cronjob-guardian
      key: url
    defaultChannel: "#alerts"
    messageTemplate: |
      :rotating_light: *{\{ .Severity | upper }\}*: {\{ .Title }\}
      
      *CronJob:* `{\{ .CronJob.Namespace }\}/{\{ .CronJob.Name }\}`
      *Monitor:* `{\{ .MonitorRef.Namespace }\}/{\{ .MonitorRef.Name }\}`
      
      {\{ .Message }\}
      
      {\{ if .Context.SuggestedFix }\}
      :bulb: *Suggested Fix:* {\{ .Context.SuggestedFix }\}
      {\{ end }\}
  rateLimiting:
    maxAlertsPerHour: 100
    burstLimit: 10

Slack Configuration

// From api/v1alpha1/alertchannel_types.go:38-50
type SlackConfig struct {
    WebhookSecretRef  NamespacedSecretKeyRef
    DefaultChannel    string
    MessageTemplate   string
}

webhookSecretRef

object

required

Reference to a Secret containing the Slack webhook URL.Create the secret:

kubectl create secret generic slack-webhook \
  -n cronjob-guardian \
  --from-literal=url='https://hooks.slack.com/services/YOUR/WEBHOOK/URL'

defaultChannel

string

Override the webhook’s default channel (e.g., #alerts, @username).

messageTemplate

string

Go template for custom message formatting. If not specified, uses a built-in template.Available variables:

{\{ .Type }\} - Alert type (JobFailed, DeadManTriggered, etc.)
{\{ .Severity }\} - critical or warning
{\{ .Title }\} - Alert title
{\{ .Message }\} - Alert message
{\{ .CronJob.Namespace }\}, {\{ .CronJob.Name }\} - CronJob reference
{\{ .MonitorRef.Namespace }\}, {\{ .MonitorRef.Name }\} - Monitor reference
{\{ .Context.Logs }\} - Pod logs
{\{ .Context.Events }\} - Kubernetes events (array)
{\{ .Context.SuggestedFix }\} - Suggested fix text
{\{ .Context.ExitCode }\} - Container exit code
{\{ .Context.Reason }\} - Termination reason

Template functions:

{\{ .Severity | upper }\} - Uppercase
{\{ .Severity | lower }\} - Lowercase
{\{ .Message | truncate 100 }\} - Truncate to N chars
{\{ formatTime .Timestamp "RFC3339" }\} - Format time

PagerDuty

Create incidents in PagerDuty for on-call escalation.

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: pagerduty-oncall
spec:
  type: pagerduty
  pagerduty:
    routingKeySecretRef:
      name: pagerduty-key
      namespace: cronjob-guardian
      key: routing-key
    severity: error  # PagerDuty severity: critical, error, warning, info

PagerDuty Configuration

// From api/v1alpha1/alertchannel_types.go:52-61
type PagerDutyConfig struct {
    RoutingKeySecretRef  NamespacedSecretKeyRef
    Severity             string  // critical, error, warning, info
}

routingKeySecretRef

object

required

Reference to a Secret containing the PagerDuty integration/routing key.Create the secret:

kubectl create secret generic pagerduty-key \
  -n cronjob-guardian \
  --from-literal=routing-key='YOUR_ROUTING_KEY'

severity

string

default:"error"

Default PagerDuty severity. Valid values: critical, error, warning, info.The dispatcher maps CronJob Guardian severities to PagerDuty:

Guardian critical → PagerDuty critical
Guardian warning → PagerDuty configured severity (default: error)

PagerDuty incidents include dedup_key based on the alert key, so repeated alerts update the same incident instead of creating duplicates.

Email

Send alerts via SMTP email with customizable templates.

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-team
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-credentials
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]
      - [email protected]
    subjectTemplate: "[{\{ .Severity | upper }\}] {\{ .CronJob.Name }\} - {\{ .Type }\}"
    bodyTemplate: |
      Alert Type: {\{ .Type }\}
      Severity: {\{ .Severity }\}
      
      CronJob: {\{ .CronJob.Namespace }\}/{\{ .CronJob.Name }\}
      Monitor: {\{ .MonitorRef.Namespace }\}/{\{ .MonitorRef.Name }\}
      
      {\{ .Message }\}
      
      {\{ if .Context.SuggestedFix }\}
      Suggested Fix:
      {\{ .Context.SuggestedFix }\}
      {\{ end }\}
      
      {\{ if .Context.Logs }\}
      Recent Logs:
      {\{ .Context.Logs }\}
      {\{ end }\}

Email Configuration

// From api/v1alpha1/alertchannel_types.go:82-100
type EmailConfig struct {
    SMTPSecretRef    NamespacedSecretRef
    From             string
    To               []string
    SubjectTemplate  string
    BodyTemplate     string
}

smtpSecretRef

object

required

Reference to a Secret containing SMTP configuration.Required secret keys:

host - SMTP server hostname
port - SMTP server port (e.g., “587”)
username - SMTP username
password - SMTP password

Create the secret:

kubectl create secret generic smtp-credentials \
  -n cronjob-guardian \
  --from-literal=host='smtp.gmail.com' \
  --from-literal=port='587' \
  --from-literal=username='[email protected]' \
  --from-literal=password='YOUR_PASSWORD'

from

string

required

Sender email address.

string[]

required

List of recipient email addresses.

subjectTemplate

string

Go template for email subject. Defaults to: CronJob Guardian Alert: {\{ .CronJob.Name }\}

bodyTemplate

string

Go template for email body. If not specified, uses a built-in template.

Webhook

Send alerts to custom HTTP endpoints with flexible payload templates.

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: webhook-custom
spec:
  type: webhook
  webhook:
    urlSecretRef:
      name: webhook-url
      namespace: cronjob-guardian
      key: url
    method: POST
    headers:
      Content-Type: application/json
      Authorization: Bearer ${TOKEN}
    payloadTemplate: |
      {
        "alert_type": "{\{ .Type }\}",
        "severity": "{\{ .Severity }\}",
        "cronjob": "{\{ .CronJob.Namespace }\}/{\{ .CronJob.Name }\}",
        "message": {\{ .Message | jsonEscape }\},
        "timestamp": "{\{ formatTime .Timestamp "RFC3339" }\}",
        "exit_code": {\{ .Context.ExitCode }\},
        "suggested_fix": {\{ .Context.SuggestedFix | jsonEscape }\}
      }

Webhook Configuration

// From api/v1alpha1/alertchannel_types.go:63-80
type WebhookConfig struct {
    URLSecretRef     NamespacedSecretKeyRef
    Method           string  // POST, PUT
    Headers          map[string]string
    PayloadTemplate  string
}

urlSecretRef

object

required

Reference to a Secret containing the webhook URL.Create the secret:

kubectl create secret generic webhook-url \
  -n cronjob-guardian \
  --from-literal=url='https://api.example.com/alerts'

method

string

default:"POST"

HTTP method: POST or PUT.

headers

object

Custom HTTP headers to include in requests.Environment variable substitution is supported:

headers:
  Authorization: Bearer ${API_TOKEN}

payloadTemplate

string

Go template for the JSON payload. If not specified, sends a default structure.Use {\{ .Field | jsonEscape }\} for string fields to ensure proper JSON escaping.

Rate Limiting

All channel types support rate limiting to prevent alert storms:

rateLimiting:
  maxAlertsPerHour: 100
  burstLimit: 10

// From api/v1alpha1/alertchannel_types.go:115-126
type RateLimitConfig struct {
    MaxAlertsPerHour  *int32  // Default: 100
    BurstLimit        *int32  // Default: 10
}

maxAlertsPerHour

int32

default:"100"

Maximum alerts per hour for this channel.

burstLimit

int32

default:"10"

Maximum alerts per minute (burst allowance).

Rate limits are per-channel. The global dispatcher also has rate limits (default: 50/min) that apply across all channels.

Test on Save

Enable test alert dispatch when the AlertChannel is created or updated:

spec:
  testOnSave: true

This sends a test alert to verify configuration. The test result is reflected in the status:

status:
  ready: true
  lastTestTime: "2026-03-04T10:30:00Z"
  lastTestResult: success

Status Fields

The controller updates the status with operational metrics:

status:
  ready: true
  lastTestTime: "2026-03-04T10:30:00Z"
  lastTestResult: success
  alertsSentTotal: 142
  lastAlertTime: "2026-03-04T12:45:00Z"
  alertsFailedTotal: 3
  lastFailedTime: "2026-03-03T08:20:00Z"
  lastFailedError: "connection timeout"
  consecutiveFailures: 0

ready

bool

Whether the channel is operational.

alertsSentTotal

int64

Total alerts successfully sent via this channel since creation.

alertsFailedTotal

int64

Total alerts that failed to send.

consecutiveFailures

int32

Number of consecutive failures. Resets to 0 on successful send.

Channel Lifecycle

The AlertChannel controller:

Registration: When an AlertChannel is created, the controller:
- Validates the configuration
- Fetches referenced secrets
- Registers the channel with the alert dispatcher
- Optionally sends a test alert (testOnSave: true)
Updates: When an AlertChannel is updated:
- Re-validates configuration
- Re-fetches secrets (in case they changed)
- Updates the dispatcher’s channel registry
- Optionally re-tests
Deletion: When an AlertChannel is deleted:
- Removes from dispatcher registry
- In-flight alerts may still be delivered
- Pending (delayed) alerts referencing the channel are cancelled

Referencing Channels

CronJobMonitors reference AlertChannels by name:

apiVersion: guardian.illenium.net/v1alpha1
kind: CronJobMonitor
metadata:
  name: my-monitor
spec:
  alerting:
    channelRefs:
      - name: slack-alerts
        severities: [critical, warning]
      - name: pagerduty-oncall
        severities: [critical]

If a referenced AlertChannel doesn’t exist, the monitor will log warnings but continue operating. Ensure channels are created before monitors reference them.

Examples

Slack Example

Complete Slack configuration

PagerDuty Example

PagerDuty integration setup

Email Example

SMTP email configuration

Webhook Example

Custom webhook integration

Next Steps

CronJobMonitor

Learn how to configure monitors

Alert Templates

Customize alert message formatting

Get Started

Core Concepts

Guides

Operations

Overview

Channel Types

Slack

PagerDuty

Email

Webhook

Slack

Slack Configuration

PagerDuty

PagerDuty Configuration

Email

Email Configuration

Webhook

Webhook Configuration

Rate Limiting

Test on Save

Status Fields

Channel Lifecycle

Referencing Channels

Examples

Slack Example

PagerDuty Example

Email Example

Webhook Example

Next Steps

CronJobMonitor

Alert Templates

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Operations

​Overview

​Channel Types

Slack

PagerDuty

Email

Webhook

​Slack

​Slack Configuration

​PagerDuty

​PagerDuty Configuration

​Email

​Email Configuration

​Webhook

​Webhook Configuration

​Rate Limiting

​Test on Save

​Status Fields

​Channel Lifecycle

​Referencing Channels

​Examples

Slack Example

PagerDuty Example

Email Example

Webhook Example

​Next Steps

CronJobMonitor

Alert Templates

Build docs developers (and LLMs) love

Overview

Channel Types

Slack

Slack Configuration

PagerDuty

PagerDuty Configuration

Email

Email Configuration

Webhook

Webhook Configuration

Rate Limiting

Test on Save

Status Fields

Channel Lifecycle

Referencing Channels

Examples

Next Steps