Email Alert Integration

Overview

Email alerts provide a traditional and widely-supported notification channel for CronJob Guardian. Email is useful for:

Teams that don’t use Slack or other chat platforms
Compliance and audit requirements
Distribution lists and mailing lists
Non-technical stakeholders who need notifications
Long-term alert archival

Quick Start

Get SMTP credentials

Gather your SMTP server details:

SMTP host (e.g., smtp.gmail.com)
SMTP port (typically 587 for TLS, 465 for SSL)
Username/email address
Password or app-specific password

Create Kubernetes Secret

Store SMTP credentials in a secret:

kubectl create secret generic smtp-credentials \
  --namespace cronjob-guardian \
  --from-literal=host=smtp.gmail.com \
  --from-literal=port=587 \
  [email protected] \
  --from-literal=password=your-app-password

Create Email AlertChannel

kubectl apply -f - <<EOF
apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-team
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-credentials
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]
      - [email protected]
EOF

Reference in CronJobMonitor

apiVersion: guardian.illenium.net/v1alpha1
kind: CronJobMonitor
metadata:
  name: critical-jobs
  namespace: production
spec:
  selector:
    matchLabels:
      tier: critical
  alerting:
    channelRefs:
      - name: email-team

Basic Email AlertChannel

Here’s the example from the repository:

alertchannels/email.yaml

# Email AlertChannel
# Sends alerts via SMTP email
apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-team
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-credentials
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]
      - [email protected]

Configuration Options

email.smtpSecretRef

object

required

Reference to a Kubernetes Secret containing SMTP credentialsThe secret must contain keys:

host: SMTP server hostname
port: SMTP server port
username: SMTP authentication username
password: SMTP authentication password

Show properties

name

string

required

Name of the Secret

namespace

string

required

Namespace where the Secret exists

email.from

string

required

Sender email address (e.g., [email protected])This address appears in the “From” field of alert emails.

email.to

array

required

List of recipient email addressesAll recipients receive every alert sent to this channel.

email.cc

array

List of CC recipient email addresses (optional)

email.bcc

array

List of BCC recipient email addresses (optional)

Common SMTP Providers

Gmail

# Create Gmail SMTP secret
kubectl create secret generic smtp-gmail \
  --namespace cronjob-guardian \
  --from-literal=host=smtp.gmail.com \
  --from-literal=port=587 \
  [email protected] \
  --from-literal=password=your-app-password

Gmail requires an App Password, not your regular password. Generate one at Google Account Settings.

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-gmail
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-gmail
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]

Office 365 / Outlook

# Create Office 365 SMTP secret
kubectl create secret generic smtp-office365 \
  --namespace cronjob-guardian \
  --from-literal=host=smtp.office365.com \
  --from-literal=port=587 \
  [email protected] \
  --from-literal=password=your-password

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-office365
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-office365
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]

Amazon SES

# Create SES SMTP secret
kubectl create secret generic smtp-ses \
  --namespace cronjob-guardian \
  --from-literal=host=email-smtp.us-east-1.amazonaws.com \
  --from-literal=port=587 \
  --from-literal=username=YOUR-SES-SMTP-USERNAME \
  --from-literal=password=YOUR-SES-SMTP-PASSWORD

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-ses
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-ses
      namespace: cronjob-guardian
    from: [email protected]  # Must be verified in SES
    to:
      - [email protected]

SendGrid

# Create SendGrid SMTP secret
kubectl create secret generic smtp-sendgrid \
  --namespace cronjob-guardian \
  --from-literal=host=smtp.sendgrid.net \
  --from-literal=port=587 \
  --from-literal=username=apikey \
  --from-literal=password=YOUR-SENDGRID-API-KEY

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-sendgrid
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-sendgrid
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]

Generic SMTP Server

# Create custom SMTP secret
kubectl create secret generic smtp-custom \
  --namespace cronjob-guardian \
  --from-literal=host=mail.example.com \
  --from-literal=port=587 \
  --from-literal=username=alerts \
  --from-literal=password=smtp-password

Multiple Email Lists

Create separate AlertChannels for different teams or alert types:

apiVersion: guardian.illenium.net/v1alpha1
kind: AlertChannel
metadata:
  name: email-ops
spec:
  type: email
  email:
    smtpSecretRef:
      name: smtp-credentials
      namespace: cronjob-guardian
    from: [email protected]
    to:
      - [email protected]
      - [email protected]

Email Alert Format

Email alerts include structured information: Subject:

[CRITICAL] CronJob Failed: production/daily-backup

Body:

CronJob Failure Alert

Job Details:
- Name: daily-backup
- Namespace: production
- Execution: job-daily-backup-28501234
- Time: 2026-03-04 08:15:23 UTC
- Duration: 5m 32s
- Exit Code: 1

Error Details:
Exit Code: 1
Reason: Error

Recent Logs (last 50 lines):
----------------------------------------
Error: Failed to connect to database
Connection timeout after 30s
Host: postgres.production.svc.cluster.local:5432
...

Kubernetes Events:
----------------------------------------
[Warning] BackoffLimitExceeded: Job has reached the specified backoff limit

Suggested Fix:
----------------------------------------
Database connection failed. Check:
1. Database pod is running: kubectl get pods -n production -l app=postgres
2. Service is healthy: kubectl get svc postgres -n production
3. Network policies allow connection from this namespace

Monitor: database-backups (namespace: databases)
Dashboard: https://guardian.example.com/jobs/production/daily-backup

---
Sent by CronJob Guardian

Customizing Email Content

Control what appears in emails via the CronJobMonitor:

apiVersion: guardian.illenium.net/v1alpha1
kind: CronJobMonitor
metadata:
  name: detailed-emails
  namespace: production
spec:
  selector:
    matchLabels:
      tier: critical
  alerting:
    channelRefs:
      - name: email-team
    
    # Customize email content
    includeContext:
      logs: true              # Include pod logs
      logLines: 100           # Number of log lines
      events: true            # Include Kubernetes events
      podStatus: true         # Include pod status details
      suggestedFixes: true    # Include fix suggestions

Routing by Severity

Send critical alerts to one list, warnings to another:

apiVersion: guardian.illenium.net/v1alpha1
kind: CronJobMonitor
metadata:
  name: tiered-emails
  namespace: production
spec:
  selector:
    matchLabels:
      tier: critical
  deadManSwitch:
    enabled: true
    maxTimeSinceLastSuccess: 25h
  sla:
    enabled: true
    minSuccessRate: 95
  alerting:
    channelRefs:
      # Critical alerts to on-call list
      - name: email-oncall
        severities: [critical]
      
      # All alerts to ops team
      - name: email-ops
        severities: [critical, warning]

Testing Email Integration

Verify SMTP credentials

Test SMTP connectivity from within the cluster:

kubectl run -it --rm smtp-test --image=nicolaka/netshoot --restart=Never -- bash

# Inside the pod
telnet smtp.gmail.com 587
# Should connect successfully

Verify AlertChannel status

kubectl get alertchannel email-team
kubectl describe alertchannel email-team

Look for:

Status:
  Conditions:
    - Type: Ready
      Status: True
      Message: SMTP configured successfully

Create a test failing job

kubectl apply -f - <<EOF
apiVersion: batch/v1
kind: CronJob
metadata:
  name: test-email-alert
  namespace: production
  labels:
    tier: critical
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: test
              image: busybox
              command: ["sh", "-c", "echo 'Testing email alert' && exit 1"]
          restartPolicy: Never
EOF

Wait for email

After the job fails, check your inbox for the alert email.If no email arrives, check:

# Controller logs
kubectl logs -n cronjob-guardian deployment/cronjob-guardian-controller-manager | grep -i email

# AlertChannel status
kubectl get alertchannel email-team -o yaml

Clean up

kubectl delete cronjob test-email-alert -n production

Troubleshooting

No emails being sent

Check SMTP credentials:

kubectl get secret smtp-credentials -n cronjob-guardian -o yaml

Decode and verify the values:

kubectl get secret smtp-credentials -n cronjob-guardian -o jsonpath='{.data.host}' | base64 -d
kubectl get secret smtp-credentials -n cronjob-guardian -o jsonpath='{.data.port}' | base64 -d
kubectl get secret smtp-credentials -n cronjob-guardian -o jsonpath='{.data.username}' | base64 -d

Test SMTP connectivity:

kubectl run -it --rm smtp-test --image=nicolaka/netshoot --restart=Never -- \
  curl -v --url "smtp://smtp.gmail.com:587" \
  --mail-from "[email protected]" \
  --mail-rcpt "[email protected]"

Check controller logs:

kubectl logs -n cronjob-guardian deployment/cronjob-guardian-controller-manager | grep -i smtp

Authentication failed

Common causes:Gmail: Using regular password instead of App Password

Go to Google Account Settings
Generate an App Password
Update the secret with the App Password

Office 365: MFA required

If MFA is enabled, you may need an app-specific password
Or configure OAuth2 (requires additional setup)

Wrong username format

Some providers require full email (e.g., [email protected])
Others require just username (e.g., user)
Check your provider’s documentation

Update the secret:

kubectl create secret generic smtp-credentials \
  --namespace cronjob-guardian \
  --from-literal=host=smtp.gmail.com \
  --from-literal=port=587 \
  [email protected] \
  --from-literal=password=correct-password \
  --dry-run=client -o yaml | kubectl apply -f -

Emails marked as spam

To improve deliverability:Use a verified domain:

Send from a domain you control
Configure SPF, DKIM, and DMARC records
Use services like SendGrid or SES for better reputation

Avoid spam triggers:

Use clear, non-promotional subject lines
Include unsubscribe instructions (if applicable)
Maintain consistent sender addresses

Test with spam checkers:

Mail Tester
Check your domain’s reputation

Wrong port or TLS errors

Common SMTP ports:

587: STARTTLS (most common)
465: SSL/TLS (implicit)
25: Plain text (rarely used, often blocked)

If you get TLS errors, verify:

# Test TLS connection
openssl s_client -connect smtp.gmail.com:587 -starttls smtp

Update port in secret if needed:

kubectl patch secret smtp-credentials -n cronjob-guardian \
  --type merge \
  -p '{"data":{"port":"NTg3"}}' # 587 in base64

Too many emails

Enable rate limiting: CronJob Guardian doesn’t have built-in rate limiting for email, but you can:Use alert suppression in monitors:

alerting:
  suppressDuplicatesFor: 1h
  alertDelay: 5m

Filter by severity:

channelRefs:
  - name: email-team
    severities: [critical]  # Only critical, no warnings

Use digest emails: Consider configuring your email provider or a proxy to batch alerts into digest emails.

Best Practices

Use Service Accounts

Create dedicated email accounts for alerts (e.g., [email protected]) rather than personal emails.

Separate by Team

Create one AlertChannel per team with their own distribution list for clear ownership.

Use Mailing Lists

Send to mailing lists/groups (e.g., [email protected]) instead of individual addresses for easier management.

Include Context

Set includeContext.logs: true to provide debugging information directly in emails.

Clear Subject Lines

Email subjects include severity and job name for easy filtering and prioritization.

Backup Channel

Use email as a backup to Slack/PagerDuty in case primary channels are down.

Email Filtering Rules

Set up filters in your email client to organize alerts:

Gmail Filters

From: [email protected]
Subject: [CRITICAL]
→ Apply label: CronJob/Critical
→ Mark as important
→ Never send to spam

From: [email protected]
Subject: [WARNING]
→ Apply label: CronJob/Warning
→ Skip inbox (archive)

Outlook Rules

From: [email protected]
AND Subject contains: CRITICAL
→ Move to folder: Alerts/Critical
→ Mark as important

Thunderbird Filters

From contains: [email protected]
AND Subject contains: Failed
→ Tag as: Alert
→ Move to: Alerts folder

Combining Email with Other Channels

Email works best in combination with other alert channels:

apiVersion: guardian.illenium.net/v1alpha1
kind: CronJobMonitor
metadata:
  name: multi-channel
  namespace: production
spec:
  selector:
    matchLabels:
      tier: critical
  alerting:
    channelRefs:
      # Immediate notification
      - name: pagerduty-oncall
        severities: [critical]
      
      # Team chat
      - name: slack-ops
        severities: [critical, warning]
      
      # Email for record-keeping
      - name: email-ops
        severities: [critical, warning]

This provides:

PagerDuty: Immediate on-call escalation
Slack: Real-time team awareness and collaboration
Email: Permanent record and distribution to non-technical stakeholders

Next Steps

Slack Alerts

Set up Slack notifications

PagerDuty Alerts

Configure on-call escalation

Webhook Alerts

Integrate with custom systems

Alert Channels Reference

Complete API documentation

Monitors

Alert Channels

Overview

Quick Start

Basic Email AlertChannel

Configuration Options

Common SMTP Providers

Gmail

Office 365 / Outlook

Amazon SES

SendGrid

Generic SMTP Server

Multiple Email Lists

Email Alert Format

Customizing Email Content

Routing by Severity

Testing Email Integration

Troubleshooting

Best Practices

Use Service Accounts

Separate by Team

Use Mailing Lists

Include Context

Clear Subject Lines

Backup Channel

Email Filtering Rules

Gmail Filters

Outlook Rules

Thunderbird Filters

Combining Email with Other Channels

Next Steps

Slack Alerts

PagerDuty Alerts

Webhook Alerts

Alert Channels Reference

Build docs developers (and LLMs) love

Monitors

Alert Channels

​Overview

​Quick Start

​Basic Email AlertChannel

​Configuration Options

​Common SMTP Providers

​Gmail

​Office 365 / Outlook

​Amazon SES

​SendGrid

​Generic SMTP Server

​Multiple Email Lists

​Email Alert Format

​Customizing Email Content

​Routing by Severity

​Testing Email Integration

​Troubleshooting

​Best Practices

Use Service Accounts

Separate by Team

Use Mailing Lists

Include Context

Clear Subject Lines

Backup Channel

​Email Filtering Rules

​Gmail Filters

​Outlook Rules

​Thunderbird Filters

​Combining Email with Other Channels

​Next Steps

Slack Alerts

PagerDuty Alerts

Webhook Alerts

Alert Channels Reference

Build docs developers (and LLMs) love

Overview

Quick Start

Basic Email AlertChannel

Configuration Options

Common SMTP Providers

Gmail

Office 365 / Outlook

Amazon SES

SendGrid

Generic SMTP Server

Multiple Email Lists

Email Alert Format

Customizing Email Content

Routing by Severity

Testing Email Integration

Troubleshooting

Best Practices

Email Filtering Rules

Gmail Filters

Outlook Rules

Thunderbird Filters

Combining Email with Other Channels

Next Steps