Skip to main content

Alerts

Alert rules allow you to monitor Mission Control entities (agents, tasks, sessions, activities) and trigger notifications when specific conditions are met. Configure automated alerts for agent failures, stuck tasks, or unusual activity patterns.

List Alert Rules

curl http://localhost:3000/api/alerts \
  -H "x-api-key: YOUR_API_KEY"

Response

rules
array
Array of alert rule configurations
id
integer
Alert rule ID
name
string
Rule name
description
string
Human-readable description
enabled
boolean
Whether rule is active
entity_type
string
Entity to monitor: agent, task, session, or activity
condition_field
string
Field to evaluate (e.g., status, priority, assigned_to)
condition_operator
string
Comparison operator: equals, not_equals, greater_than, less_than, contains, count_above, count_below, age_minutes_above
condition_value
string
Value to compare against
action_type
string
Action to take when triggered (default: notification)
action_config
object
Configuration for the action (e.g., {"recipient": "admin"})
cooldown_minutes
integer
Minimum time between triggers (default: 60 minutes)
last_triggered_at
integer
Unix timestamp of last trigger (null if never triggered)
trigger_count
integer
Total number of times this rule has triggered
created_by
string
Username who created the rule
created_at
integer
Unix timestamp

Create Alert Rule

curl -X POST http://localhost:3000/api/alerts \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "name": "Agent Offline Alert",
    "description": "Alert when agents are offline",
    "entity_type": "agent",
    "condition_field": "status",
    "condition_operator": "equals",
    "condition_value": "offline",
    "action_type": "notification",
    "action_config": {"recipient": "ops-team"},
    "cooldown_minutes": 30
  }'

Request Body

name
string
required
Rule name for identification
description
string
Human-readable description of the rule
entity_type
string
required
Entity to monitor:
  • agent - Monitor agent status, activity
  • task - Monitor task status, priority, assignments
  • session - Monitor gateway sessions
  • activity - Monitor activity log entries
condition_field
string
required
Field to evaluate. Allowed fields per entity type:
  • Agent: status, role, name, last_seen, last_activity
  • Task: status, priority, assigned_to, title
  • Activity: type, actor, entity_type
condition_operator
string
required
Comparison operator:
  • equals - Exact match
  • not_equals - Not equal
  • greater_than - Numeric comparison
  • less_than - Numeric comparison
  • contains - Substring match (case-insensitive)
  • count_above - Count of matching entities exceeds value
  • count_below - Count of matching entities below value
  • age_minutes_above - Field timestamp older than N minutes
condition_value
string
required
Value to compare against (stringified)
action_type
string
default:"notification"
Action to take when rule triggers
action_config
object
Configuration for the action. For notifications: {"recipient": "username"}
cooldown_minutes
integer
default:60
Minimum minutes between rule triggers (prevents spam)

Response

rule
object
Created alert rule object (see List response for schema)

Update Alert Rule

curl -X PUT http://localhost:3000/api/alerts \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "id": 1,
    "enabled": false,
    "cooldown_minutes": 120
  }'

Request Body

id
integer
required
Alert rule ID to update
name
string
New rule name
description
string
New description
enabled
boolean
Enable or disable rule
entity_type
string
Update entity type
condition_field
string
Update field to monitor
condition_operator
string
Update comparison operator
condition_value
string
Update comparison value
action_type
string
Update action type
action_config
object
Update action configuration
cooldown_minutes
integer
Update cooldown period

Delete Alert Rule

curl -X DELETE http://localhost:3000/api/alerts \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{"id": 1}'

Request Body

id
integer
required
Alert rule ID to delete

Evaluate Rules Manually

Trigger immediate evaluation of all enabled alert rules.
curl -X POST http://localhost:3000/api/alerts \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{"action": "evaluate"}'

Request Body

action
string
required
Must be evaluate

Response

evaluated
integer
Number of rules evaluated
triggered
integer
Number of rules that triggered
results
array
rule_id
integer
Alert rule ID
rule_name
string
Rule name
triggered
boolean
Whether rule triggered
reason
string
Result reason (e.g., “Condition met”, “In cooldown”)

Rule Examples

Alert on Agent Errors

{
  "name": "Agent Error Alert",
  "entity_type": "agent",
  "condition_field": "status",
  "condition_operator": "equals",
  "condition_value": "error",
  "action_type": "notification",
  "action_config": {"recipient": "ops-team"},
  "cooldown_minutes": 15
}

Alert on High Priority Tasks

{
  "name": "Critical Task Alert",
  "entity_type": "task",
  "condition_field": "priority",
  "condition_operator": "equals",
  "condition_value": "critical",
  "action_type": "notification",
  "cooldown_minutes": 60
}

Alert on Too Many Tasks

{
  "name": "Task Backlog Alert",
  "entity_type": "task",
  "condition_field": "status",
  "condition_operator": "count_above",
  "condition_value": "50",
  "action_type": "notification",
  "cooldown_minutes": 120
}

Alert on Stale Agent Activity

{
  "name": "Stale Agent Alert",
  "entity_type": "agent",
  "condition_field": "last_seen",
  "condition_operator": "age_minutes_above",
  "condition_value": "30",
  "action_type": "notification",
  "cooldown_minutes": 60
}

Alert on Activity Spikes

{
  "name": "High Activity Alert",
  "entity_type": "activity",
  "condition_field": "type",
  "condition_operator": "count_above",
  "condition_value": "100",
  "action_type": "notification",
  "cooldown_minutes": 30
}
The count_above operator for activities checks the count in the last hour only.

Condition Operators

Comparison Operators

OperatorDescriptionExample
equalsExact matchstatus = "error"
not_equalsNot equalstatus != "online"
greater_thanNumeric greater thanpriority > 5
less_thanNumeric less thanestimated_hours less than 2
containsSubstring (case-insensitive)title contains "urgent"

Aggregate Operators

OperatorDescriptionExample
count_aboveCount exceeds valueCount of offline agents > 3
count_belowCount below valueCount of in-progress tasks < 5
age_minutes_aboveField timestamp older than N minuteslast_seen > 30 minutes ago

Cooldown Behavior

Cooldown prevents alert spam:
  1. Rule triggers → creates notification
  2. last_triggered_at timestamp is set
  3. Rule cannot trigger again until cooldown_minutes elapses
  4. During cooldown, evaluation returns "In cooldown"
Example: A rule with cooldown_minutes: 60 can only trigger once per hour, even if the condition remains true.

Notifications

When an alert rule triggers, a notification is created:
{
  "recipient": "ops-team",
  "type": "alert",
  "title": "Alert: Agent Error Alert",
  "message": "Rule 'Agent Error Alert' triggered",
  "source_type": "alert_rule",
  "source_id": 1
}
Notifications appear in:
  • /api/notifications endpoint
  • Real-time SSE stream (/api/events)
  • Mission Control dashboard UI

Automatic Evaluation

Alert rules are evaluated automatically by the scheduler:
  • Frequency: Every 5 minutes (configurable via MC_ALERT_EVAL_INTERVAL)
  • Scope: All enabled rules across all workspaces
  • Cooldown respected: Rules in cooldown are skipped
You can also trigger manual evaluation using POST /api/alerts with {"action": "evaluate"}.

Security Considerations

  • SQL Injection Protection: Only whitelisted columns are allowed in condition_field
  • Role Requirements:
    • Create/Update: operator role
    • Delete: admin role
    • List/Evaluate: viewer role
  • Workspace Isolation: Rules only evaluate entities in their workspace

Best Practices

  1. Use appropriate cooldowns - Balance responsiveness vs. noise
  2. Test with evaluate - Manually trigger evaluation during setup
  3. Monitor trigger counts - High counts may indicate misconfigured rules
  4. Combine with webhooks - Use alerts + webhooks for external integrations
  5. Name descriptively - Use clear names like “Agent Offline > 30min”

Rate Limits

  • Creation/updates: 100 requests/minute per API key
  • Evaluation: Manual evaluation limited to 1 request/10 seconds
  • Automatic evaluation: Every 5 minutes (not user-controlled)
Alert rules that trigger frequently (due to low cooldown or persistent conditions) can generate many notifications. Monitor your notification volume.