Monitoring & Logs - llms.txt Generator

Overview

The Terraform deployment includes comprehensive monitoring with CloudWatch logs, metrics, alarms, and a dashboard. This section covers accessing logs, understanding alerts, and monitoring system health.

Monitoring Components

CloudWatch Logs

Application logs from ECS containers and Lambda functions

CloudWatch Metrics

CPU, memory, request counts, and response times

CloudWatch Alarms

10 automated alerts for critical issues

CloudWatch Dashboard

Visual overview of system health and performance

CloudWatch Log Groups

Terraform creates two log groups with 14-day retention:

ECS Application Logs

Log group: /ecs/llmstxt-api Contains:

FastAPI application logs
Crawling progress and results
Error messages and stack traces
HTTP request/response logs

Lambda Function Logs

Log group: /aws/lambda/llmstxt-auto-update Contains:

Lambda execution logs
Cron trigger events
Recrawl endpoint responses
Errors and timeouts

View Logs

Via AWS Console

Open CloudWatch Console

Navigate to CloudWatch Console

Access Log Groups

Click Logs → Log groups in left sidebar
Select log group:
- /ecs/llmstxt-api for application logs
- /aws/lambda/llmstxt-auto-update for Lambda logs

View Log Streams

Each container/Lambda execution creates a separate log stream:

ECS: ecs/llmstxt-api/[task-id]
Lambda: [date]/[execution-id]

Click a stream to view logs.

Filter Logs

Use the filter box to search:

[ERROR]

Or:

"Failed to crawl"

Via AWS CLI

Tail ECS Logs (Live)

aws logs tail /ecs/llmstxt-api \
  --follow \
  --format short \
  --region us-east-1

Press Ctrl+C to stop tailing. --follow keeps the stream open for new logs.

Tail Lambda Logs

aws logs tail /aws/lambda/llmstxt-auto-update \
  --follow \
  --format short \
  --region us-east-1

Filter for Errors

aws logs filter-log-events \
  --log-group-name /ecs/llmstxt-api \
  --filter-pattern "[ERROR]" \
  --start-time $(date -u -d '1 hour ago' +%s)000 \
  --region us-east-1

Get Recent Logs

# Last 100 lines
aws logs tail /ecs/llmstxt-api \
  --since 1h \
  --region us-east-1

# Last 5 minutes
aws logs tail /ecs/llmstxt-api \
  --since 5m \
  --region us-east-1

CloudWatch Metrics

Key Metrics

The deployment tracks these critical metrics:

ECS Service Metrics

CPUUtilization: Percentage of allocated CPU used
MemoryUtilization: Percentage of allocated memory used
RunningTaskCount: Number of active containers
DesiredTaskCount: Target number of containers

Application Load Balancer Metrics

RequestCount: Total HTTP requests
HTTPCode_Target_2XX_Count: Successful responses
HTTPCode_Target_5XX_Count: Server errors
TargetResponseTime: Average response time in seconds
UnHealthyHostCount: Number of failing targets
HealthyHostCount: Number of healthy targets

Lambda Function Metrics

Invocations: Number of executions
Errors: Failed executions
Duration: Execution time in milliseconds
Throttles: Rate-limited invocations

View Metrics in Console

Open CloudWatch Metrics

CloudWatch Console → Metrics → All metrics

Browse by Namespace

AWS/ECS: ECS service metrics
AWS/ApplicationELB: Load balancer metrics
AWS/Lambda: Lambda function metrics

Select Metrics

Choose namespace
Select dimension (e.g., Service/Cluster, LoadBalancer, Function)
Check metrics to graph

Customize Graph

Change time range (1h, 3h, 12h, 1d, 1w)
Adjust statistic (Average, Sum, Min, Max)
Set refresh interval

View Metrics via CLI

ECS CPU Utilization

aws cloudwatch get-metric-statistics \
  --namespace AWS/ECS \
  --metric-name CPUUtilization \
  --dimensions Name=ServiceName,Value=llmstxt-api-service Name=ClusterName,Value=llmstxt-cluster \
  --start-time $(date -u -d '1 hour ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 300 \
  --statistics Average \
  --region us-east-1

ALB Request Count

aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB \
  --metric-name RequestCount \
  --dimensions Name=LoadBalancer,Value=$(cd terraform && terraform output -raw alb_arn | cut -d: -f6) \
  --start-time $(date -u -d '1 hour ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 300 \
  --statistics Sum \
  --region us-east-1

CloudWatch Alarms

Terraform configures 10 alarms to detect and alert on critical issues.

Configured Alarms

Alarm Name	Metric	Threshold	Description
`llmstxt-ecs-no-running-tasks`	RunningTaskCount	< 1	ECS service has no active containers
`llmstxt-alb-unhealthy-targets`	UnHealthyHostCount	≥ 1	ALB has unhealthy targets
`llmstxt-alb-high-5xx-errors`	HTTPCode_Target_5XX	> 10 in 5 min	High server error rate
`llmstxt-lambda-errors`	Lambda Errors	≥ 1	Lambda function errors
`llmstxt-application-errors`	Custom log filter	> 5 in 5 min	Application ERROR logs
`llmstxt-ecs-high-cpu`	CPUUtilization	> 80% for 15 min	High CPU usage
`llmstxt-ecs-high-memory`	MemoryUtilization	> 85% for 15 min	High memory usage
`llmstxt-alb-high-response-time`	TargetResponseTime	> 5s for 10 min	Slow response times
`llmstxt-lambda-duration-high`	Lambda Duration	> 540s (9 min)	Lambda near timeout
`llmstxt-lambda-throttles`	Lambda Throttles	≥ 1	Lambda rate limited

View Alarm Status

Via Console

Go to CloudWatch Console → Alarms → All alarms
Filter by prefix: llmstxt-
Check alarm states:
- 🟢 OK: Normal operation
- 🔴 ALARM: Issue detected
- 🔵 INSUFFICIENT_DATA: Collecting data

Via CLI

# List all alarms
aws cloudwatch describe-alarms \
  --alarm-name-prefix llmstxt- \
  --region us-east-1

# Get alarm state
aws cloudwatch describe-alarms \
  --alarm-names llmstxt-ecs-no-running-tasks \
  --query 'MetricAlarms[0].StateValue' \
  --output text \
  --region us-east-1

Email Notifications

Alarms send notifications via Amazon SNS.

Confirm SNS Subscription

After Terraform deployment, check your email for:

Subject: “AWS Notification - Subscription Confirmation”
From: [email protected]

Click “Confirm subscription” link.

Verify Subscription

aws sns list-subscriptions-by-topic \
  --topic-arn $(cd terraform && terraform output -raw sns_topic_arn) \
  --region us-east-1

Check SubscriptionArn is not PendingConfirmation.

Receive Alerts

When an alarm triggers, you’ll receive email:

ALARM state: Issue detected
OK state: Issue resolved

You won’t receive alert emails until SNS subscription is confirmed!

Add Additional Email Recipients

aws sns subscribe \
  --topic-arn $(cd terraform && terraform output -raw sns_topic_arn) \
  --protocol email \
  --notification-endpoint [email protected] \
  --region us-east-1

Recipient must confirm subscription via email.

CloudWatch Dashboard

Terraform creates a dashboard named llmstxt-overview with key metrics.

Access Dashboard

Go to CloudWatch Console → Dashboards
Click llmstxt-overview

Dashboard Widgets

ECS Service - CPU & Memory

CPU utilization percentage (0-100%)
Memory utilization percentage (0-100%)
5-minute intervals

ALB - Requests & Errors

Total request count
2xx success responses
5xx error responses
5-minute intervals

Lambda - Invocations & Errors

Total invocations
Error count
1-hour intervals

Lambda - Duration

Average duration (ms)
Maximum duration (ms)
1-hour intervals

Customize Dashboard

Add custom widgets:

Click Actions → Add widget
Choose widget type (Line, Number, etc.)
Select metrics
Click Create widget
Click Save dashboard

Application Error Log Filter

Terraform creates a metric filter to count ERROR log entries.

View Error Metric

aws cloudwatch get-metric-statistics \
  --namespace LLMsTxt/Application \
  --metric-name ApplicationErrors \
  --start-time $(date -u -d '1 hour ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 300 \
  --statistics Sum \
  --region us-east-1

Modify Filter Pattern

Edit terraform/monitoring.tf:

resource "aws_cloudwatch_log_metric_filter" "ecs_application_errors" {
  name           = "llmstxt-ecs-application-errors"
  log_group_name = aws_cloudwatch_log_group.ecs_logs.name
  pattern        = "[ERROR]"  # Change this pattern

  metric_transformation {
    name      = "ApplicationErrors"
    namespace = "LLMsTxt/Application"
    value     = "1"
  }
}

Apply changes:

cd terraform
terraform apply

Performance Monitoring

ECS Task Performance

Monitor container resource usage:

# Get task ARN
TASK_ARN=$(aws ecs list-tasks \
  --cluster llmstxt-cluster \
  --service-name llmstxt-api-service \
  --query 'taskArns[0]' \
  --output text \
  --region us-east-1)

# Get task metrics
aws ecs describe-tasks \
  --cluster llmstxt-cluster \
  --tasks $TASK_ARN \
  --query 'tasks[0].containers[0].{CPU:cpu,Memory:memory,MemoryReservation:memoryReservation}' \
  --region us-east-1

ALB Performance

Check response times and throughput:

# Average response time (last hour)
aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB \
  --metric-name TargetResponseTime \
  --dimensions Name=LoadBalancer,Value=... \
  --start-time $(date -u -d '1 hour ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 300 \
  --statistics Average Maximum \
  --region us-east-1

Log Retention

By default, logs are retained for 14 days.

Change Retention Period

Edit terraform/main.tf and terraform/ecs.tf:

resource "aws_cloudwatch_log_group" "ecs_logs" {
  name              = "/ecs/llmstxt-api"
  retention_in_days = 30  # Change from 14 to 30 days
}

resource "aws_cloudwatch_log_group" "lambda_logs" {
  name              = "/aws/lambda/llmstxt-auto-update"
  retention_in_days = 30  # Change from 14 to 30 days
}

Apply changes:

cd terraform
terraform apply

Longer retention increases CloudWatch Logs costs. 14 days is recommended for production.

Troubleshooting with Logs

Common Issues

High 5xx error rate

Check application logs for errors:

aws logs filter-log-events \
  --log-group-name /ecs/llmstxt-api \
  --filter-pattern "[ERROR]" \
  --start-time $(date -u -d '1 hour ago' +%s)000

Common causes:

Database connection failures (Supabase)
R2 storage authentication errors
Invalid environment variables

ECS task keeps restarting

Check stopped task reason:

aws ecs describe-tasks \
  --cluster llmstxt-cluster \
  --tasks $TASK_ARN \
  --query 'tasks[0].stoppedReason'

Then check logs for the stopped task.

Lambda timeouts

Check Lambda duration metric:

aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Duration \
  --dimensions Name=FunctionName,Value=llmstxt-auto-update \
  --start-time $(date -u -d '6 hours ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 3600 \
  --statistics Maximum Average

If consistently near 600000ms (10 min), increase timeout in terraform/main.tf.

Cost Optimization

Reduce Log Costs

Decrease retention period (7 days instead of 14)
Reduce log verbosity in application
Use log sampling for high-volume debug logs

Monitor CloudWatch Costs

# Get CloudWatch Logs usage
aws cloudwatch get-metric-statistics \
  --namespace AWS/Logs \
  --metric-name IncomingBytes \
  --start-time $(date -u -d '1 month ago' --iso-8601=seconds) \
  --end-time $(date -u --iso-8601=seconds) \
  --period 2592000 \
  --statistics Sum \
  --region us-east-1

CloudWatch Logs pricing: ~

0.50 per GB ingested,

0.03 per GB stored.

Summary

You now have comprehensive monitoring with:

Real-time logs accessible via Console and CLI
10 CloudWatch alarms for critical issues
Email notifications via SNS
Visual dashboard for system health
Metric tracking for performance analysis

Complete Deployment

Your llms.txt Generator is fully deployed and monitored! Return to the Deployment Overview for next steps.

Get Started

Core Features

Guides

Deployment

​Overview

​Monitoring Components

CloudWatch Logs

CloudWatch Metrics

CloudWatch Alarms

CloudWatch Dashboard

​CloudWatch Log Groups

​ECS Application Logs

​Lambda Function Logs

​View Logs

​Via AWS Console

​Via AWS CLI

​Tail ECS Logs (Live)

​Tail Lambda Logs

​Filter for Errors

​Get Recent Logs

​CloudWatch Metrics

​Key Metrics

​View Metrics in Console

​View Metrics via CLI

​ECS CPU Utilization

​ALB Request Count

​CloudWatch Alarms

​Configured Alarms

​View Alarm Status

​Via Console

​Via CLI

​Email Notifications

​Add Additional Email Recipients

​CloudWatch Dashboard

​Access Dashboard

​Dashboard Widgets

​Customize Dashboard

​Application Error Log Filter

​View Error Metric

​Modify Filter Pattern

​Performance Monitoring

​ECS Task Performance

​ALB Performance

​Log Retention

​Change Retention Period

​Troubleshooting with Logs

​Common Issues

​Cost Optimization

​Reduce Log Costs

​Monitor CloudWatch Costs

​Summary

Complete Deployment

Build docs developers (and LLMs) love

Overview

Monitoring Components

CloudWatch Log Groups

ECS Application Logs

Lambda Function Logs

View Logs

Via AWS Console

Via AWS CLI

Tail ECS Logs (Live)

Tail Lambda Logs

Filter for Errors

Get Recent Logs

CloudWatch Metrics

Key Metrics

View Metrics in Console

View Metrics via CLI

ECS CPU Utilization

ALB Request Count

CloudWatch Alarms

Configured Alarms

View Alarm Status

Via Console

Via CLI

Email Notifications

Add Additional Email Recipients

CloudWatch Dashboard

Access Dashboard

Dashboard Widgets

Customize Dashboard

Application Error Log Filter

View Error Metric

Modify Filter Pattern

Performance Monitoring

ECS Task Performance

ALB Performance

Log Retention

Change Retention Period

Troubleshooting with Logs

Common Issues

Cost Optimization

Reduce Log Costs

Monitor CloudWatch Costs

Summary