Health Checks - AspNetRun Microservices

Overview

Health checks are implemented across all API services to monitor the health of the application and its dependencies. The system uses ASP.NET Core Health Checks with custom UI responses for detailed diagnostics.

Health Check Endpoints

All services expose a health check endpoint at:

GET /health

Response Format: JSON (using HealthChecks.UI.Client.UIResponseWriter)

Example Response

{
  "status": "Healthy",
  "totalDuration": "00:00:00.0123456",
  "entries": {
    "npgsql": {
      "status": "Healthy",
      "duration": "00:00:00.0098765",
      "data": {}
    }
  }
}

Status Values:

Healthy: All checks passed
Degraded: Some checks are in a degraded state
Unhealthy: One or more checks failed

Service Health Check Configuration

Catalog API

File: Services/Catalog/Catalog.API/Program.cs:28-29

builder.Services.AddHealthChecks()
    .AddNpgSql(builder.Configuration.GetConnectionString("Database")!);

app.UseHealthChecks("/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });

Checks:

PostgreSQL Database: Verifies connection to CatalogDb

Dependencies:

NuGet Package: AspNetCore.HealthChecks.NpgSql

Basket API

File: Services/Basket/Basket.API/Program.cs:58-60

builder.Services.AddHealthChecks()
    .AddNpgSql(builder.Configuration.GetConnectionString("Database")!)
    .AddRedis(builder.Configuration.GetConnectionString("Redis")!);

app.UseHealthChecks("/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });

Checks:

PostgreSQL Database: Verifies connection to BasketDb
Redis Cache: Verifies connection to distributed cache

Dependencies:

NuGet Package: AspNetCore.HealthChecks.NpgSql
NuGet Package: AspNetCore.HealthChecks.Redis

Ordering API

File: Services/Ordering/Ordering.API/DependencyInjection.cs:15-16

services.AddHealthChecks()
    .AddSqlServer(configuration.GetConnectionString("Database")!);

app.UseHealthChecks("/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });

Checks:

SQL Server Database: Verifies connection to OrderDb

Dependencies:

NuGet Package: AspNetCore.HealthChecks.SqlServer

Discount gRPC

Status: No health checks currently implemented Note: Consider adding SQLite health checks for production deployments:

builder.Services.AddHealthChecks()
    .AddSqlite(builder.Configuration.GetConnectionString("Database")!);

YARP API Gateway

Status: No health checks currently implemented Recommendation: Add downstream service health checks:

builder.Services.AddHealthChecks()
    .AddUrlGroup(new Uri("http://catalog.api:8080/health"), "Catalog Service")
    .AddUrlGroup(new Uri("http://basket.api:8080/health"), "Basket Service")
    .AddUrlGroup(new Uri("http://ordering.api:8080/health"), "Ordering Service");

Shopping Web

Status: No health checks currently implemented Recommendation: Add gateway health check:

builder.Services.AddHealthChecks()
    .AddUrlGroup(
        new Uri($"{builder.Configuration["ApiSettings:GatewayAddress"]}/health"),
        "API Gateway"
    );

Required NuGet Packages

Install the appropriate health check packages for each service:

<!-- PostgreSQL -->
<PackageReference Include="AspNetCore.HealthChecks.NpgSql" Version="8.0.0" />

<!-- SQL Server -->
<PackageReference Include="AspNetCore.HealthChecks.SqlServer" Version="8.0.0" />

<!-- SQLite -->
<PackageReference Include="AspNetCore.HealthChecks.Sqlite" Version="8.0.0" />

<!-- Redis -->
<PackageReference Include="AspNetCore.HealthChecks.Redis" Version="8.0.0" />

<!-- RabbitMQ -->
<PackageReference Include="AspNetCore.HealthChecks.RabbitMQ" Version="8.0.0" />

<!-- URL Group (for downstream services) -->
<PackageReference Include="AspNetCore.HealthChecks.Uris" Version="8.0.0" />

<!-- UI Response Writer -->
<PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="8.0.0" />

Testing Health Checks

Using curl

# Catalog API
curl http://localhost:6000/health

# Basket API
curl http://localhost:6001/health

# Ordering API
curl http://localhost:6003/health

Using Docker

# Inside Docker network
curl http://catalog.api:8080/health
curl http://basket.api:8080/health
curl http://ordering.api:8080/health

Expected Healthy Response

{
  "status": "Healthy",
  "totalDuration": "00:00:00.0234567",
  "entries": {
    "npgsql": {
      "status": "Healthy",
      "duration": "00:00:00.0123456",
      "data": {}
    },
    "redis": {
      "status": "Healthy",
      "duration": "00:00:00.0098765",
      "data": {}
    }
  }
}

Example Unhealthy Response

{
  "status": "Unhealthy",
  "totalDuration": "00:00:05.0000000",
  "entries": {
    "npgsql": {
      "status": "Unhealthy",
      "duration": "00:00:05.0000000",
      "exception": "Npgsql.NpgsqlException: Connection refused",
      "data": {}
    }
  }
}

Docker Health Checks

Add health check configuration to docker-compose.override.yml:

Example Configuration

catalog.api:
  environment:
    - ASPNETCORE_ENVIRONMENT=Development
    - ConnectionStrings__Database=Server=catalogdb;Port=5432;Database=CatalogDb;User Id=postgres;Password=postgres
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
    interval: 30s
    timeout: 10s
    retries: 3
    start_period: 40s
  depends_on:
    catalogdb:
      condition: service_healthy

catalogdb:
  image: postgres
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U postgres"]
    interval: 10s
    timeout: 5s
    retries: 5

Health Check Options

Option	Description	Default
`test`	Command to run	None
`interval`	Time between checks	30s
`timeout`	Max time for check	30s
`retries`	Consecutive failures before unhealthy	3
`start_period`	Initial grace period	0s

Kubernetes Liveness and Readiness Probes

Deployment Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog-api
spec:
  template:
    spec:
      containers:
      - name: catalog-api
        image: catalogapi:latest
        ports:
        - containerPort: 8080
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 3
          failureThreshold: 3

Probe Types

Liveness Probe:

Determines if the container should be restarted
Unhealthy containers are killed and restarted

Readiness Probe:

Determines if the container is ready to accept traffic
Unhealthy containers are removed from service load balancing

Advanced Health Check Configuration

Custom Health Checks

Create custom health checks for business logic:

public class DatabaseMigrationHealthCheck : IHealthCheck
{
    private readonly ApplicationDbContext _context;

    public DatabaseMigrationHealthCheck(ApplicationDbContext context)
    {
        _context = context;
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            var pendingMigrations = await _context.Database
                .GetPendingMigrationsAsync(cancellationToken);

            if (pendingMigrations.Any())
            {
                return HealthCheckResult.Degraded(
                    "Database has pending migrations");
            }

            return HealthCheckResult.Healthy("Database is up to date");
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Unhealthy(
                "Database migration check failed", ex);
        }
    }
}

// Registration
builder.Services.AddHealthChecks()
    .AddCheck<DatabaseMigrationHealthCheck>("database_migrations");

Health Check Tags

Organize health checks with tags:

builder.Services.AddHealthChecks()
    .AddNpgSql(
        builder.Configuration.GetConnectionString("Database")!,
        name: "database",
        tags: new[] { "db", "postgres", "ready" })
    .AddRedis(
        builder.Configuration.GetConnectionString("Redis")!,
        name: "cache",
        tags: new[] { "cache", "redis", "ready" });

// Filter by tags
app.UseHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready"),
    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});

app.UseHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false, // No checks, just responds 200 OK
});

Separate Endpoints

// Liveness: Is the app running?
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false
});

// Readiness: Is the app ready to serve traffic?
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready"),
    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});

// Startup: Has the app initialized?
app.MapHealthChecks("/health/startup", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("startup"),
    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});

Monitoring and Alerting

Health Checks UI

Install the Health Checks UI for dashboard monitoring:

<PackageReference Include="AspNetCore.HealthChecks.UI" Version="8.0.0" />
<PackageReference Include="AspNetCore.HealthChecks.UI.InMemory.Storage" Version="8.0.0" />

Configuration:

builder.Services
    .AddHealthChecksUI(setup =>
    {
        setup.SetEvaluationTimeInSeconds(30);
        setup.MaximumHistoryEntriesPerEndpoint(50);
        setup.AddHealthCheckEndpoint("Catalog API", "http://catalog.api:8080/health");
        setup.AddHealthCheckEndpoint("Basket API", "http://basket.api:8080/health");
        setup.AddHealthCheckEndpoint("Ordering API", "http://ordering.api:8080/health");
    })
    .AddInMemoryStorage();

app.MapHealthChecksUI(options => options.UIPath = "/health-ui");

Access the UI at: http://localhost:6004/health-ui

Integration with Monitoring Tools

Prometheus Metrics:

builder.Services.AddHealthChecks()
    .ForwardToPrometheus();

Application Insights:

builder.Services.AddApplicationInsightsTelemetry();
builder.Services.Configure<HealthCheckPublisherOptions>(options =>
{
    options.Predicate = _ => true;
});

Troubleshooting

Health Check Returns 503

Cause: One or more dependency checks are failing.Solution:

Check the JSON response for specific failed checks
Verify database connection strings
Ensure dependent services are running
Check network connectivity between services

Health Check Timeout

Cause: Health check is taking too long to respond.Solution:

Increase timeout in Docker/Kubernetes configuration
Optimize database queries used in health checks
Check for network latency issues

Container Keeps Restarting

Cause: Liveness probe is failing repeatedly.Solution:

Increase initialDelaySeconds to allow app startup time
Increase failureThreshold to tolerate transient failures
Check application logs for startup errors

Service Not Receiving Traffic

Cause: Readiness probe is failing.Solution:

Verify all dependencies are available
Check that migrations have completed
Ensure cache/message broker connections are established

Best Practices

Keep Checks Lightweight

Health checks should complete quickly (< 1 second). Avoid expensive operations.

Check All Dependencies

Include checks for databases, caches, message brokers, and downstream services.

Use Separate Endpoints

Implement distinct /health/live, /health/ready, and /health/startup endpoints.

Include Detailed Responses

Use UIResponseWriter for detailed JSON responses with timing and error information.

Docker

Configuration

Operations

​Overview

​Health Check Endpoints

​Example Response

​Service Health Check Configuration

​Catalog API

​Basket API

​Ordering API

​Discount gRPC

​YARP API Gateway

​Shopping Web

​Required NuGet Packages

​Testing Health Checks

​Using curl

​Using Docker

​Expected Healthy Response

​Example Unhealthy Response

​Docker Health Checks

​Example Configuration

​Health Check Options

​Kubernetes Liveness and Readiness Probes

​Deployment Example

​Probe Types

​Advanced Health Check Configuration

​Custom Health Checks

​Health Check Tags

​Separate Endpoints

​Monitoring and Alerting

​Health Checks UI

​Integration with Monitoring Tools

​Troubleshooting

​Best Practices

Keep Checks Lightweight

Check All Dependencies

Use Separate Endpoints

Include Detailed Responses

​Additional Resources

Build docs developers (and LLMs) love

Overview

Health Check Endpoints

Example Response

Service Health Check Configuration

Catalog API

Basket API

Ordering API

Discount gRPC

YARP API Gateway

Shopping Web

Required NuGet Packages

Testing Health Checks

Using curl

Using Docker

Expected Healthy Response

Example Unhealthy Response

Docker Health Checks

Example Configuration

Health Check Options

Kubernetes Liveness and Readiness Probes

Deployment Example

Probe Types

Advanced Health Check Configuration

Custom Health Checks

Health Check Tags

Separate Endpoints

Monitoring and Alerting

Health Checks UI

Integration with Monitoring Tools

Troubleshooting

Best Practices

Additional Resources