Skip to main content

​
Overview

The metrics middleware collects OpenTelemetry metrics for HTTP and gRPC services. It records request counts and latency histograms with labels for operation, code, and reason.

​
Installation

go get github.com/go-kratos/kratos/v2/middleware/metrics

​
Server Middleware

The Server function creates a server-side metrics middleware: 
func Server(opts ...Option) middleware.Middleware

​
Basic Usage

import (
    "github.com/go-kratos/kratos/v2/middleware/metrics"
    "github.com/go-kratos/kratos/v2/transport/http"
    
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/prometheus"
    "go.opentelemetry.io/otel/sdk/metric"
)

func main() {
    // Create Prometheus exporter
    exporter, err := prometheus.New()
    if err != nil {
        log.Fatal(err)
    }
    
    // Create meter provider
    provider := metric.NewMeterProvider(metric.WithReader(exporter))
    otel.SetMeterProvider(provider)
    
    // Get meter
    meter := provider.Meter("github.com/go-kratos/kratos/v2")
    
    // Create metrics
    counter, _ := metrics.DefaultRequestsCounter(meter, metrics.DefaultServerRequestsCounterName)
    histogram, _ := metrics.DefaultSecondsHistogram(meter, metrics.DefaultServerSecondsHistogramName)
    
    // Create HTTP server with metrics
    httpSrv := http.NewServer(
        http.Address(":8000"),
        http.Middleware(
            metrics.Server(
                metrics.WithRequests(counter),
                metrics.WithSeconds(histogram),
            ),
        ),
    )
    
    // Expose metrics endpoint
    http.HandleFunc("/metrics", func(w http.ResponseWriter, r *http.Request) {
        exporter.ServeHTTP(w, r)
    })
    
    app := kratos.New(
        kratos.Server(httpSrv),
    )
    
    if err := app.Run(); err != nil {
        log.Fatal(err)
    }
}

​
Client Middleware

The Client function creates a client-side metrics middleware: 
func Client(opts ...Option) middleware.Middleware

​
Usage Example

import (
    "github.com/go-kratos/kratos/v2/middleware/metrics"
    "github.com/go-kratos/kratos/v2/transport/http"
)

// Get meter
meter := otel.Meter("my-client")

// Create client metrics
counter, _ := metrics.DefaultRequestsCounter(meter, metrics.DefaultClientRequestsCounterName)
histogram, _ := metrics.DefaultSecondsHistogram(meter, metrics.DefaultClientSecondsHistogramName)

// Create HTTP client with metrics
conn, err := http.NewClient(
    context.Background(),
    http.WithEndpoint("127.0.0.1:8000"),
    http.WithMiddleware(
        metrics.Client(
            metrics.WithRequests(counter),
            metrics.WithSeconds(histogram),
        ),
    ),
)

​
Configuration Options

​
WithRequests

Set a custom request counter: 
import "go.opentelemetry.io/otel/metric"

// Create custom counter
counter, err := meter.Int64Counter(
    "my_service_requests_total",
    metric.WithUnit("{call}"),
)

metrics.Server(
    metrics.WithRequests(counter),
)

​
WithSeconds

Set a custom latency histogram: 
import "go.opentelemetry.io/otel/metric"

// Create custom histogram
histogram, err := meter.Float64Histogram(
    "my_service_request_duration_seconds",
    metric.WithUnit("s"),
    metric.WithExplicitBucketBoundaries(0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10),
)

metrics.Server(
    metrics.WithSeconds(histogram),
)
The latency is recorded in seconds by default.

​
Default Metrics

The middleware provides helper functions to create default metrics:

​
DefaultRequestsCounter

Creates a request counter with default configuration: 
func DefaultRequestsCounter(meter metric.Meter, histogramName string) (metric.Int64Counter, error)
Default metric names:
  • Server: server_requests_code_total
  • Client: client_requests_code_total
counter, err := metrics.DefaultRequestsCounter(
    meter,
    metrics.DefaultServerRequestsCounterName,
)

​
DefaultSecondsHistogram

Creates a latency histogram with default buckets: 
func DefaultSecondsHistogram(meter metric.Meter, histogramName string) (metric.Float64Histogram, error)
Default metric names:
  • Server: server_requests_seconds
  • Client: client_requests_seconds
Default buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1] seconds 
histogram, err := metrics.DefaultSecondsHistogram(
    meter,
    metrics.DefaultServerSecondsHistogramName,
)

​
DefaultSecondsHistogramView

Creates a view for customizing histogram aggregation: 
func DefaultSecondsHistogramView(histogramName string) metricsdk.View

import sdkmetric "go.opentelemetry.io/otel/sdk/metric"

// Create view
view := metrics.DefaultSecondsHistogramView(metrics.DefaultServerSecondsHistogramName)

// Register with meter provider
mp := sdkmetric.NewMeterProvider(
    sdkmetric.WithReader(exporter),
    sdkmetric.WithView(view),
)
otel.SetMeterProvider(mp)

​
Metric Labels

The middleware automatically adds the following labels to all metrics:
LabelDescriptionExample
kindTransport kind"grpc", "http"
operationOperation name"/api.v1.Greeter/SayHello"
codeResponse code200, 500
reasonError reason"INTERNAL_ERROR", ""

​
Metric Constants

The middleware defines the following constants for metric names: 
const (
    DefaultServerSecondsHistogramName = "server_requests_seconds"
    DefaultServerRequestsCounterName  = "server_requests_code_total"
    DefaultClientSecondsHistogramName = "client_requests_seconds"
    DefaultClientRequestsCounterName  = "client_requests_code_total"
)

​
Complete Example with Prometheus

package main

import (
    "context"
    "log"
    "net/http"
    
    "github.com/go-kratos/kratos/v2"
    "github.com/go-kratos/kratos/v2/middleware/metrics"
    "github.com/go-kratos/kratos/v2/middleware/recovery"
    httpTransport "github.com/go-kratos/kratos/v2/transport/http"
    
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/prometheus"
    "go.opentelemetry.io/otel/sdk/metric"
)

func main() {
    // Create Prometheus exporter
    exporter, err := prometheus.New()
    if err != nil {
        log.Fatal(err)
    }
    
    // Create meter provider with histogram view
    view := metrics.DefaultSecondsHistogramView(metrics.DefaultServerSecondsHistogramName)
    provider := metric.NewMeterProvider(
        metric.WithReader(exporter),
        metric.WithView(view),
    )
    otel.SetMeterProvider(provider)
    
    // Get meter
    meter := provider.Meter("github.com/go-kratos/kratos/v2")
    
    // Create metrics
    counter, err := metrics.DefaultRequestsCounter(
        meter,
        metrics.DefaultServerRequestsCounterName,
    )
    if err != nil {
        log.Fatal(err)
    }
    
    histogram, err := metrics.DefaultSecondsHistogram(
        meter,
        metrics.DefaultServerSecondsHistogramName,
    )
    if err != nil {
        log.Fatal(err)
    }
    
    // Create HTTP server with metrics
    httpSrv := httpTransport.NewServer(
        httpTransport.Address(":8000"),
        httpTransport.Middleware(
            recovery.Recovery(),
            metrics.Server(
                metrics.WithRequests(counter),
                metrics.WithSeconds(histogram),
            ),
        ),
    )
    
    // Serve metrics on separate port
    go func() {
        http.Handle("/metrics", exporter)
        log.Println("Metrics available at http://localhost:9090/metrics")
        if err := http.ListenAndServe(":9090", nil); err != nil {
            log.Fatal(err)
        }
    }()
    
    app := kratos.New(
        kratos.Name("metrics-example"),
        kratos.Server(httpSrv),
    )
    
    if err := app.Run(); err != nil {
        log.Fatal(err)
    }
}

​
Example Metrics Output

When you access the /metrics endpoint, you’ll see metrics like: 
# HELP server_requests_code_total
# TYPE server_requests_code_total counter
server_requests_code_total{code="200",kind="http",operation="/api/hello",reason=""} 42
server_requests_code_total{code="500",kind="http",operation="/api/error",reason="INTERNAL_ERROR"} 3

# HELP server_requests_seconds
# TYPE server_requests_seconds histogram
server_requests_seconds_bucket{kind="http",operation="/api/hello",le="0.005"} 10
server_requests_seconds_bucket{kind="http",operation="/api/hello",le="0.01"} 25
server_requests_seconds_bucket{kind="http",operation="/api/hello",le="0.025"} 38
server_requests_seconds_bucket{kind="http",operation="/api/hello",le="0.05"} 40
server_requests_seconds_bucket{kind="http",operation="/api/hello",le="+Inf"} 42
server_requests_seconds_sum{kind="http",operation="/api/hello"} 0.523
server_requests_seconds_count{kind="http",operation="/api/hello"} 42

​
Custom Metrics

You can create completely custom metrics: 
import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/metric"
)

// Get meter
meter := otel.Meter("my-service")

// Create custom counter
counter, err := meter.Int64Counter(
    "my_custom_requests_total",
    metric.WithDescription("Total number of requests"),
    metric.WithUnit("{request}"),
)

// Create custom histogram with custom buckets
histogram, err := meter.Float64Histogram(
    "my_custom_request_duration",
    metric.WithDescription("Request duration in seconds"),
    metric.WithUnit("s"),
    metric.WithExplicitBucketBoundaries(
        0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0,
    ),
)

// Use in metrics middleware
metrics.Server(
    metrics.WithRequests(counter),
    metrics.WithSeconds(histogram),
)

​
Best Practices

The default histogram buckets are suitable for most HTTP/gRPC services (5ms to 1s). Only customize if you have specific latency requirements.
Don’t add user IDs, IP addresses, or other high-cardinality data as labels. This can cause memory issues in Prometheus.
Instrument both your servers and clients to get complete visibility into request flows.
Serve metrics on a separate port (e.g., 9090) from your main application to avoid exposing metrics publicly.
Configure Prometheus retention based on your needs. Default is 15 days.

​
Grafana Dashboards

Create dashboards using these PromQL queries:

​
Request Rate

rate(server_requests_code_total[5m])

​
Error Rate

sum(rate(server_requests_code_total{code=~"5.."}[5m])) /
sum(rate(server_requests_code_total[5m]))

​
Latency Percentiles

histogram_quantile(0.95, 
  rate(server_requests_seconds_bucket[5m])
)

​
Request Duration by Operation

sum(rate(server_requests_seconds_sum[5m])) by (operation) /
sum(rate(server_requests_seconds_count[5m])) by (operation)

​
Source Reference

The metrics middleware implementation can be found in:
  • middleware/metrics/metrics.go:101 - Server middleware
  • middleware/metrics/metrics.go:160 - Client middleware
  • middleware/metrics/metrics.go:50 - DefaultRequestsCounter
  • middleware/metrics/metrics.go:60 - DefaultSecondsHistogram

​
Next Steps

Tracing

Add distributed tracing

Logging

Configure structured logging

Build docs developers (and LLMs) love

Get started for freeTalk to us