Monitoring - NATS Server

Overview

NATS Server provides a comprehensive set of built-in HTTP monitoring endpoints that expose real-time server metrics, connection details, and operational statistics. These endpoints enable deep observability into your NATS infrastructure without requiring external agents.

All monitoring endpoints are exposed via HTTP on a configurable port (default: 8222) and return JSON-formatted data.

HTTP Monitoring Port

Configuration

Enable monitoring by configuring the HTTP port:

monitoring.conf

http_port: 8222

# Optional: Bind to specific host
http: "0.0.0.0:8222"

# Optional: HTTPS monitoring
https_port: 8223

Command line flags:

CLI Flags

# Enable HTTP monitoring
nats-server -m 8222

# Enable HTTPS monitoring
nats-server -ms 8223

From server.go:3037-3046, the following paths are available:

/varz - Server information and statistics
/connz - Connection details
/routez - Route connection information
/subsz - Subscription details
/jsz - JetStream statistics

HTTPS Configuration

Secure monitoring endpoints with TLS:

https-monitoring.conf

https_port: 8223

tls {
    cert_file: "/path/to/monitoring-cert.pem"
    key_file: "/path/to/monitoring-key.pem"
}

Monitoring endpoints expose sensitive information. Always secure them with TLS and authentication in production.

Monitoring Endpoints

/varz - Server Information

Provides comprehensive server runtime information and statistics.

Basic Usage

curl http://localhost:8222/varz

curl http://localhost:8222/varz | jq

Key Fields (from monitor.go:1211-1283):

server_id

string

Unique server ID generated at start

version

string

NATS Server version

connections

int

Current number of active connections

total_connections

uint64

Total connections since server start

in_msgs

int64

Total inbound messages

out_msgs

int64

Total outbound messages

in_bytes

int64

Total inbound bytes

out_bytes

int64

Total outbound bytes

slow_consumers

int64

Number of slow consumers detected

subscriptions

uint32

Current active subscriptions

cpu

float64

Current CPU usage percentage

mem

int64

Current memory usage in bytes

jetstream

object

JetStream configuration and statistics (if enabled)

/connz - Connection Details

Provides detailed information about client connections.

Basic Usage

curl http://localhost:8222/connz

Query Parameters:

sort

string

default:"cid"

Sort connections by: cid, start, subs, pending, msgs_to, msgs_from, bytes_to, bytes_from, last, idle, uptime, stop, reason, rtt

auth

bool

default:"false"

Include username in connection details

subs

bool

default:"false"

Include subscription subjects

subs_detail

bool

default:"false"

Include detailed subscription information

offset

int

default:"0"

Pagination offset

limit

int

default:"1024"

Maximum connections to return

cid

uint64

Filter by specific connection ID

state

string

Filter by state: open, closed, all

mqtt_client

string

Filter by MQTT client ID

curl "http://localhost:8222/connz?state=open&limit=10"

Response Fields (from monitor.go:47-154):

Connection Info

{
  "server_id": "NDJWE4...",
  "now": "2024-01-15T12:45:30Z",
  "num_connections": 142,
  "total": 142,
  "offset": 0,
  "limit": 1024,
  "connections": [
    {
      "cid": 42,
      "kind": "Client",
      "type": "nats",
      "ip": "192.168.1.100",
      "port": 54321,
      "start": "2024-01-15T10:30:00Z",
      "last_activity": "2024-01-15T12:45:29Z",
      "rtt": "2ms",
      "uptime": "2h15m30s",
      "idle": "1s",
      "pending_bytes": 0,
      "in_msgs": 15234,
      "out_msgs": 30468,
      "in_bytes": 1523400,
      "out_bytes": 3046800,
      "subscriptions": 12,
      "name": "my-service",
      "lang": "go",
      "version": "1.31.0",
      "tls_version": "1.3",
      "tls_cipher_suite": "TLS_AES_128_GCM_SHA256",
      "authorized_user": "app-user",
      "account": "ORDERS"
    }
  ]
}

/routez - Route Information

Details about route (server-to-server) connections in a cluster.

Basic Usage

curl http://localhost:8222/routez

Query Parameters:

subs

bool

default:"false"

Include subscription details for routes

subs_detail

bool

default:"false"

Include detailed subscription information

curl http://localhost:8222/routez | jq

/subsz - Subscription Details

Information about active subscriptions across all connections.

Basic Usage

curl http://localhost:8222/subsz

Query Parameters:

subs

bool

default:"false"

Include detailed subscription list

offset

int

default:"0"

Pagination offset

limit

int

default:"1024"

Maximum subscriptions to return

test

string

Test which subscriptions match a publish subject

curl http://localhost:8222/subsz

/jsz - JetStream Statistics

Detailed JetStream metrics including streams, consumers, and storage usage.

Basic Usage

curl http://localhost:8222/jsz

Query Parameters:

accounts

bool

default:"false"

Include per-account JetStream details

streams

bool

default:"false"

Include stream details

consumers

bool

default:"false"

Include consumer details

config

bool

default:"false"

Include configuration details

leader_only

bool

default:"false"

Only return data if server is meta-leader

offset

int

default:"0"

Pagination offset for accounts

limit

int

default:"1024"

Maximum accounts to return

curl http://localhost:8222/jsz

Integration with Monitoring Tools

Prometheus

Use the NATS Prometheus Exporter:

prometheus.yml

scrape_configs:
  - job_name: 'nats'
    static_configs:
      - targets: ['localhost:7777']  # Prometheus exporter port

Run Exporter

nats-exporter -varz http://localhost:8222/varz \
              -connz http://localhost:8222/connz \
              -routez http://localhost:8222/routez \
              -subz http://localhost:8222/subsz \
              -jsz=http://localhost:8222/jsz

Grafana

Import official NATS Grafana dashboards:

NATS Server Dashboard: General server metrics
JetStream Dashboard: Stream and consumer metrics
Connection Dashboard: Client connection details

DataDog

Use the DataDog NATS integration:

datadog.yaml

instances:
  - host: localhost
    port: 8222
    tags:
      - env:production
      - cluster:us-east

Custom Monitoring

Build custom monitoring with polling:

Custom Monitor

import requests
import time

def monitor_nats():
    while True:
        varz = requests.get('http://localhost:8222/varz').json()
        
        print(f"Connections: {varz['connections']}")
        print(f"Messages/sec: {varz['in_msgs']}")
        print(f"CPU: {varz['cpu']}%")
        print(f"Memory: {varz['mem'] / 1024 / 1024:.2f} MB")
        
        if varz.get('jetstream'):
            js = varz['jetstream']['stats']
            print(f"JetStream Memory: {js['memory'] / 1024 / 1024:.2f} MB")
            print(f"JetStream Storage: {js['storage'] / 1024 / 1024 / 1024:.2f} GB")
        
        time.sleep(10)

monitor_nats()

Metrics and Observability

Key Metrics to Monitor

Connection Count

Monitor connections and total_connections for client activity trends.

Message Throughput

Track in_msgs, out_msgs, in_bytes, out_bytes for throughput.

Slow Consumers

Alert on slow_consumers increasing to detect backpressure.

Memory Usage

Monitor mem and JetStream memory to prevent OOM.

CPU Usage

Watch cpu percentage for performance issues.

Subscription Count

Track subscriptions for interest patterns.

JetStream Storage

Monitor JetStream storage vs max_storage limits.

API Errors

Alert on JetStream API errors increasing.

Health Checks

Use /healthz for health monitoring:

Health Check

curl http://localhost:8222/healthz

# Returns:
# - HTTP 200 if healthy
# - HTTP 503 if unhealthy

Configure health check options:

JetStream Health

# Check if JetStream enabled
curl "http://localhost:8222/healthz?js-enabled-only=true"

# Check specific stream
curl "http://localhost:8222/healthz?stream=ORDERS&account=ORDERS"

Real-Time Monitoring Example

Live Dashboard

watch -n 1 'curl -s http://localhost:8222/varz | jq "{connections, in_msgs, out_msgs, mem, cpu}"'

# Output:
# {
#   "connections": 142,
#   "in_msgs": 1523945,
#   "out_msgs": 3047890,
#   "mem": 12582912,
#   "cpu": 2.5
# }

Security Considerations

Monitoring endpoints expose sensitive operational data. Always secure them in production.

Authentication

Restrict monitoring endpoint access:

Secured Monitoring

http_port: 8222

authorization {
    users = [
        {
            user: "monitor"
            password: "$MONITOR_PASSWORD"
            permissions = {
                # Allow monitoring only
            }
        }
    ]
}

Network Restrictions

Bind to localhost or internal network:

Restricted Access

# Only allow localhost
http: "127.0.0.1:8222"

# Or specific internal IP
http: "10.0.0.1:8222"

Use firewall rules to restrict access to monitoring ports.

Next Steps

JetStream

Monitor JetStream streams and consumers

Configuration

Advanced monitoring configuration

Operations

Operational best practices

Prometheus Exporter

Official Prometheus integration

Get Started

Deployment

Architecture

Features

Operations

Configuration

​Overview

​HTTP Monitoring Port

​Configuration

​HTTPS Configuration

​Monitoring Endpoints

​/varz - Server Information

​/connz - Connection Details

​/routez - Route Information

​/subsz - Subscription Details

​/jsz - JetStream Statistics

​Integration with Monitoring Tools

​Prometheus

​Grafana

​DataDog

​Custom Monitoring

​Metrics and Observability

​Key Metrics to Monitor

Connection Count

Message Throughput

Slow Consumers

Memory Usage

CPU Usage

Subscription Count

JetStream Storage

API Errors

​Health Checks

​Real-Time Monitoring Example

​Security Considerations

​Authentication

​Network Restrictions

​Next Steps

JetStream

Configuration

Operations

Prometheus Exporter

Build docs developers (and LLMs) love

Overview

HTTP Monitoring Port

Configuration

HTTPS Configuration

Monitoring Endpoints

/varz - Server Information

/connz - Connection Details

/routez - Route Information

/subsz - Subscription Details

/jsz - JetStream Statistics

Integration with Monitoring Tools

Prometheus

Grafana

DataDog

Custom Monitoring

Metrics and Observability

Key Metrics to Monitor

Health Checks

Real-Time Monitoring Example

Security Considerations

Authentication

Network Restrictions

Next Steps