Skip to main content

​
Prerequisites

  • Docker and Docker Compose installed
  • Access to vLLM /metrics endpoint
  • Authentication token for remote vLLM endpoints (if applicable)

​
Environment Configuration

1

Create .env file

Copy the example environment file and populate with your values:
cd monitoring
cp .env_example .env
2

Configure credentials

Edit .env with your Grafana admin credentials:
# Grafana credentials
ADMIN_USER=admin
ADMIN_PASSWORD=your-secure-password
GRAFANA_PORT=4000

# vLLM target (local)
VLLM_TARGET=localhost:8000
SCHEME=http
VLLM_METRICS_AUTH_TOKEN=  # Not required for local

# Prometheus settings (defaults)
PROMETHEUS_PORT=9090
STORAGE_TSDB_RETENTION_TIME=15d
LOG_LEVEL=info
3

Set vLLM target

Configure the vLLM endpoint to monitor:
  • VLLM_TARGET: Hostname or IP address of your vLLM instance
  • SCHEME: Use https for remote endpoints, http for local
  • VLLM_METRICS_AUTH_TOKEN: Bearer token required for remote /metrics access
The .env file is gitignored and never committed. Keep your credentials secure.

​
Starting the Stack

​
Quick Start

The simplest way to launch the entire monitoring stack: 
make docker-up
This command orchestrates the full workflow:
  1. Stops any running containers
  2. Generates configuration files from templates
  3. Builds Docker images
  4. Starts containers in the background
  5. Streams real-time logs

​
Individual Commands

For more granular control, use these Make targets:
CommandDescription
make prometheus-confGenerates prometheus.yml from template
make grafana-confGenerates dashboard JSON files from templates
make docker-buildBuilds the Docker images
make docker-runStarts the containers in the background
make docker-stopStops containers and deletes generated config files
make docker-logsStreams real-time logs

​
Step-by-Step Deployment

1

Generate configurations

make prometheus-conf
make grafana-conf
This creates:
  • prometheus/prometheus.yml
  • grafana/provisioning/dashboards/user_metrics_overview.json
  • grafana/provisioning/dashboards/machine_metrics_overview.json
  • grafana/provisioning/dashboards/vllm_tokens.json
2

Build Docker images

make docker-build
Builds both Prometheus and Grafana containers.
3

Start services

make docker-run
Launches containers in detached mode with health checks.
4

Verify deployment

make docker-logs
Check logs for successful startup. Look for:
  • Prometheus: Server is ready to receive web requests
  • Grafana: HTTP Server Listen

​
Accessing Grafana

Once the stack is running:
  1. Open your browser to http://localhost:4000 (or your configured GRAFANA_PORT)
  2. Log in with the credentials from your .env file:
    • Username: ADMIN_USER
    • Password: ADMIN_PASSWORD
  3. The default home dashboard is User Metrics Overview
First-time setup: Grafana automatically provisions the Prometheus data source and all dashboards on startup.

​
Stopping the Stack

To stop and clean up all resources: 
make docker-stop
This command:
  • Stops and removes containers
  • Deletes generated configuration files
  • Preserves persistent Prometheus data (if configured)
Generated config files are deleted on stop for security. Always use make docker-up or regenerate configs before restarting.

​
Troubleshooting

​
Prometheus can’t reach vLLM

Check your .env configuration:
  • Verify VLLM_TARGET is correct
  • Ensure SCHEME matches your vLLM deployment (http/https)
  • For remote endpoints, confirm VLLM_METRICS_AUTH_TOKEN is valid
Test the metrics endpoint manually: 
# Local
curl http://localhost:8000/metrics

# Remote
curl -H "Authorization: Bearer your-token" https://vllm.concrete-security.com/metrics

​
Grafana shows “No Data”

  1. Check Prometheus is scraping successfully: 
    docker compose logs prometheus | grep "/metrics"
  2. Verify the Prometheus data source in Grafana:
    • Settings > Data Sources > Prometheus
    • Click “Test” to verify connectivity

​
Port conflicts

If port 4000 is already in use, change GRAFANA_PORT in .env and restart: 
make docker-stop
make docker-up

​
Advanced Configuration

​
Custom retention period

Modify STORAGE_TSDB_RETENTION_TIME in .env: 
STORAGE_TSDB_RETENTION_TIME=30d  # Keep metrics for 30 days

​
Persistent data

To persist Prometheus data across restarts, set: 
PERSISTANT_PROMETHEUS_DATA=./prometheus_data
Create the directory: 
mkdir -p prometheus_data

Build docs developers (and LLMs) love

Get started for freeTalk to us