Grafana Dashboards - Permission Mongo

Overview

Permission Mongo includes a pre-built Grafana dashboard that provides comprehensive visibility into system performance and health. The dashboard is automatically provisioned when using the Docker Compose monitoring stack.

Accessing Grafana

Start the monitoring stack

docker-compose -f docker-compose.monitoring.yaml up -d

Open Grafana

Navigate to http://localhost:3000

Use default credentials:

Username: admin
Password: admin

View the dashboard

The “Permission Mongo” dashboard is pre-loaded and ready to use

Dashboard Structure

The pre-built dashboard is organized into five main sections:

Overview Row

High-level system health metrics at a glance:

Request Rate

Current requests per second

Error Rate

Percentage of 5xx errors (color-coded: green < 1%, yellow < 5%, red ≥ 5%)

P95 Latency

95th percentile request latency (green < 100ms, yellow < 500ms, red ≥ 500ms)

Active Requests

Currently in-flight requests

Goroutines

Active goroutine count (warning at 5k, critical at 10k)

Service Status

UP/DOWN indicator

HTTP Metrics Row

Detailed HTTP request analysis:

Request Rate by Method

Type: Time series
Query: sum by (method) (rate(http_requests_total[1m])) Shows request throughput broken down by HTTP method (GET, POST, PUT, DELETE, PATCH).

Request Rate by Status Code

Type: Time series
Query: sum by (status) (rate(http_requests_total[1m])) Visualizes requests by status code with color coding:

2xx: Blue (success)
4xx: Yellow (client errors)
5xx: Red (server errors)

Request Latency Percentiles

Type: Time series
Queries:

# P50
histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

# P90
histogram_quantile(0.90, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

# P95
histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

# P99
histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

Displays multiple latency percentiles to identify outliers and tail latency.

Request Rate by Endpoint

Type: Time series
Query: sum by (path) (rate(http_requests_total[1m])) Shows which endpoints are receiving the most traffic.

MongoDB Metrics Row

Database operation insights:

MongoDB Operations by Type

Type: Time series
Query: sum by (operation) (rate(mongodb_operations_total[1m])) Breaks down database operations into:

find - Read queries
insert - Inserts
update - Updates
delete - Deletions

MongoDB Operation Latency (P95)

Type: Time series
Query:

histogram_quantile(0.95, 
  sum(rate(mongodb_operation_duration_seconds_bucket[5m])) by (le, operation)
)

Shows 95th percentile latency for each operation type.

Cache Metrics Row

Cache performance and hit rate analysis:

Cache Hit Rate

Type: Gauge
Query:

sum(rate(cache_hits_total[5m])) 
  / (sum(rate(cache_hits_total[5m])) + sum(rate(cache_misses_total[5m])))

Color-coded gauge:

Red: < 50% (poor)
Yellow: 50-80% (fair)
Green: > 80% (good)

Cache Hits vs Misses

Type: Time series
Queries:

# Hits
sum(rate(cache_hits_total[1m]))

# Misses
sum(rate(cache_misses_total[1m]))

Visualizes cache effectiveness over time with color coding:

Hits: Green
Misses: Red

RBAC & Audit Row

Security and audit trail monitoring:

RBAC Evaluations

Type: Time series
Query: sum by (result) (rate(rbac_evaluations_total[1m])) Shows authorization decisions:

Allowed: Green
Denied: Red

Audit Log Metrics

Type: Time series
Queries:

# Queue size
audit_queue_size

# Logs per second
sum(rate(audit_logs_total[1m]))

# Dropped logs per second
sum(rate(audit_logs_dropped_total[1m]))

Monitors audit system health and identifies bottlenecks.

Manual Dashboard Setup

If running Grafana separately from the Docker Compose stack:

Add Prometheus datasource

Navigate to Configuration → Data Sources
Click Add data source
Select Prometheus
Configure URL: http://localhost:9090 (or http://host.docker.internal:9090 from Docker)
Click Save & Test

Import the dashboard

Navigate to Dashboards → Import
Click Upload JSON file
Select monitoring/grafana/dashboards/permission-mongo.json
Select the Prometheus datasource
Click Import

Dashboard Location

The dashboard JSON is located at:

monitoring/grafana/dashboards/permission-mongo.json

Auto-Provisioning Configuration

Dashboards are automatically provisioned via:

monitoring/grafana/provisioning/dashboards/default.yml

apiVersion: 1

providers:
  - name: 'Default'
    orgId: 1
    folder: ''
    type: file
    disableDeletion: false
    updateIntervalSeconds: 10
    allowUiUpdates: true
    options:
      path: /etc/grafana/provisioning/dashboards

Datasource is auto-configured via:

monitoring/grafana/provisioning/datasources/prometheus.yml

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true

Creating Custom Panels

To add custom visualizations:

Edit the dashboard

Click the gear icon (⚙️) in the top right, then Settings → JSON Model

Add a new panel

Click Add panel → Add a new panel

Configure query

Select Prometheus datasource and enter your PromQL query

Customize visualization

Choose panel type (time series, gauge, stat, etc.) and configure display options

Save the dashboard

Click Save dashboard and optionally export the JSON

Useful Custom Panels

Request Success Rate

sum(rate(http_requests_total{status=~"2.."}[5m])) 
  / sum(rate(http_requests_total[5m]))

Panel Type: Gauge
Thresholds: < 0.95 (red), < 0.99 (yellow), ≥ 0.99 (green)

MongoDB Connection Pool Usage

permission_mongo_mongo_pool_size

Panel Type: Stat

Top 5 Slowest Endpoints (P95)

topk(5, 
  histogram_quantile(0.95, 
    sum(rate(http_request_duration_seconds_bucket[5m])) by (le, path)
  )
)

Panel Type: Bar gauge

Cache Efficiency by Type

sum by (type) (rate(cache_hits_total[5m])) 
  / (sum by (type) (rate(cache_hits_total[5m])) + sum by (type) (rate(cache_misses_total[5m])))

Panel Type: Bar gauge

RBAC Deny Rate

sum(rate(rbac_evaluations_total{result="denied"}[5m])) 
  / sum(rate(rbac_evaluations_total[5m]))

Panel Type: Stat

Dashboard Refresh Rate

The default refresh rate is 5 seconds. You can change this in the dashboard settings:

Click the clock icon in the top right
Select a different refresh interval (5s, 10s, 30s, 1m, etc.)

Time Range Selection

Default time range: Last 1 hour Use the time picker in the top right to adjust:

Last 5 minutes
Last 15 minutes
Last 1 hour
Last 6 hours
Last 24 hours
Custom range

Annotations

The dashboard supports Grafana annotations for marking deployments, incidents, or other events:

Click on the graph where you want to add an annotation
Enter description and tags
Save annotation

Annotations appear as vertical lines across all panels.

Variables and Templating

The current dashboard does not use variables, but you can add them for dynamic filtering:

Example: Add Instance Variable

Dashboard settings → Variables → Add variable
Name: instance
Type: Query
Data source: Prometheus
Query: label_values(up{job="permission-mongo"}, instance)
Update all panel queries to use {instance="$instance"}

Exporting Dashboards

To export your customized dashboard:

Click the share icon in the top right
Select Export
Toggle Export for sharing externally
Click Save to file

Keep a version-controlled copy of your customized dashboards in the repository.

Getting Started

Core Concepts

Configuration

Guides

Monitoring & Observability

Advanced

SDK

​Overview

​Accessing Grafana

​Dashboard Structure

​Overview Row

Request Rate

Error Rate

P95 Latency

Active Requests

Goroutines

Service Status

​HTTP Metrics Row

​Request Rate by Method

​Request Rate by Status Code

​Request Latency Percentiles

​Request Rate by Endpoint

​MongoDB Metrics Row

​MongoDB Operations by Type

​MongoDB Operation Latency (P95)

​Cache Metrics Row

​Cache Hit Rate

​Cache Hits vs Misses

​RBAC & Audit Row

​RBAC Evaluations

​Audit Log Metrics

​Manual Dashboard Setup

​Dashboard Location

​Auto-Provisioning Configuration

​Creating Custom Panels

​Useful Custom Panels

​Request Success Rate

​MongoDB Connection Pool Usage

​Top 5 Slowest Endpoints (P95)

​Cache Efficiency by Type

​RBAC Deny Rate

​Dashboard Refresh Rate

​Time Range Selection

​Annotations

​Variables and Templating

​Exporting Dashboards

Build docs developers (and LLMs) love

Overview

Accessing Grafana

Dashboard Structure

Overview Row

HTTP Metrics Row

Request Rate by Method

Request Rate by Status Code

Request Latency Percentiles

Request Rate by Endpoint

MongoDB Metrics Row

MongoDB Operations by Type

MongoDB Operation Latency (P95)

Cache Metrics Row

Cache Hit Rate

Cache Hits vs Misses

RBAC & Audit Row

RBAC Evaluations

Audit Log Metrics

Manual Dashboard Setup

Dashboard Location

Auto-Provisioning Configuration

Creating Custom Panels

Useful Custom Panels

Request Success Rate

MongoDB Connection Pool Usage

Top 5 Slowest Endpoints (P95)

Cache Efficiency by Type

RBAC Deny Rate

Dashboard Refresh Rate

Time Range Selection

Annotations

Variables and Templating

Exporting Dashboards