Observability Configuration

Overview

PentAGI provides comprehensive observability through three integrated systems:

Langfuse: LLM observability and analytics for AI agent operations
Grafana Stack: System monitoring with metrics, traces, and logs
OpenTelemetry: Unified telemetry collection and correlation

All observability components are optional and can be enabled independently.

Langfuse Integration

Langfuse provides advanced LLM observability including trace analysis, cost tracking, and performance metrics.

PentAGI Configuration

Configure PentAGI to send telemetry to Langfuse:

LANGFUSE_BASE_URL

string

URL of the Langfuse server

LANGFUSE_BASE_URL=http://langfuse-web:3000

LANGFUSE_PROJECT_ID

string

Langfuse project ID (defaults to LANGFUSE_INIT_PROJECT_ID)

LANGFUSE_PROJECT_ID=cm47619l0000872mcd2dlbqwb

LANGFUSE_PUBLIC_KEY

string

Langfuse project public key (defaults to LANGFUSE_INIT_PROJECT_PUBLIC_KEY)

LANGFUSE_PUBLIC_KEY=pk-lf-5946031c-ae6c-4451-98d2-9882a59e1707

LANGFUSE_SECRET_KEY

string

Langfuse project secret key (defaults to LANGFUSE_INIT_PROJECT_SECRET_KEY)

LANGFUSE_SECRET_KEY=sk-lf-d9035680-89dd-4950-8688-7870720bf359

Langfuse Server Configuration

LANGFUSE_LISTEN_IP

string

default:"127.0.0.1"

IP address for Langfuse web UI to bind to on host

LANGFUSE_LISTEN_IP=127.0.0.1

LANGFUSE_LISTEN_PORT

number

default:"4000"

Port for Langfuse web UI on host machine

LANGFUSE_LISTEN_PORT=4000

LANGFUSE_NEXTAUTH_URL

string

default:"http://localhost:4000"

Public URL for Langfuse NextAuth callbacks

LANGFUSE_NEXTAUTH_URL=http://localhost:4000

Database Configuration

PostgreSQL:

LANGFUSE_POSTGRES_USER

string

default:"postgres"

PostgreSQL username for Langfuse

LANGFUSE_POSTGRES_PASSWORD

string

default:"postgres"

PostgreSQL password for Langfuse

Change this default value in production

LANGFUSE_POSTGRES_DB

string

default:"langfuse"

PostgreSQL database name for Langfuse

ClickHouse (Analytics):

LANGFUSE_CLICKHOUSE_USER

string

default:"clickhouse"

ClickHouse username

LANGFUSE_CLICKHOUSE_PASSWORD

string

default:"clickhouse"

ClickHouse password

Change this default value in production

LANGFUSE_CLICKHOUSE_URL

string

default:"http://langfuse-clickhouse:8123"

ClickHouse HTTP endpoint URL

LANGFUSE_CLICKHOUSE_MIGRATION_URL

string

default:"clickhouse://langfuse-clickhouse:9000"

ClickHouse native protocol URL for migrations

LANGFUSE_CLICKHOUSE_CLUSTER_ENABLED

boolean

default:"false"

Enable ClickHouse cluster mode

Storage Configuration (S3/MinIO)

LANGFUSE_S3_BUCKET

string

default:"langfuse"

S3 bucket name for Langfuse data

LANGFUSE_S3_REGION

string

default:"auto"

S3 region (use auto for MinIO)

LANGFUSE_S3_ACCESS_KEY_ID

string

default:"accesskey"

S3/MinIO access key ID

Change this default value in production

LANGFUSE_S3_SECRET_ACCESS_KEY

string

default:"secretkey"

S3/MinIO secret access key

Change this default value in production

LANGFUSE_S3_ENDPOINT

string

default:"http://langfuse-minio:9000"

S3-compatible endpoint URL

LANGFUSE_S3_FORCE_PATH_STYLE

boolean

default:"true"

Force path-style S3 URLs (required for MinIO)

LANGFUSE_S3_EVENT_UPLOAD_PREFIX

string

default:"events/"

S3 prefix for event data

LANGFUSE_S3_MEDIA_UPLOAD_PREFIX

string

default:"media/"

S3 prefix for media uploads

LANGFUSE_S3_BATCH_EXPORT_ENABLED

boolean

default:"true"

Enable batch export to S3

Redis Configuration

LANGFUSE_REDIS_HOST

string

default:"langfuse-redis"

Redis hostname

LANGFUSE_REDIS_PORT

number

default:"6379"

Redis port

LANGFUSE_REDIS_AUTH

string

default:"redispassword"

Redis authentication password

Change this default value in production

LANGFUSE_REDIS_TLS_ENABLED

boolean

default:"false"

Enable TLS for Redis connections

LANGFUSE_REDIS_TLS_CA

string

Path to Redis TLS CA certificate

LANGFUSE_REDIS_TLS_CERT

string

Path to Redis TLS client certificate

LANGFUSE_REDIS_TLS_KEY

string

Path to Redis TLS client key

Security Settings

LANGFUSE_SALT

string

default:"salt"

Salt for hashing in Langfuse web UI

Change this to a random value in production

LANGFUSE_ENCRYPTION_KEY

string

64-character hex encryption key (32 bytes)Generate with: openssl rand -hex 32

Change this in production

LANGFUSE_NEXTAUTH_SECRET

string

default:"secret"

NextAuth secret key for session signing

Change this to a random value in production

Admin User Configuration

LANGFUSE_INIT_USER_EMAIL

string

default:"[email protected]"

Admin user email address

LANGFUSE_INIT_USER_NAME

string

default:"admin"

Admin username

LANGFUSE_INIT_USER_PASSWORD

string

default:"password"

Admin user password

Change this to a strong password in production

Project Configuration

LANGFUSE_INIT_ORG_ID

string

default:"ocm47619l0000872mcd2dlbqwb"

Initial organization ID

LANGFUSE_INIT_ORG_NAME

string

default:"PentAGI Org"

Initial organization name

LANGFUSE_INIT_PROJECT_ID

string

default:"cm47619l0000872mcd2dlbqwb"

Initial project ID

LANGFUSE_INIT_PROJECT_NAME

string

default:"PentAGI"

Initial project name

LANGFUSE_INIT_PROJECT_PUBLIC_KEY

string

default:"pk-lf-00000000-0000-0000-0000-000000000000"

Initial project public key

Change this to a random UUID in production

LANGFUSE_INIT_PROJECT_SECRET_KEY

string

default:"sk-lf-00000000-0000-0000-0000-000000000000"

Initial project secret key

Change this to a random UUID in production

Optional Settings

LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES

boolean

default:"true"

Enable experimental Langfuse features

LANGFUSE_TELEMETRY_ENABLED

boolean

default:"false"

Enable Langfuse’s own telemetry reporting

LANGFUSE_LOG_LEVEL

string

default:"info"

Log level for Langfuse services (debug, info, warn, error)

LANGFUSE_EE_LICENSE_KEY

string

Langfuse Enterprise Edition license key

Deployment

# Start PentAGI with Langfuse
docker compose -f docker-compose.yml -f docker-compose-langfuse.yml up -d

# Access Langfuse UI
open http://localhost:4000

Example Configuration

.env

# Langfuse Integration
LANGFUSE_BASE_URL=http://langfuse-web:3000
LANGFUSE_PROJECT_ID=cm47619l0000872mcd2dlbqwb
LANGFUSE_PUBLIC_KEY=pk-lf-5946031c-ae6c-4451-98d2-9882a59e1707
LANGFUSE_SECRET_KEY=sk-lf-d9035680-89dd-4950-8688-7870720bf359

# Langfuse Server
LANGFUSE_LISTEN_IP=127.0.0.1
LANGFUSE_LISTEN_PORT=4000

# Security (CHANGE IN PRODUCTION)
LANGFUSE_POSTGRES_PASSWORD=strong_pg_password
LANGFUSE_CLICKHOUSE_PASSWORD=strong_ch_password
LANGFUSE_S3_ACCESS_KEY_ID=strong_access_key
LANGFUSE_S3_SECRET_ACCESS_KEY=strong_secret_key
LANGFUSE_REDIS_AUTH=strong_redis_password
LANGFUSE_SALT=random_salt_value
LANGFUSE_ENCRYPTION_KEY=$(openssl rand -hex 32)
LANGFUSE_NEXTAUTH_SECRET=random_nextauth_secret

# Admin User
LANGFUSE_INIT_USER_EMAIL=[email protected]
LANGFUSE_INIT_USER_PASSWORD=strong_admin_password

OpenTelemetry Integration

OpenTelemetry provides unified observability with metrics, traces, and logs.

PentAGI Configuration

OTEL_HOST

string

OpenTelemetry collector endpoint (gRPC)

OTEL_HOST=otelcol:8148

OpenTelemetry Collector Configuration

OTEL_GRPC_LISTEN_IP

string

default:"127.0.0.1"

IP address for OTLP gRPC endpoint on host

OTEL_GRPC_LISTEN_PORT

number

default:"8148"

Port for OTLP gRPC endpoint on host

OTEL_HTTP_LISTEN_IP

string

default:"127.0.0.1"

IP address for OTLP HTTP endpoint on host

OTEL_HTTP_LISTEN_PORT

number

default:"4318"

Port for OTLP HTTP endpoint on host

Langfuse OpenTelemetry Integration

LANGFUSE_OTEL_EXPORTER_OTLP_ENDPOINT

string

OpenTelemetry endpoint for Langfuse to send telemetry

LANGFUSE_OTEL_EXPORTER_OTLP_ENDPOINT=http://otelcol:4318

LANGFUSE_OTEL_SERVICE_NAME

string

default:"langfuse"

Service name for Langfuse traces

Example Configuration

.env

# PentAGI to OpenTelemetry
OTEL_HOST=otelcol:8148

# OpenTelemetry Collector ports
OTEL_GRPC_LISTEN_IP=127.0.0.1
OTEL_GRPC_LISTEN_PORT=8148
OTEL_HTTP_LISTEN_IP=127.0.0.1
OTEL_HTTP_LISTEN_PORT=4318

# Langfuse to OpenTelemetry (optional)
LANGFUSE_OTEL_EXPORTER_OTLP_ENDPOINT=http://otelcol:4318
LANGFUSE_OTEL_SERVICE_NAME=langfuse

Grafana Stack

The Grafana observability stack includes metrics, traces, and logs.

Grafana Configuration

GRAFANA_LISTEN_IP

string

default:"127.0.0.1"

IP address for Grafana web UI to bind to on host

GRAFANA_LISTEN_PORT

number

default:"3000"

Port for Grafana web UI on host machine

Services Included

The observability stack includes:

Grafana: Visualization dashboards
VictoriaMetrics: Time-series metrics storage
Jaeger: Distributed tracing
Loki: Log aggregation
ClickHouse: Trace storage backend
Node Exporter: Host metrics
cAdvisor: Container metrics
OpenTelemetry Collector: Telemetry aggregation

Deployment

# Start observability stack
docker compose -f docker-compose.yml -f docker-compose-observability.yml up -d

# Access Grafana UI
open http://localhost:3000

Example Configuration

.env

# Grafana
GRAFANA_LISTEN_IP=127.0.0.1
GRAFANA_LISTEN_PORT=3000

# OpenTelemetry Integration
OTEL_HOST=otelcol:8148
OTEL_GRPC_LISTEN_IP=127.0.0.1
OTEL_GRPC_LISTEN_PORT=8148
OTEL_HTTP_LISTEN_IP=127.0.0.1
OTEL_HTTP_LISTEN_PORT=4318

Complete Observability Setup

Deploy all observability components together:

# Start PentAGI with full observability
docker compose \
  -f docker-compose.yml \
  -f docker-compose-langfuse.yml \
  -f docker-compose-observability.yml \
  up -d

Access the dashboards:

PentAGI UI: https://localhost:8443
Langfuse: http://localhost:4000
Grafana: http://localhost:3000

Complete Example Configuration

.env

# OpenTelemetry (enables all telemetry collection)
OTEL_HOST=otelcol:8148

# Langfuse Integration
LANGFUSE_BASE_URL=http://langfuse-web:3000
LANGFUSE_PUBLIC_KEY=pk-lf-5946031c-ae6c-4451-98d2-9882a59e1707
LANGFUSE_SECRET_KEY=sk-lf-d9035680-89dd-4950-8688-7870720bf359

# Langfuse to OpenTelemetry
LANGFUSE_OTEL_EXPORTER_OTLP_ENDPOINT=http://otelcol:4318

# Service Ports
LANGFUSE_LISTEN_PORT=4000
GRAFANA_LISTEN_PORT=3000

# Security - Database Passwords
LANGFUSE_POSTGRES_PASSWORD=strong_password_1
LANGFUSE_CLICKHOUSE_PASSWORD=strong_password_2
LANGFUSE_REDIS_AUTH=strong_password_3

# Security - S3/MinIO
LANGFUSE_S3_ACCESS_KEY_ID=strong_access_key
LANGFUSE_S3_SECRET_ACCESS_KEY=strong_secret_key

# Security - Langfuse
LANGFUSE_SALT=$(openssl rand -hex 16)
LANGFUSE_ENCRYPTION_KEY=$(openssl rand -hex 32)
LANGFUSE_NEXTAUTH_SECRET=$(openssl rand -hex 32)

# Admin User
LANGFUSE_INIT_USER_EMAIL=[email protected]
LANGFUSE_INIT_USER_PASSWORD=strong_admin_password

Monitoring Best Practices

Use Langfuse

Monitor LLM costs, latency, and token usage per agent type

Use Grafana

Track system resources, container health, and performance metrics

Enable Tracing

Use distributed tracing to debug complex agent workflows

Set Up Alerts

Configure Grafana alerts for critical metrics and errors

Next Steps

Security Configuration

Configure SSL, authentication, and secrets

LLM Providers

Set up OpenAI, Anthropic, and more

Get Started

Core Concepts

Configuration

Deployment

Features

Development

​Overview

​Langfuse Integration

​PentAGI Configuration

​Langfuse Server Configuration

​Database Configuration

​Storage Configuration (S3/MinIO)

​Redis Configuration

​Security Settings

​Admin User Configuration

​Project Configuration

​Optional Settings

​Deployment

​Example Configuration

​OpenTelemetry Integration

​PentAGI Configuration

​OpenTelemetry Collector Configuration

​Langfuse OpenTelemetry Integration

​Example Configuration

​Grafana Stack

​Grafana Configuration

​Services Included

​Deployment

​Example Configuration

​Complete Observability Setup

​Complete Example Configuration

​Monitoring Best Practices

Use Langfuse

Use Grafana

Enable Tracing

Set Up Alerts

​Next Steps

Security Configuration

LLM Providers

Build docs developers (and LLMs) love

Overview

Langfuse Integration

PentAGI Configuration

Langfuse Server Configuration

Database Configuration

Storage Configuration (S3/MinIO)

Redis Configuration

Security Settings

Admin User Configuration

Project Configuration

Optional Settings

Deployment

Example Configuration

OpenTelemetry Integration

PentAGI Configuration

OpenTelemetry Collector Configuration

Langfuse OpenTelemetry Integration

Example Configuration

Grafana Stack

Grafana Configuration

Services Included

Deployment

Example Configuration

Complete Observability Setup

Complete Example Configuration

Monitoring Best Practices

Next Steps