Docker Logs Source

​

Configuration

[sources.docker]
type = "docker_logs"

​

Parameters

docker_host = "unix:///var/run/docker.sock"
# or
docker_host = "tcp://localhost:2375"
# or
docker_host = "http://localhost:2375"

include_containers = ["app_", "web-", "ad08cc418cf9"]

exclude_containers = ["exclude_", "temp-"]

include_labels = [
  "org.opencontainers.image.vendor=Vector",
  "com.mycorp.logging=enabled"
]

include_images = ["nginx", "redis", "postgres"]

host_key = "hostname"

auto_partial_merge = true

partial_event_marker_field = "_partial"

retry_backoff_secs = 5

[sources.docker.multiline]
start_pattern = "^[^\\s]"
condition_pattern = "^[\\s]"
mode = "continue_through"
timeout_ms = 1000

[sources.docker.tls]
ca_file = "/path/to/ca.pem"
crt_file = "/path/to/cert.pem"
key_file = "/path/to/key.pem"

​

Output Schema

​

Examples

​

Basic Docker Logs Collection

[sources.docker]
type = "docker_logs"

​

Filter by Container Names

[sources.app_logs]
type = "docker_logs"
include_containers = ["web-", "api-", "worker-"]
exclude_containers = ["web-test", "api-debug"]

​

Filter by Labels

[sources.labeled_containers]
type = "docker_logs"
include_labels = [
  "com.example.environment=production",
  "com.example.team=backend"
]

​

Filter by Image Names

[sources.specific_images]
type = "docker_logs"
include_images = ["nginx", "postgres:13", "redis:alpine"]

​

Remote Docker Host with TLS

[sources.remote_docker]
type = "docker_logs"
docker_host = "https://docker.example.com:2376"

[sources.remote_docker.tls]
ca_file = "/etc/vector/ca.pem"
crt_file = "/etc/vector/cert.pem"
key_file = "/etc/vector/key.pem"

​

Java Stack Traces

[sources.java_containers]
type = "docker_logs"
include_images = ["openjdk", "tomcat"]

[sources.java_containers.multiline]
start_pattern = "^[^\\s]"
condition_pattern = "^[\\s]"
mode = "continue_through"
timeout_ms = 1000

​

Disable Partial Merge

[sources.raw_docker]
type = "docker_logs"
auto_partial_merge = false
partial_event_marker_field = "partial"

​

How It Works

​

Container Discovery

1. Lists all currently running containers
2. Applies filtering based on include_* and exclude_* settings
3. Starts streaming logs from matched containers
4. Watches for container start/stop events
5. Automatically begins/ends log collection as containers start/stop

​

Log Streaming

• Opens a persistent connection to the Docker daemon’s logs API
• Receives stdout and stderr streams
• Parses Docker’s log format (timestamp + stream + message)
• Enriches events with container metadata
• Handles log rotation and container restarts

​

Partial Message Handling

• Partial messages are buffered
• Complete messages are reassembled
• Only complete messages are emitted

​

Container Metadata

• Container ID (full and short)
• Container name
• Image name and tag
• Container creation timestamp
• All container labels
• Stream type (stdout/stderr)

​

Performance

• Minimal overhead compared to file-based collection
• Scales to hundreds of containers on a single host
• Memory usage scales with the number of watched containers
• Partial message buffering adds minimal latency

​

Best Practices

1. Use label-based filtering for dynamic container environments
2. Mount the Docker socket as read-only: -v /var/run/docker.sock:/var/run/docker.sock:ro
3. Run Vector in a container for consistent Docker API access
4. Enable multiline aggregation for stack traces and multi-line logs
5. Configure appropriate retry backoff for Docker daemon restarts
6. Monitor Vector’s internal metrics for container discovery issues
7. Use specific image filters to reduce overhead in large deployments
8. Test exclusion rules to ensure they don’t inadvertently exclude important containers

​

Docker Compose Example

version: "3.8"

services:
  vector:
    image: timberio/vector:latest
    volumes:
      - ./vector.toml:/etc/vector/vector.toml:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - VECTOR_CONFIG=/etc/vector/vector.toml

  app:
    image: myapp:latest
    labels:
      com.example.environment: "production"
      com.example.team: "backend"

​

Kubernetes Considerations

• Reads logs from Kubernetes API
• Includes pod and namespace metadata
• Handles Kubernetes-specific log formats
• Integrates with Kubernetes RBAC

• Development environments
• Non-Kubernetes Docker deployments
• Specific container filtering requirements

​

Troubleshooting

​

Permission Denied

# Add Vector user to docker group
sudo usermod -aG docker vector

# Or run Vector as root (not recommended)

​

Missing Containers

# List all containers
docker ps -a

# Check labels
docker inspect <container> | jq '.[0].Config.Labels'

​

High Memory Usage

• Use more specific include/exclude filters
• Filter by labels or images
• Disable auto_partial_merge if not needed

​

Connection Errors

# Test local socket
docker -H unix:///var/run/docker.sock ps

# Test remote host
docker -H tcp://remote-host:2375 ps