Skip to main content
Sources are the entry points for data in Vector. They collect events from various sources like files, network sockets, APIs, and more.

​
Source Configuration

Sources are configured in the sources section of your configuration file: 
sources:
  <source_id>:
    type: <source_type>
    # Source-specific options

​
Common Source Parameters

All sources support these base configuration options:
​
type
string
required
The type of the source component. Determines which source implementation to use.
​
proxy
object
Proxy configuration for sources that make HTTP requests.
proxy:
  http: "http://proxy.example.com:3128"
  https: "http://proxy.example.com:3128"
  no_proxy: ["localhost", "127.0.0.1"]

​
File Source

Read events from files using glob patterns: 
sources:
  apache_logs:
    type: file
    include: ["/var/log/apache2/*.log"]
    exclude: ["/var/log/apache2/*debug.log"]
    ignore_older: 86400  # 1 day in seconds
    read_from: beginning
​
include
array
required
Array of file path glob patterns to read from. Supports standard glob syntax.
​
exclude
array
Array of file path glob patterns to exclude from reading.
​
ignore_older
integer
Ignore files that haven’t been modified within this many seconds.
​
read_from
string
default:"end"
Where to start reading from: beginning or end of files.

​
Stdin Source

Read events from standard input: 
sources:
  stdin:
    type: stdin
    max_length: 102400  # 100 KB
​
max_length
integer
default:"102400"
Maximum byte size of a single event. Events exceeding this will be truncated.

​
Demo Logs Source

Generate demo log events for testing: 
sources:
  demo:
    type: demo_logs
    format: syslog
    interval: 1
    count: 100
​
format
string
required
Format of generated logs: syslog, json, apache_common, or apache_error.
​
interval
float
default:"1.0"
Number of seconds between generating events.
​
count
integer
Number of events to generate before stopping. Omit for infinite generation.

​
HTTP Source

Receive events via HTTP: 
sources:
  http_endpoint:
    type: http_server
    address: "0.0.0.0:8080"
    encoding: json
    path: "/events"
​
address
string
required
The socket address to listen on for connections.
​
encoding
string
required
Expected encoding of received data: json, text, or ndjson.
​
path
string
default:"/"
HTTP path to accept events on.

​
Syslog Source

Receive syslog messages over TCP or UDP: 
sources:
  syslog_server:
    type: syslog
    mode: tcp
    address: "0.0.0.0:514"
​
mode
string
required
Transport protocol: tcp, udp, or unix.
​
address
string
required
The socket address or Unix socket path to listen on.

​
Socket Source

Receive events over TCP, UDP, or Unix sockets: 
sources:
  socket_listener:
    type: socket
    mode: tcp
    address: "0.0.0.0:9000"
    max_length: 102400
​
mode
string
required
Transport protocol: tcp, udp, or unix.
​
address
string
required
The socket address or Unix socket path to listen on.

​
Prometheus Scrape Source

Scrape Prometheus metrics from endpoints: 
sources:
  prometheus_metrics:
    type: prometheus_scrape
    endpoints:
      - "http://localhost:9090/metrics"
    scrape_interval_secs: 15
​
endpoints
array
required
Array of HTTP endpoints to scrape metrics from.
​
scrape_interval_secs
integer
default:"15"
Interval in seconds between scrapes.

​
Multiple Sources Example

Combine multiple sources in a single configuration: 
sources:
  # Application logs from files
  app_logs:
    type: file
    include: ["/var/log/app/*.log"]
    ignore_older: 86400
  
  # Syslog messages from network
  syslog:
    type: syslog
    mode: tcp
    address: "0.0.0.0:514"
  
  # Metrics from Prometheus endpoints
  app_metrics:
    type: prometheus_scrape
    endpoints:
      - "http://localhost:9090/metrics"
      - "http://localhost:9091/metrics"
    scrape_interval_secs: 30

transforms:
  # Process all logs together
  enrich_logs:
    type: remap
    inputs: [app_logs, syslog]
    source: |
      .environment = "production"

sinks:
  elasticsearch:
    type: elasticsearch
    inputs: [enrich_logs]
    endpoint: "http://localhost:9200"

​
Source Acknowledgements

Sources can optionally wait for acknowledgements from sinks before marking data as successfully processed: 
sources:
  important_logs:
    type: file
    include: ["/var/log/critical/*.log"]
    acknowledgements:
      enabled: true

​
Next Steps

  • Transforms - Process data from sources
  • Sinks - Send processed data to destinations
  • Testing - Test your source configurations

Build docs developers (and LLMs) love

Get started for freeTalk to us