Configuration Reference

S2 Lite is configured through command-line arguments and environment variables. This guide covers all available configuration options.

Command-Line Arguments

Storage Options

--bucket

string

S3 bucket name for object storage backend. If not specified, uses in-memory storage (unless --local-root is set).

s2 lite --bucket my-s2-bucket

--local-root

string

Local filesystem directory for storage. Conflicts with --bucket.

s2 lite --local-root ./s2-data

Local filesystem mode does not provide the same durability guarantees as object storage.

--path

string

default:""

Base path prefix within object storage. Allows multiple instances to share a bucket.

s2 lite --bucket my-bucket --path s2lite/prod

Network Options

--port

integer

Port to listen on. Defaults to 443 if TLS is enabled, otherwise 80.

s2 lite --port 8080

--no-cors

boolean

default:"false"

Disable permissive CORS headers. By default, S2 Lite allows browser-based clients from any origin.

s2 lite --no-cors

TLS Options

--tls-self

boolean

default:"false"

Use a self-signed certificate for TLS. Useful for development and testing.

s2 lite --tls-self

Clients will need to use --insecure or configure SSL verification to trust self-signed certificates.

--tls-cert

string

Path to TLS certificate file (e.g., cert.pem). Must be used with --tls-key.

s2 lite --tls-cert /path/to/cert.pem --tls-key /path/to/key.pem

--tls-key

string

Path to TLS private key file (e.g., key.pem). Must be used with --tls-cert.

Initialization Options

--init-file

string

Path to JSON file defining basins and streams to create at startup. Can also be set via S2LITE_INIT_FILE environment variable.

s2 lite --init-file resources.json

See Init File Format below for details.

Environment Variables

S2 Lite Configuration

S2LITE_INIT_FILE

string

Path to initialization file. Alternative to --init-file.

export S2LITE_INIT_FILE=resources.json
s2 lite

S2LITE_PIPELINE

boolean

default:"false"

Enable append pipelining for improved performance against high-latency object storage.

export S2LITE_PIPELINE=true
s2 lite

Pipelining is currently disabled by default for safety. It will be enabled by default in a future release after further testing.

When enabled, S2 Lite pipelines up to 25 MiB of appends.

AWS Configuration

AWS_ACCESS_KEY_ID

string

AWS access key ID for S3 authentication.

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE

AWS_SECRET_ACCESS_KEY

string

AWS secret access key for S3 authentication.

export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

AWS_REGION

string

AWS region for S3 bucket.

export AWS_REGION=us-east-1

AWS_ENDPOINT_URL_S3

string

Custom S3 endpoint URL. Required for S3-compatible services like MinIO, Tigris, or Cloudflare R2.

# Tigris
export AWS_ENDPOINT_URL_S3=https://fly.storage.tigris.dev

# MinIO
export AWS_ENDPOINT_URL_S3=http://minio:9000

# Cloudflare R2
export AWS_ENDPOINT_URL_S3=https://ACCOUNT_ID.r2.cloudflarestorage.com

AWS_PROFILE

string

AWS profile name from ~/.aws/credentials.

export AWS_PROFILE=production

SlateDB Configuration

S2 Lite uses SlateDB as its storage engine. Configure SlateDB using SL8_ prefixed environment variables. See the SlateDB Settings reference for all available options.

SL8_FLUSH_INTERVAL

duration

Interval at which to flush writes to object storage.Defaults:

50ms for S3 bucket
5ms for in-memory or local filesystem

export SL8_FLUSH_INTERVAL=100ms

Lower values reduce latency but increase object storage API calls. Higher values improve throughput but increase latency.

SL8_MANIFEST_POLL_INTERVAL

duration

Interval at which to poll for manifest updates. Used for fencing out previous instances.

export SL8_MANIFEST_POLL_INTERVAL=1s

SL8_L0_COMPACTION_THRESHOLD

integer

Number of L0 SSTables before triggering compaction.

export SL8_L0_COMPACTION_THRESHOLD=8

Logging Configuration

RUST_LOG

string

default:"info"

Logging level configuration. Supports module-level filtering.

# Global info level
export RUST_LOG=info

# Debug level for s2-lite
export RUST_LOG=s2_lite=debug

# Mixed levels
export RUST_LOG=s2_lite=debug,slatedb=info,tower_http=debug

Available levels: error, warn, info, debug, trace

Init File Format

The init file is a JSON document that defines basins and streams to create at startup. It uses create-or-reconfigure semantics, making it safe for repeated restarts.

Schema

schema.json

{
  "basins": [
    {
      "name": "string",
      "config": {
        "create_stream_on_append": "boolean (optional)",
        "create_stream_on_read": "boolean (optional)",
        "default_stream_config": {
          "storage_class": "'standard' | 'express' (optional)",
          "retention_policy": "'infinite' | duration string (optional)",
          "timestamping": {
            "mode": "'client-prefer' | 'client-require' | 'arrival' (optional)",
            "uncapped": "boolean (optional)"
          },
          "delete_on_empty": {
            "min_age": "duration string (optional)"
          }
        }
      },
      "streams": [
        {
          "name": "string",
          "config": {
            "storage_class": "'standard' | 'express' (optional)",
            "retention_policy": "'infinite' | duration string (optional)",
            "timestamping": { /* same as above */ },
            "delete_on_empty": { /* same as above */ }
          }
        }
      ]
    }
  ]
}

Example

resources.json

{
  "basins": [
    {
      "name": "production",
      "config": {
        "create_stream_on_append": true,
        "create_stream_on_read": false,
        "default_stream_config": {
          "storage_class": "standard",
          "retention_policy": "7days",
          "timestamping": {
            "mode": "client-prefer",
            "uncapped": false
          },
          "delete_on_empty": {
            "min_age": "1day"
          }
        }
      },
      "streams": [
        {
          "name": "events",
          "config": {
            "retention_policy": "infinite"
          }
        },
        {
          "name": "logs",
          "config": {
            "retention_policy": "24hours",
            "storage_class": "express"
          }
        },
        {
          "name": "metrics",
          "config": {
            "retention_policy": "30days",
            "timestamping": {
              "mode": "arrival"
            }
          }
        }
      ]
    },
    {
      "name": "ephemeral",
      "config": {
        "create_stream_on_append": true,
        "default_stream_config": {
          "retention_policy": "1hour",
          "delete_on_empty": {
            "min_age": "5m"
          }
        }
      }
    }
  ]
}

Field Reference

Basin Config

create_stream_on_append

boolean

Automatically create streams on append if they don’t exist, using the default stream configuration.

create_stream_on_read

boolean

Automatically create streams on read if they don’t exist, using the default stream configuration.

default_stream_config

object

Default configuration for auto-created streams in this basin.

Stream Config

storage_class

string

Storage class for recent writes.Options:

standard - Standard S3 storage
express - S3 Express One Zone (ultra-low latency)

"storage_class": "express"

retention_policy

string

Retention policy for the stream.Options:

infinite - Retain records indefinitely
Duration string - Auto-trim older records (e.g., 7days, 24hours, 1week)

"retention_policy": "7days"

Duration formats:

1day, 7days, 30days
1hour, 24hours
1week, 2weeks
1min, 30mins

timestamping

object

Timestamping behavior for appends.

"timestamping": {
  "mode": "client-prefer",
  "uncapped": false
}

timestamping.mode

string

Timestamping mode.Options:

client-prefer - Use client timestamp if provided, otherwise arrival time
client-require - Require client timestamp (reject if not provided)
arrival - Always use server arrival time

timestamping.uncapped

boolean

Allow client-specified timestamps to exceed arrival time. If false, client timestamps are capped at arrival time.

delete_on_empty

object

Delete-on-empty configuration.

"delete_on_empty": {
  "min_age": "1day"
}

delete_on_empty.min_age

string

Minimum age before an empty stream can be deleted automatically. Set to 0 (or omit) to disable auto-deletion.

"min_age": "1day"

Performance Tuning

Flush Interval

The most important performance knob is SL8_FLUSH_INTERVAL:

Lower values (10-50ms): Lower latency, higher API call costs
Higher values (100-500ms): Higher throughput, lower costs, higher latency

# Low latency
export SL8_FLUSH_INTERVAL=10ms

# High throughput
export SL8_FLUSH_INTERVAL=500ms

Pipelining

Enable pipelining for better performance with high-latency object storage:

export S2LITE_PIPELINE=true

This allows up to 25 MiB of appends to be in-flight simultaneously.

Storage Class

Use S3 Express One Zone for ultra-low latency:

{
  "storage_class": "express"
}

S3 Express requires an Express One Zone bucket (ending in --x-s3).

Example Configurations

Development (In-Memory)

s2 lite --port 8080

Production (S3 with IAM)

export AWS_REGION=us-east-1
export SL8_FLUSH_INTERVAL=50ms
export S2LITE_PIPELINE=true

s2 lite \
  --bucket my-s2-bucket \
  --path production \
  --no-cors \
  --init-file /etc/s2-lite/resources.json

Tigris with Static Credentials

export AWS_ACCESS_KEY_ID=tid_xxx
export AWS_SECRET_ACCESS_KEY=tsec_xxx
export AWS_ENDPOINT_URL_S3=https://fly.storage.tigris.dev
export SL8_FLUSH_INTERVAL=30ms

s2 lite \
  --bucket my-tigris-bucket \
  --port 8080

Local Filesystem

export SL8_FLUSH_INTERVAL=5ms

s2 lite \
  --local-root ./s2-data \
  --port 8080

Get Started

Core Concepts

CLI

S2 Lite

SDKs

Configuration Reference

Command-Line Arguments

Storage Options

Network Options

TLS Options

Initialization Options

Environment Variables

S2 Lite Configuration

AWS Configuration

SlateDB Configuration

Logging Configuration

Init File Format

Schema

Example

Field Reference

Basin Config

Stream Config

Performance Tuning

Flush Interval

Pipelining

Storage Class

Example Configurations

Development (In-Memory)

Production (S3 with IAM)

Tigris with Static Credentials

Local Filesystem

Next Steps

Deployment

Monitoring

Build docs developers (and LLMs) love

Get Started

Core Concepts

CLI

S2 Lite

SDKs

​Command-Line Arguments

​Storage Options

​Network Options

​TLS Options

​Initialization Options

​Environment Variables

​S2 Lite Configuration

​AWS Configuration

​SlateDB Configuration

​Logging Configuration

​Init File Format

​Schema

​Example

​Field Reference

​Basin Config

​Stream Config

​Performance Tuning

​Flush Interval

​Pipelining

​Storage Class

​Example Configurations

​Development (In-Memory)

​Production (S3 with IAM)

​Tigris with Static Credentials

​Local Filesystem

​Next Steps

Deployment

Monitoring

Build docs developers (and LLMs) love

Command-Line Arguments

Storage Options

Network Options

TLS Options

Initialization Options

Environment Variables

S2 Lite Configuration

AWS Configuration

SlateDB Configuration

Logging Configuration

Init File Format

Schema

Example

Field Reference

Basin Config

Stream Config

Performance Tuning

Flush Interval

Pipelining

Storage Class

Example Configurations

Development (In-Memory)

Production (S3 with IAM)

Tigris with Static Credentials

Local Filesystem

Next Steps