Storage Configuration

The storage configuration defines how Infinitic persists workflow state, tags, and task metadata.

Package

io.infinitic.storage.config.StorageConfig

Supported Storage Backends

Infinitic supports multiple storage backends:

Redis - Fast in-memory data store (recommended for production)
PostgreSQL - Relational database
MySQL - Relational database
InMemory - In-memory storage (for testing only)

Common Properties

All storage configurations support:

compression

CompressionConfig

Compression configuration to reduce storage size

mode

String

Compression algorithm: "gzip" or "none"

compression:
  mode: gzip

cache

CacheConfig

Cache configuration to improve performance

caffeine

CaffeineCacheConfig

Caffeine cache configuration

maximumSize

Long

Maximum number of entries in cache

expireAfterAccess

Long

Seconds before cache entry expires after last access

expireAfterWrite

Long

Seconds before cache entry expires after creation

cache:
  caffeine:
    maximumSize: 10000
    expireAfterAccess: 3600

Redis Storage

Configuration Properties

redis.host

String

required

Redis server hostname

redis.port

Int

required

Redis server port

redis.username

String

Redis username (for Redis 6+ ACL)

redis.password

String

Redis password

redis.database

Int

default:"0"

Redis database number (0-15)

redis.timeout

Int

default:"2000"

Connection timeout in milliseconds

redis.ssl

Boolean

default:"false"

Enable SSL/TLS connection

redis.poolConfig

PoolConfig

Connection pool configuration

maxTotal

Int

default:"8"

Maximum total connections

maxIdle

Int

default:"8"

Maximum idle connections

minIdle

Int

default:"0"

Minimum idle connections

Example Redis YAML Configuration

storage:
  redis:
    host: redis.example.com
    port: 6379
    username: infinitic
    password: ${REDIS_PASSWORD}
    database: 0
    timeout: 5000
    ssl: true
    poolConfig:
      maxTotal: 20
      maxIdle: 10
      minIdle: 2
  compression:
    mode: gzip
  cache:
    caffeine:
      maximumSize: 10000
      expireAfterAccess: 3600

Kotlin Redis Configuration

import io.infinitic.storage.config.RedisStorageConfig
import io.infinitic.storage.config.RedisConfig
import io.infinitic.storage.compression.CompressionConfig
import io.infinitic.cache.config.CaffeineCacheConfig

val storageConfig = RedisStorageConfig.builder()
  .setHost("localhost")
  .setPort(6379)
  .setDatabase(0)
  .setPassword("secret")
  .setSsl(true)
  .setPoolConfig(
    RedisConfig.PoolConfig.builder()
      .setMaxTotal(20)
      .setMaxIdle(10)
      .setMinIdle(2)
      .build()
  )
  .setCompression(CompressionConfig.gzip())
  .setCache(
    CaffeineCacheConfig.builder()
      .setMaximumSize(10000)
      .setExpireAfterAccess(3600)
      .build()
  )
  .build()

PostgreSQL Storage

Configuration Properties

postgres.host

String

required

PostgreSQL server hostname

postgres.port

Int

required

PostgreSQL server port

postgres.username

String

required

PostgreSQL username

postgres.password

String

PostgreSQL password

postgres.database

String

default:"infinitic"

Database name

postgres.schema

String

default:"infinitic"

Schema name

postgres.keySetTable

String

default:"key_set_storage"

Table name for key-set storage

postgres.keyValueTable

String

default:"key_value_storage"

Table name for key-value storage

postgres.maximumPoolSize

Int

default:"10"

Maximum connection pool size

postgres.minimumIdle

Int

default:"10"

Minimum idle connections

postgres.idleTimeout

Long

default:"600000"

Idle timeout in milliseconds (10 minutes)

postgres.connectionTimeout

Long

default:"30000"

Connection timeout in milliseconds (30 seconds)

postgres.maxLifetime

Long

default:"1800000"

Maximum connection lifetime in milliseconds (30 minutes)

Example PostgreSQL YAML Configuration

storage:
  postgres:
    host: postgres.example.com
    port: 5432
    username: infinitic
    password: ${POSTGRES_PASSWORD}
    database: infinitic
    schema: infinitic
    keySetTable: key_set_storage
    keyValueTable: key_value_storage
    maximumPoolSize: 20
    minimumIdle: 5
    idleTimeout: 600000
    connectionTimeout: 30000
    maxLifetime: 1800000
  compression:
    mode: gzip
  cache:
    caffeine:
      maximumSize: 10000
      expireAfterAccess: 3600

Kotlin PostgreSQL Configuration

import io.infinitic.storage.config.PostgresStorageConfig
import io.infinitic.storage.compression.CompressionConfig
import io.infinitic.cache.config.CaffeineCacheConfig

val storageConfig = PostgresStorageConfig.builder()
  .setHost("localhost")
  .setPort(5432)
  .setUsername("infinitic")
  .setPassword("secret")
  .setDatabase("infinitic")
  .setSchema("infinitic")
  .setMaximumPoolSize(20)
  .setMinimumIdle(5)
  .setCompression(CompressionConfig.gzip())
  .setCache(
    CaffeineCacheConfig.builder()
      .setMaximumSize(10000)
      .setExpireAfterAccess(3600)
      .build()
  )
  .build()

Database Schema

Infinitic automatically creates required tables:

-- Key-value storage table
CREATE TABLE infinitic.key_value_storage (
  key TEXT PRIMARY KEY,
  value BYTEA NOT NULL
);

-- Key-set storage table
CREATE TABLE infinitic.key_set_storage (
  key TEXT NOT NULL,
  value TEXT NOT NULL,
  PRIMARY KEY (key, value)
);

CREATE INDEX idx_key_set_key ON infinitic.key_set_storage(key);

MySQL Storage

Configuration Properties

Similar to PostgreSQL with MySQL-specific defaults:

mysql.host

String

required

MySQL server hostname

mysql.port

Int

required

MySQL server port

mysql.username

String

required

MySQL username

mysql.password

String

MySQL password

mysql.database

String

default:"infinitic"

Database name

mysql.keySetTable

String

default:"key_set_storage"

Table name for key-set storage

mysql.keyValueTable

String

default:"key_value_storage"

Table name for key-value storage

Example MySQL YAML Configuration

storage:
  mysql:
    host: mysql.example.com
    port: 3306
    username: infinitic
    password: ${MYSQL_PASSWORD}
    database: infinitic
    keySetTable: key_set_storage
    keyValueTable: key_value_storage
    maximumPoolSize: 20
    minimumIdle: 5
  compression:
    mode: gzip
  cache:
    caffeine:
      maximumSize: 10000
      expireAfterAccess: 3600

InMemory Storage

For testing and development only:

storage:
  inMemory: {}

import io.infinitic.storage.config.InMemoryStorageConfig

val storageConfig = InMemoryStorageConfig()

Warning: InMemory storage loses all data when the worker stops.

Best Practices

Production Configuration

storage:
  redis:  # or postgres/mysql
    host: redis-cluster.prod.example.com
    port: 6379
    password: ${REDIS_PASSWORD}
    database: 0
    ssl: true
    poolConfig:
      maxTotal: 50
      maxIdle: 20
      minIdle: 5
  compression:
    mode: gzip  # Reduce storage costs
  cache:
    caffeine:
      maximumSize: 50000
      expireAfterAccess: 7200  # 2 hours

Development Configuration

storage:
  redis:
    host: localhost
    port: 6379
    database: 0
  cache:
    caffeine:
      maximumSize: 1000
      expireAfterAccess: 600

Testing Configuration

storage:
  inMemory: {}

Performance Tuning

Redis Optimization

Use connection pooling: Set appropriate maxTotal and minIdle
Enable pipelining: Reduces network round-trips
Use Redis Cluster: For horizontal scaling
Monitor memory usage: Set maxmemory policy
Use persistence: Enable AOF or RDB for durability

PostgreSQL Optimization

Index optimization: Ensure indexes on key columns
Connection pooling: Tune maximumPoolSize and minimumIdle
Vacuum regularly: Prevent table bloat
Partition tables: For large datasets
Monitor query performance: Use pg_stat_statements

Cache Tuning

cache:
  caffeine:
    maximumSize: 50000        # Adjust based on memory
    expireAfterAccess: 3600   # 1 hour
    expireAfterWrite: 7200    # 2 hours

Monitoring

Redis Metrics

# Monitor Redis
redis-cli INFO stats
redis-cli INFO memory
redis-cli DBSIZE

PostgreSQL Metrics

-- Check table sizes
SELECT 
  schemaname,
  tablename,
  pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
FROM pg_tables
WHERE schemaname = 'infinitic';

-- Check query performance
SELECT * FROM pg_stat_statements
WHERE query LIKE '%key_value_storage%'
ORDER BY total_exec_time DESC
LIMIT 10;

Backup and Recovery

Redis Backup

# Save snapshot
redis-cli SAVE

# Copy RDB file
cp /var/lib/redis/dump.rdb /backup/dump-$(date +%Y%m%d).rdb

PostgreSQL Backup

# Backup database
pg_dump -h localhost -U infinitic infinitic > backup-$(date +%Y%m%d).sql

# Restore database
psql -h localhost -U infinitic infinitic < backup-20240101.sql

Migration

To migrate between storage backends:

Stop all workers
Export data from old storage
Import data to new storage
Update configuration
Start workers with new configuration

Troubleshooting

Connection Issues

# Enable storage logging
logging:
  level:
    io.infinitic.storage: DEBUG
    io.infinitic.cache: DEBUG

High Memory Usage

Reduce cache size
Enable compression
Increase expiration times
Monitor workflow state sizes

Slow Queries

Check database indexes
Increase connection pool size
Enable query logging
Optimize database configuration

Core Classes

Annotations

Configuration

​Package

​Supported Storage Backends

​Common Properties

​Redis Storage

​Configuration Properties

​Example Redis YAML Configuration

​Kotlin Redis Configuration

​PostgreSQL Storage

​Configuration Properties

​Example PostgreSQL YAML Configuration

​Kotlin PostgreSQL Configuration

​Database Schema

​MySQL Storage

​Configuration Properties

​Example MySQL YAML Configuration

​InMemory Storage

​Best Practices

​Production Configuration

​Development Configuration

​Testing Configuration

​Performance Tuning

​Redis Optimization

​PostgreSQL Optimization

​Cache Tuning

​Monitoring

​Redis Metrics

​PostgreSQL Metrics

​Backup and Recovery

​Redis Backup

​PostgreSQL Backup

​Migration

​Troubleshooting

​Connection Issues

​High Memory Usage

​Slow Queries

​See Also

Build docs developers (and LLMs) love

Package

Supported Storage Backends

Common Properties

Redis Storage

Configuration Properties

Example Redis YAML Configuration

Kotlin Redis Configuration

PostgreSQL Storage

Configuration Properties

Example PostgreSQL YAML Configuration

Kotlin PostgreSQL Configuration

Database Schema

MySQL Storage

Configuration Properties

Example MySQL YAML Configuration

InMemory Storage

Best Practices

Production Configuration

Development Configuration

Testing Configuration

Performance Tuning

Redis Optimization

PostgreSQL Optimization

Cache Tuning

Monitoring

Redis Metrics

PostgreSQL Metrics

Backup and Recovery

Redis Backup

PostgreSQL Backup

Migration

Troubleshooting

Connection Issues

High Memory Usage

Slow Queries

See Also