Skip to main content

​
Overview

Blnk uses Redis as both a message queue for transaction processing and a cache layer. This guide covers Redis setup, connection pooling, clustering, persistence, and security.

​
Requirements

  • Redis 7.2.4 or later
  • Minimum 2GB RAM (4GB+ recommended for production)
  • Persistent storage for AOF/RDB files
  • Network connectivity between Blnk and Redis

​
Installation

redis:
  image: redis:7.2.4
  container_name: redis
  restart: on-failure
  ports:
    - "6379:6379"
  volumes:
    - redis_data:/data
  command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD}

​
Ubuntu/Debian

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update
sudo apt-get install redis-server

​
macOS

brew install redis
brew services start redis

​
Redis Configuration

Blnk’s Redis configuration is defined in internal/redis-db/redisdb.go.

​
Connection Modes

Blnk supports both standalone and cluster Redis deployments: Standalone Redis: 
addresses := []string{"localhost:6379"}
redis, err := redis_db.NewRedisClient(addresses, false, nil)
Redis Cluster: 
addresses := []string{
    "redis-node1:6379",
    "redis-node2:6379",
    "redis-node3:6379",
}
redis, err := redis_db.NewRedisClient(addresses, false, nil)

​
Connection String Formats

Blnk supports multiple Redis URL formats: Basic: 
redis:6379
With authentication: 
redis://:password@redis:6379
Full URL: 
redis://username:password@redis:6379/0
Azure Redis Cache: 
redis://:[email protected]:6380

​
Configuration in blnk.json

{
  "redis": {
    "dns": "redis://localhost:6379",
    "skip_tls_verify": false,
    "pool_size": 100,
    "min_idle_conns": 20
  }
}

​
Environment Variables

export BLNK_REDIS_DNS="redis://:password@localhost:6379"
export BLNK_REDIS_SKIP_TLS_VERIFY=false
export BLNK_REDIS_POOL_SIZE=100
export BLNK_REDIS_MIN_IDLE_CONNS=20

​
Connection Pool Configuration

​
Default Pool Settings

From config/config.go:67-70: 
defaultRedis = RedisConfig{
    PoolSize:     100,
    MinIdleConns: 20,
}

​
Custom Pool Configuration

poolConfig := &redis_db.PoolConfig{
    PoolSize:     200,
    MinIdleConns: 50,
}

redis, err := redis_db.NewRedisClient(addresses, false, poolConfig)

​
Production Pool Tuning

High-throughput workload: 
{
  "redis": {
    "pool_size": 200,
    "min_idle_conns": 50
  }
}
Memory-constrained environment: 
{
  "redis": {
    "pool_size": 50,
    "min_idle_conns": 10
  }
}

​
Pool Size Formula

pool_size = (worker_count * concurrent_jobs) + buffer
For 10 workers processing 5 concurrent jobs: 
pool_size = (10 * 5) + 50 = 100

​
Queue Configuration

Blnk uses multiple Redis queues for different operations.

​
Queue Names

From config/config.go:57-65: 
defaultQueue = QueueConfig{
    TransactionQueue:    "new:transaction",
    WebhookQueue:        "new:webhook",
    IndexQueue:          "new:index",
    InflightExpiryQueue: "new:inflight-expiry",
    NumberOfQueues:      20,
    MonitoringPort:      "5004",
    WebhookConcurrency:  20,
}

​
Queue Configuration

{
  "queue": {
    "transaction_queue": "new:transaction",
    "webhook_queue": "new:webhook",
    "index_queue": "new:index",
    "inflight_expiry_queue": "new:inflight-expiry",
    "number_of_queues": 20,
    "monitoring_port": "5004",
    "webhook_concurrency": 20,
    "insufficient_fund_retries": true,
    "max_retry_attempts": 3
  }
}

​
Environment Variables

export BLNK_QUEUE_TRANSACTION="new:transaction"
export BLNK_QUEUE_WEBHOOK="new:webhook"
export BLNK_QUEUE_INDEX="new:index"
export BLNK_QUEUE_INFLIGHT_EXPIRY="new:inflight-expiry"
export BLNK_QUEUE_NUMBER_OF_QUEUES=20
export BLNK_QUEUE_WEBHOOK_CONCURRENCY=20

​
TLS/SSL Configuration

​
Enable TLS

Blnk automatically enables TLS for Azure Redis Cache and other services requiring SSL. From internal/redis-db/redisdb.go:88-92: 
// Enable TLS for Azure Redis
if strings.Contains(host, "redis.cache.windows.net") {
    opts.TLSConfig = &tls.Config{
        MinVersion: tls.VersionTLS12,
    }
}

​
Custom TLS Configuration

{
  "redis": {
    "dns": "rediss://redis.example.com:6380",
    "skip_tls_verify": false
  }
}

​
Skip TLS Verification (Development Only)

{
  "redis": {
    "dns": "rediss://redis.example.com:6380",
    "skip_tls_verify": true
  }
}
Warning: Never use skip_tls_verify: true in production.

​
Redis Cluster Setup

​
Create Cluster

# Create 6 Redis nodes (3 masters, 3 replicas)
for port in 7000 7001 7002 7003 7004 7005; do
  mkdir -p redis-cluster/${port}
  cat > redis-cluster/${port}/redis.conf <<EOF
port ${port}
cluster-enabled yes
cluster-config-file nodes-${port}.conf
cluster-node-timeout 5000
appendonly yes
EOF
  redis-server redis-cluster/${port}/redis.conf &
done

# Create cluster
redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

​
Blnk Cluster Configuration

{
  "redis": {
    "dns": "redis://node1:7000,redis://node2:7001,redis://node3:7002"
  }
}
Or using multiple addresses in code: 
addresses := []string{
    "redis://node1:7000",
    "redis://node2:7001",
    "redis://node3:7002",
}

​
Docker Compose Cluster

version: '3.8'
services:
  redis-node1:
    image: redis:7.2.4
    command: redis-server --cluster-enabled yes --port 7000
    ports:
      - "7000:7000"

  redis-node2:
    image: redis:7.2.4
    command: redis-server --cluster-enabled yes --port 7001
    ports:
      - "7001:7001"

  redis-node3:
    image: redis:7.2.4
    command: redis-server --cluster-enabled yes --port 7002
    ports:
      - "7002:7002"

​
Persistence Configuration

​
AOF (Append-Only File)

Recommended for financial data: 
# redis.conf
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

​
RDB Snapshots

# redis.conf
save 900 1      # Save after 900s if 1 key changed
save 300 10     # Save after 300s if 10 keys changed
save 60 10000   # Save after 60s if 10000 keys changed
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
Use both AOF and RDB: 
appendonly yes
aof-use-rdb-preamble yes
save 900 1
save 300 10
save 60 10000

​
Security Configuration

​
Authentication

# redis.conf
requirepass your_strong_password_here
In Blnk configuration: 
{
  "redis": {
    "dns": "redis://:your_strong_password_here@localhost:6379"
  }
}

​
ACL (Access Control Lists)

Create limited user for Blnk: 
redis-cli
127.0.0.1:6379> ACL SETUSER blnk on >password ~* +@all
127.0.0.1:6379> ACL SAVE
Connection string: 
redis://blnk:password@localhost:6379

​
Network Security

# redis.conf
bind 127.0.0.1 ::1  # Only local connections
protected-mode yes
port 6379

# For remote access, use specific IPs
bind 0.0.0.0
requirepass strong_password

​
TLS Encryption

# redis.conf
tls-port 6380
port 0  # Disable non-TLS
tls-cert-file /path/to/redis.crt
tls-key-file /path/to/redis.key
tls-ca-cert-file /path/to/ca.crt

​
Monitoring

​
Health Check

Blnk automatically pings Redis on startup (internal/redis-db/redisdb.go:187-192): 
ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond)
defer cancel()
_, err := client.Ping(ctx).Result()

​
Redis CLI Monitoring

# Connection info
redis-cli INFO clients

# Memory usage
redis-cli INFO memory

# Queue lengths
redis-cli LLEN "new:transaction"
redis-cli LLEN "new:webhook"

# Real-time monitoring
redis-cli MONITOR

# Slow log
redis-cli SLOWLOG GET 10

​
Key Metrics

# Connected clients
redis-cli INFO clients | grep connected_clients

# Memory usage
redis-cli INFO memory | grep used_memory_human

# Operations per second
redis-cli INFO stats | grep instantaneous_ops_per_sec

# Hit rate
redis-cli INFO stats | grep keyspace_hits
redis-cli INFO stats | grep keyspace_misses

​
Queue Monitoring

Monitor Blnk worker queues: 
# Transaction queue depth
redis-cli LLEN "new:transaction"

# Webhook queue depth
redis-cli LLEN "new:webhook"

# Index queue depth
redis-cli LLEN "new:index"

# All queue keys
redis-cli KEYS "new:*"

​
Performance Tuning

​
Memory Optimization

# redis.conf
maxmemory 2gb
maxmemory-policy allkeys-lru
maxmemory-samples 5

​
Network Tuning

tcp-backlog 511
tcp-keepalive 300
timeout 0

​
Thread Configuration (Redis 6+)

io-threads 4
io-threads-do-reads yes

​
Disable Expensive Commands

rename-command FLUSHDB ""
rename-command FLUSHALL ""
rename-command KEYS ""
rename-command CONFIG "CONFIG_8f3a9c2b"

​
Backup and Recovery

​
Manual Backup

# Create RDB snapshot
redis-cli BGSAVE

# Copy dump file
cp /var/lib/redis/dump.rdb /backups/dump_$(date +%Y%m%d).rdb

# Copy AOF file
cp /var/lib/redis/appendonly.aof /backups/aof_$(date +%Y%m%d).aof

​
Automated Backup Script

#!/bin/bash
BACKUP_DIR="/backups/redis"
DATE=$(date +%Y%m%d_%H%M%S)

# Trigger background save
redis-cli BGSAVE

# Wait for save to complete
while [ $(redis-cli LASTSAVE) -eq $(redis-cli LASTSAVE) ]; do
  sleep 1
done

# Copy files
cp /var/lib/redis/dump.rdb "$BACKUP_DIR/dump_$DATE.rdb"
gzip "$BACKUP_DIR/dump_$DATE.rdb"

# Remove old backups (keep 7 days)
find "$BACKUP_DIR" -name "dump_*.rdb.gz" -mtime +7 -delete

​
Restore from Backup

# Stop Redis
sudo systemctl stop redis

# Replace dump file
cp /backups/dump_20260304.rdb /var/lib/redis/dump.rdb
chown redis:redis /var/lib/redis/dump.rdb

# Start Redis
sudo systemctl start redis

​
High Availability

​
Redis Sentinel

For automatic failover: 
# sentinel.conf
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel auth-pass mymaster your_password
sentinel down-after-milliseconds mymaster 5000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000
Start Sentinel: 
redis-sentinel /etc/redis/sentinel.conf

​
Replication Setup

Primary: 
# redis.conf
bind 0.0.0.0
requirepass primary_password
Replica: 
# redis.conf
replicaof primary-host 6379
masterauth primary_password
replica-read-only yes

​
Troubleshooting

​
Connection Refused

Check Redis is running: 
redis-cli ping
Check network binding: 
redis-cli CONFIG GET bind

​
Out of Memory

Check memory usage: 
redis-cli INFO memory
Increase maxmemory or enable eviction: 
maxmemory 4gb
maxmemory-policy allkeys-lru

​
Slow Performance

Check slow log: 
redis-cli SLOWLOG GET 10
Enable latency monitoring: 
redis-cli CONFIG SET latency-monitor-threshold 100
redis-cli LATENCY DOCTOR

​
Queue Buildup

Check worker status: 
curl http://localhost:5004/metrics
Increase worker concurrency: 
{
  "transaction": {
    "max_workers": 20
  },
  "queue": {
    "webhook_concurrency": 50
  }
}

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us