Overview
Each Redis data pod runs an instance reconciler that watches theRedisCluster CR from inside the pod. This enables live configuration updates without pod restarts.
Controller Name: instance-reconciler
Reconciliation Trigger: Any change to the RedisCluster resource
Reconciliation Steps
The reconciler executes these steps in order on every reconciliation:1. Fencing Check
Source:internal/instance-manager/reconciler/reconciler.go:96-102
Behavior:
- Reads
redis.io/fencedInstancesannotation - Parses JSON array of fenced pod names
- If current pod is in the list:
- Logs:
Pod is fenced, stopping redis-server - Sends
SIGINTto redis-server process - Skips all remaining steps
- Returns immediately
- Logs:
2. Role Reconciliation
Source:internal/instance-manager/reconciler/reconciler.go:104-108
Behavior:
The reconciler ensures the instance has the correct replication role based on status.currentPrimary and spec.replicaMode.
Replica Mode Enabled
Ifspec.replicaMode.enabled=true:
- Extract external source from
spec.replicaMode.source.hostandspec.replicaMode.source.port(default: 6379) - If
spec.replicaMode.promote=trueandstatus.currentPrimary == POD_NAME:- Issue
REPLICAOF NO ONEto promote out of replica mode - Record event:
ReplicaModePromoteRequested
- Issue
- Otherwise:
- Issue
REPLICAOF <source.host> <source.port>if not already replicating from that source - Record event:
ReplicaModeSourceUpdated
- Issue
Standard Mode
Ifspec.replicaMode.enabled=false or not set:
- Determine expected role:
isPrimary = (status.currentPrimary == POD_NAME)
- Query Redis
INFO replicationfor actual role - Compare expected vs actual:
- Issue
REPLICAOF NO ONE - Record event:
PromotedToPrimary
- Resolve primary pod IP via Kubernetes API
- Issue
REPLICAOF <primary-ip> 6379 - Record event:
DemotedToReplica
- Query current master host/port from
INFO replication - If not matching expected primary IP:
- Resolve primary pod IP
- Issue
REPLICAOF <primary-ip> 6379 - Record event:
ReplicaReconfigured
3. Config Reconciliation
Source:internal/instance-manager/reconciler/reconciler.go:110-116
Behavior:
- Iterate over
spec.redismap - Skip parameters requiring restart:
bind,port,tls-port,unixsocket,databases - For each live-reloadable parameter:
- Issue
CONFIG SET <key> <value>
- Issue
- Record event:
ConfigReloaded(only ifspec.redisis non-empty)
bindporttls-portunixsocketdatabases
maxmemorymaxmemory-policytcp-keepalivetimeoutsave
4. Secret Reconciliation
Source:internal/instance-manager/reconciler/reconciler.go:118-122
Behavior:
Reads secrets from projected volume mounts at /projected/<secret-name>/<key> and applies changes via Redis commands.
Auth Secret (spec.authSecret)
- Read password from
/projected/<authSecret.name>/password - Issue
CONFIG SET requirepass <password> - Issue
CONFIG SET masterauth <password> - Store password for replica mode auth override (see below)
ACL Config Secret (spec.aclConfigSecret)
- Read ACL rules from
/projected/<aclConfigSecret.name>/acl - Write to
/data/users.acl(path configurable viaREDIS_OPERATOR_ACL_FILE_PATH) - Issue
ACL LOADcommand
Replica Mode Auth Override
Ifspec.replicaMode.enabled=true and spec.replicaMode.source.authSecretName is set:
- Read upstream password from
/projected/<authSecretName>/password - Issue
CONFIG SET masterauth <upstream-password> - This overrides the local auth password for upstream replication
- Clear stale upstream auth:
CONFIG SET masterauth ""
5. TLS Certificate Rotation
Source:internal/instance-manager/reconciler/reconciler.go:124-128
Behavior:
Detects TLS certificate changes and reloads them without restarting redis-server.
Detection:
- Read certificate files:
/tls/tls.crt/tls/tls.key/tls/ca.crt
- Compute SHA256 checksum for each file
- Compare to cached checksums from previous reconciliation
- If any checksum changed:
- Issue
CONFIG SET tls-cert-file /tls/tls.crt - Issue
CONFIG SET tls-key-file /tls/tls.key - Issue
CONFIG SET tls-ca-cert-file /tls/ca.crt - Record event:
CertificatesRotated - Update cached checksums
- Issue
- On first reconciliation, cache checksums without issuing CONFIG SET
- Certificates are initially loaded at redis-server startup
6. Status Reporting
Source:internal/instance-manager/reconciler/reconciler.go:130-134
Behavior:
- Query Redis
INFO replication - Build
InstanceStatusobject:role:masterorslaveconnected:truereplicationOffset:master_repl_offset(primary) orslave_repl_offset(replica)connectedReplicas: Number of connected replicas (primary only)masterLinkStatus:upordown(replica only)
- Patch
status.instancesStatus[POD_NAME]using client-side merge patch
Environment Variables
Base directory for projected secret volumes
Path to write ACL configuration file
File Paths
| File | Purpose |
|---|---|
/projected/<secret-name>/<key> | Projected secret mounts |
/data/users.acl | ACL configuration file |
/tls/tls.crt | TLS certificate |
/tls/tls.key | TLS private key |
/tls/ca.crt | TLS CA certificate |
/data/dump.rdb | RDB backup file |
/data/appendonlydir/ | AOF directory |
Reconciliation Frequency
The instance reconciler is triggered by:- Watch events - Any change to the
RedisClusterresource - Requeue - Not used (no periodic requeue)
Error Handling
Fatal errors (stop reconciliation):- Cannot fetch
RedisClusterCR - Role reconciliation fails
- Status reporting fails
- Config reconciliation fails
- Secret reconciliation fails
- TLS certificate rotation fails
Events
The reconciler emits these Kubernetes events on theRedisCluster resource:
| Event Type | Reason | Trigger |
|---|---|---|
| Warning | InstanceFenced | Pod is fenced |
| Normal | PromotedToPrimary | Replica promoted to primary |
| Normal | DemotedToReplica | Primary demoted to replica |
| Normal | ReplicaReconfigured | Replica upstream changed |
| Normal | ReplicaModePromoteRequested | Promotion out of replica mode |
| Normal | ReplicaModeSourceUpdated | External source configured |
| Normal | ConfigReloaded | Redis config parameters reloaded |
| Normal | CertificatesRotated | TLS certificates reloaded |