Secret Management

Redis Operator uses Kubernetes Secrets for sensitive configuration like passwords, TLS certificates, and backup credentials. All secrets are injected as projected volumes (never environment variables).

Secret Types

The operator supports five secret types:

Secret Field	Required Keys	Purpose
`authSecret`	`password`	Redis authentication
`aclConfigSecret`	`acl`	ACL user rules
`tlsSecret`	`tls.crt`, `tls.key`	TLS encryption
`caSecret`	`ca.crt`	Client cert verification
`backupCredentialsSecret`	`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (S3) or `GOOGLE_APPLICATION_CREDENTIALS` (GCS)	Backup storage

Auto-Generated Auth Secret

If spec.authSecret is not specified, the operator auto-generates a secret with a random password. Implementation (internal/controller/cluster/secrets.go:78-117):

func (r *ClusterReconciler) ensureAuthSecret(
    ctx context.Context,
    cluster *redisv1.RedisCluster,
    secretName string,
) error {
    // Check if secret exists
    var existing corev1.Secret
    err := r.Get(ctx, types.NamespacedName{
        Name:      secretName,
        Namespace: cluster.Namespace,
    }, &existing)
    if err == nil {
        return nil // Already exists
    }
    
    // Generate random password (16 bytes hex-encoded = 32 chars)
    passwordBytes := make([]byte, 16)
    rand.Read(passwordBytes)
    password := hex.EncodeToString(passwordBytes)
    
    // Create secret
    secret := &corev1.Secret{
        ObjectMeta: metav1.ObjectMeta{
            Name:      secretName,
            Namespace: cluster.Namespace,
            Labels: map[string]string{
                "redis.io/cluster": cluster.Name,
            },
        },
        Type: corev1.SecretTypeOpaque,
        Data: map[string][]byte{
            "password": []byte(password),
        },
    }
    return r.Create(ctx, secret)
}

Example:

apiVersion: redis.io/v1
kind: RedisCluster
metadata:
  name: my-cluster
spec:
  instances: 3
  storage:
    size: 10Gi
  # authSecret not specified - auto-generated

Operator creates:

apiVersion: v1
kind: Secret
metadata:
  name: my-cluster-auth
  labels:
    redis.io/cluster: my-cluster
type: Opaque
data:
  password: YjM0ZjU2Nzg5MGFiY2RlZjEyMzQ1Njc4OTBhYmNkZWY=  # base64

Retrieve the password:

kubectl get secret my-cluster-auth -o jsonpath='{.data.password}' | base64 -d

Projected Volumes

All secrets are mounted into pods as a single projected volume at /projected. Why projected volumes?

Atomic updates (all-or-nothing)
No environment variable leakage in logs/dumps
Supports secret rotation without pod restart

Implementation (internal/controller/cluster/pods.go:189-216):

var projectedSources []corev1.VolumeProjection
projectedSecretNames := map[string]struct{}{}

secretRefs := []*redisv1.LocalObjectReference{
    cluster.Spec.AuthSecret,
    cluster.Spec.ACLConfigSecret,
}

for _, ref := range secretRefs {
    if ref != nil && ref.Name != "" {
        projectedSecretNames[ref.Name] = struct{}{}
    }
}

if sourceAuthSecretName := replicaModeSourceAuthSecretName(cluster); sourceAuthSecretName != "" {
    projectedSecretNames[sourceAuthSecretName] = struct{}{}
}

if len(projectedSecretNames) > 0 {
    secretNames := make([]string, 0, len(projectedSecretNames))
    for name := range projectedSecretNames {
        secretNames = append(secretNames, name)
    }
    sort.Strings(secretNames)  // Deterministic ordering
    for _, name := range secretNames {
        projectedSources = append(projectedSources, corev1.VolumeProjection{
            Secret: &corev1.SecretProjection{
                LocalObjectReference: corev1.LocalObjectReference{Name: name},
            },
        })
    }
}

Pod volume:

volumes:
  - name: projected-secrets
    projected:
      sources:
        - secret:
            name: my-cluster-auth
        - secret:
            name: redis-acl-rules
        - secret:
            name: redis-tls

Mounted files:

/projected/
├── password        # from my-cluster-auth
├── acl             # from redis-acl-rules
├── tls.crt         # from redis-tls
└── tls.key         # from redis-tls

Secret Rotation

The operator tracks secret ResourceVersion to detect changes and trigger reconciliation.

Status Tracking

status.secretsResourceVersion maps secret names to their current ResourceVersion:

status:
  secretsResourceVersion:
    my-cluster-auth: "12345"
    redis-acl-rules: "67890"
    redis-tls: "11223"

Implementation (internal/controller/cluster/secrets.go:19-76):

func (r *ClusterReconciler) reconcileSecrets(
    ctx context.Context,
    cluster *redisv1.RedisCluster,
) error {
    // Build map of all secret references
    secretRefs := map[string]*redisv1.LocalObjectReference{
        "authSecret":              cluster.Spec.AuthSecret,
        "aclConfigSecret":         cluster.Spec.ACLConfigSecret,
        "tlsSecret":               cluster.Spec.TLSSecret,
        "caSecret":                cluster.Spec.CASecret,
        "backupCredentialsSecret": cluster.Spec.BackupCredentialsSecret,
    }
    
    // Fetch current ResourceVersion for each secret
    newVersions := make(map[string]string)
    for refName, ref := range secretRefs {
        if ref == nil {
            continue
        }
        var secret corev1.Secret
        if err := r.Get(ctx, types.NamespacedName{
            Name:      ref.Name,
            Namespace: cluster.Namespace,
        }, &secret); err != nil {
            if errors.IsNotFound(err) {
                logger.Info("Secret not found", "secret", ref.Name, "ref", refName)
                continue
            }
            return fmt.Errorf("getting secret %s: %w", ref.Name, err)
        }
        newVersions[ref.Name] = secret.ResourceVersion
    }
    
    // Detect rotation (ResourceVersion changed)
    for name, newVer := range newVersions {
        if oldVer, ok := cluster.Status.SecretsResourceVersion[name]; ok && oldVer != newVer {
            r.Recorder.Eventf(
                cluster,
                corev1.EventTypeNormal,
                "SecretRotated",
                "Secret %s rotated (resourceVersion %s -> %s)",
                name, oldVer, newVer,
            )
        }
    }
    
    // Update status
    patch := client.MergeFrom(cluster.DeepCopy())
    cluster.Status.SecretsResourceVersion = newVersions
    return r.Status().Patch(ctx, cluster, patch)
}

Rotation Workflow

Update the secret:

kubectl create secret generic my-cluster-auth \
  --from-literal=password=new-secure-password \
  --dry-run=client -o yaml | kubectl apply -f -

Operator detects change:
- Reconciliation triggered by secret watch
- reconcileSecrets() compares ResourceVersion
- Event emitted: SecretRotated
Projected volume updated:
- Kubelet updates /projected/* files (eventually consistent, ~1 minute)
- Instance manager detects file changes via filesystem watch
- New password applied via CONFIG SET requirepass

Verify rotation:

kubectl describe rediscluster my-cluster
# Events:
#   Type    Reason         Message
#   ----    ------         -------
#   Normal  SecretRotated  Secret my-cluster-auth rotated (resourceVersion 12345 -> 12346)

ACL Rotation

ACL rotation is live (no pod restart required):

Update aclConfigSecret:

kubectl create secret generic redis-acl-rules \
  --from-file=acl=updated-acl.txt \
  --dry-run=client -o yaml | kubectl apply -f -

Instance manager detects /projected/acl change
Runs ACL LOAD to reload rules
No connection interruption

TLS Rotation

TLS rotation requires pod restart (Redis limitation):

Update tlsSecret:

kubectl create secret tls redis-tls \
  --cert=new-tls.crt \
  --key=new-tls.key \
  --dry-run=client -o yaml | kubectl apply -f -

Operator detects rotation
Rolling update triggered (replicas first, then primary)
Each pod restarts with new TLS config

Secret Examples

Auth Secret

apiVersion: v1
kind: Secret
metadata:
  name: redis-password
type: Opaque
stringData:
  password: "my-secure-password-123"

Reference:

spec:
  authSecret:
    name: redis-password

ACL Config Secret

apiVersion: v1
kind: Secret
metadata:
  name: redis-acl-rules
type: Opaque
stringData:
  acl: |
    user admin on >adminpass ~* &* +@all
    user readonly on >readpass ~* &* +@read -@write
    user app on >apppass ~app:* &* +@all -@dangerous

Reference:

spec:
  aclConfigSecret:
    name: redis-acl-rules

Connect as admin:

redis-cli -h my-cluster-leader.default.svc.cluster.local \
  --user admin --pass adminpass

TLS Certificates

Generate self-signed cert (testing only):

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=*.default.svc.cluster.local"

kubectl create secret tls redis-tls \
  --cert=tls.crt \
  --key=tls.key

Reference:

spec:
  tlsSecret:
    name: redis-tls

Connect with TLS:

redis-cli -h my-cluster-leader.default.svc.cluster.local \
  --tls --cacert ca.crt

Client CA Certificate

apiVersion: v1
kind: Secret
metadata:
  name: redis-ca
type: Opaque
data:
  ca.crt: LS0tLS...  # base64-encoded CA certificate

Reference:

spec:
  caSecret:
    name: redis-ca

Enables mutual TLS (client cert verification).

Backup Credentials (S3)

apiVersion: v1
kind: Secret
metadata:
  name: s3-backup-creds
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
  AWS_SECRET_ACCESS_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Reference:

spec:
  backupCredentialsSecret:
    name: s3-backup-creds

Backup Credentials (GCS)

apiVersion: v1
kind: Secret
metadata:
  name: gcs-backup-creds
type: Opaque
stringData:
  GOOGLE_APPLICATION_CREDENTIALS: |
    {
      "type": "service_account",
      "project_id": "my-project",
      "private_key_id": "abc123...",
      "private_key": "-----BEGIN PRIVATE KEY-----\n...",
      "client_email": "[email protected]",
      "client_id": "123456789",
      "auth_uri": "https://accounts.google.com/o/oauth2/auth",
      "token_uri": "https://oauth2.googleapis.com/token"
    }

Best Practices

Use external secret management

Integrate with External Secrets Operator:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: redis-password
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: redis-password
    template:
      type: Opaque
      data:
        password: "{{ .password }}"
  data:
    - secretKey: password
      remoteRef:
        key: redis/prod/password

Reference in RedisCluster:

spec:
  authSecret:
    name: redis-password  # Managed by ExternalSecret

Rotate secrets regularly

Automate rotation with a CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: rotate-redis-password
spec:
  schedule: "0 2 1 * *"  # Monthly at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: secret-rotator
          containers:
            - name: rotator
              image: bitnami/kubectl:latest
              command:
                - /bin/bash
                - -c
                - |
                  NEW_PASSWORD=$(openssl rand -base64 32)
                  kubectl create secret generic my-cluster-auth \
                    --from-literal=password="$NEW_PASSWORD" \
                    --dry-run=client -o yaml | kubectl apply -f -
          restartPolicy: OnFailure

Never commit secrets to Git

Use sealed secrets or SOPS for GitOps:

# Encrypt with Sealed Secrets
kubectl create secret generic redis-password \
  --from-literal=password=my-password \
  --dry-run=client -o yaml | \
  kubeseal -o yaml > sealed-redis-password.yaml

# Commit sealed-redis-password.yaml (safe to store in Git)

Principle of least privilege

Create separate auth secrets per application:

# App 1 secret
apiVersion: v1
kind: Secret
metadata:
  name: app1-redis-auth
stringData:
  password: "app1-password"

# App 2 secret
apiVersion: v1
kind: Secret
metadata:
  name: app2-redis-auth
stringData:
  password: "app2-password"

Use ACLs to restrict access:

spec:
  aclConfigSecret:
    name: redis-acl
# ACL content:
# user app1 on >app1-password ~app1:* +@all
# user app2 on >app2-password ~app2:* +@all

Troubleshooting

Secret not found error

Symptom:

$ kubectl get events --field-selector involvedObject.name=my-cluster
Warning  SecretNotFound  Secret "redis-password" not found

Solution: Create the secret before the cluster:

kubectl create secret generic redis-password \
  --from-literal=password=my-password

kubectl apply -f rediscluster.yaml

Projected volume not updating

Symptom: Updated secret but pods still use old value. Cause: Kubelet caches projected volumes (update latency ~60s). Debug:

# Check file timestamp inside pod
kubectl exec my-cluster-0 -- ls -la /projected/password

# Force refresh by deleting pod (operator recreates it)
kubectl delete pod my-cluster-0

Wrong secret key name

Symptom: Instance manager fails to start. Cause: Secret uses wrong key name (e.g., pass instead of password). Solution: Ensure exact key names:

authSecret → password
aclConfigSecret → acl
tlsSecret → tls.crt, tls.key
caSecret → ca.crt

kubectl get secret redis-password -o jsonpath='{.data}'
# Should show: {"password":"..."}

Get Started

Core Concepts

Configuration

Operations

Runbooks

Secret Types

Auto-Generated Auth Secret

Projected Volumes

Secret Rotation

Status Tracking

Rotation Workflow

ACL Rotation

TLS Rotation

Secret Examples

Auth Secret

ACL Config Secret

TLS Certificates

Client CA Certificate

Backup Credentials (S3)

Backup Credentials (GCS)

Best Practices

Use external secret management

Rotate secrets regularly

Never commit secrets to Git

Principle of least privilege

Troubleshooting

Secret not found error

Projected volume not updating

Wrong secret key name

Build docs developers (and LLMs) love

Get Started

Core Concepts

Configuration

Operations

Runbooks

​Secret Types

​Auto-Generated Auth Secret

​Projected Volumes

​Secret Rotation

​Status Tracking

​Rotation Workflow

​ACL Rotation

​TLS Rotation

​Secret Examples

​Auth Secret

​ACL Config Secret

​TLS Certificates

​Client CA Certificate

​Backup Credentials (S3)

​Backup Credentials (GCS)

​Best Practices

​Use external secret management

​Rotate secrets regularly

​Never commit secrets to Git

​Principle of least privilege

​Troubleshooting

​Secret not found error

​Projected volume not updating

​Wrong secret key name

Build docs developers (and LLMs) love

Secret Types

Auto-Generated Auth Secret

Projected Volumes

Secret Rotation

Status Tracking

Rotation Workflow

ACL Rotation

TLS Rotation

Secret Examples

Auth Secret

ACL Config Secret

TLS Certificates

Client CA Certificate

Backup Credentials (S3)

Backup Credentials (GCS)

Best Practices

Use external secret management

Rotate secrets regularly

Never commit secrets to Git

Principle of least privilege

Troubleshooting

Secret not found error

Projected volume not updating

Wrong secret key name