Skip to main content

​
Overview

Sealed Secrets provides a secure way to store Kubernetes secrets in Git repositories. It encrypts secrets using asymmetric cryptography, allowing you to commit encrypted secrets to version control while maintaining security.

​
Why Sealed Secrets?

Standard Kubernetes Secrets are base64-encoded, not encrypted. This makes them unsafe to commit to Git. Sealed Secrets solves this by:
  • Encrypting secrets with a public key
  • Storing encrypted secrets safely in Git
  • Automatically decrypting secrets in the cluster using a private key
  • Preventing secret exposure in version control

​
Security Model

​
Encryption Process

  1. Public key encryption: Secrets are encrypted using a public certificate
  2. Safe storage: Encrypted SealedSecret resources can be committed to Git
  3. Cluster decryption: The controller decrypts secrets using a private key stored only in the cluster
  4. Automatic creation: Kubernetes Secret is automatically created from the SealedSecret

​
Key Management

  • Private key: Stored securely in the kube-system namespace, never leaves the cluster
  • Public certificate: Can be safely distributed for encryption
  • Key rotation: Supports automatic key rotation for enhanced security

​
Installation

​
Install Sealed Secrets Controller

Deploy the controller using Helm: 
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets -n kube-system \
  --set-string fullnameOverride=sealed-secrets-controller \
  sealed-secrets/sealed-secrets

​
Install kubeseal CLI

Install the kubeseal client tool for encrypting secrets: 
# Set version (check latest at https://github.com/bitnami-labs/sealed-secrets/releases)
KUBESEAL_VERSION='0.29.0'

# Download and install
curl -OL "https://github.com/bitnami-labs/sealed-secrets/releases/download/v${KUBESEAL_VERSION}/kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz"
tar -xvzf kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz kubeseal
sudo install -m 755 kubeseal /usr/local/bin/kubeseal

​
Verify Installation

Fetch the public certificate to verify the controller is running: 
kubeseal --fetch-cert
You should see the public certificate output in PEM format.

​
Creating Sealed Secrets

​
Method 1: From Literal Values

Create a sealed secret directly from literal values: 
kubectl create secret generic exchange-router-secret \
  --dry-run=client \
  --from-literal=database_url="postgresql://user:pass@host:5432/db" \
  --from-literal=redis_url="redis://redis:6379" \
  --from-literal=server_addr="0.0.0.0:3000" \
  -o yaml | \
kubeseal \
  --controller-name=sealed-secrets-controller \
  --controller-namespace=kube-system \
  --format yaml > sealed-secret.yml

​
Method 2: From Existing Secret File

If you have an existing secret YAML file: 
kubeseal --format yaml < secret.yml > sealed-secret.yml

​
Method 3: Offline Encryption

For CI/CD pipelines without cluster access, fetch the certificate once: 
# Fetch and save public certificate
kubeseal \
  --controller-name=sealed-secrets-controller \
  --controller-namespace=kube-system \
  --fetch-cert > mycert.pem
Then encrypt secrets using the local certificate: 
kubectl create secret generic secret-name \
  --dry-run=client \
  --from-literal=key=value \
  -o yaml | \
kubeseal \
  --controller-name=sealed-secrets-controller \
  --controller-namespace=kube-system \
  --format yaml \
  --cert mycert.pem > sealed-secret.yml

​
Sealed Secret Structure

​
Example SealedSecret Resource

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: exchange-router-secret
  namespace: default
spec:
  encryptedData:
    database_url: AgAUEdKobv+RYQTmkqi/4zFp38QYoE0mEk8HkU4Ra8U7ZP2HtewGOspTT0PB...
    redis_url: AgA7Tzk4yAFU7Lxwv4OlgpJyXxi9hr/7qC4IyxD3bPAkaVPHi/6eS9P9P7wC...
    server_addr: AgDNCrl5xf/DQLaY/jSYWxaNGHPTwSUOWRChWedYoHRS2ElVoI+p7O569NGn...
  template:
    metadata:
      creationTimestamp: null
      name: exchange-router-secret
      namespace: default
Key fields:
  • encryptedData: Each secret key is encrypted separately
  • template.metadata: Metadata for the generated Kubernetes Secret
  • namespace: SealedSecret and Secret must be in the same namespace

​
Deploying Sealed Secrets

​
Apply to Cluster

kubectl apply -f sealed-secret.yml
The controller automatically:
  1. Detects the SealedSecret resource
  2. Decrypts the data using the private key
  3. Creates a standard Kubernetes Secret

​
Verify Deployment

Check that the Secret was created: 
# List secrets
kubectl get secret

# View decrypted secret (base64 encoded)
kubectl get secret exchange-router-secret -o yaml

# Decode a specific value
kubectl get secret exchange-router-secret \
  -o jsonpath='{.data.database_url}' | base64 --decode

​
Using Secrets in Deployments

Reference the decrypted secret in your deployments: 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: exchange-router
spec:
  template:
    spec:
      containers:
      - name: router
        image: exchange-router:latest
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: exchange-router-secret
              key: database_url
        - name: REDIS_URL
          valueFrom:
            secretKeyRef:
              name: exchange-router-secret
              key: redis_url

​
Scope and Permissions

Sealed Secrets support three encryption scopes:

​
1. Strict (default)

Secret is sealed to a specific namespace and name: 
kubeseal --scope strict < secret.yml > sealed-secret.yml

​
2. Namespace-wide

Secret can be decrypted in any Secret with any name in the same namespace: 
kubeseal --scope namespace-wide < secret.yml > sealed-secret.yml

​
3. Cluster-wide

Secret can be decrypted in any namespace: 
kubeseal --scope cluster-wide < secret.yml > sealed-secret.yml
Use cluster-wide scope sparingly. The strict scope provides the best security by binding secrets to specific namespaces and names.

​
Key Rotation and Renewal

​
Automatic Key Renewal

The controller automatically generates new keys every 30 days but keeps old keys to decrypt existing secrets.

​
Manual Key Rotation

Force key rotation: 
kubectl delete secret -n kube-system sealed-secrets-key
kubectl rollout restart deployment -n kube-system sealed-secrets-controller

​
Re-encrypting After Rotation

After key rotation, re-encrypt secrets with the new key: 
# Fetch new certificate
kubeseal --fetch-cert > new-cert.pem

# Re-encrypt secrets
kubeseal --cert new-cert.pem < secret.yml > sealed-secret.yml

​
Backup and Disaster Recovery

​
Backup Private Keys

Backup the private key for disaster recovery: 
kubectl get secret -n kube-system \
  -l sealedsecrets.bitnami.com/sealed-secrets-key=active \
  -o yaml > sealed-secrets-backup.yaml
Store this backup securely! Anyone with access to this key can decrypt all your sealed secrets.

​
Restore Private Keys

Restore keys in a new cluster: 
kubectl apply -f sealed-secrets-backup.yaml
kubectl rollout restart deployment -n kube-system sealed-secrets-controller

​
Troubleshooting

​
Secret Not Decrypting

  1. Check controller logs: 
    kubectl logs -n kube-system deployment/sealed-secrets-controller
  2. Verify SealedSecret exists: 
    kubectl get sealedsecrets
  3. Check for decryption errors: 
    kubectl describe sealedsecret exchange-router-secret

​
Wrong Namespace or Name

If using strict scope, ensure the SealedSecret name and namespace match: 
metadata:
  name: exchange-router-secret  # Must match
  namespace: default             # Must match
template:
  metadata:
    name: exchange-router-secret  # Must match
    namespace: default             # Must match

​
Certificate Fetch Fails

If kubeseal --fetch-cert fails:
  1. Verify controller is running: 
    kubectl get pods -n kube-system | grep sealed-secrets
  2. Check service exists: 
    kubectl get svc -n kube-system sealed-secrets-controller

​
Production Best Practices

​
1. Version Control Strategy

  • Commit SealedSecret YAML files to Git
  • Never commit plain Secret files
  • Use separate sealed secrets for different environments

​
2. Secret Rotation

  • Rotate secrets regularly (e.g., every 90 days)
  • Re-encrypt with new certificates after key rotation
  • Update applications to use new secrets

​
3. Access Control

  • Restrict access to the sealed-secrets-controller namespace
  • Use RBAC to limit who can view Secrets
  • Audit secret access using admission controllers

​
4. Monitoring

  • Monitor controller logs for decryption failures
  • Alert on SealedSecret resources that fail to create Secrets
  • Track certificate expiration

​
5. Multi-Cluster Strategy

  • Use separate keys per cluster
  • Avoid sharing private keys between environments
  • Use offline encryption in CI/CD pipelines

​
Resources

​
Next Steps

TLS Certificates

Automated certificate management

Best Practices

Security hardening guidelines

Build docs developers (and LLMs) love

Get started for freeTalk to us