Advanced Customization

Overview

Beyond basic configuration, charts support advanced customization for production workloads including resource management, storage, security, and scheduling.

Custom Volumes and Mounts

Basic Volume Configuration

Charts support arbitrary volumes and volumeMounts through values:

volumes:
  - name: config
    configMap:
      name: app-config
  - name: secrets
    secret:
      secretName: app-secrets

volumeMounts:
  - name: config
    mountPath: /etc/config
    readOnly: true
  - name: secrets
    mountPath: /etc/secrets
    readOnly: true

Common Volume Types

ConfigMap
Secret
EmptyDir
PersistentVolumeClaim
HostPath

volumes:
  - name: app-config
    configMap:
      name: my-config
      items:
        - key: config.json
          path: app-config.json

volumeMounts:
  - name: app-config
    mountPath: /etc/app
    readOnly: true

Use for: Configuration files, non-sensitive data

volumes:
  - name: tls-certs
    secret:
      secretName: tls-secret
      defaultMode: 0400

volumeMounts:
  - name: tls-certs
    mountPath: /etc/tls
    readOnly: true

Use for: Sensitive data, credentials, certificates

volumes:
  - name: cache
    emptyDir:
      sizeLimit: 1Gi
  - name: memory-cache
    emptyDir:
      medium: Memory
      sizeLimit: 512Mi

volumeMounts:
  - name: cache
    mountPath: /var/cache
  - name: memory-cache
    mountPath: /tmp/memory

Use for: Temporary storage, cache, shared memory

volumes:
  - name: data
    persistentVolumeClaim:
      claimName: app-data-pvc

volumeMounts:
  - name: data
    mountPath: /var/lib/app

Use for: Persistent data that survives pod restarts

volumes:
  - name: docker-sock
    hostPath:
      path: /var/run/docker.sock
      type: Socket
  - name: logs
    hostPath:
      path: /var/log/app
      type: DirectoryOrCreate

volumeMounts:
  - name: docker-sock
    mountPath: /var/run/docker.sock
  - name: logs
    mountPath: /logs

Use for: Node-specific access (use sparingly)

Real-World Examples

# Custom nginx.conf via ConfigMap
volumes:
  - name: nginx-config
    configMap:
      name: nginx-custom-config
  - name: cache
    emptyDir:
      sizeLimit: 5Gi

volumeMounts:
  - name: nginx-config
    mountPath: /etc/nginx/nginx.conf
    subPath: nginx.conf
    readOnly: true
  - name: cache
    mountPath: /var/cache/nginx

Resource Management

Resource Requests and Limits

Development
Production
Memory-Intensive
CPU-Intensive

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 50m
    memory: 64Mi

Characteristics:

Minimal resource usage
Lower requests for bin packing
Suitable for dev/test environments

resources:
  limits:
    cpu: 1000m
    memory: 2Gi
  requests:
    cpu: 500m
    memory: 1Gi

Characteristics:

Higher requests for guaranteed resources
Limits prevent resource exhaustion
Room for burst capacity

resources:
  limits:
    cpu: 2000m
    memory: 8Gi
  requests:
    cpu: 1000m
    memory: 4Gi

Use for: Databases, caches, data processing

resources:
  limits:
    cpu: 4000m
    memory: 2Gi
  requests:
    cpu: 2000m
    memory: 1Gi

Use for: API servers, computational workloads

Best Practices

Always Set Requests

Why: Kubernetes needs requests for scheduling decisions.

# Bad - no requests
resources:
  limits:
    memory: 1Gi

# Good - requests guide scheduling
resources:
  limits:
    memory: 1Gi
  requests:
    memory: 512Mi

Requests ≤ Limits

Why: Requests are guaranteed, limits are maximum.

# Bad - requests > limits
resources:
  limits:
    cpu: 500m
  requests:
    cpu: 1000m  # Invalid!

# Good
resources:
  limits:
    cpu: 1000m
  requests:
    cpu: 500m

Monitor and Adjust

Start conservative and adjust based on metrics:

# Check actual usage
kubectl top pod -l app=my-app

# View resource metrics over time (with Prometheus)
# Then adjust values.yaml

Different Resources per Component

For multi-component charts like Dify:

frontend:
  resources:
    requests:
      cpu: 100m
      memory: 128Mi

api:
  resources:
    requests:
      cpu: 500m
      memory: 1Gi

worker:
  resources:
    requests:
      cpu: 1000m
      memory: 2Gi

Pod Security Context

Pod-Level Security

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  runAsGroup: 3000
  fsGroup: 2000
  fsGroupChangePolicy: "OnRootMismatch"
  seccompProfile:
    type: RuntimeDefault

Key Settings:

runAsNonRoot: Prevents running as root
runAsUser: Specific UID for the process
fsGroup: GID for volume ownership
seccompProfile: Security compute mode profile

Container-Level Security

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000
  capabilities:
    drop:
      - ALL
    add:
      - NET_BIND_SERVICE

Key Settings:

allowPrivilegeEscalation: Prevents gaining additional privileges
readOnlyRootFilesystem: Immutable root filesystem
capabilities: Fine-grained permissions

Security Profiles

Baseline
Restricted
Privileged

podSecurityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

Use for: Standard applications

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 2000
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000
  capabilities:
    drop:
      - ALL

Use for: Highly secure environments

podSecurityContext: {}

securityContext:
  privileged: true

Use for: System-level tools (avoid if possible)

Node Scheduling

Node Selector

Schedule pods on specific nodes by label:

nodeSelector:
  kubernetes.io/hostname: node-1
  node-type: compute-optimized
  environment: production

Common Labels:

kubernetes.io/hostname: Specific node
kubernetes.io/os: Operating system
node.kubernetes.io/instance-type: Cloud instance type
Custom labels you’ve added to nodes

Tolerations

Allow pods to schedule on tainted nodes:

tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "database"
    effect: "NoSchedule"

Common Taints:

dedicated=<value>:NoSchedule: Dedicated nodes
gpu=true:NoSchedule: GPU nodes
node.kubernetes.io/not-ready: Unready nodes

Affinity Rules

Node Affinity
Pod Affinity
Pod Anti-Affinity
Combined

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: node-type
              operator: In
              values:
                - compute-optimized
                - memory-optimized
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        preference:
          matchExpressions:
            - key: zone
              operator: In
              values:
                - us-east-1a

Use for: Placing pods on specific node types

affinity:
  podAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app
              operator: In
              values:
                - cache
        topologyKey: kubernetes.io/hostname

Use for: Co-locating related pods (app + cache)

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app
              operator: In
              values:
                - my-app
        topologyKey: kubernetes.io/hostname

Use for: Spreading replicas across nodes for HA

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: node-type
              operator: In
              values:
                - compute
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                  - my-app
          topologyKey: kubernetes.io/hostname

Use for: Complex scheduling requirements

High Availability Pattern

# Ensure replicas spread across nodes and zones
replicaCount: 3

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app.kubernetes.io/name
              operator: In
              values:
                - my-app
        topologyKey: kubernetes.io/hostname
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/name
                operator: In
                values:
                  - my-app
          topologyKey: topology.kubernetes.io/zone

Pod Annotations and Labels

Pod Annotations

podAnnotations:
  prometheus.io/scrape: "true"
  prometheus.io/port: "8080"
  prometheus.io/path: "/metrics"
  linkerd.io/inject: "enabled"
  vault.hashicorp.com/agent-inject: "true"
  vault.hashicorp.com/role: "myapp"

Common Uses:

Service mesh configuration (Linkerd, Istio)
Monitoring (Prometheus)
Secret injection (Vault)
Custom automation

Pod Labels

Some charts support additional pod labels:

podLabels:
  environment: production
  team: platform
  cost-center: engineering
  app.kubernetes.io/component: backend

Complete Production Example

production-values.yaml

# Production-ready configuration with all customizations
replicaCount: 3

image:
  repository: myapp/api
  tag: "v1.2.3"
  pullPolicy: IfNotPresent

imagePullSecrets:
  - name: registry-credentials

podAnnotations:
  prometheus.io/scrape: "true"
  prometheus.io/port: "8080"

podLabels:
  environment: production
  team: platform

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 2000
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  capabilities:
    drop:
      - ALL

resources:
  limits:
    cpu: 1000m
    memory: 2Gi
  requests:
    cpu: 500m
    memory: 1Gi

volumes:
  - name: tmp
    emptyDir: {}
  - name: cache
    emptyDir:
      sizeLimit: 5Gi
  - name: config
    configMap:
      name: app-config
  - name: tls
    secret:
      secretName: app-tls

volumeMounts:
  - name: tmp
    mountPath: /tmp
  - name: cache
    mountPath: /var/cache
  - name: config
    mountPath: /etc/config
    readOnly: true
  - name: tls
    mountPath: /etc/tls
    readOnly: true

env:
  - name: LOG_LEVEL
    value: "info"
  - name: DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: db-credentials
        key: url

nodeSelector:
  node-type: compute-optimized

tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "app"
    effect: "NoSchedule"

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app.kubernetes.io/name
              operator: In
              values:
                - my-app
        topologyKey: kubernetes.io/hostname
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/name
                operator: In
                values:
                  - my-app
          topologyKey: topology.kubernetes.io/zone

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

Next Steps

Values Configuration

Learn about values.yaml basics

Ingress Setup

Configure external access

Monitoring

Set up Prometheus monitoring

Chart Reference

Production deployment patterns

Get Started

Web Services

Monitoring & Exporters

Infrastructure

Applications

Configuration

Contributing

Overview

Custom Volumes and Mounts

Basic Volume Configuration

Common Volume Types

Real-World Examples

Resource Management

Resource Requests and Limits

Best Practices

Pod Security Context

Pod-Level Security

Container-Level Security

Security Profiles

Node Scheduling

Node Selector

Tolerations

Affinity Rules

High Availability Pattern

Pod Annotations and Labels

Pod Annotations

Pod Labels

Complete Production Example

Next Steps

Values Configuration

Ingress Setup

Monitoring

Chart Reference

Build docs developers (and LLMs) love

Get Started

Web Services

Monitoring & Exporters

Infrastructure

Applications

Configuration

Contributing

​Overview

​Custom Volumes and Mounts

​Basic Volume Configuration

​Common Volume Types

​Real-World Examples

​Resource Management

​Resource Requests and Limits

​Best Practices

​Pod Security Context

​Pod-Level Security

​Container-Level Security

​Security Profiles

​Node Scheduling

​Node Selector

​Tolerations

​Affinity Rules

​High Availability Pattern

​Pod Annotations and Labels

​Pod Annotations

​Pod Labels

​Complete Production Example

​Next Steps

Values Configuration

Ingress Setup

Monitoring

Chart Reference

Build docs developers (and LLMs) love

Overview

Custom Volumes and Mounts

Basic Volume Configuration

Common Volume Types

Real-World Examples

Resource Management

Resource Requests and Limits

Best Practices

Pod Security Context

Pod-Level Security

Container-Level Security

Security Profiles

Node Scheduling

Node Selector

Tolerations

Affinity Rules

High Availability Pattern

Pod Annotations and Labels

Pod Annotations

Pod Labels

Complete Production Example

Next Steps