Kubernetes Deployment

Overview

Kubernetes deployment provides enterprise-grade orchestration, auto-scaling, and high availability for Kuest Prediction Market. This guide covers deployment using Kustomize and includes complete manifests for production use.

Kubernetes deployment is recommended for organizations requiring:

High availability and auto-scaling
Multiple environments (staging, production)
Advanced deployment strategies (blue/green, canary)
Enterprise compliance and security

Prerequisites

Working Kubernetes cluster (v1.24+)
kubectl configured with cluster access
Ingress controller installed (nginx, traefik, etc.)
Container registry access (Docker Hub, GHCR, etc.)
Environment variables configured (see Storage Options)

Quick Start with Kustomize

Clone Repository

git clone https://github.com/kuestcom/prediction-market.git
cd prediction-market

Create Secret Configuration

cp infra/kubernetes/secret.example.yaml infra/kubernetes/secret.yaml

Edit infra/kubernetes/secret.yaml with your credentials:

infra/kubernetes/secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: kuest-app-secrets
  namespace: kuest
type: Opaque
stringData:
  BETTER_AUTH_SECRET: replace-me
  CRON_SECRET: replace-me
  POSTGRES_URL: replace-me
  KUEST_ADDRESS: replace-me
  KUEST_API_KEY: replace-me
  KUEST_API_SECRET: replace-me
  KUEST_PASSPHRASE: replace-me
  ADMIN_WALLETS: replace-me
  REOWN_APPKIT_PROJECT_ID: replace-me

  # Choose one storage profile:
  # Supabase mode
  SUPABASE_URL: 'https://xxx.supabase.co'
  SUPABASE_SERVICE_ROLE_KEY: 'eyJhbGc...'

  # S3 mode (use this trio when SUPABASE_* is empty)
  # S3_BUCKET: ''
  # S3_ACCESS_KEY_ID: ''
  # S3_SECRET_ACCESS_KEY: ''

  # Optional S3 settings
  # S3_ENDPOINT: ''
  # S3_REGION: ''
  # S3_PUBLIC_URL: ''
  # S3_FORCE_PATH_STYLE: ''

Apply Manifests

kubectl apply -k infra/kubernetes

This creates:

Namespace: kuest

Deployment: kuest-web (2 replicas)

Service: kuest-web (ClusterIP)

Ingress: kuest-web (nginx)

ConfigMap: kuest-app-config

Secret: kuest-app-secrets

Verify Deployment

# Check all resources
kubectl get all -n kuest

# Check pod status
kubectl get pods -n kuest

# View logs
kubectl logs -n kuest -l app=kuest-web -f

Kubernetes Manifests

Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: kuest

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kuest-web
  namespace: kuest
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: kuest-web
  template:
    metadata:
      labels:
        app: kuest-web
    spec:
      containers:
        - name: web
          image: ghcr.io/kuestcom/prediction-market:v1.0.0
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 3000
              name: http
          envFrom:
            - configMapRef:
                name: kuest-app-config
            - secretRef:
                name: kuest-app-secrets
          readinessProbe:
            httpGet:
              path: /
              port: http
            initialDelaySeconds: 10
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /
              port: http
            initialDelaySeconds: 30
            periodSeconds: 20
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              cpu: '1'
              memory: 1024Mi

Update the image field to use your specific version tag or digest (e.g., @sha256:...) for production deployments.

Service

apiVersion: v1
kind: Service
metadata:
  name: kuest-web
  namespace: kuest
spec:
  selector:
    app: kuest-web
  ports:
    - name: http
      port: 80
      targetPort: 3000

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kuest-web
  namespace: kuest
spec:
  ingressClassName: nginx
  rules:
    - host: markets.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kuest-web
                port:
                  number: 80
  tls:
    - hosts:
        - markets.example.com
      secretName: kuest-web-tls

Update host to your actual domain. Ensure TLS certificate is available as kuest-web-tls secret or configure cert-manager for automatic SSL.

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: kuest-app-config
  namespace: kuest
data:
  NODE_ENV: production
  SITE_URL: https://markets.example.com

CronJobs (Scheduler)

Only apply CronJobs if using Postgres + S3 mode. If using Supabase mode, skip this section as Supabase handles scheduling automatically.

Edit infra/kubernetes/cronjobs.yaml to match your environment:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: kuest-sync-events
  namespace: kuest
spec:
  schedule: '1-59/5 * * * *'
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: trigger
              image: curlimages/curl:8.12.1@sha256:94e9e444bcba979c2ea12e27ae39bee4cd10bc7041a472c4727a558e213744e6
              command:
                - sh
                - -c
                - >
                  curl -fsS
                  -H "Authorization: Bearer ${CRON_SECRET}"
                  "${SITE_URL}/api/sync/events"
              env:
                - name: SITE_URL
                  value: 'https://markets.example.com'
                - name: CRON_SECRET
                  valueFrom:
                    secretKeyRef:
                      name: kuest-app-secrets
                      key: CRON_SECRET
---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: kuest-sync-resolution
  namespace: kuest
spec:
  schedule: '3-59/5 * * * *'
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: trigger
              image: curlimages/curl:8.12.1@sha256:94e9e444bcba979c2ea12e27ae39bee4cd10bc7041a472c4727a558e213744e6
              command:
                - sh
                - -c
                - >
                  curl -fsS
                  -H "Authorization: Bearer ${CRON_SECRET}"
                  "${SITE_URL}/api/sync/resolution"
              env:
                - name: SITE_URL
                  value: 'https://markets.example.com'
                - name: CRON_SECRET
                  valueFrom:
                    secretKeyRef:
                      name: kuest-app-secrets
                      key: CRON_SECRET
---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: kuest-sync-translations
  namespace: kuest
spec:
  schedule: '*/10 * * * *'
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: trigger
              image: curlimages/curl:8.12.1@sha256:94e9e444bcba979c2ea12e27ae39bee4cd10bc7041a472c4727a558e213744e6
              command:
                - sh
                - -c
                - >
                  curl -fsS
                  -H "Authorization: Bearer ${CRON_SECRET}"
                  "${SITE_URL}/api/sync/translations"
              env:
                - name: SITE_URL
                  value: 'https://markets.example.com'
                - name: CRON_SECRET
                  valueFrom:
                    secretKeyRef:
                      name: kuest-app-secrets
                      key: CRON_SECRET
---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: kuest-sync-volume
  namespace: kuest
spec:
  schedule: '14,44 * * * *'
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: trigger
              image: curlimages/curl:8.12.1@sha256:94e9e444bcba979c2ea12e27ae39bee4cd10bc7041a472c4727a558e213744e6
              command:
                - sh
                - -c
                - >
                  curl -fsS
                  -H "Authorization: Bearer ${CRON_SECRET}"
                  "${SITE_URL}/api/sync/volume"
              env:
                - name: SITE_URL
                  value: 'https://markets.example.com'
                - name: CRON_SECRET
                  valueFrom:
                    secretKeyRef:
                      name: kuest-app-secrets
                      key: CRON_SECRET

Apply CronJobs:

kubectl apply -f infra/kubernetes/cronjobs.yaml

Verify CronJobs:

# List CronJobs
kubectl get cronjobs -n kuest

# Check recent jobs
kubectl get jobs -n kuest

# View job logs
kubectl logs -n kuest job/kuest-sync-events-<timestamp>

Operations

Scaling

# Manual scaling
kubectl scale deployment/kuest-web -n kuest --replicas=5

# Auto-scaling (HPA)
kubectl autoscale deployment kuest-web -n kuest --cpu-percent=70 --min=2 --max=10

# Check HPA status
kubectl get hpa -n kuest

Updates

# Update image
kubectl set image deployment/kuest-web web=ghcr.io/kuestcom/prediction-market:v1.1.0 -n kuest

# Check rollout status
kubectl rollout status deployment/kuest-web -n kuest

# Rollback
kubectl rollout undo deployment/kuest-web -n kuest

# Rollback to specific revision
kubectl rollout undo deployment/kuest-web -n kuest --to-revision=2

Monitoring

# Watch pods
kubectl get pods -n kuest -w

# Describe pod
kubectl describe pod <pod-name> -n kuest

# View logs
kubectl logs -n kuest -l app=kuest-web --tail=100 -f

# Execute commands in pod
kubectl exec -it <pod-name> -n kuest -- sh

Debugging

# Check events
kubectl get events -n kuest --sort-by='.lastTimestamp'

# Check resource usage
kubectl top pods -n kuest
kubectl top nodes

# Port forward for local testing
kubectl port-forward -n kuest svc/kuest-web 3000:80

Production Best Practices

Resource Management

Set resource requests and limits

resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    cpu: '1'
    memory: 1024Mi

Use PodDisruptionBudgets

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: kuest-web-pdb
  namespace: kuest
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: kuest-web

Configure HorizontalPodAutoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: kuest-web-hpa
  namespace: kuest
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kuest-web
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Security

Use immutable image tags

image: ghcr.io/kuestcom/prediction-market@sha256:abc123...

NetworkPolicies for isolation

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: kuest-web-netpol
  namespace: kuest
spec:
  podSelector:
    matchLabels:
      app: kuest-web
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: nginx-ingress
  egress:
  - to:
    - namespaceSelector: {}

Security contexts

securityContext:
  runAsNonRoot: true
  runAsUser: 1001
  fsGroup: 1001
  seccompProfile:
    type: RuntimeDefault
  capabilities:
    drop:
    - ALL

High Availability

Multiple replicas: Minimum 2 replicas across availability zones

Pod anti-affinity: Distribute pods across nodes

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchLabels:
            app: kuest-web
        topologyKey: kubernetes.io/hostname

Health checks: Properly configured readiness and liveness probes

Storage Modes

Supabase Mode

Set in secret.yaml:

SUPABASE_URL: 'https://xxx.supabase.co'
SUPABASE_SERVICE_ROLE_KEY: 'eyJhbGc...'
POSTGRES_URL: 'postgresql://...'

No CronJobs needed - Supabase handles scheduling.

Postgres + S3 Mode

Set in secret.yaml:

POSTGRES_URL: 'postgresql://...'
S3_BUCKET: 'kuest-assets'
S3_ACCESS_KEY_ID: 'xxx'
S3_SECRET_ACCESS_KEY: 'xxx'

Must apply CronJobs for /api/sync/* endpoints.

Troubleshooting

Pods Not Starting

# Check pod status
kubectl get pods -n kuest

# Describe pod for events
kubectl describe pod <pod-name> -n kuest

# Check logs
kubectl logs <pod-name> -n kuest

Common issues:

Image pull errors: Verify image name and registry access
Secrets not found: Ensure kuest-app-secrets exists
Resource constraints: Check node resources

Ingress Not Working

# Check ingress
kubectl get ingress -n kuest
kubectl describe ingress kuest-web -n kuest

# Check ingress controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

Database Connection Issues

Verify POSTGRES_URL in secret is correct

Test connectivity from pod:

kubectl exec -it <pod-name> -n kuest -- sh
nc -zv <postgres-host> 5432

Next Steps

Deploy to cloud providers for managed Kubernetes
Configure monitoring with Prometheus and Grafana
Set up CI/CD for automated deployments

Get Started

Deployment

Configuration

Features

Customization

Operations

Overview

Prerequisites

Quick Start with Kustomize

Kubernetes Manifests

Namespace

Deployment

Service

Ingress

ConfigMap

CronJobs (Scheduler)

Operations

Scaling

Updates

Monitoring

Debugging

Production Best Practices

Resource Management

Security

High Availability

Storage Modes

Supabase Mode

Postgres + S3 Mode

Troubleshooting

Pods Not Starting

Ingress Not Working

Database Connection Issues

Next Steps

Build docs developers (and LLMs) love

Get Started

Deployment

Configuration

Features

Customization

Operations

​Overview

​Prerequisites

​Quick Start with Kustomize

​Kubernetes Manifests

​Namespace

​Deployment

​Service

​Ingress

​ConfigMap

​CronJobs (Scheduler)

​Operations

​Scaling

​Updates

​Monitoring

​Debugging

​Production Best Practices

​Resource Management

​Security

​High Availability

​Storage Modes

​Supabase Mode

​Postgres + S3 Mode

​Troubleshooting

​Pods Not Starting

​Ingress Not Working

​Database Connection Issues

​Next Steps

Build docs developers (and LLMs) love

Overview

Prerequisites

Quick Start with Kustomize

Kubernetes Manifests

Namespace

Deployment

Service

Ingress

ConfigMap

CronJobs (Scheduler)

Operations

Scaling

Updates

Monitoring

Debugging

Production Best Practices

Resource Management

Security

High Availability

Storage Modes

Supabase Mode

Postgres + S3 Mode

Troubleshooting

Pods Not Starting

Ingress Not Working

Database Connection Issues

Next Steps