Production Deployment

Overview

QIMEM is designed for production use with defense-in-depth security, deterministic encryption, and safe key handling. This guide covers deployment best practices, security hardening, and operational considerations.

Security Guarantees

Key Material Protection

QIMEM protects encryption keys with multiple layers of defense:

Zeroization

All key material is wrapped with zeroize::Zeroizing to ensure memory is cleared on drop:

use zeroize::Zeroizing;

pub struct Key {
    pub key_id: Uuid,
    pub material: Zeroizing<Vec<u8>>, // Zeroed on drop
    pub algorithm: Algorithm,
    // ...
}

No Logging

Key bytes are never logged. Only key IDs and metadata appear in logs:

tracing::info!("created key_id={}", key.key_id); // Safe
// NEVER: tracing::debug!("key_material={:?}", key.material)

Memory-Safe Rust

QIMEM enforces #![deny(unsafe_code)] to prevent memory vulnerabilities:

#![deny(unsafe_code)]
#![deny(missing_docs)]

Constant-Time Operations

Uses subtle::ConstantTimeEq for tag verification to prevent timing attacks:

use subtle::ConstantTimeEq;

if expected_tag.ct_eq(&computed_tag).into() {
    // Tags match (constant-time comparison)
}

Critical Security Notes:

Key material is never written to disk unencrypted (even in stateful mode)
Keys are zeroized before memory is freed
No key bytes appear in logs, metrics, or error messages
Database connections must use TLS in production

Stateful Mode with Managed PostgreSQL

Why Stateful Mode?

Stateless mode stores keys in memory:

✅ Fast and simple
❌ Keys lost on restart
❌ No key lineage across instances
❌ Not suitable for production

Stateful mode stores keys in PostgreSQL:

✅ Keys persist across restarts
✅ Key rotation with lineage tracking
✅ Multi-instance support
✅ Audit trail for key lifecycle

Managed PostgreSQL Recommendations

DATABASE_URL=postgres://qimem:${DB_PASSWORD}@qimem-db.abc123.us-east-1.rds.amazonaws.com:5432/qimem?sslmode=require

# Enable:
# - Encryption at rest (KMS)
# - Automated backups (30 days)
# - Multi-AZ deployment
# - TLS enforcement

Database Security Checklist

Enable TLS

DATABASE_URL=postgres://user:pass@host:5432/qimem?sslmode=require

Verify TLS:

SELECT ssl_is_used() FROM pg_stat_ssl WHERE pid = pg_backend_pid();

Use encrypted storage

Enable encryption at rest:

AWS RDS: KMS encryption
GCP Cloud SQL: Automatic encryption
Azure: Transparent Data Encryption (TDE)

Restrict network access

Use private subnets or VPC peering
Whitelist only QIMEM service IPs
Disable public access

Enable audit logging

ALTER SYSTEM SET log_connections = 'on';
ALTER SYSTEM SET log_disconnections = 'on';
ALTER SYSTEM SET log_statement = 'all';
SELECT pg_reload_conf();

Automate backups

Daily automated backups
30-day retention minimum
Test restore procedures quarterly

TLS/HTTPS Setup

Reverse Proxy with TLS Termination

QIMEM does not implement TLS natively. Use a reverse proxy:

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate /etc/ssl/certs/api.example.com.crt;
    ssl_certificate_key /etc/ssl/private/api.example.com.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Monitoring and Health Endpoints

Health Check Endpoint

QIMEM exposes a health endpoint at GET /health:

curl -fsS http://localhost:8080/health

Response:

{
  "status": "ok"
}

HTTP Status:

200 OK - Service is healthy
5xx - Service is unhealthy (connection refused, etc.)

Unified Platform Health

The qauth-api binary provides additional health endpoints:

# Main health check
curl http://localhost:8080/health

# Security service health
curl http://localhost:8080/v1/security/health

Health Check Configuration

apiVersion: v1
kind: Pod
metadata:
  name: qimem-api
spec:
  containers:
    - name: qimem-api
      image: qimem-api:latest
      ports:
        - containerPort: 8080
      livenessProbe:
        httpGet:
          path: /health
          port: 8080
        initialDelaySeconds: 10
        periodSeconds: 10
        timeoutSeconds: 3
        failureThreshold: 3
      readinessProbe:
        httpGet:
          path: /health
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 5
        timeoutSeconds: 2
        failureThreshold: 2

Metrics and Observability

QIMEM does not currently expose Prometheus metrics. Health checks and structured logs are the primary observability tools.

Structured Logging:

RUST_LOG=qimem=info,sqlx=warn

Log Output:

{"timestamp":"2026-03-06T12:00:00Z","level":"INFO","target":"qimem","fields":{"message":"created key_id=a1b2c3d4-..."}}
{"timestamp":"2026-03-06T12:00:05Z","level":"INFO","target":"qimem","fields":{"message":"encrypted with key_id=a1b2c3d4-..."}}
{"timestamp":"2026-03-06T12:00:10Z","level":"INFO","target":"qimem","fields":{"message":"rotated key_id=a1b2c3d4-... to new_key_id=e5f6g7h8-..."}}

Production Checklist

Enable stateful mode

QIMEM_MODE=stateful
DATABASE_URL=postgres://user:pass@host:5432/qimem?sslmode=require

Use managed PostgreSQL

Enable encryption at rest
Configure automated backups
Set up high availability
Restrict network access

Configure TLS termination

Use Nginx, ALB, or Kubernetes Ingress for HTTPS

Set production logging

RUST_LOG=info  # or warn for less verbosity

Configure health checks

Set up liveness and readiness probes for orchestrators

Implement secret management

Use Vault, AWS Secrets Manager, or Kubernetes Secrets for DATABASE_URL

Enable monitoring

Centralize logs (CloudWatch, Stackdriver, ELK)
Monitor database connections
Alert on health check failures

Test disaster recovery

Validate backup restoration
Test failover procedures
Document runbooks

Scaling Considerations

Horizontal Scaling

QIMEM is stateless at the application layer when using PostgreSQL:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: qimem-api
spec:
  replicas: 3  # Scale horizontally
  selector:
    matchLabels:
      app: qimem-api
  template:
    metadata:
      labels:
        app: qimem-api
    spec:
      containers:
        - name: qimem-api
          image: qimem-api:latest
          env:
            - name: QIMEM_MODE
              value: "stateful"
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: qimem-db-secret
                  key: url

Database Connection Pooling

sqlx automatically pools connections. Tune pool size based on workload:

let pool = PgPoolOptions::new()
    .max_connections(10)  // Adjust per replica
    .connect(&database_url)
    .await?;

Rule of thumb:

5-10 connections per replica
Monitor pg_stat_activity for connection count
Scale PostgreSQL if connection limit is reached

Rate Limiting

Implement rate limiting at the reverse proxy layer:

http {
    limit_req_zone $binary_remote_addr zone=qimem:10m rate=100r/s;

    server {
        location / {
            limit_req zone=qimem burst=20 nodelay;
            proxy_pass http://127.0.0.1:8080;
        }
    }
}

Disaster Recovery

Backup Strategy

Automated PostgreSQL backups

Daily full backups
30-day retention
Cross-region replication

Point-in-time recovery

Enable WAL archiving for sub-hour recovery:

ALTER SYSTEM SET wal_level = 'replica';
ALTER SYSTEM SET archive_mode = 'on';
ALTER SYSTEM SET archive_command = 'cp %p /backup/wal/%f';

Test restoration quarterly

# Restore to test environment
pg_restore -h test-db.example.com -U qimem -d qimem_test backup.dump

# Verify key count
psql -h test-db.example.com -U qimem -d qimem_test -c "SELECT COUNT(*) FROM keys;"

Incident Response

Key compromise:

If a key is compromised:

Rotate the affected key immediately: POST /rotate
Re-encrypt data with the new key
Audit access logs for unauthorized decrypt operations
Review database access logs

Database failure:

Check managed database status
Trigger failover to standby (if multi-AZ)
Update DATABASE_URL if endpoint changed
Restart QIMEM replicas

Service outage:

Check health endpoint: curl http://localhost:8080/health
Review logs: kubectl logs -l app=qimem-api
Check database connectivity: psql $DATABASE_URL -c 'SELECT 1'
Verify TLS certificates are valid
Scale replicas if overloaded

Security Best Practices

Critical Production Security:

Never use sslmode=disable in production
Rotate database passwords every 90 days
Use secret management for credentials (not environment variables)
Enable database audit logging
Restrict network access to database and API
Monitor for failed authentication attempts
Test disaster recovery procedures quarterly
Keep dependencies updated (Rust toolchain, PostgreSQL)
Use dedicated service accounts (not admin credentials)
Implement rate limiting to prevent abuse

Get Started

Core Concepts

Deployment

Overview

Security Guarantees

Key Material Protection

Stateful Mode with Managed PostgreSQL

Why Stateful Mode?

Managed PostgreSQL Recommendations

Database Security Checklist

TLS/HTTPS Setup

Reverse Proxy with TLS Termination

Monitoring and Health Endpoints

Health Check Endpoint

Unified Platform Health

Health Check Configuration

Metrics and Observability

Production Checklist

Scaling Considerations

Horizontal Scaling

Database Connection Pooling

Rate Limiting

Disaster Recovery

Backup Strategy

Incident Response

Security Best Practices

Build docs developers (and LLMs) love

Get Started

Core Concepts

Deployment

​Overview

​Security Guarantees

​Key Material Protection

​Stateful Mode with Managed PostgreSQL

​Why Stateful Mode?

​Managed PostgreSQL Recommendations

​Database Security Checklist

​TLS/HTTPS Setup

​Reverse Proxy with TLS Termination

​Monitoring and Health Endpoints

​Health Check Endpoint

​Unified Platform Health

​Health Check Configuration

​Metrics and Observability

​Production Checklist

​Scaling Considerations

​Horizontal Scaling

​Database Connection Pooling

​Rate Limiting

​Disaster Recovery

​Backup Strategy

​Incident Response

​Security Best Practices

Build docs developers (and LLMs) love

Overview

Security Guarantees

Key Material Protection

Stateful Mode with Managed PostgreSQL

Why Stateful Mode?

Managed PostgreSQL Recommendations

Database Security Checklist

TLS/HTTPS Setup

Reverse Proxy with TLS Termination

Monitoring and Health Endpoints

Health Check Endpoint

Unified Platform Health

Health Check Configuration

Metrics and Observability

Production Checklist

Scaling Considerations

Horizontal Scaling

Database Connection Pooling

Rate Limiting

Disaster Recovery

Backup Strategy

Incident Response

Security Best Practices