Production Deployment

Overview

This guide covers deploying VoicePact to production with a focus on security, reliability, and scalability. The architecture supports both single-server and distributed deployments.

Production Architecture

Recommended Stack

┌─────────────────────────────────────────────────────────────┐
│                         Load Balancer                        │
│                    (nginx / AWS ALB / Cloudflare)            │
└──────────────────────┬──────────────────────────────────────┘
                       │
         ┌─────────────┴─────────────┐
         │                           │
┌────────▼─────────┐      ┌─────────▼──────────┐
│   Application    │      │   Application      │
│   Server 1       │      │   Server 2         │
│   (FastAPI)      │      │   (FastAPI)        │
└────────┬─────────┘      └──────────┬─────────┘
         │                           │
         └─────────────┬─────────────┘
                       │
         ┌─────────────┴─────────────┐
         │                           │
┌────────▼─────────┐      ┌─────────▼──────────┐
│   PostgreSQL     │      │   Redis Cluster    │
│   (Primary)      │      │   (Cache/Session)  │
└──────────────────┘      └────────────────────┘
         │
┌────────▼─────────┐
│   PostgreSQL     │
│   (Replica)      │
└──────────────────┘

Component Requirements

Application Server: 2+ cores, 4GB+ RAM
PostgreSQL: 2+ cores, 8GB+ RAM, SSD storage
Redis: 1+ cores, 2GB+ RAM
Storage: Object storage for audio files (S3, MinIO, etc.)

Pre-Deployment Checklist

Security Requirements

Infrastructure Requirements

Application Requirements

Production environment variables set
Database migrations tested
Africa’s Talking production credentials
Payment provider configured
Error tracking enabled (Sentry)
Performance monitoring setup

Server Setup

1. Provision Server

Minimum specifications:

CPU: 2 cores (4+ recommended)
RAM: 4GB (8GB+ recommended)
Storage: 50GB SSD
OS: Ubuntu 22.04 LTS or later

2. Initial Server Configuration

# Update system
sudo apt update && sudo apt upgrade -y

# Install dependencies
sudo apt install -y \
    python3.11 \
    python3.11-venv \
    python3-pip \
    nginx \
    postgresql-client \
    redis-tools \
    git \
    supervisor \
    certbot \
    python3-certbot-nginx

# Create application user
sudo useradd -m -s /bin/bash voicepact
sudo usermod -aG sudo voicepact

3. Install PostgreSQL

# Install PostgreSQL 15
sudo apt install -y postgresql-15 postgresql-contrib-15

# Secure installation
sudo -u postgres psql -c "ALTER USER postgres WITH PASSWORD 'secure_password';"

# Configure PostgreSQL
sudo nano /etc/postgresql/15/main/postgresql.conf

Update postgresql.conf:

listen_addresses = 'localhost'  # Or specific IP for remote access
max_connections = 100
shared_buffers = 256MB
effective_cache_size = 1GB
maintenance_work_mem = 64MB
wal_level = replica
max_wal_size = 2GB
min_wal_size = 1GB

Configure authentication in /etc/postgresql/15/main/pg_hba.conf:

# Local connections
local   all             all                                     peer
host    all             all             127.0.0.1/32            scram-sha-256
host    all             all             ::1/128                 scram-sha-256

Restart PostgreSQL:

sudo systemctl restart postgresql
sudo systemctl enable postgresql

4. Install Redis

# Install Redis
sudo apt install -y redis-server

# Configure Redis
sudo nano /etc/redis/redis.conf

Update redis.conf:

# Security
requirepass YOUR_SECURE_REDIS_PASSWORD
bind 127.0.0.1
protected-mode yes

# Persistence
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

# Memory
maxmemory 512mb
maxmemory-policy allkeys-lru

Restart Redis:

sudo systemctl restart redis-server
sudo systemctl enable redis-server

Application Deployment

1. Clone Repository

sudo su - voicepact
cd ~
git clone https://github.com/mutuiris/voicepact.git
cd voicepact/server

2. Create Virtual Environment

python3.11 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

# Install production dependencies
pip install gunicorn uvicorn[standard]

3. Configure Environment

Create production environment file:

cd ~/voicepact/server
nano .env.production

Use strong, unique values for all secrets. Never use default or example values in production.

# Application
APP_NAME=VoicePact
APP_VERSION=1.0.0
ENVIRONMENT=production
DEBUG=false
API_V1_PREFIX=/api/v1

# Security (GENERATE UNIQUE VALUES)
SECRET_KEY=<generate-with-python-secrets>
SIGNATURE_PRIVATE_KEY=<generate-with-python-secrets>
PASSWORD_SALT=<generate-with-python-secrets>
WEBHOOK_SECRET=<generate-with-python-secrets>

# Africa's Talking (PRODUCTION CREDENTIALS)
AT_USERNAME=your_production_username
AT_API_KEY=your_production_api_key
AT_VOICE_NUMBER=+254XXXXXXXXX
AT_PAYMENT_PRODUCT_NAME=VoicePact
AT_USSD_SERVICE_CODE=*483#

# Database
DATABASE_URL=postgresql+asyncpg://voicepact:secure_db_password@localhost:5432/voicepact
DATABASE_ECHO=false

# Redis
REDIS_URL=redis://:YOUR_SECURE_REDIS_PASSWORD@localhost:6379/0
REDIS_MAX_CONNECTIONS=50
REDIS_SOCKET_TIMEOUT=30

# Webhooks
WEBHOOK_BASE_URL=https://api.yourdomain.com

# CORS (RESTRICT TO YOUR DOMAINS)
CORS_ORIGINS=https://yourdomain.com,https://app.yourdomain.com
CORS_CREDENTIALS=true

# Logging
LOG_LEVEL=WARNING
LOG_FILE=/var/log/voicepact/app.log

# Performance
CACHE_TTL=7200
MAX_WORKERS=8

Generate secure keys:

import secrets
print(f"SECRET_KEY={secrets.token_urlsafe(32)}")
print(f"SIGNATURE_PRIVATE_KEY={secrets.token_urlsafe(64)}")
print(f"PASSWORD_SALT={secrets.token_urlsafe(32)}")
print(f"WEBHOOK_SECRET={secrets.token_urlsafe(32)}")

4. Initialize Database

# Create database
sudo -u postgres psql

CREATE USER voicepact WITH PASSWORD 'secure_db_password';
CREATE DATABASE voicepact OWNER voicepact;
GRANT ALL PRIVILEGES ON DATABASE voicepact TO voicepact;
\c voicepact
GRANT ALL ON SCHEMA public TO voicepact;
\q

# Load environment and initialize
source venv/bin/activate
export $(cat .env.production | xargs)
python -c "from app.core.database import init_database; import asyncio; asyncio.run(init_database())"

5. Configure Supervisor

Create supervisor config:

sudo nano /etc/supervisor/conf.d/voicepact.conf

[program:voicepact]
command=/home/voicepact/voicepact/server/venv/bin/gunicorn main:app \
    --workers 4 \
    --worker-class uvicorn.workers.UvicornWorker \
    --bind 127.0.0.1:8000 \
    --access-logfile /var/log/voicepact/access.log \
    --error-logfile /var/log/voicepact/error.log \
    --log-level warning \
    --timeout 120 \
    --graceful-timeout 30 \
    --keep-alive 5

directory=/home/voicepact/voicepact/server
user=voicepact
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
stdout_logfile=/var/log/voicepact/supervisor.log
stderr_logfile=/var/log/voicepact/supervisor_error.log
environment=PATH="/home/voicepact/voicepact/server/venv/bin"

[group:voicepact]
programs=voicepact

Create log directory:

sudo mkdir -p /var/log/voicepact
sudo chown voicepact:voicepact /var/log/voicepact

Start application:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start voicepact
sudo supervisorctl status

Nginx Configuration

1. Configure Nginx

sudo nano /etc/nginx/sites-available/voicepact

# Rate limiting
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=100r/s;

upstream voicepact_backend {
    server 127.0.0.1:8000 fail_timeout=0;
}

server {
    listen 80;
    server_name api.yourdomain.com;
    
    # Redirect to HTTPS
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;
    
    # SSL Configuration
    ssl_certificate /etc/letsencrypt/live/api.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    
    # Security Headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    
    # Request size limits
    client_max_body_size 50M;
    client_body_timeout 120s;
    
    # Logging
    access_log /var/log/nginx/voicepact_access.log;
    error_log /var/log/nginx/voicepact_error.log warn;
    
    # API endpoints with rate limiting
    location /api/ {
        limit_req zone=api_limit burst=20 nodelay;
        
        proxy_pass http://voicepact_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        
        # Disable buffering for streaming responses
        proxy_buffering off;
    }
    
    # Webhooks with higher rate limit
    location /api/v1/webhooks/ {
        limit_req zone=webhook_limit burst=50 nodelay;
        
        proxy_pass http://voicepact_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    
    # WebSocket support
    location /ws/ {
        proxy_pass http://voicepact_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket timeout
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
    }
    
    # Health check endpoint (no rate limit)
    location /health {
        proxy_pass http://voicepact_backend;
        access_log off;
    }
}

Enable site:

sudo ln -s /etc/nginx/sites-available/voicepact /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

2. Obtain SSL Certificate

# Install Certbot
sudo apt install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d api.yourdomain.com

# Set up auto-renewal
sudo systemctl enable certbot.timer
sudo systemctl start certbot.timer

Client Deployment

The Next.js client can be deployed to Vercel, Netlify, or self-hosted.

Deploy to Vercel

Install Vercel CLI:

npm install -g vercel

Configure environment variables:

In Vercel dashboard, add:

NEXT_PUBLIC_API_BASE_URL=https://api.yourdomain.com
NEXT_PUBLIC_WEBSOCKET_BASE=wss://api.yourdomain.com
NEXT_PUBLIC_FEATURE_LOW_DATA_MODE=true
NEXT_PUBLIC_FEATURE_ESCROW_TIMELINE=true
NEXT_PUBLIC_FEATURE_ROLE_AWARE_DASHBOARD=true

Deploy:

cd client
vercel --prod

Self-Hosted Client

cd client
npm install
npm run build

# Serve with nginx
sudo nano /etc/nginx/sites-available/voicepact-client

server {
    listen 443 ssl http2;
    server_name app.yourdomain.com;
    
    ssl_certificate /etc/letsencrypt/live/app.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.yourdomain.com/privkey.pem;
    
    root /home/voicepact/voicepact/client/out;
    index index.html;
    
    location / {
        try_files $uri $uri/ /index.html;
    }
}

Security Hardening

1. Firewall Configuration

# Configure UFW
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS
sudo ufw enable
sudo ufw status

2. Fail2ban Setup

# Install fail2ban
sudo apt install fail2ban

# Configure
sudo nano /etc/fail2ban/jail.local

[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5

[nginx-limit-req]
enabled = true
port = http,https
logpath = /var/log/nginx/*error.log

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

3. Secure SSH

Edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 22

sudo systemctl reload sshd

Monitoring and Logging

1. Application Monitoring

Install Sentry:

pip install sentry-sdk[fastapi]

Add to main.py:

import sentry_sdk
from sentry_sdk.integrations.fastapi import FastApiIntegration

if settings.environment == "production":
    sentry_sdk.init(
        dsn="your_sentry_dsn",
        integrations=[FastApiIntegration()],
        traces_sample_rate=0.1,
        environment="production"
    )

2. Log Management

Configure logrotate:

sudo nano /etc/logrotate.d/voicepact

/var/log/voicepact/*.log {
    daily
    rotate 30
    compress
    delaycompress
    notifempty
    create 0640 voicepact voicepact
    sharedscripts
    postrotate
        supervisorctl restart voicepact
    endscript
}

3. Health Monitoring

Set up health check monitoring with a cron job:

crontab -e

*/5 * * * * curl -f https://api.yourdomain.com/health || echo "Health check failed" | mail -s "VoicePact Alert" [email protected]

Backup Strategy

Automated Backup Script

sudo nano /usr/local/bin/backup-voicepact.sh

#!/bin/bash
set -e

BACKUP_DIR="/var/backups/voicepact"
DATE=$(date +%Y%m%d_%H%M%S)
RETENTION_DAYS=30
S3_BUCKET="s3://your-backup-bucket/voicepact"

mkdir -p "$BACKUP_DIR"

# Database backup
pg_dump -U voicepact -d voicepact -F c -f "$BACKUP_DIR/db_$DATE.dump"

# Redis backup
redis-cli --rdb "$BACKUP_DIR/redis_$DATE.rdb"

# Application files
tar -czf "$BACKUP_DIR/app_$DATE.tar.gz" /home/voicepact/voicepact

# Upload to S3 (if using AWS)
aws s3 sync "$BACKUP_DIR" "$S3_BUCKET" --storage-class STANDARD_IA

# Clean old local backups
find "$BACKUP_DIR" -mtime +$RETENTION_DAYS -delete

echo "Backup completed: $DATE"

sudo chmod +x /usr/local/bin/backup-voicepact.sh

Schedule daily backups:

sudo crontab -e

0 3 * * * /usr/local/bin/backup-voicepact.sh >> /var/log/voicepact/backup.log 2>&1

Scaling Strategies

Horizontal Scaling

Load Balancer: Use nginx, HAProxy, or cloud load balancer
Multiple Application Servers: Deploy identical instances
Shared Session Store: Redis cluster for session management
Database Replication: PostgreSQL read replicas

Vertical Scaling

Increase server resources (CPU, RAM)
Optimize database (indexes, query optimization)
Increase worker processes in Gunicorn
Adjust PostgreSQL connection pool size

Database Optimization

-- Create indexes for frequent queries
CREATE INDEX CONCURRENTLY idx_contracts_status_created 
    ON contracts (status, created_at);

CREATE INDEX CONCURRENTLY idx_payments_status_payer 
    ON payments (status, payer_phone);

-- Analyze tables
ANALYZE contracts;
ANALYZE payments;
ANALYZE audit_logs;

Troubleshooting

Application Won’t Start

# Check supervisor logs
sudo tail -f /var/log/voicepact/supervisor_error.log

# Check application logs
sudo tail -f /var/log/voicepact/error.log

# Verify environment
sudo su - voicepact
cd ~/voicepact/server
source venv/bin/activate
export $(cat .env.production | xargs)
python -c "from app.core.config import get_settings; print(get_settings())"

High Memory Usage

# Check process memory
top -p $(pgrep -f gunicorn)

# Adjust worker count in supervisor config
# Reduce from --workers 4 to --workers 2

# Restart application
sudo supervisorctl restart voicepact

Database Performance Issues

-- Find slow queries
SELECT query, mean_exec_time, calls 
FROM pg_stat_statements 
ORDER BY mean_exec_time DESC LIMIT 10;

-- Check index usage
SELECT schemaname, tablename, indexname, idx_scan 
FROM pg_stat_user_indexes 
ORDER BY idx_scan;

-- Vacuum and analyze
VACUUM ANALYZE;

Maintenance

Regular Tasks

Daily:

Monitor application logs
Check disk space
Verify backups completed

Weekly:

Review error rates
Check database performance
Update dependencies (security patches)

Monthly:

Rotate logs
Review security alerts
Test backup restoration
Update system packages

Update Procedure

# 1. Backup current state
sudo /usr/local/bin/backup-voicepact.sh

# 2. Pull latest code
sudo su - voicepact
cd ~/voicepact
git pull origin main

# 3. Update dependencies
cd server
source venv/bin/activate
pip install -r requirements.txt

# 4. Run migrations (if any)
# alembic upgrade head

# 5. Restart application
exit
sudo supervisorctl restart voicepact

# 6. Verify deployment
curl https://api.yourdomain.com/health

Next Steps

Environment Setup - Review environment configuration
Database Configuration - Optimize database performance
API Reference - Integrate with production API

Deployment

Development

​Overview

​Production Architecture

​Recommended Stack

​Component Requirements

​Pre-Deployment Checklist

​Server Setup

​1. Provision Server

​2. Initial Server Configuration

​3. Install PostgreSQL

​4. Install Redis

​Application Deployment

​1. Clone Repository

​2. Create Virtual Environment

​3. Configure Environment

​4. Initialize Database

​5. Configure Supervisor

​Nginx Configuration

​1. Configure Nginx

​2. Obtain SSL Certificate

​Client Deployment

​Deploy to Vercel

​Self-Hosted Client

​Security Hardening

​1. Firewall Configuration

​2. Fail2ban Setup

​3. Secure SSH

​Monitoring and Logging

​1. Application Monitoring

​2. Log Management

​3. Health Monitoring

​Backup Strategy

​Automated Backup Script

​Scaling Strategies

​Horizontal Scaling

​Vertical Scaling

​Database Optimization

​Troubleshooting

​Application Won’t Start

​High Memory Usage

​Database Performance Issues

​Maintenance

​Regular Tasks

​Update Procedure

​Next Steps

Build docs developers (and LLMs) love

Overview

Production Architecture

Recommended Stack

Component Requirements

Pre-Deployment Checklist

Server Setup

1. Provision Server

2. Initial Server Configuration

3. Install PostgreSQL

4. Install Redis

Application Deployment

1. Clone Repository

2. Create Virtual Environment

3. Configure Environment

4. Initialize Database

5. Configure Supervisor

Nginx Configuration

1. Configure Nginx

2. Obtain SSL Certificate

Client Deployment

Deploy to Vercel

Self-Hosted Client

Security Hardening

1. Firewall Configuration

2. Fail2ban Setup

3. Secure SSH

Monitoring and Logging

1. Application Monitoring

2. Log Management

3. Health Monitoring

Backup Strategy

Automated Backup Script

Scaling Strategies

Horizontal Scaling

Vertical Scaling

Database Optimization

Troubleshooting

Application Won’t Start

High Memory Usage

Database Performance Issues

Maintenance

Regular Tasks

Update Procedure

Next Steps