Deployment - AI Data Science Service

This guide covers deploying the complete AI Data Science Service stack using Docker containers for production-ready, scalable deployments.

Overview

The deployment architecture consists of two containerized services:

API Service: FastAPI backend for model inference
Client Service: Web interface for interactive predictions

Both services are orchestrated using docker-compose for simplified management.

Prerequisites

Install Docker

Ensure Docker is installed on your system:

docker --version

Download Docker from docker.com if not installed.

Install Docker Compose

Verify docker-compose is available:

docker-compose --version

Docker Compose is included with Docker Desktop on macOS and Windows.

Train a Model

Ensure you have trained model files:

model/model_weights_001.pth
config/models-configs/model_config_001.yaml
processing/preprocessor.joblib

See the Training Models guide if needed.

Docker Architecture

Service Configuration

The docker-compose.yml defines the complete service stack:

services:
  api-service:
    build:
      context: .
      dockerfile: Dockerfile.api
    container_name: credit-score-api
    ports:
      - "8000:8000"
    volumes:
      - ./server:/app/server
      - ./inference:/app/inference
      - ./config:/app/config
      - ./model:/app/model
      - ./processing:/app/processing
    networks:
      - credit-score-network
    command: uvicorn server.api:app --host 0.0.0.0 --port 8000 --reload

  client-service:
    build:
      context: .
      dockerfile: Dockerfile.client
    container_name: credit-score-client
    ports:
      - "3000:3000"
    volumes:
      - ./examples/client_web:/app
    environment:
      - API_URL=http://localhost:8000
    networks:
      - credit-score-network
    command: uvicorn main:app --host 0.0.0.0 --port 3000 --reload

networks:
  credit-score-network:
    driver: bridge

Reference: docker-compose.yml:1-41

API Service Container

Dockerfile Configuration

The API container is built using a multi-stage, security-hardened Dockerfile:

# Base image with Python 3.11
FROM python:3.11-slim

# Environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# Create non-root user for security
RUN useradd -m -u 1000 appuser

WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY config/ config/
COPY inference/ inference/
COPY model/ model/
COPY processing/ processing/
COPY server/ server/

# Set ownership to non-root user
RUN chown -R appuser:appuser /app

USER appuser

EXPOSE 8000

CMD ["uvicorn", "server.api:app", "--host", "0.0.0.0", "--port", "8000"]

Reference: Dockerfile.api:1-43

Security Features

Non-root User

Container runs as appuser (UID 1000) instead of root, limiting potential security risks.

Minimal Base Image

Uses python:3.11-slim to reduce attack surface and image size.

No Cache Files

--no-cache-dir flag prevents storing unnecessary pip cache, reducing image size.

Unbuffered Output

PYTHONUNBUFFERED=1 ensures logs are immediately visible for monitoring.

Deployment Methods

Quick Start: Full Stack Deployment

Deploy both services with a single command:

docker-compose up --build

What happens during deployment?

Build Phase:
- Builds API container from Dockerfile.api
- Builds Client container from Dockerfile.client
- Installs all dependencies
- Copies application code
Network Setup:
- Creates credit-score-network bridge network
- Enables inter-container communication
Service Launch:
- Starts API service on port 8000
- Starts Client service on port 3000
- Mounts volumes for hot-reloading
Ready State:
- API available at http://localhost:8000
- Client available at http://localhost:3000
- API docs at http://localhost:8000/docs

Development Mode (Hot Reload)

For development with automatic code reloading:

docker-compose up

Volume mounts in docker-compose.yml enable hot-reloading. Changes to local files are immediately reflected in containers.

Production Mode (Detached)

Run services in the background:

docker-compose up -d --build

Manage services:

docker-compose logs -f

Service-Specific Deployment

Deploy Only API Service

docker-compose up --build api-service

Access the API:

Base URL: http://localhost:8000
Documentation: http://localhost:8000/docs
Health Check: http://localhost:8000 (redirects to docs)

Reference: server/api.py:50-55

Deploy Only Client Service

docker-compose up --build client-service

The client requires the API service to be running. Ensure API is accessible before starting the client.

Volume Mounts for Development

The docker-compose configuration mounts local directories for hot-reloading:

volumes:
  # Hot-reloading: map local directories to container
  - ./server:/app/server
  - ./inference:/app/inference
  - ./config:/app/config
  - ./model:/app/model
  - ./processing:/app/processing

Reference: docker-compose.yml:9-15 Benefits:

Instant code updates without rebuilding
Local file editing with IDE
Simplified debugging
Faster development iteration

For production, remove volume mounts to bake code into the image for better performance and security.

Networking

Services communicate through a dedicated bridge network:

networks:
  credit-score-network:
    driver: bridge

Reference: docker-compose.yml:39-41

Inter-Service Communication

Containers can reference each other by service name:

// From client-service container
fetch('http://api-service:8000/credit_score_prediction', {
  method: 'POST',
  body: JSON.stringify(data)
});

External Access

Port mappings expose services to the host machine:

API: localhost:8000 → api-service:8000
Client: localhost:3000 → client-service:3000

Environment Configuration

The client service uses environment variables for configuration:

environment:
  - API_URL=http://localhost:8000

Reference: docker-compose.yml:31-33 Customize for different environments:

Development
Staging
Production

environment:
  - API_URL=http://localhost:8000
  - DEBUG=true

environment:
  - API_URL=https://staging-api.example.com
  - DEBUG=false

environment:
  - API_URL=https://api.example.com
  - DEBUG=false
  - LOG_LEVEL=warning

Production Deployment Best Practices

Remove Development Features

Modify docker-compose.yml for production:

# Remove --reload flag
command: uvicorn server.api:app --host 0.0.0.0 --port 8000

# Remove volume mounts
# volumes:
#   - ./server:/app/server

Configure Resource Limits

Add resource constraints to prevent resource exhaustion:

api-service:
  # ... existing config
  deploy:
    resources:
      limits:
        cpus: '1'
        memory: 2G
      reservations:
        cpus: '0.5'
        memory: 1G

Use Environment Files

Store sensitive configuration in .env files:

.env

MODEL_VERSION=001
LOG_LEVEL=info
API_KEY=your-secret-key

Reference in docker-compose:

api-service:
  env_file:
    - .env

Implement Health Checks

Add health checks for monitoring:

api-service:
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:8000/docs"]
    interval: 30s
    timeout: 10s
    retries: 3
    start_period: 40s

Enable Logging

Configure log drivers for centralized logging:

api-service:
  logging:
    driver: "json-file"
    options:
      max-size: "10m"
      max-file: "3"

Scaling Deployments

Horizontal Scaling

Run multiple API instances behind a load balancer:

docker-compose up --scale api-service=3

Port conflicts will occur with multiple instances using the same port. Use a reverse proxy or load balancer.

Example with NGINX Load Balancer

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - api-service

  api-service:
    # Remove ports mapping
    expose:
      - "8000"
    deploy:
      replicas: 3

Monitoring and Observability

View Service Logs

docker-compose logs -f

Container Metrics

Monitor resource usage:

docker stats credit-score-api credit-score-client

Output shows:

CPU usage percentage
Memory usage and limit
Network I/O
Block I/O

Access Running Container

For debugging, access a running container:

docker exec -it credit-score-api /bin/bash

Inside the container:

# Check files
ls -la /app/model/

# Test API internally
curl http://localhost:8000/docs

# View Python packages
pip list

Troubleshooting

Port already in use

Error: Bind for 0.0.0.0:8000 failed: port is already allocatedSolution:

# Find process using the port
lsof -i :8000

# Kill the process
kill -9 <PID>

# Or change port in docker-compose.yml
ports:
  - "8080:8000"

Container exits immediately

Debugging steps:

# View container logs
docker-compose logs api-service

# Check container status
docker-compose ps

# Inspect container
docker inspect credit-score-api

Common causes:

Missing dependencies
File permission issues
Configuration errors

Cannot connect to API from client

Solution:

Verify both services are running:
```
docker-compose ps
```

Check network connectivity:

docker exec credit-score-client ping api-service

Verify API_URL environment variable:

docker exec credit-score-client env | grep API_URL

Volume mount permissions error

Error: Permission denied when accessing mounted filesSolution:

# Fix file permissions
sudo chown -R $USER:$USER .

# Or run with sudo (not recommended for production)
sudo docker-compose up

Out of memory errors

Solution:Increase Docker memory allocation:

Docker Desktop: Settings → Resources → Memory
Linux: Modify /etc/docker/daemon.json

Or add memory limits to containers:

api-service:
  mem_limit: 2g

Cleanup

Stop and Remove Containers

# Stop services
docker-compose down

# Stop and remove volumes
docker-compose down -v

# Stop and remove images
docker-compose down --rmi all

Remove Unused Docker Resources

# Remove dangling images
docker image prune

# Remove all unused containers, networks, images
docker system prune -a

docker system prune -a removes ALL unused Docker resources. Use with caution.

Advanced Deployment Options

Docker Swarm

For orchestrated, multi-node deployments:

# Initialize swarm
docker swarm init

# Deploy stack
docker stack deploy -c docker-compose.yml credit-score

Kubernetes

For production-grade orchestration, convert to Kubernetes manifests using Kompose:

# Install kompose
curl -L https://github.com/kubernetes/kompose/releases/download/v1.28.0/kompose-linux-amd64 -o kompose
chmod +x kompose

# Convert docker-compose to Kubernetes
./kompose convert

# Deploy to Kubernetes
kubectl apply -f .

Cloud Deployments

AWS ECS

Deploy using Amazon Elastic Container Service with Fargate for serverless containers.

Google Cloud Run

Deploy directly from container images with automatic scaling.

Azure Container Instances

Simple container hosting on Microsoft Azure.

Next Steps

Running Inference

Test your deployed API with predictions

MLflow Tracking

Monitor deployed model performance

API Reference

Explore complete API documentation

Model Configuration

Customize models for production

Get Started

Core Concepts

Guides

Use Cases

​Overview

​Prerequisites

​Docker Architecture

​Service Configuration

​API Service Container

​Dockerfile Configuration

​Security Features

Non-root User

Minimal Base Image

No Cache Files

Unbuffered Output

​Deployment Methods

​Quick Start: Full Stack Deployment

​Development Mode (Hot Reload)

​Production Mode (Detached)

​Service-Specific Deployment

​Deploy Only API Service

​Deploy Only Client Service

​Volume Mounts for Development

​Networking

​Inter-Service Communication

​External Access

​Environment Configuration

​Production Deployment Best Practices

​Scaling Deployments

​Horizontal Scaling

​Example with NGINX Load Balancer

​Monitoring and Observability

​View Service Logs

​Container Metrics

​Access Running Container

​Troubleshooting

​Cleanup

​Stop and Remove Containers

​Remove Unused Docker Resources

​Advanced Deployment Options

​Docker Swarm

​Kubernetes

​Cloud Deployments

AWS ECS

Google Cloud Run

Azure Container Instances

​Next Steps

Running Inference

MLflow Tracking

API Reference

Model Configuration

Build docs developers (and LLMs) love

Overview

Prerequisites

Docker Architecture

Service Configuration

API Service Container

Dockerfile Configuration

Security Features

Deployment Methods

Quick Start: Full Stack Deployment

Development Mode (Hot Reload)

Production Mode (Detached)

Service-Specific Deployment

Deploy Only API Service

Deploy Only Client Service

Volume Mounts for Development

Networking

Inter-Service Communication

External Access

Environment Configuration

Production Deployment Best Practices

Scaling Deployments

Horizontal Scaling

Example with NGINX Load Balancer

Monitoring and Observability

View Service Logs

Container Metrics

Access Running Container

Troubleshooting

Cleanup

Stop and Remove Containers

Remove Unused Docker Resources

Advanced Deployment Options

Docker Swarm

Kubernetes

Cloud Deployments

Next Steps