CI/CD Pipeline

OddsEngine uses GitHub Actions to automate testing, quality checks, and deployment processes, ensuring code quality and reliable releases.

Overview

Our CI/CD pipeline automates:

Continuous Integration

Automated testing and quality checks on every push and PR

Continuous Deployment

Automated deployment to staging and production environments

Code Quality

SonarQube analysis for maintainability and security

Docker Builds

Containerized builds for consistent environments

CI Workflow

The Continuous Integration workflow runs on every push and pull request to ensure code quality.

Trigger Events

CI runs automatically on:

on:
  push:
    branches: [ "main", "develop" ]
  pull_request:
    branches: [ "main", "develop" ]

Push to main: Production code validation
Push to develop: Integration branch checks
Pull Requests: Pre-merge validation

CI Pipeline Stages

Checkout Code

- uses: actions/checkout@v4

Retrieves the latest code from the repository.

Smoke Test

- name: Smoke
  run: echo "CI running"

Basic verification that the workflow is executing.

This is a minimal implementation. In a full pipeline, this stage would include actual smoke tests to verify critical functionality.

Full CI Pipeline (Recommended)

While the current implementation is minimal, a complete CI pipeline should include:

Environment Setup

- name: Set up Python
  uses: actions/setup-python@v4
  with:
    python-version: '3.10'

- name: Cache dependencies
  uses: actions/cache@v3
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}

Install Dependencies

- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
    pip install pytest pytest-cov flake8

Linting & Style Checks

- name: Lint with flake8
  run: |
    # Stop build if there are Python syntax errors
    flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
    # Check for PEP 8 compliance
    flake8 . --count --max-complexity=10 --max-line-length=127 --statistics

Run Tests

- name: Test with pytest
  run: |
    pytest tests/ --cov=src --cov-report=xml --cov-report=term

Code Quality Analysis

- name: SonarQube Scan
  uses: sonarsource/sonarqube-scan-action@master
  env:
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
    SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}

Build Docker Image

- name: Build Docker image
  run: |
    docker build -t oddsengine:${{ github.sha }} .
    docker tag oddsengine:${{ github.sha }} oddsengine:latest

CD Workflow

The CD workflow file (cd.yml) is currently empty. Deployment automation should be configured based on your hosting environment.

Recommended CD Pipeline

A complete deployment pipeline should include:

Build & Test

Run full CI pipeline
Ensure all tests pass
Validate code quality gates

Build Artifacts

Build Docker images
Tag with version/commit SHA
Push to container registry

Deploy to Staging

Deploy to staging environment
Run smoke tests
Validate deployment health

Deploy to Production

Require manual approval
Deploy to production
Monitor for errors
Rollback capability

Deployment Triggers

Recommended trigger strategy:

Automatic Deployment
Manual Deployment
Release Tags

on:
  push:
    branches: [ "develop" ]

Auto-deploy develop branch to staging environment for integration testing.

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy'
        required: true
        type: choice
        options:
          - staging
          - production

Manual trigger for controlled production deployments.

on:
  push:
    tags:
      - 'v*.*.*'

Deploy on version tag creation (e.g., v1.0.0).

Quality Gates

Code must pass these quality gates before merging:

Test Coverage

Minimum: 80% code coverage
Measurement: pytest-cov
Enforcement: Fail pipeline if below threshold

pytest --cov=src --cov-fail-under=80

Code Quality

Tool: SonarQube
Metrics: Maintainability, Reliability, Security
Rating: B or higher required
Debt: < 5% technical debt ratio

Linting

Tool: flake8
Standard: PEP 8 compliance
Complexity: Max cyclomatic complexity of 10
No: Syntax errors or undefined names

Security

Dependencies: No critical vulnerabilities
Code: No security hotspots in SonarQube
Secrets: No hardcoded credentials

Branch Protection Rules

Protect important branches with required checks:

Main Branch

Require PR before merging
Require at least 1 approval
Require CI pipeline to pass
Require branch to be up to date
No direct pushes allowed

Develop Branch

Require PR before merging
Require CI pipeline to pass
Allow maintainers to bypass (emergency fixes)

Docker Integration

Our pipeline uses Docker for consistent builds and deployments:

Dockerfile

Ensure your Dockerfile is optimized for CI/CD:

FROM python:3.10-slim

WORKDIR /app

# Install dependencies first (better caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Run tests during build (optional)
RUN pytest tests/ || true

CMD ["python", "main.py"]

Docker Compose for Local CI

Test CI pipeline locally:

docker-compose -f docker-compose.ci.yml up --build

Monitoring & Notifications

Pipeline Status

Monitor pipeline health:

GitHub Actions Dashboard: View all workflow runs
Badges: Add status badges to README
Notifications: Configure GitHub notifications

Status Badge

Add to your README:

![CI](https://github.com/nicosanlucon/OddsEngine/workflows/CI/badge.svg)

Notifications

Configure Slack/Discord notifications for:

Pipeline failures on main or develop
Deployment completions
Critical security vulnerabilities

Secrets Management

Never commit secrets to the repository. Use GitHub Secrets for sensitive data.

Required Secrets

Configure these in GitHub Settings > Secrets:

SONAR_TOKEN: SonarQube authentication
SONAR_HOST_URL: SonarQube server URL
DOCKER_USERNAME: Docker registry username
DOCKER_PASSWORD: Docker registry password
DEPLOY_KEY: SSH key for deployment

Using Secrets

env:
  SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Best Practices

Fast Feedback

Keep CI runs under 10 minutes. Use caching and parallel jobs.

Fail Fast

Run quick checks (linting) before expensive ones (integration tests).

Reproducible Builds

Use Docker and locked dependencies for consistency.

Clear Failures

Provide actionable error messages when pipeline fails.

Troubleshooting

Common Issues

Pipeline Fails on Dependencies

Problem: pip install fails or times outSolution:

Use caching for pip dependencies
Check requirements.txt for version conflicts
Verify Python version matches local environment

Tests Pass Locally but Fail in CI

Problem: Environment differencesSolution:

Run tests in Docker locally: docker-compose up --build
Check for hardcoded paths or environment variables
Verify database or API mocks are configured

Docker Build Timeout

Problem: Image build takes too longSolution:

Use layer caching effectively
Install dependencies in separate layer
Use .dockerignore to exclude unnecessary files

Expanding the Pipeline

As the project grows, consider adding:

Performance Testing

Load testing for API endpoints
Response time monitoring
Memory and CPU profiling

Security Scanning

Dependency vulnerability scanning (Snyk, Dependabot)
Static security analysis (Bandit)
Container image scanning

Database Migrations

Automated migration testing
Rollback verification
Data integrity checks

E2E Testing

Selenium or Playwright tests
User flow validation
Cross-browser testing

Resources

The CI/CD pipeline is continuously evolving. Contribute improvements through pull requests following our contributing guidelines.

Academic Context: This CI/CD implementation is part of the DevOps practices taught in Fundamentos de Ingeniería de Software at Pontificia Universidad Javeriana (2026).

Architecture

Contributing

Reference

Overview

Continuous Integration

Continuous Deployment

Code Quality

Docker Builds

CI Workflow

Trigger Events

CI Pipeline Stages

Full CI Pipeline (Recommended)

CD Workflow

Recommended CD Pipeline

Deployment Triggers

Quality Gates

Branch Protection Rules

Main Branch

Develop Branch

Docker Integration

Dockerfile

Docker Compose for Local CI

Monitoring & Notifications

Pipeline Status

Status Badge

Notifications

Secrets Management

Required Secrets

Using Secrets

Best Practices

Fast Feedback

Fail Fast

Reproducible Builds

Clear Failures

Troubleshooting

Common Issues

Expanding the Pipeline

Resources

Build docs developers (and LLMs) love

Architecture

Contributing

Reference

​Overview

Continuous Integration

Continuous Deployment

Code Quality

Docker Builds

​CI Workflow

​Trigger Events

​CI Pipeline Stages

​Full CI Pipeline (Recommended)

​CD Workflow

​Recommended CD Pipeline

​Deployment Triggers

​Quality Gates

​Branch Protection Rules

​Main Branch

​Develop Branch

​Docker Integration

​Dockerfile

​Docker Compose for Local CI

​Monitoring & Notifications

​Pipeline Status

​Status Badge

​Notifications

​Secrets Management

​Required Secrets

​Using Secrets

​Best Practices

Fast Feedback

Fail Fast

Reproducible Builds

Clear Failures

​Troubleshooting

​Common Issues

​Expanding the Pipeline

​Resources

Build docs developers (and LLMs) love

Overview

CI Workflow

Trigger Events

CI Pipeline Stages

Full CI Pipeline (Recommended)

CD Workflow

Recommended CD Pipeline

Deployment Triggers

Quality Gates

Branch Protection Rules

Main Branch

Develop Branch

Docker Integration

Dockerfile

Docker Compose for Local CI

Monitoring & Notifications

Pipeline Status

Status Badge

Notifications

Secrets Management

Required Secrets

Using Secrets

Best Practices

Troubleshooting

Common Issues

Expanding the Pipeline

Resources