CI/CD Workflows - Product Distribution Dashboard

Overview

The Product Distribution Dashboard uses GitHub Actions for automated testing, code quality analysis, and validation. The CI/CD pipeline is organized into PR validation workflows and main branch workflows.

PR Validation Orchestrator

The main PR validation workflow (pr-validation.yml) orchestrates all checks when a pull request is created or updated.

Change Detection

The workflow uses path filtering to run only relevant checks:

frontend:
  - 'frontend/**'
  - '.github/workflows/pr-frontend-analysis.yml'
backend:
  - 'backend/**'
  - '.github/workflows/pr-backend-test-and-sonar.yml'
data:
  - 'data/**'
  - '.github/workflows/pr-data-validation.yml'

Benefits:

Only runs workflows affected by changes
Reduces CI time and resource usage
Provides faster feedback to developers

PR Gate

The PR gate job ensures all required workflows pass before allowing merge:

if [ "${{ needs.frontend.result }}" == "failure" ] || \
   [ "${{ needs.backend.result }}" == "failure" ] || \
   [ "${{ needs.data.result }}" == "failure" ]; then
  echo "❌ One or more workflows failed"
  exit 1
fi

Frontend Analysis Workflow

File: .github/workflows/pr-frontend-analysis.yml Trigger: Called by PR validation workflow when frontend changes detected

Steps

Environment Setup
- Checkout code
- Set up Node.js 20
- Cache npm dependencies based on package-lock.json
Dependency Installation
```
npm ci
```
Code Quality Checks
- ESLint: npm run lint
- Prettier: npm run format:check
Build Verification
```
npm run build
```

Working Directory: ./frontend

Backend Test & Sonar Workflow

File: .github/workflows/pr-backend-test-and-sonar.yml Trigger: Called by PR validation workflow when backend changes detected

Job 1: Backend Tests

Runs unit and integration tests with code coverage:

mvn -B clean verify

Configuration:

Java 17 (Temurin distribution)
Maven cache enabled
Working directory: ./backend

Artifacts:

JaCoCo coverage reports uploaded to jacoco-coverage
Retained for 1 day
Path: backend/target/site/jacoco/

Job 2: Backend Sonar Analysis

Performs static code analysis using SonarCloud: Prerequisites:

Requires backend-tests job to complete
Downloads JaCoCo coverage reports

Steps:

Checkout with full git history (fetch-depth: 0)
Set up Java 17
Cache Sonar packages
Run Sonar scanner:
```
mvn -B compile sonar:sonar
```

Environment Variables:

GITHUB_TOKEN: Automatic GitHub token
SONAR_TOKEN: SonarCloud authentication (secret)

Data Validation Workflow

File: .github/workflows/pr-data-validation.yml Trigger: Called by PR validation workflow when data changes detected Validates the structure and integrity of JSON data files across three environments:

Job 1: Validate Current Folder

Required Files:

products.json
warehouses.json
stores.json

Validations:

Exactly 3 files present
All required files exist
Valid JSON syntax

Job 2: Validate Test Folder

Same validations as current folder for data/test/ directory.

Job 3: Validate Archive Folder

Special Requirements:

Must contain multiple of 3 files
Files must follow naming pattern: {base}_v{number}.json
Versions must be consecutive starting from v1
Valid JSON syntax for all files

Example:

products_v1.json
products_v2.json
warehouses_v1.json
warehouses_v2.json
stores_v1.json
stores_v2.json

Main Branch Workflows

Backend Sonar Analysis

File: .github/workflows/main-backend-sonar.yml Triggers:

Push to main branch
Changes to:
- backend/**
- .github/workflows/main-backend-sonar.yml
- render.yaml
Manual workflow dispatch

Process:

mvn -B clean verify sonar:sonar

Configuration:

Full git history fetch for accurate blame information
Java 17 with Maven cache
Sonar package caching
Combined test execution and analysis

Purpose:

Maintains code quality metrics on main branch
Tracks quality trends over time
Provides baseline for PR comparisons

Workflow Triggers Summary

Workflow	Trigger	Path Filters
PR Validation	`pull_request`	All paths
Frontend Analysis	`workflow_call`	`frontend/**`
Backend Test & Sonar	`workflow_call`	`backend/**`
Data Validation	`workflow_call`	`data/**`
Main Backend Sonar	`push` to `main`	`backend/**`, `render.yaml`

Required Secrets

Configure these secrets in GitHub repository settings:

SONAR_TOKEN: SonarCloud authentication token
- Generate from SonarCloud account settings
- Required for both PR and main branch analysis

Manual Workflow Execution

All workflows support manual triggering via workflow_dispatch:

Go to Actions tab in GitHub
Select workflow from left sidebar
Click Run workflow button
Select branch and confirm

Best Practices

For Developers

Run locally first: Execute npm run lint and mvn verify before pushing
Check workflow results: Review failed checks and fix issues promptly
Keep PRs focused: Smaller PRs trigger fewer workflows and get faster feedback

For Maintainers

Monitor workflow duration: Optimize if workflows take too long
Update dependencies: Keep actions and tools up to date
Review failed workflows: Investigate and fix flaky tests
Cache effectively: Ensure dependency caching is working

Troubleshooting

Workflow Not Triggered

Issue: PR created but workflows not running Solutions:

Check if changes match path filters
Verify workflow files are in .github/workflows/
Ensure GitHub Actions is enabled for repository

SonarCloud Analysis Failed

Issue: Backend Sonar job fails Solutions:

Verify SONAR_TOKEN secret is configured
Check SonarCloud organization and project key in pom.xml
Ensure JaCoCo reports are generated

Data Validation Failed

Issue: JSON validation errors Solutions:

Validate JSON syntax using python3 -m json.tool file.json
Check file names match expected patterns
Verify archive versions are consecutive

Frontend Build Failed

Issue: Angular build errors Solutions:

Run npm ci locally to ensure clean install
Check for TypeScript compilation errors
Verify all dependencies are in package.json

Backend

Frontend

CI/CD

​Overview

​PR Validation Orchestrator

​Change Detection

​PR Gate

​Frontend Analysis Workflow

​Steps

​Backend Test & Sonar Workflow

​Job 1: Backend Tests

​Job 2: Backend Sonar Analysis

​Data Validation Workflow

​Job 1: Validate Current Folder

​Job 2: Validate Test Folder

​Job 3: Validate Archive Folder

​Main Branch Workflows

​Backend Sonar Analysis

​Workflow Triggers Summary

​Required Secrets

​Manual Workflow Execution

​Best Practices

​For Developers

​For Maintainers

​Troubleshooting

​Workflow Not Triggered

​SonarCloud Analysis Failed

​Data Validation Failed

​Frontend Build Failed

Build docs developers (and LLMs) love