Configuration Validation - SGIVU Config Repository

Overview

Proper validation prevents configuration errors that can cause service failures or security vulnerabilities. This guide covers tools and techniques for validating the SGIVU config repository.

Invalid YAML syntax or missing required properties can prevent services from starting. Always validate before merging configuration changes.

YAML Syntax Validation

Using yamllint

The yamllint tool checks YAML syntax and formatting:

# Install yamllint
pip install yamllint

# Or with apt
sudo apt-get install yamllint

Validating Configuration Files

Navigate to Config Repository

cd ~/workspace/source

Run yamllint

yamllint *.yml

No output means all files are valid. Errors will be reported with line numbers:

sgivu-auth-dev.yml
  12:1      error    too many blank lines (3 > 2)  (empty-lines)
  23:81     error    line too long (87 > 80 characters)  (line-length)

Fix Reported Issues

Address any syntax errors or formatting issues before committing.

yamllint Configuration

Create a .yamllint file in the repository root to customize validation rules:

extends: default

rules:
  line-length:
    max: 120
    level: warning
  indentation:
    spaces: 2
    indent-sequences: true
  comments:
    min-spaces-from-content: 1
  empty-lines:
    max: 2

Adjust rules based on your team’s coding standards. The above configuration is more lenient than yamllint defaults.

Testing Configuration Locally

Starting the Config Server

Before testing, ensure the Config Server is running and pointing to your local repository:

# Using docker-compose from sgivu-docker-compose
cd ../sgivu/infra/compose/sgivu-docker-compose
docker compose up -d sgivu-config

# Verify it's running
docker compose ps sgivu-config

Verifying Config Server Setup

Check that the Config Server can access the Git repository:

curl http://localhost:8888/actuator/health

Expected response:

{
  "status": "UP",
  "components": {
    "configServer": {
      "status": "UP",
      "details": {
        "repositories": [
          {
            "name": "sgivu-config-repo",
            "profiles": ["dev", "prod"],
            "label": "main"
          }
        ]
      }
    }
  }
}

Testing Configuration Retrieval

Testing Service Configurations

Use curl to fetch configuration and verify it’s correct:

curl http://localhost:8888/sgivu-auth/dev

Understanding the Response

The Config Server returns merged configuration from base and profile-specific files:

{
  "name": "sgivu-auth",
  "profiles": ["dev"],
  "label": null,
  "version": "abc123def",
  "state": null,
  "propertySources": [
    {
      "name": "file:///config-repo/sgivu-auth-dev.yml",
      "source": {
        "spring.datasource.url": "jdbc:postgresql://host.docker.internal:5432/sgivu_auth_dev",
        "management.endpoint.health.show-details": "always"
      }
    },
    {
      "name": "file:///config-repo/sgivu-auth.yml",
      "source": {
        "server.port": 9000,
        "management.endpoints.web.exposure.include": "health, info",
        "management.tracing.sampling.probability": 0.1
      }
    }
  ]
}

Configuration from profile-specific files (e.g., -dev.yml) overrides values from base files.

Validating Specific Properties

Extracting Property Values

Use jq to extract specific properties for validation:

# Install jq if needed
sudo apt-get install jq

# Check database URL
curl -s http://localhost:8888/sgivu-auth/dev | \
  jq '.propertySources[].source["spring.datasource.url"]' | \
  grep -v null

# Verify Zipkin endpoint
curl -s http://localhost:8888/sgivu-gateway/default | \
  jq '.propertySources[].source["management.zipkin.tracing.endpoint"]' | \
  grep -v null

# Check Eureka URL
curl -s http://localhost:8888/sgivu-user/dev | \
  jq '.propertySources[].source["eureka.client.service-url.defaultZone"]' | \
  grep -v null

Validating Required Properties

Create a validation script to check that required properties are set:

#!/bin/bash
# validate-config.sh

SERVICE=$1
PROFILE=$2

echo "Validating $SERVICE configuration for profile $PROFILE..."

# Fetch configuration
CONFIG=$(curl -s http://localhost:8888/$SERVICE/$PROFILE)

# Check for required properties
REQUIRED_PROPS=(
  "server.port"
  "spring.application.name"
  "eureka.client.service-url.defaultZone"
)

for PROP in "${REQUIRED_PROPS[@]}"; do
  VALUE=$(echo $CONFIG | jq -r ".propertySources[].source[\"$PROP\"]" | grep -v null | head -1)
  if [ -z "$VALUE" ]; then
    echo "❌ Missing required property: $PROP"
    exit 1
  else
    echo "✅ $PROP = $VALUE"
  fi
done

echo "Validation successful!"

Usage:

chmod +x validate-config.sh
./validate-config.sh sgivu-auth dev

Pre-Commit Validation

Git Pre-Commit Hook

Automate validation before commits using a Git hook:

#!/bin/bash
# .git/hooks/pre-commit

echo "Running YAML validation..."

# Get list of staged YAML files
YAML_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep '\.ya\?ml$')

if [ -z "$YAML_FILES" ]; then
  exit 0
fi

# Run yamllint on staged files
echo "$YAML_FILES" | xargs yamllint

if [ $? -ne 0 ]; then
  echo "❌ YAML validation failed. Please fix errors before committing."
  exit 1
fi

echo "✅ YAML validation passed"
exit 0

Make it executable:

chmod +x .git/hooks/pre-commit

This hook runs automatically before every commit, preventing invalid YAML from being committed.

Pre-Merge Validation Checklist

Before merging configuration changes to main:

Testing Configuration Changes

Local Development Workflow

Make Configuration Changes

Edit the relevant YAML files in ~/workspace/source:

cd ~/workspace/source
vim sgivu-auth-dev.yml

Validate Syntax

yamllint sgivu-auth-dev.yml

Commit Locally

git add sgivu-auth-dev.yml
git commit -m "Update auth service database pool size"

Test with Config Server

If using a local file-based Config Server, refresh it:

docker restart sgivu-config

Then verify the change:

curl http://localhost:8888/sgivu-auth/dev | jq

Test with Service

Restart or refresh the service:

docker restart sgivu-auth
# Or use /actuator/refresh if supported

Check that it starts successfully:

docker logs sgivu-auth
curl http://localhost:9000/actuator/health

Push to Remote

Once validated locally:

git push origin feature/update-auth-config

Common Validation Errors

YAML Syntax Errors

Indentation Error

Error:

mapping values are not allowed here

Cause: Inconsistent indentation or missing space after colon.Fix:

# ❌ Wrong
spring:
  datasource:
    url:jdbc:postgresql://localhost:5432/db

# ✅ Correct
spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/db

Unquoted Special Characters

Error:

could not find expected ':'

Cause: Special characters in values need quoting.Fix:

# ❌ Wrong
secret-key: ${SERVICE_SECRET}

# ✅ Correct
secret-key: "${SERVICE_SECRET}"

Duplicate Keys

Error:

found duplicate key

Cause: Same key defined multiple times at the same level.Fix: Remove or merge the duplicate keys.

Configuration Errors

Missing Required Property

Symptom: Service fails to start with error like:

Binding to target failed: Property 'spring.datasource.url' is required

Solution: Add the required property or provide a default value:

spring:
  datasource:
    url: ${DB_URL:jdbc:postgresql://localhost:5432/defaultdb}

Environment Variable Not Resolved

Symptom: Service uses literal ${VAR_NAME} instead of the value.Solution:

Verify the environment variable is set
Check that Spring Boot can access it
Provide a default value: ${VAR_NAME:default_value}

Automated Validation in CI/CD

GitHub Actions Example

Add automated validation to your CI pipeline:

name: Validate Configuration

on:
  pull_request:
    paths:
      - '**.yml'
      - '**.yaml'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install yamllint
        run: pip install yamllint
      
      - name: Validate YAML syntax
        run: yamllint *.yml
      
      - name: Start Config Server
        run: |
          docker run -d -p 8888:8888 \
            -v $PWD:/config-repo \
            -e SPRING_CLOUD_CONFIG_SERVER_GIT_URI=file:///config-repo \
            hyness/spring-cloud-config-server:latest
          sleep 10
      
      - name: Test configuration retrieval
        run: |
          services=("sgivu-auth" "sgivu-gateway" "sgivu-user" "sgivu-client" "sgivu-vehicle")
          for service in "${services[@]}"; do
            echo "Testing $service..."
            curl -f http://localhost:8888/$service/dev || exit 1
          done

Best Practices

Validate Early

Run yamllint before committing. Use pre-commit hooks to enforce this.

Test Locally

Always test configuration changes with Config Server and target services before pushing.

Automate Checks

Use CI/CD pipelines to automatically validate all configuration changes.

Document Changes

Explain why configuration changed in commit messages and pull requests.

Use Version Control

Leverage Git history to track configuration changes over time.

Review Together

Require peer review for production configuration changes.

Security

Validate that secrets are not committed

Configuration Refresh

Apply validated changes without restarts

Testing Changes

Comprehensive guide to testing configuration

Troubleshooting

Debug configuration issues

Configuration Refresh

⌘I

Overview

Getting Started

Service Configuration

Environment Management

Operations

​Overview

​YAML Syntax Validation

​Using yamllint

​Validating Configuration Files

​yamllint Configuration

​Testing Configuration Locally

​Starting the Config Server

​Verifying Config Server Setup

​Testing Configuration Retrieval

​Testing Service Configurations

​Understanding the Response

​Validating Specific Properties

​Extracting Property Values

​Validating Required Properties

​Pre-Commit Validation

​Git Pre-Commit Hook

​Pre-Merge Validation Checklist

​Testing Configuration Changes

​Local Development Workflow

​Common Validation Errors

​YAML Syntax Errors

​Configuration Errors

​Automated Validation in CI/CD

​GitHub Actions Example

​Best Practices

Validate Early

Test Locally

Automate Checks

Document Changes

Use Version Control

Review Together

​Related Resources

Security

Configuration Refresh

Testing Changes

Troubleshooting

Build docs developers (and LLMs) love

Overview

YAML Syntax Validation

Using yamllint

Validating Configuration Files

yamllint Configuration

Testing Configuration Locally

Starting the Config Server

Verifying Config Server Setup

Testing Configuration Retrieval

Testing Service Configurations

Understanding the Response

Validating Specific Properties

Extracting Property Values

Validating Required Properties

Pre-Commit Validation

Git Pre-Commit Hook

Pre-Merge Validation Checklist

Testing Configuration Changes

Local Development Workflow

Common Validation Errors

YAML Syntax Errors

Configuration Errors

Automated Validation in CI/CD

GitHub Actions Example

Best Practices

Related Resources