Database Migrations - SASCOP BME SubTec

Overview

Django migrations are version control for your database schema. SASCOP BME SubTec uses migrations to manage database changes across multiple apps including operaciones, core, costa_fuera, and reportes.

How Migrations Work

Django migrations track changes to your models and apply them to the database:

Model Change - You modify a model in your code
Create Migration - Run makemigrations to generate migration files
Apply Migration - Run migrate to apply changes to the database
Track State - Django tracks which migrations have been applied

Creating Migrations

Automatic Migration Generation

When you modify models, Django can automatically generate migrations:

python manage.py makemigrations

For a specific app:

python manage.py makemigrations operaciones

Example Output:

Migrations for 'operaciones':
  operaciones/migrations/0002_add_contrato_fields.py
    - Add field fecha_inicio to contrato
    - Add field fecha_termino to contrato
    - Add field monto_mn to contrato
    - Add field monto_usd to contrato

Custom Migration Names

Provide a descriptive name for your migration:

python manage.py makemigrations operaciones --name add_contract_dates

Empty Migrations

Create an empty migration for custom SQL or data migrations:

python manage.py makemigrations operaciones --empty --name populate_default_data

Viewing Migrations

List All Migrations

View migration status for all apps:

python manage.py showmigrations

Example Output:

operaciones
 [X] 0001_initial
 [X] 0002_add_contrato_fields
 [ ] 0003_add_anexo_models
core
 [X] 0001_initial

[X] = Applied
[ ] = Not applied

View Specific App

python manage.py showmigrations operaciones

View Migration SQL

See the SQL that will be executed:

python manage.py sqlmigrate operaciones 0002

Example Output:

BEGIN;
--
-- Add field fecha_inicio to contrato
--
ALTER TABLE "contrato" ADD COLUMN "fecha_inicio" date NULL;
--
-- Add field fecha_termino to contrato
--
ALTER TABLE "contrato" ADD COLUMN "fecha_termino" date NULL;
COMMIT;

Applying Migrations

Apply All Pending Migrations

python manage.py migrate

Apply Migrations for Specific App

python manage.py migrate operaciones

Apply to Specific Migration

python manage.py migrate operaciones 0002

Non-Interactive Mode (Production)

Use --noinput to skip confirmations:

python manage.py migrate --noinput

Always use --noinput in automated deployment scripts.

Migration Files

Migration files are stored in each app’s migrations/ directory:

operaciones/migrations/
├── __init__.py
├── 0001_initial.py
├── 0002_add_contrato_fields.py
└── 0003_add_anexo_models.py

Migration File Structure

operaciones/migrations/0002_add_contrato_fields.py

from django.db import migrations, models

class Migration(migrations.Migration):

    dependencies = [
        ('operaciones', '0001_initial'),
    ]

    operations = [
        migrations.AddField(
            model_name='contrato',
            name='fecha_inicio',
            field=models.DateField(blank=True, null=True),
        ),
        migrations.AddField(
            model_name='contrato',
            name='fecha_termino',
            field=models.DateField(blank=True, null=True),
        ),
    ]

Data Migrations

Create migrations that modify data, not schema:

Create Empty Migration

python manage.py makemigrations operaciones --empty --name populate_default_statuses

Edit Migration File

operaciones/migrations/0004_populate_default_statuses.py

from django.db import migrations

def create_default_statuses(apps, schema_editor):
    Estatus = apps.get_model('operaciones', 'Estatus')
    
    default_statuses = [
        {'descripcion': 'Pendiente', 'nivel_afectacion': 1, 'activo': True},
        {'descripcion': 'En Proceso', 'nivel_afectacion': 1, 'activo': True},
        {'descripcion': 'Completado', 'nivel_afectacion': 1, 'activo': True},
    ]
    
    for status_data in default_statuses:
        Estatus.objects.create(**status_data)

def reverse_default_statuses(apps, schema_editor):
    Estatus = apps.get_model('operaciones', 'Estatus')
    Estatus.objects.filter(
        descripcion__in=['Pendiente', 'En Proceso', 'Completado']
    ).delete()

class Migration(migrations.Migration):

    dependencies = [
        ('operaciones', '0003_add_anexo_models'),
    ]

    operations = [
        migrations.RunPython(create_default_statuses, reverse_default_statuses),
    ]

Apply Migration

python manage.py migrate operaciones

Always provide a reverse operation for data migrations to support rollback.

Production Migration Strategy

Pre-Deployment Checklist

Test Migrations Locally

# Create fresh database
createdb sascop_test

# Set test database in .env
RDS_DB_NAME=sascop_test

# Run all migrations
python manage.py migrate

# Verify application works
python manage.py runserver

Review Migration SQL

# Check SQL for each new migration
python manage.py sqlmigrate operaciones 0005

Look for:

Table locks
Long-running operations
Data loss risks

Backup Production Database

pg_dump -h $RDS_HOSTNAME -U $RDS_USERNAME $RDS_DB_NAME > backup_$(date +%Y%m%d_%H%M%S).sql

Plan Downtime (if needed)

Some migrations require downtime:

Adding NOT NULL columns to large tables
Changing column types
Adding indexes to large tables

Deployment Migration Process

# 1. Deploy code (migrations not applied yet)
git pull origin main
pip install -r requirements.txt
python manage.py collectstatic --noinput

# 2. Apply migrations
python manage.py migrate --noinput

# 3. Restart application
sudo systemctl restart gunicorn

Common Migration Operations

Adding a Field

migrations.AddField(
    model_name='contrato',
    name='observaciones',
    field=models.TextField(blank=True, null=True),
)

Removing a Field

migrations.RemoveField(
    model_name='contrato',
    name='campo_obsoleto',
)

Renaming a Field

migrations.RenameField(
    model_name='contrato',
    old_name='descripcion',
    new_name='descripcion_contrato',
)

Adding an Index

migrations.AddIndex(
    model_name='conceptomaestro',
    index=models.Index(fields=['partida_ordinaria'], name='idx_partida_ord'),
)

Creating a Table

migrations.CreateModel(
    name='NuevoModelo',
    fields=[
        ('id', models.BigAutoField(primary_key=True)),
        ('descripcion', models.CharField(max_length=200)),
        ('activo', models.BooleanField(default=True)),
    ],
    options={
        'db_table': 'nuevo_modelo',
    },
)

Running Custom SQL

migrations.RunSQL(
    sql="CREATE INDEX CONCURRENTLY idx_pte_fecha ON pte_header (fecha_creacion);",
    reverse_sql="DROP INDEX idx_pte_fecha;"
)

Migration Dependencies

Migrations can depend on migrations from other apps:

class Migration(migrations.Migration):

    dependencies = [
        ('operaciones', '0004_previous_migration'),
        ('core', '0002_core_migration'),  # Dependency on core app
    ]

    operations = [
        # ...
    ]

Squashing Migrations

Combine multiple migrations into one for better performance:

python manage.py squashmigrations operaciones 0001 0010

This creates a new migration that replaces migrations 0001-0010.

Only squash migrations after they’ve been applied to all environments.

Handling Migration Conflicts

Conflict Scenario

Two developers create migrations independently:

Branch A: 0005_add_field_a.py
Branch B: 0005_add_field_b.py

Resolution

Identify Conflict

python manage.py migrate

Error: “Conflicting migrations detected”

Merge Migrations

Rename one migration:

mv operaciones/migrations/0005_add_field_b.py operaciones/migrations/0006_add_field_b.py

Update dependencies in the renamed file:

dependencies = [
    ('operaciones', '0005_add_field_a'),
]

Apply Migrations

python manage.py migrate

Testing Migrations

Unit Test Migrations

operaciones/tests/test_migrations.py

from django.test import TestCase
from django.apps import apps

class MigrationTest(TestCase):
    def test_default_statuses_created(self):
        Estatus = apps.get_model('operaciones', 'Estatus')
        
        # Verify default statuses exist
        self.assertTrue(
            Estatus.objects.filter(descripcion='Pendiente').exists()
        )
        self.assertTrue(
            Estatus.objects.filter(descripcion='En Proceso').exists()
        )

Integration Testing

# Create test database
createdb sascop_migration_test

# Run migrations from scratch
RDS_DB_NAME=sascop_migration_test python manage.py migrate

# Run tests
RDS_DB_NAME=sascop_migration_test python manage.py test

# Clean up
dropdb sascop_migration_test

Rollback Migrations

Rollback to Specific Migration

# Rollback to migration 0003
python manage.py migrate operaciones 0003

Rollback All Migrations for an App

python manage.py migrate operaciones zero

Rolling back migrations can cause data loss. Always backup before rollback.

Fake Migrations

Mark migrations as applied without running them:

# Mark as applied
python manage.py migrate operaciones 0005 --fake

# Mark as not applied
python manage.py migrate operaciones 0004 --fake

Use cases:

Manual schema changes already applied
Fixing migration state inconsistencies
Testing migration rollbacks

Best Practices

Always Review Generated Migrations

Never blindly trust auto-generated migrations. Review:

SQL output (sqlmigrate)
Field types and constraints
Default values
Data loss potential

Keep Migrations Small

One logical change per migration:

Add field: one migration
Populate data: separate migration
Remove old field: third migration

Test Before Production

# Test on staging
python manage.py migrate --plan
python manage.py migrate

# Verify application works
# Test rollback

Never Edit Applied Migrations

Once a migration is applied to any environment:

Never modify it
Create a new migration instead
Exception: Migrations not yet in production

Commit Migrations to Version Control

git add operaciones/migrations/0005_new_migration.py
git commit -m "Add migration for contract dates"

Document Complex Migrations

class Migration(migrations.Migration):
    """
    Adds contract date fields and populates them from existing data.
    
    This migration:
    1. Adds fecha_inicio and fecha_termino fields
    2. Populates dates from anexo records
    3. Adds database indexes for performance
    
    Rollback: Removes added fields and indexes
    """
    # ...

Troubleshooting

Migration Fails Midway

# Check migration status
python manage.py showmigrations

# If migration is marked as applied but failed:
python manage.py migrate operaciones 0004 --fake
python manage.py migrate operaciones 0005

Inconsistent Migration State

# Clear migration records
python manage.py migrate operaciones zero --fake

# Reapply all migrations
python manage.py migrate operaciones --fake-initial

Missing Migration Files

If migration files are lost but database has them applied:

# Recreate initial migration
python manage.py makemigrations operaciones --name recovered_initial

# Mark as applied
python manage.py migrate operaciones --fake

Monitoring Migration Performance

import time
from django.db import migrations

def time_consuming_operation(apps, schema_editor):
    start = time.time()
    # ... operation ...
    duration = time.time() - start
    print(f"Operation took {duration:.2f} seconds")

class Migration(migrations.Migration):
    operations = [
        migrations.RunPython(time_consuming_operation),
    ]

Setup

Architecture

Deployment

​Overview

​How Migrations Work

​Creating Migrations

​Automatic Migration Generation

​Custom Migration Names

​Empty Migrations

​Viewing Migrations

​List All Migrations

​View Specific App

​View Migration SQL

​Applying Migrations

​Apply All Pending Migrations

​Apply Migrations for Specific App

​Apply to Specific Migration

​Non-Interactive Mode (Production)

​Migration Files

​Migration File Structure

​Data Migrations

​Production Migration Strategy

​Pre-Deployment Checklist

​Deployment Migration Process

​Common Migration Operations

​Adding a Field

​Removing a Field

​Renaming a Field

​Adding an Index

​Creating a Table

​Running Custom SQL

​Migration Dependencies

​Squashing Migrations

​Handling Migration Conflicts

​Conflict Scenario

​Resolution

​Testing Migrations

​Unit Test Migrations

​Integration Testing

​Rollback Migrations

​Rollback to Specific Migration

​Rollback All Migrations for an App

​Fake Migrations

​Best Practices

​Troubleshooting

​Migration Fails Midway

​Inconsistent Migration State

​Missing Migration Files

​Monitoring Migration Performance

​Next Steps

Static Files

Environment Variables

Build docs developers (and LLMs) love

Overview

How Migrations Work

Creating Migrations

Automatic Migration Generation

Custom Migration Names

Empty Migrations

Viewing Migrations

List All Migrations

View Specific App

View Migration SQL

Applying Migrations

Apply All Pending Migrations

Apply Migrations for Specific App

Apply to Specific Migration

Non-Interactive Mode (Production)

Migration Files

Migration File Structure

Data Migrations

Production Migration Strategy

Pre-Deployment Checklist

Deployment Migration Process

Common Migration Operations

Adding a Field

Removing a Field

Renaming a Field

Adding an Index

Creating a Table

Running Custom SQL

Migration Dependencies

Squashing Migrations

Handling Migration Conflicts

Conflict Scenario

Resolution

Testing Migrations

Unit Test Migrations

Integration Testing

Rollback Migrations

Rollback to Specific Migration

Rollback All Migrations for an App

Fake Migrations

Best Practices

Troubleshooting

Migration Fails Midway

Inconsistent Migration State

Missing Migration Files

Monitoring Migration Performance

Next Steps