AWX Architecture
AWX is built on a modern, scalable architecture combining web technologies, message queuing, distributed task execution, and container orchestration. This guide provides a comprehensive look at how AWX works internally.High-Level Architecture
AWX consists of several interconnected components:Core Components
Django Backend
AWX’s backend is built on Django 4.x with Django REST Framework, providing the web framework, ORM, and API layer.Key Modules
- Models (awx/main/models/)
- API (awx/api/)
- Settings (awx/conf/)
AWX’s data models define the core domain objects:Key model hierarchies:
UnifiedJobTemplate→JobTemplate,ProjectUpdate,InventoryUpdateUnifiedJob→Job,ProjectUpdate,InventoryUpdate,WorkflowJob- Both use polymorphic inheritance for shared behavior
Database Schema
AWX uses PostgreSQL with several key design patterns:Partitioned Event Tables
Partitioned Event Tables
Job events are stored in partitioned tables for performance:Partitioning enables efficient retention policies and query performance.
main_jobevent- Parent tablemain_jobevent_*- Monthly partitions (e.g.,main_jobevent_202403)
awx/main/models/events.py:Polymorphic Tables
Polymorphic Tables
Unified jobs use polymorphic inheritance via django-polymorphic:Single table with a type discriminator allows querying across all job types.
RBAC Tables
RBAC Tables
AWX uses django-ansible-base for RBAC:
main_roledefinition- Role definitions (e.g., “Admin”, “Execute”)main_roleuserassignment- User assignments to roles on objectsmain_roleevaluation- Cached permission evaluations
React Frontend (ansible-ui)
The AWX UI is built with React and the ansible-ui framework.Location
Source:
awx/ui/The UI is built separately and static files are served by Django via collectstatic.Build Process
awx/ui/build/static/.- Real-time updates: WebSocket connection for job output streaming
- PatternFly components: Consistent Red Hat design language
- React Router: Client-side routing
- API integration: Axios-based HTTP client communicating with Django REST API
PostgreSQL Database
The PostgreSQL database stores all AWX state:- Tables
- Connection Pooling
- Migrations
- Organizations, Teams, Users:
main_organization,main_team,auth_user - Credentials:
main_credential,main_credentialtype - Inventories:
main_inventory,main_host,main_group - Projects:
main_project,main_projectupdate - Jobs:
main_job,main_jobtemplate,main_unifiedjob - Events:
main_jobevent(partitioned),main_inventoryupdateevent - Schedules:
main_schedule - Notifications:
main_notification,main_notificationtemplate - RBAC:
main_roledefinition,main_roleuserassignment
Redis
Redis provides caching and message brokering:Redis Usage:
- Settings cache (TTL: 60 seconds)
- Django session storage
- Django Channels layer for WebSockets
- Dispatcher job queue (via dispatcherd)
Task Execution Engine
The task execution engine is AWX’s core: the system that schedules, dispatches, and executes automation jobs.Dispatcher (awx/main/dispatch/)
AWX uses a custom task queue system called “dispatcher” (replacing the older Celery-based approach):Dispatcher Architecture
Dispatcher Architecture
Components (from
awx/main/dispatch/):- Pool (
pool.py): Worker process pool management - Reaper (
reaper.py): Cleanup of orphaned processes - Worker (
worker/): Worker process implementationstask.py- Task execution workercallback.py- Job event callback receiverdispatcherd.py- Dispatcher daemon
- Task published to Redis queue
- Dispatcher daemon polls queue
- Worker process consumes message
- Task callable executed
- Result stored (if applicable)
Task Manager
The task manager decides which jobs run when and where:Instance Group Selection
Chooses which instance group (control plane, execution nodes, container groups) should run the job.
Receptor Mesh Network
Receptor provides the communication layer between AWX control plane and execution nodes.- Purpose
- Configuration
- Work Units
- Secure mesh networking: Encrypted connections between nodes
- Job routing: Routes work units to appropriate execution nodes
- Bidirectional communication: Callback events from nodes to control plane
- Flexible topologies: Direct connections, hop nodes, peered networks
ansible-runner
Jobs execute via the ansible-runner library:Execution Environments: Container images (Podman/Docker) with Ansible + dependencies. Provides isolation, reproducibility, and version control for automation environments.
Job Lifecycle
Complete flow from job launch to completion:Job Created (status: new)
User clicks Launch or API receives POST to
/api/v2/job_templates/{id}/launch/- Job record created in database
- Status:
new - Job placed in task manager queue
Task Manager Evaluation (status: pending)
Task manager periodically evaluates pending jobs:Checks:
- Capacity available on execution nodes?
- Dependencies satisfied (e.g., project synced)?
- Concurrent job limits not exceeded?
Job Dispatch (status: waiting → running)
Once resources available:
- Job status →
waiting - Task manager selects execution node (or container group)
- Work unit submitted to Receptor
- Execution node receives work unit
- ansible-runner starts playbook execution
- Job status →
running
Event Streaming
During execution:
- ansible-runner emits events (task started, task completed, etc.)
- Events sent to AWX via callback plugin
- Stored in
main_jobeventpartitioned table - Broadcast to UI clients via WebSocket (Django Channels)
- UI displays real-time output
Job Completion (status: successful/failed/error)
When playbook finishes:
- Final status based on Ansible return code
- Job stats calculated (ok, changed, failed, skipped counts)
- Notifications triggered (if configured)
- Workflow successors evaluated (if part of workflow)
- Job status →
successful,failed, orerror
WebSocket Real-Time Updates
AWX uses Django Channels for WebSocket support:- UI connects to
wss://awx/websocket/ - Subscribes to job channel:
jobs-<job_id> - Job events published to channel via
emit_channel_notification():
- UI receives events and updates job output display in real-time
Instance and Instance Groups
AWX supports clustered deployments with multiple nodes:- Instance Types
- Instance Groups
- Capacity Algorithm
From
awx/main/models/ha.py:- Control: Web, API, task manager (no job execution)
- Execution: Job execution only (via Receptor)
- Hybrid: Both control and execution
- Hop: Receptor routing only (no AWX services)
Credential System
AWX’s credential system securely manages authentication:Credential Types
Fromawx/main/models/credential.py:
- Machine (SSH keys, passwords, privilege escalation)
- Source Control (Git, SVN)
- Vault (Ansible Vault passwords)
- Cloud (AWS, Azure, GCP, OpenStack)
- Network (for network devices)
- Container Registry
- HashiCorp Vault (external secret lookup)
Encryption
Credentials are encrypted at rest:SECRET_KEY in Django settings.
Credential Injection
Credentials are injected into jobs via environment variables or files:Notification System
AWX sends notifications on job success/failure: Notification Types:- Email (SMTP)
- Slack
- PagerDuty
- Webhooks
- IRC
- Mattermost
- Grafana
- Twilio (SMS)
- On job start
- On job success
- On job failure
Performance and Scaling
Horizontal Scaling
Add more control plane nodes for API capacity:
Execution Scaling
Add execution nodes for job capacity:
- Register new instances in AWX
- Add to instance groups
- Configure Receptor connections
- Capacity increases automatically
Database Performance
- Partitioned event tables reduce query times
- Indexes on frequently queried fields
- Connection pooling via PgBouncer
- Read replicas for reporting (if needed)
Caching Strategy
- Redis cache for settings (60s TTL)
- In-memory cache for RBAC evaluations (5s TTL)
- Browser caching for static assets
- API result caching (selective)
Security Considerations
- Authentication
- Secrets Management
- Network Security
- Username/password (local or LDAP)
- SAML 2.0
- OAuth2 (social auth: GitHub, Google, etc.)
- Token-based for API access
- Session management via Django
Monitoring and Observability
AWX provides multiple monitoring interfaces:Prometheus Metrics
Prometheus Metrics
AWX exports Prometheus metrics:
- Job counts by status
- Instance capacity and utilization
- Task manager queue depth
- Database connection pool stats
- HTTP request rates and latencies
/api/v2/metrics/ (requires authentication).OpenTelemetry (Experimental)
OpenTelemetry (Experimental)
AWX can export traces and logs to OpenTelemetry collectors:Enables distributed tracing across AWX components.
Activity Stream
Activity Stream
AWX logs all changes to resources in
main_activitystream table:- User actions (create, update, delete)
- Job launches
- Configuration changes
- Login/logout events
External Logging
External Logging
Forward logs to external aggregators:
- Splunk
- ELK stack (Elasticsearch, Logstash, Kibana)
- Grafana Loki
- Sumologic
Development Architecture
For developers working on AWX:Development Environment
Fromtools/docker-compose/README.md:
Project Structure
Testing
Summary
AWX’s architecture combines:Modern Web Stack
Django + DRF + React for robust web application foundation
Scalable Execution
Distributed task execution via Receptor mesh and ansible-runner
Enterprise Features
RBAC, audit logging, HA, secrets management, and monitoring
Related Documentation
Introduction
Overview of AWX features and capabilities
Getting Started
Install AWX and run your first job
AWX Source Code
Explore the complete source on GitHub
Ansible Docs
Official AWX documentation