Architecture Overview
Temporal Server is a distributed, durable execution platform that executes application logic (Workflows) reliably in the face of failures. This page provides a high-level overview of Temporal’s system architecture.Core Design Principles
Temporal’s architecture is built on several fundamental principles:Event Sourcing
The system functions via event sourcing: an append-only history of events is stored for each workflow execution, and all required workflow state can be recreated at any time by replaying this history.Separation of Concerns
User code defining workflows is segregated into:- Workflow definitions: Must be deterministic and have no side effects (with specific exceptions)
- Activity definitions: Must either be idempotent or non-retryable (at-least-once or at-most-once semantics)
Durable Execution
Workflows must execute correctly in the face of transient failures in server processes and user-hosted processes. The system guarantees that workflows will continue to make progress regardless of infrastructure failures.High-Level Architecture
The Temporal architecture is divided into two main categories of processes:User-Hosted Processes
User Application
User Application
Uses one of the Temporal SDKs as a gRPC client to communicate with the Temporal server to:
- Start and cancel workflows
- Send signals to running workflows
- Query workflow state
- Update running workflows
Worker Processes
Worker Processes
Users segregate application code into Temporal Workflow and Activity definitions, and host Worker processes which execute their Workflow and Activity code.Workers communicate with the Temporal server in two ways:
- Polling: Continuously poll the server for Workflow and Activity tasks
- Completion: Send commands (for Workflow Tasks) or results (for Activity Tasks) back to the server
Temporal Cluster Services
The Temporal cluster consists of four internal services:Frontend Service
The Frontend Service is the entry point for all external communication:- Handles all gRPC API requests from user applications and workers
- Performs request validation and rate limiting
- Routes requests to appropriate History or Matching service instances
- Manages namespace operations and metadata
- Exposes Admin, Operator, and Workflow service APIs
History Service
The History Service is the core of Temporal’s durable execution:- Manages individual Workflow Executions through sharding
- Maintains workflow execution state (Mutable State) and event history
- Processes requests from user applications and workers
- Drives workflow execution by enqueuing tasks
- Handles workflow lifecycle from start to completion
Matching Service
The Matching Service manages task queues:- Holds task queues polled by Temporal Worker processes
- Matches tasks from History Service with polling workers
- Manages task queue partitions for load balancing
- Implements task forwarding between partitions
- Handles both Workflow Tasks and Activity Tasks
Worker Service (Internal)
The Worker Service hosts background processing for the cluster:- Replicator: Handles replication tasks from remote clusters in multi-region setups
- Archiver: Archives workflow histories to long-term storage
- Scanner: Performs maintenance tasks like history scavenging
- Batcher: Processes batch operations across multiple workflows
- Parent Close Policy Worker: Handles child workflow cleanup
Task Types
Temporal uses different types of tasks to coordinate workflow execution:Workflow Tasks
Processed by resuming execution of user workflow code until it becomes blocked or completes. Workers send back commands specifying what is required to advance the workflow.Activity Tasks
Processed by attempting to execute an Activity. Workers send back the activity outcome (success or failure).History Tasks (Internal)
Internal tasks managed by the History Service:- Transfer Tasks: Available for immediate execution (e.g., enqueue Workflow/Activity tasks)
- Timer Tasks: Execute at a specific trigger time (e.g., workflow timers, timeouts)
- Visibility Tasks: Update workflow metadata in visibility storage
- Replication Tasks: Replicate events to remote clusters
- Archival Tasks: Archive workflow histories
Workflow Execution Flow
A typical workflow execution follows this pattern:Scaling and Sharding
History Shards
Workflow executions are partitioned into History Shards:- Fixed number of shards defined at cluster creation
- Each shard is owned by one History Service instance
- Shard ownership can move between instances for load balancing
- Each shard maintains its own task queues and processes workflows independently
Matching Partitions
Task queues are divided into partitions:- Default of 4 partitions per task queue
- Partitions can be loaded/unloaded dynamically
- Tasks and pollers can be forwarded between partitions
- Enables horizontal scaling of task processing
Persistence Architecture
Temporal requires two types of storage:Core Database
Stores essential workflow execution data:- Workflow execution state (Mutable State)
- Workflow history events
- Task queue information
- Namespace and cluster metadata
Visibility Store
Stores workflow metadata for search and filtering:- Workflow execution status
- Search attributes
- Start/close times
- Enables complex queries via List APIs
Consistency Guarantees
Temporal maintains strong consistency through:Database Transactions: Mutable State and History Tasks are persisted atomically using database transactions
Event Versioning: History Events are versioned and validated against Mutable State to ensure consistency
Transactional Outbox Pattern: Transfer Tasks ensure that operations in History Service are eventually reflected in Matching Service
Multi-Region Architecture
Temporal supports multi-cluster deployments with:- Active-passive or active-active configurations
- Cross-cluster replication via Replication Tasks
- Namespace-level replication and failover
- Conflict resolution for concurrent updates
Further Reading
History Service
Deep dive into workflow execution management
Matching Service
Task queue management and worker polling
Workflow Lifecycle
Detailed sequence diagrams of workflow execution
Event Sourcing
How Temporal implements event sourcing