Skip to main content

Frontend Service

The Frontend Service is the entry point for all external communication with the Temporal cluster. It acts as an API gateway, handling request validation, rate limiting, and routing to appropriate internal services.

Overview

The Frontend Service exposes three main gRPC APIs:
  1. Workflow Service API: Primary API for workflow operations (start, signal, query, etc.)
  2. Operator Service API: Cluster operations and maintenance (add search attributes, etc.)
  3. Admin Service API: Advanced administrative operations (workflow reset, shard management)

Core Responsibilities

API Gateway

The Frontend Service acts as the single entry point for all external requests:
  • Protocol Translation: Converts external gRPC requests to internal service calls
  • Request Routing: Directs requests to the appropriate History or Matching Service instance
  • Response Aggregation: Collects responses from internal services and returns to clients

Request Validation

Before forwarding requests, the Frontend validates:
  • Verifies namespace exists and is active
  • Checks namespace state (registered, deprecated, deleted)
  • For global namespaces, checks if cluster is active
  • Workflow input payload size
  • Activity input payload size
  • Memo and search attribute sizes
  • Configurable via dynamic config
  • Workflow ID length
  • Activity ID length
  • Task queue name length
  • Prevents database key length issues
  • Required fields are present
  • Enum values are valid
  • Timeout values are reasonable
  • Retry policies are valid
// Code entrypoint: service/frontend/workflow_handler.go
// Main handler for Workflow Service API
// Contains validation logic for all workflow operations

Rate Limiting

The Frontend Service enforces multiple layers of rate limiting:

Global Rate Limits

  • Global RPS: Total requests per second across all namespaces
  • Operator RPS: Separate limit for operator APIs
  • Protects the cluster from overload

Namespace Rate Limits

  • Per-Namespace RPS: Requests per second per namespace
  • Visibility RPS: Separate limit for list/scan operations
  • Long-Running Request Concurrency: Limits concurrent long-poll requests
Rate limits are configured via dynamic configuration:
// Code entrypoint: service/frontend/configs/quotas.go
// Rate limiting configuration and enforcement

Per-API Rate Limits

Some APIs have specialized rate limits:
  • Namespace replication-inducing APIs (create, update, delete namespace)
  • Worker versioning APIs (build ID operations)
  • Batch operations

Request Routing

The Frontend routes requests to the correct service instance:

History Service Routing

For workflow operations:
  1. Calculate shard ID from workflow ID (consistent hashing)
  2. Look up History Service instance owning that shard
  3. Forward request via gRPC to that instance
// Shard ID calculation:
// shardID = hash(namespaceID, workflowID) % numShards

Matching Service Routing

For task queue operations:
  1. Hash task queue name to determine partition
  2. Look up Matching Service instance owning that partition
  3. Forward request via gRPC to that instance

Namespace-Based Routing

For multi-cluster setups:
  • Check if namespace is active in current cluster
  • If not, optionally forward to active cluster
  • Controlled by EnableNamespaceNotActiveAutoForwarding config

Health Checking

The Frontend provides gRPC health check endpoints:
  • Standard gRPC health check protocol
  • Returns SERVING when all dependencies are healthy
  • Returns NOT_SERVING during shutdown or degraded state
// Code entrypoint: service/frontend/health_check.go
// Health check implementation

API Handlers

Workflow Service API

The primary API for workflow operations: Workflow Lifecycle:
  • StartWorkflowExecution
  • SignalWorkflowExecution
  • SignalWithStartWorkflowExecution
  • RequestCancelWorkflowExecution
  • TerminateWorkflowExecution
Queries and Updates:
  • QueryWorkflowExecution
  • UpdateWorkflowExecution
  • PollWorkflowExecutionUpdate
History and Metadata:
  • GetWorkflowExecutionHistory
  • DescribeWorkflowExecution
  • ListOpenWorkflowExecutions
  • ListClosedWorkflowExecutions
  • ScanWorkflowExecutions
  • CountWorkflowExecutions
Task Operations:
  • PollWorkflowTaskQueue
  • PollActivityTaskQueue
  • RespondWorkflowTaskCompleted
  • RespondActivityTaskCompleted
  • RecordActivityTaskHeartbeat
// Code entrypoint: service/frontend/workflow_handler.go
// Main implementation of Workflow Service API
// Over 7000 lines of handler code

Operator Service API

Cluster operations and configuration: Search Attributes:
  • AddSearchAttributes
  • RemoveSearchAttributes
  • ListSearchAttributes
Cluster Management:
  • AddOrUpdateRemoteCluster
  • RemoveRemoteCluster
  • ListClusters
Nexus Endpoints:
  • CreateNexusEndpoint
  • UpdateNexusEndpoint
  • DeleteNexusEndpoint
  • ListNexusEndpoints
// Code entrypoint: service/frontend/operator_handler.go
// Operator API implementation

Admin Service API

Advanced administrative operations: Workflow Maintenance:
  • DescribeMutableState
  • GetWorkflowExecutionRawHistory
  • ResendReplicationTasks
  • RefreshWorkflowTasks
Shard Management:
  • CloseShard
  • GetShard
DLQ Operations:
  • GetDLQMessages
  • PurgeDLQMessages
  • MergeDLQMessages
History Imports:
  • ImportWorkflowExecution
// Code entrypoint: service/frontend/admin_handler.go
// Admin API implementation

Namespace Handling

Namespace Registry

The Frontend maintains a cache of namespace metadata:
  • Loaded from persistence on startup
  • Updated via watch mechanism
  • Provides fast lookup for request validation
  • Handles namespace state transitions
// Code entrypoint: service/frontend/namespace_handler.go
// Namespace operations and registry management

Global Namespaces

For multi-cluster deployments:
  • Namespaces can be global (replicated across clusters)
  • One cluster is “active” for write operations
  • Frontend checks if current cluster is active
  • Can auto-forward requests to active cluster

Namespace State Machine

Namespaces have states:
  • Registered: Normal operational state
  • Deprecated: Marked for deletion, read-only
  • Deleted: Namespace is removed

HTTP APIs

Nexus HTTP Handler

The Frontend exposes an HTTP endpoint for Nexus operations:
  • Implements Nexus protocol over HTTP
  • Used for cross-namespace workflow invocations
  • Supports synchronous and asynchronous operations
// Code entrypoint: service/frontend/nexus_http_handler.go
// HTTP handler for Nexus protocol

OpenAPI HTTP Handler

Provides OpenAPI/Swagger documentation:
  • Serves OpenAPI specification
  • UI for exploring APIs
  • Useful for development and debugging

Version Checking

The Frontend checks client and server version compatibility:
  • Validates SDK versions are supported
  • Checks worker build IDs
  • Enforces minimum version requirements
  • Provides upgrade recommendations
// Code entrypoint: service/frontend/version_checker.go
// Version validation logic

Error Handling

The Frontend translates internal errors to gRPC status codes:
  • Maps service errors to appropriate gRPC codes
  • Adds error details for client consumption
  • Sanitizes error messages to avoid leaking internals
// Code entrypoint: service/frontend/errors.go
// Error handling and translation

Common Error Patterns

InvalidArgument: Request validation failed NotFound: Workflow or namespace not found AlreadyExists: Workflow ID already in use FailedPrecondition: Namespace not active, wrong cluster ResourceExhausted: Rate limit exceeded Unavailable: Temporary failure, retry PermissionDenied: Authorization failed

Configuration

Frontend Service configuration includes:
// Code entrypoint: service/frontend/service.go
type Config struct {
    NumHistoryShards int32  // Total number of history shards
    RPS              int     // Global requests per second
    GlobalRPS        int     // Global RPS across all instances
    
    // Per-namespace limits
    MaxNamespaceRPSPerInstance int
    MaxNamespaceVisibilityRPSPerInstance int
    
    // Size limits
    BlobSizeLimitError int  // Max payload size (error)
    BlobSizeLimitWarn  int  // Max payload size (warning)
    MemoSizeLimitError int  // Max memo size
    MaxIDLengthLimit   int  // Max ID length
    
    // Feature flags
    EnableNamespaceNotActiveAutoForwarding bool
    DisableListVisibilityByFilter bool
}
Most configuration is dynamic and can be changed at runtime via dynamic configuration.

Metrics and Observability

The Frontend emits metrics for:
  • Request counts per API
  • Request latency percentiles
  • Error rates by error type
  • Rate limiting events
  • Long-poll queue depth

Shutdown Behavior

During graceful shutdown:
  1. Health check returns NOT_SERVING
  2. Drain period allows in-flight requests to complete
  3. New requests are rejected
  4. gRPC server stops accepting connections
Configured via:
  • ShutdownDrainDuration: Time to drain in-flight requests
  • ShutdownFailHealthCheckDuration: When to fail health checks

Further Reading

History Service

Where Frontend routes workflow operations

Matching Service

Where Frontend routes task operations

Architecture Overview

High-level system architecture

Workflow Lifecycle

How requests flow through services

Build docs developers (and LLMs) love