Running Tests

Temporal Server has a comprehensive test suite covering unit, integration, and functional tests.

Test Categories

Temporal defines three categories of tests:

Unit Tests

Tests with no external dependencies beyond the test target and Go mocks. Run fast and should have maximum coverage. Location: Throughout the codebase in *_test.go files

Integration Tests

Tests covering integration between the server and its dependencies (Cassandra, PostgreSQL, MySQL, Elasticsearch, etc.). Locations:

./common/persistence/tests - Database integration tests
./tools/tests - Database tool integration tests
./temporaltest - Embedded server tests

Functional Tests

End-to-end tests covering the complete functionality of Temporal Server. Locations:

./tests - Main functional test suite
./tests/xdc - Cross-DC replication tests
./tests/ndc - Named datacenter tests

Run Unit Tests

make unit-test

Unit tests do not require runtime dependencies.

Run Integration Tests

make integration-test

Integration tests require runtime dependencies. Start them first:

make start-dependencies

Run Functional Tests

make functional-test

Optionally specify persistence type and driver:

PERSISTENCE_TYPE=nosql PERSISTENCE_DRIVER=cassandra make functional-test
PERSISTENCE_TYPE=sql PERSISTENCE_DRIVER=mysql8 make functional-test
PERSISTENCE_TYPE=sql PERSISTENCE_DRIVER=postgres12 make functional-test

Run All Tests

make test

This runs unit, integration, and functional tests sequentially.

Test Options

Build Tags

Tests require specific build tags:

test_dep - Enables test hooks (required for most tests)
integration - Includes integration test dependencies
disable_grpc_modules - Reduces binary size during tests

The Makefile automatically applies: -tags disable_grpc_modules,test_dep

Test Timeout

Default timeout is 35 minutes (configurable via TEST_TIMEOUT):

TEST_TIMEOUT=60m make test

Race Detection

Race detection is enabled by default. To disable:

TEST_RACE_FLAG=off make unit-test

Test Shuffling

Test order is randomized by default. To disable:

TEST_SHUFFLE_FLAG=off make unit-test

Running Individual Tests

Run a specific test or test suite:

go test -v -tags test_dep <package_path> -run <TestName>

Examples:

# Run a specific test
go test -v -tags test_dep ./common/persistence -run TestPersistenceStartWorkflow

# Run a test suite
go test -v -tags test_dep github.com/temporalio/temporal/common/persistence -run TestCassandraPersistenceSuite

# Run with testify method filter
go test -v -tags test_dep ./common/persistence -run TestCassandraPersistenceSuite -testify.m TestPersistenceStartWorkflow

Code Coverage

Generate coverage reports:

Unit Test Coverage

make unit-test-coverage

Integration Test Coverage

make integration-test-coverage

Functional Test Coverage

make functional-test-coverage

Coverage reports are written to .testoutput/ and available on Codecov.

Install the Codecov Browser Extension to see code coverage directly in GitHub PRs.

Test Configuration

Build Tags

test_dep - Enables test hooks implementation for production code paths that are difficult to test
TEMPORAL_DEBUG - Extends functional test timeouts for debugging sessions
disable_grpc_modules - Disables gRPC modules for faster compilation

Environment Variables

Logging:

TEMPORAL_TEST_LOG_FORMAT - Output format (json or console)
TEMPORAL_TEST_LOG_LEVEL - Log verbosity (debug, info, warn, error, fatal)

OpenTelemetry:

TEMPORAL_TEST_OTEL_OUTPUT - File path for OTEL trace output on test failures

Cluster Configuration:

TEMPORAL_TEST_SHARED_CLUSTERS - Number of shared test clusters (default varies)
TEMPORAL_TEST_DEDICATED_CLUSTERS - Number of dedicated test clusters (default varies)

Data Encoding:

TEMPORAL_TEST_DATA_ENCODING - Persistence encoding (proto3 or json, default: proto3)

CGO:

CGO_ENABLED - Set to 0 to disable CGO for faster builds (default: 0)

Writing Tests

See the development guidelines for best practices on writing tests. Key test utilities:

Test Cluster

Use testcore.NewEnv(t) for functional tests:

func TestMyFeature(t *testing.T) {
    s := testcore.NewEnv(t)
    // Each test gets its own isolated namespace
    namespace := s.Namespace()
    // ...
}

Test Variables

Use testvars package for generating test identifiers:

func TestWorkflow(t *testing.T) {
    tv := testvars.New(t)
    req := &workflowservice.StartWorkflowExecutionRequest{
        RequestId:    tv.Any().String(),
        Namespace:    tv.NamespaceName().String(),
        WorkflowId:   tv.WorkflowID(),
        WorkflowType: tv.WorkflowType(),
        TaskQueue:    tv.TaskQueue(),
    }
}

Task Poller

For end-to-end workflow testing:

s := testcore.NewEnv(t)
poller := s.TaskPoller
// Use poller to handle workflow and activity tasks

Debugging Tests

IDE Debugging (GoLand)

Add build tags to “Go tool arguments” in Run/Debug configuration:

-tags disable_grpc_modules,test_dep

For integration tests, add the integration tag:

-tags disable_grpc_modules,test_dep,integration

See GoLand Debugging documentation for details.

OpenTelemetry Tracing

Debug tests using OTEL traces:

export OTEL_BSP_SCHEDULE_DELAY=100
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
export OTEL_TRACES_EXPORTER=otlp
export TEMPORAL_OTEL_DEBUG=true

# Start OTEL collector (e.g., Grafana Tempo)
make start-dependencies

# Run your test
go test -v -tags test_dep ./tests -run TestMyFeature

Traces are available in Grafana at localhost:3000. See tracing.md for more details.

Fault Injection Tests

Run functional tests with fault injection enabled:

make functional-with-fault-injection-test

Or manually:

go test ./tests -tags test_dep -enableFaultInjection=true

Test Helpers

softassert Package

softassert.That logs errors without stopping test execution:

softassert.That(t, condition, "error message")
// Test continues even if condition is false
// Test ultimately fails if any soft assertions failed

testhooks Package

Injects test-specific behavior into production code (last resort - prefer mocking):

// Only available with -tags=test_dep
testhooks.RegisterHook("UpdateWithStartInBetweenLockAndStart", func() {
    // Inject behavior at specific execution point
})

Test Parallelization

All tests should use t.Parallel() unless there’s a specific reason not to:

func TestMyFeature(t *testing.T) {
    t.Parallel()
    // test code
}

Automate parallelization:

make parallelize-tests

Opt out with //parallelize:ignore comment.

Mixed Brain Tests

Tests that require CGO:

make mixed-brain-test

Troubleshooting

Tests Timeout

Increase timeout:

TEST_TIMEOUT=60m make test

Tests Fail with “image not found”

Start dependencies:

make start-dependencies

Tests Pass Locally but Fail in CI

Ensure you’re using the correct build tags:

go test -tags disable_grpc_modules,test_dep ./...

Clean Test Cache

make clean-test-output

Contributing

Internal APIs

​Test Categories

​Unit Tests

​Integration Tests

​Functional Tests

​Running Tests

​Run Unit Tests

​Run Integration Tests

​Run Functional Tests

​Run All Tests

​Test Options

​Build Tags

​Test Timeout

​Race Detection

​Test Shuffling

​Running Individual Tests

​Code Coverage

​Unit Test Coverage

​Integration Test Coverage

​Functional Test Coverage

​Test Configuration

​Build Tags

​Environment Variables

​Writing Tests

​Test Cluster

​Test Variables

​Task Poller

​Debugging Tests

​IDE Debugging (GoLand)

​OpenTelemetry Tracing

​Fault Injection Tests

​Test Helpers

​softassert Package

​testhooks Package

​Test Parallelization

​Mixed Brain Tests

​Troubleshooting

​Tests Timeout

​Tests Fail with “image not found”

​Tests Pass Locally but Fail in CI

​Clean Test Cache

Build docs developers (and LLMs) love

Test Categories

Unit Tests

Integration Tests

Functional Tests

Running Tests

Run Unit Tests

Run Integration Tests

Run Functional Tests

Run All Tests

Test Options

Build Tags

Test Timeout

Race Detection

Test Shuffling

Running Individual Tests

Code Coverage

Unit Test Coverage

Integration Test Coverage

Functional Test Coverage

Test Configuration

Build Tags

Environment Variables

Writing Tests

Test Cluster

Test Variables

Task Poller

Debugging Tests

IDE Debugging (GoLand)

OpenTelemetry Tracing

Fault Injection Tests

Test Helpers

softassert Package

testhooks Package

Test Parallelization

Mixed Brain Tests

Troubleshooting

Tests Timeout

Tests Fail with “image not found”

Tests Pass Locally but Fail in CI

Clean Test Cache