Event Sourcing Overview
Event sourcing is a persistence pattern where state changes are stored as a sequence of events rather than by updating records in place. The current state of any entity is reconstructed by replaying its events from the beginning. Benefits for durable workflows:- Complete audit trail: Every state change is recorded with its timestamp and context
- Debugging: Replay the exact sequence of events that led to any state
- Consistency: Events provide a single source of truth for all entity state
- Recoverability: State can be reconstructed from the event log after failures
- Runs: Workflow execution instances (materialized in storage)
- Steps: Individual atomic operations within a workflow (materialized in storage)
- Hooks: Suspension points that can receive external data (materialized in storage)
- Waits: Sleep or delay operations (materialized in storage)
Entity Lifecycles
Each entity type follows a specific lifecycle defined by the events that can affect it. Events transition entities between states, and certain states are terminal—once reached, no further transitions are possible.Run Lifecycle
A run represents a single execution of a workflow function. Runs begin inpending state when created, transition to running when execution starts, and end in one of three terminal states.
Run states:
pending: Created but not yet executingrunning: Actively executing workflow codecompleted: Finished successfully with an output valuefailed: Terminated due to an unrecoverable errorcancelled: Explicitly cancelled by the user or system
Step Lifecycle
A step represents a single invocation of a step function. Steps can retry on failure, either transitioning back topending via step_retrying or being re-executed directly with another step_started event.
Step states:
pending: Created but not yet executing, or waiting to retryrunning: Actively executing step codecompleted: Finished successfully with a result valuefailed: Terminated after exhausting all retry attempts
Hook Lifecycle
A hook represents a suspension point that can receive external data. Hooks can receive multiple payloads while active and are disposed when no longer needed. Hook states:active: Ready to receive payloads (hook exists in storage)disposed: No longer accepting payloads (hook is deleted from storage)conflicted: Hook creation failed because the token is already in use by another workflow
Wait Lifecycle
A wait represents a sleep operation. Waits track when a delay period has elapsed. Wait states:waiting: Delay period has not yet elapsedcompleted: Delay period has elapsed, workflow can resume
Event Types Reference
Events are categorized by the entity type they affect. Each event contains metadata including a timestamp and acorrelationId that links the event to a specific entity.
Run Events
| Event | Description |
|---|---|
run_created | Creates a new workflow run in pending state. Contains the deployment ID, workflow name, input arguments, and optional execution context. |
run_started | Transitions the run to running state when execution begins. |
run_completed | Transitions the run to completed state with the workflow’s return value. |
run_failed | Transitions the run to failed state with error details and optional error code. |
run_cancelled | Transitions the run to cancelled state. Can be triggered from pending or running states. |
Step Events
| Event | Description |
|---|---|
step_created | Creates a new step in pending state. Contains the step name and serialized input arguments. |
step_started | Transitions the step to running state. Includes the current attempt number for retries. |
step_completed | Transitions the step to completed state with the step’s return value. |
step_failed | Transitions the step to failed state with error details. The step will not be retried. |
step_retrying | (Optional) Transitions the step back to pending state for retry. Contains the error that caused the retry and optional delay before the next attempt. |
Hook Events
| Event | Description |
|---|---|
hook_created | Creates a new hook in active state. Contains the hook token and optional metadata. |
hook_conflict | Records that hook creation failed because the token is already in use by another active hook. |
hook_received | Records that a payload was delivered to the hook. The hook remains active and can receive more payloads. |
hook_disposed | Deletes the hook from storage (conceptually transitioning to disposed state). |
Wait Events
| Event | Description |
|---|---|
wait_created | Creates a new wait in waiting state. Contains the timestamp when the wait should complete. |
wait_completed | Transitions the wait to completed state when the delay period has elapsed. |
Event Consumption and Replay
TheEventsConsumer class manages the sequential processing of events during workflow replay. It implements a subscription-based model where different parts of the workflow runtime can subscribe to specific event types.
How it works:
- Sequential Processing: Events are consumed in order, one at a time
- Callback Subscription: Components subscribe with callbacks that inspect each event
- Result Handling: Callbacks return:
Consumed: Event processed, continue to next eventNotConsumed: Event doesn’t match, try next callbackFinished: Event processed, remove this callback
Entity IDs
All entities in the Workflow DevKit use a consistent ID format: a 4-character prefix followed by an underscore and a ULID (Universally Unique Lexicographically Sortable Identifier).| Entity | Prefix | Example |
|---|---|---|
| Run | wrun_ | wrun_01HXYZ123ABC456DEF789GHJ |
| Step | step_ | step_01HXYZ123ABC456DEF789GHJ |
| Hook | hook_ | hook_01HXYZ123ABC456DEF789GHJ |
| Wait | wait_ | wait_01HXYZ123ABC456DEF789GHJ |
| Event | evnt_ | evnt_01HXYZ123ABC456DEF789GHJ |
| Stream | strm_ | strm_01HXYZ123ABC456DEF789GHJ |
- Prefixes enable introspection: Given any ID, you can immediately identify what type of entity it refers to
- ULIDs enable chronological ordering: ULIDs encode a timestamp in their first 48 bits, making them lexicographically sortable by creation time
Terminal States and Consistency
Terminal states represent the end of an entity’s lifecycle. Once an entity reaches a terminal state, no further events can transition it to another state. Run terminal states:completed: Workflow finished successfullyfailed: Workflow encountered an unrecoverable errorcancelled: Workflow was explicitly cancelled
completed: Step finished successfullyfailed: Step failed after all retry attempts
disposed: Hook has been deleted from storageconflicted: Hook creation failed due to token conflict
completed: Delay period has elapsed