Skip to main content

​
Overview

The dagster create command compiles your Metaflow flow into a self-contained Dagster definitions file. Each Metaflow step becomes a Dagster @op, and the flow’s DAG structure is preserved as a Dagster @job.

​
Command Syntax

python my_flow.py dagster create [OPTIONS] [DEFINITIONS_FILE]

​
Arguments

​
DEFINITIONS_FILE
string
Path to write the generated Dagster definitions file. If not specified, defaults to <job_name>_dagster.py (lowercased).

​
Options

​
--name
string
Dagster job name. Defaults to the flow name. If the flow uses @project(name=...), the job name is automatically prefixed with the project name (e.g., myproject_FlowName).
​
--with
string
Inject a Metaflow step decorator at deploy time. Can be specified multiple times. Examples: --with=sandbox or --with='resources:cpu=4,memory=8000'.
​
--workflow-timeout
int
Maximum wall-clock seconds for the entire job run. The job will be terminated if it exceeds this duration.
​
--tag
string
Tag all objects produced by Dagster job runs with this tag. Can be specified multiple times. Tags are forwarded to every metaflow step subprocess.
​
--namespace
string
Metaflow namespace for the production run. Sets the namespace for all steps executed by the Dagster job.

​
Examples

​
Basic Usage

Generate a Dagster definitions file with default settings: 
python my_flow.py dagster create
# Creates: myflow_dagster.py

​
Custom Job Name

Specify a custom name for the Dagster job: 
python my_flow.py dagster create --name nightly_pipeline
# Creates: nightly_pipeline_dagster.py
# Job name in Dagster: nightly_pipeline

​
Inject Step Decorators

Add step decorators at deploy time without modifying the flow source: 
python my_flow.py dagster create --with=sandbox

​
Set Workflow Timeout

Cap the total execution time for the entire job: 
python my_flow.py dagster create --workflow-timeout 3600
# Job will timeout after 1 hour

​
Add Tags

Tag all runs for tracking and organization: 
python my_flow.py dagster create --tag env:prod

​
Complete Example

Combine multiple options for production deployment: 
python train_flow.py dagster create train_flow_prod.py \
  --name production_training \
  --with=sandbox \
  --with='resources:cpu=8,memory=16000' \
  --workflow-timeout 7200 \
  --tag env:prod \
  --tag version:3.1 \
  --namespace production

​
Output

On success, the command writes the generated Dagster definitions file and displays: 
Compiling MyFlow to Dagster job MyFlow...
Dagster job MyFlow for flow MyFlow written to myflow_dagster.py.
Load it in Dagster with:
    dagster dev -f myflow_dagster.py

​
Generated File Contents

The generated definitions file includes:
  • One @op for each Metaflow step
  • A @job that preserves the flow’s DAG structure
  • Proper handling of splits, joins, and foreach branches
  • RetryPolicy for steps with @retry decorator
  • Execution timeouts for steps with @timeout decorator
  • ScheduleDefinition if the flow has @schedule decorator
  • SensorDefinition for @trigger and @trigger_on_finish decorators
  • Typed Config class for Metaflow Parameter definitions
  • Environment variables from @environment decorators

​
Next Steps

Launch a Run

Trigger a Dagster job execution from the CLI

Resume Failed Runs

Resume a failed run and skip completed steps

​
See Also