Skip to main content
Metaflow provides robust production deployment capabilities that allow you to schedule and orchestrate your workflows on various platforms. Once your flow is tested locally, you can deploy it to production with a single command.

​
Deployment Platforms

Metaflow supports multiple orchestration platforms for production deployments:

AWS Step Functions

Deploy to AWS Step Functions for serverless orchestration

Argo Workflows

Deploy to Kubernetes-native Argo Workflows

Apache Airflow

Generate Airflow DAGs from Metaflow flows

​
Key Concepts

​
Production Tokens

Production tokens are used to organize and secure production deployments. Each deployment is associated with a production token that:
  • Creates a unique namespace for the flow’s runs
  • Controls access to redeploy or modify the flow
  • Ensures multiple deployments don’t conflict
When you deploy a flow for the first time, Metaflow generates a production token: 
python myflow.py step-functions create
The token is displayed and cached locally. To authorize team members to redeploy: 
python myflow.py step-functions create --authorize <token>

​
Namespaces

Production runs are organized in namespaces with the format production:<token>. To analyze production results in notebooks: 
from metaflow import namespace, Flow

namespace('production:<token>')
run = Flow('MyFlow').latest_run

​
Project Branches

When using the @project decorator, Metaflow automatically manages namespaces for different branches: 
from metaflow import FlowSpec, step, project

@project(name='my_project')
class MyFlow(FlowSpec):
    @step
    def start(self):
        pass
Deploy to different branches: 
# Deploy to production branch
python myflow.py --branch prod step-functions create

# Deploy to user branch
python myflow.py --branch user.alice step-functions create

​
Deployment Workflow

A typical production deployment workflow:
  1. Develop and test locally 
    python myflow.py run
  2. Deploy to staging 
    python myflow.py --branch staging step-functions create
  3. Test the production deployment 
    python myflow.py step-functions trigger
  4. Deploy to production 
    python myflow.py --branch prod step-functions create
  5. Monitor execution 
    python myflow.py step-functions list-runs

​
Best Practices

Always commit your flow code to version control before deploying. This ensures reproducibility and enables rollbacks if needed.
Before deploying to production, test your flow with a representative subset of data to catch potential issues early.
Use decorators like @resources to set appropriate CPU and memory limits for production workloads.
Use @retry and @catch decorators to handle transient failures and make your flows more resilient.
Use --tag to annotate deployments with version numbers or commit hashes for easier tracking.

​
Configuration

Production deployments require proper configuration of:
  • Datastore: S3, Azure Blob Storage, or Google Cloud Storage for artifact storage
  • Metadata service: For tracking run metadata and lineage
  • Compute environment: AWS Batch, Kubernetes, or other compute platforms
Refer to the Configuration Guide for details.

​
Next Steps

Scheduling Flows

Learn how to schedule flows with cron expressions

Event Triggering

Trigger flows based on events and upstream dependencies

Monitoring

Monitor and debug production flows

Orchestrators

Deep dive into specific orchestrator platforms

Build docs developers (and LLMs) love

Get started for freeTalk to us