The zenml stack command group manages your MLOps stacks - collections of infrastructure components that define where and how your pipelines execute.
Overview A ZenML stack consists of:
Artifact Store (required) - Stores pipeline outputsOrchestrator (required) - Executes pipeline stepsContainer Registry (optional) - Stores Docker imagesModel Registry (optional) - Manages ML modelsStep Operator (optional) - Runs individual steps on specialized infrastructureFeature Store (optional) - Manages feature dataModel Deployer (optional) - Deploys models to productionExperiment Tracker (optional) - Tracks experiments and metricsAlerter (optional) - Sends notificationsAnnotator (optional) - Manages data labelingData Validator (optional) - Validates data qualityImage Builder (optional) - Builds Docker imagesDeployer (optional) - Manages infrastructure deploymentsLog Store (optional) - Aggregates and stores logs
Command Structure zenml stack < comman d > [OPTIONS] [ARGUMENTS]
Core Stack Commands Register a Stack Create a new stack by combining components:
zenml stack register < STACK_NAM E > [OPTIONS]
Required component options: Option Component Type -a, --artifact-storeArtifact store name -o, --orchestratorOrchestrator name
Optional component options: Option Component Type -c, --container_registryContainer registry -r, --model_registryModel registry -s, --step_operatorStep operator -f, --feature_storeFeature store -d, --model_deployerModel deployer -e, --experiment_trackerExperiment tracker -al, --alerterAlerter -an, --annotatorAnnotator -dv, --data_validatorData validator -i, --image_builderImage builder -D, --deployerDeployer -l, --log_storeLog store
Additional options: Option Description --setImmediately set this stack as active -p, --providerCloud provider (aws, azure, gcp) -sc, --connectorService connector name --secretSecrets to attach (can be specified multiple times) --envEnvironment variables in KEY=VALUE format (can be specified multiple times)
Examples: # Basic local stack zenml stack register local-stack \ -a local_artifact_store \ -o local_orchestrator # Production stack with multiple components zenml stack register production \ -a s3_artifact_store \ -o kubernetes_orchestrator \ -c aws_container_registry \ -r mlflow_model_registry \ -e mlflow_experiment_tracker \ --set # Stack with environment variables zenml stack register dev-stack \ -a local_artifact_store \ -o local_orchestrator \ --env AWS_REGION=us-west-2 \ --env LOG_LEVEL=DEBUG # Stack with cloud provider auto-configuration zenml stack register aws-stack \ --provider aws \ --connector my_aws_connector
List Stacks View all registered stacks:
zenml stack list [OPTIONS]
Filter options: Option Description --nameFilter by name --userFilter by user --component_typeFilter by component type --is_sharedFilter shared stacks (True/False) --sort_bySort results --columnsSelect columns to display --output, -oOutput format (table, json, yaml, csv, tsv)
Examples: # List all stacks zenml stack list # Filter by name zenml stack list --name "contains:prod" # List shared stacks zenml stack list --is_shared True # Custom columns zenml stack list --columns name,orchestrator,artifact_store # Export to JSON zenml stack list --output json
Example output: ┏━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━┯━━━━━━━━┓ ┃ Name │ Orchestrator │ Artifact Store │ Active ┃ ┠───────────────┼─────────────────┼─────────────────┼────────┨ ┃ default │ local │ local │ ✔ ┃ ┃ production │ kubernetes │ s3 │ ┃ ┃ staging │ kubernetes │ s3 │ ┃ ┗━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━┷━━━━━━━━┛
Set Active Stack Switch to a different stack:
zenml stack set < STACK_NAM E >
Example: zenml stack set production
Output:
✔ Active stack set to: production
Get Active Stack Show the currently active stack:
Output:
Describe a Stack View detailed information about a stack:
zenml stack describe [STACK_NAME]
Without a name, describes the active stack.
Example: zenml stack describe production
Output:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Stack: production ┃ ┠───────────────────────────────────────────────────┨ ┃ ID │ abc123... ┃ ┃ Created │ 2024-01-15 10:30:00 ┃ ┃ Updated │ 2024-03-01 15:45:00 ┃ ┃ Shared │ Yes ┃ ┠───────────────────────────────────────────────────┨ ┃ Component Type │ Component Name ┃ ┠───────────────────────────────────────────────────┨ ┃ Orchestrator │ kubernetes_orchestrator ┃ ┃ Artifact Store │ s3_artifact_store ┃ ┃ Container Reg. │ aws_container_registry ┃ ┃ Model Registry │ mlflow_model_registry ┃ ┃ Exp. Tracker │ mlflow_experiment_tracker ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Update a Stack Modify stack components:
zenml stack update < STACK_NAM E > [OPTIONS]
Examples: # Update orchestrator zenml stack update production --orchestrator new_orchestrator # Add container registry zenml stack update production --container_registry my_registry # Remove component (set to empty string) zenml stack update production --model_deployer "" # Add environment variable zenml stack update production --env NEW_VAR=new_value
Rename a Stack zenml stack rename < OLD_NAM E > < NEW_NAM E >
Example: zenml stack rename staging pre-production
Copy a Stack Duplicate an existing stack:
zenml stack copy < SOURCE_STAC K > < NEW_STACK_NAM E >
Example: zenml stack copy production production-backup
Delete a Stack zenml stack delete < STACK_NAM E >
Example: zenml stack delete old-stack
You cannot delete the active stack. Switch to a different stack first.
Share a Stack Make a stack available to all users:
zenml stack share < STACK_NAM E >
Stack Components Each component type has its own CLI command group for management.
Register a Component General syntax:
zenml < component-typ e > register < NAM E > --flavor < FLAVO R > [OPTIONS]
Examples: # Register S3 artifact store zenml artifact-store register s3_store \ --flavor s3 \ --path s3://my-bucket/artifacts # Register Kubernetes orchestrator zenml orchestrator register k8s_orchestrator \ --flavor kubernetes \ --kubernetes_context my-cluster # Register MLflow experiment tracker zenml experiment-tracker register mlflow_tracker \ --flavor mlflow \ --tracking_uri https://mlflow.example.com # Register component with labels zenml artifact-store register prod_store \ --flavor s3 \ --path s3://prod-bucket/artifacts \ -l environment=production \ -l team=ml-platform
List Components zenml < component-typ e > list [OPTIONS]
Examples: # List all artifact stores zenml artifact-store list # List orchestrators zenml orchestrator list # List with filters zenml artifact-store list --flavor s3 zenml orchestrator list --name "contains:kubernetes"
Describe a Component zenml < component-typ e > describe < NAM E >
Example: zenml artifact-store describe s3_store
Update a Component zenml < component-typ e > update < NAM E > [OPTIONS]
Example: zenml artifact-store update s3_store --path s3://new-bucket/artifacts
Delete a Component zenml < component-typ e > delete < NAM E >
Example: zenml orchestrator delete old_orchestrator
You cannot delete a component that’s currently used by any stack.
Available Component Types Use these command names to manage components:
Command Component Type artifact-storeArtifact storage orchestratorPipeline orchestration container-registryDocker image storage model-registryML model registry step-operatorSpecial step execution feature-storeFeature management model-deployerModel serving experiment-trackerExperiment tracking alerterAlerting and notifications annotatorData annotation data-validatorData quality checks image-builderImage building deployerInfrastructure deployment log-storeLog aggregation
Component Flavors List Available Flavors zenml < component-typ e > flavor list
Example: zenml orchestrator flavor list
Output:
┏━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓ ┃ Name │ Integration │ Service Connector ┃ ┠─────────────────┼───────────────┼────────────────────┨ ┃ local │ built-in │ - ┃ ┃ kubernetes │ kubernetes │ kubernetes ┃ ┃ kubeflow │ kubeflow │ kubernetes ┃ ┃ airflow │ airflow │ - ┃ ┃ sagemaker │ aws │ aws ┃ ┗━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛
Describe a Flavor zenml < component-typ e > flavor describe < FLAVOR_NAM E >
Example: zenml orchestrator flavor describe kubernetes
Shows configuration options, requirements, and usage examples.
Service Connectors Service connectors provide authentication to cloud resources.
Connect Component to Service Connector zenml < component-typ e > connect < COMPONENT_NAM E > -c < CONNECTOR_NAM E >
Example: zenml artifact-store connect s3_store -c aws_connector
Disconnect Component zenml < component-typ e > disconnect < COMPONENT_NAM E >
Stack Validation Check if a stack is properly configured:
zenml stack describe < STACK_NAM E >
The describe command validates:
All components are accessible
Required integrations are installed
Configuration is valid
Authentication works
Stack Export and Import Export Stack Configuration zenml stack export < STACK_NAM E > --filename stack.yaml
Import Stack Configuration zenml stack import < FILENAM E >
Example workflow: # On machine A: export stack zenml stack export production --filename prod_stack.yaml # Transfer file to machine B # On machine B: import stack zenml stack import prod_stack.yaml
Environment Variables Attach environment variables to stacks for runtime configuration:
# During registration zenml stack register my-stack \ -a local_artifact_store \ -o local_orchestrator \ --env AWS_REGION=us-west-2 \ --env LOG_LEVEL=DEBUG # Update existing stack zenml stack update my-stack --env NEW_VAR=value
These variables are available to all pipeline steps running on the stack.
Best Practices Stack Naming Use descriptive names that indicate environment and purpose:
zenml stack register prod-training-gpu zenml stack register staging-inference zenml stack register dev-local
Component Organization Register reusable components once, use in multiple stacks:
# Register components zenml artifact-store register prod_s3 --flavor s3 --path s3://prod-bucket zenml orchestrator register k8s_prod --flavor kubernetes zenml orchestrator register k8s_staging --flavor kubernetes # Use in different stacks zenml stack register prod -a prod_s3 -o k8s_prod zenml stack register staging -a prod_s3 -o k8s_staging
Stack Validation Always validate after registration:
zenml stack register new-stack -a store -o orch zenml stack describe new-stack
Development vs Production Maintain separate stacks for different environments:
# Local development zenml stack register dev -a local_store -o local_orch # Staging zenml stack register staging -a s3_staging -o k8s_staging # Production zenml stack register prod -a s3_prod -o k8s_prod
Troubleshooting Component Not Found Error: Orchestrator 'my_orchestrator' not found.
Solution: Register the component first:zenml orchestrator list # Check available orchestrators zenml orchestrator register my_orchestrator --flavor kubernetes
Integration Not Installed Error: The 'kubernetes' integration is not installed.
Solution: Install required integration:zenml integration install kubernetes
Stack Already Exists Error: Stack 'my-stack' already exists.
Solution: Use a different name or update existing stack:zenml stack update my-stack -o new_orchestrator
Next Steps
Install Integrations Add integrations for cloud providers and orchestrators
Run Pipelines Execute pipelines on your configured stack
Service Connectors Learn about cloud authentication
Stack Components Guide Detailed documentation for each component type
