Skip to main content
Manage Metaflow execution environments for reproducible dependency management. The metaflow environment command allows you to resolve, create, inspect, and share Conda-based environments.

Synopsis

metaflow environment [OPTIONS] COMMAND [ARGS]...

Global Options

--quiet
flag
default:"false"
Suppress unnecessary messages
--metadata
choice
default:"service"
Metadata service type. Available options: local, service
--datastore
choice
default:"s3"
Datastore type. Available options: s3, local
--environment
choice
default:"conda"
Type of environment to manage. Currently only conda is supported
--conda-root
string
Root path for Conda cached information. If not set, looks for METAFLOW_CONDA_S3ROOT (for S3)

Commands

metaflow environment resolve

Resolve an environment from dependency specifications.
metaflow environment resolve [OPTIONS]

Options

--alias
string
Alias the resolved environment. Can be specified multiple times. Format: name or name:tag
--force
flag
default:"false"
Force resolution of already resolved environments
--local-only
flag
default:"false"
Only look locally for using environments
--dry-run
flag
default:"false"
Dry-run — only resolve, do not download, cache, persist or alias anything
--set-default
flag
default:"true"
Set the resolved environment as the default environment for this set of requirements
--arch
string
Architecture to resolve for. Can be specified multiple times. If not specified, defaults to current machine architecture
--python
string
Python version for the environment. Defaults to current version. Example: ">=3.8,<3.9"
-r, --requirement
path
Get the environment definition from a requirements.txt file (pip format)
-f, --file
path
Get the environment definition from a environment.yml file (conda-lock format)
-p, --pyproject
path
Get the environment definition from a pyproject.toml file (poetry format)
--using-pathspec
string
Environment created starting with the environment for the Pathspec specified
--using
string
Environment created starting with the environment referenced here
--from-pathspec
string
Environment created cloning the requirements in the Pathspec specified
--from
string
Environment created cloning the requirements in the environment referenced here

Examples

Resolve an environment from a requirements.txt file:
metaflow environment resolve --python ">=3.8,<3.9" -r req_numpy.txt
Resolve an environment from an environment.yml file:
metaflow environment resolve --python ">=3.8,<3.9" -f env_numpy.yml
Resolve and alias an environment:
metaflow environment resolve --python ">=3.8,<3.9" --alias mlp/metaflow/conda_example/numpy_test_env -f env_numpy.yml

metaflow environment create

Create a local environment based on an environment specification.
metaflow environment create [OPTIONS] ENV-NAME

Arguments

env-name
string
required
Environment identifier. Can be:
  • A pathspec: <flowname>/<runid>/<stepname>
  • A partial hash: <req id> (assumes default environment)
  • A full hash: <req id>:<full id> or <full-id>
  • An alias name: as added using --alias when resolving

Options

--name
string
Name of the environment to create
--local-only
flag
default:"false"
Only create if environment is known locally
--force
flag
default:"false"
Recreate the environment if it already exists and remove the --into-dir directory if it exists
--strict
flag
default:"true"
If True, fails if it cannot install the original Metaflow environment
--into-dir
path
If the env-name refers to a Metaflow executed Step, downloads the step’s code package into this directory
--install-notebook
flag
default:"false"
Install the created environment as a Jupyter kernel. Requires --name
--pathspec
flag
default:"false"
The environment name given is a pathspec

Examples

Create an environment with a Jupyter kernel:
metaflow environment create --name testenv --install-notebook --force --into-dir testdir --pathspec CondaTestFlow/118/start
Use the environment in the shell:
cd testdir && /usr/local/libexec/metaflow-condav2-20230809/envs/testenv/bin/python -c "from mymodule import hello; hello(); import pandas as pd; print(pd.__version__)"

metaflow environment get

Locally fetch an environment already resolved and present remotely.
metaflow environment get [OPTIONS] SOURCE_ENV

Arguments

source_env
string
required
Environment identifier (pathspec, hash, or alias name)

Options

--default
flag
default:"true"
Set the downloaded environment as default for its requirement ID
--arch
string
Request this architecture. Defaults to current architecture if not specified
--pathspec
flag
default:"false"
The environment name given is a pathspec

Examples

Fetch a named environment:
metaflow environment get mlp/metaflow/conda_example/numpy_test_env

metaflow environment show

Show information about an environment.
metaflow environment show [OPTIONS] ENVS...

Arguments

envs
string
required
One or more environment identifiers (pathspec, hash, or alias name)

Options

--local-only
flag
default:"false"
Only resolve source env using local information
--arch
string
default:"current"
Show environment for this architecture
--pathspec
flag
default:"false"
The environments given are pathspecs

Examples

Show environment from a pathspec:
metaflow environment show --pathspec CondaTestFlow/118/start

metaflow environment alias

Alias an existing environment.
metaflow environment alias [OPTIONS] SOURCE_ENV ALIAS

Arguments

source_env
string
required
Source environment identifier
alias
string
required
Alias name to assign. Format: name or name:tag

Options

--local-only
flag
default:"false"
Only resolve source env using local information
--pathspec
flag
default:"false"
The source environment given is a pathspec

Examples

Alias an environment:
metaflow environment alias --pathspec CondaTestFlow/118/start myproject/myenv:v1

See Also

Build docs developers (and LLMs) love