Flyte supports multi-tenancy through two concepts: projects and domains. Together, they provide a namespace for all Flyte resources — tasks, workflows, launch plans, and executions.
Projects
A project is a logical namespace that groups related workflows, tasks, and resources together. Think of a project as a team or application boundary:
recommendation-enginefraud-detectiondata-ingestion
Multiple users and teams can share the same Flyte cluster while working in separate projects. Each project has its own isolated view of tasks, workflows, and executions.
Domains
A domain represents an execution environment within a project. Flyte ships with three default domains:
| Domain | Purpose |
|---|
development | For iterating on workflows during development. Lower resource quotas, less strict access controls. |
staging | For pre-production validation and integration testing. Mirrors production configuration. |
production | For live, mission-critical workloads. Full resource quotas, stricter controls. |
production domain instead of development.
How they work together
Every Flyte resource — task, workflow, launch plan, execution — is identified by a four-part key:
{project} / {domain} / {name} / {version}
recommendation-engine / development / train_wf / abc123
recommendation-engine / production / train_wf / abc123
abc123) can exist in both development and production within the same project. Executions in each domain are fully isolated from each other.
Using projects and domains with pyflyte
When registering workflows to a cluster, specify the project and domain with flags:
# Register to the development domain
pyflyte register --project recommendation-engine --domain development workflows/
# Register to the production domain
pyflyte register --project recommendation-engine --domain production workflows/
pyflyte run --remote \
--project recommendation-engine \
--domain development \
workflow.py train_wf
Using projects and domains with FlyteRemote
Set the default project and domain when creating a FlyteRemote object:
from flytekit.configuration import Config
from flytekit.remote import FlyteRemote
# Target the development domain
remote_dev = FlyteRemote(
config=Config.auto(),
default_project="recommendation-engine",
default_domain="development",
)
# Target the production domain
remote_prod = FlyteRemote(
config=Config.auto(),
default_project="recommendation-engine",
default_domain="production",
)
from workflows.training import train_wf
# Run in development
execution = remote_dev.execute(train_wf, inputs={"learning_rate": 0.01})
# Promote to production — same workflow code, different domain
execution = remote_prod.execute(train_wf, inputs={"learning_rate": 0.01})
Resource isolation
Domains can be configured with different resource quotas, Kubernetes namespaces, and access controls. This means:
- A runaway workflow in
development cannot consume resources allocated to production.
- Service accounts and IAM roles can differ between domains, giving
production access to production data sources while development uses sandboxed data.
- Notifications and alerting can be scoped to
production only, reducing noise from development runs.
Managing projects with flytectl
Create a new project:
flytectl create project \
--id recommendation-engine \
--name "Recommendation Engine" \
--description "ML pipelines for the recommendation system"
flytectl update project --id recommendation-engine --archive
Dev-to-production workflow
A typical promotion flow in Flyte:
Develop in the development domain
Register and run workflows with --domain development. Iterate quickly, use smaller datasets, and lower resource requests.pyflyte register --project my-project --domain development workflows/
pyflyte run --remote --project my-project --domain development \
workflows/train.py train_wf
Validate in the staging domain
Once the workflow looks correct, register it to staging for integration testing against production-like data and infrastructure.pyflyte register --project my-project --domain staging workflows/
Promote to production
Register the same code to production. Activate the scheduled launch plan to start running on the production cadence.pyflyte register --project my-project --domain production workflows/
flytectl update launchplan -p my-project -d production \
nightly_train_lp --version <VERSION> --activate
The workflow code does not need to change between domains. Domain promotion is purely a matter of which --domain flag you pass to pyflyte register or which default_domain you set in FlyteRemote.
