Skip to main content
Flyte is a workflow orchestrator that unifies machine learning, data engineering, and data analytics stacks for building robust and reliable applications. It is built for scalability and reproducibility, leveraging Kubernetes as its underlying platform. With Flyte, teams write pipelines in Python using the flytekit SDK and deploy them on cloud or on-premises environments, enabling distributed processing and efficient resource utilization. Organizations like LinkedIn, Spotify, Freenome, Pachama, and Warner Bros. use Flyte for mission-critical workloads.

Quickstart

Run your first Flyte workflow locally in under five minutes.

Key Concepts

Understand tasks, workflows, launch plans, and how Flyte executes pipelines.

User Guide

Dive into data types, advanced composition, and productionizing workflows.

Deployment

Deploy Flyte on AWS, GCP, Azure, or on-premises with Helm and Kubernetes.

​
Why Flyte?

Strongly typed

Every task input and output is type-annotated. Flyte validates data at workflow boundaries, catching bugs before they reach production.

Reproducible by design

Immutable executions, versioned tasks, and full data lineage mean any run can be reproduced exactly — including the inputs, outputs, and environment.

Kubernetes-native

Flyte runs on any Kubernetes cluster. Scale from a local sandbox to thousands of parallel tasks on AWS, GCP, or Azure without changing your code.

Multi-language

Write tasks in Python, Java, Scala, or any language via raw containers. The primary SDK is flytekit for Python.

Dynamic workflows

Build workflows that change shape at runtime based on computed values, branching logic, and conditional execution paths.

Plugin ecosystem

Native integrations with Spark, Ray, PyTorch, Kubeflow, Snowflake, and more through a first-class plugin system.

Data lineage

Track the movement and transformation of data throughout the full lifecycle of your data and ML workflows.

Map tasks

Achieve parallel code execution with minimal configuration using map tasks — no manual thread or process management.

Scheduling

Schedule workflows on cron or fixed-rate intervals using launch plans. Enable and disable schedules without redeploying.

​
Architecture overview

Flyte is made up of a user plane, a control plane, and a data plane.
  • The user plane contains the tools you use to develop and register workflows: flytekit (the Python SDK) and flytectl (the CLI). Data scientists and ML engineers primarily work in the user plane.
  • The control plane is the Flyte backend configured by platform engineers. It consists of FlyteConsole (the web UI) and FlyteAdmin, which serves as the main Flyte API. The control plane sends workflow execution requests to the data plane and stores current and historical execution state.
  • The data plane contains FlytePropeller, the core execution engine. FlytePropeller is a Kubernetes controller that runs workflows as Kubernetes Pods. It sends status events back to the control plane so information can be surfaced to end users.
ComponentRole
FlyteAdminControl plane. Stores workflow definitions, launches executions, exposes gRPC and REST API.
FlytePropellerExecution engine. Kubernetes operator that reconciles workflow states as a Custom Resource.
FlyteIDLProtobuf-based interface definition language. The contract between all Flyte components.
DataCatalogArtifact caching layer. Enables task output memoization across runs.
FlyteConsoleWeb UI for monitoring workflows, executions, and artifacts.
flytectlCLI for managing all Flyte resources and running local sandboxes.

​
Key features

Flyte supports memoization of task outputs to ensure that identical invocations of a task are not executed repeatedly, saving compute resources and time. Enable caching by passing cache=True and a cache_version to the @task decorator:
from flytekit import task

@task(cache=True, cache_version="1")
def compute_mean(data: list) -> float:
    return sum(data) / len(data)
Flyte provides built-in retry strategies for both user errors and system-level failures such as spot instance preemptions, network issues, and hardware failures. Configure retries at the task level:
@task(retries=3)
def my_task() -> None:
    ...
Express heterogeneous resource requirements at the task level. Flyte allocates CPU, memory, and GPU per task, enabling workflows composed of tasks with very different infrastructure needs:
from flytekit import Resources

@task(requests=Resources(cpu="16", mem="16Gi", gpu="1"))
def train_model() -> None:
    ...
Each Flyte task runs in its own container on a Kubernetes Pod, maintaining separate dependency sets so no conflicts arise between tasks in the same workflow.
Multiple users and teams share the same Flyte platform while maintaining their own distinct data and configurations through projects and domains.

​
Next steps

Quickstart

Install flytekit, write your first workflow, and run it — locally and on a cluster.

Key Concepts

Learn how tasks, workflows, and launch plans fit together in the Flyte programming model.

Build docs developers (and LLMs) love

Get started for freeTalk to us