After you have developed and tested your agent locally, you need to deploy it to a Kubernetes cluster. This page covers deploying agents to the Flyte sandbox for integration testing and to a production cluster.
Deploying to the Flyte sandbox
The Flyte sandbox (flytectl demo start) is the easiest way to test an agent end-to-end before moving to a production cluster.
Build your agent image
Create a Dockerfile for your agent. The following example is for the Databricks agent:FROM python:3.9-slim-bookworm
RUN apt-get update && apt-get install build-essential git -y
RUN pip install prometheus-client grpcio-health-checking
RUN pip install --no-cache-dir -U flytekit \\
git+https://github.com/flyteorg/flytekit.git@<gitsha>#subdirectory=plugins/flytekit-spark \\
&& apt-get clean autoclean \\
&& apt-get autoremove --yes \\
&& rm -rf /var/lib/{apt,dpkg,cache,log}/
CMD pyflyte serve agent --port 8000
docker buildx build -t localhost:30000/flyteagent:example -f Dockerfile.agent . --load
docker push localhost:30000/flyteagent:example
Deploy the agent image to the cluster
Edit the flyteagent deployment to use your new image:kubectl edit deployment flyteagent -n flyte
image key and update its value:image: localhost:30000/flyteagent:example
Configure secrets
Store agent credentials as Kubernetes secrets. The agent deployment mounts the flyteagent secret in the flyte namespace. Use the following command to add a secret value (the Databricks token is used here as an example):SECRET_VALUE=$(echo -n "<DATABRICKS_TOKEN>" | base64) && \\
kubectl patch secret flyteagent -n flyte \\
--patch "{\"data\":{\"flyte_databricks_access_token\":\"$SECRET_VALUE\"}}"
Secret names must contain only lowercase English letters. Secret values must be Base64-encoded.
Restart the agent deployment
Apply the new image and secrets by restarting the deployment:kubectl rollout restart deployment flyteagent -n flyte
Test your agent remotely
Run a workflow that uses the agent:pyflyte run --remote agent_workflow.py agent_task
You must build an image that includes the plugin package for the task and specify it with the --image flag or via an ImageSpec definition in your workflow file.
Managing secrets
All agent secrets are stored in the flyteagent Kubernetes secret in the flyte namespace. To add a secret for any agent:
SECRET_VALUE=$(<YOUR_SECRET_VALUE> | base64) && \
kubectl patch secret flyteagent -n flyte \
--patch "{\"data\":{\"your_agent_secret_name\":\"$SECRET_VALUE\"}}"
Always Base64-encode secret values before patching. Storing plaintext secrets in the Kubernetes secret object will cause authentication failures.
Deploying to a production cluster
For production deployments, update the flyteagent deployment’s image directly:
kubectl set image deployment/flyteagent flyteagent=ghcr.io/flyteorg/flyteagent:latest
Ensure your FlytePropeller image version is >= 1.13.0. Propeller 1.13.0+ automatically fetches the list of supported task types from the agent deployment, removing the need to manually specify task types in the configmap.
Configuring FlytePropeller to use the agent service
FlytePropeller needs to know the endpoint of the agent service. Configure this in the propeller configmap:
plugins:
agent-service:
defaultAgent:
endpoint: "k8s://flyteagent.flyte:8000"
insecure: true
timeouts:
# CreateTask, GetTask, DeleteTask are for async agents
CreateTask: 5s
GetTask: 5s
DeleteTask: 5s
# ExecuteTaskSync is for sync agents
ExecuteTaskSync: 10s
defaultTimeout: 10s
Routing task types to custom agent services
If you have multiple agent services (for example, a dedicated service for a custom integration), configure routing rules:
plugins:
agent-service:
defaultAgent:
endpoint: "k8s://flyteagent.flyte:8000"
insecure: true
defaultTimeout: 10s
agents:
custom_agent:
endpoint: "dns:///custom-flyteagent.flyte.svc.cluster.local:8000"
insecure: false
defaultServiceConfig: '{"loadBalancingConfig": [{"round_robin":{}}]}'
timeouts:
GetTask: 5s
defaultTimeout: 10s
agentForTaskTypes:
# Requests for custom_task go to custom_agent; all other types go to defaultAgent
- custom_task: custom_agent
Enabling agents in a managed deployment
If you are using a managed Flyte deployment, you cannot modify the agent configuration directly. Contact your deployment administrator to:
- Enable the agent plugin package in the agent service image.
- Add any required secrets to the
flyteagent Kubernetes secret.
- Restart the agent deployment.
For per-agent configuration guides, see the Agent setup documentation.
Verifying the deployment
After deploying, verify the agent is running and reachable:
# Check that the agent pod is running
kubectl get pods -n flyte -l app=flyteagent
# Tail agent logs for debugging
kubectl logs -n flyte -l app=flyteagent -f
# Check the rollout status
kubectl rollout status deployment/flyteagent -n flyte
Enable DEBUG logging in the agent service to trace gRPC calls from FlytePropeller. Set the FLYTE_INTERNAL_LOG_LEVEL=DEBUG environment variable in the agent deployment spec.
