Development Guide - Tekton Pipelines

This guide walks you through setting up a development environment for Tekton Pipelines and covers common development workflows.

Prerequisites

Before starting, you should familiarize yourself with:

Kubernetes and Custom Resource Definitions (CRDs)
Controller Logic - How Tekton extends Kubernetes
Tekton Pipelines documentation
Getting Started tutorial - Define and run Tasks and Pipelines

Setting Up Development Environment

1. Setup GitHub Account

GitHub is used for source code management using SSH for authentication.

Create GitHub account

Create a GitHub account if you don’t already have one.

Setup SSH access

Setup GitHub access via SSH.

2. Install Required Tools

Install git

Install git for source control.

Install pre-commit

Install pre-commit to run git hooks locally:

# After installing, run at the root directory to install git hooks
pre-commit install
# Run the hooks against all files
pre-commit run --all-files

Install Go

Install go - Tekton is built in Go.

Go version v1.15 or higher is recommended.

Install ko

Install ko to build and deploy container images.

ko version v0.5.1 or higher is required.

Install kubectl

Install kubectl to interact with Kubernetes.

The user interacting with your K8s cluster must be a cluster admin to create role bindings.

Google Cloud Platform example:

# Using gcloud to get your current user
USER=$(gcloud config get-value core/account)
# Make that user a cluster admin
kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user="${USER}"

Install bash

Install bash v4 or higher for build scripts.On MacOS, use Homebrew to install a newer version.

Install go-licenses

Install go-licenses - used in e2e tests.

3. Install Optional Tools

yamllint

yamllint is run against every PR as part of pre-commit. Install it so pre-commit can use it.

golangci-lint

golangci-lint is run against every PR. Install and run it locally to iterate quickly on linter issues.

Linter findings depend on your Go version. Match the version in go.mod to match PR findings.

woke

woke checks for offensive language in every PR. Install to run checks locally.

delve

delve is needed for debugging the Tekton controller in VSCode or your IDE.

4. Configure Environment

Set these environment variables to build, deploy, and run Tekton with ko:

Set GOROOT (optional)

Set GOROOT to the Go installation location you want ko to use:

export GOROOT=/path/to/go

Only needed if you installed Go to a non-default location or have multiple Go versions.

Set KO_DOCKER_REPO

Set the docker repository for pushing developer images:Using Google Container Registry (GCR):

# format: gcr.io/${GCP-PROJECT-NAME}
export KO_DOCKER_REPO='gcr.io/my-gcloud-project-name'

Using Docker Desktop (Docker Hub):

# format: docker.io/${DOCKER_HUB_USERNAME}
export KO_DOCKER_REPO='docker.io/my-dockerhub-username'

Using a self-hosted Docker Registry:

# format: ${localhost:port}/{}
export KO_DOCKER_REPO='localhost:5000/mypipelineimages'

Add Go binaries to PATH (optional)

Add $HOME/go/bin to your system PATH:

export PATH="${PATH}:$HOME/go/bin"

Add these environment variables to your shell’s configuration files (e.g., ~/.bash_profile or ~/.bashrc).

5. Setup a Fork

Create a fork

Create a fork of the tektoncd/pipeline repository in your GitHub account.

Clone your fork

git clone [email protected]:${YOUR_GITHUB_USERNAME}/pipeline.git

Tekton uses Go Modules, so you can clone to any location.

Configure remotes

cd pipeline

# Add Tekton as upstream
git remote add upstream [email protected]:tektoncd/pipeline.git

# Prevent accidental pushing to upstream
git remote set-url --push upstream no_push

# Configure your fork as origin
git remote add origin [email protected]:${YOUR_GITHUB_USERNAME}/pipeline.git

6. Configure Container Registry

Docker Desktop
Google Container Registry

Docker Desktop provides seamless integration with Docker Hub. Configure Docker Desktop with your Docker ID and password in its dashboard.

Configure authentication for your KO_DOCKER_REPO:

gcloud auth configure-docker

To pull images from gcr.io/<project>, configure IAM policies following the GCR access control instructions.If running GKE and GCR in the same project, add storage-full to the --scopes when creating your cluster, or grant read access:

gcloud projects add-iam-policy-binding <project-number> \
  --member='serviceAccount:<project-number>[email protected]' \
  --role='roles/storage.objectViewer'

Setup a Kubernetes Cluster

Recommended Minimum Configuration

Kubernetes version 1.28 or later
4 (virtual) CPU nodes
8 GB of platform memory
Node autoscaling, up to 3 nodes

Kind
Minikube
Docker Desktop
GKE

Kind is great for testing locally.

Install Docker

Install Docker.

Create cluster

kind create cluster

Configure ko

export KO_DOCKER_REPO="kind.local"
export KIND_CLUSTER_NAME="kind"  # only if you used a custom name

The Tekton plumbing project provides a ‘tekton_in_kind.sh’ script that creates a cluster with Tekton components installed.

Setup GCP Project

Set up a GCP Project and enable the GKE API.

Create GKE cluster

export PROJECT_ID=my-gcp-project
export CLUSTER_NAME=mycoolcluster

gcloud container clusters create $CLUSTER_NAME \
  --enable-autoscaling \
  --min-nodes=1 \
  --max-nodes=3 \
  --scopes=cloud-platform \
  --no-issue-client-certificate \
  --project=$PROJECT_ID \
  --region=us-central1 \
  --machine-type=e2-standard-4 \
  --num-nodes=1 \
  --cluster-version=1.28

Recommended machine type: e2-standard-4

Grant cluster-admin permissions

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user=$(gcloud config get-value core/account)

Building and Deploying

Install Pipeline

Deploy Tekton using your local code:

ko apply -R -f config/

Verify Installation

Check that Tekton pipeline pods are running:

kubectl get pods -n tekton-pipelines

Redeploy Controller

As you make code changes, redeploy your controller:

ko apply -f config/controller.yaml

Delete Pipeline

Clean up everything:

# Without deleting the namespace
ko delete -f config/

# Including the namespace
ko delete -R -f config/

If using the same namespace as other components (dashboard, triggers), ko delete -R -f config/ deletes those components too.

Development Workflows

Iterating on Code Changes

While developing:

Make your code changes
Run update scripts as needed:
- ./hack/update-deps.sh - Update dependencies
- ./hack/update-codegen.sh - Update type definitions
- ./hack/update-openapigen.sh - Update OpenAPI specs
Redeploy the controller
Verify the installation and check logs

Accessing Logs

Controller logs:

kubectl -n tekton-pipelines logs $(kubectl -n tekton-pipelines get pods -l app=tekton-pipelines-controller -o name)

Webhook logs:

kubectl -n tekton-pipelines logs $(kubectl -n tekton-pipelines get pods -l app=tekton-pipelines-webhook -o name)

TaskRun/PipelineRun logs: See docs on accessing logs.

Testing

For comprehensive testing documentation, see the Testing Guide.

Adding New CRD Types

If you need to add a new CRD type:

Add YAML definition

Add a yaml definition in config/

Update cluster roles

Add the type to cluster roles in:

config/200-clusterrole.yaml
config/clusterrole-aggregate-edit.yaml
config/clusterrole-aggregate-view.yaml

Add Go structs

Add go structs in pkg/apis/pipeline/v1alpha1 implementing:

Defaultable interface
Validatable interface

Add to known types

Add the new type to the list of known types in pkg/apis/pipeline/v1alpha1/register.go

See the API compatibility policy for more information.

Debugging

ko has built-in support for the delve debugger.

Setup Debugging

Comment out probes

Update config/controller.yaml and comment out the liveness and readiness probes to prevent timeouts.

Build in debug mode

ko apply -f config/controller.yaml --debug --disable-optimizations

Forward debugging port

kubectl port-forward -n tekton-pipelines deployments/tekton-pipelines-controller 40000:40000

Add VSCode configuration

Add to launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Attach to Delve (Tekton Controller)",
      "type": "go",
      "request": "attach",
      "mode": "remote",
      "port": 40000,
      "host": "127.0.0.1",
      "apiVersion": 2,
      "substitutePath": [
        {
          "from": "${workspaceFolder}",
          "to": "github.com/tektoncd/pipeline"
        }
      ]
    }
  ]
}

Now you can attach to delve in VSCode, set breakpoints, and debug PipelineRun execution.

Development

​Prerequisites

​Setting Up Development Environment

​1. Setup GitHub Account

​2. Install Required Tools

​3. Install Optional Tools

​4. Configure Environment

​5. Setup a Fork

​6. Configure Container Registry

​Setup a Kubernetes Cluster

​Recommended Minimum Configuration

​Building and Deploying

​Install Pipeline

​Verify Installation

​Redeploy Controller

​Delete Pipeline

​Development Workflows

​Iterating on Code Changes

​Accessing Logs

​Testing

​Adding New CRD Types

​Debugging

​Setup Debugging

Build docs developers (and LLMs) love

Prerequisites

Setting Up Development Environment

1. Setup GitHub Account

2. Install Required Tools

3. Install Optional Tools

4. Configure Environment

5. Setup a Fork

6. Configure Container Registry

Setup a Kubernetes Cluster

Recommended Minimum Configuration

Building and Deploying

Install Pipeline

Verify Installation

Redeploy Controller

Delete Pipeline

Development Workflows

Iterating on Code Changes

Accessing Logs

Testing

Adding New CRD Types

Debugging

Setup Debugging