Skip to main content
The K8s Scheduler implements a three-tier secrets management system using HashiCorp Vault:
  1. Organization Secrets - Shared across all deployments in an organization
  2. Deployment Secrets - Scoped to a specific deployment
  3. Service Secrets - Scoped to a specific service within a deployment
Secrets are stored in Vault and automatically injected into deployments via the init-container pattern.

​
Deployment-Scoped Secrets

​
List Secrets

GET /api/secrets
List all secrets for the authenticated user, optionally filtered by deployment. Authentication: Required (session or API key with secrets:read scope) Query Parameters:
​
deployment
string
Filter by deployment name
Response:
​
id
string
Secret UUID
​
key
string
Secret key (environment variable name)
​
deploymentName
string
Deployment name (empty for user-wide secrets)
​
serviceName
string
Service name (empty for deployment-wide secrets)
​
description
string
Human-readable description
Example: 
curl -X GET "https://your-domain.com/api/secrets?deployment=abc123xyz7" \
  -H "Authorization: Bearer YOUR_API_KEY"

[
  {
    "id": "secret_abc123",
    "key": "DATABASE_PASSWORD",
    "deploymentName": "abc123xyz7",
    "serviceName": "postgres",
    "description": "PostgreSQL admin password"
  },
  {
    "id": "secret_def456",
    "key": "API_KEY",
    "deploymentName": "abc123xyz7",
    "serviceName": "",
    "description": "External API key"
  }
]
Secret values are never returned by the API for security reasons. Only keys and metadata are returned.

​
Create Secret

POST /api/secrets
Create a new secret at deployment or service scope. Authentication: Required (secrets:write scope) Request Body:
​
key
string
required
Secret key (must be valid environment variable name: [A-Za-z_][A-Za-z0-9_]*)
​
value
string
required
Secret value (stored encrypted in Vault)
​
deploymentName
string
Deployment name for deployment-scoped secret. Can use template: prefix for template-scoped secrets (e.g., template:nginx)
​
serviceName
string
Service name for service-scoped secret
​
description
string
Human-readable description
Response: 201 Created
​
id
string
Secret UUID
​
key
string
Secret key
​
deploymentName
string
Deployment name
​
serviceName
string
Service name
​
description
string
Description
Example: 
curl -X POST "https://your-domain.com/api/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "DATABASE_PASSWORD",
    "value": "super-secret-password",
    "deploymentName": "abc123xyz7",
    "serviceName": "postgres",
    "description": "PostgreSQL admin password"
  }'
Validation:
  • Key must match pattern: ^[A-Za-z_][A-Za-z0-9_]*$
  • Value cannot be empty
  • If deploymentName is specified, user must own the deployment
  • If deploymentName starts with template:, the template must exist

​
Organization Secrets

Organization secrets are shared across all deployments within an organization. Managing org secrets requires org_owner or org_admin role.

​
List Organization Secrets

GET /api/org/secrets
List all secrets for the current organization (determined by active team context). Authentication: Required (must be org owner or admin) Response:
​
id
string
Secret UUID
​
key
string
Secret key
​
description
string
Description
Example: 
curl -X GET "https://your-domain.com/api/org/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY"

[
  {
    "id": "secret_org_123",
    "key": "STRIPE_API_KEY",
    "description": "Stripe payment processing key"
  },
  {
    "id": "secret_org_456",
    "key": "SENDGRID_API_KEY",
    "description": "SendGrid email API key"
  }
]

​
Create Organization Secret

POST /api/org/secrets
Create a new organization-wide secret. Authentication: Required (org owner or admin) Request Body:
​
key
string
required
Secret key (must be valid environment variable name)
​
value
string
required
Secret value
​
description
string
Description
Response: 201 Created Example: 
curl -X POST "https://your-domain.com/api/org/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "STRIPE_API_KEY",
    "value": "sk_live_abc123xyz",
    "description": "Stripe payment processing key"
  }'

​
Update Organization Secret

PUT /api/org/secrets
Update an existing organization secret’s value and/or description. Authentication: Required (org owner or admin) Request Body:
​
id
string
required
Secret UUID
​
value
string
required
New secret value
​
description
string
New description
Response: 200 OK 
{
  "status": "updated"
}
Example: 
curl -X PUT "https://your-domain.com/api/org/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "secret_org_123",
    "value": "sk_live_new_key_789",
    "description": "Updated Stripe key"
  }'

​
Delete Organization Secret

DELETE /api/org/secrets
Delete an organization secret. Authentication: Required (org owner or admin) Request Body:
​
id
string
required
Secret UUID
Response: 200 OK 
{
  "status": "deleted"
}
Example: 
curl -X DELETE "https://your-domain.com/api/org/secrets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"id": "secret_org_123"}'

​
Secrets Hierarchy

Secrets are loaded in the following order (later values override earlier ones):
  1. Organization Secrets - Applied to all deployments in the org
  2. Deployment Secrets - Applied to all services in the deployment
  3. Service Secrets - Applied only to the specific service
This allows for flexible secret management:
  • Shared credentials (database URLs, API keys) at org level
  • Deployment-specific config at deployment level
  • Service-specific secrets at service level

​
Example Hierarchy

# Organization level (all deployments)
STRIPE_API_KEY=sk_live_abc123
SENDGRID_API_KEY=SG.xyz789

# Deployment level (all services in deployment abc123xyz7)
DATABASE_URL=postgres://host/db
REDIS_URL=redis://host:6379

# Service level (only postgres service)
POSTGRES_PASSWORD=secret123
POSTGRES_USER=admin

​
Vault Storage Paths

Secrets are stored in Vault using the following path structure: 
# Organization secrets
orgs/{orgId}/secrets/{key}

# Deployment secrets
deployments/{deploymentName}/secrets/{key}

# Service secrets
deployments/{deploymentName}/services/{serviceName}/secrets/{key}

# Template secrets (copied to deployments on creation)
templates/{templateName}/secrets/{key}
templates/{templateName}/services/{serviceName}/secrets/{key}

​
Secret Injection

Secrets are injected into deployments using an init container pattern:
  1. Init Container - Fetches secrets from Vault using deployment’s service account token
  2. Shared Volume - Writes secrets to a shared emptyDir volume
  3. Main Container - Reads secrets as environment variables on startup
This approach:
  • Avoids storing secrets in Kubernetes Secrets
  • Provides audit trail in Vault
  • Supports dynamic secret rotation
  • Works with any Vault authentication method

​
Template Secrets

Templates can define required secrets, which are then copied to each deployment: 
services:
  - name: postgres
    image: postgres:15
    secrets:
      - key: POSTGRES_PASSWORD
        description: "Database admin password"
      - key: POSTGRES_USER
        description: "Database admin user"
When creating a deployment from this template:
  1. Template secrets are copied to deployments/{deploymentName}/secrets/
  2. Service-specific secrets are copied to deployments/{deploymentName}/services/{serviceName}/secrets/
  3. Original template secrets remain unchanged
This allows:
  • Templates to define required secrets
  • Each deployment to have unique secret values
  • Secrets to be updated per-deployment without affecting other deployments

​
Security Considerations

​
Authentication

  • Deployments authenticate to Vault using Kubernetes service account tokens
  • The operator configures Kubernetes auth backend in Vault
  • Each deployment has a unique service account with minimal permissions

​
Authorization

  • Organization secrets require org_owner or org_admin role
  • Deployment secrets require deployment ownership
  • Service secrets require deployment ownership
  • Template secrets require template ownership

​
Encryption

  • All secrets are encrypted at rest in Vault
  • Secrets are transmitted over TLS
  • Init container runs as non-root user
  • Secret files have 0600 permissions

​
Audit

  • All secret operations are logged to the audit log
  • Vault maintains its own audit log
  • Secret access is tracked per-deployment

​
Backend Configuration

The secrets store backend is configured at server startup:

​
Vault Backend (Production)

vault:
  address: https://vault.example.com
  token: ${VAULT_TOKEN}
  namespace: "shipyard"

​
In-Memory Backend (Development)

secrets:
  backend: memory
The in-memory backend is for development only and does not persist secrets across restarts.

​
Error Responses

​
error
string
Human-readable error message

​
Common Error Codes

  • 400 Bad Request - Invalid key format or missing required field
  • 403 Forbidden - Deployment not found, insufficient permissions, or org admin required
  • 404 Not Found - Secret not found
  • 409 Conflict - Secret with this key already exists at this scope
  • 500 Internal Server Error - Vault communication error
  • 503 Service Unavailable - Secrets management not configured

​
Key Validation

Secret keys must match the environment variable name pattern:
  • Start with letter or underscore: [A-Za-z_]
  • Followed by letters, numbers, or underscores: [A-Za-z0-9_]*
Examples:
  • ✅ DATABASE_URL
  • ✅ API_KEY
  • ✅ _PRIVATE_VAR
  • ❌ 1PASSWORD (starts with number)
  • ❌ MY-SECRET (contains hyphen)
  • ❌ my.secret (contains dot)

Build docs developers (and LLMs) love

Get started for freeTalk to us