Skip to main content
Learn how to update retention policies within a Halo CMMS deployment to automatically manage the lifecycle of measurements and exchanges.
This guide assumes familiarity with Kubernetes and an existing CMMS instance with a Kingdom deployed locally or through GKE.See the Local Kubernetes Configuration or Kingdom Deployment Guide for setup instructions.

​
Retention Policy CronJobs

Retention policies are enforced through configured CronJobs that are enabled by default on Kingdom deployments.

​
Viewing Configured CronJobs

List all retention policy CronJobs: 
kubectl get cronjobs
Example output: 
NAME                                      SCHEDULE    SUSPEND ACTIVE LAST SCHEDULE AGE
completed-measurements-deletion-cronjob   15 * * * *  False   0      52m           5h32m
exchanges-deletion-cronjob                40 6 * * *  False   0      10h27m        5h32m
pending-measurements-cancellation-cronjob 45 * * * *  False   0      22m           5h32m

​
Understanding the Schedule

The SCHEDULE column uses Cron syntax to specify execution frequency:
  • 15 * * * * - Runs hourly at 15 minutes past the hour
  • 40 6 * * * - Runs daily at 6:40 AM
  • 45 * * * * - Runs hourly at 45 minutes past the hour

​
Configuration Flags

Each CronJob has two primary configuration flags:
FlagPurposeValues
--dry-runControls whether operations are executed or just loggedtrue (log only) or false (execute)
--time-to-live / --days-to-liveHow long a resource must be in a state before the retention operation appliesDuration (e.g., 7d) or integer (days)
Dry Run ModeBy default, CronJobs may be in dry-run mode. When --dry-run=true, the job will only log operations that would be performed without actually executing them.Set --dry-run=false to enable actual retention operations.

​
Kingdom Retention CronJobs

The Kingdom deployment includes three retention policy CronJobs:
Purpose: Deletes measurements in terminal states after a specified time.Behavior: Measurements are deleted if:
  • They are in a terminal state (SUCCEEDED, FAILED, or CANCELLED)
  • The time since their last update exceeds the --time-to-live duration
Default Schedule: Hourly at 15 minutes past the hourExample Configuration:
args:
  - --dry-run=false
  - --time-to-live=30d
This configuration deletes completed measurements that haven’t been updated in 30 days.
Purpose: Deletes exchanges after a specified number of days.Behavior: Exchanges are deleted if:
  • The days since their exchange date exceeds --days-to-live
Default Schedule: Daily at 6:40 AMExample Configuration:
args:
  - --dry-run=false
  - --days-to-live=90
This configuration deletes exchanges older than 90 days from their exchange date.
Purpose: Cancels measurements that remain in non-terminal states too long.Behavior: Measurements are transitioned to CANCELLED state if:
  • They are in a non-terminal state (PENDING, RUNNING, etc.)
  • The time since their creation exceeds the --time-to-live duration
Default Schedule: Hourly at 45 minutes past the hourExample Configuration:
args:
  - --dry-run=false
  - --time-to-live=7d
This configuration cancels measurements that have been pending for more than 7 days.

​
Modifying Retention CronJobs

Important: Modifications to a CronJob only affect future Jobs. Currently running Jobs will not be updated.

​
Interactive Editing

Use kubectl edit to modify a CronJob interactively: 
kubectl edit cronjob exchanges-deletion-cronjob
This opens the CronJob configuration in your default text editor.

​
Configuration Paths

When editing, modify these paths in the CronJobSpec:
1

Update schedule

Path: spec.schedule
spec:
  schedule: "30 * * * *"  # Runs hourly at 30 minutes past
Use Cron syntax to specify the frequency.
2

Update retention flags

Path: spec.jobTemplate.spec.template.spec.containers[0].args
args:
  - --dry-run=false          # Enable actual operations
  - --time-to-live=14d       # 14 days retention

​
Non-Interactive Patching

For automated updates, use kubectl patch: 
# Update schedule
kubectl patch cronjob exchanges-deletion-cronjob -p '{"spec":{"schedule":"0 2 * * *"}}'

# Update dry-run flag
kubectl patch cronjob completed-measurements-deletion-cronjob --type=json -p='[
  {"op": "replace", "path": "/spec/jobTemplate/spec/template/spec/containers/0/args/0", "value": "--dry-run=false"}
]'

​
Duration Format Reference

The --time-to-live flag uses human-readable duration format consisting of a sequence of decimal numbers with unit suffixes:
SuffixUnitExample
dDays7d = 7 days
hHours24h = 24 hours
mMinutes30m = 30 minutes
sSeconds60s = 60 seconds
msMilliseconds500ms = 500 milliseconds
nsNanoseconds1000ns = 1000 nanoseconds

​
Examples

# 3 hours and 50 minutes
--time-to-live=3h50m

# 30 days
--time-to-live=30d

# 1 day and 12 hours
--time-to-live=1d12h

# 90 days
--time-to-live=90d

​
Best Practices

Testing Retention Policies
  1. Always test in a non-production environment first
  2. Start with --dry-run=true to verify what would be deleted
  3. Review logs to ensure the policy targets the correct resources
  4. Gradually decrease retention periods based on storage needs
Monitoring Retention Jobs
  • Check CronJob execution history: kubectl get jobs
  • View job logs: kubectl logs job/<job-name>
  • Monitor deleted resource counts in job output
  • Set up alerts for failed retention jobs

​
Troubleshooting

Check if the CronJob is suspended:
kubectl get cronjob <name> -o jsonpath='{.spec.suspend}'
If true, resume it:
kubectl patch cronjob <name> -p '{"spec":{"suspend":false}}'
View the job logs:
# Get the most recent job
kubectl get jobs --sort-by=.metadata.creationTimestamp

# View logs
kubectl logs job/<job-name>
Common issues:
  • Insufficient permissions (RBAC)
  • Database connection errors
  • Invalid duration format
Check if resources are being deleted:
# Enable retention operations
kubectl patch cronjob <name> -p '{"spec":{"jobTemplate":{"spec":{"template":{"spec":{"containers":[{"name":"<container-name>","args":["--dry-run=false","--time-to-live=30d"]}]}}}}}}'

# Monitor the next job execution
kubectl logs -f job/<next-job-name>
Look for deletion counts in the job output.

Build docs developers (and LLMs) love

Get started for freeTalk to us