Key Expiry - ValKeyper

Overview

ValKeyper implements automatic key expiration using Go’s time-based channels and goroutines. When a key is set with a TTL (Time To Live), a background goroutine monitors the expiration and automatically removes the key from the store when the TTL expires.

Expiry Data Structures

The expiry system uses two key components:

type KVStore struct {
    store     map[string]string      // Main key-value storage
    expiryMap map[string]chan int    // Expiry control channels
    // ...
}

Fields

store: Main in-memory map holding key-value pairs
expiryMap: Maps each key with TTL to a control channel
- Key: The store key (string)
- Value: Channel to signal expiry cancellation

Setting Keys with Expiry

The Set method handles key creation with optional TTL:

func (kv *KVStore) Set(key, value string, expiry int) {
    if expiry != -1 {
        timeout := time.After(time.Duration(expiry) * time.Millisecond)
        go kv.handleExpiry(timeout, key)
    }
    
    kv.store[key] = value
}

Flow

Check Expiry: If expiry != -1, TTL is enabled
Create Timeout: Convert milliseconds to time.Duration
Launch Goroutine: Spawn handleExpiry in background
Store Value: Write key-value to store (synchronous)

Usage via SET Command

SET key value PX milliseconds

Implementation in processCommand:

case "SET":
    key := buff[1]
    val := buff[2]
    ex := -1
    if len(buff) > 4 {
        ex, err = strconv.Atoi(buff[4])
        if err != nil {
            panic(err)
        }
    }
    kv.Set(key, val, ex)
    res = []byte("+OK\r\n")

Example

# Set key with 5000ms (5 second) expiry
> SET session:abc token123 PX 5000
+OK

# Key exists immediately
> GET session:abc
"token123"

# After 5 seconds, key is automatically deleted
> GET session:abc
$-1

The handleExpiry Goroutine

The core expiry mechanism is implemented in the handleExpiry method:

func (kv *KVStore) handleExpiry(timeout <-chan time.Time, key string) {
    closeCh := make(chan int)
    kv.expiryMap[key] = closeCh
    for {
        select {
        case <-closeCh:
            return
        case <-timeout:
            delete(kv.store, key)
        }
    }
}

Goroutine Lifecycle

Channel Creation
```
closeCh := make(chan int)
kv.expiryMap[key] = closeCh
```
- Creates unbuffered channel for cancellation signals
- Stores channel in expiryMap for external access

Event Loop

for {
    select {
    case <-closeCh:
        return
    case <-timeout:
        delete(kv.store, key)
    }
}

closeCh receives: Goroutine exits (early cancellation)
timeout receives: Delete key and continue (TTL expired)

Goroutine Termination
- Expires naturally when timeout fires (one-time delete)
- Exits early if cancellation signal received

After the timeout fires and the key is deleted, the goroutine continues looping. However, since timeout is a one-time channel created by time.After(), it will never fire again. The goroutine effectively waits indefinitely on closeCh unless signaled.

Expiry Cancellation

Keys can be deleted manually before expiry using the DEL command:

case "DEL":
    key := buff[1]
    _, ok := kv.store[key]
    if !ok {
        res = []byte(":0\r\n")
        break switchLoop
    }
    delete(kv.store, key)
    
    ch, ok := kv.expiryMap[key]
    if ok {
        ch <- 1  // Signal goroutine to exit
    }
    
    res = []byte(":1\r\n")

Cancellation Flow

Check Existence: Verify key exists in store
Delete from Store: Remove key-value pair
Lookup Channel: Check if key has expiry goroutine
Signal Goroutine: Send value to closeCh
Goroutine Exits: Receives signal and returns

Example

# Set key with 60 second expiry
> SET temp:data value PX 60000
+OK

# Delete before expiry
> DEL temp:data
:1

# Expiry goroutine has been terminated

Loading Expiry from RDB

When loading persisted data, ValKeyper recalculates remaining TTL:

func (kv *KVStore) LoadFromRDB(rdb *rdb.RDB) {
    if len(rdb.Dbs) < 1 {
        return
    }
    kv.store = rdb.Dbs[0].DbStore
    
    for _, x := range rdb.Dbs[0].ExpiryStore {
        kv.store[x.Key] = x.Value
        duration := time.Duration(int64(x.Expiry)-time.Now().UnixMilli()) * time.Millisecond
        go kv.handleExpiry(time.After(duration), x.Key)
    }
}

RDB Expiry Restoration

Load Regular Keys: Copy DbStore to in-memory store
Process Expiry Keys: Iterate ExpiryStore

Calculate Remaining TTL:

duration = (stored_expiry_time - current_time)

Launch Goroutines: Start handleExpiry with remaining duration

Time-Based Expiry

RDB stores absolute expiry timestamps (Unix milliseconds). ValKeyper:

Subtracts current time to get remaining duration
Handles already-expired keys (negative duration)

If a key’s expiry timestamp is in the past when loading from RDB, time.After() with a negative duration will fire immediately, deleting the key.

Expiry Precision

ValKeyper uses millisecond precision for all expiry operations:

timeout := time.After(time.Duration(expiry) * time.Millisecond)

Timing Characteristics

Resolution: Milliseconds
Mechanism: Go’s time.After() timer
Accuracy: Subject to Go scheduler and system timer precision
Guarantee: Key expires at or shortly after the specified time

Go’s timer accuracy is typically within a few milliseconds on modern systems, but is not guaranteed to be exact. Keys may expire slightly after the specified TTL.

Memory Management

The expiry system has memory implications:

Per-Key Overhead

// For each key with TTL:
closeCh := make(chan int)           // Channel: ~96 bytes
kv.expiryMap[key] = closeCh         // Map entry: ~32 bytes
go kv.handleExpiry(...)             // Goroutine stack: ~2KB minimum

Estimated overhead per expiring key: ~2-4 KB

Goroutine Cleanup

After a key expires naturally, its goroutine continues running indefinitely, waiting on the close channel. This is a potential goroutine leak.

Mitigation strategies:

Always use DEL to explicitly remove keys
Let natural expiry happen for short-lived keys
Monitor goroutine count in production

Edge Cases and Behaviors

Overwriting Keys with TTL

Setting a new value for an existing key with TTL:

> SET key1 value1 PX 10000
+OK
> SET key1 value2 PX 5000  # New TTL
+OK

Behavior:

New handleExpiry goroutine is launched
Old goroutine continues running (not terminated)
Result: Goroutine leak accumulates

Repeated SET operations on the same key with TTL will spawn multiple expiry goroutines without cleaning up old ones.

Setting Key Without TTL

Removing TTL from an existing key:

> SET key value PX 10000
+OK
> SET key newvalue  # No expiry
+OK

Behavior:

Key is updated in store
Old expiry goroutine still runs
When old TTL expires, key is deleted unexpectedly

ValKeyper does not cancel existing expiry when overwriting with a key without TTL. The original expiry remains active.

Zero or Negative Expiry

> SET key value PX 0

Behavior:

time.After(0) fires immediately
Key is deleted almost instantly after being set

Best Practices

Use DEL for Early Removal

Always use DEL to remove keys before expiry to properly clean up goroutines.

Avoid Frequent Overwrites

Minimize repeated SET operations on the same key with TTL to prevent goroutine leaks.

Monitor Memory

Track memory usage and goroutine count when using many expiring keys.

Millisecond Precision

Use millisecond values (PX) for precise expiry control.

Common Patterns

Session Management

# Create session with 30-minute expiry
SET session:user123 {"user_id": 123} PX 1800000

# Extend session on activity
SET session:user123 {"user_id": 123} PX 1800000

# Logout (explicit cleanup)
DEL session:user123

Cache with TTL

# Cache API response for 5 minutes
SET cache:api:/users/123 {"name":"Alice"} PX 300000

# Invalidate cache early
DEL cache:api:/users/123

Rate Limiting

# Allow 100 requests per minute
SET rate:ip:192.168.1.1 0 PX 60000
INCR rate:ip:192.168.1.1
# Check if count < 100

Implementation Reference

Key source locations in store.go:

KVStore struct: Lines 43-51
Set method: Lines 53-62
handleExpiry goroutine: Lines 94-105
DEL cancellation: Lines 193-208
RDB loading: Lines 81-92
SET command parsing: Lines 209-220

Getting Started

Core Concepts

Commands

Operations

Advanced

​Overview

​Expiry Data Structures

​Fields

​Setting Keys with Expiry

​Flow

​Usage via SET Command

​Example

​The handleExpiry Goroutine

​Goroutine Lifecycle

​Expiry Cancellation

​Cancellation Flow

​Example

​Loading Expiry from RDB

​RDB Expiry Restoration

​Time-Based Expiry

​Expiry Precision

​Timing Characteristics

​Memory Management

​Per-Key Overhead

​Goroutine Cleanup

​Edge Cases and Behaviors

​Overwriting Keys with TTL

​Setting Key Without TTL

​Zero or Negative Expiry

​Best Practices

Use DEL for Early Removal

Avoid Frequent Overwrites

Monitor Memory

Millisecond Precision

​Common Patterns

​Session Management

​Cache with TTL

​Rate Limiting

​Implementation Reference

Build docs developers (and LLMs) love

Overview

Expiry Data Structures

Fields

Setting Keys with Expiry

Flow

Usage via SET Command

Example

The handleExpiry Goroutine

Goroutine Lifecycle

Expiry Cancellation

Cancellation Flow

Example

Loading Expiry from RDB

RDB Expiry Restoration

Time-Based Expiry

Expiry Precision

Timing Characteristics

Memory Management

Per-Key Overhead

Goroutine Cleanup

Edge Cases and Behaviors

Overwriting Keys with TTL

Setting Key Without TTL

Zero or Negative Expiry

Best Practices

Common Patterns

Session Management

Cache with TTL

Rate Limiting

Implementation Reference