PII Tokenization

Blnk provides built-in tokenization for Personally Identifiable Information (PII) using AES-256-GCM encryption. The tokenization service converts sensitive data into secure tokens while maintaining format compatibility for business logic.

Architecture

The tokenization system is implemented in /internal/tokenization/tokenization.go and provides:

Field-Level Tokenization: Tokenize specific identity fields
Format-Preserving Tokenization: Maintain original data format (e.g., email remains email-like)
AES-256-GCM Encryption: Industry-standard encryption with authenticated encryption
Reversible Tokens: Detokenize when authorized access is needed

Configuration

Enable Tokenization

Set a 32-byte encryption key in your blnk.json:

{
  "tokenization_secret": "your-32-byte-secret-key-here-12"
}

Or via environment variable:

BLNK_TOKENIZATION_SECRET=your-32-byte-secret-key-here-12

Security Requirements:

Key must be exactly 32 bytes (256 bits)
Use a cryptographically secure random generator
Store key in a secrets management system (e.g., AWS Secrets Manager, HashiCorp Vault)
Rotate keys periodically with a migration plan

Generate a Secure Key:

# Linux/macOS
openssl rand -base64 32

# Or using Go
go run -
package main
import (
    "crypto/rand"
    "encoding/base64"
    "fmt"
)
func main() {
    key := make([]byte, 32)
    rand.Read(key)
    fmt.Println(base64.StdEncoding.EncodeToString(key))
}

Tokenizable Fields

The following Identity fields support tokenization (tokenization.go:28-36):

var TokenizableFields = []string{
    "FirstName",
    "LastName",
    "OtherNames",
    "EmailAddress",
    "PhoneNumber",
    "Street",
    "PostCode",
}

Tokenization Service

Service Initialization

The service is initialized on startup:

// Source: tokenization.go:55-69
func NewTokenizationService(encryptionKey []byte) *TokenizationService {
    enabled := len(encryptionKey) == 32
    
    return &TokenizationService{
        key:     encryptionKey,
        enabled: enabled,
    }
}

Error Handling: If tokenization is not enabled (invalid/missing key), operations return:

var ErrTokenizationDisabled = errors.New(
    "tokenization is disabled: BLNK_TOKENIZATION_SECRET or tokenization_secret in your blnk.json file must be set to a 32-byte value",
)

Tokenization Modes

Blnk supports two tokenization modes:

1. Standard Mode (Default)

Standard tokenization uses AES-GCM encryption with base64 encoding:

// Source: tokenization.go:92-124
func (s *TokenizationService) TokenizeWithMode(value string, mode TokenizationMode) (string, error) {
    if err := s.ensureEnabled(); err != nil {
        return "", err
    }
    
    // Standard tokenization using AES encryption
    block, err := aes.NewCipher(s.key)
    if err != nil {
        return "", err
    }
    
    // Create GCM
    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return "", err
    }
    
    // Create nonce
    nonce := make([]byte, gcm.NonceSize())
    if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
        return "", err
    }
    
    // Encrypt
    ciphertext := gcm.Seal(nonce, nonce, []byte(value), nil)
    
    // Return base64 encoded token
    return base64.StdEncoding.EncodeToString(ciphertext), nil
}

Example:

Input:  [email protected]
Output: dGVzdF90b2tlbl9mb3JfZGVtb25zdHJhdGlvbg==

Use Cases:

Long-term storage
Fields not used in display/logic
Maximum security

2. Format-Preserving Mode

Format-preserving tokenization maintains the original data format:

// Source: tokenization.go:208-230
func (s *TokenizationService) formatPreservingTokenize(value string) (string, error) {
    // 1. Create a deterministic seed using HMAC of the value
    h := hmac.New(sha256.New, s.key)
    h.Write([]byte(value))
    seed := h.Sum(nil)
    
    // 2. Use the seed to generate a format-preserving token
    visibleToken, err := generateTokenWithFormat(seed, value)
    if err != nil {
        return "", err
    }
    
    // 3. Encrypt the original value using standard encryption
    standardToken, err := s.Tokenize(value)
    if err != nil {
        return "", err
    }
    
    // 4. Create the final token by combining the format-preserving token and its identifier
    // We'll prefix format-preserving tokens with FPT: to identify them
    finalToken := fmt.Sprintf("FPT:%s:%s", visibleToken, standardToken)
    
    return finalToken, nil
}

Format Preservation Rules (tokenization.go:261-285):

Uppercase letters → Random uppercase letters
Lowercase letters → Random lowercase letters
Digits → Random digits
Special characters → Preserved as-is

Example:

Input:  [email protected]
Output: FPT:[email protected]:dGVzdF90b2tlbl9mb3JfZGVtb25zdHJhdGlvbg==

Use Cases:

Email validation (looks like email)
Phone number formatting (maintains country code structure)
Display purposes (preserves readability patterns)
Business logic requiring format

API Usage

Tokenizing Identities

When creating or updating identities, PII fields are automatically tokenized:

POST /identities
Content-Type: application/json

{
  "identity_type": "individual",
  "first_name": "John",
  "last_name": "Doe",
  "email_address": "[email protected]",
  "phone_number": "+1234567890",
  "street": "123 Main St",
  "city": "San Francisco",
  "country": "US"
}

Stored (tokenized):

{
  "identity_id": "idt_abc123",
  "identity_type": "individual",
  "first_name": "dGVzdF90b2tlbl8x",
  "last_name": "dGVzdF90b2tlbl8y",
  "email_address": "FPT:[email protected]:dGVzdF90b2tlbl8z",
  "phone_number": "dGVzdF90b2tlbl80",
  "street": "dGVzdF90b2tlbl81",
  "city": "San Francisco",
  "country": "US"
}

Note: Non-PII fields (city, country) remain in plaintext.

Detokenizing Data

Retrieve original values when authorized:

GET /identities/idt_abc123?detokenize=true
Authorization: Bearer <admin-token>

Response:

{
  "identity_id": "idt_abc123",
  "first_name": "John",
  "last_name": "Doe",
  "email_address": "[email protected]",
  "phone_number": "+1234567890",
  "street": "123 Main St"
}

Detokenization Logic (tokenization.go:134-143):

func (s *TokenizationService) Detokenize(token string) (string, error) {
    // Auto-detect the token type based on prefix
    if strings.HasPrefix(token, "FPT:") {
        // Format-preserving token
        return s.formatPreservingDetokenize(token)
    }
    
    // Standard token
    return s.standardDetokenize(token)
}

Security Considerations

Encryption Strength

Blnk uses AES-256-GCM which provides:

Confidentiality: 256-bit AES encryption
Authenticity: GCM authenticated encryption mode
Tamper Detection: Modification attempts fail decryption
Unique IVs: Random nonce per encryption

Key Management

Best Practices:

External Secrets Manager: Store keys in AWS Secrets Manager, HashiCorp Vault, or similar

# Example: AWS Secrets Manager
export BLNK_TOKENIZATION_SECRET=$(aws secretsmanager get-secret-value \
  --secret-id blnk/tokenization-key \
  --query SecretString \
  --output text)

Key Rotation: Implement periodic key rotation
- Generate new 32-byte key
- Update configuration with new key
- Re-tokenize existing data (migration script)
- Decommission old key

Environment Separation: Use different keys per environment

# Development
BLNK_TOKENIZATION_SECRET=dev-key-32-bytes-long-here

# Production
BLNK_TOKENIZATION_SECRET=prod-key-32-bytes-long-here

Access Control: Limit who can:
- View tokenization keys
- Call detokenize API
- Access raw database

Compliance

Tokenization helps meet compliance requirements: PCI DSS:

Tokenize cardholder data (if storing card info)
Reduce PCI scope by removing plaintext card data

GDPR:

Pseudonymization of personal data
Right to be forgotten (delete keys instead of data)
Data minimization (only detokenize when needed)

HIPAA:

Protect Protected Health Information (PHI)
Audit access to detokenized data

SOC 2:

Demonstrate encryption at rest
Access controls for sensitive data

Audit Logging

Log all tokenization operations:

{
  "timestamp": "2024-01-15T10:30:00Z",
  "operation": "detokenize",
  "user_id": "usr_admin_123",
  "identity_id": "idt_abc123",
  "fields": ["email_address", "phone_number"],
  "reason": "Customer support request #12345"
}

Performance Impact

Tokenization Overhead

Tokenization adds minimal latency:

Tokenize: ~0.1ms per field
Detokenize: ~0.1ms per field
Format-preserving: ~0.2ms per field (additional HMAC)

Benchmark (1000 operations):

BenchmarkTokenize-8                 10000    105 µs/op
BenchmarkDetokenize-8               10000    102 µs/op
BenchmarkFormatPreserving-8          5000    215 µs/op

Optimization Tips

Batch Operations: Tokenize multiple fields in parallel

var wg sync.WaitGroup
for _, field := range fields {
    wg.Add(1)
    go func(f string) {
        defer wg.Done()
        tokenized[f] = service.Tokenize(data[f])
    }(field)
}
wg.Wait()

Cache Detokenized Values: If same identity accessed frequently

// Cache for 5 minutes
cache.Set(identityID, detokenizedData, 5*time.Minute)

Use Standard Mode: Format-preserving adds ~2x overhead

Migration Guide

Enabling Tokenization on Existing System

Generate Key:

openssl rand -base64 32 > tokenization.key

Update Configuration:

{
  "tokenization_secret": "<paste-key-here>"
}

Restart Blnk: New identities will be tokenized automatically

Migrate Existing Data (optional):

# Custom migration script
curl -X POST http://localhost:5001/admin/migrate-tokenization

Key Rotation

Generate New Key:
```
NEW_KEY=$(openssl rand -base64 32)
```

Dual-Key Configuration (supports both old and new):

{
  "tokenization_secret": "new-key",
  "tokenization_secret_old": "old-key"
}

Re-tokenize Data:

-- Identify records needing re-tokenization
SELECT identity_id FROM blnk.identities WHERE updated_at < '2024-01-01';

Remove Old Key after all data migrated:
```
{
  "tokenization_secret": "new-key"
}
```

Troubleshooting

Tokenization Disabled Error

Symptom:

Error: tokenization is disabled: BLNK_TOKENIZATION_SECRET must be set to a 32-byte value

Solutions:

Verify key is exactly 32 bytes:

echo -n "$BLNK_TOKENIZATION_SECRET" | wc -c
# Should output: 32

Check environment variable is set:
```
echo $BLNK_TOKENIZATION_SECRET
```
Restart Blnk after configuration change

Detokenization Fails

Symptom:

Error: failed to decrypt token: cipher: message authentication failed

Causes:

Wrong Key: Token encrypted with different key
Corrupted Token: Token modified/truncated
Version Mismatch: Token format changed between versions

Solutions:

Verify correct tokenization key is configured
Check token not corrupted in database
Use original key that encrypted the data

Format-Preserving Not Working

Symptom: Format not preserved (getting base64 output) Solution: Explicitly specify mode:

token, err := service.TokenizeWithMode(value, tokenization.FormatPreservingMode)

Best Practices

Tokenize by Default: Enable tokenization for all new identities

Minimize Detokenization: Only detokenize when absolutely necessary

// Bad: Detokenize for display
displayName := service.Detokenize(firstName)

// Good: Use identifier instead
displayName := fmt.Sprintf("User %s", identityID[:8])

Audit Detokenization: Log all detokenization requests

logrus.WithFields(logrus.Fields{
    "user_id": userID,
    "identity_id": identityID,
    "reason": reason,
}).Info("Detokenization performed")

Use Format-Preserving Sparingly: Only when format is needed for business logic
Regular Key Rotation: Rotate keys annually or after security incidents
Secure Key Storage: Never commit keys to version control
```
# .gitignore
*.key
tokenization.key
blnk.json
```

Getting Started

Core Concepts

Guides

Advanced

Deployment

PII Tokenization

Architecture

Configuration

Enable Tokenization

Tokenizable Fields

Tokenization Service

Service Initialization

Tokenization Modes

1. Standard Mode (Default)

2. Format-Preserving Mode

API Usage

Tokenizing Identities

Detokenizing Data

Security Considerations

Encryption Strength

Key Management

Compliance

Audit Logging

Performance Impact

Tokenization Overhead

Optimization Tips

Migration Guide

Enabling Tokenization on Existing System

Key Rotation

Troubleshooting

Tokenization Disabled Error

Detokenization Fails

Format-Preserving Not Working

Best Practices

Build docs developers (and LLMs) love

Getting Started

Core Concepts

Guides

Advanced

Deployment

​Architecture

​Configuration

​Enable Tokenization

​Tokenizable Fields

​Tokenization Service

​Service Initialization

​Tokenization Modes

​1. Standard Mode (Default)

​2. Format-Preserving Mode

​API Usage

​Tokenizing Identities

​Detokenizing Data

​Security Considerations

​Encryption Strength

​Key Management

​Compliance

​Audit Logging

​Performance Impact

​Tokenization Overhead

​Optimization Tips

​Migration Guide

​Enabling Tokenization on Existing System

​Key Rotation

​Troubleshooting

​Tokenization Disabled Error

​Detokenization Fails

​Format-Preserving Not Working

​Best Practices

​Related Resources

Build docs developers (and LLMs) love

Architecture

Configuration

Enable Tokenization

Tokenizable Fields

Tokenization Service

Service Initialization

Tokenization Modes

1. Standard Mode (Default)

2. Format-Preserving Mode

API Usage

Tokenizing Identities

Detokenizing Data

Security Considerations

Encryption Strength

Key Management

Compliance

Audit Logging

Performance Impact

Tokenization Overhead

Optimization Tips

Migration Guide

Enabling Tokenization on Existing System

Key Rotation

Troubleshooting

Tokenization Disabled Error

Detokenization Fails

Format-Preserving Not Working

Best Practices

Related Resources