Security Features - Camera Workflow

Camera Workflow is designed with multiple layers of security to protect your media files during conversion and archival.

Safety Guarantees

Core Principles

Originals preserved by default

Files are never deleted unless explicitly requested with --keep-originals=false.

Atomic operations

All conversions use temporary files with atomic renames to prevent corruption.

Triple verification

Files are verified for existence, integrity, and size before any destructive operations.

Automatic recovery

Incomplete conversions are detected and cleaned up on resume.

Camera Workflow is 100% safe and idempotent - you can run it multiple times without risk.

Atomic Operations

Temporary File Pattern

All conversions happen in three atomic steps:

1. Convert to .tmp
2. Verify integrity
3. Atomic rename

tempPath := outputPath + ".tmp"
cmd := exec.CommandContext(ctx, "magick", inputPath,
    "-quality", fmt.Sprintf("%d", quality),
    fmt.Sprintf("%s:%s", format, tempPath))
cmd.Run()

Files are written with .tmp suffix to indicate in-progress conversion.

if err := c.security.VerifyOutputFile(inputPath, tempPath, "photo", format); err != nil {
    return fmt.Errorf("output verification failed: %w", err)
}

Comprehensive verification before making file visible.

if err := os.Rename(tempPath, outputPath); err != nil {
    return fmt.Errorf("failed to finalize conversion: %w", err)
}

Atomic operation makes the file visible. Either succeeds completely or fails with no side effects.

Why this matters:

If interrupted (Ctrl+C, crash), .tmp files are cleaned up automatically
No partially-converted files ever appear in destination
Re-running continues from where it left off

Processing Markers

Active conversions are tracked with .processing markers:

func (s *SecurityChecker) CreateProcessingMarker(filePath string) error {
    markerPath := filePath + ".processing"
    
    content := fmt.Sprintf("PID:%d\nStarted:%s\nFile:%s\n",
        os.Getpid(),
        time.Now().Format(time.RFC3339),
        filePath)
    
    return os.WriteFile(markerPath, []byte(content), 0644)
}

Example marker file:

PID:12345
Started:2024-03-15T14:23:45Z
File:/dest/2024/03-March/2024-03-15/images/20240315_IMG_1234.avif

Markers are removed after successful conversion or cleaned up on crash recovery.

File Integrity Verification

Output Verification

Every converted file undergoes multi-stage verification:

func (s *SecurityChecker) VerifyOutputFile(inputPath, outputPath, fileType, outputFormat string) error {
    // 1. Check file exists
    if _, err := os.Stat(outputPath); os.IsNotExist(err) {
        return fmt.Errorf("output file does not exist")
    }

    // 2. Check file not empty
    outputInfo, _ := os.Stat(outputPath)
    if outputInfo.Size() == 0 {
        os.Remove(outputPath)
        return fmt.Errorf("output file is empty")
    }

    // 3. Check minimum size ratio
    inputInfo, _ := os.Stat(inputPath)
    minSize := int64(float64(inputInfo.Size()) * ratio)
    if outputInfo.Size() < minSize {
        os.Remove(outputPath)
        return fmt.Errorf("output file too small")
    }

    // 4. Verify format integrity
    switch fileType {
    case "photo":
        return s.verifyImageIntegrity(outputPath)
    case "video":
        return s.verifyVideoIntegrity(outputPath)
    }
}

Image Integrity Check

Uses ImageMagick to verify image structure:

func (s *SecurityChecker) verifyImageIntegrity(imagePath string) error {
    cmd := exec.Command("magick", "identify", imagePath)
    if err := cmd.Run(); err != nil {
        os.Remove(imagePath)  // Clean up corrupted file
        return fmt.Errorf("image is corrupted: %s", imagePath)
    }
    return nil
}

Tests:

File header is valid
Image can be decoded
Metadata is readable
No corruption in structure

Video Integrity Check

Uses FFprobe to verify video structure:

func (s *SecurityChecker) verifyVideoIntegrity(videoPath string) error {
    cmd := exec.Command("ffprobe", videoPath)
    cmd.Stdout = nil
    cmd.Stderr = nil
    if err := cmd.Run(); err != nil {
        os.Remove(videoPath)  // Clean up corrupted file
        return fmt.Errorf("video is corrupted: %s", videoPath)
    }
    return nil
}

Tests:

Container format is valid
Video streams are readable
Audio streams are readable
Duration is determinable

Size Ratio Validation

Format-specific minimum size ratios prevent corrupted files:

type SecurityChecker struct {
    minOutputSizeRatio     float64  // Generic default
    minOutputSizeRatioAVIF float64  // AVIF-specific
    minOutputSizeRatioWebP float64  // WebP-specific
}

// From config (internal/config/config.go)
MinOutputSizeRatioAVIF: 0.01  // Output must be ≥1% of input
MinOutputSizeRatioWebP: 0.02  // Output must be ≥2% of input

Files smaller than the minimum ratio are considered corrupted and deleted automatically.

Disk Space Checks

Pre-flight Verification

Before starting conversion, disk space is verified:

func (s *SecurityChecker) CheckDiskSpace(sourceDir, destDir string) error {
    sourceSize, err := getDirSize(sourceDir)
    if err != nil {
        return fmt.Errorf("failed to get source directory size: %w", err)
    }

    destAvailable, err := getAvailableSpace(destDir)
    if err != nil {
        return fmt.Errorf("failed to get available space: %w", err)
    }

    // Estimate needed space (50% of original for safety)
    estimatedNeeded := sourceSize / 2

    if destAvailable < estimatedNeeded {
        return fmt.Errorf("insufficient disk space! Available: %s, Estimated needed: %s",
            formatBytes(destAvailable), formatBytes(estimatedNeeded))
    }

    return nil
}

Safety margin:

Assumes output will be 50% of input size (conservative estimate)
AVIF typically produces 30-40% of original size
H.265 typically produces 40-60% of original size

Skip Disk Check

Only use --skip-disk-check if you’re confident you have enough space.

media-converter --skip-disk-check /source /dest

Skipping the check is useful for:

Network drives with unreliable space reporting
Very large archives where calculation is slow
Testing/development

Conversion Timeouts

Timeout Protection

Conversions have configurable timeouts to prevent hanging:

ctx, cancel := context.WithTimeout(context.Background(), c.config.ConversionTimeoutPhoto)
defer cancel()

cmd := exec.CommandContext(ctx, "magick", inputPath, ...)
if err := cmd.Run(); err != nil {
    // Timeout or conversion error
    return err
}

Default Timeouts

Photos
Videos

Default: 300 seconds (5 minutes)

# Increase for large RAW files
media-converter --timeout-photo 600 /source /dest

Typical conversion times:

JPEG: 1-3 seconds
RAW (24MP): 5-15 seconds
RAW (50MP+): 15-45 seconds

Default: 1800 seconds (30 minutes)

# Increase for 4K+ videos
media-converter --timeout-video 3600 /source /dest

Typical conversion times:

1080p 5min: 2-5 minutes (hardware)
1080p 5min: 10-20 minutes (software)
4K 10min: 10-15 minutes (hardware)
4K 10min: 40-60 minutes (software)

If a timeout occurs, the file is marked as failed and the .tmp file is cleaned up.

Safe Deletion

Triple Verification

If --keep-originals=false, files are only deleted after triple verification:

func (s *SecurityChecker) SafeDelete(filePath, outputPath string) error {
    // 1. Verify output file exists
    outputInfo, err := os.Stat(outputPath)
    if err != nil {
        return fmt.Errorf("cannot verify output file before deletion: %w", err)
    }

    // 2. Ensure output file is not empty and has reasonable size
    if outputInfo.Size() < 1000 {
        return fmt.Errorf("deletion cancelled for safety: output file too small")
    }

    // 3. Perform the deletion
    if err := os.Remove(filePath); err != nil {
        return fmt.Errorf("failed to delete original file: %w", err)
    }

    return nil
}

Safety checks:

✅ Output file exists
✅ Output file is ≥1KB (not empty/corrupted)
✅ No errors during deletion

If any check fails, deletion is cancelled and original is preserved.

Deletion Workflow

Convert successfully

IMG_1234.CR2 → 20240315_IMG_1234.avif.tmp

Verify output

✅ Verify image integrity with ImageMagick
✅ Check file size ≥1KB
✅ Atomic rename to final name

Safe delete (if requested)

✅ Output exists: 20240315_IMG_1234.avif (13.5 MB)
✅ Output size ≥1KB: true
✅ Delete original: IMG_1234.CR2

Log output:

✅ IMG_1234.CR2 -> 20240315_IMG_1234.avif | -68% (42.3->13.5 MB) | 3.2s
🔒 Safe deletion: IMG_1234.CR2

Crash Recovery

Recovery Phase

Every run starts with automatic recovery:

func (c *Converter) performRecovery() error {
    c.logger.Info("🔍 Performing recovery check...")

    // Cleanup abandoned files
    if err := c.security.CleanupAbandonedFiles(c.config.DestDir); err != nil {
        return fmt.Errorf("cleanup failed: %w", err)
    }

    // Find and remove abandoned markers
    abandonedMarkers, err := c.security.FindAbandonedMarkers(c.config.DestDir)
    if err != nil {
        return fmt.Errorf("failed to find abandoned markers: %w", err)
    }

    if len(abandonedMarkers) > 0 {
        c.logger.Info(fmt.Sprintf("🔄 Found %d abandoned conversion markers", len(abandonedMarkers)))
        for _, marker := range abandonedMarkers {
            os.Remove(marker)
        }
    }

    // Verify existing files and mark corrupted ones for re-conversion
    if err := c.verifyExistingFiles(); err != nil {
        return fmt.Errorf("verification failed: %w", err)
    }

    c.logger.Success("✅ Recovery check completed")
    return nil
}

Abandoned File Cleanup

func (s *SecurityChecker) CleanupAbandonedFiles(dir string) error {
    filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
        if !info.IsDir() {
            // Remove .tmp files
            if strings.HasSuffix(path, ".tmp") {
                os.Remove(path)
            }

            // Remove abandoned .processing markers
            if strings.HasSuffix(path, ".processing") && s.isMarkerAbandoned(path) {
                os.Remove(path)
            }
        }
        return nil
    })
}

Abandoned Marker Detection

func (s *SecurityChecker) isMarkerAbandoned(markerPath string) bool {
    content, _ := os.ReadFile(markerPath)
    lines := strings.Split(string(content), "\n")
    
    for _, line := range lines {
        if strings.HasPrefix(line, "PID:") {
            pidStr := strings.TrimPrefix(line, "PID:")
            pid, _ := strconv.Atoi(pidStr)
            
            if !processExists(pid) {
                return true  // Process is dead
            }
            
            return false  // Process still running
        }
    }
    
    return true  // No PID found = abandoned
}

Corruption Detection

Existing files are verified on resume:

func (c *Converter) verifyExistingFiles() error {
    filepath.Walk(c.config.DestDir, func(path string, info os.FileInfo, err error) error {
        if strings.HasSuffix(strings.ToLower(path), ".avif") ||
           strings.HasSuffix(strings.ToLower(path), ".webp") {
            if c.security.IsFileCorrupted(path, "photo") {
                c.logger.Warn(fmt.Sprintf("🔍 Corrupted image detected: %s (will be re-converted)", filepath.Base(path)))
                os.Remove(path)
                c.stats.recoveredFiles++
            } else {
                c.stats.verifiedFiles++
            }
        }
        return nil
    })
}

Recovery statistics:

✅ Files processed: 234/234
🔄 Files recovered from corruption: 3
🧹 Abandoned files cleaned: 2
🔍 Files verified for integrity: 229

Idempotency

Safe Re-runs

Camera Workflow can be run multiple times safely:

# First run - converts all files
media-converter /source /dest

# Interrupted (Ctrl+C)
# ...

# Resume - skips completed, re-converts corrupted
media-converter /source /dest

# Third run - skips everything (nothing to do)
media-converter /source /dest

Idempotency Checks

// Check if file already exists and is valid
if existingInfo, err := os.Stat(baseOutputPath); err == nil {
    if !c.security.IsFileCorrupted(baseOutputPath, "photo") {
        c.logger.Info(fmt.Sprintf("📷 %s -> %s (already exists and valid, skipping)", filename, baseName))
        c.stats.skippedFiles++
        return nil
    } else {
        c.logger.Warn(fmt.Sprintf("📷 %s -> %s (corrupted file detected, re-converting)", filename, baseName))
        os.Remove(baseOutputPath)
        c.stats.recoveredFiles++
    }
}

Behavior:

✅ Existing valid files → skipped
❌ Existing corrupted files → re-converted
🆕 Missing files → converted
🔄 .tmp files → cleaned up and re-converted

Checksum Verification (Copy-Only)

In copy-only mode, checksums provide extra safety:

media-converter --copy-only --verify-checksum /source /dest

Checksum Process

Calculate source

srcChecksum, err := c.hasher.Calculate(srcPath)
// XXHash64: 0x1a2b3c4d5e6f7890

Copy data

copyFileData(srcPath, tmpPath)

Verify copy

copiedChecksum, err := c.hasher.Calculate(tmpPath)
if copiedChecksum != srcChecksum {
    os.Remove(tmpPath)
    return fmt.Errorf("checksum mismatch")
}

Finalize

os.Rename(tmpPath, destPath)

XXHash64 is used for its speed (~GB/s) while providing reliable duplicate detection.

Safety Test

Pre-flight Conversion Test

Before batch processing, a safety test is run:

func (c *Converter) runSafetyTest() error {
    c.logger.Info("Running safety test...")

    // Find a JPG/JPEG test file (prefer small files)
    testFile := findTestFile(c.config.SourceDir)
    if testFile == "" {
        c.logger.Warn("No test file found, skipping safety test")
        return nil
    }

    // Create isolated test directory
    testDir := filepath.Join(c.config.DestDir, ".safety_test")
    os.MkdirAll(testDir, 0755)
    defer os.RemoveAll(testDir)

    // Test conversion
    c.logger.Info(fmt.Sprintf("Testing conversion on: %s", filepath.Base(testFile)))
    err := c.convertFile(testCopy, "photo")

    if err != nil {
        return fmt.Errorf("safety test failed: %w", err)
    }

    c.logger.Success("Safety test passed ✅")
    return nil
}

Purpose:

Verifies ImageMagick/FFmpeg are working
Tests conversion settings before processing hundreds of files
Catches configuration errors early

Skipped in:

Dry-run mode (--dry-run)
Copy-only mode (--copy-only)
When no suitable test file is found

Platform-Specific Security

Process Detection (Unix)

// +build !windows

func processExists(pid int) bool {
    process, err := os.FindProcess(pid)
    if err != nil {
        return false
    }
    
    err = process.Signal(syscall.Signal(0))
    return err == nil
}

Disk Space (Platform-specific)

Implementation varies by OS:

macOS/Linux: Uses syscall.Statfs
Windows: Uses syscall.GetDiskFreeSpaceEx

See internal/security/disk_*.go for platform implementations.

Best Practices

Recommended Flags

Conservative (safest)
Balanced
Aggressive (delete originals)

media-converter \
  --keep-originals \
  --dry-run \
  /source /dest

# Review output, then run for real:
media-converter \
  --keep-originals \
  /source /dest

media-converter \
  --keep-originals \
  --organize-by-date \
  --jobs 4 \
  /source /dest

# Test first!
media-converter \
  --keep-originals=false \
  --dry-run \
  /source /dest

# Verify dry-run output, then:
media-converter \
  --keep-originals=false \
  /source /dest

Never Do This

Do not use the same directory for source and destination without testing:

# ❌ DANGEROUS - could overwrite originals
media-converter --keep-originals=false /photos /photos

# ✅ SAFE - test with dry-run first
media-converter --keep-originals=false --dry-run /photos /photos

Do not skip disk checks on small drives:

# ❌ RISKY - could fill disk and fail
media-converter --skip-disk-check /photos /small-drive

# ✅ SAFE - let disk check protect you
media-converter /photos /small-drive

Getting Started

CLI Reference

Core Features

Raycast Extension

Advanced

Development

​Safety Guarantees

​Core Principles

​Atomic Operations

​Temporary File Pattern

​Processing Markers

​File Integrity Verification

​Output Verification

​Image Integrity Check

​Video Integrity Check

​Size Ratio Validation

​Disk Space Checks

​Pre-flight Verification

​Skip Disk Check

​Conversion Timeouts

​Timeout Protection

​Default Timeouts

​Safe Deletion

​Triple Verification

​Deletion Workflow

​Crash Recovery

​Recovery Phase

​Abandoned File Cleanup

​Abandoned Marker Detection

​Corruption Detection

​Idempotency

​Safe Re-runs

​Idempotency Checks

​Checksum Verification (Copy-Only)

​Checksum Process

​Safety Test

​Pre-flight Conversion Test

​Platform-Specific Security

​Process Detection (Unix)

​Disk Space (Platform-specific)

​Best Practices

​Recommended Flags

​Never Do This

Build docs developers (and LLMs) love

Safety Guarantees

Core Principles

Atomic Operations

Temporary File Pattern

Processing Markers

File Integrity Verification

Output Verification

Image Integrity Check

Video Integrity Check

Size Ratio Validation

Disk Space Checks

Pre-flight Verification

Skip Disk Check

Conversion Timeouts

Timeout Protection

Default Timeouts

Safe Deletion

Triple Verification

Deletion Workflow

Crash Recovery

Recovery Phase

Abandoned File Cleanup

Abandoned Marker Detection

Corruption Detection

Idempotency

Safe Re-runs

Idempotency Checks

Checksum Verification (Copy-Only)

Checksum Process

Safety Test

Pre-flight Conversion Test

Platform-Specific Security

Process Detection (Unix)

Disk Space (Platform-specific)

Best Practices

Recommended Flags

Never Do This