Memory Management

const totalDeviceRAM = await DeviceInfo.getTotalMemory(); // bytes
const memoryBudget = totalDeviceRAM * 0.60; // 60% of total RAM

requiredRAM = fileSize × 1.5

requiredRAM = (modelFileSize + mmProjSize) × 1.5

requiredRAM = fileSize × 1.8

// 1. Get device memory info
const deviceInfo = await hardwareService.getDeviceInfo();
const totalRAM = deviceInfo.totalMemory / (1024 ** 3); // Convert to GB
const memoryBudget = totalRAM * 0.60;

// 2. Estimate model RAM requirement
const model = getModelById(modelId);
const requiredRAM = estimateModelMemory(model, modelType);

// 3. Check if currently loaded models fit with new model
const currentlyLoaded = getCurrentlyLoadedMemory();
const totalRequired = requiredRAM + currentlyLoaded;

// 4. Compare against budget
if (totalRequired > memoryBudget) {
  return {
    canLoad: false,
    severity: 'critical',
    message: `Cannot load ${model.name} (~${requiredRAM.toFixed(1)}GB required) - would exceed device safe limit of ${memoryBudget.toFixed(1)}GB. Unload current model or choose smaller.`
  };
}

// src/services/llmHelpers.ts
export function getGpuLayersForDevice(
  totalMemoryBytes: number,
  requestedLayers: number
): number {
  const totalGB = totalMemoryBytes / (1024 ** 3);
  
  // Force CPU-only on ≤4GB devices (prevent Metal abort)
  if (totalGB <= 4) {
    return 0;
  }
  
  return requestedLayers;
}

// src/services/llmHelpers.ts
export function getMaxContextForDevice(
  totalMemoryBytes: number
): number {
  const totalGB = totalMemoryBytes / (1024 ** 3);
  
  if (totalGB <= 4) return 2048;
  if (totalGB <= 6) return 2048;
  if (totalGB <= 8) return 4096;
  return 8192;
}

// Vision models disable CLIP GPU on ≤4GB devices
const useGpuForClip = totalMemoryGB > 4;

await llmService.initializeMultimodal({
  modelPath,
  mmProjPath,
  useGpuForClip // false on ≤4GB devices
});

import { hardwareService } from '../services/hardware';

// Get app-specific memory usage
const memoryUsage = await hardwareService.getAppMemoryUsage();
console.log('Used:', memoryUsage.used / (1024 ** 3), 'GB');
console.log('Available:', memoryUsage.available / (1024 ** 3), 'GB');
console.log('Total:', memoryUsage.total / (1024 ** 3), 'GB');

// Refresh memory info (force fresh fetch)
const deviceInfo = await hardwareService.refreshMemoryInfo();
console.log('Available RAM:', deviceInfo.availableMemory);

// Get combined size (model + mmproj for vision models)
const totalSize = hardwareService.getModelTotalSize(model);

// Format for display
const sizeString = hardwareService.formatModelSize(model); // "1.2 GB"

// Estimate RAM usage
const ramEstimate = hardwareService.estimateModelRam(model, 1.5);
const ramString = hardwareService.formatModelRam(model, 1.5); // "~1.8 GB"

Cannot load Qwen3-7B-Q4_K_M (~5.5GB required) - would exceed
device safe limit of 4.8GB. Unload current model or choose smaller.

Loading Llama 3.2 3B Q4_K_M will use ~3.0GB (~55% of available RAM).
Performance may degrade if other apps are running.

Your device has limited memory (4GB). Only the smallest models will
work well. GPU acceleration disabled automatically for stability.

Running in emulator. Performance may be significantly slower than
real device. Memory estimates may be inaccurate.

await activeModelService.unloadTextModel();
await activeModelService.loadTextModel(newModelId);