Skip to main content

Overview

Upload photos of receipts, invoices, or tickets, and Sistema Financiero will automatically extract:
  • Total amount
  • Merchant/store name
  • Suggested category
  • List of items purchased
  • Transaction date
The OCR feature uses Google Gemini 2.5 Flash with vision capabilities for accurate text extraction from images.
The system validates that uploaded images are actually receipts before processing, preventing accidental uploads of unrelated photos.

How It Works

1

Upload image

Take or upload a photo of your receipt. The image is stored in Supabase Storage.
2

Image validation

AI checks if the image is a valid receipt, invoice, or purchase document.
  • If valid: proceeds to data extraction
  • If invalid: returns a helpful error message
3

OCR extraction

Gemini 2.5 Flash analyzes the image and extracts:
  • Total amount (monto)
  • Store name (comercio)
  • Suggested category (categoria_sugerida)
  • Items purchased (items)
  • Date (fecha)
4

Structured response

Data is returned in both human-readable and JSON formats for easy integration.

Example Extractions

Gas Station Receipt

Image uploaded: Receipt from Pemex AI Response:
📸 TICKET ANALIZADO:

💰 Monto: $450.50
🏪 Comercio: Pemex
📁 Categoría sugerida: Transporte
📋 Items: Magna Premium 30L, Total
📅 Fecha: 2025-10-06

📝 Descripción: Llenado de combustible en Pemex
Structured JSON:
{
  "es_ticket": true,
  "monto": 450.50,
  "comercio": "Pemex",
  "categoria_sugerida": "Transporte",
  "items": ["Magna Premium 30L", "Total"],
  "fecha": "2025-10-06",
  "descripcion": "Llenado de combustible en Pemex"
}

Grocery Store Receipt

Image uploaded: Receipt from Walmart AI Response:
📸 TICKET ANALIZADO:

💰 Monto: $350.00
🏪 Comercio: Walmart
📁 Categoría sugerida: Alimentación
📋 Items: Leche, Pan, Huevos, Verduras
📅 Fecha: 2025-10-09

📝 Descripción: Compra de despensa en Walmart

Restaurant Receipt

Image uploaded: Receipt from a restaurant AI Response:
📸 TICKET ANALIZADO:

💰 Monto: $280.00
🏪 Comercio: Restaurante La Casa
📁 Categoría sugerida: Alimentación
📋 Items: 2x Tacos, 1x Refresco, Propina
📅 Fecha: 2025-10-09

📝 Descripción: Comida en restaurante

Invalid Image Detection

If you upload an image that’s not a receipt (screenshot, random photo, etc.):
⚠️ IMAGEN NO RECONOCIDA COMO TICKET

Esta es una captura de pantalla de una conversación de texto, 
no un ticket o factura.

💡 Sugerencia: Sube una foto de un ticket, factura o recibo de 
compra para que pueda analizarlo.

Si quieres registrar algo manualmente, dime:
- ¿Es gasto o ingreso?
- Monto
- Comercio/Proveedor
- Categoría
The AI is trained to detect non-receipt images like chat screenshots, random photos, or unrelated documents to prevent false extractions.

Category Mapping

The AI automatically suggests the most appropriate category from your system’s valid categories:

Expense Categories

  • Alimentación (Food/Groceries)
  • Transporte (Transportation/Fuel)
  • Vivienda (Housing/Rent/Utilities)
  • Salud (Health/Medical)
  • Entretenimiento (Entertainment)
  • Educación (Education)
  • Otros Gastos (Other Expenses)

Income Categories

  • Salario (Salary)
  • Ventas (Sales)
  • Servicios (Services)
  • Inversiones (Investments)
  • Otros Ingresos (Other Income)

Technical Implementation

API Endpoint

POST /api/upload-image Request: multipart/form-data
{
  image: File // Image file from form upload
}
Response:
{
  success: true,
  url: string,        // Public URL of uploaded image
  analysis: string,   // Formatted text response
  data: {             // Structured JSON data
    es_ticket: boolean,
    monto?: number,
    comercio?: string,
    categoria_sugerida?: string,
    items?: string[],
    fecha?: string,
    descripcion?: string,
    razon?: string,      // If not a ticket
    sugerencia?: string  // If not a ticket
  }
}

Storage Flow

  1. Upload to Supabase Storage:
const timestamp = Date.now()
const fileName = `ticket_${timestamp}_${image.name}`

await supabase.storage
  .from('facturas')
  .upload(fileName, imageBytes, {
    contentType: image.type,
  })
  1. Get public URL:
const { data } = supabase.storage
  .from('facturas')
  .getPublicUrl(fileName)
See /app/api/upload-image/route.ts:17-37

Vision API Integration

The system sends the image to Gemini 2.5 Flash with a detailed extraction prompt:
await fetch('https://openrouter.ai/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.OPENROUTER_API_KEY}`,
    'Content-Type': 'application/json',
    'HTTP-Referer': process.env.NEXT_PUBLIC_SITE_URL,
    'X-Title': 'Sistema Financiero - OCR',
  },
  body: JSON.stringify({
    model: 'google/gemini-2.5-flash',
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: '...' // Detailed extraction prompt
          },
          {
            type: 'image_url',
            image_url: { url: imageUrl }
          }
        ]
      }
    ],
    max_tokens: 400,
    temperature: 0.1,
    response_format: { type: 'json_object' }
  })
})
See /app/api/upload-image/route.ts:40-129

OCR Prompt Engineering

The system uses a carefully crafted prompt to ensure accurate extraction:
Analiza esta imagen y determina si es un ticket/factura válido.

PASO 1: VALIDAR SI ES UN TICKET
- ¿La imagen muestra un ticket, factura, recibo o comprobante de compra?
- ¿Tiene información de comercio, monto, items comprados?
- Si es screenshot de chat, foto aleatoria, o documento que NO sea ticket
  → marca "es_ticket": false

INSTRUCCIONES SI ES TICKET:
1. Extrae el MONTO TOTAL (solo número, sin símbolos)
2. Identifica el COMERCIO/ESTABLECIMIENTO
3. Sugiere UNA categoría de la lista válida (la más apropiada)
4. Lista los items principales si son visibles
5. Extrae fecha si está visible

RESPONDE SOLO CON JSON (sin markdown, sin explicaciones)
Full prompt in /app/api/upload-image/route.ts:56-114
The response_format: { type: 'json_object' } parameter ensures structured JSON responses for reliable parsing.

Integration with Chat

The OCR feature integrates with the AI Chat Assistant:
// In chat API
if (images.length > 0) {
  openRouterMessages[openRouterMessages.length - 1].content += 
    `\n\n[El usuario subió ${images.length} imagen(es). 
    Analiza las imágenes para extraer información del ticket.]`
}
See /app/api/chat/route.ts:69-74
You can combine natural language with receipt images: upload a photo and say “registra este gasto” for seamless entry.

Error Handling

Upload Errors

if (uploadError) {
  throw new Error(`Upload error: ${uploadError.message}`)
}

Vision API Errors

if (!visionResponse.ok) {
  console.error('Vision API error:', errorText)
  throw new Error(`Vision API error: ${visionResponse.statusText}`)
}

JSON Parsing Fallback

try {
  ocrData = JSON.parse(analysisText)
} catch (parseError) {
  // Fallback to plain text
  ocrData = {
    monto: null,
    comercio: 'Desconocido',
    categoria_sugerida: 'Otros Gastos',
    items: [],
    descripcion: analysisText,
  }
}
See /app/api/upload-image/route.ts:145-158

Storage Bucket Configuration

Images are stored in the facturas bucket in Supabase Storage.
Ensure the facturas bucket exists and has proper public access policies configured in your Supabase project.

Model Selection

The system uses Gemini 2.5 Flash for optimal performance:
  • Fast processing (typically < 2 seconds)
  • Strong OCR capabilities
  • Multimodal support (text + images)
  • Cost-effective for high-volume usage

Best Practices

Good lighting

Take photos in well-lit conditions for better text recognition

Clear focus

Ensure the receipt is in focus and all text is readable

Full receipt

Capture the entire receipt including header and total

Flat surface

Place receipt on a flat surface to avoid distortion

Next Steps

AI Chat

Use natural language to register extracted transactions

Configuration

Configure your OpenRouter API key and model settings

Build docs developers (and LLMs) love