Overview
Processes user-uploaded files (images and PDFs) to extract transaction information using AI. Similar to email processing but handles direct file uploads from the frontend application.Endpoint
Authentication
Requires valid Supabase authentication. The JWT token must be provided in theAuthorization header.
Request
Headers
Bearer token with Supabase JWTFormat:
Bearer {token}MIME type of the uploaded file:
image/jpegimage/jpgimage/pngapplication/pdf
Original filename (optional, defaults to “unknown”)
Request Body
Raw binary file data (not multipart/form-data).Supported File Types
- Images: JPEG, JPG, PNG
- Documents: PDF
File Size Limit
Maximum: 5 MBExample Request
Response
Success Response
When a transaction is successfully extracted:Indicates successful transaction creation
Complete transaction object as stored in database
Unique transaction ID (UUID)
User ID who uploaded the document
Always “manual-upload” for uploaded files
Unique identifier in format
manual-{timestamp}-{random}Transaction amount extracted by AI
Currency code (USD, EUR, GBP, etc.)
Type of transaction: “expense” or “income”
Description extracted by AI
Date in YYYY-MM-DD format
Merchant/vendor name extracted by AI
Transaction category (e.g., “food”, “transport”, “entertainment”)
No Transaction Found
When AI cannot extract transaction data:Error Responses
400 Bad Request
Missing File
File Too Large
Unsupported File Type
PDF Processing Error
401 Unauthorized
Returned when authentication fails or token is invalid.405 Method Not Allowed
Returned when using any HTTP method other than POST.500 Internal Server Error
PDF Processing Failed
Database Error
General Error
Processing Flow
1. Authentication
Verifies Supabase JWT token fromAuthorization header.
2. File Validation
- Checks file size (max 5 MB)
- Validates content type
- Reads file as binary data
3. File Processing
For PDFs
Usesunpdf library to extract text:
For Images
Converts image bytes toImageAttachment format:
4. User Context
Retrieves user’s full name from Supabase Auth metadata to provide context for AI analysis.5. AI Analysis
CallsextractTransactionFromEmail() with:
- Document content description
- User’s full name
- Image attachments (for images)
- Extracted text (for PDFs)
- Receipt/invoice structure
- Transaction amounts
- Merchant information
- Dates
- Item descriptions
- Currency
6. Transaction Storage
If AI successfully extracts transaction data, stores intransactions table:
7. Langfuse Flush
Flushes AI observability events:Implementation Details
Environment Variables Required
SUPABASE_URL- Supabase project URLSUPABASE_SERVICE_ROLE_KEY- Service role key for database access
Image Format Normalization
The function normalizesimage/jpg to image/jpeg for consistency:
Source Identification
Manually uploaded transactions are marked with:source_email: “manual-upload”source_message_id: Unique ID in formatmanual-{timestamp}-{random}
PDF Text Extraction
Uses theunpdf NPM package:
mergePages: true option to combine all pages.
User Metadata Access
Retrieves user metadata via Admin API:Best Practices
Frontend Integration
- Use
Content-Typeheader matching the actual file type - Send raw binary data (not form-encoded)
- Include
x-file-nameheader with original filename - Handle both success (
success: true) and no-transaction (success: false) responses - Implement retry logic for 500 errors
File Optimization
- Resize large images before upload to stay under 5 MB
- Compress images while maintaining readability
- For PDFs, ensure text is extractable (not scanned images)
Error Handling
- Display clear error messages from
errorfield - For “no transaction found” responses, allow users to manually enter data
- Log failures for debugging
Related Functions
- gmail-webhook - Similar AI processing for email-based transactions
- seed-emails - Bulk processing of historical emails