Overview
SpendWisely George’s AI assistant uses Google’s Gemini Flash model to provide intelligent expense parsing, contextual insights, and financial analysis through natural language interaction.
Core Capabilities
Expense Parsing Extracts amount, description, and category from natural language
Smart Categorization Automatically assigns expenses to Food, Transport, Tech, etc.
Contextual Insights Compares new expenses to spending history
Financial Analysis Answers questions about your spending patterns
Two Modes of Operation
ADD Mode: Expense Entry
Processes natural language input to create structured expense entries.
User Flow
Frontend sends to Apps Script
const res = await fetch ( SCRIPT_URL , {
method: 'POST' ,
body: JSON . stringify ({
action: "process_text" ,
text: "Coffee 200" ,
mode: "add"
})
});
AI extracts structured data
{
"description" : "Coffee" ,
"amount" : 200 ,
"category" : "Food" ,
"type" : "DEBIT" ,
"insight" : "Your second coffee this week!"
}
Saved to Google Sheets
Transaction is appended to sheet and balance is recalculated
ADD Mode Prompt Engineering
The system provides rich context to the AI:
if ( mode === 'add' ) {
// Fetch last 40 transactions for context
const historyData = sheet . getRange (
Math . max ( 1 , sheet . getLastRow () - 40 ),
1 , 41 , 4
). getValues ();
const historyContext = historyData
. map ( r => ` ${ r [ 1 ] } ( ${ r [ 2 ] } ) [ ${ r [ 3 ] } ]` )
. join ( " \n " );
prompt = `
You are a financial assistant.
CONTEXT: The user wants to ADD a transaction.
USER INPUT: " ${ userText } "
HISTORY:
${ historyContext }
TASK:
1. Extract: Description, Amount, Category (Food, Transport, Tech, Invest, Income, Misc).
2. Analyze: Compare this new expense to the HISTORY. Is it higher than usual? Is it a repeat?
3. Insight: Write a very short, witty, or helpful insight (max 15 words) about this specific spend.
OUTPUT JSON STRICTLY:
{
"description": "String",
"amount": Number,
"category": "String",
"type": "DEBIT" or "CREDIT",
"insight": "String"
}
` ;
}
ASK Mode: Financial Analysis
Answers questions about spending patterns using transaction history.
User Flow
User asks question
How much did I spend on food this month?
AI analyzes transaction history
The system provides the last 40 transactions as context
Returns concise answer
{
"answer" : "You spent ₹3,450 on food across 12 transactions this month."
}
ASK Mode Implementation
if ( mode === 'ask' ) {
prompt = `
You are a financial analyst.
CONTEXT: The user is ASKING a question about their finances.
USER INPUT: " ${ userText } "
HISTORY:
${ historyContext }
TASK:
Answer the user's question based strictly on the HISTORY data provided.
Be concise (max 30 words).
OUTPUT JSON STRICTLY:
{
"answer": "String"
}
` ;
}
if ( mode === 'ask' ) {
card . className = "card p-4 bg-purple-50 border-purple-200 min-h-[80px]" ;
tip . innerText = "Analyzing history..." ;
tip . className = "text-[11px] font-bold text-purple-800 italic leading-relaxed flex-1" ;
}
// After AI response
if ( data . status === "Success" ) {
icon . innerText = "🧠" ;
tip . innerText = data . ai_response ;
}
Browser Speech Recognition
The app uses the Web Speech API for voice-to-text input:
const SpeechRecognition = window . SpeechRecognition || window . webkitSpeechRecognition ;
let recognition = null ;
if ( SpeechRecognition ) {
recognition = new SpeechRecognition ();
recognition . continuous = false ;
recognition . lang = 'en-IN' ; // Indian English
recognition . interimResults = false ;
recognition . onstart = () => {
document . getElementById ( 'micBtn' ). classList . add ( 'mic-active' );
};
recognition . onend = () => {
document . getElementById ( 'micBtn' ). classList . remove ( 'mic-active' );
};
recognition . onresult = ( event ) => {
const transcript = event . results [ 0 ][ 0 ]. transcript ;
const current = document . getElementById ( 'omniInput' ). value ;
document . getElementById ( 'omniInput' ). value = current
? current + " " + transcript
: transcript ;
};
}
function toggleVoice () {
if ( recognition ) recognition . start ();
else alert ( "Voice recognition not supported on this browser." );
}
Visual Feedback
The microphone button pulses red during recording:
.mic-active {
animation : pulse-red 1.5 s infinite ;
color : #ef4444 !important ;
}
@keyframes pulse-red {
0% {
box-shadow : 0 0 0 0 rgba ( 239 , 68 , 68 , 0.7 );
}
70% {
box-shadow : 0 0 0 10 px rgba ( 239 , 68 , 68 , 0 );
}
100% {
box-shadow : 0 0 0 0 rgba ( 239 , 68 , 68 , 0 );
}
}
Gemini AI Integration
API Configuration
Users configure their Gemini API key in Settings:
const SCRIPT_PROP = PropertiesService . getScriptProperties ();
// Saving API key
if ( body . action === "save_settings" ) {
SCRIPT_PROP . setProperty ( "DAILY_BUDGET" , body . budget . toString ());
SCRIPT_PROP . setProperty ( "PRIVACY_MODE" , body . privacy . toString ());
if ( body . geminiKey ) SCRIPT_PROP . setProperty ( "GEMINI_KEY" , body . geminiKey );
return ContentService . createTextOutput (
JSON . stringify ({ status: "Saved" })
). setMimeType ( ContentService . MimeType . JSON );
}
// Retrieving for AI calls
const geminiKey = SCRIPT_PROP . getProperty ( "GEMINI_KEY" );
API Request Structure
const geminiKey = SCRIPT_PROP . getProperty ( "GEMINI_KEY" );
const aiUrl = `https://generativelanguage.googleapis.com/v1beta/models/gemini-flash-latest:generateContent?key= ${ geminiKey } ` ;
const aiPayload = {
contents: [{
parts: [{ text: prompt }]
}]
};
try {
const aiRes = UrlFetchApp . fetch ( aiUrl , {
method: 'post' ,
contentType: 'application/json' ,
payload: JSON . stringify ( aiPayload )
});
const aiJson = JSON . parse ( aiRes . getContentText ());
const rawText = aiJson . candidates [ 0 ]. content . parts [ 0 ]. text ;
// Clean markdown code blocks from response
const cleanJson = rawText
. replace ( /```json/ g , '' )
. replace ( /```/ g , '' )
. trim ();
const parsed = JSON . parse ( cleanJson );
// Process parsed data...
} catch ( e ) {
return ContentService . createTextOutput (
JSON . stringify ({
status: "Error" ,
message: "AI Failed: " + e . message
})
). setMimeType ( ContentService . MimeType . JSON );
}
The system uses gemini-flash-latest model which provides fast responses with Gemini’s free tier quota.
AI Response Handling
Success State
if ( data . status === "Success" ) {
icon . innerText = mode === 'add' ? "✅" : "🧠" ;
tip . innerText = data . ai_response ;
if ( mode === 'add' ) {
appData . balance = parseFloat ( data . balance );
if ( data . parsed ) appData . expenses . push ( data . parsed );
renderUI ();
}
}
Error State with Fallback
try {
const res = await fetch ( SCRIPT_URL , {
method: 'POST' ,
body: JSON . stringify ({ action: "process_text" , text: txt , mode: mode })
});
const data = await res . json ();
if ( data . status === "Success" ) {
// Handle success...
} else {
throw new Error ( "API Limit or Error" );
}
} catch ( e ) {
if ( mode === 'add' ) {
handleManualFallback ( txt ); // Regex-based parsing
} else {
icon . innerText = "⚠️" ;
tip . innerText = "AI Offline. Can't analyze right now." ;
}
}
Visual Feedback System
The AI card provides real-time status updates:
Loading State
icon . innerHTML = '<div class="loader"></div>' ;
if ( mode === 'add' ) {
card . className = "card p-4 bg-blue-50 border-blue-200 min-h-[80px]" ;
tip . innerText = "Processing..." ;
tip . className = "text-[11px] font-bold text-blue-800 italic leading-relaxed flex-1" ;
} else {
card . className = "card p-4 bg-purple-50 border-purple-200 min-h-[80px]" ;
tip . innerText = "Analyzing history..." ;
tip . className = "text-[11px] font-bold text-purple-800 italic leading-relaxed flex-1" ;
}
Success State
icon . innerText = mode === 'add' ? "✅" : "🧠" ;
tip . innerText = data . ai_response ; // AI's insight or answer
Fallback State
icon . innerText = "⚡" ;
tip . innerText = `Manual Fallback: Saved " ${ desc } " (₹ ${ amount } )` ;
Smart Insights Examples
The AI generates contextual insights by analyzing your transaction history:
Repeat Expense
High Spend
New Category
Income
Input: Coffee 200
Insight: "Your second coffee this week!"
Input: Dinner 2500
Insight: "3x higher than your average dinner spend"
Input: Gym membership 1500
Insight: "First fitness expense this month!"
Input: Freelance project 15000
Insight: "Nice! This covers 2 weeks of average spending."
Contextual History
The AI receives the last 40 transactions formatted as context:
const historyData = sheet . getRange (
Math . max ( 1 , sheet . getLastRow () - 40 ),
1 , 41 , 4
). getValues ();
const historyContext = historyData
. map ( r => ` ${ r [ 1 ] } ( ${ r [ 2 ] } ) [ ${ r [ 3 ] } ]` )
. join ( " \n " );
// Example output:
// Coffee (200) [Food]
// Uber (150) [Transport]
// Netflix (199) [Tech]
The AI only has access to the last 40 transactions due to Gemini API token limits. For comprehensive analysis spanning months, consider exporting data to CSV.
Example Questions for ASK Mode
“How much did I spend on food this week?”
“What’s my average Uber cost?”
“Show me my tech subscriptions”
“Am I spending more on transport this month?”
“Compare my food expenses to last month”
“What category do I spend most on?”
“How many coffees did I buy this week?”
“What’s my largest expense recently?”
“Am I staying within budget?”
Configuration
Setting up Gemini API Key
Open Settings
Click the “G” button in the top-left corner
Enter API key
Paste your key in the “Gemini API Key” field and click “Save Key to Cloud”
Test it
Try adding an expense to verify the AI is working
< div class = "pt-4 border-t border-slate-100" >
< label class = "block text-xs font-bold text-slate-400 uppercase mb-2" > Gemini API Key </ label >
< input type = "password" id = "geminiKeyInput" placeholder = "Enter new key"
class = "w-full p-3 bg-slate-50 rounded-lg text-xs font-mono outline-none mb-2" >
< button onclick = " pushSettings ( true )"
class = "w-full p-3 bg-slate-900 text-white rounded-lg font-bold text-xs" >
Save Key to Cloud
</ button >
</ div >
Best Practices
Be specific “Coffee at Starbucks 250” works better than just “coffee”
Include context “Uber to airport” helps AI categorize correctly
Review insights AI insights help identify spending patterns
Ask natural questions ASK mode understands conversational queries