Skip to main content

Overview

SpendWisely George’s AI assistant uses Google’s Gemini Flash model to provide intelligent expense parsing, contextual insights, and financial analysis through natural language interaction.

Core Capabilities

Expense Parsing

Extracts amount, description, and category from natural language

Smart Categorization

Automatically assigns expenses to Food, Transport, Tech, etc.

Contextual Insights

Compares new expenses to spending history

Financial Analysis

Answers questions about your spending patterns

Two Modes of Operation

ADD Mode: Expense Entry

Processes natural language input to create structured expense entries.

User Flow

1

User enters text

Coffee 200
2

Frontend sends to Apps Script

const res = await fetch(SCRIPT_URL, {
    method: 'POST',
    body: JSON.stringify({ 
        action: "process_text", 
        text: "Coffee 200", 
        mode: "add" 
    })
});
3

AI extracts structured data

{
  "description": "Coffee",
  "amount": 200,
  "category": "Food",
  "type": "DEBIT",
  "insight": "Your second coffee this week!"
}
4

Saved to Google Sheets

Transaction is appended to sheet and balance is recalculated

ADD Mode Prompt Engineering

The system provides rich context to the AI:
google_script.js
if (mode === 'add') {
    // Fetch last 40 transactions for context
    const historyData = sheet.getRange(
        Math.max(1, sheet.getLastRow() - 40), 
        1, 41, 4
    ).getValues();
    const historyContext = historyData
        .map(r => `${r[1]} (${r[2]}) [${r[3]}]`)
        .join("\n");

    prompt = `
You are a financial assistant. 
CONTEXT: The user wants to ADD a transaction.
USER INPUT: "${userText}"
HISTORY: 
${historyContext}

TASK:
1. Extract: Description, Amount, Category (Food, Transport, Tech, Invest, Income, Misc).
2. Analyze: Compare this new expense to the HISTORY. Is it higher than usual? Is it a repeat?
3. Insight: Write a very short, witty, or helpful insight (max 15 words) about this specific spend.

OUTPUT JSON STRICTLY:
{
  "description": "String",
  "amount": Number,
  "category": "String",
  "type": "DEBIT" or "CREDIT",
  "insight": "String"
}
`;
}

ASK Mode: Financial Analysis

Answers questions about spending patterns using transaction history.

User Flow

1

User asks question

How much did I spend on food this month?
2

AI analyzes transaction history

The system provides the last 40 transactions as context
3

Returns concise answer

{
  "answer": "You spent ₹3,450 on food across 12 transactions this month."
}

ASK Mode Implementation

google_script.js
if (mode === 'ask') {
    prompt = `
You are a financial analyst.
CONTEXT: The user is ASKING a question about their finances.
USER INPUT: "${userText}"
HISTORY: 
${historyContext}

TASK: 
Answer the user's question based strictly on the HISTORY data provided. 
Be concise (max 30 words).

OUTPUT JSON STRICTLY:
{
  "answer": "String"
}
`;
}
index.html
if (mode === 'ask') {
    card.className = "card p-4 bg-purple-50 border-purple-200 min-h-[80px]";
    tip.innerText = "Analyzing history...";
    tip.className = "text-[11px] font-bold text-purple-800 italic leading-relaxed flex-1";
}

// After AI response
if (data.status === "Success") {
    icon.innerText = "🧠";
    tip.innerText = data.ai_response;
}

Voice Input

Browser Speech Recognition

The app uses the Web Speech API for voice-to-text input:
index.html
const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
let recognition = null;

if (SpeechRecognition) {
    recognition = new SpeechRecognition();
    recognition.continuous = false;
    recognition.lang = 'en-IN';  // Indian English
    recognition.interimResults = false;
    
    recognition.onstart = () => { 
        document.getElementById('micBtn').classList.add('mic-active'); 
    };
    
    recognition.onend = () => { 
        document.getElementById('micBtn').classList.remove('mic-active'); 
    };
    
    recognition.onresult = (event) => {
        const transcript = event.results[0][0].transcript;
        const current = document.getElementById('omniInput').value;
        document.getElementById('omniInput').value = current 
            ? current + " " + transcript 
            : transcript;
    };
}

function toggleVoice() {
    if (recognition) recognition.start();
    else alert("Voice recognition not supported on this browser.");
}

Visual Feedback

The microphone button pulses red during recording:
index.html
.mic-active {
    animation: pulse-red 1.5s infinite;
    color: #ef4444 !important;
}

@keyframes pulse-red {
    0% {
        box-shadow: 0 0 0 0 rgba(239, 68, 68, 0.7);
    }
    70% {
        box-shadow: 0 0 0 10px rgba(239, 68, 68, 0);
    }
    100% {
        box-shadow: 0 0 0 0 rgba(239, 68, 68, 0);
    }
}

Gemini AI Integration

API Configuration

Users configure their Gemini API key in Settings:
google_script.js
const SCRIPT_PROP = PropertiesService.getScriptProperties();

// Saving API key
if (body.action === "save_settings") {
    SCRIPT_PROP.setProperty("DAILY_BUDGET", body.budget.toString());
    SCRIPT_PROP.setProperty("PRIVACY_MODE", body.privacy.toString());
    if (body.geminiKey) SCRIPT_PROP.setProperty("GEMINI_KEY", body.geminiKey);
    return ContentService.createTextOutput(
        JSON.stringify({ status: "Saved" })
    ).setMimeType(ContentService.MimeType.JSON);
}

// Retrieving for AI calls
const geminiKey = SCRIPT_PROP.getProperty("GEMINI_KEY");

API Request Structure

google_script.js
const geminiKey = SCRIPT_PROP.getProperty("GEMINI_KEY");
const aiUrl = `https://generativelanguage.googleapis.com/v1beta/models/gemini-flash-latest:generateContent?key=${geminiKey}`;

const aiPayload = { 
    contents: [{ 
        parts: [{ text: prompt }] 
    }] 
};

try {
    const aiRes = UrlFetchApp.fetch(aiUrl, { 
        method: 'post', 
        contentType: 'application/json', 
        payload: JSON.stringify(aiPayload) 
    });
    
    const aiJson = JSON.parse(aiRes.getContentText());
    const rawText = aiJson.candidates[0].content.parts[0].text;
    
    // Clean markdown code blocks from response
    const cleanJson = rawText
        .replace(/```json/g, '')
        .replace(/```/g, '')
        .trim();
    
    const parsed = JSON.parse(cleanJson);
    // Process parsed data...
} catch (e) {
    return ContentService.createTextOutput(
        JSON.stringify({ 
            status: "Error", 
            message: "AI Failed: " + e.message 
        })
    ).setMimeType(ContentService.MimeType.JSON);
}
The system uses gemini-flash-latest model which provides fast responses with Gemini’s free tier quota.

AI Response Handling

Success State

index.html
if (data.status === "Success") {
    icon.innerText = mode === 'add' ? "✅" : "🧠";
    tip.innerText = data.ai_response;
    
    if (mode === 'add') {
        appData.balance = parseFloat(data.balance);
        if (data.parsed) appData.expenses.push(data.parsed);
        renderUI();
    }
}

Error State with Fallback

index.html
try {
    const res = await fetch(SCRIPT_URL, {
        method: 'POST',
        body: JSON.stringify({ action: "process_text", text: txt, mode: mode })
    });
    const data = await res.json();
    
    if (data.status === "Success") {
        // Handle success...
    } else {
        throw new Error("API Limit or Error");
    }
} catch (e) {
    if (mode === 'add') {
        handleManualFallback(txt);  // Regex-based parsing
    } else {
        icon.innerText = "⚠️";
        tip.innerText = "AI Offline. Can't analyze right now.";
    }
}

Visual Feedback System

The AI card provides real-time status updates:

Loading State

index.html
icon.innerHTML = '<div class="loader"></div>';

if (mode === 'add') {
    card.className = "card p-4 bg-blue-50 border-blue-200 min-h-[80px]";
    tip.innerText = "Processing...";
    tip.className = "text-[11px] font-bold text-blue-800 italic leading-relaxed flex-1";
} else {
    card.className = "card p-4 bg-purple-50 border-purple-200 min-h-[80px]";
    tip.innerText = "Analyzing history...";
    tip.className = "text-[11px] font-bold text-purple-800 italic leading-relaxed flex-1";
}

Success State

icon.innerText = mode === 'add' ? "✅" : "🧠";
tip.innerText = data.ai_response;  // AI's insight or answer

Fallback State

icon.innerText = "⚡";
tip.innerText = `Manual Fallback: Saved "${desc}" (₹${amount})`;

Smart Insights Examples

The AI generates contextual insights by analyzing your transaction history:
Input: Coffee 200
Insight: "Your second coffee this week!"

Contextual History

The AI receives the last 40 transactions formatted as context:
google_script.js
const historyData = sheet.getRange(
    Math.max(1, sheet.getLastRow() - 40), 
    1, 41, 4
).getValues();

const historyContext = historyData
    .map(r => `${r[1]} (${r[2]}) [${r[3]}]`)
    .join("\n");

// Example output:
// Coffee (200) [Food]
// Uber (150) [Transport]
// Netflix (199) [Tech]
The AI only has access to the last 40 transactions due to Gemini API token limits. For comprehensive analysis spanning months, consider exporting data to CSV.

Example Questions for ASK Mode

  • “How much did I spend on food this week?”
  • “What’s my average Uber cost?”
  • “Show me my tech subscriptions”
  • “Am I spending more on transport this month?”
  • “Compare my food expenses to last month”
  • “What category do I spend most on?”

Configuration

Setting up Gemini API Key

1

Get API key

Visit Google AI Studio to generate a free Gemini API key
2

Open Settings

Click the “G” button in the top-left corner
3

Enter API key

Paste your key in the “Gemini API Key” field and click “Save Key to Cloud”
4

Test it

Try adding an expense to verify the AI is working
index.html
<div class="pt-4 border-t border-slate-100">
    <label class="block text-xs font-bold text-slate-400 uppercase mb-2">Gemini API Key</label>
    <input type="password" id="geminiKeyInput" placeholder="Enter new key"
        class="w-full p-3 bg-slate-50 rounded-lg text-xs font-mono outline-none mb-2">
    <button onclick="pushSettings(true)"
        class="w-full p-3 bg-slate-900 text-white rounded-lg font-bold text-xs">
        Save Key to Cloud
    </button>
</div>

Best Practices

Be specific

“Coffee at Starbucks 250” works better than just “coffee”

Include context

“Uber to airport” helps AI categorize correctly

Review insights

AI insights help identify spending patterns

Ask natural questions

ASK mode understands conversational queries

Build docs developers (and LLMs) love