Data Storage - Price Tracker Bot

Overview

The Price Tracker Bot uses a file-based JSON storage system with in-memory caching. All data is stored in a single file (prices.json) located in the bot’s working directory.

Trade-offs: File-based storage is simple and requires no external dependencies, but may not scale well beyond ~1000 products or multiple concurrent bot instances.

Storage Architecture

In-Memory State

Two global variables maintain runtime state:

let priceData = {};  // Object keyed by sanitized product URLs
let chats = new Set();  // Set of active Telegram chat IDs

Location: index.mjs:21-22

Persistence Functions

`loadData()`

Loads data from disk on bot startup:

function loadData() {
  if (fs.existsSync(DATA_FILE)) {
    try {
      const saved = JSON.parse(fs.readFileSync(DATA_FILE, "utf8"));
      priceData = saved.products || {};
      chats = new Set(saved.chats || []);
      console.log(`✅ Datos cargados. Productos: ${Object.keys(priceData).length}, Chats: ${chats.size}`);
    } catch (err) {
      console.error("❌ Error al cargar datos:", err.message);
      priceData = {};
      chats = new Set();
    }
  } else {
    console.log("ℹ️ No existe prices.json — se creará al guardar.");
  }
}

Location: index.mjs:24-39 Error handling:

If file doesn’t exist, initializes empty state
If JSON parsing fails, resets to empty state
Logs success/failure status

loadData() is called once at startup (index.mjs:50). Data is never reloaded during runtime.

`saveData()`

Persists current state to disk:

function saveData() {
  try {
    fs.writeFileSync(DATA_FILE, JSON.stringify({
      products: priceData,
      chats: [...chats]
    }, null, 2));
  } catch (err) {
    console.error("❌ Error guardando datos:", err.message);
  }
}

Location: index.mjs:41-48 Characteristics:

Synchronous write (blocks until complete)
Pretty-printed JSON (2-space indentation)
Converts Set to array for serialization
Silently fails on error (only logs)

saveData() is called after every mutation: adding products, updating prices, removing products, or registering chats.

File Structure

Root Schema

The prices.json file has two top-level keys:

{
  "products": {
    "https://amazon.com/dp/B08N5WRWNW": { /* product object */ },
    "https://amazon.com/dp/B09JQMJHXY": { /* product object */ }
  },
  "chats": [123456789, 987654321]
}

Field	Type	Description
`products`	Object	Map of sanitized URLs to product objects
`chats`	Array	List of active Telegram chat IDs (numbers)

Product Object Schema

Each entry in the products object has this structure:

{
  url: "https://amazon.com/dp/B08N5WRWNW",
  title: "Sony WH-1000XM4 Wireless Headphones",
  price: 278.00,
  lowestPrice: 248.00,
  imageUrl: "https://m.media-amazon.com/images/I/71o8Q5XJS5L._AC_SL1500_.jpg",
  lastChecked: "2026-03-10T15:30:45.123Z",
  addedDate: "2026-03-01T10:20:15.000Z",
  addedBy: 123456789,
  history: [
    { date: "2026-03-01T10:20:15.000Z", price: 298.00 },
    { date: "2026-03-01T18:30:20.456Z", price: 295.00 },
    { date: "2026-03-02T08:15:33.789Z", price: 278.00 }
  ]
}

Field Definitions

url (string)

Sanitized Amazon product URL (no query parameters or hash fragments)

Generated by sanitizeAmazonURL() (index.mjs:53-61)
Used as the object key in priceData
Example: https://amazon.com/dp/B08N5WRWNW

title (string)

Product title extracted from Amazon page

Scraped from multiple selectors (see Web Scraping)
Displayed in notifications and product lists
Example: "Sony WH-1000XM4 Wireless Headphones"

price (number)

Current price in USD (or currency detected from page)

Parsed from price text by removing non-numeric characters
Updated during each price check
Used for price drop detection
Example: 278.00

lowestPrice (number)

Lowest price ever seen for this product

Initialized to first scraped price
Updated when scraped.price < stored.lowestPrice
Used in notifications to show historical context
Example: 248.00

Update logic (index.mjs:190):

lowestPrice: Math.min(scraped.price, originalLowest)

imageUrl (string | null)

Product image URL from Amazon

Scraped from multiple image selectors
May be null if no image found
Used in Telegram messages with bot.sendPhoto()
Example: "https://m.media-amazon.com/images/I/71o8Q5XJS5L._AC_SL1500_.jpg"

lastChecked (string)

ISO 8601 timestamp of last price check

Updated even if scraping fails
Used to display “last updated” information
Format: YYYY-MM-DDTHH:mm:ss.sssZ
Example: "2026-03-10T15:30:45.123Z"

Set with (index.mjs:187):

lastChecked: new Date().toISOString()

addedDate (string)

ISO 8601 timestamp when product was added

Set once when product is first tracked
Preserved during URL edits
Used in product details view
Example: "2026-03-01T10:20:15.000Z"

addedBy (number | null)

Telegram chat ID of user who added the product

Used for attribution (not currently displayed)
May be null for legacy products
Example: 123456789

history (array)

Array of price history entries (max 120)

Each entry: { date: string, price: number }
Appended on every price check (even if price unchanged)
Trimmed to last 120 entries when limit exceeded
Used for chart generation

Example:

[
  { "date": "2026-03-01T10:20:15.000Z", "price": 298.00 },
  { "date": "2026-03-01T18:30:20.456Z", "price": 295.00 },
  { "date": "2026-03-02T08:15:33.789Z", "price": 278.00 }
]

Limit enforcement (index.mjs:196):

if (updated.history.length > HISTORY_LIMIT) {
  updated.history = updated.history.slice(-HISTORY_LIMIT);
}

Data Operations

Creating a Product

When a user runs /add [url], a new product object is created:

priceData[sanitized] = {
  url: sanitized,
  title: scraped.title,
  price: scraped.price,
  lowestPrice: scraped.price,
  imageUrl: scraped.imageUrl || null,
  addedDate: new Date().toISOString(),
  addedBy: chatId,
  lastChecked: new Date().toISOString(),
  history: [{ date: new Date().toISOString(), price: scraped.price }]
};

saveData();

Location: index.mjs:427-439

The initial history array contains a single entry with the scraped price.

Updating a Product

During price checks, products are updated in place:

const updated = {
  url: sanitized,
  title: scraped.title,
  price: scraped.price,
  imageUrl: scraped.imageUrl || stored.imageUrl || null,
  lastChecked: new Date().toISOString(),
  addedDate: stored.addedDate || new Date().toISOString(),
  addedBy: stored.addedBy || null,
  lowestPrice: Math.min(scraped.price, originalLowest),
  history: Array.isArray(stored.history) ? stored.history.slice() : []
};

// Append history
updated.history.push({ date: new Date().toISOString(), price: scraped.price });
if (updated.history.length > HISTORY_LIMIT) {
  updated.history = updated.history.slice(-HISTORY_LIMIT);
}

priceData[finalKey] = updated;
saveData();

Location: index.mjs:182-218 Key behaviors:

Preserves addedDate and addedBy from previous version
Falls back to stored imageUrl if scraper doesn’t find one
Always appends to history (even if price unchanged)
Recalculates lowestPrice on every update

Deleting a Product

Products are removed using JavaScript’s delete operator:

delete priceData[sanitized];
saveData();

Location: index.mjs:527, index.mjs:536, index.mjs:667

When deleting all products (delete_all callback), the entire priceData object is replaced:

priceData = {};
saveData();

Editing a Product URL

The /edit command creates a new entry and deletes the old one:

// Create new entry
priceData[newKey] = {
  url: newKey,
  title: scraped.title,
  price: scraped.price,
  lowestPrice: Math.min(scraped.price, lowest),
  imageUrl: scraped.imageUrl || prev.imageUrl || null,
  addedDate: prev.addedDate || new Date().toISOString(),
  addedBy: prev.addedBy || chatId,
  lastChecked: new Date().toISOString(),
  history: Array.isArray(prev.history) ? prev.history.slice() : []
};

// Append new history entry
priceData[newKey].history.push({ 
  date: new Date().toISOString(), 
  price: scraped.price 
});

// Delete old entry
delete priceData[oldKey];
saveData();

Location: index.mjs:578-596

The entire history is preserved when editing URLs, allowing users to fix incorrect URLs without losing data.

Chat Tracking

The chats Set stores Telegram chat IDs for notification delivery:

let chats = new Set();

Location: index.mjs:22

Registration

Chats are registered automatically in two ways:

On /start command (index.mjs:361-364):

if (!chats.has(chatId)) {
  chats.add(chatId);
  saveData();
}

On any message (index.mjs:705-711):

bot.on("message", (msg) => {
  const chatId = msg.chat.id;
  if (!chats.has(chatId)) {
    chats.add(chatId);
    saveData();
  }
});

Usage

Registered chats receive notifications during price checks:

for (const chatId of chats) {
  for (const p of productsChanged) {
    await bot.sendPhoto(chatId, p.imageUrl, {
      caption: p.message,
      parse_mode: "Markdown"
    });
  }
}

Location: index.mjs:223-244

There is currently no /unsubscribe command. Once registered, a chat continues receiving notifications indefinitely.

History Management

History Limit

The HISTORY_LIMIT constant controls how many price points are retained:

const HISTORY_LIMIT = 120;

Location: index.mjs:69

Trimming Logic

When appending history, older entries are removed if limit exceeded:

updated.history.push({ date: new Date().toISOString(), price: scraped.price });
if (updated.history.length > HISTORY_LIMIT) {
  updated.history = updated.history.slice(-HISTORY_LIMIT);
}

Location: index.mjs:195-196

Array.slice(-HISTORY_LIMIT) keeps only the last 120 entries, discarding older ones.

Chart Generation

History data is sorted chronologically before charting:

const sorted = product.history
  .map(h => ({ date: new Date(h.date), price: h.price }))
  .sort((a, b) => a.date - b.date);

const labels = sorted.map(s => 
  s.date.toLocaleString("es-MX", { 
    day: "2-digit", 
    month: "short", 
    hour: "2-digit", 
    minute: "2-digit" 
  })
);
const prices = sorted.map(s => s.price);

Location: index.mjs:317-320

Data Integrity

Sanitization on Keys

All URLs used as keys are sanitized to prevent duplicates:

const sanitized = sanitizeAmazonURL(raw);
if (priceData[sanitized]) {
  return bot.sendMessage(chatId, "⚠️ Este producto ya está en seguimiento");
}

Example: These URLs become the same key:

https://amazon.com/dp/B08N5WRWNW?tag=xyz
https://amazon.com/dp/B08N5WRWNW#reviews
https://amazon.com/dp/B08N5WRWNW

Fallback Values

The code defensively handles missing fields:

title: stored.title || u,
imageUrl: scraped.imageUrl || stored.imageUrl || null,
addedDate: stored.addedDate || new Date().toISOString(),
addedBy: stored.addedBy || null,
history: Array.isArray(stored.history) ? stored.history.slice() : []

Array Cloning

The history array is cloned to prevent mutations:

history: Array.isArray(stored.history) ? stored.history.slice() : []

Without cloning, modifications would affect the original array.

Migration Considerations

Adding New Fields

New fields can be added with default values:

const updated = {
  // ... existing fields
  newField: stored.newField || "default value"
};

Old products will automatically get the default value.

Changing Schema

For breaking changes, implement a migration function:

function migrateData() {
  for (const key in priceData) {
    const product = priceData[key];
    
    // Example: rename field
    if (product.oldFieldName) {
      product.newFieldName = product.oldFieldName;
      delete product.oldFieldName;
    }
  }
  
  saveData();
}

loadData();
migrateData();

Backup Strategy

For production deployments, implement periodic backups:

function backupData() {
  const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
  const backupFile = `prices-backup-${timestamp}.json`;
  fs.copyFileSync(DATA_FILE, backupFile);
}

cron.schedule("0 0 * * *", backupData); // Daily at midnight

Performance Analysis

File Size Estimation

With 120 history entries per product:

One product ≈ 500 bytes (base fields) + 120 × 50 bytes (history) = 6.5 KB
100 products ≈ 650 KB
1000 products ≈ 6.5 MB

JSON parsing performance degrades around 10-50 MB depending on hardware. Consider migrating to a database beyond 1000 products.

Write Frequency

Data is written to disk on:

Every /add command
Every /remove command
Every /edit command
Every price check cycle (every 2 hours)
Every chat registration
Every delete_all operation

With 100 products and 2-hour checks:

Writes per day ≈ 12 (price checks) + ~10 (user commands) = ~22 writes/day

Optimization Opportunities

Lazy writes: Buffer changes and write every N seconds
Incremental saves: Use append-only logs (event sourcing)
Database migration: PostgreSQL, MongoDB, or Redis
Compression: gzip the JSON file
Indexing: Add secondary indices for faster lookups

Example Data File

A complete prices.json example:

{
  "products": {
    "https://amazon.com/dp/B08N5WRWNW": {
      "url": "https://amazon.com/dp/B08N5WRWNW",
      "title": "Sony WH-1000XM4 Wireless Headphones",
      "price": 278,
      "lowestPrice": 248,
      "imageUrl": "https://m.media-amazon.com/images/I/71o8Q5XJS5L.jpg",
      "lastChecked": "2026-03-10T15:30:45.123Z",
      "addedDate": "2026-03-01T10:20:15.000Z",
      "addedBy": 123456789,
      "history": [
        { "date": "2026-03-01T10:20:15.000Z", "price": 298 },
        { "date": "2026-03-01T18:30:20.456Z", "price": 295 },
        { "date": "2026-03-02T08:15:33.789Z", "price": 278 }
      ]
    },
    "https://amazon.com/dp/B09JQMJHXY": {
      "url": "https://amazon.com/dp/B09JQMJHXY",
      "title": "Apple AirPods Pro (2nd Generation)",
      "price": 199,
      "lowestPrice": 199,
      "imageUrl": "https://m.media-amazon.com/images/I/61SUj2aKoEL.jpg",
      "lastChecked": "2026-03-10T15:31:12.789Z",
      "addedDate": "2026-03-05T14:45:30.000Z",
      "addedBy": 987654321,
      "history": [
        { "date": "2026-03-05T14:45:30.000Z", "price": 199 }
      ]
    }
  },
  "chats": [
    123456789,
    987654321
  ]
}

Getting Started

Features

Commands

Configuration

Technical Details

Support

​Overview

​Storage Architecture

​In-Memory State

​Persistence Functions

​loadData()

​saveData()

​File Structure

​Root Schema

​Product Object Schema

​Field Definitions

​Data Operations

​Creating a Product

​Updating a Product

​Deleting a Product

​Editing a Product URL

​Chat Tracking

​Registration

​Usage

​History Management

​History Limit

​Trimming Logic

​Chart Generation

​Data Integrity

​Sanitization on Keys

​Fallback Values

​Array Cloning

​Migration Considerations

​Adding New Fields

​Changing Schema

​Backup Strategy

​Performance Analysis

​File Size Estimation

​Write Frequency

​Optimization Opportunities

​Example Data File

Build docs developers (and LLMs) love

Overview

Storage Architecture

In-Memory State

Persistence Functions

`loadData()`

`saveData()`

File Structure

Root Schema

Product Object Schema

Field Definitions

Data Operations

Creating a Product

Updating a Product

Deleting a Product

Editing a Product URL

Chat Tracking

Registration

Usage

History Management

History Limit

Trimming Logic

Chart Generation

Data Integrity

Sanitization on Keys

Fallback Values

Array Cloning

Migration Considerations

Adding New Fields

Changing Schema

Backup Strategy

Performance Analysis

File Size Estimation

Write Frequency

Optimization Opportunities

Example Data File