External Integrations

Overview

HTTP Ledger provides two powerful integration points: getIpInfo for enriching logs with IP geolocation data, and onLog for sending logs to external services. These enable seamless integration with your existing infrastructure.

getIpInfo

(ip: string) => Promise<IpInfo>

Optional async function that takes an IP address and returns IP information like country, city, region, and timezone.

Basic Usage

app.use(logger({
  getIpInfo: async (ip) => {
    const response = await fetch(`https://ipapi.co/${ip}/json/`);
    return response.json();
  }
}));

Popular IP Geolocation Services

ipapi.co
ip-api.com
ipinfo.io

app.use(logger({
  getIpInfo: async (ip) => {
    try {
      const response = await fetch(`https://ipapi.co/${ip}/json/`);
      const data = await response.json();
      return {
        ip: data.ip,
        country: data.country_name,
        city: data.city,
        region: data.region,
        timezone: data.timezone
      };
    } catch (error) {
      console.error('IP lookup failed:', error);
      return {}; // Return empty object on error
    }
  }
}));

app.use(logger({
  getIpInfo: async (ip) => {
    try {
      const response = await fetch(`http://ip-api.com/json/${ip}`);
      const data = await response.json();
      return {
        ip: data.query,
        country: data.country,
        city: data.city,
        region: data.regionName,
        timezone: data.timezone
      };
    } catch (error) {
      return {};
    }
  }
}));

app.use(logger({
  getIpInfo: async (ip) => {
    try {
      const response = await fetch(
        `https://ipinfo.io/${ip}/json?token=${process.env.IPINFO_TOKEN}`
      );
      const data = await response.json();
      return {
        ip: data.ip,
        country: data.country,
        city: data.city,
        region: data.region,
        timezone: data.timezone
      };
    } catch (error) {
      return {};
    }
  }
}));

Caching IP Lookups

IP lookups can be expensive. Implement caching for better performance:

const NodeCache = require('node-cache');
const ipCache = new NodeCache({ stdTTL: 3600 }); // Cache for 1 hour

app.use(logger({
  getIpInfo: async (ip) => {
    // Check cache first
    const cached = ipCache.get(ip);
    if (cached) return cached;
    
    try {
      const response = await fetch(`https://ipapi.co/${ip}/json/`);
      const data = await response.json();
      
      const ipInfo = {
        ip: data.ip,
        country: data.country_name,
        city: data.city,
        region: data.region,
        timezone: data.timezone
      };
      
      // Store in cache
      ipCache.set(ip, ipInfo);
      
      return ipInfo;
    } catch (error) {
      return {};
    }
  }
}));

With Timeout

Prevent slow IP lookups from blocking:

const fetchWithTimeout = (url, timeout = 1000) => {
  return Promise.race([
    fetch(url),
    new Promise((_, reject) => 
      setTimeout(() => reject(new Error('Timeout')), timeout)
    )
  ]);
};

app.use(logger({
  getIpInfo: async (ip) => {
    try {
      const response = await fetchWithTimeout(
        `https://ipapi.co/${ip}/json/`,
        1000 // 1 second timeout
      );
      return response.json();
    } catch (error) {
      console.warn('IP lookup timeout or error:', error.message);
      return {};
    }
  }
}));

Example Log Output

With IP information enrichment:

{
  "method": "GET",
  "url": "/api/products",
  "statusCode": 200,
  "timeTaken": 45.23,
  "ipInfo": {
    "ip": "203.0.113.0",
    "country": "United States",
    "city": "San Francisco",
    "region": "California",
    "timezone": "America/Los_Angeles"
  }
}

onLog

(logData: LogData) => void | Promise<void>

Optional callback that receives the complete log data after each request. Use this to send logs to external services, databases, or monitoring platforms.

Basic Usage

app.use(logger({
  onLog: async (logData) => {
    // Send to external logging service
    await fetch('https://logs.example.com/api', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(logData)
    });
  }
}));

Integration Examples

Elasticsearch
MongoDB
Datadog
CloudWatch

const { Client } = require('@elastic/elasticsearch');
const client = new Client({ node: 'http://localhost:9200' });

app.use(logger({
  onLog: async (logData) => {
    try {
      await client.index({
        index: 'http-logs',
        document: {
          '@timestamp': logData.timestamp.request,
          method: logData.method,
          url: logData.url,
          status_code: logData.statusCode,
          response_time: logData.timeTaken,
          user_agent: logData.userAgent,
          geo: logData.ipInfo
        }
      });
    } catch (error) {
      console.error('Failed to send to Elasticsearch:', error);
    }
  }
}));

const { MongoClient } = require('mongodb');
const client = new MongoClient(process.env.MONGODB_URI);
const db = client.db('logs');
const collection = db.collection('http_requests');

app.use(logger({
  onLog: async (logData) => {
    try {
      await collection.insertOne({
        ...logData,
        timestamp: new Date(logData.timestamp.request)
      });
    } catch (error) {
      console.error('Failed to save to MongoDB:', error);
    }
  }
}));

const StatsD = require('hot-shots');
const dogstatsd = new StatsD();

app.use(logger({
  onLog: (logData) => {
    // Send metrics to Datadog
    dogstatsd.increment('http.requests', 1, [
      `method:${logData.method}`,
      `status:${logData.statusCode}`,
      `path:${logData.url}`
    ]);
    
    dogstatsd.histogram('http.response_time', logData.timeTaken, [
      `method:${logData.method}`
    ]);
    
    if (logData.statusCode >= 400) {
      dogstatsd.increment('http.errors', 1, [
        `status:${logData.statusCode}`
      ]);
    }
  }
}));

const AWS = require('aws-sdk');
const cloudwatch = new AWS.CloudWatchLogs();

app.use(logger({
  onLog: async (logData) => {
    try {
      await cloudwatch.putLogEvents({
        logGroupName: '/aws/lambda/my-api',
        logStreamName: new Date().toISOString().split('T')[0],
        logEvents: [{
          timestamp: Date.now(),
          message: JSON.stringify(logData)
        }]
      }).promise();
    } catch (error) {
      console.error('Failed to send to CloudWatch:', error);
    }
  }
}));

Batching Logs

Batch multiple logs before sending to reduce network overhead:

const logBatch = [];
const BATCH_SIZE = 100;
const FLUSH_INTERVAL = 5000; // 5 seconds

const flushLogs = async () => {
  if (logBatch.length === 0) return;
  
  const logsToSend = [...logBatch];
  logBatch.length = 0;
  
  try {
    await fetch('https://logs.example.com/batch', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(logsToSend)
    });
  } catch (error) {
    console.error('Failed to send batch:', error);
  }
};

// Flush periodically
setInterval(flushLogs, FLUSH_INTERVAL);

app.use(logger({
  onLog: (logData) => {
    logBatch.push(logData);
    
    // Flush if batch is full
    if (logBatch.length >= BATCH_SIZE) {
      flushLogs();
    }
  }
}));

// Flush on process exit
process.on('SIGTERM', async () => {
  await flushLogs();
  process.exit(0);
});

Error Tracking

Send only errors to error tracking services:

const Sentry = require('@sentry/node');

app.use(logger({
  onLog: (logData) => {
    // Only send errors to Sentry
    if (logData.statusCode >= 400 || logData.error) {
      Sentry.captureMessage(`HTTP ${logData.statusCode}: ${logData.method} ${logData.url}`, {
        level: logData.statusCode >= 500 ? 'error' : 'warning',
        extra: {
          method: logData.method,
          url: logData.url,
          statusCode: logData.statusCode,
          timeTaken: logData.timeTaken,
          error: logData.error,
          requestId: logData.requestId
        }
      });
    }
  }
}));

Error Handling

Both getIpInfo and onLog are async-safe. If they throw errors, the middleware continues logging normally. Errors are logged to console but don’t break request handling.

From src/index.ts:151-158 and 189-195:

// Get IP info if function is provided
let ipInfo: Record<string, any> = {};
if (getIpInfo && typeof getIpInfo === 'function') {
  try {
    const ip = req.ip || req.connection.remoteAddress || '';
    ipInfo = await getIpInfo(ip);
  } catch (ipError) {
    console.warn('Failed to get IP info:', ipError);
  }
}

// Call onLog callback if provided
if (onLog) {
  try {
    await onLog(logData);
  } catch (cbErr) {
    console.warn('onLog callback threw:', cbErr);
  }
}

Complete Integration Example

import express from 'express';
import logger, { LogData, IpInfo } from 'http-ledger';
import { Client as ElasticsearchClient } from '@elastic/elasticsearch';
import NodeCache from 'node-cache';

const app = express();
const ipCache = new NodeCache({ stdTTL: 3600 });
const esClient = new ElasticsearchClient({ node: 'http://localhost:9200' });

app.use(logger({
  // IP geolocation with caching
  getIpInfo: async (ip: string): Promise<IpInfo> => {
    const cached = ipCache.get<IpInfo>(ip);
    if (cached) return cached;
    
    try {
      const response = await fetch(`https://ipapi.co/${ip}/json/`, {
        signal: AbortSignal.timeout(1000)
      });
      const data = await response.json();
      
      const ipInfo: IpInfo = {
        ip: data.ip,
        country: data.country_name,
        city: data.city,
        region: data.region,
        timezone: data.timezone
      };
      
      ipCache.set(ip, ipInfo);
      return ipInfo;
    } catch {
      return {};
    }
  },
  
  // Send to Elasticsearch
  onLog: async (logData: LogData) => {
    try {
      await esClient.index({
        index: `http-logs-${new Date().toISOString().split('T')[0]}`,
        document: {
          '@timestamp': logData.timestamp.request,
          http: {
            method: logData.method,
            url: logData.url,
            status_code: logData.statusCode,
            response_time_ms: logData.timeTaken,
            request_size_bytes: logData.requestSize,
            response_size_bytes: logData.responseSize,
            version: logData.httpVersion
          },
          user_agent: logData.userAgent,
          geo: logData.ipInfo,
          error: logData.error,
          request_id: logData.requestId
        }
      });
    } catch (error) {
      console.error('Elasticsearch indexing failed:', error);
    }
  },
  
  // Other options
  maskFields: ['password', 'token'],
  autoGenerateRequestId: true,
  logSampling: 0.1
}));

app.listen(3000);

Best Practices

Always Return Empty Object

In getIpInfo, return {} on errors instead of throwing

Use Timeouts

Set timeouts for IP lookups to prevent slow requests

Implement Caching

Cache IP lookup results to reduce external API calls

Handle onLog Errors

Wrap external service calls in try-catch blocks

Batch When Possible

Batch logs before sending to reduce network overhead

Monitor Integration Health

Track success/failure rates of external integrations

Custom Logging

Format logs before sending to external services

Production Setup

Complete production integration examples

Getting Started

Configuration

Guides

External Integrations

Overview

getIpInfo

Basic Usage

Popular IP Geolocation Services

Caching IP Lookups

With Timeout

Example Log Output

onLog

Basic Usage

Integration Examples

Batching Logs

Error Tracking

Error Handling

Complete Integration Example

Best Practices

Always Return Empty Object

Use Timeouts

Implement Caching

Handle onLog Errors

Batch When Possible

Monitor Integration Health

Custom Logging

Production Setup

Build docs developers (and LLMs) love

Getting Started

Configuration

Guides

​Overview

​getIpInfo

​Basic Usage

​Popular IP Geolocation Services

​Caching IP Lookups

​With Timeout

​Example Log Output

​onLog

​Basic Usage

​Integration Examples

​Batching Logs

​Error Tracking

​Error Handling

​Complete Integration Example

​Best Practices

Always Return Empty Object

Use Timeouts

Implement Caching

Handle onLog Errors

Batch When Possible

Monitor Integration Health

​Related Configuration

Custom Logging

Production Setup

Build docs developers (and LLMs) love

Overview

getIpInfo

Basic Usage

Popular IP Geolocation Services

Caching IP Lookups

With Timeout

Example Log Output

onLog

Basic Usage

Integration Examples

Batching Logs

Error Tracking

Error Handling

Complete Integration Example

Best Practices

Related Configuration