Monitor Duckling’s operations through logs and metrics endpoints.
Logs API
Get server logs
Maximum number of log entries to return
Number of log entries to skip
Filter by log level: debug, info, warn, error
curl http://localhost:3001/api/logs?limit= 50 & level = error \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"logs" : [
{
"timestamp" : "2024-03-01T10:30:00Z" ,
"level" : "error" ,
"message" : "Connection timeout to MySQL" ,
"context" : {
"database" : "lms" ,
"duration" : "30000ms"
}
}
],
"total" : 125 ,
"limit" : 50 ,
"offset" : 0
}
Get synchronization logs
Database ID to filter logs (required for multi-database setups)
Maximum number of log entries to return
Number of log entries to skip
Filter by status: success or error
Filter by specific table name
curl "http://localhost:3001/api/sync-logs?db=lms&status=success&limit=20" \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"logs" : [
{
"id" : 1523 ,
"table_name" : "User" ,
"sync_type" : "incremental" ,
"records_processed" : 1250 ,
"duration_ms" : 2340 ,
"status" : "success" ,
"error_message" : null ,
"watermark_before" : { "lastProcessedId" : 98500 , "lastProcessedTimestamp" : "2024-03-01T10:25:00Z" },
"watermark_after" : { "lastProcessedId" : 99750 , "lastProcessedTimestamp" : "2024-03-01T10:30:00Z" },
"created_at" : "2024-03-01T10:30:15Z"
}
],
"total" : 1523 ,
"limit" : 20 ,
"offset" : 0
}
Type of sync: watermark, sequential, full
Watermark state before sync operation
Watermark state after sync operation
Metrics API
System metrics
Get real-time system resource metrics.
curl http://localhost:3001/api/metrics/system \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"timestamp" : "2024-03-01T10:30:00Z" ,
"cpu" : {
"usage" : 45.2 ,
"count" : 8
},
"memory" : {
"used" : 2147483648 ,
"total" : 8589934592 ,
"percentage" : 25.0 ,
"heapUsed" : 524288000 ,
"heapTotal" : 1073741824
},
"eventLoop" : {
"lag" : 2.5 ,
"maxLag" : 15.3
},
"uptime" : 86400
}
CPU usage percentage (0-100)
Current event loop lag in milliseconds
Query metrics
Get query performance metrics and patterns.
curl http://localhost:3001/api/metrics/queries \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"timestamp" : "2024-03-01T10:30:00Z" ,
"totalQueries" : 125430 ,
"avgDuration" : 45.3 ,
"slowQueries" : 23 ,
"errorRate" : 0.02 ,
"topQueries" : [
{
"pattern" : "SELECT COUNT(*) FROM User WHERE status = ?" ,
"count" : 1250 ,
"avgDuration" : 15.3 ,
"maxDuration" : 89.2
}
]
}
Total number of queries executed
Average query duration in milliseconds
Number of queries exceeding timeout threshold
Most frequently executed query patterns
Worker pool statistics
Get worker thread pool statistics for row sanitization.
curl http://localhost:3001/api/workers/stats \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"threadCount" : 7 ,
"activeThreads" : 3 ,
"queuedTasks" : 12 ,
"completedTasks" : 45230 ,
"totalProcessingTime" : 1523400
}
Total number of worker threads
Currently active worker threads
Real-time monitoring
Enhanced metrics endpoint
Get comprehensive metrics for all configured databases.
curl http://localhost:3001/metrics \
-H "Authorization: Bearer $DUCKLING_API_KEY "
Response:
{
"timestamp" : "2024-03-01T10:30:00Z" ,
"databases" : [
{
"id" : "lms" ,
"name" : "LMS" ,
"status" : "healthy" ,
"lastSync" : "2024-03-01T10:25:00Z" ,
"tableCount" : 181 ,
"totalRecords" : 1234567 ,
"syncDuration" : 2340
}
],
"system" : {
"cpu" : 45.2 ,
"memory" : 25.0 ,
"uptime" : 86400
}
}
Use cases
Monitoring dashboard
const response = await fetch ( 'http://localhost:3001/api/metrics/system' , {
headers: { 'Authorization' : `Bearer ${ apiKey } ` }
});
const metrics = await response . json ();
console . log ( `CPU: ${ metrics . cpu . usage } %` );
console . log ( `Memory: ${ metrics . memory . percentage } %` );
console . log ( `Event Loop Lag: ${ metrics . eventLoop . lag } ms` );
Alerting on sync failures
#!/bin/bash
FAILED = $( curl -s "http://localhost:3001/api/sync-logs?status=error&limit=1" \
-H "Authorization: Bearer $DUCKLING_API_KEY " | jq '.total' )
if [ " $FAILED " -gt 0 ]; then
echo "Alert: $FAILED sync failures detected"
# Send alert to monitoring system
fi
const response = await fetch ( 'http://localhost:3001/api/metrics/queries' , {
headers: { 'Authorization' : `Bearer ${ apiKey } ` }
});
const metrics = await response . json ();
// Alert on high error rate
if ( metrics . errorRate > 0.05 ) {
console . warn ( `High query error rate: ${ metrics . errorRate * 100 } %` );
}
// Alert on slow queries
if ( metrics . slowQueries > 100 ) {
console . warn ( ` ${ metrics . slowQueries } slow queries detected` );
}
Metrics are collected every 30 seconds and kept for 30 minutes in memory. For long-term storage, export to external monitoring systems like Prometheus or Grafana.
Health checks Database connectivity and system health
Sync status Synchronization status and progress