Skill Tracker
The Skill Tracker monitors skill usage patterns and calculates confidence scores based on success rates. It tracks the complete adoption lifecycle from discovery (50% confidence) to graduation (85%+ confidence).
createSkillTracker
Creates a skill tracker instance backed by SQLite.
import { createSkillTracker } from 'auto-skill/core/skill-tracker';
const tracker = createSkillTracker('/path/to/skills_tracking.db');
Path to SQLite database. Defaults to ~/.claude/auto-skill/skills_tracking.db
Returns a SkillTracker instance with usage tracking and graduation detection methods
SkillAdoption Type
Tracks adoption metrics for a skill.
interface SkillAdoption {
skillId: string;
skillName: string;
source: string; // "external" | "local" | "mental-hint"
initialConfidence: number; // Starting confidence (0-1)
currentConfidence: number; // Current confidence (0-1)
usageCount: number;
successCount: number;
failureCount: number;
firstUsed: string; // ISO-8601 timestamp
lastUsed: string; // ISO-8601 timestamp
graduatedToLocal: boolean;
}
Confidence Evolution
Confidence scores evolve based on usage patterns:
| Source | Initial | Proven (3+ uses, 70%+ success) | Graduation (5+ uses, 80%+ success) |
|---|
| External | 50% | 75% | 85%+ |
| Mental Hint | 60% | 75% | 85%+ |
| Local | 80% | 85%+ | 100% (cap) |
- Usage Factor:
min(usageCount / 10.0, 1.0) (caps at 10 uses)
- Weighted Score:
successRate * 0.7 + usageFactor * 0.3
- Blended:
initialConfidence * 0.3 + weightedScore * 0.7
- External Cap: 0.95 max for external skills, 1.0 for local
Methods
trackUsage
Record a skill usage event and update confidence score.
tracker.trackUsage(
'skill-123',
'fix-typescript-errors',
'external',
true // success
);
Unique identifier for the skill
Human-readable skill name
Skill source: "external" | "local" | "mental-hint"
Whether the skill execution succeeded
- Creates new adoption record if skill is new
- Increments usage/success/failure counts
- Recalculates confidence score
- Updates
last_used timestamp
getAdoption
Get adoption details for a specific skill.
const adoption = tracker.getAdoption('skill-123');
if (adoption) {
console.log(`Confidence: ${adoption.currentConfidence}`);
console.log(`Success rate: ${adoption.successCount / adoption.usageCount}`);
}
Skill identifier to look up
adoption
SkillAdoption | undefined
Adoption record if found, otherwise undefined
listAdoptions
Get all skill adoptions above a minimum confidence threshold.
const highConfidence = tracker.listAdoptions(0.75);
Minimum confidence threshold (0-1)
Array of adoptions sorted by confidence (descending), then usage count (descending)
shouldGraduate
Check if a skill meets graduation criteria.
if (tracker.shouldGraduate('skill-123')) {
console.log('Skill is ready for graduation!');
}
Skill identifier to check
true if skill meets all graduation criteria:
- Confidence >= 0.85
- Usage count >= 5
- Success rate >= 0.8
- Source is “external”
- Not already graduated
markGraduated
Mark a skill as graduated to local.
tracker.markGraduated('skill-123');
Skill identifier to mark as graduated
- Sets
graduated_to_local flag to 1
- Changes
source to “local”
- Sets
current_confidence to 0.85
- Updates
updated_at timestamp
getGraduationCandidates
Get all skills eligible for graduation.
const candidates = tracker.getGraduationCandidates();
console.log(`Found ${candidates.length} graduation candidates`);
Array of skills that meet graduation criteria (confidence >= 0.8)
getStats
Get tracker statistics.
const stats = tracker.getStats();
console.log(`Total skills tracked: ${stats.total_skills}`);
console.log(`Average confidence: ${stats.avg_confidence.toFixed(2)}`);
console.log(`Graduated: ${stats.graduated_count}`);
Object with tracking statistics:
total_skills: numberavg_confidence: numbergraduated_count: numbertotal_usage: number (sum of all usage counts)by_source: Record of skill counts by source type
close
Close the database connection.
Usage Example
import { createSkillTracker } from 'auto-skill/core/skill-tracker';
const tracker = createSkillTracker();
// Track skill usage
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', true);
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', true);
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', false);
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', true);
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', true);
tracker.trackUsage('ts-debug', 'typescript-debugger', 'external', true);
// Check adoption
const adoption = tracker.getAdoption('ts-debug');
if (adoption) {
console.log(`Usage: ${adoption.usageCount}`);
console.log(`Success rate: ${(adoption.successCount / adoption.usageCount * 100).toFixed(1)}%`);
console.log(`Confidence: ${(adoption.currentConfidence * 100).toFixed(1)}%`);
}
// Output:
// Usage: 6
// Success rate: 83.3%
// Confidence: 81.5%
// Check if ready for graduation
if (tracker.shouldGraduate('ts-debug')) {
console.log('Ready for graduation!');
tracker.markGraduated('ts-debug');
}
// View all high-confidence skills
const proven = tracker.listAdoptions(0.75);
console.log(`\nProven skills (75%+ confidence):`);
proven.forEach(skill => {
console.log(` - ${skill.skillName}: ${(skill.currentConfidence * 100).toFixed(1)}%`);
});
// Get statistics
const stats = tracker.getStats();
console.log(`\nTracker Stats:`);
console.log(` Total skills: ${stats.total_skills}`);
console.log(` Avg confidence: ${(stats.avg_confidence * 100).toFixed(1)}%`);
console.log(` Graduated: ${stats.graduated_count}`);
console.log(` Total usage: ${stats.total_usage}`);
console.log(` By source:`, stats.by_source);
tracker.close();
Database Schema
The tracker uses a SQLite table:
CREATE TABLE skill_adoptions (
skill_id TEXT PRIMARY KEY,
skill_name TEXT NOT NULL,
source TEXT NOT NULL,
initial_confidence REAL NOT NULL,
current_confidence REAL NOT NULL,
usage_count INTEGER DEFAULT 0,
success_count INTEGER DEFAULT 0,
failure_count INTEGER DEFAULT 0,
first_used TEXT NOT NULL,
last_used TEXT NOT NULL,
graduated_to_local INTEGER DEFAULT 0,
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_confidence ON skill_adoptions(current_confidence);
CREATE INDEX idx_source ON skill_adoptions(source);
Source Reference
src/core/skill-tracker.ts:102-366