track_agent_performance
Monitor and evaluate AI agent performance by tracking key metrics such as tokens used, time elapsed, validation score, security score, and test coverage to optimize efficiency and quality.
Instructions
Track and analyze AI agent performance metrics
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| featureName | Yes | Name of the feature being tracked | |
| improvements | No | ||
| metrics | Yes | ||
| timestamp | Yes | ISO timestamp of the feature completion |
Implementation Reference
- Core handler function that processes input metrics, calculates scores, updates agent memory/context, generates report with trends, recommendations, and achievements.export async function trackAgentPerformance( metrics: PerformanceMetrics ): Promise<PerformanceReport> { const report: PerformanceReport = { success: false, summary: { overallScore: 0, efficiency: '', quality: '', security: '', }, recommendations: [], trends: { tokenUsage: 'stable', quality: 'stable', speed: 'stable', }, achievements: [], }; try { // Calculate overall score const scores = { validation: metrics.metrics.validationScore, security: metrics.metrics.securityScore, coverage: metrics.metrics.testCoverage, efficiency: calculateEfficiencyScore(metrics), errorFree: calculateErrorScore(metrics), }; const overallScore = Math.round( (scores.validation * 0.25 + scores.security * 0.25 + scores.coverage * 0.2 + scores.efficiency * 0.15 + scores.errorFree * 0.15) ); // Update report summary report.summary.overallScore = overallScore; report.summary.efficiency = getEfficiencyRating(metrics); report.summary.quality = getQualityRating(scores.validation, scores.coverage); report.summary.security = getSecurityRating(scores.security); // Update agent memory with performance data updateAgentMemory(metrics, overallScore); // Analyze trends report.trends = analyzeTrends(metrics); // Generate recommendations report.recommendations = generateRecommendations(metrics, scores); // Check for achievements report.achievements = checkAchievements(metrics, overallScore); // Update context with current metrics updateAgentContext(metrics); report.success = true; } catch (error) { report.success = false; report.recommendations = [`Error tracking performance: ${error}`]; } return report; }
- MCP tool definition with input schema for validating tool arguments.{ name: 'track_agent_performance', description: 'Track and analyze AI agent performance metrics', inputSchema: { type: 'object', properties: { featureName: { type: 'string', description: 'Name of the feature being tracked', }, timestamp: { type: 'string', description: 'ISO timestamp of the feature completion', }, metrics: { type: 'object', properties: { tokensUsed: { type: 'number' }, timeElapsed: { type: 'number' }, validationScore: { type: 'number' }, securityScore: { type: 'number' }, testCoverage: { type: 'number' }, hallucinations: { type: 'object', properties: { detected: { type: 'number' }, prevented: { type: 'number' }, examples: { type: 'array', items: { type: 'string' }, }, }, }, errors: { type: 'object', properties: { syntax: { type: 'number' }, runtime: { type: 'number' }, type: { type: 'number' }, }, }, }, required: ['tokensUsed', 'timeElapsed', 'validationScore', 'securityScore', 'testCoverage'], }, improvements: { type: 'object', properties: { tokenReduction: { type: 'number' }, timeReduction: { type: 'number' }, qualityIncrease: { type: 'number' }, }, }, }, required: ['featureName', 'timestamp', 'metrics'], }, },
- src/tools/index.ts:185-230 (registration)Tool registration in the MCP server request handler switch statement, including Zod parsing and execution of the handler.case 'track_agent_performance': { const params = z.object({ featureName: z.string(), timestamp: z.string(), metrics: z.object({ tokensUsed: z.number(), timeElapsed: z.number(), validationScore: z.number(), securityScore: z.number(), testCoverage: z.number(), hallucinations: z.object({ detected: z.number(), prevented: z.number(), examples: z.array(z.string()), }).optional().default({ detected: 0, prevented: 0, examples: [], }), errors: z.object({ syntax: z.number(), runtime: z.number(), type: z.number(), }).optional().default({ syntax: 0, runtime: 0, type: 0, }), }), improvements: z.object({ tokenReduction: z.number().optional(), timeReduction: z.number().optional(), qualityIncrease: z.number().optional(), }).optional(), }).parse(args); const result = await trackAgentPerformance(params); return { content: [ { type: 'text', text: JSON.stringify(result, null, 2), }, ], }; }
- TypeScript interfaces defining the input metrics structure and output report structure for the tool.interface PerformanceMetrics { featureName: string; timestamp: string; metrics: { tokensUsed: number; timeElapsed: number; // in seconds validationScore: number; // 0-100 securityScore: number; // 0-100 testCoverage: number; // percentage hallucinations: { detected: number; prevented: number; examples: string[]; }; errors: { syntax: number; runtime: number; type: number; }; }; improvements?: { tokenReduction?: number; timeReduction?: number; qualityIncrease?: number; }; } interface PerformanceReport { success: boolean; summary: { overallScore: number; efficiency: string; quality: string; security: string; }; recommendations: string[]; trends: { tokenUsage: 'improving' | 'stable' | 'degrading'; quality: 'improving' | 'stable' | 'degrading'; speed: 'improving' | 'stable' | 'degrading'; }; achievements: string[]; }