get_performance_stats
Analyze and compare performance statistics between Chain of Draft (CoD) and Chain of Thought (CoT) approaches, filtering by specific domains to evaluate reasoning efficiency and accuracy.
Instructions
Get performance statistics for CoD vs CoT approaches
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | No | Filter for specific domain |
Implementation Reference
- server.py:142-169 (handler)Primary handler for get_performance_stats tool in Python MCP server. Fetches stats from AnalyticsService and formats response.@app.tool() async def get_performance_stats( domain: str = None ) -> str: """Get performance statistics for CoD vs CoT approaches. Args: domain: Filter for specific domain (optional) """ stats = await analytics.get_performance_by_domain(domain) result = "Performance Comparison (CoD vs CoT):\n\n" if not stats: return "No performance data available yet." for stat in stats: result += f"Domain: {stat['domain']}\n" result += f"Approach: {stat['approach']}\n" result += f"Average tokens: {stat['avg_tokens']:.1f}\n" result += f"Average time: {stat['avg_time_ms']:.1f}ms\n" if stat['accuracy'] is not None: result += f"Accuracy: {stat['accuracy'] * 100:.1f}%\n" result += f"Sample size: {stat['count']}\n\n" return result
- analytics.py:84-113 (helper)Core helper method in AnalyticsService that queries the database for performance statistics by domain and approach.async def get_performance_by_domain(self, domain=None): """Get performance statistics by domain.""" session = self.Session() try: query = session.query( InferenceRecord.domain, InferenceRecord.approach, func.avg(InferenceRecord.tokens_used).label("avg_tokens"), func.avg(InferenceRecord.execution_time_ms).label("avg_time"), func.avg(InferenceRecord.is_correct).label("accuracy"), func.count(InferenceRecord.id).label("count") ).group_by(InferenceRecord.domain, InferenceRecord.approach) if domain: query = query.filter(InferenceRecord.domain == domain) results = query.all() return [ { "domain": r.domain, "approach": r.approach, "avg_tokens": r.avg_tokens, "avg_time_ms": r.avg_time, "accuracy": r.accuracy if r.accuracy is not None else None, "count": r.count } for r in results ] finally: session.close()
- index.js:684-711 (handler)Handler for get_performance_stats in JavaScript MCP server implementation. Uses in-memory analyticsDb.if (name === "get_performance_stats") { const stats = analyticsDb.getPerformanceByDomain(args.domain); let result = "Performance Comparison (CoD vs CoT):\n\n"; if (!stats || stats.length === 0) { result = "No performance data available yet."; } else { for (const stat of stats) { result += `Domain: ${stat.domain}\n`; result += `Approach: ${stat.approach}\n`; result += `Average tokens: ${stat.avg_tokens.toFixed(1)}\n`; result += `Average time: ${stat.avg_time_ms.toFixed(1)}ms\n`; if (stat.accuracy !== null) { result += `Accuracy: ${(stat.accuracy * 100).toFixed(1)}%\n`; } result += `Sample size: ${stat.count}\n\n`; } } return { content: [{ type: "text", text: result }] };
- index.js:528-540 (schema)Input schema definition for the get_performance_stats tool in JS server.const PERFORMANCE_TOOL = { name: "get_performance_stats", description: "Get performance statistics for CoD vs CoT approaches", inputSchema: { type: "object", properties: { domain: { type: "string", description: "Filter for specific domain" } } } };
- index.js:581-591 (registration)Registration of available tools including get_performance_stats via ListToolsRequestHandler.server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [ CHAIN_OF_DRAFT_TOOL, MATH_TOOL, CODE_TOOL, LOGIC_TOOL, PERFORMANCE_TOOL, TOKEN_TOOL, COMPLEXITY_TOOL ], }));