list_metrics
Discover available evaluation metrics for AI explanation methods, filtering by metric type to assess faithfulness, stability, or fairness.
Instructions
List available evaluation metrics in OpenXAI
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| metric_type | No | Filter by metric type (faithfulness, stability, fairness) |
Implementation Reference
- index.js:670-766 (handler)The primary handler function `listMetrics(metricType)` that executes the tool logic. It defines metrics grouped by categories (faithfulness, stability, ground_truth) and returns filtered list based on input metric_type or 'all', formatted as MCP content response.async listMetrics(metricType) { const metrics = { faithfulness: { PGI: { name: 'Prediction Gap on Important feature perturbation', description: 'Measures the difference in prediction probability when perturbing important features', higher_is_better: true }, PGU: { name: 'Prediction Gap on Unimportant feature perturbation', description: 'Measures the difference in prediction probability when perturbing unimportant features', higher_is_better: false } }, stability: { RIS: { name: 'Relative Input Stability', description: 'Measures maximum change in explanation relative to changes in inputs', higher_is_better: false }, RRS: { name: 'Relative Representation Stability', description: 'Measures maximum change in explanation relative to changes in internal representation', higher_is_better: false }, ROS: { name: 'Relative Output Stability', description: 'Measures maximum change in explanation relative to changes in output predictions', higher_is_better: false } }, ground_truth: { FA: { name: 'Feature Agreement', description: 'Fraction of top-K features common between explanation and ground truth', higher_is_better: true }, RA: { name: 'Rank Agreement', description: 'Fraction of top-K features with same rank in explanation and ground truth', higher_is_better: true }, SA: { name: 'Sign Agreement', description: 'Fraction of top-K features with same sign in explanation and ground truth', higher_is_better: true }, SRA: { name: 'Signed Rank Agreement', description: 'Fraction of top-K features with same sign and rank in explanation and ground truth', higher_is_better: true }, RC: { name: 'Rank Correlation', description: 'Spearman rank correlation between explanation and ground truth rankings', higher_is_better: true }, PRA: { name: 'Pairwise Rank Agreement', description: 'Fraction of feature pairs with same relative ordering in explanation and ground truth', higher_is_better: true } } }; let result = []; if (metricType === 'all') { result = Object.entries(metrics).map(([category, categoryMetrics]) => ({ category, metrics: Object.entries(categoryMetrics).map(([key, value]) => ({ metric: key, ...value })) })); } else { const categoryMetrics = metrics[metricType]; if (categoryMetrics) { result = [{ category: metricType, metrics: Object.entries(categoryMetrics).map(([key, value]) => ({ metric: key, ...value })) }]; } } return { content: [ { type: 'text', text: `Available OpenXAI evaluation metrics:\n\n` + JSON.stringify(result, null, 2) } ] }; }
- index.js:157-171 (registration)Registration of the 'list_metrics' tool in the ListToolsRequest handler, including name, description, and inputSchema.{ name: 'list_metrics', description: 'List available evaluation metrics in OpenXAI', inputSchema: { type: 'object', properties: { metric_type: { type: 'string', description: 'Filter by metric type (faithfulness, stability, fairness)', enum: ['faithfulness', 'stability', 'fairness', 'all'] } }, required: [] } },
- index.js:160-169 (schema)The inputSchema for the 'list_metrics' tool, defining optional metric_type parameter with enum ['faithfulness', 'stability', 'fairness', 'all']. Note: enum has 'fairness' but code uses 'ground_truth'; possible discrepancy.inputSchema: { type: 'object', properties: { metric_type: { type: 'string', description: 'Filter by metric type (faithfulness, stability, fairness)', enum: ['faithfulness', 'stability', 'fairness', 'all'] } }, required: []
- index.js:273-274 (handler)The switch case dispatcher in CallToolRequestHandler that routes 'list_metrics' calls to the listMetrics method.case 'list_metrics': return await this.listMetrics(args.metric_type || 'all');