# ml-inspector-mcp

[![PyPI](https://img.shields.io/pypi/v/ml-inspector-mcp)](https://pypi.org/project/ml-inspector-mcp/)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Tests](https://github.com/jaiminee/ml-inspector-mcp/actions/workflows/test.yml/badge.svg)](https://github.com/jaiminee/ml-inspector-mcp/actions/workflows/test.yml)

Framework-agnostic ML model analysis MCP server. Drop in any trained model
and test data — Claude evaluates it, explains predictions, detects drift,
and generates PDF reports via natural language.

## Installation

```bash
pip install ml-inspector-mcp                                          # minimal
pip install "ml-inspector-mcp[full]"                                  # everything
pip install "ml-inspector-mcp[sklearn-onnx,explain,reports]"          # common combo
```

## Claude Desktop setup

Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (Mac):

```json
{
  "mcpServers": {
    "ml-inspector": {
      "command": "ml-inspector",
      "env": {
        "ANTHROPIC_API_KEY": "your-key-here"
      }
    }
  }
}
```

Or use `uvx` (no install step needed):

```json
{
  "mcpServers": {
    "ml-inspector": {
      "command": "uvx",
      "args": ["ml-inspector-mcp"]
    }
  }
}
```

## Quick start

```bash
# Clone and generate demo files
git clone https://github.com/jaiminee/ml-inspector-mcp
cd ml-inspector-mcp
pip install "ml-inspector-mcp[sklearn-onnx,explain]"
python examples/train_demo_model.py
```

Then in Claude Desktop:
> "Load the demo model from examples/demo_model.onnx"
> "Load test data from examples/demo_test.csv"
> "Evaluate the model and tell me how it's performing"
> "Explain what drove the prediction for sample 5"
> "Generate a PDF evaluation report"

## Model compatibility

| Format | Framework | Install |
|--------|-----------|---------|
| `.onnx` | Any | Always works — recommended |
| `.pkl` / `.joblib` | scikit-learn | `pip install "ml-inspector-mcp[sklearn-onnx]"` |
| `.h5` / `.keras` | TensorFlow/Keras | `pip install "ml-inspector-mcp[tensorflow]"` |
| `.pt` / `.pth` | PyTorch (full model only) | `pip install "ml-inspector-mcp[pytorch]"` |

### Version mismatch fix

If you get version errors loading a `.pkl` or `.pt` file, export to ONNX first:

```bash
# scikit-learn — use the convert_to_onnx tool after loading, or:
python -c "
import joblib
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
model = joblib.load('model.pkl')
onnx_model = convert_sklearn(model, initial_types=[('input', FloatTensorType([None, N_FEATURES]))])
open('model.onnx', 'wb').write(onnx_model.SerializeToString())
"

# PyTorch
torch.onnx.export(model, dummy_input, "model.onnx", opset_version=17)

# TensorFlow / Keras
python -m tf2onnx.convert --keras model.h5 --output model.onnx
```

## All 17 tools

| Tool | Description |
|------|-------------|
| `load_model` | Load any model file (`.pkl`, `.h5`, `.pt`, `.onnx`) — auto-detects framework |
| `get_model_info` | Info about the currently loaded model |
| `convert_to_onnx` | Convert loaded model to ONNX format |
| `list_supported_formats` | Show all supported formats and install instructions |
| `load_test_data` | Load a CSV as test dataset |
| `evaluate_model` | Full evaluation — accuracy, F1, AUC, confusion matrix, per-class metrics |
| `find_worst_predictions` | Find samples the model struggled most with |
| `evaluate_by_slice` | Evaluate on a data subset (e.g. by group or label) |
| `threshold_analysis` | Sweep decision threshold — precision/recall/F1/FPR trade-offs |
| `explain_prediction` | SHAP explanation for a single sample |
| `global_feature_importance` | Mean absolute SHAP values across all samples |
| `plot_shap_summary` | SHAP beeswarm summary plot saved as PNG |
| `data_quality_report` | Null counts, class imbalance, outliers, data type warnings |
| `detect_drift` | Statistical drift detection between two datasets (Evidently) |
| `plot_confusion_matrix` | Confusion matrix heatmap (raw + normalized) saved as PNG |
| `plot_roc_curve` | ROC curve with per-class AUC scores saved as PNG |
| `generate_report` | Full PDF / HTML / Markdown report with metrics, charts, and optional AI narrative |

## Contributing

Issues and PRs welcome at [github.com/jaiminee/ml-inspector-mcp](https://github.com/jaiminee/ml-inspector-mcp).

## License

MIT