Allows loading and analyzing Keras models in .h5/.keras format, including evaluation, prediction explanation, and report generation.
Allows loading and analyzing ONNX models, providing cross-framework evaluation, explanation, drift detection, and PDF report generation.
Allows loading and analyzing PyTorch models (full model .pt/.pth files), with evaluation, SHAP explanations, and report generation.
Allows loading and analyzing scikit-learn models (.pkl/.joblib), with tools for evaluation, feature importance, threshold analysis, and report generation.
Allows loading and analyzing TensorFlow models in .h5/.keras format, including evaluation, prediction explanation, drift detection, and report generation.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ml-inspector-mcp Load the demo model and generate a PDF evaluation report"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Framework-agnostic ML model analysis MCP server. Drop in any trained model and test data — Claude evaluates it, explains predictions, detects drift, and generates PDF reports via natural language.
pip install ml-inspector-mcp # minimal
pip install "ml-inspector-mcp[full]" # everything
pip install "ml-inspector-mcp[sklearn-onnx,explain,reports]" # common comboRelated MCP server: mlctl
Add to ~/Library/Application Support/Claude/claude_desktop_config.json (Mac):
{
"mcpServers": {
"ml-inspector": {
"command": "ml-inspector",
"env": {
"ANTHROPIC_API_KEY": "your-key-here"
}
}
}
}Or use uvx (no install step needed):
{
"mcpServers": {
"ml-inspector": {
"command": "uvx",
"args": ["ml-inspector-mcp"]
}
}
}# Clone and generate demo files
git clone https://github.com/jaiminee/ml-inspector-mcp
cd ml-inspector-mcp
pip install "ml-inspector-mcp[sklearn-onnx,explain]"
python examples/train_demo_model.pyThen in Claude Desktop:
"Load the demo model from examples/demo_model.onnx" "Load test data from examples/demo_test.csv" "Evaluate the model and tell me how it's performing" "Explain what drove the prediction for sample 5" "Generate a PDF evaluation report"
Format | Framework | Install |
| Any | Always works — recommended |
| scikit-learn |
|
| TensorFlow/Keras |
|
| PyTorch (full model only) |
|
If you get version errors loading a .pkl or .pt file, export to ONNX first:
# scikit-learn — use the convert_to_onnx tool after loading, or:
python -c "
import joblib
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
model = joblib.load('model.pkl')
onnx_model = convert_sklearn(model, initial_types=[('input', FloatTensorType([None, N_FEATURES]))])
open('model.onnx', 'wb').write(onnx_model.SerializeToString())
"
# PyTorch
torch.onnx.export(model, dummy_input, "model.onnx", opset_version=17)
# TensorFlow / Keras
python -m tf2onnx.convert --keras model.h5 --output model.onnxTool | Description |
| Load any model file ( |
| Info about the currently loaded model |
| Convert loaded model to ONNX format |
| Show all supported formats and install instructions |
| Load a CSV as test dataset |
| Full evaluation — accuracy, F1, AUC, confusion matrix, per-class metrics |
| Find samples the model struggled most with |
| Evaluate on a data subset (e.g. by group or label) |
| Sweep decision threshold — precision/recall/F1/FPR trade-offs |
| SHAP explanation for a single sample |
| Mean absolute SHAP values across all samples |
| SHAP beeswarm summary plot saved as PNG |
| Null counts, class imbalance, outliers, data type warnings |
| Statistical drift detection between two datasets (Evidently) |
| Confusion matrix heatmap (raw + normalized) saved as PNG |
| ROC curve with per-class AUC scores saved as PNG |
| Full PDF / HTML / Markdown report with metrics, charts, and optional AI narrative |
Issues and PRs welcome at github.com/jaiminee/ml-inspector-mcp.
MIT
This server cannot be deployed
Sentiment, toxicity, entity extraction, PII, translation, summary, QA, fraud scoring, safety audit.
Train, explain, optimise and deploy transparent glass-box ML models via workflow tools.
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
DriftOracle - 15 tools for model/data drift monitoring: PSI, KS-test, alerts, evidence packs.
