# MCP-Grounded 🩺

**A multi-agent pipeline for medical image classification with verification-aware abstention, coordinated via the Model Context Protocol (MCP).**

> *"Instead of always guessing, the AI says — I'm not confident enough, I'll skip this one."*

---

## What is this?

MCP-Grounded is a 4-agent AI pipeline that classifies skin lesion images from the [HAM10000](https://www.kaggle.com/datasets/kmader/skin-cancer-mnist-ham10000) dataset. What makes it novel: the final agent can **abstain** from answering when it isn't confident — making it safer for medical use.

All four agents are real [MCP tools](https://www.anthropic.com/news/model-context-protocol), not just described as such.

---

## Pipeline

```
Skin lesion image
       │
       ▼
┌─────────────────┐
│  BiomedCLIP     │  Agent 1: Extract 512-dim embedding
│  (Extract)      │
└────────┬────────┘
         │
         ▼  ┌─────────────────────────────────────────┐
            │              MCP Server                  │
            │                                          │
            │  ┌──────────┐       ┌──────────┐        │
            │  │ Retrieve │──────▶│  Rerank  │        │
            │  │ Agent 2  │       │  Agent 3 │        │
            │  └──────────┘       └────┬─────┘        │
            │                          │               │
            │                 ┌────────▼──────────┐   │
            │                 │  Verify / Abstain  │   │
            │                 │     Agent 4        │   │
            │                 └────────┬───────────┘   │
            └────────────────────────── │ ─────────────┘
                                        │
                          ┌─────────────┴─────────────┐
                          │                           │
                    conf ≥ τ                     conf < τ
                          │                           │
                       PREDICT                    ABSTAIN
```

---

## Results

### Retrieval Quality

| Metric | Value |
|--------|-------|
| Recall@1 | 77.9% |
| Recall@5 | 93.5% |
| Recall@10 | **96.3%** |
| Recall@50 | 99.2% |

### Verification-Aware Abstention (key result)

| Threshold τ | Coverage | Selective Accuracy |
|-------------|----------|--------------------|
| 0.0 (answer all) | 100.0% | 67.0% |
| 0.5 | 96.9% | 69.0% |
| 0.6 | 83.4% | 77.0% |
| **0.7** | **52.0%** | **91.3%** |
| 0.8 | 4.9% | 98.6% |

At τ = 0.7, selective accuracy improves **+24 percentage points** over the no-abstention baseline.

### Risk–Coverage Curve

![Risk-Coverage Curve](risk_coverage.png)

*As the confidence threshold rises, coverage drops but selective accuracy climbs sharply — proving abstention makes the system safer.*

### Calibration

| Metric | Value |
|--------|-------|
| ECE before temperature scaling | 0.191 |
| ECE after temperature scaling | **0.185** |
| Learned temperature T | 0.944 |

---

## Dataset

[HAM10000](https://www.kaggle.com/datasets/kmader/skin-cancer-mnist-ham10000) — 10,015 dermoscopic images across 7 skin lesion categories:

`akiec` · `bcc` · `bkl` · `df` · `mel` · `nv` · `vasc`

Split: 70% train / 15% validation / 15% test (stratified).

---

## How to Run

### Step 1 — Generate embeddings (Google Colab, GPU)

Open `notebook1_embeddings.py` in [Google Colab](https://colab.research.google.com) with a T4 GPU runtime. Run all cells top to bottom. Downloads HAM10000 and produces `embeddings.npz`.

### Step 2 — Run experiments (Google Colab)

Open `notebook2_experiments.py` in a new Colab notebook. Upload `embeddings.npz`. Run all cells. Produces:
- All result tables (Recall@K, accuracy, calibration, abstention)
- `risk_coverage.png`
- `clf_weights.npz`

### Step 3 — Run the MCP server (local)

```bash
pip install "mcp[cli]" numpy torch
python mcp_grounded_server.py
```

Starts a live MCP server with three callable tools: `retrieve`, `rerank`, `classify_and_verify`.

---

## Requirements

```
mcp[cli]
numpy
torch
open_clip_torch
scikit-learn
pandas
pillow
tqdm
matplotlib
```

See `requirements.txt`.

---

## File Structure

```
mcp_grounded/
├── notebook1_embeddings.py     # Colab: download HAM10000, extract BiomedCLIP embeddings
├── notebook2_experiments.py    # Colab: retrieval, calibration, abstention experiments
├── mcp_grounded_server.py      # Local: FastMCP server exposing 4 agents as tools
├── risk_coverage.png           # Figure 2: risk-coverage curve
├── requirements.txt
└── README.md
```

---

## Citation

If you use this work, please cite:

```bibtex
@inproceedings{mcpgrounded2025,
  title     = {MCP-Grounded: A Multi-Agent Pipeline with Verification-Aware Abstention for Medical Image Classification},
  author    = {[Your Name]},
  booktitle = {[Conference Name]},
  year      = {2025}
}
```

---

## License

MIT License. Dataset (HAM10000) is CC-BY-NC-SA-4.0 — see Kaggle for terms.

---

*Built with [BiomedCLIP](https://huggingface.co/microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224) · [FastMCP](https://github.com/jlowin/fastmcp) · HAM10000*