# Orthanc DICOM + Nerve Segmentation MCP Server

An MCP (Model Context Protocol) server that lets any MCP client — Claude
Desktop, Claude Code, CrewAI agents, or the MCP Inspector — navigate an
[Orthanc](https://orthanc.uclouvain.be/) PACS through the full DICOM
hierarchy (patients → studies → series → instances), extract text from
encapsulated PDF reports, and run a nerve segmentation pipeline
(classifier + U-Net) against ultrasound instances, pushing the resulting
color-overlay DICOM back into Orthanc as a new series under the same study.

## Features

- **Full DICOM hierarchy navigation** — search patients, studies, series,
  and instances with flexible filters.
- **One-shot natural-language tools** — resolve common requests (e.g. "run
  segmentation on Patient's study") in a single tool call instead of
  chaining 4–5 calls by hand.
- **PDF report extraction** — pull text out of Encapsulated PDF instances
  (scanned radiology reports) stored in Orthanc.
- **Nerve segmentation pipeline** — classifier + U-Net inference on
  multi-frame ultrasound DICOM instances, with a color overlay result
  uploaded back to Orthanc as a new series.
- **UID or Orthanc ID accepted everywhere** — pass either an Orthanc
  internal ID or a DICOM UID (PatientID, StudyInstanceUID,
  SeriesInstanceUID, SOPInstanceUID); the client resolves it automatically.

## Architecture

```
mcp_server.py         MCP tool definitions (FastMCP)
helpers/
  orthanc_client.py    REST client for the Orthanc API
  nerve_seg.py         Model definitions + inference pipeline (DICOM in -> DICOM out)
```

### Tool design: step-by-step vs. one-shot

The step-by-step tools mirror how a human would navigate a PACS:

```
query_patients -> query_studies -> query_series -> query_instances -> run_nerve_segmentation
```

These are useful when the caller needs to inspect intermediate results
(e.g. "which series are in this study?"). But chaining several tool calls
per request also gives an LLM more chances to produce malformed
intermediate output. So most steps also have a **one-shot** counterpart
that resolves a whole natural-language request in a single call:

| Step-by-step chain | One-shot equivalent |
|---|---|
| `query_patients` → `query_studies` | `find_studies_by_patient_name` |
| `list_recent_studies` | `list_recent_studies` |
| `query_patients` → ... → `run_nerve_segmentation` | `run_nerve_segmentation_for_patient` |
| `list_recent_studies` → ... → `run_nerve_segmentation` | `run_nerve_segmentation_on_latest_study` |

Prefer the one-shot tools for common requests; fall back to the
step-by-step tools when the caller needs to inspect or disambiguate along
the way.

## Prerequisites

- Python 3.10+
- A running Orthanc instance, reachable over HTTP(S)
- Trained model checkpoints:
  - `best_cls_model.pth` — nerve presence/absence classifier (ResNet-18 backbone)
  - `best_model.pth` — U-Net segmentation model
- (Optional) a CUDA-capable GPU for faster inference

## Installation

```bash
git clone <this-repo-url>
cd <this-repo>
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
```

`requirements.txt` should include at minimum:

```
mcp
requests
pydicom
PyPDF2
torch
torchvision
opencv-python
numpy
albumentations
scipy
```

Place your checkpoints somewhere on disk (e.g. `Trained-models/`), or set
the environment variables below to point at their location.

## Configuration

The server is configured entirely through environment variables:

| Variable | Description | Default |
|---|---|---|
| `ORTHANC_URL` | Base URL of the Orthanc REST API | `http://localhost:8042` |
| `ORTHANC_USERNAME` | Orthanc HTTP basic auth username | *(none)* |
| `ORTHANC_PASSWORD` | Orthanc HTTP basic auth password | *(none)* |
| `ORTHANC_VERIFY_SSL` | Set to `false` to skip TLS verification | `true` |
| `NERVE_CLS_CKPT` | Path to the classifier checkpoint | `Trained-models/best_cls_model.pth` |
| `NERVE_SEG_CKPT` | Path to the segmentation checkpoint | `Trained-models/best_model.pth` |
| `NERVE_DEVICE` | `cuda` or `cpu` | auto-detected |

## Running the server

Standalone (for local testing):

```bash
python mcp_server.py
```

The server communicates over stdio, so it's normally launched by an MCP
client rather than run interactively.

### Claude Desktop

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "orthanc-nerve-segmentation": {
      "command": "python",
      "args": ["/absolute/path/to/mcp_server.py"],
      "env": {
        "ORTHANC_URL": "http://localhost:8042",
        "NERVE_CLS_CKPT": "/absolute/path/to/best_cls_model.pth",
        "NERVE_SEG_CKPT": "/absolute/path/to/best_model.pth",
        "NERVE_DEVICE": "cuda"
      }
    }
  }
}
```

### MCP Inspector

```bash
npx @modelcontextprotocol/inspector python mcp_server.py
```

Model inference is CPU/GPU-bound and can take a while, especially on the
first call (checkpoint loading) or when processing multi-frame studies —
see [Troubleshooting](#troubleshooting) below for adjusting Inspector's
timeout.

### CrewAI

Wire the server in via `MCPServerAdapter`, pointing it at the same
`python mcp_server.py` command and environment variables shown above.

## Available tools

### Navigation

| Tool | Purpose |
|---|---|
| `query_patients` | Search patients by name / ID / birth date |
| `query_studies` | Search studies, filtered by patient ID, date, modality, etc. |
| `find_studies_by_patient_name` | One-shot server-side patient-name → study search |
| `list_recent_studies` | List the most recently received studies |
| `query_series` | Search series within a study |
| `list_series_for_study` | List every series in a study, unfiltered |
| `query_instances` | Search instances within a series |
| `list_instances_for_series` | List instance IDs within a series, unfiltered |
| `get_instance_tags` | Fetch simplified DICOM tags for one instance |

### Reports

| Tool | Purpose |
|---|---|
| `extract_pdf_text_from_dicom` | Extract text from an Encapsulated PDF instance, page by page |

### Segmentation

| Tool | Purpose |
|---|---|
| `run_nerve_segmentation` | Run the pipeline on one instance; uploads a color-overlay series back to Orthanc |
| `run_nerve_segmentation_for_patient` | One-shot: find a patient's latest study and segment every instance in it |
| `run_nerve_segmentation_on_latest_study` | One-shot: segment every instance in the most recently received study |

`run_nerve_segmentation` accepts these tuning parameters (all optional):

| Parameter | Default | Description |
|---|---|---|
| `cls_threshold` | `0.4` | Probability above which a frame is treated as containing a nerve |
| `seg_threshold` | `0.9` | Probability threshold to binarize the segmentation output |
| `min_component_size` | `100` | Connected components smaller than this (pixels) are discarded as noise |
| `overlay_alpha` | `0.4` | Blend strength (0–1) of the colored mask over the original image |
| `overlay_color_rgb` | `[255, 0, 0]` | Overlay color |
| `also_upload_mask_only_series` | `false` | Also upload a second series containing just the raw binary mask |

## Example prompts

- *"Find Patient's most recent study and run nerve segmentation on it."*
- *"What does the report say for the latest study on patient MRN12345?"*
- *"List the series in study `1.2.840.113619...`."*
- *"Process the current study with a lower classification threshold, 0.3."*

## Troubleshooting

**Tool calls time out in MCP Inspector.** This is usually Inspector's own
client-side timeout (10–30s by default), not a server crash — segmentation
on multi-frame ultrasound can genuinely take longer than that, especially
on the first call (model checkpoints load lazily on first use) or on CPU.
Open Inspector's *Configuration* panel and raise the request timeout to a
few minutes, and set `NERVE_DEVICE=cuda` if a GPU is available. The
one-shot study-level tools (`run_nerve_segmentation_for_patient`,
`run_nerve_segmentation_on_latest_study`) process every instance in every
series in a single call, so they take proportionally longer than
`run_nerve_segmentation` on a single instance.

**`extract_pdf_text_from_dicom` returns an error.** The instance must have
`SOPClassUID` = Encapsulated PDF Storage (`1.2.840.10008.5.1.4.1.1.104.1`).
Use `query_series` with `modality="DOC"` or `"SR"` as a hint for which
series/instance actually holds the report.

**Model checkpoints fail to load.** Confirm `NERVE_CLS_CKPT` /
`NERVE_SEG_CKPT` point at valid `.pth` files whose architecture matches
`NerveClassifier` (ResNet-18 backbone) and `UNetIN` in `helpers/nerve_seg.py`
exactly — `load_state_dict` will raise if the architectures don't match.