Orthanc DICOM + Nerve Segmentation MCP Server

An MCP (Model Context Protocol) server that lets any MCP client — Claude Desktop, Claude Code, CrewAI agents, or the MCP Inspector — navigate an Orthanc PACS through the full DICOM hierarchy (patients → studies → series → instances), extract text from encapsulated PDF reports, and run a nerve segmentation pipeline (classifier + U-Net) against ultrasound instances, pushing the resulting color-overlay DICOM back into Orthanc as a new series under the same study.

Features

• Full DICOM hierarchy navigation — search patients, studies, series, and instances with flexible filters.

• One-shot natural-language tools — resolve common requests (e.g. "run segmentation on Patient's study") in a single tool call instead of chaining 4–5 calls by hand.

• PDF report extraction — pull text out of Encapsulated PDF instances (scanned radiology reports) stored in Orthanc.

• Nerve segmentation pipeline — classifier + U-Net inference on multi-frame ultrasound DICOM instances, with a color overlay result uploaded back to Orthanc as a new series.

• UID or Orthanc ID accepted everywhere — pass either an Orthanc internal ID or a DICOM UID (PatientID, StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID); the client resolves it automatically.

Architecture

mcp_server.py         MCP tool definitions (FastMCP)
helpers/
  orthanc_client.py    REST client for the Orthanc API
  nerve_seg.py         Model definitions + inference pipeline (DICOM in -> DICOM out)

Tool design: step-by-step vs. one-shot

The step-by-step tools mirror how a human would navigate a PACS:

query_patients -> query_studies -> query_series -> query_instances -> run_nerve_segmentation

These are useful when the caller needs to inspect intermediate results (e.g. "which series are in this study?"). But chaining several tool calls per request also gives an LLM more chances to produce malformed intermediate output. So most steps also have a one-shot counterpart that resolves a whole natural-language request in a single call:

Prefer the one-shot tools for common requests; fall back to the step-by-step tools when the caller needs to inspect or disambiguate along the way.

Prerequisites

• Python 3.10+

• A running Orthanc instance, reachable over HTTP(S)

• Trained model checkpoints:

  • best_cls_model.pth — nerve presence/absence classifier (ResNet-18 backbone)

  • best_model.pth — U-Net segmentation model

• (Optional) a CUDA-capable GPU for faster inference

Installation

git clone <this-repo-url>
cd <this-repo>
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

requirements.txt should include at minimum:

mcp
requests
pydicom
PyPDF2
torch
torchvision
opencv-python
numpy
albumentations
scipy

Place your checkpoints somewhere on disk (e.g. Trained-models/), or set the environment variables below to point at their location.

Configuration

The server is configured entirely through environment variables:

Running the server

Standalone (for local testing):

python mcp_server.py

The server communicates over stdio, so it's normally launched by an MCP client rather than run interactively.

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "orthanc-nerve-segmentation": {
      "command": "python",
      "args": ["/absolute/path/to/mcp_server.py"],
      "env": {
        "ORTHANC_URL": "http://localhost:8042",
        "NERVE_CLS_CKPT": "/absolute/path/to/best_cls_model.pth",
        "NERVE_SEG_CKPT": "/absolute/path/to/best_model.pth",
        "NERVE_DEVICE": "cuda"
      }
    }
  }
}

MCP Inspector

npx @modelcontextprotocol/inspector python mcp_server.py

Model inference is CPU/GPU-bound and can take a while, especially on the first call (checkpoint loading) or when processing multi-frame studies — see Troubleshooting below for adjusting Inspector's timeout.

CrewAI

Wire the server in via MCPServerAdapter, pointing it at the same python mcp_server.py command and environment variables shown above.

Available tools

Navigation

Reports

Segmentation

run_nerve_segmentation accepts these tuning parameters (all optional):

Example prompts

• "Find Patient's most recent study and run nerve segmentation on it."

• "What does the report say for the latest study on patient MRN12345?"

• "List the series in study 1.2.840.113619...."

• "Process the current study with a lower classification threshold, 0.3."

Troubleshooting

Tool calls time out in MCP Inspector. This is usually Inspector's own client-side timeout (10–30s by default), not a server crash — segmentation on multi-frame ultrasound can genuinely take longer than that, especially on the first call (model checkpoints load lazily on first use) or on CPU. Open Inspector's Configuration panel and raise the request timeout to a few minutes, and set NERVE_DEVICE=cuda if a GPU is available. The one-shot study-level tools (run_nerve_segmentation_for_patient, run_nerve_segmentation_on_latest_study) process every instance in every series in a single call, so they take proportionally longer than run_nerve_segmentation on a single instance.

extract_pdf_text_from_dicom returns an error. The instance must have SOPClassUID = Encapsulated PDF Storage (1.2.840.10008.5.1.4.1.1.104.1). Use query_series with modality="DOC" or "SR" as a hint for which series/instance actually holds the report.

Model checkpoints fail to load. Confirm NERVE_CLS_CKPT / NERVE_SEG_CKPT point at valid .pth files whose architecture matches NerveClassifier (ResNet-18 backbone) and UNetIN in helpers/nerve_seg.py exactly — load_state_dict will raise if the architectures don't match.