Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@open-med-mcp segment the liver in ct.nii.gz, check it in three planes, and report the volume"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
open-med-mcp (OMM) is a Model Context Protocol server for medical image analysis. It gives AI coding agents (Claude Code, Codex CLI, Claude Desktop, Cursor, ...) tools to inspect a CT/MRI/X-ray, run segmentation and other models, view and measure the result, and write a report — all on your own machine.
Beyond segmentation it covers classification, detection, vision-language questions, radiomics, and computable clinical criteria (RECIST 1.1, Fleischner, Lung-RADS, LI-RADS, TI-RADS, Agatston). Three pillars:
pillar | what it means |
1. Containerized specialised models | Every model is an adapter with a manifest, a |
2. Preset / custom guidelines | Markdown protocols with YAML front matter that tell an agent how to do a task well (which model, which window, what to check, what to report). Presets ship with the package; drop your own into a folder to override or extend them. They are exposed as MCP prompts and resources. |
3. Modular, code-customizable viewer | For the agent, |
Images never leave the machine.
# 1. install (Python 3.10+)
pip install "open-med-mcp @ git+https://github.com/d0ng231/open-med-mcp"
# optional local model backends (or use containers, see below):
pip install "open-med-mcp[sam2,totalsegmentator] @ git+https://github.com/d0ng231/open-med-mcp" # pick extras: sam2,
# totalsegmentator, lungmask, hdbet, nnunet, monai, torchxrayvision - or [models] for all
# 2. check the environment and fetch weights you want to use
open-med-mcp doctor
open-med-mcp models download medsam2
# 3. register the server with your agent (run inside the folder that holds your images)
claude mcp add open-med-mcp -e OMM_WORKSPACE=$PWD -- open-med-mcp serve # Claude Code
codex mcp add open-med-mcp --env OMM_WORKSPACE=$PWD -- open-med-mcp serve # Codex CLI
open-med-mcp client-config claude-desktop # prints JSON for other clientsThen talk to your agent:
"Segment the liver in
ct.nii.gz, check the result in three planes, and give me the volume."
A typical run looks like this (tool calls made by the agent):
get_conventions() -> coordinate + prompt conventions
inspect_image("ct.nii.gz") -> 512x512x210, 0.8x0.8x1.5 mm, CT, RAS, preview
get_guideline("segmentation-3d-ct") -> protocol: TotalSegmentator first, MedSAM2 to refine
run_model("totalsegmentator", {"image": "ct.nii.gz"}, params={"roi_subset": ["liver"]})
render_view("ct.nii.gz", masks=[mask], layout="three-plane", window="soft-tissue") -> PNG (agent looks)
mask_to_prompts(mask) -> box prompt for a promptable model
segment("ct.nii.gz", model="medsam2", prompts=[{"type": "box", "coords": [...], "slice": 97}])
compare_masks(ts_mask, medsam2_mask, image="ct.nii.gz") -> Dice, HD95, volume difference
postprocess_mask(mask, ["largest_component", "fill_holes"])
mask_stats(mask, image="ct.nii.gz") -> 1432 mL, mean 58 HU
write_report("Liver volumetry", sections=[...])Related MCP server: CBCTScope MCP Server
group | tools |
inspect |
|
models |
|
clinical criteria |
|
masks |
|
processing |
|
viewer |
|
guidance |
|
Full reference with every parameter: docs/tools.md. Coordinate conventions: docs/coordinates.md.
model | what it does | modalities | prompts | how it runs |
| MedSAM2: promptable 2D + 3D segmentation (slice propagation), medical fine-tune of SAM 2.1 | CT, MR, PET, US, endoscopy | box, points | local |
| SAM 2.1 tiny / small / base+ / large | any | box, points | local |
| VoxTell (CVPR 2026): free-text prompts ("liver", "left kidney", "liver tumor") -> 3D masks | CT, MR, PET | text | local |
| 117 CT structures, MR variant, vessels, body regions ... | CT, MR | - | local |
| lungs (R231) and lobes (LTRCLobes) + LAA% emphysema index | CT | - | local |
| HD-BET 2.0 brain extraction (mask + stripped image) | MR | - | local |
| FreeSurfer SynthStrip skull stripping, official image wrapped | MR, CT, PET | - | Docker, Apptainer (host |
| any nnU-Net v2 model (results folder, exported zip, dataset name) | any | - | local |
| MONAI Model Zoo bundles: segmentation (spleen, pancreas, whole body, BraTS, prostate ...) and detection (lung nodules, RetinaNet) | CT, MR | - | local |
| chest X-ray: 18-finding classification, 14-structure anatomy segmentation (-> cardiothoracic ratio), biological age | XR | - | local |
| vision-language: MedGemma 4B by default (any HF image-text-to-text model, e.g. Qwen2.5-VL) - describe, answer, draft | any | text | local |
| pyradiomics: IBSI feature extraction (shape, first order, GLCM, GLRLM, GLSZM, GLDM, NGTDM, filters) per label | CT, MR, PET | - | local (Python 3.9 venv), Docker, Apptainer |
| threshold / Otsu / multi-range / seeded region growing | any | seeds | local (no extras), Docker, Apptainer |
open-med-mcp models list # what is available
open-med-mcp models check hdbet # which backend can run it here
open-med-mcp models pull medsam2 # docker pull ghcr.io/d0ng231/open-med-mcp-sam2:0.1
open-med-mcp models pull synthstrip --engine apptainer # HPC: official image -> .sif
open-med-mcp models build lungmask --engine apptainer # or build from the Dockerfile (converted to a .def)
open-med-mcp run lungmask --image chest_ct.nii.gz --task lobes
open-med-mcp run medsam2 --image ct.nii.gz --prompts '[{"type":"box","coords":[60,80,20,140,170,20]}]'
open-med-mcp run voxtell --image ct.nii.gz --prompts '[{"type":"text","text":"liver"},{"type":"text","text":"spleen"}]'Adding a model = copy src/open_med_mcp/zoo/_template/, edit the manifest, implement run.py
against the job contract, add a Dockerfile. Wrapping an existing image (as
synthstrip does) needs only a manifest with a command template. User models can live outside the
package (OMM_MODEL_DIRS). See docs/models.md.
Workflow presets: getting-started, segmentation-3d-ct, segmentation-3d-mri, segmentation-2d-prompted,
brain-mri-preprocessing, chest-xray-triage, chest-xray-anatomy-and-ctr, lung-ct-analysis,
registration-followup, batch-processing, multi-organ-ct-report, compare-two-segmentations, qc-checklist.
Clinical criteria presets (each cites its source and states its scope): recist-1-1, fleischner-2017,
lung-rads-2022, li-rads-2018, acr-ti-rads-2017, coronary-calcium-agatston,
organ-volume-reference-ranges.
Add your own: put *.md files with YAML front matter into omm_guidelines/ in the workspace (or any
directory in OMM_GUIDELINE_DIRS). A file with the same name overrides the preset. Guidelines
are also exposed as MCP prompts (/mcp__open-med-mcp__segmentation-3d-ct in Claude Code) and as
guideline://<name> resources. See docs/guidelines.md.
render_view(...) - PNG figures for the agent, with native-voxel tick labels.
export_viewer(...) - one self-contained HTML file (no server, no CDN): cine/scroll through slices,
toggle the overlay, invert, zoom/pan, and click or drag for a point/box prompt in native
coordinates (with a millimetre size) that feeds back into segment.
open-med-mcp serve-viewer ct.nii.gz -m mask.nii.gz - 3D view (NiiVue) on localhost.
write_report(...) - Markdown + HTML with embedded figures.
Custom renderers: implement render(image, masks, spec) and register with @register_renderer.
See docs/viewer.md.
automatic anatomy: | free-text prompts: |
|
|
skull stripping: | chest X-ray: |
|
|
2D promptable: | agent QC view: |
|
|
chest X-ray anatomy + | classify + measure + criteria |
| RECIST |
All figures are real tool outputs on public sample data (abdominal CT, MNI152 template, NIH chest
X-ray; see examples/get_sample_data.sh).
Everything is extensible from the workspace, without forking:
open-med-mcp new plugin lesion-count # omm_plugins/lesion_count.py: register(server) + your @server.tool()s
open-med-mcp new model my-unet # omm_models/my-unet/: manifest + run.py + Dockerfile (job contract)
open-med-mcp new model synthseg --wrapped-image freesurfer/synthseg # manifest only, drives the official image
open-med-mcp new guideline my-protocol # omm_guidelines/my-protocol.md
open-med-mcp plugins list # what loads, and why something did notPlug-in tools use open_med_mcp.plugin_api (path resolution, cached image loading, previews,
result packaging) and appear next to the built-in tools; packaged plug-ins register through the
open_med_mcp.plugins entry point. See docs/plugins.md and examples/plugins/.
stdio hygiene - the server never writes to stdout; logs go to stderr and $OMM_HOME/logs/server.log.
Long runs - model tools stream MCP progress notifications; pass wait=false to get a job id
immediately and poll get_job (results, previews and logs are also on disk under omm_outputs/).
Concurrency - tools run in worker threads; rendering, caches and provenance are lock-protected.
Payload limits - inline previews are capped (OMM_MAX_IMAGE_BYTES, default 1.5 MB) and can be
switched off for text-only clients (OMM_RETURN_IMAGES=0); every result also names the saved file.
Transports - open-med-mcp serve (stdio), --transport streamable-http / sse for remote
agents; over HTTP file access is confined to the workspace by default.
Errors - every tool returns a readable is_error result instead of crashing the session.
Schemas - every parameter is typed and described; the test-suite validates all tool schemas.
variable | default | meaning |
| current directory | root for relative paths; outputs go to |
|
| weights and Apptainer images |
|
|
|
|
|
|
| current interpreter | interpreter of a dedicated venv for a local adapter (e.g. |
| - | extra adapters / guidelines ( |
|
| registry prefix for container images |
|
| confine file access to the workspace |
| - | extra plug-in directories ( |
|
| inline preview images in results |
|
| cap for one inline image |
|
| logging |
|
| in-memory image cache budget |
No Docker on the cluster? open-med-mcp models pull <model> --engine apptainer fetches the GHCR
image as a .sif, or models build --engine apptainer converts the Dockerfile to an Apptainer
definition and builds it. Set OMM_RUNNER=apptainer; GPU passthrough (--nv) is automatic when a
GPU is visible. Alternatively point OMM_ZOO_SAM2_PYTHON at a venv with PyTorch and run models on
a GPU node with the local backend. See docs/hpc.md.
Architecture - components, data flow, design decisions
Quick start and client setup - Claude Code, Codex, Claude Desktop, Cursor, HTTP transport
Plug-ins - tools, models, guidelines, renderers
git clone https://github.com/d0ng231/open-med-mcp && cd open-med-mcp
uv venv && uv pip install -e ".[dev]"
.venv/bin/ruff check src tests && .venv/bin/python -m pytest -q # CPU-only tests, ~1 minThe test-suite covers the whole stack (I/O, coordinates, processing, viewer, job contract, MCP server, CLI) with synthetic data; the real models are verified on GPU/CPU before a release.
SAM 2 (Meta, Apache-2.0) and MedSAM2 (Ma et al., Apache-2.0)
TotalSegmentator (Wasserthal et al., Apache-2.0), lungmask (Hofmanninger et al., Apache-2.0), HD-BET and nnU-Net (Isensee et al., Apache-2.0), SynthStrip (Hoopes et al., FreeSurfer license), MONAI (Apache-2.0), TorchXRayVision (Cohen et al., Apache-2.0)
SimpleITK, NiiVue, the MCP Python SDK
Model weights are downloaded from the upstream projects and remain subject to their licenses.
open-med-mcp is research software. It is not a medical device and must not be used for clinical decision making. Outputs are model predictions that need expert review.
Apache-2.0. See LICENSE. If you use it in research, please cite via CITATION.cff.
This server cannot be deployed
Telehealth support for Claude, Codex and Cursor: investigate cases and connect your tools.
Physician-reviewed medical opinions and prescriptions for AI agents.
Medical RAG: semantic search for clinical guidelines, drug interactions, diagnoses & EHR data.
Medical RAG: semantic search for clinical guidelines, drug interactions, diagnoses & EHR data.