medimgq: labeled CT images for exam questions

An MCP server that lets an AI build image questions for medical-school practice exams. The AI asks for structures ("label the pancreas, the left adrenal gland and both kidneys"); medimgq finds a real CT slice where they are all clearly visible, draws numbered arrows, and returns the image together with an answer key. It can also export click-the-structure (hotspot) masks that line up exactly with the image.

Images come from the TotalSegmentator dataset: 1,228 real CT scans, each with expert-reviewed outlines of 117 structures. CT comes first; X-ray and MRI are planned.

If you are an AI agent, read ai_read_this.md instead. It is the same information in a form built for tool use.

Contents

• How it works

• Setup

• Connect it to an AI

• What you can ask for

• Options, with pictures

• What comes back

• The tools

• Examples

• Real pathology

• Limits and caveats

• Project layout and development

• Data and attribution

Related MCP server: offline-mcp-educational-server

How it works

1. Ask. The AI names the structures to label. Names are forgiving: "IVC", "inferior vena cava" and "postcava" all work; "kidney" means both kidneys; "lumbar spine" means L1–L5.

2. Find a slice. medimgq searches a precomputed index of every structure on every slice of every scan. By default it uses the textbook view and vertebral level (the pancreas at L1, the thyroid at C6–T1) and skips noisy scans, cut-off anatomy and arms in the field of view.

3. Render. It draws numbered arrows, placed so they don't cover other labeled structures or each other, picks a sensible window (lung window for lung lobes) and crops to the right region (neck, chest or abdomen).

4. Get back. A PNG, an answer key such as {"1": "pancreas", "2": "left kidney, right kidney"}, the exact pixel geometry of every arrow and label, and optional hotspot masks.

The AI writes the question itself. medimgq only supplies the images and the facts about them.

Setup

You need Python 3.12+, uv, about 30 GB of disk for the dataset, and a few minutes to build the index.

git clone https://github.com/agadin/medical-image-question-creator.git
cd medical-image-question-creator
uv sync

1. Download the data (not included in this repo). Put it in the project folder, or point MEDIMGQ_TS_ROOT at it.

• TotalSegmentator dataset v2.0.1 → unzip to Totalsegmentator_dataset_v201/

2. Build the indexes (one time):

uv run python -m medimgq.indexer --workers 12   # ~4 min: every structure on every slice
uv run python -m medimgq.quality --workers 12   # ~4 min: noise, contrast phase, cut-off body, arms
uv run python -m medimgq.findings --workers 12  # ~1 min: aneurysms, goiter, cysts, organ enlargement

They are written to data/index/ (about 160 MB).

3. Check it works:

uv run pytest            # unit tests on synthetic scans + an end-to-end test that calls every tool

Connect it to an AI

Claude Code

claude mcp add medimgq -- uv run --directory /path/to/medical-image-question-creator medimgq-mcp

Claude Desktop and other MCP clients: add this to the client's MCP config.

{
  "mcpServers": {
    "medimgq": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/medical-image-question-creator", "medimgq-mcp"]
    }
  }
}

Then ask your AI something like: "Make a CT question where the student has to recognize the left brachiocephalic vein. Use numbers on the image and add four distractors."

What you can ask for

• Any of 117 structures: abdominal and pelvic organs, lung lobes, heart and great vessels, neck vessels, thyroid, trachea, esophagus, every vertebra C1–S1, every rib, major muscles and bones. Ask the list_structures tool for the full list with synonyms.

• Several structures per label. {"label": "2", "structures": ["kidneys"]} puts one number on both kidneys.

• Unlabeled arrows. A label of "" draws an arrow with no text ("What is the structure at the arrow?").

• Shuffled numbers. shuffle_labels: true reassigns the numbers randomly, so the answer isn't always the first label you listed.

• Real pathology: aortic aneurysms, goiters, renal cysts, cardiomegaly, enlarged spleen, liver or prostate, found by measuring every scan (see Real pathology).

• Bare images with hotspot masks for click-the-structure questions.

• Patients by filter: pathology category (tumor, vascular, trauma, inflammation, …), body region, scan type, sex, age, estimated contrast phase, or a specific case.

Options, with pictures

All images below are real output. They can be regenerated with uv run python scripts/make_docs_images.py.

Label style: render.preset

Every part of a preset can be overridden on its own: marker, tip, outline, shared_mode, text_style (halo / boxed / boxed_dark), marker_color, text_color, font_size, size (longest side in pixels, default 768) and format (png / webp / jpeg).

View: view

auto picks each structure's textbook view: axial for most organs, coronal for lung lobes and long bones, sagittal for vertebrae and the spinal cord. You can always force one.

Window: render.window

auto uses the lung window when any lung lobe is labeled, the bone window when most targets are bones, and soft tissue otherwise. Presets: soft_tissue, abdomen, liver, mediastinum, lung, bone, brain, angio, or any [width, level] pair.

Crop: render.crop

auto picks a region from the slice: neck (shoulders cut away), chest (arms cut away) or abdomen (whole trunk). You can force neck, chest or abdomen, or use body, targets (tight zoom) or full. A labeled structure is never cropped out.

Arrow tips and shared labels: render.tip, render.shared_mode

Which piece to point at: labels[].component

Some structures appear as two pieces on one slice. On an axial slice at T8, for example, you can see both the aortic root and the descending aorta. By default the arrow goes to the largest piece. Set component to a patient direction (anterior, posterior, left, right, superior, inferior) to pick the piece lying furthest that way. Directions that a view can't show (e.g. superior on an axial slice) are rejected.

Hotspot masks: hotspot_zones

Each zone becomes a transparent PNG the same size as the image, with the structure's pixels painted. Give one zone as the answer and the others as known wrong answers, so a wrong click can explain what the student actually clicked.

Choosing the slice: strategy, level, filters

labeled_image picks randomly among the best top_k (default 10) cases, so repeated calls vary. Pass a seed to make a result reproducible.

Quality filters are on by default and can each be relaxed:

Patient filters: pathology (no_pathology, tumor, vascular, trauma, inflammation, bleeding, other, unclear), pathology_location and study_type (substring match), gender, age_min, age_max, and phase (non_contrast, arterial, venous; off by default).

What comes back

{
  "image_path": "outputs/mcp/q001.png",
  "image_size": [768, 695],
  "answer_key": { "1": "pancreas", "2": "left kidney, right kidney" },
  "key": [
    {
      "label": "1",
      "structures": ["pancreas"],
      "display_names": ["pancreas"],
      "anchors_px": [[480.0, 318.5]],
      "label_box_px": [536.4, 302.0, 564.4, 335.0],
      "arrows_px": [[[536.4, 318.5], [481.5, 318.5]]],
      "outlines_px": [],
      "area_mm2": { "pancreas": 2250.0 }
    }
  ],
  "hotspot_masks": [{ "id": "pancreas", "structures": ["pancreas"], "path": "outputs/mcp/q001_mask_pancreas.png" }],
  "source": {
    "dataset": "TotalSegmentator v2.0.1",
    "case_id": "s1426", "view": "axial", "slice": 71, "vertebral_level": "L1",
    "window": "soft_tissue", "crop_region": "abdomen",
    "case": { "age": 68, "sex": "m", "study_type": "unknown", "pathology": "unknown", "contrast_phase_estimate": "non_contrast" },
    "attribution": "TotalSegmentator dataset v2.0.1 (Wasserthal et al., Radiology: AI 2023), CC BY 4.0"
  },
  "selection": { "strategy": "classical", "visibility": { "pancreas": 1.0 }, "areas_mm2": { "pancreas": 2250.0 } },
  "image_base64": "iVBORw0KG...",
  "mime_type": "image/png"
}

Arrows are [tail, tip] pairs in output-image pixels. outlines_px is filled when outlines are drawn. image_base64 is included unless you pass include_base64: false.

The tools

render_image, labeled_image and batch requests also take shuffle_labels (and render_image a seed), and every label takes component.

Examples

One call, five labels:

{
  "tool": "labeled_image",
  "arguments": {
    "labels": [
      { "label": "1", "structures": ["left brachiocephalic vein"] },
      { "label": "2", "structures": ["right brachiocephalic vein"] },
      { "label": "3", "structures": ["brachiocephalic trunk"] },
      { "label": "4", "structures": ["trachea"] },
      { "label": "5", "structures": ["esophagus"] }
    ]
  }
}

A specific vertebral level: "strategy": "level", "level": "T8"

An unlabeled arrow on the aortic arch: "labels": [{ "label": "", "structures": ["aorta"] }], "view": "axial", "strategy": "max_area"

A click-the-structure question:

{
  "tool": "labeled_image",
  "arguments": {
    "labels": [],
    "render": { "preset": "bare" },
    "hotspot_zones": [
      { "id": "answer", "structures": ["esophagus"] },
      { "id": "aorta", "structures": ["aorta"] },
      { "id": "heart", "structures": ["heart"] }
    ],
    "strategy": "level", "level": "T8"
  }
}

A disease question: call find_findings with finding: "thoracic_aortic_aneurysm", then render_image with the returned case_id and slice, labeling aorta (with component: "posterior"), esophagus and heart.

Choosing distractors yourself: call find_slices, then slice_contents on the chosen slice to see what else is visible, then render_image with the labels you want.

A full worked example is in examples/examsim_ct_bank.py. It is the script that made the images for a 14-question practice bank, talking to the server exactly as an AI would.

Real pathology

Most scans show normal anatomy, and the dataset's own pathology labels are coarse (for example "vascular, thorax"). So medimgq measures every scan and flags real findings:

list_findings returns this table, and find_findings returns the cases, most severe first, with the axial slice that shows each finding best. The thresholds are screening values for finding teaching cases, not diagnoses, so look at every image. Each threshold was checked by viewing flagged cases near the cutoff and at the extreme. A hiatal hernia detector was tried and dropped because it mostly flagged normal stomachs.

Limits and caveats

• Only segmented structures can be labeled. Nerves, the azygos vein, the internal jugular veins, laryngeal cartilages and bowel segments aren't available.

• A structure that appears twice on a slice (e.g. the aortic root and the descending aorta) gets its arrow on the larger piece unless you set component.

• Resolution is 1.5 mm, so images are slightly softer than clinical CT.

• Pathology metadata describes the whole scan, not the labeled structure.

• Contrast phase is an estimate from the brightness of the aorta and portal vein.

• Check every image before you publish it. Masks come from a model and were reviewed by the dataset authors, but mistakes are possible. Add bad cases to qc/blocklist.csv.

Project layout and development

src/medimgq/
  server.py      MCP server (9 tools)
  catalog.py     structure names, synonyms, groups, name resolution
  selection.py   slice search: strategies, textbook levels, filters
  render.py      labels, arrows, outlines, crop, window, hotspot masks
  volume.py      CT loading, radiological orientation, windows
  indexer.py     builds the structure/slice index
  quality.py     builds the quality index (noise, phase, truncation, arms)
  findings.py    builds the findings index (aneurysm, goiter, cysts, organ enlargement)
scripts/
  compare_strategies.py   contact sheet comparing slice strategies
  showcase_labels.py      contact sheet comparing label presets
  make_docs_images.py     regenerates the images in this README
examples/
  examsim_ct_bank.py      worked example: an MCP client building a question bank
qc/blocklist.csv          cases rejected after review
tests/test_unit.py        unit tests on synthetic scans (no dataset needed)
tests/test_server.py      end-to-end test over stdio (needs the dataset and indexes)

uv run pytest                                  # tests
uv run python scripts/compare_strategies.py    # compare slice strategies
uv run python scripts/showcase_labels.py       # compare label presets

Data and attribution

Images are derived from the TotalSegmentator dataset v2.0.1 (Wasserthal J. et al., TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images, Radiology: Artificial Intelligence, 2023), licensed CC BY 4.0. Every generated image carries this attribution in its footer.

The code is released under the MIT License.

For educational use only. Not for clinical decision-making.