svg-annotate-mcp

Annotate SVG paper figures in the browser — click an element (hovering live-highlights the semantic element under the cursor, Claude Design style) or lasso a region. The annotation (including the hit SVG element info) is sent back to Claude over MCP; Claude edits the source file (the SVG or the generating script), and the page watches for file changes and auto-refreshes — forming a closed "annotate → modify → refresh → re-annotate" loop.

• Backend: Python + official mcp SDK (MCPServer, stdio), with an in-process stdlib HTTP thread (127.0.0.1 ephemeral port).

• Frontend: single file src/svg_annotate_mcp/web/index.html. The SVG is fetched and injected via DOMParser into a Shadow DOM (bidirectionally isolated from the page's styles and IDs); annotations are drawn on a separate overlay, and hit testing happens browser-side at submission.

Installation and registration

cd ~/Projects/svg-annotate-mcp && uv sync
claude mcp add --scope user svg-annotate -- \
  uv run --directory /Users/boryant/Projects/svg-annotate-mcp svg-annotate-mcp

Optional: loosen the MCP client timeout so a single wait_for_annotations call can wait longer (it works without this too; the loop just runs more often): set MCP_TOOL_TIMEOUT=600000 in your shell profile or in the env block of the Claude Code settings.

Tool surface (4 tools)

Element-level click selection (v2)

Under the default "Select" tool, hovering immediately highlights the semantic element under the cursor (the g[id] groups from matplotlib such as text_N / line2d_N / legend_N, taking the minimum-area hit, with a small tag showing id + text). Clicking enters the element inspector (the side panel shows the id, verbatim text, and an ancestor breadcrumb — click the breadcrumb to reselect a parent group, e.g. text_54 → legend_1); an annotation is only produced when you type the "modification notes" (a bare click is just the inspector — no junk annotations). Esc or "Cancel selection" exits. Element annotations appear as dashed boxes + numbers on the figure.

Annotation payload structure (designed for Claude to locate)

Each annotation contains:

• kind (element/rect/arrow/freehand/text), note (the user's modification instruction), number (the number on the canvas);

• For kind:"element", a target field: the clicked element's tag/id/text (verbatim)/ancestors/d_prefix/bbox_svg — this is the strongest locator anchor; use it first (grep the id or text directly in the generating script or SVG);

• geometry_norm (0-1) and geometry_svg (in viewBox coordinates, already converted server-side);

• hits[]: the SVG elements the selection hit; each includes tag / id (semantic groups such as text_N/line2d_N in matplotlib) / text (element text, verbatim) / ancestors (ancestor id chain, e.g. ["figure_1","legend_1","text_54"]) / bbox_svg / coverage / d_prefix (first 30 chars of the path's d — a grep anchor when editing the SVG directly). Already denoised: background containers removed, semantic groups preferred (leaves inside groups not repeated), max 10 per annotation;

• texts_in_region: all the text inside the selected region (reading order, deduplicated) — the first lead for grep-based locating in the generating script.

What gets changed is Claude's call: when an annotation carries source_script, prefer editing the script and rerunning (edits to the SVG artifact would be overwritten by the next rerun); for SVGs with no script, edit the SVG file directly. The page makes no assumption about the editing method — it only watches for file mtime changes (500 ms polling, double-tick stabilization, and a </svg> tail check to avoid acting on half-written files).

Annotation lifecycle

Unsubmitted drafts survive figure refreshes. After clicking "Submit to Claude", the batch stays on the figure at 35% transparency (numbers retained, for easy cross-reference); the next refresh triggered by Claude's file edit clears the translucent annotations.

Typical closed loop

用户: 帮我改 figure11,我来圈
Claude: open_svg("/path/figures/figure11.svg", source_script="/path/fig_tri_complement.py")
        wait_for_annotations()          # 挂起
用户: (浏览器里圈图例写「图例移到右上」,点提交)
Claude: 收到批注 → set_status("正在改 fig_tri_complement.py…")
        → 改脚本 → 重跑出图 → 页面自动刷新
        → wait_for_annotations()        # 等下一轮

Testing

uv run python tests/smoke_test.py     # 端到端:握手/HTTP/阻塞等待/坐标换算/SSE reload/复用 tab
uv run python tests/manual_driver.py <svg> [script]   # 起 server 供手动/浏览器自动化测试,批次落盘 tests/out/batches.jsonl

Page debug parameters: ?nosse=1 skips SSE (for headless screenshots); ?autotest=x,y,w,h auto-draws a rectangle annotation and submits it after load (0-1 normalized coordinates); ?autotest_click=x,y click-selects the element at that point and submits it as a real annotation (add &autotest_stage=pick to just do the click-highlight, for inspector screenshots).

Environment variables: SVG_ANNOTATE_NO_OPEN=1 prevents open_svg from auto-opening the browser (for testing).

Known limitations and (first version)

• Single-session instance (one figure at a time; another open swaps the figure; multiple Claude sessions = independent server instances, no mutual interference);

• No annotations.json export (annotations are only returned through memory and are lost on server restart — Claude's context is the fallback copy);

• External resources in an SVG are not proxied (open_svg will warn; matplotlib output is all embedded, so this does not trigger);

• Browser opens only via macOS open;

• No auth on localhost.