detection-mcp
This server is a local MCP tool for object-detection annotation workflows: it manages datasets, categories, images, boxes (axis-aligned and rotated), previews, and JSONL exports without modifying source images.
Register, list, soft-delete, restore, and fetch datasets tied to authorized directory roots
Add, edit, delete, restore, list, and fetch categories per dataset
Discover and page through dataset images, filter by annotation status (unannotated, in_progress, completed), and order by name or deterministic random seed
Set an image’s workflow status without altering the image file
Preview orientation-corrected images as PNGs with size/scale metadata
Preview in-memory annotation overlays, optionally filtered by annotation IDs, geometry type, or deleted-category visibility
List annotations with filters for image, category, annotation ID, and geometry type
Atomically add axis-aligned bounding boxes using normalized xyxy coordinates
Edit or hard-delete bbox annotations while preserving their geometry type
Atomically add rotated bounding boxes from normalized 8-coordinate polygons, with correction diagnostics (submitted vs stored geometry, corrected flag, deviation, warning)
Edit or hard-delete rotated bbox annotations
Export completed-image metadata to JSONL in AutoTrain mode (pixel xywh bboxes, zero-based category mapping, negative samples, rotated boxes omitted) or extended mode (adds polygon/polygon_category, nested paths, WebP support, deleted-category exclusion reporting)
Export uses atomic file replacement with locking, preflight checks, overwrite control, and reports counts plus category mappings
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@detection-mcp list all images in the training dataset that are still pending annotation"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
detection-mcp
detection-mcp is a local STDIO Model Context Protocol server for object-detection annotation. It stores workflow state and annotations in SQLite, renders review previews in memory, and exports JSONL without modifying source images.
Status
The v1 implementation is available for review. It exposes 23 tools for datasets, categories, images, axis-aligned boxes, rotated boxes, previews, and export. The package targets Python 3.12 or newer and uses fastmcp>=3.4.7,<4.0.0, allowing compatible 3.x security updates.
Related MCP server: z_ai_vision_mcp_server_clone
Install
Install uv, then install the published package:
uv tool install detection-mcp
detection-mcp --versionFor repository development, clone the project and run:
uv sync --locked --all-groups
uv run pre-commit install --install-hooks --hook-type pre-commit --hook-type commit-msg --hook-type pre-pushConfigure an MCP client
Use an installed executable and grant only the directories the task needs:
{
"mcpServers": {
"detection-mcp": {
"command": "detection-mcp",
"args": [
"--db-path", "/var/lib/detection-mcp/annotations.db",
"--allowed-dataset-root", "/srv/datasets",
"--allowed-export-root", "/srv/exports"
]
}
}
}The server uses STDIO: protocol messages go to stdout and logs go to stderr. CLI options override DETECTION_MCP_* environment variables, followed by built-in defaults.
Configuration reference
Purpose | CLI | Environment | Default |
SQLite file |
|
| platform user-data directory |
Dataset roots | repeated |
| unrestricted |
Export roots | repeated |
| unrestricted |
Random order seed |
|
|
|
Preview width |
|
|
|
Preview height |
|
|
|
Rotated correction |
|
| enabled |
Correction threshold |
|
|
|
Rejection threshold |
|
|
|
Logging |
|
|
|
Environment root lists use the operating-system path separator. Empty allowed-root lists retain the portable default, but deployments should configure explicit roots. Dataset permission never grants export permission. Run detection-mcp --version without starting an MCP session.
Annotation workflow
Register a dataset root and define categories.
List images by status or deterministic random order.
Preview an orientation-corrected image.
Add normalized axis-aligned or rotated boxes in atomic batches.
Preview the overlay and correct annotations before marking the image completed.
Export completed images to AutoTrain or extended JSONL.
Install the Agent Skills
The installable Agent Skills for detection-mcp users live in the repository's root
skills/ directory. They are not included in the Python wheel, source distribution,
or container image. Install either Skill directly from GitHub with the skills CLI:
npx skills add ryan-minato/detection-mcp --skill object-detection-annotation
npx skills add ryan-minato/detection-mcp --skill detection-mcp-setupThe default installation is project-local. Add --global when the Skill should be available across projects.
Tool groups
Area | Tools |
Datasets |
|
Categories |
|
Images and review |
|
Annotations |
|
Export |
|
Every ordinary success returns { "ok": true, "data": ... }. Domain failures return { "ok": false, "error": { "code": ..., "message": ... } } as an MCP error result. Preview tools return PNG image content plus the same structured envelope. The schemas returned by list_tools are the machine-readable contract.
Export format
export_metadata_jsonl exports only images marked completed. It writes to a temporary file, flushes it, and atomically replaces the destination while holding an exclusive lock file.
AutoTrain mode
AutoTrain mode requires at least five completed images in a flat dataset root. Files must use .jpg, .jpeg, or .png. Each JSONL object contains file_name and objects; bbox values are pixel [x, y, width, height] coordinates and categories are zero-based export indices.
Rotated boxes are not representable in the AutoTrain bbox schema. They are counted in ignored_rotated_annotations and omitted. Images without annotations remain negative samples with empty arrays.
Extended mode
Extended mode also emits polygon and polygon_category arrays. Each polygon is eight pixel coordinates in canonical point order. Nested portable image paths and WebP images are allowed.
Deleted-category annotations are excluded and reported. Existing output is rejected unless overwrite=true; a concurrent lock is also reported as an existing-output error.
Development commands
just is the repository command executor:
just sync # synchronize the locked environment
just test # run ordinary tests
just quality-control # run CI checks without tests
just quality # run the complete local commit gate
just hooks # run repository hooks against tracked files
just check # run quality and hooksNever bypass Git hooks. Every commit requires the full quality gate and a staged secret/PII scan. See CONTRIBUTING.md, SECURITY.md, and AGENTS.md.
Containers
Release images are published to GitHub Container Registry. Pull latest for the
current stable release or use a version tag for a fixed release:
docker pull ghcr.io/ryan-minato/detection-mcp:latestThe production image runs as the non-root detection-mcp system user and starts the STDIO server directly. To build it from this repository, set two host paths and launch the example:
export DETECTION_MCP_DATASET_PATH=/srv/datasets
export DETECTION_MCP_OUTPUT_PATH=/srv/detection-mcp-output
docker compose -f docker-compose.example.yml run --rm detection-mcpThe example mounts /datasets read-only, /output writable, and /state from a named persistent volume. Pass container paths such as /datasets/coco-subset and /output/metadata.jsonl to MCP tools; host paths are not visible inside the container.
An MCP desktop client can use docker run --interactive with the same mounts as its STDIO command. Do not mount a home directory or filesystem root. The database must not share the read-only dataset mount. Back up the state volume and export output independently; source images remain outside this server's ownership.
License
Apache-2.0. See LICENSE.
Available Tools
23 toolsadd_bbox_annotationsC
Add normalized xyxy annotations in one transaction.
Inputs: dataset_id (required), image_path (required), annotations (required). Returns: ok, data
add_categoriesB
Add categories atomically to an active dataset.
Inputs: categories (required), dataset_id (required). Returns: ok, data
add_rotated_bbox_annotationsC
Validate, correct, and add rotated annotations atomically.
Inputs: dataset_id (required), image_path (required), annotations (required). Returns: ok, data
create_datasetC
Register a dataset root without changing any source image.
Inputs: root_path (required), name. Returns: ok, data
delete_bbox_annotationA
Hard-delete one or more bbox annotations atomically.
Inputs: dataset_id (required), annotation_ids (required). Returns: ok, data
delete_categoryB
Soft-delete a category and retain historical annotations.
Inputs: dataset_id (required), category_id (required). Returns: ok, data
delete_datasetA
Soft-delete a dataset and preserve all related state.
Inputs: dataset_id (required). Returns: ok, data
delete_rotated_bbox_annotationB
Hard-delete one or more rotated annotations atomically.
Inputs: dataset_id (required), annotation_ids (required). Returns: ok, data
edit_bbox_annotationB
Edit a bbox annotation without changing its type.
Inputs: dataset_id (required), annotation_id (required), bbox, category_id. Returns: ok, data
edit_categoryC
Change a category name or authoritative description.
Inputs: dataset_id (required), category_id (required), name, description. Returns: ok, data
edit_rotated_bbox_annotationC
Edit a rotated annotation without changing its type.
Inputs: dataset_id (required), annotation_id (required), polygon, category_id. Returns: ok, data
export_metadata_jsonlB
Preflight and atomically export completed-image metadata.
Inputs: dataset_id (required), output_path (required), overwrite, export_mode. Returns: ok, data
get_categoryB
Get a category, including a soft-deleted category.
Inputs: dataset_id (required), category_id (required). Returns: ok, data
get_datasetC
Get dataset metadata, including a deleted dataset.
Inputs: dataset_id (required). Returns: ok, data
list_annotationsB
List annotations with stable filters and pagination.
Inputs: dataset_id (required), offset, image_path, max_results, +4 more. Returns: ok, data
list_categoriesB
List categories for a dataset.
Inputs: dataset_id (required), include_deleted. Returns: ok, data
list_datasetsB
List registered datasets.
Inputs: include_deleted. Returns: ok, data
list_imagesB
Discover dataset images with status filtering and stable ordering.
Inputs: dataset_id (required), offset, status, order_by, +2 more. Returns: ok, data
preview_annotationsC
Return an in-memory annotation overlay and metadata.
Inputs: dataset_id (required), image_path (required), max_width, max_height, +3 more. Returns: ok, data
preview_imageC
Return an orientation-corrected preview and size metadata.
Inputs: dataset_id (required), image_path (required), max_width, max_height, +1 more. Returns: ok, data
restore_categoryC
Restore a category, optionally under a new name.
Inputs: dataset_id (required), category_id (required), new_name. Returns: ok, data
restore_datasetB
Restore a soft-deleted dataset record.
Inputs: dataset_id (required). Returns: ok, data
set_image_statusA
Set annotation workflow status without changing the image.
Inputs: status (required), dataset_id (required), image_path (required). Returns: ok, data
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 23 tools
Each tool targets a distinct resource (dataset, category, image, bbox, rotated bbox, export) with clear action verbs. Even the two annotation types are unambiguously separated by bbox vs rotated_bbox in their names.
All tools follow a consistent verb_noun snake_case pattern, with clear verbs like create, delete, list, edit, add, get, restore, set, preview, export. Minor pluralization inconsistencies (e.g., add_bbox_annotations vs edit_bbox_annotation) do not undermine overall predictability.
At 23 tools, the server sits in the heavy range (16-25). However, the number is justified by the broad domain covering datasets, categories, images, two annotation types, and export, so it feels borderline rather than excessive.
The surface covers full CRUD lifecycles for datasets, categories, and both annotation types, plus image status management and export. A notable gap is the lack of a dataset update operation, but core workflows are well-supported.
Related MCP Connectors
An MCP server for search and retrieval over a local LanceDB database
Collaborative whiteboard MCP server — create objects, connectors, C4 diagrams, and manage boards
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
Hosted MCP server for AI-driven data ops. Create apps, manage schemas, and CRUD structured data.
Related MCP Servers
AlicenseNot gradedqualityDmaintenanceAn MCP server that enables automated dataset creation and custom object detection model training through natural language interactions. It integrates foundation models like GroundedSAM for auto-labeling and supports training specialized YOLOv8 models using local or Unsplash images.4MIT- FlicenseBqualityDmaintenanceOpenAI-compatible MCP server for running image analysis tools against your own vision model endpoint.717 npm-
- AlicenseNot gradedqualityDmaintenanceA filesystem-backed MCP server with REST API for AI tool integration and project workflow tracking. Zero database required.547 npmMIT
- AlicenseAqualityCmaintenanceA local (stdio) MCP server for PixelVault — agent-first image hosting, enabling upload of local files by path without base64.613 npmMIT