This server provides an evidence-grade motion tracking and validation pipeline using pinned MMPose RTMW whole-body inference. It includes:
track_motion: Run inference on a source video, producing 133 raw landmarks per frame, normalized pose tracks, overlays, trajectories, and a manifest. Optional crop, device, model preset, and minimum score threshold. All tracks start in "tracked/unreviewed" state.
validate_track: Check schema, chronology, required joint coverage (hips, feet, body wrists), review attestations, and event lock. With production gate enabled, it fails unless all conditions are met.
inspect_track: Summarize coverage gaps, landmark uncertainty, events, review state, and gate results.
compare_motion: Mechanically compare a candidate track to a source across hips, feet, wrists, and hands, returning a tri-state judgment (completed/blocked) with per-lane blockers if applicable.
export_artifacts: Verify manifest integrity (SHA-256 hashes) and export a deterministic, content-addressed review bundle.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@grounded-motion-mcp track motion from video.mp4"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
grounded-motion-mcp is a standalone, agent-free motion tracking appliance. It runs pinned
MMPose RTMW whole-body inference, preserves all 133 raw landmarks and their exact detector scores,
normalizes the motion into a reviewable grounded-motion-track/v2 graph, and produces evidence for
feet, pelvis/root, hands, detector score, occlusion, and source chronology. RTMW SimCC scores are
backend-native maximum responses, not calibrated probabilities; values above 1 are preserved.
The model is a sensor. It does not author motion or certify its own output.
Every content-addressed job contains:
raw-predictions.json — immutable detector output.
pose-track.json — normalized landmarks with provenance.
pose-track-report.json — structural and production-gate findings.
trajectories.svg — pelvis, wrists, heels, and big-toe paths.
overlay.mp4 and overlay-slow.mp4 — full-speed and slow evidence.
manifest.json — artifact paths, sizes, and SHA-256 hashes.
receipt.json — exact input, backend, versions, device, status, and job identity.
Inference deliberately ends in tracked/unreviewed. validate_track with the production gate
enabled fails until every required hip, foot, and body-wrist landmark is source-witnessed, the
review attestations are present, and the event map is reviewed and locked. A claimed review flag
without witnessed landmark coverage fails closed.
Related MCP server: mcp-orchestrator-framework
track_motion
validate_track
inspect_track
compare_motion
export_artifacts
The default transport is local STDIO. The optional Streamable HTTP lane uses the same service code and expects files to be mounted under the configured workspace root.
Core tools and MCP server:
uv sync --extra devMMPose inference requires OpenMMLab's compiled runtime. The repeatable path is the supplied container:
docker build -t grounded-motion-mcp .For a native install, sync the pinned inference stack, then let OpenMIM install the matching compiled MMCV build into that environment:
uv sync --extra inference --extra dev
uv run mim install "mmcv==2.1.0"uv run grounded-motion --workspace /absolute/path/workspace track \
/absolute/path/source.mp4 \
--device cpu
uv run grounded-motion --workspace /absolute/path/workspace validate \
/absolute/path/workspace/grounded-motion/jobs/<job-id>/pose-track.json \
--production
uv run grounded-motion --workspace /absolute/path/workspace export \
/absolute/path/workspace/grounded-motion/jobs/<job-id>Use --crop x,y,width,height to lock a single subject crop. Coordinates are source pixels.
The entire source interval is decoded without resampling.
STDIO:
{
"mcpServers": {
"grounded-motion": {
"command": "uv",
"args": [
"--directory",
"/absolute/path/grounded-motion-mcp",
"run",
"grounded-motion-mcp"
],
"env": {
"GROUNDED_MOTION_WORKSPACE": "/absolute/path/motion-workspace"
}
}
}
}Agent-free HTTP appliance:
docker run --rm --gpus all -p 8000:8000 \
-v /absolute/path/data:/data \
-e GROUNDED_MOTION_TRANSPORT=streamable-http \
-e GROUNDED_MOTION_WORKSPACE=/data \
grounded-motion-mcpThe endpoint is /mcp. Do not expose it publicly without authentication and an origin policy.
Production uses MMPose 1.3.2 RTMW-X Cocktail14 at 384×288:
133 COCO-WholeBody landmarks.
Apache-2.0 MMPose code.
Explicit body, six foot, face, and 21 landmarks per hand.
The fixed Vanguard canary retains whole-image single-subject inference.
Arbitrary uploads use the hash-pinned RTMDet-m COCO-person detector and preserve every detected person in conservative identity segments. Ambiguous crossings, gaps, exits, and re-entry start new segments instead of inventing re-identification.
The preset records the upstream config and checkpoint URL in every receipt. Downloaded weights must be cached and hashed before production use.
tracked → reviewed → event-locked → keyed → transferred →
mechanically-compared → human-accepted
No earlier state implies a later one.
The comparison result is tri-state. judgment_status=completed permits a boolean
mechanical_pass; judgment_status=blocked always returns mechanical_pass=null with explicit
per-lane blockers. Structural inference is never silently promoted into judgment.
Render registration and pelvis/root translation are diagnostics. Root-relative mechanics subtract each lane's per-frame pelvis and normalize by that lane's hip-to-ankle scale before comparing hips, feet, and COCO body wrists. Render registration never moves root-relative mechanics. For sprite fixtures, all 42 detailed COCO hand landmarks remain in raw and normalized evidence but are explicitly quarantined from mechanical judgment; body wrists remain eligible after review.
The production profile accepts arbitrary hash-locked MP4s through either a ChatGPT attachment or a direct signed upload:
start_motion_tracking(source_file, expected_sha256?, crop?, minimum_score?, request_id?)
create_motion_upload(file_name, size_bytes, sha256, request_id?)
finalize_motion_upload(execution_id, crop?, minimum_score?, request_id?)
get_motion_status(execution_id)
get_motion_result(execution_id)
submit_motion_review(execution_id, tracked_evidence_sha256, subjects, excluded_subjects?, request_id?)
Attachments are copied from an approved temporary host into private GCS. Direct upload returns a one-use, 15-minute signed PUT URL bound to video/mp4 and a new object generation. Both paths lock the exact GCS generation and declared SHA-256, then run a CPU-only full-decode preflight before joining a FIFO queue. One user GPU execution runs at a time; the fixed health canary has a separate lock. Inputs are capped at 200 MiB and 10,000 decoded frames. Request ids make starts replay-safe.
The GPU worker runs the pinned RTMDet-m + RTMW-X stack and publishes immutable raw predictions, a multi-person track set, conservative identity findings, overlays with stable subject colors, trajectories, manifest, receipt, input lock, and reproducibility index. get_motion_result returns fresh 24-hour signed GET URLs. Tracking ends at tracked/unreviewed and event_lock_status=unlocked.
Review is a separate authenticated, human-only promotion. Every detector segment must be explicitly included or excluded with a reason. Included identities require interval, continuity, required-landmark, and event-map attestations; merges cannot overlap; sparse corrections cannot edit detector confidence; and unreliable landmarks may be quarantined with a reason. Successful review publishes reviewed tracks, event maps, reviewed overlays, trajectories, audit report, submission, and a new signed evidence index while retaining all raw detector evidence.
The immutable health canary remains available separately:
start_vanguard_canary()
get_vanguard_canary_status(execution_id)
get_vanguard_canary_result(execution_id)
start launches a one-task Cloud Run L4 GPU Job. The job tracks the immutable canonical Vanguard
Walk v1 and quarantined WalkSwordCarryV2 candidate 003 through the same GroundedMotionService
used by the CLI, verifies both manifests, runs the existing mechanical comparison, and publishes
private GCS evidence. result issues fresh 24-hour signed URLs for every artifact and the complete
evidence index. pipeline_pass proves real pinned MMPose inference, structural track validity,
artifact readback, and diagnostic completion. Because candidate 003 remains human-review pending,
the canary honestly returns judgment_status=blocked, mechanical_pass=null, and
human_accepted=false; it does not manufacture a reviewed event lock.
The fixtures preserve the eight source PNGs at Pose Lab commit
90ca534c46a47c660e7bf5ef7bd2efcf35dbeb9e and the eight candidate PNGs at immutable revision
b2c5bde5d91325726af34e5daea17b96d78b46f3. They are assembled as 82 frames at 100 fps using
repeats 11/9/10/11/11/9/10/11, with no interpolation. The paths, Git blob ids, file hashes,
video hashes, candidate quarantine status, and timing live in
src/grounded_motion_mcp/data/vanguard_canary.json.
Production authentication delegates identity only to Home Center OAuth. Tokens must be bound to
the production /mcp resource, carry grounded-motion:vanguard-canary, and identify
dclarke1005@gmail.com. Grounded Motion receives no Drive scope or Google refresh token.
Infrastructure is prepared by infra/bootstrap_gcp.sh inside the existing billed
home-center-dclar project, using isolated Grounded Motion service accounts, Artifact Registry,
bucket, Cloud Run service, GPU job, and a repository/main-constrained provider in the existing
github-actions WIF pool. The bootstrap refuses to continue unless the exact materialized videos
pass their stored SHA-256 values. Pull requests run .github/workflows/ci.yml. Every merge to main runs
.github/workflows/deploy-production.yml, builds the exact commit, deploys that image to the
CPU control service, CPU preflight/review job, and one-task L4 job, verifies the OAuth challenge, and completes a real GPU
canary before the deployment is green.
Compare candidate hips, feet, wrists, and detailed hands to source motion.
| Name | Required | Description | Default |
|---|---|---|---|
| report_path | No | ||
| trajectory_path | No | ||
| source_track_path | Yes | ||
| candidate_track_path | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully disclose behavior. It only states a compare operation without mentioning whether it is read-only, destructive, permissions required, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise, but it under-specifies the tool's behavior and parameters, making it less useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 4 parameters, 0% schema coverage, and no output schema details, the description is incomplete. It does not explain the return format or how results are presented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It does not mention source_track_path, candidate_track_path, or optional parameters. No parameter semantics are conveyed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the verb 'compare' and specifies the resources (hips, feet, wrists, hands) from candidate and source motion. It distinguishes from siblings like export_artifacts or track_motion, though it could be more specific about the nature of the comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Context signals show sibling tools are related but distinct, but the description offers no selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify the immutable manifest and export a deterministic review bundle.
| Name | Required | Description | Default |
|---|---|---|---|
| job_path | Yes | ||
| destination_path | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It hints at 'verify' (read-only check) and 'export' (write action), but does not clarify side effects, required permissions, or whether the manifest is modified. The description is insufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff, making it concise. However, it could be slightly more structured or informative without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is too brief given the tool's complexity (2 parameters, output schema present). It omits context like what kind of artifacts, how verification works, and what the bundle contains. The presence of an output schema reduces the need to explain returns, but other gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions neither job_path nor destination_path. With 0% schema description coverage, it fails to explain the parameters' roles or provide constraints beyond their names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description says 'verify the immutable manifest and export a deterministic review bundle,' which indicates a verification and export action, but the exact nature of the manifest and bundle is vague. It does not distinguish the tool from sibling tools like track_motion or validate_track.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There are no prerequisites, exclusions, or usage context mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize coverage gaps, uncertainty, events, review state, and gate results.
| Name | Required | Description | Default |
|---|---|---|---|
| track_path | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits like read-only nature or side effects, but it only states the output type (summary). It fails to mention whether the tool modifies data or requires special permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single focused sentence that front-loads the action and lists key outputs. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description omits the parameter role and behavioral context. It lists summary topics but doesn't anchor them to the parameter's source (track). Adequate but not fully informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter (track_path) has no description in the schema (0% coverage) and the tool description does not explain its meaning or format. The agent cannot determine what constitutes a valid track_path.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool summarizes specific aspects (coverage gaps, uncertainty, events, review state, gate results), distinguishing it from sibling tools like track_motion and validate_track which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites or context, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Track every source frame with RTMW whole-body landmarks and emit review artifacts.
The source must be inside GROUNDED_MOTION_WORKSPACE. Supply all four crop fields or none. Inference ends in tracked/unreviewed; it never implies motion acceptance.
| Name | Required | Description | Default |
|---|---|---|---|
| crop_x | No | ||
| crop_y | No | ||
| device | No | auto | |
| crop_width | No | ||
| crop_height | No | ||
| source_path | Yes | ||
| model_preset | No | rtmw-x-cocktail14-384x288 | |
| minimum_score | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that inference ends in tracked/unreviewed and never implies motion acceptance, adding behavioral context. However, it lacks details on side effects (e.g., artifact emission specifics) and error handling, which would be needed without annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using three short sentences to convey key points. It is front-loaded with the main action and constraints, but could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain what the emitted review artifacts are or how to interpret results. It also omits prerequisites like workspace access and fallback behaviors, leaving significant gaps for an 8-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate but only explains the crop fields grouping. It does not clarify the meaning of model_preset, minimum_score, device, or source_path beyond the required status.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool tracks every source frame with RTMW whole-body landmarks and emits review artifacts. It distinguishes itself from siblings like export_artifacts and validate_track by focusing on the tracking process.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit constraints: source must be in GROUNDED_MOTION_WORKSPACE, crop fields must be all-or-none, and inference ends in tracked/unreviewed without implying acceptance. It offers clear context but does not compare with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate schema, complete chronology, required joints, review state, and event lock.
| Name | Required | Description | Default |
|---|---|---|---|
| production | No | ||
| track_path | Yes | ||
| report_path | No | ||
| trajectory_path | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It lists validation items but does not state if the tool is read-only, has side effects, requires specific permissions, or what the output contains. The behavior is implied as validation but not explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and uses a comma-separated list. While concise, it is cryptic and lacks sentence structure. It could be more efficient by explaining the tool's purpose in a phrase rather than a list.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters with 0% schema coverage, an output schema exists but is not described. The description does not cover the return value or behavior. It is incomplete for effective tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any parameters. It does not mention track_path, production, report_path, or trajectory_path. The agent receives no help understanding parameter meaning beyond the schema names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description lists multiple validation aspects (schema, chronology, joints, review state, event lock) but does not clearly state the tool's primary function or scope. The verb 'validate' is present but the resource 'track' is implicit. It lacks distinction from sibling tools like inspect_track, which may also validate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There are no exclusions, prerequisites, or context indicating appropriate scenarios. The description does not help an agent decide between validate_track and sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0compare_motionexport_artifactsinspect_tracktrack_motionvalidate_trackScored across 5 tools
Each tool has a clear and distinct purpose: tracking frames, validating schema, inspecting coverage, comparing motion, and exporting artifacts. There is no overlap in functionality.
Most tools follow a verb_noun pattern (export_artifacts, track_motion, validate_track, inspect_track), but compare_motion uses 'motion' instead of 'track', causing a slight inconsistency.
With 5 tools, the server is well-scoped for motion capture and review tasks. Each tool contributes a distinct step in the workflow without being excessive.
The tools cover tracking, validation, inspection, comparison, and export, but lack lifecycle management tools such as updating review status, deleting tracks, or listing all tracks, which creates gaps in a full workflow.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for the Inistate platform: module discovery, entry management, and activity submission.
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
