cosplay-vp
Converts COLMAP camera solves into Unreal Engine camera parameters (location, pitch/yaw/roll, FOV, focal length) and provides a UE5 Python script to import a solved camera as a keyed CineCamera, enabling virtual production compositing in Unreal.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@cosplay-vpProbe take3.mp4, key it, and solve the camera move for Unreal"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
cosplay-vp
Open-source tools, AI agent skills and a knowledge base for CG cosplay virtual production.
Film a real cosplayer on a phone. Cut them out, recover the camera move, put them inside an Unreal Engine world, and finish with anime-style cut-ins and impact frames, for recorded shots or a live event booth.
Plan ─► Shoot ─► Prep ─► Matte ─► Track ─► Unreal comp ─► Anime FX ─► DeliverDocs site: https://astralrc.github.io/cosplay-vp/ · Start with: Prompt library · Techniques A–Z · Showcase · How we built this
What's in the box:
Part | What it is |
Knowledge base: shooting, matting, tracking, Unreal comp, anime FX, live booth architectures (Aximmetry vs Pixel Streaming), rights and consent, open questions | |
Python library and MCP server with 14 tools built on ffmpeg and COLMAP | |
6 Agent Skills (SKILL.md) that teach an AI agent the pipeline step by step | |
Unreal Engine 5 Python script that imports a solved camera as a keyed CineCamera | |
Round-trip tests for the COLMAP → Unreal maths and ffmpeg tools on synthetic footage |
MCP tools
Tool | Does |
| Resolution, fps, rotation, alpha, with warnings for phone quirks |
| Constant frame rate, rotation baked in |
| Numbered frames for COLMAP or matting |
| Green/blue key + despill to ProRes 4444, VP9 alpha or PNG |
| Runs an AI matting model you installed (RVM, MatAnyone, ...) |
| Alpha foreground over a background to check edges |
| COLMAP sparse reconstruction, exported as TXT |
| COLMAP cameras → Unreal location (cm), pitch/yaw/roll, FOV, focal length |
| Anime "on twos" timing |
| Invert or white flash frames on hits |
| Raw vs final demo |
| 9:16 crop for TikTok, Reels and Shorts |
| Plan a live booth's latency against a target |
| Read any page of the knowledge base |
Related MCP server: AfterEffectsMCP
Quick start
Requirements: Python 3.10+, ffmpeg. Optional: COLMAP, Unreal Engine 5, Blender, OBS.
git clone https://github.com/AstralRC/cosplay-vp.git
cd cosplay-vp
pip install -e ".[dev]"
pytest # 14 tests, including a live MCP stdio sessionConnect the MCP server
Claude Code
claude mcp add cosplay-vp -- cosplay-vp-mcpClaude Desktop (claude_desktop_config.json), or any MCP client:
{
"mcpServers": {
"cosplay-vp": { "command": "cosplay-vp-mcp" }
}
}If the command isn't on your PATH, use "command": "python", "args": ["-m", "cosplay_vp.server"].
Install the skills
Claude Code: copy the folders in
skills/to~/.claude/skills/(all projects) or.claude/skills/(one project).Claude apps: zip a skill folder and upload it under Settings → Capabilities → Skills.
Then ask, for example:
Plan a 15-second ultimate-move cut-in for an original character, then prep
take3.mp4, key it, and track the camera into Unreal.
Without an AI agent
cosplay-vp-colmap2ue colmap_ws/sparse_txt camera.csv --scale 100and in Unreal's Python console:
py "path/to/unreal/import_camera_csv.py" "path/to/camera.csv" 24Status
Area | Status |
ffmpeg tools, COLMAP → UE conversion | Working, tested in CI |
| Wraps the COLMAP CLI; needs COLMAP installed |
Unreal import script | Written for UE 5.x, not yet tested across versions; reports welcome |
Live booth architectures B and C (cloud) |
Roadmap
Local booth prototype: Unreal + OBS key, measured latency.
Cloud experiment: Pixel Streaming with a camera uplink, latency against the local build.
Matting benchmark: hair and lace edges, speed per model.
Unreal MCP bridge: place media plates and queue Movie Render Queue jobs from an agent.
Rotation-only (nodal) solve for tripod pans.
Ground rules
Only film people who agreed, with guardian consent for anyone under 18.
Paid work uses characters the client owns or licensed, or original characters.
No ripped game assets, logos or music in this repo.
Label ads, and label AI-generated content where platforms require it.
Details: docs/08-rights-consent-disclosure.md.
Contributing
Anyone can contribute: fork, change, pull request. Benchmarks with methods, fixes for new Unreal versions, matting comparisons, techniques, prompts, showcase entries and translations are all welcome. Issue templates are ready for each. Regular contributors become triage collaborators: see GOVERNANCE.md and CONTRIBUTING.md.
Licence
Code: MIT. Docs and skills: CC BY 4.0. External tools keep their own licences (doc 10).
Started by Idea (Phakpaphon Jaiwanglok) of AstralRC / CELESTI.
Available Tools
14 toolschroma_keyB
Green/blue screen key to a clip with alpha (.mov ProRes 4444, .webm VP9, or %05d.png). A fast first pass.
| Name | Required | Description | Default |
|---|---|---|---|
| blend | No | ||
| color | No | 0x00FF00 | |
| input | Yes | ||
| output | Yes | ||
| despill | No | ||
| similarity | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations the description must carry the full behavioral burden. It usefully discloses the resulting alpha output and container/codec options (.mov ProRes 4444, .webm VP9, PNG sequence) and a speed characteristic, but says nothing about overwrite behavior, quality tradeoffs, or how 'fast first pass' differs in fidelity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the core operation followed by the output-format and speed detail. No wasted words, though the brevity comes at the cost of explaining any of the six parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a six-parameter, no-annotation, no-output-schema mutation tool with 0% schema coverage, the description is too thin. It gives output formats and a speed hint but leaves keying parameters and the meaning of the returned alpha entirely undocumented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there are six parameters (blend, color, despill, similarity, input, output). The description explains none of the tuning parameters and only loosely implies the output format, so it fails to compensate for the schema's complete lack of parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific operation (green/blue screen key) and its result (clip with alpha), and even enumerates the supported output codecs. It does not explicitly name a sibling, but 'a fast first pass' implicitly separates it from a slower, higher-quality matte tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'A fast first pass' implies a quick/preview use case, hinting that a more thorough sibling (e.g. matte_external) should be used for final quality. However, there is no explicit when-to-use/when-not guidance or named alternative, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
colmap_solveC
Solve the camera from a frame folder with COLMAP (must be installed) and export the model as TXT.
| Name | Required | Description | Default |
|---|---|---|---|
| quality | No | medium | |
| image_dir | Yes | ||
| workspace | Yes | ||
| single_camera | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It discloses the COLMAP installation prerequisite and that the model is exported as TXT, but it omits critical behavioral details such as whether it writes files to the workspace, expected runtime, GPU requirements, overwrite behavior, and error modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words. The purpose, prerequisite, and output format are packed efficiently, and every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 0% schema coverage, no annotations, no output schema, and four parameters, the description is too sparse. It covers the high-level purpose and prerequisite but leaves parameter meanings, usage routing, and behavioral side effects largely unexplained, so an agent cannot invoke it confidently with correct arguments.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all four parameters. It only indirectly clarifies image_dir as 'a frame folder' and gives no meaning for workspace, quality, or single_camera, leaving most parameter semantics undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Solve the camera'), resource ('from a frame folder'), and implementation ('with COLMAP'), plus an output format ('export the model as TXT'). It is clear what the tool does, but it does not explicitly differentiate itself from the sibling tool colmap_to_ue or other frame-processing siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives, nor any when-not conditions. The only usage-adjacent information is the prerequisite that COLMAP must be installed, which is useful but does not constitute selection guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
colmap_to_ueA
Convert COLMAP cameras.txt + images.txt into Unreal camera keys (.csv or .json): location in cm, pitch/yaw/roll, FOV. scale = UE centimetres per COLMAP unit. Import with unreal/import_camera_csv.py.
| Name | Required | Description | Default |
|---|---|---|---|
| scale | No | ||
| output | Yes | ||
| sparse_txt_dir | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does disclose output format (.csv or .json), the units of the emitted values, and the meaning of scale, but says nothing about whether existing output files are overwritten, whether the source directory is modified, or what happens on parse failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences, front-loaded with the conversion and output contents before the scale semantics and the downstream import hint. Every clause carries information; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description does well on the transformation and result shape, but leaves two of three parameters undocumented and omits error/overwrite behavior. Adequate but with clear gaps for a 3-parameter file-writing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains scale well (UE centimetres per COLMAP unit), but output and sparse_txt_dir are only inferable from context, with no format/path expectations stated for either.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a precise verb+resource: converting COLMAP cameras.txt + images.txt into Unreal camera keys, and enumerates the output fields (location in cm, pitch/yaw/roll, FOV). This is clearly distinct from the sibling colmap_solve, which only produces the COLMAP reconstruction this tool consumes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the pipeline framing (convert COLMAP output, then import with unreal/import_camera_csv.py), but there is no explicit when-to-use, prerequisites, or statement of when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
composite_previewB
Preview an alpha foreground over a background still or render. For checking edges, not the final comp.
| Name | Required | Description | Default |
|---|---|---|---|
| height | No | ||
| output | Yes | ||
| background | Yes | ||
| foreground | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It signals a non-final preview operation, but says nothing about whether it writes files, what format 'output' takes, or what permissions/alpha conventions are required. For a tool that takes an output path, that is a meaningful gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and immediately followed by the scoping caveat. Every word earns its place, with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A 4-parameter tool with 0% schema coverage, no annotations, and no output schema leaves the description responsible for format, resolution defaults, and return behavior — none of which are covered. The definition is too thin for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 4 parameters. The description only maps the foreground as an 'alpha foreground' composited over 'background', leaving output, height, and the expected file formats entirely unstated. It compensates only minimally for the zero-coverage schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: preview an alpha foreground over a background. It also qualifies the intent ('checking edges, not the final comp'), which separates it from sibling compositing tools like chroma_key or matte_external. It stops short of naming a sibling explicitly, so it lands at 4 rather than 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'For checking edges, not the final comp' tells the agent both when to reach for it and, by exclusion, when it is the wrong tool. No alternative tool is named for producing the final comp, which keeps it below 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
conformA
Re-encode phone footage to a constant frame rate with rotation baked in. Do this before tracking or matting.
| Name | Required | Description | Default |
|---|---|---|---|
| fps | No | ||
| input | Yes | ||
| output | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It usefully discloses the transformation semantics (CFR normalization, rotation baked into pixels) and sequencing, but says nothing about overwrite behavior on the output path, whether input is modified, codec/audio handling, or runtime cost for long footage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, action front-loaded and the timing constraint placed second. No filler, no restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema and no annotations, and the tool writes a new media file from a required input/output pair. The description covers the core transformation but omits overwrite semantics, output format behavior, and failure modes that an agent would need to invoke this safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does add meaning for fps by tying it to the 'constant frame rate' target, but input/output are left entirely to the schema names and the default of 24 fps is never surfaced in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb (re-encode) and resource (phone footage) plus the two transforms applied: constant frame rate and baked-in rotation. It does not name siblings explicitly, but the phrase 'before tracking or matting' implicitly positions it relative to other tools in the workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Do this before tracking or matting' gives an explicit ordering constraint that tells the agent when this tool belongs in a pipeline. It stops short of naming alternative tools or stating when not to use it, so it is clear context without full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
extract_framesB
Export numbered frames (00001.jpg ...) for COLMAP or matting. Use fps to thin long clips.
| Name | Required | Description | Default |
|---|---|---|---|
| ext | No | jpg | |
| fps | No | ||
| input | Yes | ||
| start | No | ||
| out_dir | Yes | ||
| duration | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses the output naming convention, but says nothing about overwriting existing files in out_dir, directory creation, codec/quality defaults, or return values - all relevant for a write-heavy export operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the core action and output naming front-loaded ahead of the usage tip. Nothing needs trimming.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A 6-parameter tool with no annotations, no output schema, and 0% schema coverage needs substantially more description than two sentences. The agent lacks guidance on start/duration semantics and on what the export produces or overwrites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 6 parameters, so the description must compensate. It clarifies only 'fps' (thinning rate) and hints at 'ext' via the .jpg example; 'start', 'duration', 'input', and 'out_dir' receive no explanation beyond their bare names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb+resource ('Export numbered frames') and even gives the naming pattern (00001.jpg). It identifies downstream targets (COLMAP, matting), but does not explicitly differentiate itself from related siblings like impact_frames or on_twos.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the intended pipelines ('for COLMAP or matting') and gives one actionable usage tip ('Use fps to thin long clips'), which implies when to set fps. However, it offers no when-not guidance and does not route the agent away from similar frame-producing siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
impact_framesC
Add anime impact flashes at frame numbers. mode: 'invert' or 'white'.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | invert | |
| input | Yes | ||
| frames | Yes | ||
| length | No | ||
| output | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It does not state that this is a write operation producing a new file, whether the input is preserved, whether frames are 0- or 1-indexed, or any permission/format requirements. 'mode' options add minor signal but little behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with zero padding, and the primary action is front-loaded ahead of the mode detail. Appropriately sized, though the brevity reflects under-specification rather than disciplined compression.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter, no-annotation, no-output-schema mutation tool operating on video files, the description is too thin. It omits input/output file semantics, the meaning of 'length', frame indexing, and return behavior, leaving an agent unable to call it confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the schema provides only bare titles. The description explains the 'mode' values ('invert' or 'white') and ties 'frames' to impact placement, partially compensating. However, 'input', 'output', and 'length' remain completely undocumented, leaving most parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Add anime impact flashes') plus a scoping constraint ('at frame numbers'), so an agent understands it applies effects at given frames. It does not differentiate itself from siblings like chroma_key or composite_preview, but the core purpose is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this versus the many sibling effects tools (chroma_key, matte_external, composite_preview) or any prerequisites. Usage is implied only by the narrow description of the effect.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
knowledgeA
Read the project's knowledge base. Call with no topic to list topics.
| Name | Required | Description | Default |
|---|---|---|---|
| topic | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden. It does reveal the dual-mode behavior (omit topic to enumerate topics, supply topic to read) that the schema does not express, but it never says what a read returns, how unknown topics fail, or whether the knowledge base is large enough to need paging.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short imperative sentences with no filler, and the primary action is front-loaded ahead of the optional-argument hint. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-optional-parameter read tool with no output schema and no annotations, the description covers the essential calling contract, including the no-argument mode. Only failure behavior for invalid topics is left unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the undocumented 'topic' parameter. It partially does by explaining the omitting-topic case, implying topic is a selector, but it gives no format, case sensitivity, or valid-value examples for a free-string parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Read) and resource (the project's knowledge base), which is more than the bare name 'knowledge' conveys. It does not reference any sibling, but the surrounding siblings (probe, conform, chroma_key, colmap_solve) are all media-pipeline tools, so there is no plausible confusable alternative to differentiate against.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives one concrete calling rule: 'Call with no topic to list topics,' which tells the agent how to discover valid input before reading. It stops short of saying when to prefer this tool over other discovery routes or what to do on an unknown topic, but for a single-parameter tool the guidance is actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
latency_budgetC
Sum estimated latency per stage of a live booth (camera, key, render, stream, display) against a target.
| Name | Required | Description | Default |
|---|---|---|---|
| stages_ms | Yes | ||
| target_ms | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It implies a pure estimation/comparison, but never states that it is side-effect free and non-destructive, that it measures nothing real ('estimated'), or what happens when a stage is absent or the total exceeds target — meaningful gaps for a zero-annotation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence with the computation and the comparison target front-loaded; there is no filler. It is slightly crammed, packing stage enumeration and target comparison without any visual separation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and 0% schema description coverage over a nested-object parameter, the description should do more. It conveys the inputs conceptually but never says what comes back (per-stage breakdown, total, pass/fail against target), leaving the agent to guess the result shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% with a nested free-form object, so the description must compensate, and it partially does: the parenthetical stage names (camera, key, render, stream, display) hint at expected keys for stages_ms, which the schema's additionalProperties does not convey. However units beyond the parameter name, the target_ms default of 100, and whether keys are arbitrary or fixed remain unstated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb ('sum') and resource (estimated latency per stage), and enumerates the concrete stages (camera, key, render, stream, display), which is far more than a restated title. It is clearly distinct from every sibling (probe, chroma_key, colmap_solve, etc.), but it never explicitly contrasts itself with any of them, so sibling differentiation is implied rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no exclusions; nothing tells the agent when to reach for this versus probe or composite_preview. The only cue is the implied planning/design context of a 'live booth', which the agent must infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
matte_externalB
Run an AI matting tool the user installed (e.g. RVM, MatAnyone) using {input}/{output} placeholders. Check that tool's licence first.
| Name | Required | Description | Default |
|---|---|---|---|
| input | Yes | ||
| output | Yes | ||
| command_template | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It usefully discloses that the binary is user-supplied/external and that a licence check gates execution, which are non-obvious behavioral facts. It says nothing about failure modes when the external tool is missing, output format, or runtime behaviour.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action and no filler. The licence warning is effectively placed second as a caveat. Efficient, though extremely terse given the 0% schema coverage it is meant to offset.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and three wholly undocumented required parameters, an agent is left guessing at command_template syntax and what a run returns. The description covers the concept and one prerequisite but is not complete for a tool this unannotated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain that {input} and {output} are placeholders substituted with the input/output paths, which gives those two parameters meaning the schema lacks. The third required parameter, command_template, is never explained — how to build it or what syntax is expected.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — run an external AI matting tool the user installed — with concrete examples (RVM, MatAnyone). The 'external/user-installed' framing distinguishes it reasonably from sibling tools like chroma_key, though it never names a sibling explicitly. An agent can tell what it does without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description supplies one real prerequisite ('check that tool's licence first') and implies the placeholder convention for invocation. It does not say when this is preferable to chroma_key or other matting/keying siblings, nor when not to use it. Usage is implied rather than routed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
on_twosC
Give motion the anime 'on twos' feel by holding each frame hold times at the same frame rate.
| Name | Required | Description | Default |
|---|---|---|---|
| fps | No | ||
| hold | No | ||
| input | Yes | ||
| output | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose the core mechanism: each frame is held `hold` times at an unchanged frame rate. It says nothing about input/output formats, whether source footage is altered, or side effects, which leaves the mutation profile under-specified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tightly-written sentence with the mechanism front-loaded and no filler. It is efficient, though arguably too terse for a 4-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple transformation with no output schema to rely on, the description covers the core behavior but omits input/output expectations and edge cases (e.g. non-integer hold, fps mismatch). It is minimally adequate rather than complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains `hold` concretely ('holding each frame hold times') and hints at `fps` ('same frame rate'), but says nothing about `input`/`output` paths or accepted formats, leaving most required parameters undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description conveys a specific effect (anime 'on twos' look) via frame holding, which is more than a tautology. However, it never states the operation in plain terms (e.g. 'convert a video'), relies on the niche term 'on twos', and gives no signal that separates it from siblings like extract_frames or impact_frames.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use, when-not-to-use, or alternative guidance. The agent must infer that this is for post-processing an existing frame stream and cannot tell how it relates to the frame-source siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
probeB
Inspect a clip: resolution, fps, duration, codec, rotation, alpha, plus warnings (variable frame rate, rotation flags).
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It does add real behavioral value by disclosing the diagnostic content returned, specifically variable-frame-rate and rotation warnings, which an agent would not guess from the name alone. It still omits whether the tool touches the file, what happens on an unreadable path, and whether output is a structured report.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the verb and resource, with the return contents trailing behind. Every clause earns its place as a distinct inspection field; the parenthetical warning list is efficient rather than padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with no annotations and no output schema, the description does the heavy lifting by enumerating the returned fields, which is what an agent needs to decide whether to call it. Remaining gaps (error behavior, exact output shape) are minor given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the single 'path' parameter is undocumented in the schema, and the description does not compensate by stating whether it is a file path, directory, or whether globs/URLs are accepted. The parameter name is largely self-explanatory, which keeps this at a minimum-viable 3 rather than lower.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Inspect') and resource ('a clip') and then enumerates exactly what is surfaced: resolution, fps, duration, codec, rotation, alpha and warnings. That is far more concrete than a tautology. It stops short of a 5 only because it never contrasts itself with siblings, though no sibling appears to perform inspection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use guidance, no prerequisite (e.g. 'call before transcoding to detect variable frame rate'), and no mention of alternatives such as directly reading metadata elsewhere. The purpose implies inspection, but nothing tells the agent when this tool is the right choice versus the rest of the pipeline.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
side_by_sideC
Raw plate next to the finished shot, for showreels and pitch demos.
| Name | Required | Description | Default |
|---|---|---|---|
| after | Yes | ||
| before | Yes | ||
| height | No | ||
| output | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, and it discloses almost nothing: not the output artifact type, format, whether the source files are modified, or any permission requirements. 'Raw plate next to the finished shot' describes intent only, leaving the agent unable to predict the side effects of the call.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no padding or repetition. Its brevity is efficient, though it flirts with under-specification rather than being truly well-sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a four-parameter tool with no output schema, no annotations, and 0% parameter coverage, the description is not complete enough. An agent cannot determine what artifact is produced, in what format, or at what size, which are the core facts needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% across four parameters, so the description must compensate. It partially maps semantics for two of them ('raw plate' clarifies before, 'finished shot' clarifies after), but says nothing about what output refers to or how height (default 1080) affects the result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The phrase 'Raw plate next to the finished shot' conveys the substance (a before/after comparison render), but it is a noun phrase with no verb and never states that it produces a file or image. It also does nothing to distinguish itself from nearby siblings like composite_preview or vertical. The purpose is inferable but not stated crisply.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'For showreels and pitch demos' gives an implied use context, which is more than nothing. However, it never says when to choose this over siblings such as composite_preview or vertical, and gives no prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verticalC
Fill-crop a render to 9:16 for TikTok, Reels and Shorts.
| Name | Required | Description | Default |
|---|---|---|---|
| input | Yes | ||
| width | No | ||
| height | No | ||
| output | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It says 'fill-crop' but never states whether the source is replaced or written to a new path, whether content outside the crop is lost, what codec/quality is used, or whether the input file is mutated. For a mutation-style render tool this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single well-formed sentence with the operation and target front-loaded and zero filler. It is appropriately sized, though its brevity is also the source of the specification gaps.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with no annotations, no output schema, and 0% schema description coverage, one sentence is not enough. Nothing describes return values, output file handling, or how width/height interact with the stated 9:16 target.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% across 4 parameters, so the description must compensate and does not. '9:16' loosely maps to the default width/height of 1080x1920, but the description never explains that width/height override the aspect ratio, nor what 'input' and 'output' accept (paths, formats).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Fill-crop a render') plus the target aspect ratio and destination platforms, which disambiguates the otherwise opaque name 'vertical'. It does not explicitly differentiate itself from siblings like 'conform' or 'side_by_side', but the operation is concrete enough to act on.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Naming TikTok/Reels/Shorts implies the use case, but there is no explicit when-to-use guidance, no mention of when NOT to use it (e.g. versus 'conform' or 'side_by_side' for other output shapes), and no prerequisites. The agent must infer the trigger condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
v0.1.0- First observed
chroma_key - First observed
colmap_solve - First observed
colmap_to_ue - First observed
composite_preview - First observed
conform - First observed
extract_frames - First observed
impact_frames - First observed
knowledge - First observed
latency_budget - First observed
matte_external - First observed
on_twos - First observed
probe - First observed
side_by_side - First observed
vertical
TDQS
Scored across 14 tools
Each tool targets a distinct pipeline stage (probe, conform, key, matte, solve, anime effects, delivery), and the descriptions clearly frame the boundaries. The only real overlap is chroma_key vs matte_external, but the 'fast first pass' vs 'AI matting tool you installed' framing makes the choice workable, as does composite_preview vs side_by_side vs vertical.
All names are readable snake_case, but grammatical structure is mixed: verb-only (probe, conform), verb_noun (extract_frames, impact_frames), noun_noun (chroma_key, latency_budget), noun_verb (colmap_solve), and prepositional/adjectival forms (on_twos, side_by_side, vertical). The vocabulary is descriptive but not a predictable pattern an agent can rely on.
14 tools is well within the sweet spot and each maps to a concrete pipeline stage from ingest through anime stylization to social delivery. Nothing feels padded or redundant enough to remove.
The surface covers ingest, camera solve, UE export, keying/matting, anime effects, social reframing, and booth latency planning. The main gap is a final composite/render step — composite_preview explicitly is 'not the final comp' — plus no audio or delivery/export tooling, though those can be handled in Unreal.
Maintenance
Related MCP Connectors
Build editable 3D scenes, direct characters and cameras, and export AI video references with MCP.
214 tools for AI agents over one MCP endpoint: image, video, audio, 3D, data, storage, mail, GPUs.
FFmpeg as a service for AI agents: typed video editing tools, async jobs, downloadable outputs.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceEnables LLM agents to drive a self-hosted video toolkit through MCP tools, including lipsync, face restoration, ffmpeg operations, and metadata retrieval.1Do What The F*ck You Want To Public
- AlicenseNot gradedqualityBmaintenanceEnables MCP clients like Claude Code, Claude Desktop, and VS Code to build and edit inside a running Adobe After Effects, covering comps, layers, keyframes, effects, text animators, markers, and renders. It supports batched tool calls with single-step undo, frame previews for self-verification, reusable style tokens, and four one-call studio rigs.MIT
- AlicenseBqualityCmaintenanceProvides Claude Code with local ffmpeg-based tools to probe, trim, concat, transform, overlay, mix audio, render JSON timelines, and run automated QA on video files—no API key required.9Apache 2.0
- AlicenseBqualityCmaintenanceEnables AI agents to programmatically edit videos via FFmpeg, providing tools for probing, trimming, concatenating, timeline composition, LUT generation/application, rendering, and edit verification over MCP.162MIT