Converts COLMAP camera solves into Unreal Engine camera parameters (location, pitch/yaw/roll, FOV, focal length) and provides a UE5 Python script to import a solved camera as a keyed CineCamera, enabling virtual production compositing in Unreal.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@cosplay-vp Probe take3.mp4, key it, and solve the camera move for Unreal"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Open-source tools, AI agent skills and a knowledge base for CG cosplay virtual production.
Film a real cosplayer on a phone. Cut them out, recover the camera move, put them inside an Unreal Engine world, and finish with anime-style cut-ins and impact frames, for recorded shots or a live event booth.
Plan ─► Shoot ─► Prep ─► Matte ─► Track ─► Unreal comp ─► Anime FX ─► DeliverDocs site: https://astralrc.github.io/cosplay-vp/ · Start with: Prompt library · Techniques A–Z · Showcase · How we built this
What's in the box:
Part | What it is |
Knowledge base: shooting, matting, tracking, Unreal comp, anime FX, live booth architectures (Aximmetry vs Pixel Streaming), rights and consent, open questions | |
Python library and MCP server with 14 tools built on ffmpeg and COLMAP | |
6 Agent Skills (SKILL.md) that teach an AI agent the pipeline step by step | |
Unreal Engine 5 Python script that imports a solved camera as a keyed CineCamera | |
Round-trip tests for the COLMAP → Unreal maths and ffmpeg tools on synthetic footage |
Tool | Does |
| Resolution, fps, rotation, alpha, with warnings for phone quirks |
| Constant frame rate, rotation baked in |
| Numbered frames for COLMAP or matting |
| Green/blue key + despill to ProRes 4444, VP9 alpha or PNG |
| Runs an AI matting model you installed (RVM, MatAnyone, ...) |
| Alpha foreground over a background to check edges |
| COLMAP sparse reconstruction, exported as TXT |
| COLMAP cameras → Unreal location (cm), pitch/yaw/roll, FOV, focal length |
| Anime "on twos" timing |
| Invert or white flash frames on hits |
| Raw vs final demo |
| 9:16 crop for TikTok, Reels and Shorts |
| Plan a live booth's latency against a target |
| Read any page of the knowledge base |
Related MCP server: video-editor-mcp
Requirements: Python 3.10+, ffmpeg. Optional: COLMAP, Unreal Engine 5, Blender, OBS.
git clone https://github.com/AstralRC/cosplay-vp.git
cd cosplay-vp
pip install -e ".[dev]"
pytest # 14 tests, including a live MCP stdio sessionClaude Code
claude mcp add cosplay-vp -- cosplay-vp-mcpClaude Desktop (claude_desktop_config.json), or any MCP client:
{
"mcpServers": {
"cosplay-vp": { "command": "cosplay-vp-mcp" }
}
}If the command isn't on your PATH, use "command": "python", "args": ["-m", "cosplay_vp.server"].
Claude Code: copy the folders in skills/ to ~/.claude/skills/ (all projects) or .claude/skills/ (one project).
Claude apps: zip a skill folder and upload it under Settings → Capabilities → Skills.
Then ask, for example:
Plan a 15-second ultimate-move cut-in for an original character, then prep
take3.mp4, key it, and track the camera into Unreal.
cosplay-vp-colmap2ue colmap_ws/sparse_txt camera.csv --scale 100and in Unreal's Python console:
py "path/to/unreal/import_camera_csv.py" "path/to/camera.csv" 24Area | Status |
ffmpeg tools, COLMAP → UE conversion | Working, tested in CI |
| Wraps the COLMAP CLI; needs COLMAP installed |
Unreal import script | Written for UE 5.x, not yet tested across versions; reports welcome |
Live booth architectures B and C (cloud) |
Local booth prototype: Unreal + OBS key, measured latency.
Cloud experiment: Pixel Streaming with a camera uplink, latency against the local build.
Matting benchmark: hair and lace edges, speed per model.
Unreal MCP bridge: place media plates and queue Movie Render Queue jobs from an agent.
Rotation-only (nodal) solve for tripod pans.
Only film people who agreed, with guardian consent for anyone under 18.
Paid work uses characters the client owns or licensed, or original characters.
No ripped game assets, logos or music in this repo.
Label ads, and label AI-generated content where platforms require it.
Details: docs/08-rights-consent-disclosure.md.
Anyone can contribute: fork, change, pull request. Benchmarks with methods, fixes for new Unreal versions, matting comparisons, techniques, prompts, showcase entries and translations are all welcome. Issue templates are ready for each. Regular contributors become triage collaborators: see GOVERNANCE.md and CONTRIBUTING.md.
Code: MIT. Docs and skills: CC BY 4.0. External tools keep their own licences (doc 10).
Started by Idea (Phakpaphon Jaiwanglok) of AstralRC / CELESTI.
Green/blue screen key to a clip with alpha (.mov ProRes 4444, .webm VP9, or %05d.png). A fast first pass.
Inputs: input (required), output (required), blend, color, +2 more.
Solve the camera from a frame folder with COLMAP (must be installed) and export the model as TXT.
Inputs: image_dir (required), workspace (required), quality, single_camera.
Convert COLMAP cameras.txt + images.txt into Unreal camera keys (.csv or .json): location in cm, pitch/yaw/roll, FOV. scale = UE centimetres per COLMAP unit. Import with unreal/import_camera_csv.py.
Inputs: output (required), sparse_txt_dir (required), scale.
Preview an alpha foreground over a background still or render. For checking edges, not the final comp.
Inputs: output (required), background (required), foreground (required), height.
Re-encode phone footage to a constant frame rate with rotation baked in. Do this before tracking or matting.
Inputs: input (required), output (required), fps.
Export numbered frames (00001.jpg ...) for COLMAP or matting. Use fps to thin long clips.
Inputs: input (required), out_dir (required), ext, fps, +2 more.
Add anime impact flashes at frame numbers. mode: 'invert' or 'white'.
Inputs: input (required), frames (required), output (required), mode, +1 more.
Read the project's knowledge base. Call with no topic to list topics.
Inputs: topic.
Sum estimated latency per stage of a live booth (camera, key, render, stream, display) against a target.
Inputs: stages_ms (required), target_ms.
Run an AI matting tool the user installed (e.g. RVM, MatAnyone) using {input}/{output} placeholders. Check that tool's licence first.
Inputs: input (required), output (required), command_template (required).
Give motion the anime 'on twos' feel by holding each frame hold times at the same frame rate.
Inputs: input (required), output (required), fps, hold.
Inspect a clip: resolution, fps, duration, codec, rotation, alpha, plus warnings (variable frame rate, rotation flags).
Inputs: path (required).
Raw plate next to the finished shot, for showreels and pitch demos.
Inputs: after (required), before (required), output (required), height.
Fill-crop a render to 9:16 for TikTok, Reels and Shorts.
Inputs: input (required), output (required), width, height.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 14 tools
Each tool targets a distinct pipeline stage (probe, conform, key, matte, solve, anime effects, delivery), and the descriptions clearly frame the boundaries. The only real overlap is chroma_key vs matte_external, but the 'fast first pass' vs 'AI matting tool you installed' framing makes the choice workable, as does composite_preview vs side_by_side vs vertical.
All names are readable snake_case, but grammatical structure is mixed: verb-only (probe, conform), verb_noun (extract_frames, impact_frames), noun_noun (chroma_key, latency_budget), noun_verb (colmap_solve), and prepositional/adjectival forms (on_twos, side_by_side, vertical). The vocabulary is descriptive but not a predictable pattern an agent can rely on.
14 tools is well within the sweet spot and each maps to a concrete pipeline stage from ingest through anime stylization to social delivery. Nothing feels padded or redundant enough to remove.
The surface covers ingest, camera solve, UE export, keying/matting, anime effects, social reframing, and booth latency planning. The main gap is a final composite/render step — composite_preview explicitly is 'not the final comp' — plus no audio or delivery/export tooling, though those can be handled in Unreal.
Build editable 3D scenes, direct characters and cameras, and export AI video references with MCP.
214 tools for AI agents over one MCP endpoint: image, video, audio, 3D, data, storage, mail, GPUs.
FFmpeg as a service for AI agents: typed video editing tools, async jobs, downloadable outputs.
MCP server for Clipkit — gives AI agents a video toolbox via the Clipkit schema.
