ca-scene-mcp
The ca-scene-mcp server provides a comprehensive API for creating, editing, simulating, rendering, and optimizing physics-based computer animation scenes, supporting rigid bodies, cloth, fluids, constraints, fields, and actions.
Scene Management
Create, list, retrieve, and modify animation scenes
Export scenes to JSON or other formats, and as standalone Python programs
Retrieve scene schemas and server capabilities
Scene Content Definition
Objects: Rigid bodies (boxes, spheres, capsules, compound shapes), cloth, fluids (smoke/liquid), and containers
Constraints: Fixed-point and distance constraints
Fields: Uniform and radial force or acceleration fields
Actions: Transform (keyframe), impulse, continuous force, and fluid emission actions
Update or remove any existing object, constraint, field, or action; apply transactional merge patches to entire scenes
Set a fixed look-at camera (position, target, field of view)
Simulation & Rendering
Validate scenes (structured errors, suggestions, resource estimates)
Run low-cost preview simulations returning keyframes and metrics
Submit full simulations (Newton physics executor) and rendering jobs asynchronously
Poll job status or cancel queued jobs
Numerical Optimization
Create, validate, start, and manage optimization plans and trials
List, retrieve, compare, and apply trial results to find optimal physical parameters for a scene
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@ca-scene-mcp Create a scene with a cube falling onto a plane"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Computer Animation - Exercise
Overview
This repository contains the code for the computer animation exercise. The code uses Newton@2a6df66 as a backend for the physics simulation.
Related MCP server: Blender MCP
Agent scene framework
The repository also contains a prompt-driven framework for generating animations:
ca_framework.scenedefines a backend-neutral, JSON-serializable scene model for rigid bodies, cloth, fluids, constraints, and external fields.ca_framework.mcpprovides protocol-independent scene editing tools.mcp_serverexposes those tools through the official Python MCP SDK as a standalone stdio or Streamable HTTP server.knowledge/skillscontains focused simulation knowledge that an agent can load when translating a prompt into a scene.
The runtime and agent-facing files intentionally contain no reference scenes or
reference answers. Black-box task prompts and benchmark commands live under
evaluation/; do not expose that directory to an agent during a prompt-to-video
experiment.
Start the stdio MCP server from the repository root:
uv run --project mcp_server ca-scene-mcpScene files are stored in .ca-scenes by default. Set CA_SCENE_WORKSPACE to use another directory. SceneExecutorLocal compiles rigid bodies and cloth to Newton XPBD/VBD, advances smoke and APIC/FLIP fluid routes, caches every simulated frame, and renders an output bundle. Final jobs use run_scene(scene_name, output_dir) and write animation.mp4, scene.json, program.py, metrics.json, diagnostics.jsonl, and cache/.
Development tests remain under tests/. Evaluation assets and instructions are
documented separately in evaluation/README.md.
Numerical optimization
Scene optimization is a separate, reproducible workflow: an
OptimizationPlan defines bounded physical parameters, an optional TaskSpec
defines task losses, and Optuna proposes Random, TPE, or CMA-ES
trials. Every trial passes through static validation, a short coarse rejection
run, a full-resolution metric run, and optional finalist rendering. Optimization
history is never stored in Scene.metadata.
The MCP server exposes the complete lifecycle through
create_optimization_plan, validate_optimization_plan,
start_optimization, get_optimization_job, list_optimization_trials,
get_optimization_trial, compare_optimization_trials, and
apply_optimization_trial. A stored plan has three independent inputs:
.ca-scenes/.optimizations/plans/<name>/
├── scene.json
├── optimization_plan.json
└── task_spec.json # optionalTrial directories contain the applied scene and parameters, constraint-first
metrics, diagnostics, structured telemetry, and the final report. The report
selects the best feasible, fastest acceptable, and most robust candidates and
writes best_so_far.svg without requiring a plotting dependency.
Installation
Install Git and uv on your system. On Windows, uv can be simply installed by running the following command in PowerShell:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"Clone this repository:
git clone http://dalab.se.sjtu.edu.cn/gitlab/courses/ca-framework-2026.gitInstall dependencies:
uv sync --extra examplesUse any IDE (PyCharm is recommended) of your choice to open the project and run the script in
examples/ca_exercises/directory. PyCharm will automatically recognize the virtual environments, if not, you can activate in the terminal using the command below:./.venv/Scripts/activateThen run the python script in the terminal:
python ./newton/examples/ca_exercises/exercise2_xxx_xxx.pyIf everything's alright, you will see a window like below:
Controls:WASD: move camera
QE: move camera up/down
Left click: lock around
Handin
You only need to modify the TODO sections in the code. You can run the code to test your implementation, but please do not modify the code structure or add any new files.
After you finish one exercise, You only need to submit the modified .py files in newton/_src/solvers/ca_exercises/. Please compress these files into a zip file and submit it to Canvas. The file name should be in the format of ca_exercise<n>_<student_id>.zip.
Notes: each exercise may have its own additional requirements, please pay attention.
Available Tools
25 toolsadd_actionB
Add a transform, impulse, force, or fluid emission action.
Inputs: spec (required), scene_name (required). Returns: object
add_constraintB
Add a constraint; spec.kind is fixed-point or distance.
Inputs: spec (required), scene_name (required). Returns: object
add_fieldB
Add a force or acceleration field; spec.kind is uniform or radial.
Inputs: spec (required), scene_name (required). Returns: object
add_objectC
Add a rigid body, cloth, smoke/liquid fluid, or compound container.
Inputs: spec (required), scene_name (required). Returns: object
apply_scene_patchB
Transactionally apply a recursive merge patch to a scene.
Inputs: patch (required), scene_name (required). Returns: object
cancel_jobC
Cancel a queued scene job.
Inputs: job_id (required). Returns: object
create_sceneC
Create an empty animation scene.
Inputs: name (required), overwrite. Returns: object
export_programC
Export a standalone Python program and matching scene JSON.
Inputs: scene_name (required), scene_output (required), program_output (required). Returns: object
export_sceneC
Export a scene; the initial executor supports JSON.
Inputs: output (required), scene_name (required), format. Returns: object
get_capabilitiesC
Describe supported objects, actions, solvers, and outputs.
Inputs: no named parameters. Returns: object
get_jobC
Poll an asynchronous scene job.
Inputs: job_id (required). Returns: object
get_sceneB
Read a complete animation scene.
Inputs: name (required). Returns: object
get_scene_schemaA
Return the discriminated JSON schema for scene IR version 2.
Inputs: no named parameters. Returns: object
list_scenesA
List all animation scenes in the workspace.
Inputs: no named parameters. Returns: result
preview_sceneB
Run a short low-cost preview and return keyframes and metrics.
Inputs: scene_name (required), frames. Returns: object
remove_itemC
Remove one scene item when it has no remaining references.
Inputs: item_id (required), collection (required), scene_name (required). Returns: result
render_sceneC
Validate and submit a scene to the configured Newton renderer.
Inputs: output (required), scene_name (required). Returns: object
run_sceneC
Submit a persistent simulation and rendering bundle asynchronously.
Inputs: output_dir (required), scene_name (required). Returns: object
set_cameraC
Set a fixed look-at camera; position and target use metres.
Inputs: camera (required), scene_name (required). Returns: object
simulate_sceneC
Validate and submit a scene to the configured Newton executor.
Inputs: scene_name (required), frames. Returns: object
update_actionB
Update one action; patch.kind must match its existing action kind.
Inputs: patch (required), item_id (required), scene_name (required). Returns: object
update_constraintB
Update one constraint; patch.kind must match its existing constraint kind.
Inputs: patch (required), item_id (required), scene_name (required). Returns: object
update_fieldC
Update one field; patch.kind must match its existing field kind.
Inputs: patch (required), item_id (required), scene_name (required). Returns: object
update_objectC
Update one object; patch.kind must match its existing object kind.
Inputs: patch (required), item_id (required), scene_name (required). Returns: object
validate_sceneC
Return structured errors, suggestions, and resource estimates.
Inputs: scene_name (required). Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 25 tools
Most tools have distinct purposes, but simulate_scene and run_scene could be confused (both involve submission to Newton executor/renderer). Other tools are clearly separated.
All tools follow a consistent verb_noun pattern in snake_case (e.g., add_action, update_object, validate_scene). No mixing of styles.
25 tools is high but still reasonable given the broad domain covering objects, actions, constraints, fields, simulation, rendering, export, and job management. Slightly above ideal but not chaotic.
Covers most lifecycle operations: create, read, update, delete for items, plus actions/constraints/fields. Notable missing tool to delete an entire scene, though apply_scene_patch may cover it indirectly.
Maintenance
Related MCP Connectors
Generate 3D models, worlds, PBR materials, and coordinated asset packs from AI agents.
One sentence to a finished animation: your AI writes motion graphics as code and checks real frames.
Turn prompts into production-ready game assets: animated characters, textures, terrain, VFX, HUD.
AI visual generation agent: multi-pipeline rendering, prompt crafting, and image composition.
Related MCP Servers
- AlicenseAqualityDmaintenanceOrchestrates 3D scene creation, camera paths, and physics-driven animations, bridging physics simulations with video rendering by defining scene graphs, binding objects to physics bodies, and exporting to React Three Fiber or Remotion.91MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to create, inspect, and animate 3D scenes in Blender through natural language and structured tool calls.6MIT
- AlicenseBqualityCmaintenanceEnables natural language creation and refinement of Blender scenes through structured MCP tools, with persistent object identity, visual validation, and reversible edits.23MIT
- AlicenseAqualityCmaintenanceAn agentic system for sim-ready interactive scene generation in Isaac Sim61Apache 2.0