Skip to main content
Glama

Computer Animation - Exercise

Overview

This repository contains the code for the computer animation exercise. The code uses Newton@2a6df66 as a backend for the physics simulation.

Related MCP server: Blender MCP

Agent scene framework

The repository also contains a prompt-driven framework for generating animations:

  • ca_framework.scene defines a backend-neutral, JSON-serializable scene model for rigid bodies, cloth, fluids, constraints, and external fields.

  • ca_framework.mcp provides protocol-independent scene editing tools.

  • mcp_server exposes those tools through the official Python MCP SDK as a standalone stdio or Streamable HTTP server.

  • knowledge/skills contains focused simulation knowledge that an agent can load when translating a prompt into a scene.

The runtime and agent-facing files intentionally contain no reference scenes or reference answers. Black-box task prompts and benchmark commands live under evaluation/; do not expose that directory to an agent during a prompt-to-video experiment.

Start the stdio MCP server from the repository root:

uv run --project mcp_server ca-scene-mcp

Scene files are stored in .ca-scenes by default. Set CA_SCENE_WORKSPACE to use another directory. SceneExecutorLocal compiles rigid bodies and cloth to Newton XPBD/VBD, advances smoke and APIC/FLIP fluid routes, caches every simulated frame, and renders an output bundle. Final jobs use run_scene(scene_name, output_dir) and write animation.mp4, scene.json, program.py, metrics.json, diagnostics.jsonl, and cache/.

Development tests remain under tests/. Evaluation assets and instructions are documented separately in evaluation/README.md.

Numerical optimization

Scene optimization is a separate, reproducible workflow: an OptimizationPlan defines bounded physical parameters, an optional TaskSpec defines task losses, and Optuna proposes Random, TPE, or CMA-ES trials. Every trial passes through static validation, a short coarse rejection run, a full-resolution metric run, and optional finalist rendering. Optimization history is never stored in Scene.metadata.

The MCP server exposes the complete lifecycle through create_optimization_plan, validate_optimization_plan, start_optimization, get_optimization_job, list_optimization_trials, get_optimization_trial, compare_optimization_trials, and apply_optimization_trial. A stored plan has three independent inputs:

.ca-scenes/.optimizations/plans/<name>/
├── scene.json
├── optimization_plan.json
└── task_spec.json          # optional

Trial directories contain the applied scene and parameters, constraint-first metrics, diagnostics, structured telemetry, and the final report. The report selects the best feasible, fastest acceptable, and most robust candidates and writes best_so_far.svg without requiring a plotting dependency.

Installation

  1. Install Git and uv on your system. On Windows, uv can be simply installed by running the following command in PowerShell:

    powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
  2. Clone this repository:

    git clone http://dalab.se.sjtu.edu.cn/gitlab/courses/ca-framework-2026.git
  3. Install dependencies:

    uv sync --extra examples
  4. Use any IDE (PyCharm is recommended) of your choice to open the project and run the script in examples/ca_exercises/ directory. PyCharm will automatically recognize the virtual environments, if not, you can activate in the terminal using the command below:

    ./.venv/Scripts/activate
  5. Then run the python script in the terminal:

    python ./newton/examples/ca_exercises/exercise2_xxx_xxx.py

    If everything's alright, you will see a window like below: empty Controls:

    • WASD: move camera

    • QE: move camera up/down

    • Left click: lock around

Handin

You only need to modify the TODO sections in the code. You can run the code to test your implementation, but please do not modify the code structure or add any new files.

After you finish one exercise, You only need to submit the modified .py files in newton/_src/solvers/ca_exercises/. Please compress these files into a zip file and submit it to Canvas. The file name should be in the format of ca_exercise<n>_<student_id>.zip.

Notes: each exercise may have its own additional requirements, please pay attention.

Available Tools

25 tools
add_actionB

Add a transform, impulse, force, or fluid emission action.

Inputs: spec (required), scene_name (required). Returns: object

add_constraintB

Add a constraint; spec.kind is fixed-point or distance.

Inputs: spec (required), scene_name (required). Returns: object

add_fieldB

Add a force or acceleration field; spec.kind is uniform or radial.

Inputs: spec (required), scene_name (required). Returns: object

add_objectC

Add a rigid body, cloth, smoke/liquid fluid, or compound container.

Inputs: spec (required), scene_name (required). Returns: object

apply_scene_patchB

Transactionally apply a recursive merge patch to a scene.

Inputs: patch (required), scene_name (required). Returns: object

cancel_jobC

Cancel a queued scene job.

Inputs: job_id (required). Returns: object

create_sceneC

Create an empty animation scene.

Inputs: name (required), overwrite. Returns: object

export_programC

Export a standalone Python program and matching scene JSON.

Inputs: scene_name (required), scene_output (required), program_output (required). Returns: object

export_sceneC

Export a scene; the initial executor supports JSON.

Inputs: output (required), scene_name (required), format. Returns: object

get_capabilitiesC

Describe supported objects, actions, solvers, and outputs.

Inputs: no named parameters. Returns: object

get_jobC

Poll an asynchronous scene job.

Inputs: job_id (required). Returns: object

get_sceneB

Read a complete animation scene.

Inputs: name (required). Returns: object

get_scene_schemaA

Return the discriminated JSON schema for scene IR version 2.

Inputs: no named parameters. Returns: object

list_scenesA

List all animation scenes in the workspace.

Inputs: no named parameters. Returns: result

preview_sceneB

Run a short low-cost preview and return keyframes and metrics.

Inputs: scene_name (required), frames. Returns: object

remove_itemC

Remove one scene item when it has no remaining references.

Inputs: item_id (required), collection (required), scene_name (required). Returns: result

render_sceneC

Validate and submit a scene to the configured Newton renderer.

Inputs: output (required), scene_name (required). Returns: object

run_sceneC

Submit a persistent simulation and rendering bundle asynchronously.

Inputs: output_dir (required), scene_name (required). Returns: object

set_cameraC

Set a fixed look-at camera; position and target use metres.

Inputs: camera (required), scene_name (required). Returns: object

simulate_sceneC

Validate and submit a scene to the configured Newton executor.

Inputs: scene_name (required), frames. Returns: object

update_actionB

Update one action; patch.kind must match its existing action kind.

Inputs: patch (required), item_id (required), scene_name (required). Returns: object

update_constraintB

Update one constraint; patch.kind must match its existing constraint kind.

Inputs: patch (required), item_id (required), scene_name (required). Returns: object

update_fieldC

Update one field; patch.kind must match its existing field kind.

Inputs: patch (required), item_id (required), scene_name (required). Returns: object

update_objectC

Update one object; patch.kind must match its existing object kind.

Inputs: patch (required), item_id (required), scene_name (required). Returns: object

validate_sceneC

Return structured errors, suggestions, and resource estimates.

Inputs: scene_name (required). Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.2/5.0

    Scored across 25 tools

    Disambiguation4/5

    Most tools have distinct purposes, but simulate_scene and run_scene could be confused (both involve submission to Newton executor/renderer). Other tools are clearly separated.

    Naming Consistency5/5

    All tools follow a consistent verb_noun pattern in snake_case (e.g., add_action, update_object, validate_scene). No mixing of styles.

    Tool Count4/5

    25 tools is high but still reasonable given the broad domain covering objects, actions, constraints, fields, simulation, rendering, export, and job management. Slightly above ideal but not chaotic.

    Completeness4/5

    Covers most lifecycle operations: create, read, update, delete for items, plus actions/constraints/fields. Notable missing tool to delete an entire scene, though apply_scene_patch may cover it indirectly.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers