Skip to main content
Glama
SimScene MCP Server

SimScene MCP Server

by ros-claw

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
SIMSCENE_HOMENoSimScene home directory containing saved scenes and run state; used by run to locate a scene ID.
USD_SEARCH_API_URLNoURL of the USD Search API. The public gateway is the default; set this environment variable or use --search-url to choose another.
USD_SEARCH_API_TOKENNoToken for USD Search credentials. Alternatively, supply USD_SEARCH_API_USERNAME and USD_SEARCH_API_PASSWORD.
SIMSCENE_ISAAC_PYTHONNoAbsolute path to the installed Isaac Sim python.sh. Required for experimental Isaac simulation jobs.
SIMSCENE_PHYSICS_DEVICENoPhysics device selection. CPU rigid-body physics is the default; set to gpu for GPU physics or default to use Isaac's defaults.cpu
USD_SEARCH_API_PASSWORDNoPassword for USD Search credentials. Used with USD_SEARCH_API_USERNAME when USD_SEARCH_API_TOKEN is not supplied.
USD_SEARCH_API_USERNAMENoUsername for USD Search credentials. Used with USD_SEARCH_API_PASSWORD when USD_SEARCH_API_TOKEN is not supplied.

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}
prompts
{
  "listChanged": false
}
resources
{
  "subscribe": false,
  "listChanged": false
}
experimental
{}

Tools

Functions exposed to the LLM to take actions

NameDescription
catalog_searchA

Search reviewed sources with bounded metadata only. Read known source IDs from simscene://catalog/sources or simscene catalog sources; copy those IDs into filters.source_ids for explicit selection. Detailed records are linked, not inlined.

catalog_discoverB

Discover bounded HF repository candidates, not enabled libraries. No models, conversion or simulation; review access/license/layout facts before registering a source.

scene_compile_recipeA

Compile a versioned recipe into ordinary scene_apply operations at the current base_revision. Supply role bindings with asset_path and pinned asset_binding. This does not edit or run the scene and gives no physical pass; apply the full batch atomically, then freeze and verify a new trial.

asset_generateB

Start a durable job for an explicitly configured independent Sand3D runtime and a shipped fixed template (guide_plate_v1 or open_receiver_v1). Accepts structured dimensions only, never external programs. Completion returns a generated AssetRef for ordinary asset_plan/asset_prepare; generation alone gives no physical pass.

asset_planA

Freeze a selected revision, role, toolchain and transfer budget without downloading models or starting physics. Unknown constraints block preparation.

asset_prepareB

Start a persistent acquisition/preparation job; poll job_status. Preparation alone does not establish runtime physics success. Reuse request_id to reconnect.

environment_probeA

Inspect configured runtime and GPU without starting Isaac Sim.

environment_configureC

Save non-secret local defaults; environment variables take precedence.

asset_searchC

Find candidate assets online without downloading 3D assets.

huggingface_search_datasetsB

Discover Hugging Face dataset repositories by name.

huggingface_list_assetsB

List USD files from one page of a dataset at a supplied revision.

huggingface_acquire_assetB

Download one selected dataset USD and resolvable dependencies at a pinned commit; poll job_status.

asset_thumbnailC

Return a preview by search hit thumbnail_image_key or asset URL.

asset_inspectB

Retrieve indexed metadata, dependency graph and download estimate.

asset_inspect_localB

Inspect a USD file on the simulation host. Physics behavior is not verified.

asset_acquireA

Start downloading a selected asset and dependencies. Poll job_status until completed; its result.root is the local USD path for scene_apply. Check dependency_status and unresolved_dependencies before use.

robot_solve_ikB

Find a URDF IK candidate in the base frame; does not check collisions.

robot_prepare_profileB

Prepare experimental controller files for one acquired USD on the simulation host. Models: kawasaki_rs007l_rg2, franka_panda, ur10_short_suction. Poll job_status; validate compatibility and physical behavior before use.

robot_suggest_configurationB

Suggest an experimental starting controller configuration for a generated profile and world base pose; validate actual reachability, collisions and behavior in Isaac.

robot_plan_pick_cycleC

Propose experimental robot-specific phases for an approximately 4 cm cube; use $slot and $item to expand with scene_plan_stream. Validate all paths and physical results.

scene_openC

Create or reopen a scene specification.

scene_getC

Get the current scene specification.

scene_applyB

Atomically edit a revision. For a prepared asset, use asset_path=result.root and asset_binding={"manifest_path":result.manifest_path,"manifest_sha256":result.manifest_sha256} with exactly these two binding keys from the preparation result. A rigid_prop_v1 asset requires dynamic=true. Set rotation with normalized orientation_wxyz=[w,x,y,z]. Supported kinds: box, sphere, cylinder, capsule, compound, joint, asset, camera, light, counter; make ground from a static box. A light can use light={"type":"dome"}, {"type":"sphere","radius_m":0.2}, or {"type":"rect","width_m":2,"height_m":1}; default is distant. Configure intensity, optional light color/exposure and transform. A counter uses counter={"source":"spawned"} or {"source":"verified_archives","class_label":"red"}; verified counts require case-checked archiving. A non-colliding static box can use indicator={"robot_id":"arm"} to show the observed controller phase. A pose_follower can opt into reactive obstacle planning with avoid_obstacles listing static collision-enabled primitive IDs and obstacle_margin_m; this is not collision safety certification. A USD asset requires the local simulation-host asset_path returned by acquisition, not its online URL. Example: {"op":"add","id":"box1","object":{"kind":"box","size":[0.2,0.2,0.2],"position":[0,0,0.1]}}. IDs belong on the operation; object uses kind. Update uses {"op":"update","id":"box1","changes":{...}}. Also supports remove, place_on, arrange_line, arrange_grid, align and look_at. compound uses 1..128 local primitive parts under one unscaled rigid body; part visible=false hides its rendered proxy while retaining collision. An acquired asset with asset_physics=visual_only and asset_lights=disabled can set rigid_attachment={body:compound_ID}; its position/orientation then define a local pose under that dynamic compound. It follows USD hierarchy, with no runtime pose commands. Measure the original mesh and disclose its collision proxy approximation. joint={type:revolute|prismatic|spherical|fixed|d6,body0:null|ID,body1:ID,axis:X|Y|Z,local_pos0:[...],local_pos1:[...],lower_limit:...,upper_limit:...}; angular limits are degrees, linear limits metres. D6 instead requires dof_limits with all6 transX/transY/transZ/rotX/rotY/rotZ values: free, locked, or [low,high] with low<high; no axis/lower_limit fields. Joint X is twist; local_rot0_wxyz/local_rot1_wxyz define frames. D6 swing bounds form a pyramid, not a cone; use measured relative frame quaternion rather than global Euler components. Primitive kind=joint remains passive, with no drive targets. A prepared articulated_prop_v1 asset can use mechanism={"commands":[{"joint_id":"main","trigger":{"type":"at_step","physics_step":0},"target_position":0.5,"position_unit":"rad","stiffness":400,"damping":12,"max_effort":22}]}; use manifest joint IDs and limits, rad for hinges or m for sliders. Commands declare a schedule; actual joint response is observed separately. Initial body poses define initial state. Nonuniform legacy primitives need a baked compound frame when jointed. set_observation with observation={sample_interval_steps:1,strict_start:true,record_contact_pairs:true} requests every actual physics step, a measured reset zero after SDK warmup, and actual per-step contact records; no body setters run during the task. Imported supports need support_surfaces={name:{min_xy:[x,y],max_xy:[x,y],z_m:z,exclude:[]}} in normalized local metres; place_on uses support_surface and offset_xy with optional avoid_ids/keepout_bounds/clearance_m. arrange_grid uses ids,support,support_surface,columns,gap_m; align uses ids,axis,edge,value_m. set_presentation accepts presentation.profile=technical_showcase_1080p and capture/output/budget/diagnostics; camera settings live under camera={resolution:[width,height],focal_length_mm:35,horizontal_aperture_mm:36,...}. Native 30fps records every two physics steps. For a static box surface use {"op":"bind_conveyor","id":"belt","speed_m_s":0.1,"direction":[1,0,0]}; imported assets cannot be conveyor surfaces directly. A camera can use camera_orbit={"center":[0,0,0.5],"radius_m":2,"height_m":1,"duration_s":8} for a circular view.

scene_measureA

Measure authored world render bounds and bottom anchors, including imported units/axes/variants and rigid visual attachments. Optional include_overlap_candidates returns AABB intersections among selected objects to review initial layout interference; empty compound space and deliberate contact can be false positives. It does not certify physical clearance. Imported outer bbox is not a support surface.

scene_plan_ropeB

Propose real colliding capsule links connected by passive spherical joints. points are a metric world polyline, including deliberate slack. Optional start/end attachments each use body and local_position_m. Apply returned ordinary operations after review. This is not a distance-only or animated rope; routing over a free pulley and actual tension transfer need physical trials and broken-end controls.

scene_plan_presentationB

Propose measured overview/detail cameras and a capture profile. regions use id,min,max, optional action/subject_ids. Review occlusion, lighting and existing camera budget; apply operations and preview before native recording.

run_presentation_eventsA

Read semantic phase/spawn events from the completed run's frozen spec. No event implies physical success; inspect run_verify separately.

video_exportB

Edit intact real run recordings. Segments require run_directory,camera,start_sim_time_s,end_sim_time_s; optional speed,title. Preserve native cadence, save source hashes and sim/output time mapping. Upscaling requires explicit opt-in.

video_inspectB

Probe/decode real video; when job_id/camera supplied, bind to intact source and verify native capture dimensions and physics frame clock. Representative PNG paths are returned for actual visual review.

scene_plan_streamA

Propose scheduled workpieces and per-class robot cycles without editing the scene. Each routes[label] uses robot_id, wait={axis,minimum,maximum,max_steps}, cycle=[phases], and optional slots. Use the cycle returned by robot_plan_pick_cycle; apply returned operations with base_revision.

scene_plan_stream_caseC

Propose a scene-derived physical evidence case using explicit per-class inner bin bounds and forbidden objects. This is not independent acceptance or path-safety certification.

scene_plan_collision_auditC

Propose discrete overlap monitoring and model-specific mechanical exceptions for supported robot paths or matching generated profiles. Include every robot in the stream case, review exclusions, apply operations, regenerate the case, add collision_policy and run verification. This does not certify swept-path or real-robot safety.

scene_plan_synchronizeC

Propose interleaved spawn times from observed pickup delays in an intact completed stream run. Review and apply the proposal, regenerate the case, and verify simultaneous cycles and safety in a new run; this is not a guarantee.

scene_plan_retimeB

Propose retiming selected scheduled objects, preserving their order and first spawn. Review and apply the operations, then rerun.

scene_plan_interleaveC

Propose a class-aware round-robin spawn schedule while preserving each class's item order. Review robot timing and other objects, apply operations, regenerate the case, and rerun physics.

scene_plan_swap_routesB

Propose swapping two equal-sized, single-class scheduled robot routes. Review bin labels, reachability and physical results.

scene_plan_translate_groupA

Propose moving objects by a world-space delta and optionally moving one robot's explicitly selected nearby absolute waypoints. Review the returned phase indices and rerun.

scene_validateC

Check supported object schemas. This is not physics validation.

scene_exportC

Export supported scene objects to a USD file on this server.

simulation_startB

Start or queue an experimental Isaac job, optionally pinning an inline case or server-side JSON/YAML case. Reusing the same request_id with identical parameters returns the original job; use a new ID for a distinct run. continue_on_controller_error keeps physics running after a controller failure. archive_verified_objects requires a pinned stream case.

job_statusA

Check a job; optionally wait machine-side and save a compact completion receipt. For wait_seconds>0, the caller must choose a new unused absolute receipt_path in their own output directory. A timeout preserves the same job_id; wait again with a new unused receipt path. Waiting grants no physical acceptance. Configure a client timeout longer than wait_seconds.

run_observeB

Read the latest state while running or after failure, or a sample from a completed run.

run_contactsB

Page recorded contact_counts, robot_contacts, or optional overlap_counts from a completed run. Overlap counts include expected contacts and do not measure penetration.

run_verify_lockB

Recheck saved run inputs and external dependencies against recorded hashes.

run_verifyA

Check completed physical evidence against an independent pinned local case. Types: contact, suction, moving, repeated, domino, parallel, stream, mechanism, passive_network. Passive schemas infer their type. Mechanism detached_drawer and network no_trigger/broken_rope/locked_door modes require the matching reference_job_id and isolated declared changes. A single positive network run does not accept the whole stage: use run_verify_passive_suite with its frozen required controls. This verifier only reads measurements and never releases or moves a mechanism.

run_verify_passive_suiteA

Read-only acceptance of a frozen passive-network positive plus its required isolated controls. control_job_ids maps no_trigger, broken_rope and/or locked_door to completed distinct job IDs. Case bytes, source changes, sampled physical clocks, runtime settings and evidence locks are checked; missing controls cannot accept a P2/P3 stage. This checks physical acceptance, not final video or artistic quality.

run_verify_evidenceB

Recheck saved output hashes; this is not a signature or task-success check.

run_video_chunkC

Read a bounded, base64-encoded chunk of a completed camera recording.

scene_observeC

Return a live/last preview (when enabled) or a completed camera image.

job_cancelB

Request cancellation; poll status for confirmation.

simulation_pauseB

Request a cooperative timeline pause; poll job_status for acknowledgment.

simulation_resumeA

Request resumption of a paused simulation; poll job_status.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription
catalog_sources_resourceReviewed source states and provenance; metadata availability is not integration acceptance.

TDQS

C2.9/5.0

Scored across 52 tools

Disambiguation3/5

Most tools have distinct roles, but several clusters overlap: scene_plan_synchronize/interleave/retime/stream, asset_search/catalog_search/catalog_discover/huggingface_search_datasets, and run_verify/run_verify_lock/run_verify_evidence. Detailed descriptions help, but an agent must read carefully to avoid misselection.

Naming Consistency4/5

Almost all names are snake_case with domain prefixes such as scene_, asset_, run_, job_, robot_, and video_, mostly following context_verb or context_verb_noun patterns. Minor deviations include huggingface_* tools breaking the asset_ prefix and job_status/simulation_start using noun-first ordering.

Tool Count2/5

52 tools is far above the recommended 3-15 range and near the 50+ extreme; the domain is broad, but the surface is very heavy and increases selection load. Many specialized tools earn their place, yet the total count is still excessive for reliable agent use.

Completeness4/5

Core workflows—scene editing, asset acquisition, robot planning, simulation control, run verification, and video export—are well covered, with multiple specialized verification tools. Gaps include no explicit scene/run/job listing or deletion tools and no catalog registration tool despite discovery mentioning registration.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues