SimScene MCP Server
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| SIMSCENE_HOME | No | SimScene home directory containing saved scenes and run state; used by run to locate a scene ID. | |
| USD_SEARCH_API_URL | No | URL of the USD Search API. The public gateway is the default; set this environment variable or use --search-url to choose another. | |
| USD_SEARCH_API_TOKEN | No | Token for USD Search credentials. Alternatively, supply USD_SEARCH_API_USERNAME and USD_SEARCH_API_PASSWORD. | |
| SIMSCENE_ISAAC_PYTHON | No | Absolute path to the installed Isaac Sim python.sh. Required for experimental Isaac simulation jobs. | |
| SIMSCENE_PHYSICS_DEVICE | No | Physics device selection. CPU rigid-body physics is the default; set to gpu for GPU physics or default to use Isaac's defaults. | cpu |
| USD_SEARCH_API_PASSWORD | No | Password for USD Search credentials. Used with USD_SEARCH_API_USERNAME when USD_SEARCH_API_TOKEN is not supplied. | |
| USD_SEARCH_API_USERNAME | No | Username for USD Search credentials. Used with USD_SEARCH_API_PASSWORD when USD_SEARCH_API_TOKEN is not supplied. |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| catalog_searchA | Search reviewed sources with bounded metadata only. Read known source IDs from simscene://catalog/sources or simscene catalog sources; copy those IDs into filters.source_ids for explicit selection. Detailed records are linked, not inlined. |
| catalog_discoverB | Discover bounded HF repository candidates, not enabled libraries. No models, conversion or simulation; review access/license/layout facts before registering a source. |
| scene_compile_recipeA | Compile a versioned recipe into ordinary scene_apply operations at the current base_revision. Supply role bindings with asset_path and pinned asset_binding. This does not edit or run the scene and gives no physical pass; apply the full batch atomically, then freeze and verify a new trial. |
| asset_generateB | Start a durable job for an explicitly configured independent Sand3D runtime and a shipped fixed template (guide_plate_v1 or open_receiver_v1). Accepts structured dimensions only, never external programs. Completion returns a generated AssetRef for ordinary asset_plan/asset_prepare; generation alone gives no physical pass. |
| asset_planA | Freeze a selected revision, role, toolchain and transfer budget without downloading models or starting physics. Unknown constraints block preparation. |
| asset_prepareB | Start a persistent acquisition/preparation job; poll job_status. Preparation alone does not establish runtime physics success. Reuse request_id to reconnect. |
| environment_probeA | Inspect configured runtime and GPU without starting Isaac Sim. |
| environment_configureC | Save non-secret local defaults; environment variables take precedence. |
| asset_searchC | Find candidate assets online without downloading 3D assets. |
| huggingface_search_datasetsB | Discover Hugging Face dataset repositories by name. |
| huggingface_list_assetsB | List USD files from one page of a dataset at a supplied revision. |
| huggingface_acquire_assetB | Download one selected dataset USD and resolvable dependencies at a pinned commit; poll job_status. |
| asset_thumbnailC | Return a preview by search hit thumbnail_image_key or asset URL. |
| asset_inspectB | Retrieve indexed metadata, dependency graph and download estimate. |
| asset_inspect_localB | Inspect a USD file on the simulation host. Physics behavior is not verified. |
| asset_acquireA | Start downloading a selected asset and dependencies. Poll job_status until completed; its result.root is the local USD path for scene_apply. Check dependency_status and unresolved_dependencies before use. |
| robot_solve_ikB | Find a URDF IK candidate in the base frame; does not check collisions. |
| robot_prepare_profileB | Prepare experimental controller files for one acquired USD on the simulation host. Models: kawasaki_rs007l_rg2, franka_panda, ur10_short_suction. Poll job_status; validate compatibility and physical behavior before use. |
| robot_suggest_configurationB | Suggest an experimental starting controller configuration for a generated profile and world base pose; validate actual reachability, collisions and behavior in Isaac. |
| robot_plan_pick_cycleC | Propose experimental robot-specific phases for an approximately 4 cm cube; use $slot and $item to expand with scene_plan_stream. Validate all paths and physical results. |
| scene_openC | Create or reopen a scene specification. |
| scene_getC | Get the current scene specification. |
| scene_applyB | Atomically edit a revision. For a prepared asset, use asset_path=result.root and asset_binding={"manifest_path":result.manifest_path,"manifest_sha256":result.manifest_sha256} with exactly these two binding keys from the preparation result. A rigid_prop_v1 asset requires dynamic=true. Set rotation with normalized orientation_wxyz=[w,x,y,z]. Supported kinds: box, sphere, cylinder, capsule, compound, joint, asset, camera, light, counter; make ground from a static box. A light can use light={"type":"dome"}, {"type":"sphere","radius_m":0.2}, or {"type":"rect","width_m":2,"height_m":1}; default is distant. Configure intensity, optional light color/exposure and transform. A counter uses counter={"source":"spawned"} or {"source":"verified_archives","class_label":"red"}; verified counts require case-checked archiving. A non-colliding static box can use indicator={"robot_id":"arm"} to show the observed controller phase. A pose_follower can opt into reactive obstacle planning with avoid_obstacles listing static collision-enabled primitive IDs and obstacle_margin_m; this is not collision safety certification. A USD asset requires the local simulation-host asset_path returned by acquisition, not its online URL. Example: {"op":"add","id":"box1","object":{"kind":"box","size":[0.2,0.2,0.2],"position":[0,0,0.1]}}. IDs belong on the operation; object uses kind. Update uses {"op":"update","id":"box1","changes":{...}}. Also supports remove, place_on, arrange_line, arrange_grid, align and look_at. compound uses 1..128 local primitive parts under one unscaled rigid body; part visible=false hides its rendered proxy while retaining collision. An acquired asset with asset_physics=visual_only and asset_lights=disabled can set rigid_attachment={body:compound_ID}; its position/orientation then define a local pose under that dynamic compound. It follows USD hierarchy, with no runtime pose commands. Measure the original mesh and disclose its collision proxy approximation. joint={type:revolute|prismatic|spherical|fixed|d6,body0:null|ID,body1:ID,axis:X|Y|Z,local_pos0:[...],local_pos1:[...],lower_limit:...,upper_limit:...}; angular limits are degrees, linear limits metres. D6 instead requires dof_limits with all6 transX/transY/transZ/rotX/rotY/rotZ values: free, locked, or [low,high] with low<high; no axis/lower_limit fields. Joint X is twist; local_rot0_wxyz/local_rot1_wxyz define frames. D6 swing bounds form a pyramid, not a cone; use measured relative frame quaternion rather than global Euler components. Primitive kind=joint remains passive, with no drive targets. A prepared articulated_prop_v1 asset can use mechanism={"commands":[{"joint_id":"main","trigger":{"type":"at_step","physics_step":0},"target_position":0.5,"position_unit":"rad","stiffness":400,"damping":12,"max_effort":22}]}; use manifest joint IDs and limits, rad for hinges or m for sliders. Commands declare a schedule; actual joint response is observed separately. Initial body poses define initial state. Nonuniform legacy primitives need a baked compound frame when jointed. set_observation with observation={sample_interval_steps:1,strict_start:true,record_contact_pairs:true} requests every actual physics step, a measured reset zero after SDK warmup, and actual per-step contact records; no body setters run during the task. Imported supports need support_surfaces={name:{min_xy:[x,y],max_xy:[x,y],z_m:z,exclude:[]}} in normalized local metres; place_on uses support_surface and offset_xy with optional avoid_ids/keepout_bounds/clearance_m. arrange_grid uses ids,support,support_surface,columns,gap_m; align uses ids,axis,edge,value_m. set_presentation accepts presentation.profile=technical_showcase_1080p and capture/output/budget/diagnostics; camera settings live under camera={resolution:[width,height],focal_length_mm:35,horizontal_aperture_mm:36,...}. Native 30fps records every two physics steps. For a static box surface use {"op":"bind_conveyor","id":"belt","speed_m_s":0.1,"direction":[1,0,0]}; imported assets cannot be conveyor surfaces directly. A camera can use camera_orbit={"center":[0,0,0.5],"radius_m":2,"height_m":1,"duration_s":8} for a circular view. |
| scene_measureA | Measure authored world render bounds and bottom anchors, including imported units/axes/variants and rigid visual attachments. Optional include_overlap_candidates returns AABB intersections among selected objects to review initial layout interference; empty compound space and deliberate contact can be false positives. It does not certify physical clearance. Imported outer bbox is not a support surface. |
| scene_plan_ropeB | Propose real colliding capsule links connected by passive spherical joints. points are a metric world polyline, including deliberate slack. Optional start/end attachments each use body and local_position_m. Apply returned ordinary operations after review. This is not a distance-only or animated rope; routing over a free pulley and actual tension transfer need physical trials and broken-end controls. |
| scene_plan_presentationB | Propose measured overview/detail cameras and a capture profile. regions use id,min,max, optional action/subject_ids. Review occlusion, lighting and existing camera budget; apply operations and preview before native recording. |
| run_presentation_eventsA | Read semantic phase/spawn events from the completed run's frozen spec. No event implies physical success; inspect run_verify separately. |
| video_exportB | Edit intact real run recordings. Segments require run_directory,camera,start_sim_time_s,end_sim_time_s; optional speed,title. Preserve native cadence, save source hashes and sim/output time mapping. Upscaling requires explicit opt-in. |
| video_inspectB | Probe/decode real video; when job_id/camera supplied, bind to intact source and verify native capture dimensions and physics frame clock. Representative PNG paths are returned for actual visual review. |
| scene_plan_streamA | Propose scheduled workpieces and per-class robot cycles without editing the scene. Each routes[label] uses robot_id, wait={axis,minimum,maximum,max_steps}, cycle=[phases], and optional slots. Use the cycle returned by robot_plan_pick_cycle; apply returned operations with base_revision. |
| scene_plan_stream_caseC | Propose a scene-derived physical evidence case using explicit per-class inner bin bounds and forbidden objects. This is not independent acceptance or path-safety certification. |
| scene_plan_collision_auditC | Propose discrete overlap monitoring and model-specific mechanical exceptions for supported robot paths or matching generated profiles. Include every robot in the stream case, review exclusions, apply operations, regenerate the case, add collision_policy and run verification. This does not certify swept-path or real-robot safety. |
| scene_plan_synchronizeC | Propose interleaved spawn times from observed pickup delays in an intact completed stream run. Review and apply the proposal, regenerate the case, and verify simultaneous cycles and safety in a new run; this is not a guarantee. |
| scene_plan_retimeB | Propose retiming selected scheduled objects, preserving their order and first spawn. Review and apply the operations, then rerun. |
| scene_plan_interleaveC | Propose a class-aware round-robin spawn schedule while preserving each class's item order. Review robot timing and other objects, apply operations, regenerate the case, and rerun physics. |
| scene_plan_swap_routesB | Propose swapping two equal-sized, single-class scheduled robot routes. Review bin labels, reachability and physical results. |
| scene_plan_translate_groupA | Propose moving objects by a world-space delta and optionally moving one robot's explicitly selected nearby absolute waypoints. Review the returned phase indices and rerun. |
| scene_validateC | Check supported object schemas. This is not physics validation. |
| scene_exportC | Export supported scene objects to a USD file on this server. |
| simulation_startB | Start or queue an experimental Isaac job, optionally pinning an inline case or server-side JSON/YAML case. Reusing the same request_id with identical parameters returns the original job; use a new ID for a distinct run. continue_on_controller_error keeps physics running after a controller failure. archive_verified_objects requires a pinned stream case. |
| job_statusA | Check a job; optionally wait machine-side and save a compact completion receipt. For wait_seconds>0, the caller must choose a new unused absolute receipt_path in their own output directory. A timeout preserves the same job_id; wait again with a new unused receipt path. Waiting grants no physical acceptance. Configure a client timeout longer than wait_seconds. |
| run_observeB | Read the latest state while running or after failure, or a sample from a completed run. |
| run_contactsB | Page recorded contact_counts, robot_contacts, or optional overlap_counts from a completed run. Overlap counts include expected contacts and do not measure penetration. |
| run_verify_lockB | Recheck saved run inputs and external dependencies against recorded hashes. |
| run_verifyA | Check completed physical evidence against an independent pinned local case. Types: contact, suction, moving, repeated, domino, parallel, stream, mechanism, passive_network. Passive schemas infer their type. Mechanism detached_drawer and network no_trigger/broken_rope/locked_door modes require the matching reference_job_id and isolated declared changes. A single positive network run does not accept the whole stage: use run_verify_passive_suite with its frozen required controls. This verifier only reads measurements and never releases or moves a mechanism. |
| run_verify_passive_suiteA | Read-only acceptance of a frozen passive-network positive plus its required isolated controls. control_job_ids maps no_trigger, broken_rope and/or locked_door to completed distinct job IDs. Case bytes, source changes, sampled physical clocks, runtime settings and evidence locks are checked; missing controls cannot accept a P2/P3 stage. This checks physical acceptance, not final video or artistic quality. |
| run_verify_evidenceB | Recheck saved output hashes; this is not a signature or task-success check. |
| run_video_chunkC | Read a bounded, base64-encoded chunk of a completed camera recording. |
| scene_observeC | Return a live/last preview (when enabled) or a completed camera image. |
| job_cancelB | Request cancellation; poll status for confirmation. |
| simulation_pauseB | Request a cooperative timeline pause; poll job_status for acknowledgment. |
| simulation_resumeA | Request resumption of a paused simulation; poll job_status. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| catalog_sources_resource | Reviewed source states and provenance; metadata availability is not integration acceptance. |
TDQS
Scored across 52 tools
Most tools have distinct roles, but several clusters overlap: scene_plan_synchronize/interleave/retime/stream, asset_search/catalog_search/catalog_discover/huggingface_search_datasets, and run_verify/run_verify_lock/run_verify_evidence. Detailed descriptions help, but an agent must read carefully to avoid misselection.
Almost all names are snake_case with domain prefixes such as scene_, asset_, run_, job_, robot_, and video_, mostly following context_verb or context_verb_noun patterns. Minor deviations include huggingface_* tools breaking the asset_ prefix and job_status/simulation_start using noun-first ordering.
52 tools is far above the recommended 3-15 range and near the 50+ extreme; the domain is broad, but the surface is very heavy and increases selection load. Many specialized tools earn their place, yet the total count is still excessive for reliable agent use.
Core workflows—scene editing, asset acquisition, robot planning, simulation control, run verification, and video export—are well covered, with multiple specialized verification tools. Gaps include no explicit scene/run/job listing or deletion tools and no catalog registration tool despite discovery mentioning registration.