IsaacSim-MCP
Provides tools to control NVIDIA Isaac Sim, including USD scene creation and manipulation, robot articulation and motion planning, camera and LiDAR sensors, PhysX simulation, Action Graph management, Replicator SDG, and simulation control.
Provides integration with ROS 2, including management of ROS 2 publishers and related capabilities within Isaac Sim.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@IsaacSim-MCP Create a factory scene with 10 workstations, 3 AGVs, robot arms, and conveyor belts"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
IsaacSim-MCP
IsaacSim-MCP 讓支援 MCP 的 AI client 透過具名、可驗證的 tools 控制 NVIDIA Isaac Sim。涵蓋 USD 場景、機器人、感測器、物理、Action Graph、ROS 2、Replicator SDG、動畫人物、NVIDIA 資產與模擬控制。
主要驗證環境是 Windows 與 Isaac Sim 6.0.1。本專案延伸自 whats2000/isaacsim-mcp-server,沿用 MIT License。
Table of Contents
Related MCP server: Isaac Sim MCP Server
核心功能
建立、查詢、變形、組合、儲存與驗證 USD Stage。
控制 articulation、joint drive、motion planning、gripper 與 mobile base。
在 Stage/robot replacement 後重建失效的 physics tensor 與 articulation binding,並保持 joint name/value 對應一致。
擷取 Camera RGB 與 typed RTX outputs;RGB 可選擇 MCP-native
ImageContent,供支援影像內容的遠端 client 顯示或交給 vision model。建立 PhysX scene、body、collider、joint、PBR material 與 physics material。
管理 Action Graph、ScriptNode、ROS 2 publisher、Replicator SDG job 與 human behavior lifecycle。
透過 managed artifact 傳輸大型輸出,支援 hash、bounded chunk、TTL 與 cleanup。
提供 command ID、idempotency、policy limit、job、cancel、read-back、rollback 與 redacted diagnostics。
MCP Tool Inventory 由 source decorators 自動產生;Tool profiles 說明 129-tool 相容模式與 98-tool 合併模式。目前 runtime 支援狀態以 get_capabilities 為準。
架構
LLM → Skill → MCP Server → TCP → Isaac Extension → Handler → Adapter → Isaac Simflowchart LR
A[LLM / MCP Client] --> B[Project Skill]
B --> C[Python MCP Server]
C -->|127.0.0.1:8766| D[Isaac Sim Extension]
D --> E[Handler]
E --> F[V5 / V6 Adapter]
F --> G[Isaac Sim / USD / PhysX]各層責任、request lifecycle、runtime routes 與權威來源見 ARCHITECTURE.md。
Installation
安裝 MCP Server:
git clone https://github.com/Tim0320/IsaacSim-MCP.git
cd IsaacSim-MCP
py -3.10 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -e .Windows 的完整環境需求與安裝方式見 Windows 安裝指南。
Configuration
stdio 是預設 transport。Codex/Claude Desktop 可以沿用現有設定,不需要新增 HTTP 環境變數:
{
"mcpServers": {
"isaac-sim-live": {
"command": "F:\\IsaacSim-MCP\\.venv\\Scripts\\python.exe",
"args": ["-m", "isaac_mcp.server"],
"env": {
"ISAAC_MCP_HOST": "127.0.0.1",
"ISAAC_MCP_PORT": "8766"
}
}
}
}Streamable HTTP 使用另一組設定:
Variable | Default | Purpose |
|
| 設為 |
|
| MCP HTTP listener address。 |
|
| MCP HTTP listener port。 |
|
|
|
| 未設定 | 額外允許送到 HTTP endpoint 的 exact Host header,以逗號分隔;loopback hosts 永遠保留。 |
|
| Supervisor 與 MCP Server 共用的 bounded crash/restart 狀態檔。 |
|
|
|
ISAAC_MCP_HOST=127.0.0.1 與 ISAAC_MCP_PORT=8766 仍屬於 Python MCP Server 和 Isaac Sim Extension 間的 runtime TCP socket。不要把它們改成 HTTP endpoint 設定。
Tool profiles
預設 legacy 不改任何既有 tool name 或 schema。ChatGPT 等需要較小 action space 的 client,可在啟動 MCP Server 前設定:
$env:ISAAC_MCP_TOOL_PROFILE = "consolidated"consolidated 用 action 或 publisher_type 合併同一資源的讀寫/控制操作,公開 98 tools。它只調整 MCP tool surface,仍呼叫原本的 Isaac Extension commands。修改 profile 後必須重新啟動 MCP Server,讓 client 重新取得 tool list。完整對照見 Tool profiles。
.env.example 只列出可用設定,server 不會自動載入它。請在啟動 python -m isaac_mcp.server 的同一個 process environment 設定 profile。啟動後可用 MCP tools/list 與 get_capabilities.data.mcp_server 核對實際公開 surface;consolidated 應回報 98 tools。
Running IsaacSim-MCP
建議用 supervisor 啟動 Isaac Sim 與 Extension:
$env:ISAACSIM_ROOT = "C:\isaacsim"
.\scripts\run_isaac_sim_supervised.ps1Supervisor 會沿用既有 launcher 的 Isaac Sim 6.0.1、Extension、port 與 Physics GPU guard。非零 exit code 視為異常退出,預設最多在 300 秒內重啟 3 次並使用 exponential backoff;exit code 0 視為正常關閉,不會自動重開。啟動前若 protocol health probe 已找到健康 runtime,或 8766 已被無回應程序占用,supervisor 都會拒絕再啟動一份 Isaac Sim。需要 one-shot 行為時仍可使用 .\scripts\run_isaac_sim.ps1。
Supervisor 與 MCP Server 是兩個獨立程序。Codex/Claude 仍由 client 用 stdio 啟動 isaac_mcp.server;Streamable HTTP 仍用下方命令啟動。Supervisor 狀態預設寫入 %LOCALAPPDATA%\IsaacSim-MCP\runtime-state.json,不包含 environment、command source 或 log 內容。
當 Isaac Sim crash、正在重啟或超過 restart budget 時,MCP tools 會回傳 ISAAC_RUNTIME_RECOVERING、ISAAC_RUNTIME_CRASHED 或 ISAAC_RUNTIME_UNAVAILABLE。呼叫 get_runtime_status 可在 8766 關閉時讀到 exit code、時間、attempt、restart count、health 狀態與建議動作。Connection loss 後不會自動 replay write;runtime 恢復後必須先 read-back,再決定是否用原 idempotency key 重送。
完整 process ownership、state schema、restart budget 與 Agent recovery contract 見 Runtime supervision 與 crash recovery。
本機 Codex/Claude Desktop 會依上一節設定,以 stdio 自動啟動 MCP Server。需要 Streamable HTTP 時,在另一個 PowerShell session 啟動:
$env:ISAAC_MCP_TRANSPORT = "streamable-http"
$env:ISAAC_MCP_HTTP_HOST = "127.0.0.1"
$env:ISAAC_MCP_HTTP_PORT = "8000"
$env:ISAAC_MCP_TOOL_PROFILE = "consolidated"
$env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1"
.\.venv\Scripts\python.exe -m isaac_mcp.serverMCP HTTP endpoint 是 http://127.0.0.1:8000/mcp。先呼叫 get_capabilities,再呼叫 get_scene_info,確認 runtime 與 Stage 後才執行 write。
Remote MCP Access
IsaacSim-MCP 預設只在本機使用。ChatGPT 等雲端 MCP client 無法直接連到 localhost、127.0.0.1 或 192.168.x.x 私有位址。它們需要能從 public internet 存取的 HTTPS endpoint,例如 Tailscale Funnel:
ChatGPT
↓ HTTPS
Public MCP Endpoint
↓
IsaacSim-MCP
↓ TCP 127.0.0.1:8766
Isaac SimMethod | Tailnet only | Public Internet | ChatGPT can access |
localhost | No | No | No |
Tailscale Serve | Yes | No | No |
Tailscale Funnel | Yes | Yes | Yes |
Tailscale Serve 只把服務提供給同一個 tailnet;Tailscale Funnel 會建立 public internet 可達的 HTTPS endpoint。ChatGPT MCP Connector 需要 Funnel 或其他 public HTTPS deployment。
Tailscale Funnel
從 Tailscale Download 安裝適合目前作業系統的版本,登入後啟用本機 Tailscale:
tailscale up啟動 Isaac Sim Extension,接著在 repository 根目錄啟動 Streamable HTTP server。將範例 hostname 換成
tailscale funnel顯示的完整 hostname:$env:ISAAC_MCP_TRANSPORT = "streamable-http" $env:ISAAC_MCP_HTTP_HOST = "127.0.0.1" $env:ISAAC_MCP_HTTP_PORT = "8000" $env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1,your-device.your-tailnet.ts.net" .\.venv\Scripts\python.exe -m isaac_mcp.serverMCP_ALLOWED_HOSTS使用 FastMCP 的 exact Host matching。專案會替每個完整 hostname 接受有 port 與無 port 形式,但不接受.ts.netsuffix 或*.ts.netwildcard。loopback hosts 永遠保留,所以加入外部 hostname 不會破壞本機 HTTP 存取。在另一個 terminal 建立背景 Funnel。
8000是 repository 的預設 HTTP MCP port:tailscale funnel --bg 8000預期輸出類似:
Available on the internet: https://your-device.your-tailnet.ts.net |-- / proxy http://127.0.0.1:8000公開 MCP URL 是
https://your-device.your-tailnet.ts.net/mcp。查看狀態或停止同一個 Funnel:
tailscale funnel status tailscale funnel --bg 8000 offtailscale funnel reset會清除本機全部 Funnel configuration。完整參數以 Tailscale Funnel CLI 為準。從外部網路驗證:
curl -i https://your-device.your-tailnet.ts.net/mcpStreamable HTTP 對一般 GET 的回應會依 request header 和 transport 狀態而異,
200、405或406都可能表示請求已到達 endpoint。結果不應再是421 Misdirected Request/Invalid Host header,也不應出現502 Bad Gateway或 timeout。
ChatGPT MCP Connector
Funnel 驗證成功後,依 OpenAI Developer mode 與 MCP apps 官方說明 在 ChatGPT Developer Mode 新增 MCP Connector:
Name:
IsaacSim
MCP Server URL:
https://your-device.your-tailnet.ts.net/mcp
Authentication:
No authentication需要讓 MCP client 直接取得 RGB 圖片時,呼叫:
capture_camera_output(
prim_path="/World/Camera",
output_type="rgb",
return_mode="image"
)return_mode="image" 會回傳 schema 1.0 文字 metadata 與 MCP-native ImageContent。Server 會驗證 PNG base64、byte size 與 SHA-256,且不會把大型 base64 重複放進文字 metadata。artifact 仍是預設模式,因此既有 Codex/Claude stdio 與下載流程不變。Client 是否在 UI 顯示圖片、把圖片交給 vision model,取決於該 client 對 MCP image content 的支援;要宣稱 ChatGPT 可見,必須完成實際 Connector acceptance,不能只依 server unit test 推論。
capture_image 與 capture_camera_output 的 MCP input schema 會把 return_mode 公開為 metadata | artifact | inline | image enum,Agent 不需要從說明文字猜測。Camera 新建或解析度變更後若第一個 RTX read 回 CAMERA_FRAME_NOT_READY,MCP Server 會等待一次 render tick 並自動重試一次;一次 tool call 即可完成 Render → Capture → ImageContent。若第二次仍未取得 frame,錯誤會原樣回傳,不會無限重試。
IsaacSim-MCP 目前沒有 HTTP authentication。MCP_ALLOWED_HOSTS 只防止不受信任的 Host header,不是身分驗證。公開 Funnel 會讓任何能連到該 URL 的人嘗試呼叫 MCP tools,可能控制本機 Isaac Sim。只在你接受這個風險時啟用,使用完立即停止 Funnel;正式共享環境應在 MCP server 前加入 authentication、authorization 與存取稽核。
支援版本
Component | 狀態 |
Isaac Sim 6.0.1 on Windows | 主要驗證 runtime |
PhysX | 透過 V6 adapter 與 guarded live verifier 支援 |
Newton | 只有 active backend matrix 回報 supported 且 verified 的功能才能使用,其餘 fail closed |
Isaac Sim 5.1.x | Legacy adapter;不在目前 6.0.1 release gate 範圍 |
Isaac Lab MCP | 明確延後,與目前 Isaac Sim MCP 分開處理 |
Package、extension、response、capability 與 backend-matrix version 各有獨立相容規則,見 Protocol versions 與 migration。
文件入口
從 docs/README.md 開始:
getting-started/:安裝與第一次連線。concepts/:protocol、transport、governance 與 job 共用模型。reference/:目前 API 與 capability 契約。development/:測試、scratch-stage 與 release 流程。research/:有日期的 1.x~6.x tasks 與 verification snapshots。
Agent 工作流程在 .agents/skills/omniverse-windows-workspace/SKILL.md。Tool、version 與 capability 的權威來源見 Authority and Generated Metadata。 Agent 的 retry、reconnect、read-back 與 fail-closed 行為見 Error Codes and Agent Recovery。 Robot articulation、physics tensor、joint mapping、DriveAPI 與 IK lifecycle 見 Robot Runtime Lifecycle。
Safety 與 verification
isaac-sim-live透過 TCP8766控制 Stage;documentation MCP 無法證明 live Stage 已改變。Write 必須通過 timeline、backend、extension、ownership 與 path prerequisites。Destructive verification 只允許 exact scratch Stage 與 MCP-owned namespace。
Registry presence 只證明 tool 可發現。Live pass 必須有 operation-specific read-back 與 cleanup evidence。
execute_script/reload_script只接受可信任程式碼,且受 policy 限制;優先使用 named tools。API keys 只供外部資產 provider 選用,不可寫入 source、MCP JSON、report 或 commit。
Release 前建立 verified backup 並執行 strict release gate。Commit 與 push 仍需使用者明確授權。
Troubleshooting
421 Misdirected Request / Invalid Host header
Tailscale Funnel 已成功轉發,但 Funnel hostname 尚未被 MCP Server 信任。把完整 hostname 加入 MCP_ALLOWED_HOSTS,例如:
$env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1,your-device.your-tailnet.ts.net"不要使用 .ts.net 或 *.ts.net;FastMCP 只接受設定中的 exact hostname。專案會自動接受該 hostname 有 port 與無 port 的形式。
502 Bad Gateway
Funnel 可用,但本機 MCP Server 沒有在 Funnel 指向的 port listening。確認 ISAAC_MCP_HTTP_PORT 與 tailscale funnel --bg 8000 的 port 相同,並保持 MCP Server process 執行中。
Connection timeout
執行 tailscale funnel status,確認 Funnel 仍啟用,再檢查 firewall 與 Tailscale connection 狀態。
Works locally but ChatGPT cannot connect
確認公開 URL 使用 HTTPS 且以 /mcp 結尾,並確認使用 Tailscale Funnel。Tailscale Serve 只有 tailnet 內部可達,ChatGPT 無法透過它連線。
Camera 成功但 ChatGPT 沒有顯示圖片
確認呼叫的是 RGB capture_camera_output(..., return_mode="image") 或 capture_image(..., return_mode="image")。artifact 只回 managed handle,inline 只把 base64 放進 JSON;兩者都不是 MCP-native image content。若 response 已含 ImageContent,但 UI 仍無圖片,代表目前 client 沒有渲染或轉交該 content block;保留 artifact handle 作為下載路徑,並把結果標記為 client-side partial support。
若 response 是 CAMERA_FRAME_NOT_READY,代表 Server 已完成一次 bounded warm-up retry,但第二次 RTX read 仍未準備。確認 Camera prim、timeline 與 render runtime 正常,再建立新的 capture attempt;不要在單次 request 內無限重播。
ChatGPT 顯示 legacy tools,但呼叫回 Unknown tool
這代表 connector 保存的 tool schema 與目前 server profile 不一致。先對實際 HTTP endpoint 執行 MCP tools/list,再呼叫 get_capabilities:
legacy應公開 129 tools,包含play_simulation、open_gripper、list_available_robots。consolidated應公開 98 tools,改用control_timeline、control_gripper、robot_library。
確認 server process 啟動前已設定 ISAAC_MCP_TOOL_PROFILE,重新啟動 server,接著讓 ChatGPT Connector 重新連線並重新取得 tool schema。若 connector migration 期間仍保存舊名稱,可暫時使用 full 同時公開新舊名稱;完成 schema 更新後切回 consolidated。不要永久使用 full,它會把 action space 增加到 151 tools。
COMMAND_FAILED / Not connected to Isaac
MCP Server 仍在執行,但 Isaac Sim Extension 沒有回應。改用 .\scripts\run_isaac_sim_supervised.ps1 啟動 runtime,並呼叫 get_runtime_status。Agent 應依 availability_code 處理:ISAAC_RUNTIME_RECOVERING 等待 bounded recovery;ISAAC_RUNTIME_CRASHED 檢查 last_crash 並修正 root cause;ISAAC_RUNTIME_UNAVAILABLE 啟動 supervisor。任何 write 在連線中斷後都必須先 read-back,不能盲目重送。
IK_FAILED / physics tensor entity is not valid
目前版本會在 Stage 或 robot identity 改變後丟棄舊 articulation wrapper,並重建 invalid Physics SimulationView。若仍看到此訊息,先確認 MCP Server 與 Isaac Extension 都來自同一個最新 checkout,再重新啟動 Isaac Sim 載入新 Extension。不要用重複 Play/Pause 當成修復,也不要重送未確認結果的 write。
PhysicsDriveAPI, a non-empty instance name must be provided
目前 USD fallback 會依 joint type 使用 angular 或 linear DriveAPI instance。若舊 runtime 仍回此錯誤,重新啟動 Isaac Sim 以載入更新後的 Extension,接著先用 get_joint_config 做 read-only 驗證。
Development
uv sync --dev
uv run pytest -q -m "not live and not windows_launcher and not unix_launcher" -k "not test_detect_version_returns_zero_on_failure"
uv run ruff check .
.\.venv\Scripts\python.exe .\scripts\generate_tool_inventory.py --checkLive tests 是 opt-in,並受 scratch-stage harness 保護。禁止對使用者 Stage 執行 legacy destructive integration suite。
License
本專案使用 MIT License。散布修改版時需保留授權、copyright notices 與 upstream attribution。
Available Tools
129 toolsapply_materialA
Bind a material to an object.
Args: material_path: Prim path of the material. target_prim_path: Prim path of the object to apply the material to. material_purpose: auto, physics, or visual. Auto selects from the material schema.
Inputs: material_path (required), target_prim_path (required), command_id, idempotency_key, +1 more. Returns: result
apply_stage_batchA
Preview or atomically apply up to 100 layer/arc/variant/semantic/attribute edits. Any failed operation restores the stage snapshot.
Inputs: operations (required), preview, command_id, idempotency_key, +1 more. Returns: result
cancel_jobC
Request cancellation and return a predictable terminal lifecycle state.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
cancel_motionB
Cancel a running motion job and hold its last commanded position.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
cancel_sdg_jobB
Preview or request cancellation; resources detach at the next safe point.
Inputs: job_id (required), preview, command_id, idempotency_key. Returns: result
capture_camera_outputB
Capture a typed RTX camera annotator output.
Args: prim_path: Prim path of the camera. output_type: depth, distance_to_image_plane, semantic_segmentation, instance_segmentation, instance_id_segmentation, normals, or motion_vectors. output_path: Optional explicit .npy path for artifact mode. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum raw bytes allowed in inline mode. Maximum is 4 MiB.
Inputs: prim_path, command_id, output_path, output_type, +3 more. Returns: result
capture_imageB
Capture an RGB image from a camera sensor.
Args: prim_path: Prim path of the camera. output_path: Optional explicit .png path for artifact mode. If omitted, uses the managed artifact root. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum PNG bytes allowed in inline mode. Maximum configurable value is 4 MiB.
Inputs: prim_path, command_id, output_path, return_mode, +2 more. Returns: result
cleanup_artifactsC
Delete expired managed artifacts and report reclaimed capacity.
Inputs: command_id, idempotency_key. Returns: result
clear_sceneA
Remove all prims from the scene.
Also empties any environment loaded by load_environment, so a later create_physics_scene(floor=True) does not stack a second ground under the first. The stage's defaultLight is always kept — a stage with no light renders black, which looks like a broken camera.
Args: keep_physics: If True, keep physics scene prims. keep_environment: If True, keep the loaded environment. Reloading one costs seconds, so pass this when clearing objects between attempts.
Inputs: command_id, keep_physics, idempotency_key, keep_environment. Returns: result
clone_objectA
Duplicate an existing object to a new prim path.
Args: source_path: Prim path of the object to clone. target_path: Prim path for the cloned object. position: [x, y, z] position for the clone. Keeps original position if not set.
Inputs: source_path (required), target_path (required), position, command_id, +1 more. Returns: result
close_gripperA
Close a gripper to the profile's declared closed width with immediate target read-back.
Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result
compute_ikA
Solve bounded Lula inverse kinematics without moving the robot.
Quaternion order is scalar-first [w, x, y, z]. The explicit warm-start and random seed make repeated requests reproducible. The result reports achieved end-effector errors and whether collision checking was done.
Inputs: prim_path (required), target_position (required), command_id, timeout_ms, +9 more. Returns: result
configure_physics_bodyB
Atomically configure rigid/static body, collider approximation and mass.
body_type is dynamic, kinematic, or static. approximation is one of none, convex_hull, convex_decomposition, mesh_simplification, bounding_cube, or bounding_sphere. Mass uses kg and density uses kg/m^3. The timeline must be stopped.
Inputs: prim_path (required), mass_kg, body_type, command_id, +4 more. Returns: result
configure_script_nodeB
Preview or configure one exact ScriptNode in explicit inline/file mode.
mode='inline' requires inline_script and forbids script_file.
mode='file' requires script_file and forbids inline_script.
The extension validates the mode before mutating the graph.
Inputs: graph_path (required), mode, preview, node_path, +4 more. Returns: result
connect_action_graphC
Preview or connect one source output to one target input in an Action Graph.
Inputs: graph_path (required), source_attr (required), target_attr (required), preview, +2 more. Returns: result
create_action_graphA
Create and wire an OmniGraph Action Graph.
Builds a complete Action Graph with nodes, connections and attribute values using og.Controller.edit(). This is the programmatic equivalent of creating an Action Graph in the visual editor.
Args: graph_path: USD prim path for the graph (default "/World/ActionGraph"). nodes: List of node definitions. Each dict has: - "path": Node path relative to graph (e.g. "OnPlaybackTick") - "type": OmniGraph node type (e.g. "omni.graph.action.OnPlaybackTick") connections: List of [source_attr, target_attr] pairs for wiring nodes. Each attr is "NodePath.outputs:attrName" or "NodePath.inputs:attrName". values: List of attribute value overrides. Each dict has: - "attr": Full attribute path (e.g. "ScriptNode.inputs:script") - "value": The value to set evaluator: Graph evaluator type (default "execution", what Action Graphs use). "push" evaluates every application update regardless of the timeline, so an OnPlaybackTick-driven ScriptNode would keep running even while the simulation is stopped. script_file: Convenience shortcut — path to a local Python script file. When provided, automatically creates OnPlaybackTick → ScriptNode nodes, wires them, and attaches the script file (sets usePath + scriptPath). The nodes and connections parameters are ignored when script_file is set. RECOMMENDED for anything you will iterate on — edit the file and reload_script "just works", with the better reload story. inline_script: Convenience shortcut — inline Python (must define setup(db)/compute(db)). Auto-creates OnPlaybackTick → ScriptNode, wires them, and sets the script inline (usePath=False). For small, static graphs. For anything you will iterate on, prefer script_file — it has the better reload story (edit the file + reload_script "just works"; inline edits need edit_action_graph).
Example (inline script — one-step): create_action_graph( inline_script="def setup(db): pass\ndef compute(db): return True" )
Example (script file — one-step, recommended for iteration): create_action_graph( script_file="/path/to/controller.py" )
Inputs: nodes, values, evaluator, command_id, +5 more. Returns: result
create_cameraB
Add a camera sensor to the scene.
Args: prim_path: Prim path for the camera. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. resolution: [width, height] image resolution. Default 1280x720.
Inputs: position, rotation, prim_path, command_id, +2 more. Returns: result
create_collision_groupC
Create a USD collision group and return relationship read-back.
Inputs: group_path (required), collider_paths (required), command_id, idempotency_key, +3 more. Returns: result
create_lidarB
Add a lidar sensor to the scene.
Args: prim_path: Prim path for the lidar. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. config: Isaac Sim 6 supported preset name, such as Example_Rotary or OS1. variant: Optional preset variant string or variant-set mapping. horizontal_fov_deg: Generic sensor horizontal field of view in degrees. vertical_fov_deg: Generic sensor vertical field of view in degrees. horizontal_resolution_deg: Generic sensor horizontal angular resolution. vertical_resolution_deg: Generic sensor vertical angular resolution. rotation_rate_hz: Integer generic rotary scan rate from 1 to 100 Hz. min_range_m: Generic sensor minimum range in meters. max_range_m: Generic sensor maximum range in meters.
Inputs: config, variant, position, rotation, +10 more. Returns: result
create_lightB
Create a light in the scene.
Args: light_type: Type of light — DistantLight, DomeLight, SphereLight, RectLight, DiskLight, or CylinderLight. position: [x, y, z] world position. intensity: Light intensity. color: [r, g, b] light color (0-1). rotation: [rx, ry, rz] rotation in degrees. prim_path: Custom prim path. Auto-generated if not provided.
Inputs: color, position, rotation, intensity, +4 more. Returns: result
create_materialA
Create a PBR or physics material.
Args: material_type: "pbr" for visual material or "physics" for physics material. prim_path: Prim path for the material. Auto-generated if not set. color: [r, g, b] diffuse color (0-1). PBR only. roughness: Surface roughness (0-1). PBR only. metallic: Metallic value (0-1). PBR only. static_friction: Static friction coefficient (>=0). Physics only. dynamic_friction: Dynamic friction coefficient (>=0 and <= static). Physics only. restitution: Bounciness from 0 to 1. Physics only.
Inputs: color, metallic, prim_path, roughness, +6 more. Returns: result
create_objectA
Create a primitive object (Cube, Sphere, Cylinder, Cone, Capsule, Plane).
Prefer size for absolute sizing: size is the target in METERS
(default 1.0), so size=0.3 gives a 0.3 m object regardless of type.
scale is a RAW MULTIPLIER of the primitive's NATIVE size, not meters.
Native sizes: Cube/Sphere/Cylinder/Cone/Capsule = 2 m, Plane = 1 m.
So scale=0.5 on a Cube -> 1 m, and scale=[0.4,0.4,0.3] -> a
0.8 x 0.8 x 0.6 m box (0.4 * 2 m), which surprises callers who expect
0.4 m. Use scale only for deliberate non-uniform shaping; otherwise
use size. If both are given, scale wins and size is ignored.
Returns prim_path, actual_size [x, y, z] in meters, and bounding_box (min/max corners in world coordinates) so you can accurately place other objects relative to this one.
Args:
object_type: Type of primitive — Cube, Sphere, Cylinder, Cone, Capsule, or Plane
(case-insensitive; "cube" is normalized to "Cube").
position: [x, y, z] world position.
rotation: [rx, ry, rz] rotation in degrees.
scale: [sx, sy, sz] RAW multiplier of the native size (2 m for most
prims, 1 m for Plane). NOT meters. Overrides size.
size: Target size in METERS (default 1.0). Absolute; independent of
the primitive's native size. Ignored if scale is provided.
color: [r, g, b] color values (0-1).
physics_enabled: Enable physics on this object.
prim_path: Custom prim path. Auto-generated if not provided.
Inputs: size, color, scale, position, +6 more. Returns: result
create_physics_jointA
Create fixed, revolute, or prismatic joint with explicit local frames.
Positions and prismatic limits are meters; revolute limits are degrees. Quaternion rotations use [w, x, y, z]. Axis is X, Y, or Z.
Inputs: body1 (required), joint_path (required), joint_type (required), axis, +10 more. Returns: result
create_physics_sceneB
Create a physics scene with ground plane. Call get_scene_info first to verify connection.
Args: gravity: Gravity vector [x, y, z]. Default is standard gravity. scene_name: Name for the physics scene prim.
Inputs: gravity, command_id, scene_name, idempotency_key. Returns: result
create_robotA
Create a robot in the scene from the Isaac Sim asset library.
Supports fuzzy matching — e.g. "franka", "spot", "g1", "go1". Call list_available_robots first to see all available robots. Call create_physics_scene before creating robots.
Returns prim_path, robot_key, joint_names, and num_dof so you can immediately use set_joint_positions without a follow-up get_robot_info call.
Args: robot_type: Robot name or search term. Fuzzy matched against available robots. position: [x, y, z] world position. name: Custom name for the robot prim. prim_path: Exact USD prim path (e.g. "/World/Franka"). Overrides name-based path.
Inputs: name, position, prim_path, command_id, +2 more. Returns: result
create_ros2_camera_publisherA
Preview or create a ROS2CameraHelper publisher for an MCP camera runtime.
The camera prim must first be created with create_camera. Its owned render product is resolved automatically; render_product_path is an explicit override for externally-created render products.
Inputs: graph_path (required), camera_prim_path (required), preview, frame_id, +9 more. Returns: result
create_ros2_clock_publisherB
Preview or create a playback-driven ROS 2 Clock publisher graph.
Inputs: preview, domain_id, command_id, graph_path, +5 more. Returns: result
create_ros2_joint_state_publisherC
Preview or create an IsaacReadJointState ROS 2 JointState publisher graph.
Inputs: graph_path (required), target_prim (required), preview, domain_id, +5 more. Returns: result
create_ros2_lidar_publisherB
Preview or create a ROS2RtxLidarHelper publisher for an MCP RTX LiDAR runtime.
The LiDAR prim must first be created with create_lidar. Its owned render product is resolved automatically; render_product_path is an explicit override for externally-created render products.
Inputs: graph_path (required), lidar_prim_path (required), preview, frame_id, +9 more. Returns: result
create_ros2_tf_publisherC
Preview or create a 6.0-style ComputeTransformTree to ROS 2 TF publisher.
Inputs: graph_path (required), target_prims (required), preview, domain_id, +7 more. Returns: result
create_sdg_jobB
Preview or create a bounded manual-trigger BasicWriter SDG job.
Randomizers are typed records. transform accepts prim_paths and
optional position/rotation/scale min/max vectors. light accepts
prim_paths and optional intensity/color min/max values.
Inputs: annotations (required), frame_count (required), camera_prim_path (required), seed, +7 more. Returns: result
delete_action_graphB
Preview or delete one exact Action Graph, with deletion read-back.
Inputs: graph_path (required), preview, command_id, idempotency_key. Returns: result
delete_artifactA
Delete one managed artifact after validating its opaque handle.
Inputs: handle (required), command_id, idempotency_key. Returns: result
delete_humanC
Delete one MCP-owned human. Preview is enabled by default.
Inputs: human_path (required), preview, command_id, idempotency_key, +1 more. Returns: result
delete_objectB
Delete an object; Camera/LiDAR use verified lifecycle teardown.
Args: prim_path: The prim path of the object to delete. post_delete_updates: Camera/LiDAR verification updates, from 1 to 240. Ignored for ordinary prims.
Inputs: prim_path (required), command_id, idempotency_key, post_delete_updates. Returns: result
delete_ros2_workflowB
Preview or delete one MCP-owned ROS 2 workflow; foreign graphs are refused.
Inputs: graph_path (required), preview, command_id, idempotency_key. Returns: result
delete_sdg_jobC
Preview or delete one terminal job record and optionally its artifacts.
Inputs: job_id (required), preview, command_id, idempotency_key, +1 more. Returns: result
delete_sensorA
Release and delete one Camera or LiDAR, then verify it stays absent.
Args: prim_path: Managed Camera or LiDAR prim path. post_delete_updates: Kit updates used to detect delayed prim or render-product reappearance. Range 1 to 240.
Inputs: prim_path (required), command_id, idempotency_key, post_delete_updates. Returns: result
disconnect_action_graphC
Preview or remove one exact connection from an Action Graph.
Inputs: graph_path (required), source_attr (required), target_attr (required), preview, +2 more. Returns: result
edit_action_graphA
Edit an existing OmniGraph Action Graph: set attribute values or add connections.
Use this to update ScriptNode scripts (inline or file path), change attribute values, or add new connections on an already-created graph.
For ScriptNode with a local file script, set both usePath and scriptPath: values=[ {"attr": "ScriptNode.inputs:usePath", "value": true}, {"attr": "ScriptNode.inputs:scriptPath", "value": "/path/to/script.py"} ]
For ScriptNode with inline script: values=[ {"attr": "ScriptNode.inputs:usePath", "value": false}, {"attr": "ScriptNode.inputs:script", "value": "def compute(db): ..."} ]
Args: graph_path: USD prim path of the existing graph (default "/World/ActionGraph"). values: List of attribute value overrides. Each dict has: - "attr": Attribute path relative to graph (e.g. "ScriptNode.inputs:script") - "value": The value to set connections: List of [source_attr, target_attr] pairs to add.
Inputs: values, command_id, graph_path, connections, +1 more. Returns: result
edit_composition_arcB
Preview or edit a reference/payload arc. Payloads also support load/unload; every mutation is read back and rolled back on failure.
Inputs: action (required), arc_type (required), prim_path (required), preview, +4 more. Returns: result
edit_sublayerC
Preview or add/remove a root-layer subLayer path with stopped-timeline and exact read-back checks.
Inputs: action (required), layer_path (required), index, preview, +2 more. Returns: result
evaluate_action_graphC
Explicitly evaluate one exact Action Graph and return post-evaluation status.
Inputs: graph_path (required), command_id, idempotency_key. Returns: result
execute_scriptA
Escape hatch: execute arbitrary Python code in Isaac Sim.
PREFER named tools over this for: reading/setting joints (set_joint_positions, get_joint_positions), inspecting state (get_prim_info, get_physics_state, get_joint_config), stepping simulation (step_simulation), and checking logs (get_isaac_logs).
USE this only for a trusted operation that no named tool covers. Execution is policy-gated, cooperatively timed, output-bounded, cwd-restricted, and audited by code hash. Background scheduling is denied unless both the request and administrator policy explicitly opt in.
CAUTION: touching an articulation controlled by a running ScriptNode / Action Graph can silently break its control path (no error is raised). While a graph is running, read-only diagnostics (get_prim_info, get_physics_state, get_joint_positions, get_isaac_logs) are safe, but stop_simulation before using execute_script or named write tools on the same articulation.
For persistent controllers (>20 lines), write a .py file and load it with reload_script instead of pasting code here.
Args: code: Python code to execute in the Isaac Sim context. cwd: Optional working directory to add to sys.path before execution. timeout_s: Cooperative Python deadline; capped by extension policy. max_output_bytes: Combined per-stream capture limit; capped by policy. allow_background: Request background scheduling. Default false and rejected unless the extension policy also enables it.
Inputs: code (required), cwd, timeout_s, command_id, +3 more. Returns: result
execute_trajectoryC
Start a trajectory job and return immediately without blocking the MCP worker.
Inputs: trajectory_id (required), command_id, timeout_ms, idempotency_key. Returns: result
generate_3dB
Generate a 3D model from text or image using Beaver3D, then load it into the scene.
Args: text_prompt: Text description for 3D generation. image_url: URL of an image for 3D generation. position: [x, y, z] world position for the generated model. scale: [sx, sy, sz] scale factors.
Inputs: scale, position, image_url, command_id, +2 more. Returns: result
get_action_graphA
Read one Action Graph's nodes, edges, state, and optional values or script source.
Script source is omitted by default because inline source and local file contents may be large or sensitive.
Inputs: graph_path (required), command_id, include_values, idempotency_key, +1 more. Returns: result
get_action_graph_statusA
Read one Action Graph's enabled, evaluation, and error status.
Inputs: graph_path (required), command_id, idempotency_key. Returns: result
get_artifact_infoB
Get size, type, hash, expiry, and producer metadata for a managed artifact.
Inputs: handle (required), command_id, idempotency_key. Returns: result
get_camera_calibrationC
Read camera intrinsics, extrinsics, projection, resolution, and units.
Inputs: prim_path, command_id, idempotency_key. Returns: result
get_capabilitiesA
Discover the live Isaac Sim MCP runtime and its explicit limitations.
Returns the Isaac Sim version, selected adapter, active physics backend, adapter-owned PhysX/Newton matrix, extension states, sensor warm-up policy, feature flags, and unsupported arguments. This query is read-only and works while the USD stage is still starting.
Inputs: command_id, idempotency_key. Returns: result
get_collision_groupB
Read collision group members, filters, inversion and merge name.
Inputs: group_path (required), command_id, idempotency_key. Returns: result
get_humanC
Return USD and live Behavior Agent state for one IRA human.
Inputs: human_path (required), command_id, idempotency_key. Returns: result
get_isaac_logsA
Diagnostic tool: recent WARN/ERROR logs plus captured print() output.
Captures carb.log_*/omni.log WARN+ERROR and stdout from execute_script / reload_script (tagged [PRINT]). Plain print() outside those captured contexts may not appear.
Defaults are agent-friendly: non-destructive (clear=False) and scoped to the current run (since_last_play=True) so you see logs from what you just did, not stale entries from previous runs.
Args: clear: If True, empty the buffer after reading. Default False. count: Maximum number of log entries to return. since_last_play: If True (default), return only entries since the last timeline Play. Set False for the full buffer. filter_command_id: Return only structured records for this command. severity: Optional structured-record severity filter. source: Optional structured-record source filter (for example kit or dispatcher).
Inputs: clear, count, source, severity, +4 more. Returns: result
get_job_statusB
Read a managed, motion, or SDG job without executing it again.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
get_joint_configA
Diagnostic tool: get joint drive configuration for a robot articulation.
Returns stiffness, damping, limits, target vs actual positions, and position error for each joint. Call this when:
Joint drives are not tracking targets (check position_error)
Joints are oscillating or unstable (check stiffness/damping ratio)
Joints hit limits unexpectedly (check lower_limit/upper_limit)
Args: prim_path: USD path to the robot articulation root.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_joint_positionsA
Read current joint positions from a robot.
Units: radians for revolute joints, meters for prismatic joints. Joint order matches the joint_names from get_robot_info. For a combined step-and-read, prefer step_simulation with observe_joints.
Args: prim_path: The prim path of the robot.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_joint_stateA
Read measured joint state and active command targets.
Returns position, velocity, measured effort, position/velocity/effort targets, joint name/index mapping, joint type, and explicit units. Use either joint_names or joint_indices for a subset; the selectors are mutually exclusive. Revolute units are radians, radians_per_second, and newton_meters. Prismatic units are meters, meters_per_second, and newtons.
Args: prim_path: USD path of the articulation root. joint_names: Optional exact, case-sensitive joint names in requested order. joint_indices: Optional DOF indices in requested order.
Inputs: prim_path (required), command_id, joint_names, joint_indices, +1 more. Returns: result
get_lidar_configB
Read effective LiDAR FOV, angular resolution, rate, range, and USD schema values.
Inputs: prim_path, command_id, idempotency_key. Returns: result
get_lidar_point_cloudB
Get typed point cloud fields from a lidar sensor.
Args: prim_path: Prim path of the lidar sensor. output_path: Optional explicit .npz path for artifact mode. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum encoded NPZ bytes allowed in inline mode.
Inputs: prim_path, command_id, output_path, return_mode, +2 more. Returns: result
get_materialB
Read PBR or physics material parameters and explicit units.
Inputs: material_path (required), command_id, idempotency_key. Returns: result
get_material_bindingA
Read the resolved physics or visual material binding on a prim.
Inputs: target_prim_path (required), command_id, idempotency_key, material_purpose. Returns: result
get_motion_statusC
Read motion job state, progress, timing, and terminal error details.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
get_physics_bodyA
Read authored rigid body, collider, approximation, mass and density.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_physics_jointC
Read joint type, body targets, frames, axis, limits and units.
Inputs: joint_path (required), command_id, idempotency_key. Returns: result
get_physics_stateA
Diagnostic tool: get physics state for a prim.
Returns rigid body status, mass, velocities, kinematic flag, and collision info. Velocity units: linear_velocity in m/s, angular_velocity in rad/s. Velocities are only non-zero once the simulation has advanced — step the simulation (or play) before reading them. Call this when:
Objects fall through the ground (check collision enabled)
Objects don't move when expected (check is_kinematic, mass)
Grasping fails (check collision on gripper fingers and target object)
Args: prim_path: USD path to the prim to inspect.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_prim_infoA
Get detailed information about a specific prim.
Returns type, children, and a transform block holding position, rotation [rx, ry, rz] in degrees (XYZ order, the same convention transform_object accepts), and scale. For geometric prims (Cube, Sphere, Cylinder, Cone, Capsule), also returns actual_size [x, y, z] in meters accounting for scale and default primitive dimensions.
Args: prim_path: The USD prim path to inspect.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_replicator_statusC
Read Replicator extension, orchestrator, job, writer, and trigger state.
Inputs: command_id, idempotency_key. Returns: result
get_robot_infoA
Get robot joint information including names, DOF count, joint types, and limits.
Call this after create_robot to understand the robot's kinematic structure. Returns joint names ordered by DOF index, joint types (revolute/prismatic), and joint limits (degrees for revolute, meters for prismatic).
Args: prim_path: The prim path of the robot.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_ros2_statusB
Read ROS 2 bridge/core/nodes state, domain source, distro, and workflow count.
Inputs: command_id, idempotency_key. Returns: result
get_runtime_statusA
Read supervisor, crash, restart, and protocol-health state without requiring Isaac Sim.
This local diagnostic remains available when the extension socket is closed. It never starts, stops, or mutates Isaac Sim and never replays a failed command.
Inputs: command_id, idempotency_key. Returns: result
get_scene_infoC
Ping the Isaac Sim extension server and return scene information including stage path, assets root, and prim count.
Inputs: command_id, idempotency_key. Returns: result
get_script_audit_logC
Return bounded script audit metadata. Source code is represented only by SHA-256.
Inputs: count, command_id, idempotency_key. Returns: result
get_script_policyB
Return the current execute/reload policy without exposing script source or credentials.
Inputs: command_id, idempotency_key. Returns: result
get_sdg_job_statusC
Read bounded SDG job progress, lifecycle state, and cleanup read-back.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
get_sdg_manifestC
Return a terminal job manifest and managed artifact handles.
Inputs: job_id (required), command_id, idempotency_key. Returns: result
get_semantic_labelsA
Read Isaac Sim 6.0.1 LabelsAPI taxonomies and labels, plus any legacy semantic labels.
Inputs: prim_path (required), command_id, idempotency_key. Returns: result
get_simulation_stateA
Get the current simulation state: timeline status (playing/stopped/paused), simulation time, and physics dt. step_simulation does NOT require a running timeline — do not play just to step.
Inputs: command_id, idempotency_key. Returns: result
get_stage_compositionB
Read the root layer, full layer stack, prim count, references/payloads, variant selections, semantics, and stage metadata.
Inputs: root_path, command_id, idempotency_key. Returns: result
get_typed_attributeA
Read a USD attribute value, declared type, and authored-value state in JSON-safe form.
Inputs: attribute (required), prim_path (required), command_id, idempotency_key. Returns: result
import_urdfB
Import a robot from a URDF file into the scene.
Args: urdf_path: Path to the URDF file. prim_path: Prim path for the imported robot. position: [x, y, z] world position.
Inputs: urdf_path (required), position, prim_path, command_id, +1 more. Returns: result
list_action_graphsA
List Action Graphs below a USD root, including disabled graphs by default.
Args: root_path: USD subtree to inspect. include_disabled: Include graphs whose evaluation is disabled.
Inputs: root_path, command_id, idempotency_key, include_disabled. Returns: result
list_available_robotsA
List all available robots discovered from the Isaac Sim asset server. Returns robot keys, descriptions, manufacturers, and asset paths. The list is auto-discovered at startup and reflects the actual assets available in your Isaac Sim version.
Inputs: command_id, idempotency_key. Returns: result
list_controller_profilesA
List the exact gripper and mobile-base profiles accepted by high-level controller tools.
Inputs: command_id, idempotency_key. Returns: result
list_environmentsA
List all available environments discovered from the Isaac Sim asset server. Includes warehouses, offices, outdoor scenes, and more.
Inputs: command_id, idempotency_key. Returns: result
list_humansA
List IRA human characters without changing the stage.
Inputs: command_id, root_prim_path, idempotency_key, include_external. Returns: result
list_jobsC
List the bounded retained managed-job registry.
Inputs: count, command_id, idempotency_key, include_terminal. Returns: result
list_nvidia_assetsB
List NVIDIA assets that can be spawned through MCP.
Combines a verified Isaac/SimReady/Vegetation prop catalog with the robot library discovered from the running Isaac Sim 6.0 asset root.
Args:
category: Optional category: robot_arm, quadruped, agv,
robot_other, conveyor, warehouse, factory, or
vegetation, lifting, or instrumentation.
query: Optional case-insensitive text filter across key,
description, manufacturer, and asset path.
max_results: Maximum returned entries, from 1 to 250.
Inputs: query, category, command_id, max_results, +1 more. Returns: result
list_primsA
List all prims in the scene, optionally filtered by type.
Args: root_path: Root path to start listing from. prim_type: Filter by prim type (e.g. "Mesh", "Xform").
Inputs: prim_type, root_path, command_id, idempotency_key. Returns: result
list_ros2_workflowsB
List MCP-owned ROS 2 publisher graphs below a USD root.
Inputs: root_path, command_id, idempotency_key. Returns: result
load_environmentA
Load a pre-built environment into the scene. Supports fuzzy matching. Call list_environments first to see available options.
Many shipped environments are authored Y-up and/or in centimeters; those are rotated and rescaled to match the stage, and the response reports what was applied under "corrections". It also returns "bounds" with the environment's extent and floor_height, so objects can be placed on the ground without a second query. Read prim_path from the response rather than assuming it — it defaults to a named child of /Environment.
Args: environment: Environment name or search term (e.g. "warehouse", "hospital", "office"). prim_path: Prim path for the loaded environment. Defaults to /Environment/, which keeps it separate from the stage's default lighting and lets clear_scene remove it.
Inputs: environment (required), prim_path, command_id, idempotency_key. Returns: result
load_usdA
Load a USD asset from a URL or file path into the scene.
Args: usd_url: URL or local path to the USD file. prim_path: Prim path for the loaded asset. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.
Inputs: usd_url (required), scale, position, prim_path, +2 more. Returns: result
modify_lightC
Modify properties of an existing light.
Args: prim_path: The prim path of the light to modify. intensity: New intensity value. color: [r, g, b] new light color (0-1).
Inputs: prim_path (required), color, intensity, command_id, +1 more. Returns: result
new_stageA
Preview or create a new empty stage. Requires scratch_stage=true, a scratch_root, stopped timeline, and defaults to preview-only.
Inputs: preview, command_id, scratch_root, scratch_stage, +1 more. Returns: result
open_gripperB
Open a gripper to the profile's declared open width with immediate target read-back.
Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result
open_stageB
Preview or open a local USD stage inside scratch_root. Destructive stage replacement requires explicit scratch_stage=true.
Inputs: path (required), preview, command_id, scratch_root, +3 more. Returns: result
pause_simulationC
Pause the physics simulation.
Inputs: command_id, idempotency_key. Returns: result
plan_joint_trajectoryA
Plan a bounded joint trajectory and return an opaque trajectory_id.
planner="rrt" uses NVIDIA Lula RRT and reports collision_checked=true; planner="cspace" creates a deterministic spline and explicitly reports collision_checked=false. start_joint_positions defaults to measured state; pass it explicitly for reproducible offline planning. This call never executes.
Inputs: prim_path (required), goal_joint_positions (required), planner, command_id, +6 more. Returns: result
play_simulationC
Start the physics simulation.
Inputs: command_id, idempotency_key. Returns: result
read_artifactB
Read one bounded base64 chunk from a managed artifact.
Inputs: handle (required), length, offset, command_id, +1 more. Returns: result
refresh_robot_libraryA
Force re-scan the asset server for available robots. Use this if new robot assets were added.
Inputs: command_id, idempotency_key. Returns: result
reload_scriptA
Reload a Python controller from a file on disk.
Two modes, chosen automatically:
If any Action-Graph ScriptNode references this file (inputs:scriptPath), those ScriptNodes are force-recompiled so your on-disk edits take effect on the running graph. This is how you iterate on a ScriptNode controller.
Otherwise the file is (re-)executed as a standalone controller, the way you would use execute_script for code longer than ~20 lines.
Workflow:
Write the controller as a .py file (attach via create_action_graph script_file=... for ScriptNode use)
reload_script to load / recompile it
step_simulation to debug (frozen timeline) or play for a ScriptNode demo
Edit the file and reload_script again to iterate
The file's directory is auto-added to sys.path.
Args: file_path: Path to the Python file on disk. module_name: Optional module name to reload (e.g. 'my_controller'). timeout_s: Cooperative Python deadline; capped by extension policy. max_output_bytes: Per-stream capture limit; capped by policy.
Inputs: file_path (required), timeout_s, command_id, module_name, +2 more. Returns: result
reload_script_nodeA
Preview or recompile one exact ScriptNode without cross-graph fallback.
Supply mode='inline' with inline_script to replace inline source.
For mode='file', script_file may select a new canonical local file;
when omitted, the node's existing canonical file path is reloaded. Omitting
mode asks the extension to retain and validate the node's current mode.
Inputs: graph_path (required), mode, preview, node_path, +4 more. Returns: result
save_stage_asA
Preview or export the current stage to a different local USD file inside scratch_root. Source overwrite is always rejected.
Inputs: path (required), preview, overwrite, command_id, +4 more. Returns: result
search_usdA
Search the NVIDIA USD asset library by text description, then load the best match.
Args: text_prompt: Text description of the 3D asset to search for. target_path: Prim path for the loaded result. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.
Inputs: text_prompt (required), scale, position, command_id, +2 more. Returns: result
set_action_graph_enabledB
Preview or enable/disable one Action Graph and read back its effective state.
Inputs: enabled (required), graph_path (required), preview, command_id, +1 more. Returns: result
set_gripper_widthA
Set total gripper opening width using an explicit robot profile.
The profile validates exact joint names and types before applying any position target. width_m is the total distance between both fingers.
Inputs: profile (required), width_m (required), prim_path (required), command_id, +1 more. Returns: result
set_human_behaviorC
Update supported live Behavior Agent settings with read-back.
Inputs: human_path (required), enabled, preview, speed_mps, +5 more. Returns: result
set_human_idleC
Cancel the active action by issuing Idle, optionally facing one target.
Inputs: human_path (required), preview, command_id, facing_position, +2 more. Returns: result
set_human_look_atC
Issue a Behavior Agent LookAt task; zero duration means indefinite.
Inputs: human_path (required), preview, command_id, idempotency_key, +3 more. Returns: result
set_human_targetC
Issue a Behavior Agent MoveTo task to exactly one target.
Inputs: human_path (required), preview, speed_mps, auto_brake, +4 more. Returns: result
set_joint_commandA
Apply an atomic position, velocity, or effort joint command.
All selectors and values are validated before the adapter is called, so an unknown name, invalid index, duplicate selector, non-finite value, or length mismatch applies nothing. Effort commands must be renewed every simulation update; this tool applies one update's effort command.
Args: prim_path: USD path of the articulation root. mode: One of position, velocity, or effort. values: Command values ordered by the selected joints. joint_names: Optional exact, case-sensitive joint subset. joint_indices: Optional DOF-index subset; mutually exclusive with joint_names.
Inputs: mode (required), values (required), prim_path (required), command_id, +3 more. Returns: result
set_joint_drive_configA
Atomically update drive gains, limits, or drive type for selected joints.
The timeline must be stopped. Numeric fields must be finite and non-negative. Values use runtime SI units: revolute gains are per radian, revolute velocity is radians_per_second, and prismatic velocity is meters_per_second. drive_type is force or acceleration. All inputs are validated before writing, and an apply failure triggers rollback.
Args: prim_path: USD path of the articulation root. stiffness: Optional proportional gain applied to every selected joint. damping: Optional derivative gain applied to every selected joint. max_force: Optional maximum drive effort. max_velocity: Optional maximum joint velocity; PhysX only. drive_type: Optional force or acceleration drive mode. joint_names: Optional exact, case-sensitive joint subset. joint_indices: Optional DOF-index subset; mutually exclusive with joint_names.
Inputs: prim_path (required), damping, max_force, stiffness, +6 more. Returns: result
set_joint_positionsA
Set target joint positions on a robot via ArticulationAction.
Units: radians for revolute joints, meters for prismatic joints (e.g. gripper fingers). Use get_robot_info to discover joint names, types, and limits first. After calling this, use step_simulation to advance and observe the result — do not use play_simulation + sleep.
Args: prim_path: The prim path of the robot. joint_positions: List of target joint position values. joint_indices: Optional list of joint indices to set. Sets all joints if not provided.
Inputs: prim_path (required), joint_positions (required), command_id, joint_indices, +1 more. Returns: result
set_mobile_base_velocityA
Apply a profiled differential or holonomic base velocity target.
The timeline must be playing for a non-zero command. Targets persist until replaced, so always call stop_mobile_base at the end of motion. Differential profiles reject non-zero lateral velocity.
Inputs: profile (required), prim_path (required), forward_mps (required), yaw_radps, +3 more. Returns: result
set_physics_paramsA
Atomically configure Isaac Sim 6 PhysX scene parameters.
The timeline must be stopped. time_step must be within
[0.0001, 1.0] seconds and map to an integer PhysX steps-per-second
value (for example, 1/60 or 1/120). Isaac Sim's Stage time codes and
minimum simulation frame-rate are synchronized to that rate. Enabling GPU physics selects GPU
dynamics and GPU broadphase; disabling it selects MBP broadphase.
This does not change the launcher-controlled physics GPU ordinal.
Success includes both USD and runtime read-back; failed application is
rolled back as one transaction.
Args: gravity: Gravity vector [x, y, z]. time_step: Physics time step in seconds. gpu_enabled: Enable or disable GPU dynamics and matching broadphase.
Inputs: gravity, time_step, command_id, gpu_enabled, +1 more. Returns: result
set_semantic_labelsA
Preview or apply Isaac Sim 6.0.1 UsdSemantics LabelsAPI labels under one taxonomy with read-back.
Inputs: labels (required), taxonomy (required), prim_path (required), preview, +3 more. Returns: result
set_typed_attributeA
Preview or set a finite, explicitly typed USD attribute. Existing attributes require overwrite=true and cannot change type.
Inputs: value (required), attribute (required), prim_path (required), type_name (required), +5 more. Returns: result
set_variant_selectionB
Preview or select an existing USD variant after validating the set and available names.
Inputs: prim_path (required), selection (required), variant_set (required), preview, +2 more. Returns: result
spawn_humanA
Spawn animated NVIDIA human characters into the CURRENT USD stage.
Uses Isaac Sim 6.0+'s isaacsim.replicator.agent.core (IRA 1.x)
character loader and SimReady assets. Unlike IRA.setup_simulation(),
this tool does not reopen the environment USD, so existing scene edits
are preserved.
A NavMesh is required. The tool can bake an authored volume and waits up to 2000 update frames for complex factory stages. The extension is enabled automatically, but the tool fails closed when no usable NavMesh or character assets are available. Stop the timeline first.
execute_script remains useful for one-off IRA experiments.
reload_script is the better companion for reusable interaction or
event logic after characters have been spawned.
Args:
count: Number of new characters to add, from 1 to 25.
group_name: USD-safe IRA group name. Reusing it appends characters.
root_prim_path: Parent prim for all characters.
asset_path: Character asset directory. Defaults to
Isaac/People/Characters/.
motion_library_path: Human motion-library USD. Defaults to the IRA
extension setting / Isaac Sim asset root.
behavior: wander, patrol, stop, or manual. Use
manual when interaction scripts will issue Behavior Agent
tasks directly.
spawn_areas: Optional NavMesh area names used for spawning.
speed_range: Two walking speeds in m/s.
distance_range: Two wander-leg distances in meters.
idle_time_range: Two idle/stop durations in seconds.
patrol_points: Required for patrol; reachable NavMesh points as
[[x,y,z], ...].
position: Optional exact world position for one spawned character.
The point should lie on the NavMesh.
rotation: Optional [rx, ry, rz] degrees; requires count=1.
auto_create_navmesh_volume: Create one include-volume when the
stage has none. This modifies the current stage and requires
navmesh_volume_size.
navmesh_volume_center: Center of the new NavMesh volume. Defaults
to [0, 0, 1].
navmesh_volume_size: Full [x, y, z] size in stage units. Make
it cover the walkable floor and surrounding obstacles.
seed: Deterministic 32-bit IRA randomization seed.
Inputs: seed, count, behavior, position, +15 more. Returns: result
spawn_nvidia_assetA
Spawn one catalog asset into the CURRENT USD stage.
Call list_nvidia_assets first to obtain an exact asset_key.
Robot keys create controllable articulations through the existing robot
handler. Props, conveyors, warehouse equipment, and vegetation are
referenced from verified NVIDIA USD paths and remain compatible with
transform_object, clone_object, and delete_object.
Args:
asset_key: Exact key returned by list_nvidia_assets.
prim_path: Optional absolute destination prim path. A unique path
under /World/Assets is generated when omitted.
position: Optional world [x, y, z].
rotation: Optional XYZ Euler degrees [rx, ry, rz].
scale: Optional [sx, sy, sz] multiplier.
Inputs: asset_key (required), scale, position, rotation, +3 more. Returns: result
start_jobB
Start an eligible asset or sensor command as a bounded background job.
Motion and SDG starts retain their typed tools; their returned IDs can still be queried and cancelled through the unified job tools.
Inputs: command_type (required), params, command_id, deadline_ms, +1 more. Returns: result
start_sdg_jobC
Preview or asynchronously start one configured SDG job.
Inputs: job_id (required), preview, command_id, idempotency_key. Returns: result
step_simulationA
Advance the simulation by exactly N physics frames on a FROZEN timeline.
step is self-contained: it initialises physics on first call and operates on a paused/stopped timeline, so N is always exact and observations correlate to a known frame count.
Do NOT call play_simulation before or during the debug loop; step is for a frozen timeline. If the timeline is already playing, step returns an error (a free run cannot be counted frame-by-frame). Use play_simulation ONLY for a final continuous run / ScriptNode-driven demo, never for debugging.
Typical debug loop (no play):
set_joint_positions to command the robot
step_simulation with observe_prims and observe_joints
get_joint_config if drives are not tracking correctly
get_physics_state if objects are not behaving as expected
Adjust and repeat
Args: num_steps: Number of simulation frames to step. observe_prims: List of prim paths to observe (returns position + velocity). observe_joints: List of articulation prim paths to observe (returns joint positions).
Inputs: num_steps, command_id, observe_prims, observe_joints, +1 more. Returns: result
stop_mobile_baseB
Set every profiled wheel velocity target to zero and verify the stop target read-back.
Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result
stop_simulationA
Stop the physics simulation and reset to spawn state.
Resets articulations and rigid bodies to their spawn pose (the state captured at first Play), like the Isaac UI Stop button. Call this to return the scene to a clean starting point before another run.
Inputs: command_id, idempotency_key. Returns: result
transform_objectA
Set the transform (position, rotation, scale) of an existing object.
Args: prim_path: The prim path of the object to transform. position: [x, y, z] new world position. rotation: [rx, ry, rz] new rotation in degrees. scale: [sx, sy, sz] new scale factors.
Inputs: prim_path (required), scale, position, rotation, +2 more. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 129 tools
Most tools target distinct resources and actions, but several pairs overlap meaningfully: set_joint_positions vs set_joint_command(mode=position), get_joint_positions vs get_joint_state, and start_job vs start_sdg_job/execute_trajectory. The detailed descriptions help, but an agent could easily misselect between the joint and job-related tools.
The overwhelming majority follow a clear lowercase snake_case verb_noun pattern (create_camera, get_prim_info, delete_sdg_job). Minor deviations like new_stage, save_stage_as, cleanup_artifacts, and get_capabilities break the uniformity slightly, but the convention is otherwise predictable and consistent.
129 tools is far beyond the 50+ extreme threshold for any server, even one covering a broad domain like Isaac Sim. The count creates a massive surface that would overwhelm an agent, and many subsystems could be consolidated or exposed via narrower interfaces.
The toolset is remarkably broad, covering scene, physics, robots, sensors, materials, lights, SDG, ROS2, humans, action graphs, scripting, and asset workflows. Minor gaps exist—no delete_material, delete_light, modify_material, or explicit environment teardown—but most lifecycle paths are covered and workarounds are plausible.
Maintenance
Related MCP Connectors
Control Unreal Engine to browse assets, import content, and manage levels and sequences. Automate…
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to control Unreal E…
Generate 3D models, worlds, PBR materials, and coordinated asset packs from AI agents.
Connect assistants to your workspace knowledge, skills, and work tools.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to read, inspect, and manipulate OpenUSD 3D scenes, including scene graph traversal, property querying, mesh export, and variant switching.10MIT
- AlicenseAqualityBmaintenanceEnables natural language control of NVIDIA Isaac Sim through the Model Context Protocol, allowing you to create robots, build scenes, run simulations, and debug physics from any MCP-compatible IDE.4258 PyPI70MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI tools like Windsurf and Claude to control NVIDIA Isaac Sim and Isaac Lab through natural language, providing tools for scene inspection, prim management, physics simulation, and robot spawning.8Apache 2.0
- AlicenseAqualityBmaintenanceLets an AI assistant work in NVIDIA Isaac Sim and other Omniverse Kit apps: inspect and edit USD scenes, capture viewports with segmentation, run physics, drive robots and record. A Python MCP server on PyPI, with a Kit extension it reaches over a token-checked loopback link.44209 PyPIMIT