Skip to main content
Glama

IsaacSim-MCP

IsaacSim-MCP 讓支援 MCP 的 AI client 透過具名、可驗證的 tools 控制 NVIDIA Isaac Sim。涵蓋 USD 場景、機器人、感測器、物理、Action Graph、ROS 2、Replicator SDG、動畫人物、NVIDIA 資產與模擬控制。

主要驗證環境是 Windows 與 Isaac Sim 6.0.1。本專案延伸自 whats2000/isaacsim-mcp-server,沿用 MIT License。

Table of Contents

Related MCP server: Isaac Sim MCP Server

核心功能

  • 建立、查詢、變形、組合、儲存與驗證 USD Stage。

  • 控制 articulation、joint drive、motion planning、gripper 與 mobile base。

  • 在 Stage/robot replacement 後重建失效的 physics tensor 與 articulation binding,並保持 joint name/value 對應一致。

  • 擷取 Camera RGB 與 typed RTX outputs;RGB 可選擇 MCP-native ImageContent,供支援影像內容的遠端 client 顯示或交給 vision model。

  • 建立 PhysX scene、body、collider、joint、PBR material 與 physics material。

  • 管理 Action Graph、ScriptNode、ROS 2 publisher、Replicator SDG job 與 human behavior lifecycle。

  • 透過 managed artifact 傳輸大型輸出,支援 hash、bounded chunk、TTL 與 cleanup。

  • 提供 command ID、idempotency、policy limit、job、cancel、read-back、rollback 與 redacted diagnostics。

MCP Tool Inventory 由 source decorators 自動產生;Tool profiles 說明 129-tool 相容模式與 98-tool 合併模式。目前 runtime 支援狀態以 get_capabilities 為準。

架構

LLM → Skill → MCP Server → TCP → Isaac Extension → Handler → Adapter → Isaac Sim
flowchart LR
    A[LLM / MCP Client] --> B[Project Skill]
    B --> C[Python MCP Server]
    C -->|127.0.0.1:8766| D[Isaac Sim Extension]
    D --> E[Handler]
    E --> F[V5 / V6 Adapter]
    F --> G[Isaac Sim / USD / PhysX]

各層責任、request lifecycle、runtime routes 與權威來源見 ARCHITECTURE.md。

Installation

安裝 MCP Server:

git clone https://github.com/Tim0320/IsaacSim-MCP.git
cd IsaacSim-MCP
py -3.10 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -e .

Windows 的完整環境需求與安裝方式見 Windows 安裝指南。

Configuration

stdio 是預設 transport。Codex/Claude Desktop 可以沿用現有設定,不需要新增 HTTP 環境變數:

{
  "mcpServers": {
    "isaac-sim-live": {
      "command": "F:\\IsaacSim-MCP\\.venv\\Scripts\\python.exe",
      "args": ["-m", "isaac_mcp.server"],
      "env": {
        "ISAAC_MCP_HOST": "127.0.0.1",
        "ISAAC_MCP_PORT": "8766"
      }
    }
  }
}

Streamable HTTP 使用另一組設定:

Variable

Default

Purpose

ISAAC_MCP_TRANSPORT

stdio

設為 streamable-http 或 alias http 以啟用 HTTP transport。

ISAAC_MCP_HTTP_HOST

127.0.0.1

MCP HTTP listener address。

ISAAC_MCP_HTTP_PORT

8000

MCP HTTP listener port。

ISAAC_MCP_TOOL_PROFILE

legacy

legacy 保留既有 129 tools;consolidated 提供 98-tool 合併 surface;full 僅供遷移與測試。

MCP_ALLOWED_HOSTS

未設定

額外允許送到 HTTP endpoint 的 exact Host header,以逗號分隔;loopback hosts 永遠保留。

ISAAC_MCP_RUNTIME_STATE_FILE

%LOCALAPPDATA%\IsaacSim-MCP\runtime-state.json

Supervisor 與 MCP Server 共用的 bounded crash/restart 狀態檔。

ISAAC_MCP_RUNTIME_PROBE_TIMEOUT_SECONDS

1

get_runtime_status protocol health probe timeout。

ISAAC_MCP_HOST=127.0.0.1 與 ISAAC_MCP_PORT=8766 仍屬於 Python MCP Server 和 Isaac Sim Extension 間的 runtime TCP socket。不要把它們改成 HTTP endpoint 設定。

Tool profiles

預設 legacy 不改任何既有 tool name 或 schema。ChatGPT 等需要較小 action space 的 client,可在啟動 MCP Server 前設定:

$env:ISAAC_MCP_TOOL_PROFILE = "consolidated"

consolidated 用 action 或 publisher_type 合併同一資源的讀寫/控制操作,公開 98 tools。它只調整 MCP tool surface,仍呼叫原本的 Isaac Extension commands。修改 profile 後必須重新啟動 MCP Server,讓 client 重新取得 tool list。完整對照見 Tool profiles。

.env.example 只列出可用設定,server 不會自動載入它。請在啟動 python -m isaac_mcp.server 的同一個 process environment 設定 profile。啟動後可用 MCP tools/list 與 get_capabilities.data.mcp_server 核對實際公開 surface;consolidated 應回報 98 tools。

Running IsaacSim-MCP

建議用 supervisor 啟動 Isaac Sim 與 Extension:

$env:ISAACSIM_ROOT = "C:\isaacsim"
.\scripts\run_isaac_sim_supervised.ps1

Supervisor 會沿用既有 launcher 的 Isaac Sim 6.0.1、Extension、port 與 Physics GPU guard。非零 exit code 視為異常退出,預設最多在 300 秒內重啟 3 次並使用 exponential backoff;exit code 0 視為正常關閉,不會自動重開。啟動前若 protocol health probe 已找到健康 runtime,或 8766 已被無回應程序占用,supervisor 都會拒絕再啟動一份 Isaac Sim。需要 one-shot 行為時仍可使用 .\scripts\run_isaac_sim.ps1。

Supervisor 與 MCP Server 是兩個獨立程序。Codex/Claude 仍由 client 用 stdio 啟動 isaac_mcp.server;Streamable HTTP 仍用下方命令啟動。Supervisor 狀態預設寫入 %LOCALAPPDATA%\IsaacSim-MCP\runtime-state.json,不包含 environment、command source 或 log 內容。

當 Isaac Sim crash、正在重啟或超過 restart budget 時,MCP tools 會回傳 ISAAC_RUNTIME_RECOVERING、ISAAC_RUNTIME_CRASHED 或 ISAAC_RUNTIME_UNAVAILABLE。呼叫 get_runtime_status 可在 8766 關閉時讀到 exit code、時間、attempt、restart count、health 狀態與建議動作。Connection loss 後不會自動 replay write;runtime 恢復後必須先 read-back,再決定是否用原 idempotency key 重送。

完整 process ownership、state schema、restart budget 與 Agent recovery contract 見 Runtime supervision 與 crash recovery。

本機 Codex/Claude Desktop 會依上一節設定,以 stdio 自動啟動 MCP Server。需要 Streamable HTTP 時,在另一個 PowerShell session 啟動:

$env:ISAAC_MCP_TRANSPORT = "streamable-http"
$env:ISAAC_MCP_HTTP_HOST = "127.0.0.1"
$env:ISAAC_MCP_HTTP_PORT = "8000"
$env:ISAAC_MCP_TOOL_PROFILE = "consolidated"
$env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1"

.\.venv\Scripts\python.exe -m isaac_mcp.server

MCP HTTP endpoint 是 http://127.0.0.1:8000/mcp。先呼叫 get_capabilities,再呼叫 get_scene_info,確認 runtime 與 Stage 後才執行 write。

Remote MCP Access

IsaacSim-MCP 預設只在本機使用。ChatGPT 等雲端 MCP client 無法直接連到 localhost、127.0.0.1 或 192.168.x.x 私有位址。它們需要能從 public internet 存取的 HTTPS endpoint,例如 Tailscale Funnel:

ChatGPT
   ↓ HTTPS
Public MCP Endpoint
   ↓
IsaacSim-MCP
   ↓ TCP 127.0.0.1:8766
Isaac Sim

Method

Tailnet only

Public Internet

ChatGPT can access

localhost

No

No

No

Tailscale Serve

Yes

No

No

Tailscale Funnel

Yes

Yes

Yes

Tailscale Serve 只把服務提供給同一個 tailnet;Tailscale Funnel 會建立 public internet 可達的 HTTPS endpoint。ChatGPT MCP Connector 需要 Funnel 或其他 public HTTPS deployment。

Tailscale Funnel

  1. 從 Tailscale Download 安裝適合目前作業系統的版本,登入後啟用本機 Tailscale:

    tailscale up
  2. 啟動 Isaac Sim Extension,接著在 repository 根目錄啟動 Streamable HTTP server。將範例 hostname 換成 tailscale funnel 顯示的完整 hostname:

    $env:ISAAC_MCP_TRANSPORT = "streamable-http"
    $env:ISAAC_MCP_HTTP_HOST = "127.0.0.1"
    $env:ISAAC_MCP_HTTP_PORT = "8000"
    $env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1,your-device.your-tailnet.ts.net"
    
    .\.venv\Scripts\python.exe -m isaac_mcp.server

    MCP_ALLOWED_HOSTS 使用 FastMCP 的 exact Host matching。專案會替每個完整 hostname 接受有 port 與無 port 形式,但不接受 .ts.net suffix 或 *.ts.net wildcard。loopback hosts 永遠保留,所以加入外部 hostname 不會破壞本機 HTTP 存取。

  3. 在另一個 terminal 建立背景 Funnel。8000 是 repository 的預設 HTTP MCP port:

    tailscale funnel --bg 8000

    預期輸出類似:

    Available on the internet:
    
    https://your-device.your-tailnet.ts.net
    
    |-- / proxy http://127.0.0.1:8000

    公開 MCP URL 是 https://your-device.your-tailnet.ts.net/mcp。

  4. 查看狀態或停止同一個 Funnel:

    tailscale funnel status
    tailscale funnel --bg 8000 off

    tailscale funnel reset 會清除本機全部 Funnel configuration。完整參數以 Tailscale Funnel CLI 為準。

  5. 從外部網路驗證:

    curl -i https://your-device.your-tailnet.ts.net/mcp

    Streamable HTTP 對一般 GET 的回應會依 request header 和 transport 狀態而異,200、405 或 406 都可能表示請求已到達 endpoint。結果不應再是 421 Misdirected Request/Invalid Host header,也不應出現 502 Bad Gateway 或 timeout。

ChatGPT MCP Connector

Funnel 驗證成功後,依 OpenAI Developer mode 與 MCP apps 官方說明 在 ChatGPT Developer Mode 新增 MCP Connector:

Name:
IsaacSim

MCP Server URL:
https://your-device.your-tailnet.ts.net/mcp

Authentication:
No authentication

需要讓 MCP client 直接取得 RGB 圖片時,呼叫:

capture_camera_output(
  prim_path="/World/Camera",
  output_type="rgb",
  return_mode="image"
)

return_mode="image" 會回傳 schema 1.0 文字 metadata 與 MCP-native ImageContent。Server 會驗證 PNG base64、byte size 與 SHA-256,且不會把大型 base64 重複放進文字 metadata。artifact 仍是預設模式,因此既有 Codex/Claude stdio 與下載流程不變。Client 是否在 UI 顯示圖片、把圖片交給 vision model,取決於該 client 對 MCP image content 的支援;要宣稱 ChatGPT 可見,必須完成實際 Connector acceptance,不能只依 server unit test 推論。

capture_image 與 capture_camera_output 的 MCP input schema 會把 return_mode 公開為 metadata | artifact | inline | image enum,Agent 不需要從說明文字猜測。Camera 新建或解析度變更後若第一個 RTX read 回 CAMERA_FRAME_NOT_READY,MCP Server 會等待一次 render tick 並自動重試一次;一次 tool call 即可完成 Render → Capture → ImageContent。若第二次仍未取得 frame,錯誤會原樣回傳,不會無限重試。

IsaacSim-MCP 目前沒有 HTTP authentication。MCP_ALLOWED_HOSTS 只防止不受信任的 Host header,不是身分驗證。公開 Funnel 會讓任何能連到該 URL 的人嘗試呼叫 MCP tools,可能控制本機 Isaac Sim。只在你接受這個風險時啟用,使用完立即停止 Funnel;正式共享環境應在 MCP server 前加入 authentication、authorization 與存取稽核。

支援版本

Component

狀態

Isaac Sim 6.0.1 on Windows

主要驗證 runtime

PhysX

透過 V6 adapter 與 guarded live verifier 支援

Newton

只有 active backend matrix 回報 supported 且 verified 的功能才能使用,其餘 fail closed

Isaac Sim 5.1.x

Legacy adapter;不在目前 6.0.1 release gate 範圍

Isaac Lab MCP

明確延後,與目前 Isaac Sim MCP 分開處理

Package、extension、response、capability 與 backend-matrix version 各有獨立相容規則,見 Protocol versions 與 migration。

文件入口

從 docs/README.md 開始:

  • getting-started/:安裝與第一次連線。

  • concepts/:protocol、transport、governance 與 job 共用模型。

  • reference/:目前 API 與 capability 契約。

  • development/:測試、scratch-stage 與 release 流程。

  • research/:有日期的 1.x~6.x tasks 與 verification snapshots。

Agent 工作流程在 .agents/skills/omniverse-windows-workspace/SKILL.md。Tool、version 與 capability 的權威來源見 Authority and Generated Metadata。 Agent 的 retry、reconnect、read-back 與 fail-closed 行為見 Error Codes and Agent Recovery。 Robot articulation、physics tensor、joint mapping、DriveAPI 與 IK lifecycle 見 Robot Runtime Lifecycle。

Safety 與 verification

  • isaac-sim-live 透過 TCP 8766 控制 Stage;documentation MCP 無法證明 live Stage 已改變。

  • Write 必須通過 timeline、backend、extension、ownership 與 path prerequisites。Destructive verification 只允許 exact scratch Stage 與 MCP-owned namespace。

  • Registry presence 只證明 tool 可發現。Live pass 必須有 operation-specific read-back 與 cleanup evidence。

  • execute_script/reload_script 只接受可信任程式碼,且受 policy 限制;優先使用 named tools。

  • API keys 只供外部資產 provider 選用,不可寫入 source、MCP JSON、report 或 commit。

  • Release 前建立 verified backup 並執行 strict release gate。Commit 與 push 仍需使用者明確授權。

Troubleshooting

421 Misdirected Request / Invalid Host header

Tailscale Funnel 已成功轉發,但 Funnel hostname 尚未被 MCP Server 信任。把完整 hostname 加入 MCP_ALLOWED_HOSTS,例如:

$env:MCP_ALLOWED_HOSTS = "localhost,127.0.0.1,your-device.your-tailnet.ts.net"

不要使用 .ts.net 或 *.ts.net;FastMCP 只接受設定中的 exact hostname。專案會自動接受該 hostname 有 port 與無 port 的形式。

502 Bad Gateway

Funnel 可用,但本機 MCP Server 沒有在 Funnel 指向的 port listening。確認 ISAAC_MCP_HTTP_PORT 與 tailscale funnel --bg 8000 的 port 相同,並保持 MCP Server process 執行中。

Connection timeout

執行 tailscale funnel status,確認 Funnel 仍啟用,再檢查 firewall 與 Tailscale connection 狀態。

Works locally but ChatGPT cannot connect

確認公開 URL 使用 HTTPS 且以 /mcp 結尾,並確認使用 Tailscale Funnel。Tailscale Serve 只有 tailnet 內部可達,ChatGPT 無法透過它連線。

Camera 成功但 ChatGPT 沒有顯示圖片

確認呼叫的是 RGB capture_camera_output(..., return_mode="image") 或 capture_image(..., return_mode="image")。artifact 只回 managed handle,inline 只把 base64 放進 JSON;兩者都不是 MCP-native image content。若 response 已含 ImageContent,但 UI 仍無圖片,代表目前 client 沒有渲染或轉交該 content block;保留 artifact handle 作為下載路徑,並把結果標記為 client-side partial support。

若 response 是 CAMERA_FRAME_NOT_READY,代表 Server 已完成一次 bounded warm-up retry,但第二次 RTX read 仍未準備。確認 Camera prim、timeline 與 render runtime 正常,再建立新的 capture attempt;不要在單次 request 內無限重播。

ChatGPT 顯示 legacy tools,但呼叫回 Unknown tool

這代表 connector 保存的 tool schema 與目前 server profile 不一致。先對實際 HTTP endpoint 執行 MCP tools/list,再呼叫 get_capabilities:

  • legacy 應公開 129 tools,包含 play_simulation、open_gripper、list_available_robots。

  • consolidated 應公開 98 tools,改用 control_timeline、control_gripper、robot_library。

確認 server process 啟動前已設定 ISAAC_MCP_TOOL_PROFILE,重新啟動 server,接著讓 ChatGPT Connector 重新連線並重新取得 tool schema。若 connector migration 期間仍保存舊名稱,可暫時使用 full 同時公開新舊名稱;完成 schema 更新後切回 consolidated。不要永久使用 full,它會把 action space 增加到 151 tools。

COMMAND_FAILED / Not connected to Isaac

MCP Server 仍在執行,但 Isaac Sim Extension 沒有回應。改用 .\scripts\run_isaac_sim_supervised.ps1 啟動 runtime,並呼叫 get_runtime_status。Agent 應依 availability_code 處理:ISAAC_RUNTIME_RECOVERING 等待 bounded recovery;ISAAC_RUNTIME_CRASHED 檢查 last_crash 並修正 root cause;ISAAC_RUNTIME_UNAVAILABLE 啟動 supervisor。任何 write 在連線中斷後都必須先 read-back,不能盲目重送。

IK_FAILED / physics tensor entity is not valid

目前版本會在 Stage 或 robot identity 改變後丟棄舊 articulation wrapper,並重建 invalid Physics SimulationView。若仍看到此訊息,先確認 MCP Server 與 Isaac Extension 都來自同一個最新 checkout,再重新啟動 Isaac Sim 載入新 Extension。不要用重複 Play/Pause 當成修復,也不要重送未確認結果的 write。

PhysicsDriveAPI, a non-empty instance name must be provided

目前 USD fallback 會依 joint type 使用 angular 或 linear DriveAPI instance。若舊 runtime 仍回此錯誤,重新啟動 Isaac Sim 以載入更新後的 Extension,接著先用 get_joint_config 做 read-only 驗證。

Development

uv sync --dev
uv run pytest -q -m "not live and not windows_launcher and not unix_launcher" -k "not test_detect_version_returns_zero_on_failure"
uv run ruff check .
.\.venv\Scripts\python.exe .\scripts\generate_tool_inventory.py --check

Live tests 是 opt-in,並受 scratch-stage harness 保護。禁止對使用者 Stage 執行 legacy destructive integration suite。

License

本專案使用 MIT License。散布修改版時需保留授權、copyright notices 與 upstream attribution。

Available Tools

129 tools
apply_materialA

Bind a material to an object.

Args: material_path: Prim path of the material. target_prim_path: Prim path of the object to apply the material to. material_purpose: auto, physics, or visual. Auto selects from the material schema.

Inputs: material_path (required), target_prim_path (required), command_id, idempotency_key, +1 more. Returns: result

apply_stage_batchA

Preview or atomically apply up to 100 layer/arc/variant/semantic/attribute edits. Any failed operation restores the stage snapshot.

Inputs: operations (required), preview, command_id, idempotency_key, +1 more. Returns: result

bake_navmeshB

Bake the current stage NavMesh with a bounded wait; preview by default.

Inputs: preview, command_id, max_frames, idempotency_key. Returns: result

cancel_jobC

Request cancellation and return a predictable terminal lifecycle state.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

cancel_motionB

Cancel a running motion job and hold its last commanded position.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

cancel_sdg_jobB

Preview or request cancellation; resources detach at the next safe point.

Inputs: job_id (required), preview, command_id, idempotency_key. Returns: result

capture_camera_outputB

Capture a typed RTX camera annotator output.

Args: prim_path: Prim path of the camera. output_type: depth, distance_to_image_plane, semantic_segmentation, instance_segmentation, instance_id_segmentation, normals, or motion_vectors. output_path: Optional explicit .npy path for artifact mode. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum raw bytes allowed in inline mode. Maximum is 4 MiB.

Inputs: prim_path, command_id, output_path, output_type, +3 more. Returns: result

capture_imageB

Capture an RGB image from a camera sensor.

Args: prim_path: Prim path of the camera. output_path: Optional explicit .png path for artifact mode. If omitted, uses the managed artifact root. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum PNG bytes allowed in inline mode. Maximum configurable value is 4 MiB.

Inputs: prim_path, command_id, output_path, return_mode, +2 more. Returns: result

cleanup_artifactsC

Delete expired managed artifacts and report reclaimed capacity.

Inputs: command_id, idempotency_key. Returns: result

clear_sceneA

Remove all prims from the scene.

Also empties any environment loaded by load_environment, so a later create_physics_scene(floor=True) does not stack a second ground under the first. The stage's defaultLight is always kept — a stage with no light renders black, which looks like a broken camera.

Args: keep_physics: If True, keep physics scene prims. keep_environment: If True, keep the loaded environment. Reloading one costs seconds, so pass this when clearing objects between attempts.

Inputs: command_id, keep_physics, idempotency_key, keep_environment. Returns: result

clone_objectA

Duplicate an existing object to a new prim path.

Args: source_path: Prim path of the object to clone. target_path: Prim path for the cloned object. position: [x, y, z] position for the clone. Keeps original position if not set.

Inputs: source_path (required), target_path (required), position, command_id, +1 more. Returns: result

close_gripperA

Close a gripper to the profile's declared closed width with immediate target read-back.

Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result

compute_ikA

Solve bounded Lula inverse kinematics without moving the robot.

Quaternion order is scalar-first [w, x, y, z]. The explicit warm-start and random seed make repeated requests reproducible. The result reports achieved end-effector errors and whether collision checking was done.

Inputs: prim_path (required), target_position (required), command_id, timeout_ms, +9 more. Returns: result

configure_physics_bodyB

Atomically configure rigid/static body, collider approximation and mass.

body_type is dynamic, kinematic, or static. approximation is one of none, convex_hull, convex_decomposition, mesh_simplification, bounding_cube, or bounding_sphere. Mass uses kg and density uses kg/m^3. The timeline must be stopped.

Inputs: prim_path (required), mass_kg, body_type, command_id, +4 more. Returns: result

configure_script_nodeB

Preview or configure one exact ScriptNode in explicit inline/file mode.

mode='inline' requires inline_script and forbids script_file. mode='file' requires script_file and forbids inline_script. The extension validates the mode before mutating the graph.

Inputs: graph_path (required), mode, preview, node_path, +4 more. Returns: result

connect_action_graphC

Preview or connect one source output to one target input in an Action Graph.

Inputs: graph_path (required), source_attr (required), target_attr (required), preview, +2 more. Returns: result

create_action_graphA

Create and wire an OmniGraph Action Graph.

Builds a complete Action Graph with nodes, connections and attribute values using og.Controller.edit(). This is the programmatic equivalent of creating an Action Graph in the visual editor.

Args: graph_path: USD prim path for the graph (default "/World/ActionGraph"). nodes: List of node definitions. Each dict has: - "path": Node path relative to graph (e.g. "OnPlaybackTick") - "type": OmniGraph node type (e.g. "omni.graph.action.OnPlaybackTick") connections: List of [source_attr, target_attr] pairs for wiring nodes. Each attr is "NodePath.outputs:attrName" or "NodePath.inputs:attrName". values: List of attribute value overrides. Each dict has: - "attr": Full attribute path (e.g. "ScriptNode.inputs:script") - "value": The value to set evaluator: Graph evaluator type (default "execution", what Action Graphs use). "push" evaluates every application update regardless of the timeline, so an OnPlaybackTick-driven ScriptNode would keep running even while the simulation is stopped. script_file: Convenience shortcut — path to a local Python script file. When provided, automatically creates OnPlaybackTick → ScriptNode nodes, wires them, and attaches the script file (sets usePath + scriptPath). The nodes and connections parameters are ignored when script_file is set. RECOMMENDED for anything you will iterate on — edit the file and reload_script "just works", with the better reload story. inline_script: Convenience shortcut — inline Python (must define setup(db)/compute(db)). Auto-creates OnPlaybackTick → ScriptNode, wires them, and sets the script inline (usePath=False). For small, static graphs. For anything you will iterate on, prefer script_file — it has the better reload story (edit the file + reload_script "just works"; inline edits need edit_action_graph).

Example (inline script — one-step): create_action_graph( inline_script="def setup(db): pass\ndef compute(db): return True" )

Example (script file — one-step, recommended for iteration): create_action_graph( script_file="/path/to/controller.py" )

Inputs: nodes, values, evaluator, command_id, +5 more. Returns: result

create_cameraB

Add a camera sensor to the scene.

Args: prim_path: Prim path for the camera. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. resolution: [width, height] image resolution. Default 1280x720.

Inputs: position, rotation, prim_path, command_id, +2 more. Returns: result

create_collision_groupC

Create a USD collision group and return relationship read-back.

Inputs: group_path (required), collider_paths (required), command_id, idempotency_key, +3 more. Returns: result

create_lidarB

Add a lidar sensor to the scene.

Args: prim_path: Prim path for the lidar. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. config: Isaac Sim 6 supported preset name, such as Example_Rotary or OS1. variant: Optional preset variant string or variant-set mapping. horizontal_fov_deg: Generic sensor horizontal field of view in degrees. vertical_fov_deg: Generic sensor vertical field of view in degrees. horizontal_resolution_deg: Generic sensor horizontal angular resolution. vertical_resolution_deg: Generic sensor vertical angular resolution. rotation_rate_hz: Integer generic rotary scan rate from 1 to 100 Hz. min_range_m: Generic sensor minimum range in meters. max_range_m: Generic sensor maximum range in meters.

Inputs: config, variant, position, rotation, +10 more. Returns: result

create_lightB

Create a light in the scene.

Args: light_type: Type of light — DistantLight, DomeLight, SphereLight, RectLight, DiskLight, or CylinderLight. position: [x, y, z] world position. intensity: Light intensity. color: [r, g, b] light color (0-1). rotation: [rx, ry, rz] rotation in degrees. prim_path: Custom prim path. Auto-generated if not provided.

Inputs: color, position, rotation, intensity, +4 more. Returns: result

create_materialA

Create a PBR or physics material.

Args: material_type: "pbr" for visual material or "physics" for physics material. prim_path: Prim path for the material. Auto-generated if not set. color: [r, g, b] diffuse color (0-1). PBR only. roughness: Surface roughness (0-1). PBR only. metallic: Metallic value (0-1). PBR only. static_friction: Static friction coefficient (>=0). Physics only. dynamic_friction: Dynamic friction coefficient (>=0 and <= static). Physics only. restitution: Bounciness from 0 to 1. Physics only.

Inputs: color, metallic, prim_path, roughness, +6 more. Returns: result

create_objectA

Create a primitive object (Cube, Sphere, Cylinder, Cone, Capsule, Plane).

Prefer size for absolute sizing: size is the target in METERS (default 1.0), so size=0.3 gives a 0.3 m object regardless of type.

scale is a RAW MULTIPLIER of the primitive's NATIVE size, not meters. Native sizes: Cube/Sphere/Cylinder/Cone/Capsule = 2 m, Plane = 1 m. So scale=0.5 on a Cube -> 1 m, and scale=[0.4,0.4,0.3] -> a 0.8 x 0.8 x 0.6 m box (0.4 * 2 m), which surprises callers who expect 0.4 m. Use scale only for deliberate non-uniform shaping; otherwise use size. If both are given, scale wins and size is ignored.

Returns prim_path, actual_size [x, y, z] in meters, and bounding_box (min/max corners in world coordinates) so you can accurately place other objects relative to this one.

Args: object_type: Type of primitive — Cube, Sphere, Cylinder, Cone, Capsule, or Plane (case-insensitive; "cube" is normalized to "Cube"). position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. scale: [sx, sy, sz] RAW multiplier of the native size (2 m for most prims, 1 m for Plane). NOT meters. Overrides size. size: Target size in METERS (default 1.0). Absolute; independent of the primitive's native size. Ignored if scale is provided. color: [r, g, b] color values (0-1). physics_enabled: Enable physics on this object. prim_path: Custom prim path. Auto-generated if not provided.

Inputs: size, color, scale, position, +6 more. Returns: result

create_physics_jointA

Create fixed, revolute, or prismatic joint with explicit local frames.

Positions and prismatic limits are meters; revolute limits are degrees. Quaternion rotations use [w, x, y, z]. Axis is X, Y, or Z.

Inputs: body1 (required), joint_path (required), joint_type (required), axis, +10 more. Returns: result

create_physics_sceneB

Create a physics scene with ground plane. Call get_scene_info first to verify connection.

Args: gravity: Gravity vector [x, y, z]. Default is standard gravity. scene_name: Name for the physics scene prim.

Inputs: gravity, command_id, scene_name, idempotency_key. Returns: result

create_robotA

Create a robot in the scene from the Isaac Sim asset library.

Supports fuzzy matching — e.g. "franka", "spot", "g1", "go1". Call list_available_robots first to see all available robots. Call create_physics_scene before creating robots.

Returns prim_path, robot_key, joint_names, and num_dof so you can immediately use set_joint_positions without a follow-up get_robot_info call.

Args: robot_type: Robot name or search term. Fuzzy matched against available robots. position: [x, y, z] world position. name: Custom name for the robot prim. prim_path: Exact USD prim path (e.g. "/World/Franka"). Overrides name-based path.

Inputs: name, position, prim_path, command_id, +2 more. Returns: result

create_ros2_camera_publisherA

Preview or create a ROS2CameraHelper publisher for an MCP camera runtime.

The camera prim must first be created with create_camera. Its owned render product is resolved automatically; render_product_path is an explicit override for externally-created render products.

Inputs: graph_path (required), camera_prim_path (required), preview, frame_id, +9 more. Returns: result

create_ros2_clock_publisherB

Preview or create a playback-driven ROS 2 Clock publisher graph.

Inputs: preview, domain_id, command_id, graph_path, +5 more. Returns: result

create_ros2_joint_state_publisherC

Preview or create an IsaacReadJointState ROS 2 JointState publisher graph.

Inputs: graph_path (required), target_prim (required), preview, domain_id, +5 more. Returns: result

create_ros2_lidar_publisherB

Preview or create a ROS2RtxLidarHelper publisher for an MCP RTX LiDAR runtime.

The LiDAR prim must first be created with create_lidar. Its owned render product is resolved automatically; render_product_path is an explicit override for externally-created render products.

Inputs: graph_path (required), lidar_prim_path (required), preview, frame_id, +9 more. Returns: result

create_ros2_tf_publisherC

Preview or create a 6.0-style ComputeTransformTree to ROS 2 TF publisher.

Inputs: graph_path (required), target_prims (required), preview, domain_id, +7 more. Returns: result

create_sdg_jobB

Preview or create a bounded manual-trigger BasicWriter SDG job.

Randomizers are typed records. transform accepts prim_paths and optional position/rotation/scale min/max vectors. light accepts prim_paths and optional intensity/color min/max values.

Inputs: annotations (required), frame_count (required), camera_prim_path (required), seed, +7 more. Returns: result

delete_action_graphB

Preview or delete one exact Action Graph, with deletion read-back.

Inputs: graph_path (required), preview, command_id, idempotency_key. Returns: result

delete_artifactA

Delete one managed artifact after validating its opaque handle.

Inputs: handle (required), command_id, idempotency_key. Returns: result

delete_humanC

Delete one MCP-owned human. Preview is enabled by default.

Inputs: human_path (required), preview, command_id, idempotency_key, +1 more. Returns: result

delete_objectB

Delete an object; Camera/LiDAR use verified lifecycle teardown.

Args: prim_path: The prim path of the object to delete. post_delete_updates: Camera/LiDAR verification updates, from 1 to 240. Ignored for ordinary prims.

Inputs: prim_path (required), command_id, idempotency_key, post_delete_updates. Returns: result

delete_ros2_workflowB

Preview or delete one MCP-owned ROS 2 workflow; foreign graphs are refused.

Inputs: graph_path (required), preview, command_id, idempotency_key. Returns: result

delete_sdg_jobC

Preview or delete one terminal job record and optionally its artifacts.

Inputs: job_id (required), preview, command_id, idempotency_key, +1 more. Returns: result

delete_sensorA

Release and delete one Camera or LiDAR, then verify it stays absent.

Args: prim_path: Managed Camera or LiDAR prim path. post_delete_updates: Kit updates used to detect delayed prim or render-product reappearance. Range 1 to 240.

Inputs: prim_path (required), command_id, idempotency_key, post_delete_updates. Returns: result

disconnect_action_graphC

Preview or remove one exact connection from an Action Graph.

Inputs: graph_path (required), source_attr (required), target_attr (required), preview, +2 more. Returns: result

edit_action_graphA

Edit an existing OmniGraph Action Graph: set attribute values or add connections.

Use this to update ScriptNode scripts (inline or file path), change attribute values, or add new connections on an already-created graph.

For ScriptNode with a local file script, set both usePath and scriptPath: values=[ {"attr": "ScriptNode.inputs:usePath", "value": true}, {"attr": "ScriptNode.inputs:scriptPath", "value": "/path/to/script.py"} ]

For ScriptNode with inline script: values=[ {"attr": "ScriptNode.inputs:usePath", "value": false}, {"attr": "ScriptNode.inputs:script", "value": "def compute(db): ..."} ]

Args: graph_path: USD prim path of the existing graph (default "/World/ActionGraph"). values: List of attribute value overrides. Each dict has: - "attr": Attribute path relative to graph (e.g. "ScriptNode.inputs:script") - "value": The value to set connections: List of [source_attr, target_attr] pairs to add.

Inputs: values, command_id, graph_path, connections, +1 more. Returns: result

edit_composition_arcB

Preview or edit a reference/payload arc. Payloads also support load/unload; every mutation is read back and rolled back on failure.

Inputs: action (required), arc_type (required), prim_path (required), preview, +4 more. Returns: result

edit_sublayerC

Preview or add/remove a root-layer subLayer path with stopped-timeline and exact read-back checks.

Inputs: action (required), layer_path (required), index, preview, +2 more. Returns: result

evaluate_action_graphC

Explicitly evaluate one exact Action Graph and return post-evaluation status.

Inputs: graph_path (required), command_id, idempotency_key. Returns: result

execute_scriptA

Escape hatch: execute arbitrary Python code in Isaac Sim.

PREFER named tools over this for: reading/setting joints (set_joint_positions, get_joint_positions), inspecting state (get_prim_info, get_physics_state, get_joint_config), stepping simulation (step_simulation), and checking logs (get_isaac_logs).

USE this only for a trusted operation that no named tool covers. Execution is policy-gated, cooperatively timed, output-bounded, cwd-restricted, and audited by code hash. Background scheduling is denied unless both the request and administrator policy explicitly opt in.

CAUTION: touching an articulation controlled by a running ScriptNode / Action Graph can silently break its control path (no error is raised). While a graph is running, read-only diagnostics (get_prim_info, get_physics_state, get_joint_positions, get_isaac_logs) are safe, but stop_simulation before using execute_script or named write tools on the same articulation.

For persistent controllers (>20 lines), write a .py file and load it with reload_script instead of pasting code here.

Args: code: Python code to execute in the Isaac Sim context. cwd: Optional working directory to add to sys.path before execution. timeout_s: Cooperative Python deadline; capped by extension policy. max_output_bytes: Combined per-stream capture limit; capped by policy. allow_background: Request background scheduling. Default false and rejected unless the extension policy also enables it.

Inputs: code (required), cwd, timeout_s, command_id, +3 more. Returns: result

execute_trajectoryC

Start a trajectory job and return immediately without blocking the MCP worker.

Inputs: trajectory_id (required), command_id, timeout_ms, idempotency_key. Returns: result

generate_3dB

Generate a 3D model from text or image using Beaver3D, then load it into the scene.

Args: text_prompt: Text description for 3D generation. image_url: URL of an image for 3D generation. position: [x, y, z] world position for the generated model. scale: [sx, sy, sz] scale factors.

Inputs: scale, position, image_url, command_id, +2 more. Returns: result

get_action_graphA

Read one Action Graph's nodes, edges, state, and optional values or script source.

Script source is omitted by default because inline source and local file contents may be large or sensitive.

Inputs: graph_path (required), command_id, include_values, idempotency_key, +1 more. Returns: result

get_action_graph_statusA

Read one Action Graph's enabled, evaluation, and error status.

Inputs: graph_path (required), command_id, idempotency_key. Returns: result

get_artifact_infoB

Get size, type, hash, expiry, and producer metadata for a managed artifact.

Inputs: handle (required), command_id, idempotency_key. Returns: result

get_camera_calibrationC

Read camera intrinsics, extrinsics, projection, resolution, and units.

Inputs: prim_path, command_id, idempotency_key. Returns: result

get_capabilitiesA

Discover the live Isaac Sim MCP runtime and its explicit limitations.

Returns the Isaac Sim version, selected adapter, active physics backend, adapter-owned PhysX/Newton matrix, extension states, sensor warm-up policy, feature flags, and unsupported arguments. This query is read-only and works while the USD stage is still starting.

Inputs: command_id, idempotency_key. Returns: result

get_collision_groupB

Read collision group members, filters, inversion and merge name.

Inputs: group_path (required), command_id, idempotency_key. Returns: result

get_humanC

Return USD and live Behavior Agent state for one IRA human.

Inputs: human_path (required), command_id, idempotency_key. Returns: result

get_isaac_logsA

Diagnostic tool: recent WARN/ERROR logs plus captured print() output.

Captures carb.log_*/omni.log WARN+ERROR and stdout from execute_script / reload_script (tagged [PRINT]). Plain print() outside those captured contexts may not appear.

Defaults are agent-friendly: non-destructive (clear=False) and scoped to the current run (since_last_play=True) so you see logs from what you just did, not stale entries from previous runs.

Args: clear: If True, empty the buffer after reading. Default False. count: Maximum number of log entries to return. since_last_play: If True (default), return only entries since the last timeline Play. Set False for the full buffer. filter_command_id: Return only structured records for this command. severity: Optional structured-record severity filter. source: Optional structured-record source filter (for example kit or dispatcher).

Inputs: clear, count, source, severity, +4 more. Returns: result

get_job_statusB

Read a managed, motion, or SDG job without executing it again.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

get_joint_configA

Diagnostic tool: get joint drive configuration for a robot articulation.

Returns stiffness, damping, limits, target vs actual positions, and position error for each joint. Call this when:

  • Joint drives are not tracking targets (check position_error)

  • Joints are oscillating or unstable (check stiffness/damping ratio)

  • Joints hit limits unexpectedly (check lower_limit/upper_limit)

Args: prim_path: USD path to the robot articulation root.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_joint_positionsA

Read current joint positions from a robot.

Units: radians for revolute joints, meters for prismatic joints. Joint order matches the joint_names from get_robot_info. For a combined step-and-read, prefer step_simulation with observe_joints.

Args: prim_path: The prim path of the robot.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_joint_stateA

Read measured joint state and active command targets.

Returns position, velocity, measured effort, position/velocity/effort targets, joint name/index mapping, joint type, and explicit units. Use either joint_names or joint_indices for a subset; the selectors are mutually exclusive. Revolute units are radians, radians_per_second, and newton_meters. Prismatic units are meters, meters_per_second, and newtons.

Args: prim_path: USD path of the articulation root. joint_names: Optional exact, case-sensitive joint names in requested order. joint_indices: Optional DOF indices in requested order.

Inputs: prim_path (required), command_id, joint_names, joint_indices, +1 more. Returns: result

get_lidar_configB

Read effective LiDAR FOV, angular resolution, rate, range, and USD schema values.

Inputs: prim_path, command_id, idempotency_key. Returns: result

get_lidar_point_cloudB

Get typed point cloud fields from a lidar sensor.

Args: prim_path: Prim path of the lidar sensor. output_path: Optional explicit .npz path for artifact mode. return_mode: metadata, artifact, or inline. Defaults to artifact. inline_max_bytes: Maximum encoded NPZ bytes allowed in inline mode.

Inputs: prim_path, command_id, output_path, return_mode, +2 more. Returns: result

get_materialB

Read PBR or physics material parameters and explicit units.

Inputs: material_path (required), command_id, idempotency_key. Returns: result

get_material_bindingA

Read the resolved physics or visual material binding on a prim.

Inputs: target_prim_path (required), command_id, idempotency_key, material_purpose. Returns: result

get_motion_statusC

Read motion job state, progress, timing, and terminal error details.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

get_navmesh_statusC

Return Navigation Core, NavMeshVolume, bake and area status.

Inputs: command_id, idempotency_key. Returns: result

get_physics_bodyA

Read authored rigid body, collider, approximation, mass and density.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_physics_jointC

Read joint type, body targets, frames, axis, limits and units.

Inputs: joint_path (required), command_id, idempotency_key. Returns: result

get_physics_stateA

Diagnostic tool: get physics state for a prim.

Returns rigid body status, mass, velocities, kinematic flag, and collision info. Velocity units: linear_velocity in m/s, angular_velocity in rad/s. Velocities are only non-zero once the simulation has advanced — step the simulation (or play) before reading them. Call this when:

  • Objects fall through the ground (check collision enabled)

  • Objects don't move when expected (check is_kinematic, mass)

  • Grasping fails (check collision on gripper fingers and target object)

Args: prim_path: USD path to the prim to inspect.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_prim_infoA

Get detailed information about a specific prim.

Returns type, children, and a transform block holding position, rotation [rx, ry, rz] in degrees (XYZ order, the same convention transform_object accepts), and scale. For geometric prims (Cube, Sphere, Cylinder, Cone, Capsule), also returns actual_size [x, y, z] in meters accounting for scale and default primitive dimensions.

Args: prim_path: The USD prim path to inspect.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_replicator_statusC

Read Replicator extension, orchestrator, job, writer, and trigger state.

Inputs: command_id, idempotency_key. Returns: result

get_robot_infoA

Get robot joint information including names, DOF count, joint types, and limits.

Call this after create_robot to understand the robot's kinematic structure. Returns joint names ordered by DOF index, joint types (revolute/prismatic), and joint limits (degrees for revolute, meters for prismatic).

Args: prim_path: The prim path of the robot.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_ros2_statusB

Read ROS 2 bridge/core/nodes state, domain source, distro, and workflow count.

Inputs: command_id, idempotency_key. Returns: result

get_runtime_statusA

Read supervisor, crash, restart, and protocol-health state without requiring Isaac Sim.

This local diagnostic remains available when the extension socket is closed. It never starts, stops, or mutates Isaac Sim and never replays a failed command.

Inputs: command_id, idempotency_key. Returns: result

get_scene_infoC

Ping the Isaac Sim extension server and return scene information including stage path, assets root, and prim count.

Inputs: command_id, idempotency_key. Returns: result

get_script_audit_logC

Return bounded script audit metadata. Source code is represented only by SHA-256.

Inputs: count, command_id, idempotency_key. Returns: result

get_script_policyB

Return the current execute/reload policy without exposing script source or credentials.

Inputs: command_id, idempotency_key. Returns: result

get_sdg_job_statusC

Read bounded SDG job progress, lifecycle state, and cleanup read-back.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

get_sdg_manifestC

Return a terminal job manifest and managed artifact handles.

Inputs: job_id (required), command_id, idempotency_key. Returns: result

get_semantic_labelsA

Read Isaac Sim 6.0.1 LabelsAPI taxonomies and labels, plus any legacy semantic labels.

Inputs: prim_path (required), command_id, idempotency_key. Returns: result

get_simulation_stateA

Get the current simulation state: timeline status (playing/stopped/paused), simulation time, and physics dt. step_simulation does NOT require a running timeline — do not play just to step.

Inputs: command_id, idempotency_key. Returns: result

get_stage_compositionB

Read the root layer, full layer stack, prim count, references/payloads, variant selections, semantics, and stage metadata.

Inputs: root_path, command_id, idempotency_key. Returns: result

get_typed_attributeA

Read a USD attribute value, declared type, and authored-value state in JSON-safe form.

Inputs: attribute (required), prim_path (required), command_id, idempotency_key. Returns: result

import_urdfB

Import a robot from a URDF file into the scene.

Args: urdf_path: Path to the URDF file. prim_path: Prim path for the imported robot. position: [x, y, z] world position.

Inputs: urdf_path (required), position, prim_path, command_id, +1 more. Returns: result

list_action_graphsA

List Action Graphs below a USD root, including disabled graphs by default.

Args: root_path: USD subtree to inspect. include_disabled: Include graphs whose evaluation is disabled.

Inputs: root_path, command_id, idempotency_key, include_disabled. Returns: result

list_available_robotsA

List all available robots discovered from the Isaac Sim asset server. Returns robot keys, descriptions, manufacturers, and asset paths. The list is auto-discovered at startup and reflects the actual assets available in your Isaac Sim version.

Inputs: command_id, idempotency_key. Returns: result

list_controller_profilesA

List the exact gripper and mobile-base profiles accepted by high-level controller tools.

Inputs: command_id, idempotency_key. Returns: result

list_environmentsA

List all available environments discovered from the Isaac Sim asset server. Includes warehouses, offices, outdoor scenes, and more.

Inputs: command_id, idempotency_key. Returns: result

list_humansA

List IRA human characters without changing the stage.

Inputs: command_id, root_prim_path, idempotency_key, include_external. Returns: result

list_jobsC

List the bounded retained managed-job registry.

Inputs: count, command_id, idempotency_key, include_terminal. Returns: result

list_nvidia_assetsB

List NVIDIA assets that can be spawned through MCP.

Combines a verified Isaac/SimReady/Vegetation prop catalog with the robot library discovered from the running Isaac Sim 6.0 asset root.

Args: category: Optional category: robot_arm, quadruped, agv, robot_other, conveyor, warehouse, factory, or vegetation, lifting, or instrumentation. query: Optional case-insensitive text filter across key, description, manufacturer, and asset path. max_results: Maximum returned entries, from 1 to 250.

Inputs: query, category, command_id, max_results, +1 more. Returns: result

list_primsA

List all prims in the scene, optionally filtered by type.

Args: root_path: Root path to start listing from. prim_type: Filter by prim type (e.g. "Mesh", "Xform").

Inputs: prim_type, root_path, command_id, idempotency_key. Returns: result

list_ros2_workflowsB

List MCP-owned ROS 2 publisher graphs below a USD root.

Inputs: root_path, command_id, idempotency_key. Returns: result

load_environmentA

Load a pre-built environment into the scene. Supports fuzzy matching. Call list_environments first to see available options.

Many shipped environments are authored Y-up and/or in centimeters; those are rotated and rescaled to match the stage, and the response reports what was applied under "corrections". It also returns "bounds" with the environment's extent and floor_height, so objects can be placed on the ground without a second query. Read prim_path from the response rather than assuming it — it defaults to a named child of /Environment.

Args: environment: Environment name or search term (e.g. "warehouse", "hospital", "office"). prim_path: Prim path for the loaded environment. Defaults to /Environment/, which keeps it separate from the stage's default lighting and lets clear_scene remove it.

Inputs: environment (required), prim_path, command_id, idempotency_key. Returns: result

load_usdA

Load a USD asset from a URL or file path into the scene.

Args: usd_url: URL or local path to the USD file. prim_path: Prim path for the loaded asset. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Inputs: usd_url (required), scale, position, prim_path, +2 more. Returns: result

modify_lightC

Modify properties of an existing light.

Args: prim_path: The prim path of the light to modify. intensity: New intensity value. color: [r, g, b] new light color (0-1).

Inputs: prim_path (required), color, intensity, command_id, +1 more. Returns: result

new_stageA

Preview or create a new empty stage. Requires scratch_stage=true, a scratch_root, stopped timeline, and defaults to preview-only.

Inputs: preview, command_id, scratch_root, scratch_stage, +1 more. Returns: result

open_gripperB

Open a gripper to the profile's declared open width with immediate target read-back.

Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result

open_stageB

Preview or open a local USD stage inside scratch_root. Destructive stage replacement requires explicit scratch_stage=true.

Inputs: path (required), preview, command_id, scratch_root, +3 more. Returns: result

pause_simulationC

Pause the physics simulation.

Inputs: command_id, idempotency_key. Returns: result

plan_joint_trajectoryA

Plan a bounded joint trajectory and return an opaque trajectory_id.

planner="rrt" uses NVIDIA Lula RRT and reports collision_checked=true; planner="cspace" creates a deterministic spline and explicitly reports collision_checked=false. start_joint_positions defaults to measured state; pass it explicitly for reproducible offline planning. This call never executes.

Inputs: prim_path (required), goal_joint_positions (required), planner, command_id, +6 more. Returns: result

play_simulationC

Start the physics simulation.

Inputs: command_id, idempotency_key. Returns: result

read_artifactB

Read one bounded base64 chunk from a managed artifact.

Inputs: handle (required), length, offset, command_id, +1 more. Returns: result

refresh_robot_libraryA

Force re-scan the asset server for available robots. Use this if new robot assets were added.

Inputs: command_id, idempotency_key. Returns: result

reload_scriptA

Reload a Python controller from a file on disk.

Two modes, chosen automatically:

  • If any Action-Graph ScriptNode references this file (inputs:scriptPath), those ScriptNodes are force-recompiled so your on-disk edits take effect on the running graph. This is how you iterate on a ScriptNode controller.

  • Otherwise the file is (re-)executed as a standalone controller, the way you would use execute_script for code longer than ~20 lines.

Workflow:

  1. Write the controller as a .py file (attach via create_action_graph script_file=... for ScriptNode use)

  2. reload_script to load / recompile it

  3. step_simulation to debug (frozen timeline) or play for a ScriptNode demo

  4. Edit the file and reload_script again to iterate

The file's directory is auto-added to sys.path.

Args: file_path: Path to the Python file on disk. module_name: Optional module name to reload (e.g. 'my_controller'). timeout_s: Cooperative Python deadline; capped by extension policy. max_output_bytes: Per-stream capture limit; capped by policy.

Inputs: file_path (required), timeout_s, command_id, module_name, +2 more. Returns: result

reload_script_nodeA

Preview or recompile one exact ScriptNode without cross-graph fallback.

Supply mode='inline' with inline_script to replace inline source. For mode='file', script_file may select a new canonical local file; when omitted, the node's existing canonical file path is reloaded. Omitting mode asks the extension to retain and validate the node's current mode.

Inputs: graph_path (required), mode, preview, node_path, +4 more. Returns: result

save_stage_asA

Preview or export the current stage to a different local USD file inside scratch_root. Source overwrite is always rejected.

Inputs: path (required), preview, overwrite, command_id, +4 more. Returns: result

search_usdA

Search the NVIDIA USD asset library by text description, then load the best match.

Args: text_prompt: Text description of the 3D asset to search for. target_path: Prim path for the loaded result. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Inputs: text_prompt (required), scale, position, command_id, +2 more. Returns: result

set_action_graph_enabledB

Preview or enable/disable one Action Graph and read back its effective state.

Inputs: enabled (required), graph_path (required), preview, command_id, +1 more. Returns: result

set_gripper_widthA

Set total gripper opening width using an explicit robot profile.

The profile validates exact joint names and types before applying any position target. width_m is the total distance between both fingers.

Inputs: profile (required), width_m (required), prim_path (required), command_id, +1 more. Returns: result

set_human_behaviorC

Update supported live Behavior Agent settings with read-back.

Inputs: human_path (required), enabled, preview, speed_mps, +5 more. Returns: result

set_human_idleC

Cancel the active action by issuing Idle, optionally facing one target.

Inputs: human_path (required), preview, command_id, facing_position, +2 more. Returns: result

set_human_look_atC

Issue a Behavior Agent LookAt task; zero duration means indefinite.

Inputs: human_path (required), preview, command_id, idempotency_key, +3 more. Returns: result

set_human_targetC

Issue a Behavior Agent MoveTo task to exactly one target.

Inputs: human_path (required), preview, speed_mps, auto_brake, +4 more. Returns: result

set_joint_commandA

Apply an atomic position, velocity, or effort joint command.

All selectors and values are validated before the adapter is called, so an unknown name, invalid index, duplicate selector, non-finite value, or length mismatch applies nothing. Effort commands must be renewed every simulation update; this tool applies one update's effort command.

Args: prim_path: USD path of the articulation root. mode: One of position, velocity, or effort. values: Command values ordered by the selected joints. joint_names: Optional exact, case-sensitive joint subset. joint_indices: Optional DOF-index subset; mutually exclusive with joint_names.

Inputs: mode (required), values (required), prim_path (required), command_id, +3 more. Returns: result

set_joint_drive_configA

Atomically update drive gains, limits, or drive type for selected joints.

The timeline must be stopped. Numeric fields must be finite and non-negative. Values use runtime SI units: revolute gains are per radian, revolute velocity is radians_per_second, and prismatic velocity is meters_per_second. drive_type is force or acceleration. All inputs are validated before writing, and an apply failure triggers rollback.

Args: prim_path: USD path of the articulation root. stiffness: Optional proportional gain applied to every selected joint. damping: Optional derivative gain applied to every selected joint. max_force: Optional maximum drive effort. max_velocity: Optional maximum joint velocity; PhysX only. drive_type: Optional force or acceleration drive mode. joint_names: Optional exact, case-sensitive joint subset. joint_indices: Optional DOF-index subset; mutually exclusive with joint_names.

Inputs: prim_path (required), damping, max_force, stiffness, +6 more. Returns: result

set_joint_positionsA

Set target joint positions on a robot via ArticulationAction.

Units: radians for revolute joints, meters for prismatic joints (e.g. gripper fingers). Use get_robot_info to discover joint names, types, and limits first. After calling this, use step_simulation to advance and observe the result — do not use play_simulation + sleep.

Args: prim_path: The prim path of the robot. joint_positions: List of target joint position values. joint_indices: Optional list of joint indices to set. Sets all joints if not provided.

Inputs: prim_path (required), joint_positions (required), command_id, joint_indices, +1 more. Returns: result

set_mobile_base_velocityA

Apply a profiled differential or holonomic base velocity target.

The timeline must be playing for a non-zero command. Targets persist until replaced, so always call stop_mobile_base at the end of motion. Differential profiles reject non-zero lateral velocity.

Inputs: profile (required), prim_path (required), forward_mps (required), yaw_radps, +3 more. Returns: result

set_physics_paramsA

Atomically configure Isaac Sim 6 PhysX scene parameters.

The timeline must be stopped. time_step must be within [0.0001, 1.0] seconds and map to an integer PhysX steps-per-second value (for example, 1/60 or 1/120). Isaac Sim's Stage time codes and minimum simulation frame-rate are synchronized to that rate. Enabling GPU physics selects GPU dynamics and GPU broadphase; disabling it selects MBP broadphase. This does not change the launcher-controlled physics GPU ordinal. Success includes both USD and runtime read-back; failed application is rolled back as one transaction.

Args: gravity: Gravity vector [x, y, z]. time_step: Physics time step in seconds. gpu_enabled: Enable or disable GPU dynamics and matching broadphase.

Inputs: gravity, time_step, command_id, gpu_enabled, +1 more. Returns: result

set_semantic_labelsA

Preview or apply Isaac Sim 6.0.1 UsdSemantics LabelsAPI labels under one taxonomy with read-back.

Inputs: labels (required), taxonomy (required), prim_path (required), preview, +3 more. Returns: result

set_typed_attributeA

Preview or set a finite, explicitly typed USD attribute. Existing attributes require overwrite=true and cannot change type.

Inputs: value (required), attribute (required), prim_path (required), type_name (required), +5 more. Returns: result

set_variant_selectionB

Preview or select an existing USD variant after validating the set and available names.

Inputs: prim_path (required), selection (required), variant_set (required), preview, +2 more. Returns: result

spawn_humanA

Spawn animated NVIDIA human characters into the CURRENT USD stage.

Uses Isaac Sim 6.0+'s isaacsim.replicator.agent.core (IRA 1.x) character loader and SimReady assets. Unlike IRA.setup_simulation(), this tool does not reopen the environment USD, so existing scene edits are preserved.

A NavMesh is required. The tool can bake an authored volume and waits up to 2000 update frames for complex factory stages. The extension is enabled automatically, but the tool fails closed when no usable NavMesh or character assets are available. Stop the timeline first.

execute_script remains useful for one-off IRA experiments. reload_script is the better companion for reusable interaction or event logic after characters have been spawned.

Args: count: Number of new characters to add, from 1 to 25. group_name: USD-safe IRA group name. Reusing it appends characters. root_prim_path: Parent prim for all characters. asset_path: Character asset directory. Defaults to Isaac/People/Characters/. motion_library_path: Human motion-library USD. Defaults to the IRA extension setting / Isaac Sim asset root. behavior: wander, patrol, stop, or manual. Use manual when interaction scripts will issue Behavior Agent tasks directly. spawn_areas: Optional NavMesh area names used for spawning. speed_range: Two walking speeds in m/s. distance_range: Two wander-leg distances in meters. idle_time_range: Two idle/stop durations in seconds. patrol_points: Required for patrol; reachable NavMesh points as [[x,y,z], ...]. position: Optional exact world position for one spawned character. The point should lie on the NavMesh. rotation: Optional [rx, ry, rz] degrees; requires count=1. auto_create_navmesh_volume: Create one include-volume when the stage has none. This modifies the current stage and requires navmesh_volume_size. navmesh_volume_center: Center of the new NavMesh volume. Defaults to [0, 0, 1]. navmesh_volume_size: Full [x, y, z] size in stage units. Make it cover the walkable floor and surrounding obstacles. seed: Deterministic 32-bit IRA randomization seed.

Inputs: seed, count, behavior, position, +15 more. Returns: result

spawn_nvidia_assetA

Spawn one catalog asset into the CURRENT USD stage.

Call list_nvidia_assets first to obtain an exact asset_key. Robot keys create controllable articulations through the existing robot handler. Props, conveyors, warehouse equipment, and vegetation are referenced from verified NVIDIA USD paths and remain compatible with transform_object, clone_object, and delete_object.

Args: asset_key: Exact key returned by list_nvidia_assets. prim_path: Optional absolute destination prim path. A unique path under /World/Assets is generated when omitted. position: Optional world [x, y, z]. rotation: Optional XYZ Euler degrees [rx, ry, rz]. scale: Optional [sx, sy, sz] multiplier.

Inputs: asset_key (required), scale, position, rotation, +3 more. Returns: result

start_jobB

Start an eligible asset or sensor command as a bounded background job.

Motion and SDG starts retain their typed tools; their returned IDs can still be queried and cancelled through the unified job tools.

Inputs: command_type (required), params, command_id, deadline_ms, +1 more. Returns: result

start_sdg_jobC

Preview or asynchronously start one configured SDG job.

Inputs: job_id (required), preview, command_id, idempotency_key. Returns: result

step_simulationA

Advance the simulation by exactly N physics frames on a FROZEN timeline.

step is self-contained: it initialises physics on first call and operates on a paused/stopped timeline, so N is always exact and observations correlate to a known frame count.

Do NOT call play_simulation before or during the debug loop; step is for a frozen timeline. If the timeline is already playing, step returns an error (a free run cannot be counted frame-by-frame). Use play_simulation ONLY for a final continuous run / ScriptNode-driven demo, never for debugging.

Typical debug loop (no play):

  1. set_joint_positions to command the robot

  2. step_simulation with observe_prims and observe_joints

  3. get_joint_config if drives are not tracking correctly

  4. get_physics_state if objects are not behaving as expected

  5. Adjust and repeat

Args: num_steps: Number of simulation frames to step. observe_prims: List of prim paths to observe (returns position + velocity). observe_joints: List of articulation prim paths to observe (returns joint positions).

Inputs: num_steps, command_id, observe_prims, observe_joints, +1 more. Returns: result

stop_mobile_baseB

Set every profiled wheel velocity target to zero and verify the stop target read-back.

Inputs: profile (required), prim_path (required), command_id, idempotency_key. Returns: result

stop_simulationA

Stop the physics simulation and reset to spawn state.

Resets articulations and rigid bodies to their spawn pose (the state captured at first Play), like the Isaac UI Stop button. Call this to return the scene to a clean starting point before another run.

Inputs: command_id, idempotency_key. Returns: result

transform_objectA

Set the transform (position, rotation, scale) of an existing object.

Args: prim_path: The prim path of the object to transform. position: [x, y, z] new world position. rotation: [rx, ry, rz] new rotation in degrees. scale: [sx, sy, sz] new scale factors.

Inputs: prim_path (required), scale, position, rotation, +2 more. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    C2.9/5.0

    Scored across 129 tools

    Disambiguation3/5

    Most tools target distinct resources and actions, but several pairs overlap meaningfully: set_joint_positions vs set_joint_command(mode=position), get_joint_positions vs get_joint_state, and start_job vs start_sdg_job/execute_trajectory. The detailed descriptions help, but an agent could easily misselect between the joint and job-related tools.

    Naming Consistency4/5

    The overwhelming majority follow a clear lowercase snake_case verb_noun pattern (create_camera, get_prim_info, delete_sdg_job). Minor deviations like new_stage, save_stage_as, cleanup_artifacts, and get_capabilities break the uniformity slightly, but the convention is otherwise predictable and consistent.

    Tool Count1/5

    129 tools is far beyond the 50+ extreme threshold for any server, even one covering a broad domain like Isaac Sim. The count creates a massive surface that would overwhelm an agent, and many subsystems could be consolidated or exposed via narrower interfaces.

    Completeness4/5

    The toolset is remarkably broad, covering scene, physics, robots, sensors, materials, lights, SDG, ROS2, humans, action graphs, scripting, and asset workflows. Minor gaps exist—no delete_material, delete_light, modify_material, or explicit environment teardown—but most lifecycle paths are covered and workarounds are plausible.

    Maintenance

    ActivityStale
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI assistants to read, inspect, and manipulate OpenUSD 3D scenes, including scene graph traversal, property querying, mesh export, and variant switching.
      10
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      Enables natural language control of NVIDIA Isaac Sim through the Model Context Protocol, allowing you to create robots, build scenes, run simulations, and debug physics from any MCP-compatible IDE.
      42
      58 PyPI
      70
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI tools like Windsurf and Claude to control NVIDIA Isaac Sim and Isaac Lab through natural language, providing tools for scene inspection, prim management, physics simulation, and robot spawning.
      8
      Apache 2.0
    • A
      license
      A
      quality
      B
      maintenance
      Lets an AI assistant work in NVIDIA Isaac Sim and other Omniverse Kit apps: inspect and edit USD scenes, capture viewports with segmentation, run physics, drive robots and record. A Python MCP server on PyPI, with a Kit extension it reaches over a token-checked loopback link.
      44
      209 PyPI
      MIT