Ping the Isaac Sim extension server and return scene information including stage path, assets root, and prim count.

Create a physics scene, adding a ground plane only if the stage lacks one. Call get_scene_info first to verify connection.

A loaded environment usually brings its own collision floor, and this does not add a second one on top of it — provided load_environment ran BEFORE this call. The check happens once, here: call this first and the environment's floor arrives afterwards, leaving two collision floors with the engine deciding which one objects land on. load_environment reports that case as "collision_floor_warning". The response reports "ground_plane" — the floor objects will actually land on — and "ground_plane_created", false when the stage already had one. Read the floor's height from that prim rather than assuming z=0; an environment's floor is not always at the origin.

The check recognises a collision-enabled prim of type Plane, which is what the shipped environments author. An environment whose floor is a Mesh is NOT recognised, so a second plane is added and two collision floors end up on the stage — which one wins is the physics engine's decision. When "ground_plane_created" is true after loading an environment, verify the floor before placing anything on it.

Args: gravity: Gravity vector [x, y, z]. Default is standard gravity. scene_name: Name for the physics scene prim.

Remove all prims from the scene.

Also empties any environment loaded by load_environment, which removes that environment's collision floor along with it — so a later create_physics_scene finds no floor and supplies its own. The stage's defaultLight is always kept — a stage with no light renders black, which looks like a broken camera.

Args: keep_physics: If True, keep physics scene prims. keep_environment: If True, keep the loaded environment. Reloading one costs seconds, so pass this when clearing objects between attempts.

List the prims directly under root_path, optionally filtered by type.

One level deep by default, so list_prims("/") names /World and /Environment rather than everything inside them — a robot alone is hundreds of prims. The response echoes recursive so a shallow answer is never mistaken for a complete one.

Pass recursive=True to walk the whole subtree. That is the one you want when checking whether something was really deleted, or when hunting a prim nested under a robot: a Camera at /World/Arm/EyeCam does not appear in a shallow listing of /World.

Args: root_path: Root path to start listing from. prim_type: Filter by prim type (e.g. "Mesh", "Xform"). With recursive=True, non-matching prims are still descended into, so a Camera under an Xform is found. recursive: Walk the entire subtree instead of one level.

Get detailed information about a specific prim.

Returns type, children, and a transform block. Position is reported in both frames, under explicit names — there is no bare "position": position_local — parent-relative, the value transform_object writes. position_world — where the prim actually is on the stage. Use this to reason about distances, reach, or contact. For a robot link such as /World/Franka/fr3_hand_tcp the two differ by the robot's own pose. position_world_source is "usd" (derived from the authored transform) or "physics" (measured, on Newton). On Newton a body that has been simulated may carry position_warning saying both values are its spawn pose; read it through get_physics_state instead.

Also returns rotation [rx, ry, rz] in degrees (XYZ order, the same convention transform_object accepts) and scale — both local, like position_local. For geometric prims (Cube, Sphere, Cylinder, Cone, Capsule), also returns actual_size [x, y, z] in meters accounting for scale and default primitive dimensions (world-space, like position_world).

Args: prim_path: The USD prim path to inspect.

List all available environments discovered from the Isaac Sim asset server. Includes warehouses, offices, outdoor scenes, and more.

Load a pre-built environment into the scene. Supports fuzzy matching. Call list_environments first to see available options.

Many shipped environments are authored Y-up and/or in centimeters; those are rotated and rescaled to match the stage, and the response reports what was applied under "corrections". Read prim_path from the response rather than assuming it — it defaults to a named child of /Environment.

"bounds" carries two different heights, so use the right one: floor_height — the surface objects rest on, measured from the environment's collision floor. Place with position=[x, y, floor_height]. bounds_min_z — the lowest authored geometry (trim, a recessed drain, a sunk prop). Not a placement height. floor_height_source says which was used. When it reads "bounds_min_z" no collision floor could be measured and floor_height is a fallback that may be below the real surface — floor_height_warning explains it.

Args: environment: Environment name or search term (e.g. "warehouse", "hospital", "office"). prim_path: Prim path for the loaded environment. Defaults to /Environment/, which keeps it separate from the stage's default lighting and lets clear_scene remove it.

Create a primitive object (Cube, Sphere, Cylinder, Cone, Capsule, Plane).

Prefer size for absolute sizing: size is the target in METERS (default 1.0), so size=0.3 gives a 0.3 m object regardless of type.

scale is a RAW MULTIPLIER of the primitive's NATIVE size, not meters. Native sizes: Cube/Sphere/Cylinder/Cone/Capsule = 2 m, Plane = 1 m. So scale=0.5 on a Cube -> 1 m, and scale=[0.4,0.4,0.3] -> a 0.8 x 0.8 x 0.6 m box (0.4 * 2 m), which surprises callers who expect 0.4 m. Use scale only for deliberate non-uniform shaping; otherwise use size. If both are given, scale wins and size is ignored.

For the geometric prims (Cube, Sphere, Cylinder, Cone, Capsule) this returns prim_path, actual_size [x, y, z] in meters, and bounding_box (min/max corners in world coordinates) so you can accurately place other objects relative to this one. A Plane has no such extent and returns prim_path only.

Args: object_type: Type of primitive — Cube, Sphere, Cylinder, Cone, Capsule, or Plane (case-insensitive; "cube" is normalized to "Cube"). position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. scale: [sx, sy, sz] RAW multiplier of the native size (2 m for most prims, 1 m for Plane). NOT meters. Overrides size. size: Target size in METERS (default 1.0). Absolute; independent of the primitive's native size. Ignored if scale is provided. color: [r, g, b] color values (0-1). physics_enabled: Enable physics on this object. prim_path: Custom prim path. Auto-generated if not provided.

Delete an object from the scene.

Args: prim_path: The prim path of the object to delete.

Set the transform (position, rotation, scale) of an existing object.

Args: prim_path: The prim path of the object to transform. position: [x, y, z] new world position. rotation: [rx, ry, rz] new rotation in degrees. scale: [sx, sy, sz] new scale factors.

Duplicate an existing object to a new prim path.

Args: source_path: Prim path of the object to clone. target_path: Prim path for the cloned object. position: [x, y, z] position for the clone. Keeps original position if not set.

Create a light in the scene.

Args: light_type: Type of light — DistantLight, DomeLight, SphereLight, RectLight, DiskLight, or CylinderLight. position: [x, y, z] world position. intensity: Light intensity. color: [r, g, b] light color (0-1). rotation: [rx, ry, rz] rotation in degrees. prim_path: Custom prim path. Auto-generated if not provided.

Modify properties of an existing light.

Args: prim_path: The prim path of the light to modify. intensity: New intensity value. color: [r, g, b] new light color (0-1).

Create a robot in the scene from the Isaac Sim asset library.

Supports fuzzy matching — e.g. "franka", "spot", "g1", "go1". Call list_available_robots first to see all available robots. Call create_physics_scene before creating robots.

Returns prim_path, robot_key, joint_names, and num_dof so you can immediately use set_joint_positions without a follow-up get_robot_info call.

Args: robot_type: Robot name or search term. Fuzzy matched against available robots. position: [x, y, z] world position. name: Custom name for the robot prim. prim_path: Exact USD prim path (e.g. "/World/Franka"). Overrides name-based path.

List all available robots discovered from the Isaac Sim asset server. Returns robot keys, descriptions, manufacturers, and asset paths. The list is auto-discovered at startup and reflects the actual assets available in your Isaac Sim version.

Force re-scan the asset server for available robots. Use this if new robot assets were added.

Get robot joint information including names, DOF count, joint types, and limits.

Call this after create_robot to understand the robot's kinematic structure. Returns joint names ordered by DOF index, joint types (revolute/prismatic), and joint limits (radians for revolute, meters for prismatic — each entry carries its own units).

Args: prim_path: The prim path of the robot.

Set target joint positions on a robot via ArticulationAction.

Units: radians for revolute joints, meters for prismatic joints (e.g. gripper fingers). Use get_robot_info to discover joint names, types, and limits first. After calling this, use step_simulation to advance and observe the result — do not use play_simulation + sleep.

Args: prim_path: The prim path of the robot. joint_positions: List of target joint position values. joint_indices: Optional list of joint indices to set. Sets all joints if not provided.

Read current joint positions from a robot.

Units: radians for revolute joints, meters for prismatic joints. Joint order matches the joint_names from get_robot_info. For a combined step-and-read, prefer step_simulation with observe_joints.

Args: prim_path: The prim path of the robot.

Add a camera sensor to the scene.

Prefer target= over rotation= for aiming: cameras look down their local -Z and carry a built-in orientation, so hand-computed euler angles are easy to get wrong and give you a picture of the sky. The response echoes the rotation that was applied under "rotation" and the point under "aimed_at".

Args: prim_path: Prim path for the camera. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. Ignored if target is given. resolution: [width, height] image resolution. Default 1280x720. target: [x, y, z] world point to look at, using +Z as up. Needs a position — either passed here or already on the prim.

Capture an RGB image from a camera sensor.

Args: prim_path: Prim path of the camera. output_path: File path to save the image. Returns metadata only if not set.

Add a lidar sensor to the scene.

Args: prim_path: Prim path for the lidar. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. config: Lidar configuration name (e.g. "Example_Rotary").

Get point cloud data from a lidar sensor.

Requires the timeline to be playing — RTX lidar data is produced by Replicator while the sim runs, and a sweep only completes on some frames, so an empty read means "not this frame", not "saw nothing".

By default returns a summary rather than the raw cloud: point_count, bounds, and the nearest hit. A full sweep is tens of thousands of points and megabytes of JSON, which is rarely what you want in a response.

Args: prim_path: Prim path of the lidar sensor. max_points: Include this many points in the response, sampled at an even stride across the sweep. Omit for summary only. output_path: Write the complete cloud to this .npy file and return its path; numpy.load() reads it back as an (N, 3) array.

Create a PBR or physics material.

Args: material_type: "pbr" for visual material or "physics" for physics material. prim_path: Prim path for the material. Auto-generated if not set. material_path: Alias for prim_path. apply_material names this argument material_path, and an unknown argument is dropped silently rather than rejected — so asking for material_path="/World/Looks/Red" used to succeed while creating the material somewhere else entirely, and the follow-up apply_material then failed on a path that was never used. color: [r, g, b] diffuse color (0-1). PBR only. roughness: Surface roughness (0-1). PBR only. metallic: Metallic value (0-1). PBR only.

Bind a material to an object.

Args: material_path: Prim path of the material. target_prim_path: Prim path of the object to apply the material to.

Import a robot from a URDF file into the scene.

Args: urdf_path: Path to the URDF file. prim_path: Prim path for the imported robot. position: [x, y, z] world position.

Load a USD asset from a URL or file path into the scene.

Args: usd_url: URL or local path to the USD file. prim_path: Prim path for the loaded asset. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Search the NVIDIA USD asset library by text description, then load the best match.

Args: text_prompt: Text description of the 3D asset to search for. target_path: Prim path for the loaded result. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Generate a 3D model from text or image using Beaver3D, then load it into the scene.

Args: text_prompt: Text description for 3D generation. image_url: URL of an image for 3D generation. position: [x, y, z] world position for the generated model. scale: [sx, sy, sz] scale factors.

Start the physics simulation.

Pause the physics simulation.

Stop the physics simulation and reset to spawn state.

Resets articulations and rigid bodies to their spawn pose (the state captured at first Play), like the Isaac UI Stop button. Call this to return the scene to a clean starting point before another run.

Advance the simulation by exactly N physics frames on a FROZEN timeline.

step is self-contained: it initialises physics on first call and operates on a paused/stopped timeline, so N is always exact and observations correlate to a known frame count.

Do NOT call play_simulation before or during the debug loop; step is for a frozen timeline. If the timeline is already playing, step returns an error (a free run cannot be counted frame-by-frame). Use play_simulation ONLY for a final continuous run / ScriptNode-driven demo, never for debugging.

Typical debug loop (no play):

1. set_joint_positions to command the robot

2. step_simulation with observe_prims and observe_joints

3. get_joint_config if drives are not tracking correctly

4. get_physics_state if objects are not behaving as expected

5. Adjust and repeat

Args: num_steps: Number of simulation frames to step. observe_prims: List of prim paths to observe (returns position + velocity). observe_joints: List of articulation prim paths to observe (returns joint positions).

Configure physics engine parameters.

Args: gravity: Gravity vector [x, y, z]. time_step: Physics time step in seconds. gpu_enabled: Enable GPU-accelerated physics.

Diagnostic tool: recent WARN/ERROR logs plus captured print() output.

Captures carb.log_*/omni.log WARN+ERROR and stdout from execute_script / reload_script (tagged [PRINT]). Plain print() outside those captured contexts may not appear.

Defaults are agent-friendly: non-destructive (clear=False) and scoped to the current run (since_last_play=True) so you see logs from what you just did, not stale entries from previous runs.

Args: clear: If True, empty the buffer after reading. Default False. count: Maximum number of log entries to return. since_last_play: If True (default), return only entries since the last timeline Play. Set False for the full buffer.

Get the current simulation state: timeline status (playing/stopped/paused), simulation time, and physics dt. step_simulation does NOT require a running timeline — do not play just to step.

Diagnostic tool: get physics state for a prim.

Returns rigid body status, velocities, kinematic flag, and collision info. mass is included only when the prim carries a UsdPhysics MassAPI — objects created by create_object do not, and take their mass from the collider's density. Velocity units: linear_velocity in m/s, angular_velocity in rad/s. Velocities are only non-zero once the simulation has advanced — step the simulation (or play) before reading them. Call this when:

• Objects fall through the ground (check collision enabled)

• Objects don't move when expected (check is_kinematic, mass)

• Grasping fails (check collision on gripper fingers and target object)

Args: prim_path: USD path to the prim to inspect.

Diagnostic tool: get joint drive configuration for a robot articulation.

Returns stiffness, damping, limits, target vs actual positions, and position error for each joint. Call this when:

• Joint drives are not tracking targets (check position_error)

• Joints are oscillating or unstable (check stiffness/damping ratio)

• Joints hit limits unexpectedly (check lower_limit/upper_limit)

Units: gains are per-radian (angular) or per-meter (linear), per each joint's gain_units. USD stores angular gains per-degree — divide by 180/pi before writing one back via execute_script, or the drive lands 57.3x stiff, silently.

Args: prim_path: USD path to the robot articulation root.

Escape hatch: execute arbitrary Python code in Isaac Sim.

PREFER named tools over this for: reading/setting joints (set_joint_positions, get_joint_positions), inspecting state (get_prim_info, get_physics_state, get_joint_config), stepping simulation (step_simulation), and checking logs (get_isaac_logs).

USE this for: operations no named tool covers, such as creating Action Graphs, computing IK, setting up physics callbacks, or configuring advanced USD properties.

CAUTION: touching an articulation controlled by a running ScriptNode / Action Graph can silently break its control path (no error is raised). While a graph is running, read-only diagnostics (get_prim_info, get_physics_state, get_joint_positions, get_isaac_logs) are safe, but stop_simulation before using execute_script or named write tools on the same articulation.

For persistent controllers (>20 lines), write a .py file and load it with reload_script instead of pasting code here.

Args: code: Python code to execute in the Isaac Sim context. cwd: Optional working directory to add to sys.path before execution.

Reload a Python controller from a file on disk.

Two modes, chosen automatically:

• If any Action-Graph ScriptNode references this file (inputs:scriptPath), those ScriptNodes are force-recompiled so your on-disk edits take effect on the running graph. This is how you iterate on a ScriptNode controller.

• Otherwise the file is (re-)executed as a standalone controller, the way you would use execute_script for code longer than ~20 lines.

Workflow:

1. Write the controller as a .py file (attach via create_action_graph script_file=... for ScriptNode use)

2. reload_script to load / recompile it

3. step_simulation to debug (frozen timeline) or play for a ScriptNode demo

4. Edit the file and reload_script again to iterate

The file's directory is auto-added to sys.path.

Args: file_path: Path to the Python file on disk. script_file: Alias for file_path. create_action_graph names this argument script_file, and an unknown argument is dropped silently rather than rejected, so both spellings are accepted here instead of one of them quietly doing nothing. module_name: Optional module name to reload (e.g. 'my_controller').

Create and wire an OmniGraph Action Graph.

Builds a complete Action Graph with nodes, connections and attribute values using og.Controller.edit(). This is the programmatic equivalent of creating an Action Graph in the visual editor.

Args: graph_path: USD prim path for the graph (default "/World/ActionGraph"). nodes: List of node definitions. Each dict has: - "path": Node path relative to graph (e.g. "OnPlaybackTick") - "type": OmniGraph node type (e.g. "omni.graph.action.OnPlaybackTick") connections: List of [source_attr, target_attr] pairs for wiring nodes. Each attr is "NodePath.outputs:attrName" or "NodePath.inputs:attrName". values: List of attribute value overrides. Each dict has: - "attr": Full attribute path (e.g. "ScriptNode.inputs:script") - "value": The value to set evaluator: Graph evaluator type (default "execution", what Action Graphs use). "push" evaluates every application update regardless of the timeline, so an OnPlaybackTick-driven ScriptNode would keep running even while the simulation is stopped. script_file: Convenience shortcut — path to a local Python script file. When provided, automatically creates OnPlaybackTick → ScriptNode nodes, wires them, and attaches the script file (sets usePath + scriptPath). The nodes and connections parameters are ignored when script_file is set. RECOMMENDED for anything you will iterate on — edit the file and reload_script "just works", with the better reload story. inline_script: Convenience shortcut — inline Python (must define setup(db)/compute(db)). Auto-creates OnPlaybackTick → ScriptNode, wires them, and sets the script inline (usePath=False). For small, static graphs. For anything you will iterate on, prefer script_file — it has the better reload story (edit the file + reload_script "just works"; inline edits need edit_action_graph).

Example (inline script — one-step): create_action_graph( inline_script="def setup(db): pass\ndef compute(db): return True" )

Example (script file — one-step, recommended for iteration): create_action_graph( script_file="/path/to/controller.py" )

Edit an existing OmniGraph Action Graph: set attribute values or add connections.

Use this to update ScriptNode scripts (inline or file path), change attribute values, or add new connections on an already-created graph.

For ScriptNode with a local file script, set both usePath and scriptPath: values=[ {"attr": "ScriptNode.inputs:usePath", "value": true}, {"attr": "ScriptNode.inputs:scriptPath", "value": "/path/to/script.py"} ]

For ScriptNode with inline script: values=[ {"attr": "ScriptNode.inputs:usePath", "value": false}, {"attr": "ScriptNode.inputs:script", "value": "def compute(db): ..."} ]

Args: graph_path: USD prim path of the existing graph (default "/World/ActionGraph"). values: List of attribute value overrides. Each dict has: - "attr": Attribute path relative to graph (e.g. "ScriptNode.inputs:script") - "value": The value to set connections: List of [source_attr, target_attr] pairs to add.