ui_describe_point

Instructions

Implementation Reference

• src/index.ts:531-569 (handler)

  The async handler function that executes the tool logic. It resolves the booted device UDID (via getBootedDeviceId), calls `idb ui describe-point --udid <udid> --json -- <x> <y>`, and returns the accessibility element at the given coordinates. On error, returns a user-friendly troubleshooting message.

    async ({ udid, x, y }) => {
      try {
        const actualUdid = await getBootedDeviceId(udid);
  
        const { stdout, stderr } = await idb(
          "ui",
          "describe-point",
          "--udid",
          actualUdid,
          "--json",
          // When passing user-provided values to a command, it's crucial to use `--`
          // to separate the command's options from positional arguments.
          // This prevents the shell from misinterpreting the arguments as options.
          "--",
          String(x),
          String(y)
        );
  
        if (stderr) throw new Error(stderr);
  
        return {
          isError: false,
          content: [{ type: "text", text: stdout }],
        };
      } catch (error) {
        return {
          isError: true,
          content: [
            {
              type: "text",
              text: errorWithTroubleshooting(
                `Error describing point (${x}, ${y}): ${toError(error).message}`
              ),
            },
          ],
        };
      }
    }
  );

• src/index.ts:521-529 (schema)

  Input schema defined using Zod for the tool. Defines three parameters: `udid` (optional, UUID format), `x` (number, the x-coordinate), and `y` (number, the y-coordinate).

  {
    udid: z
      .string()
      .regex(UDID_REGEX)
      .optional()
      .describe("Udid of target, can also be set with the IDB_UDID env var"),
    x: z.number().describe("The x-coordinate"),
    y: z.number().describe("The y-coordinate"),
  },

• src/index.ts:517-570 (registration)

  Registration of the tool via `server.tool()` with the name "ui_describe_point", a description, the Zod schema, metadata options (title, readOnlyHint, openWorldHint), and the handler function. Wrapped in a conditional check `!isToolFiltered("ui_describe_point")` to allow disabling via the IOS_SIMULATOR_MCP_FILTERED_TOOLS environment variable.

  if (!isToolFiltered("ui_describe_point")) {
    server.tool(
      "ui_describe_point",
      "Returns the accessibility element at given co-ordinates on the iOS Simulator's screen",
      {
        udid: z
          .string()
          .regex(UDID_REGEX)
          .optional()
          .describe("Udid of target, can also be set with the IDB_UDID env var"),
        x: z.number().describe("The x-coordinate"),
        y: z.number().describe("The y-coordinate"),
      },
      { title: "Describe UI Point", readOnlyHint: true, openWorldHint: true },
      async ({ udid, x, y }) => {
        try {
          const actualUdid = await getBootedDeviceId(udid);
  
          const { stdout, stderr } = await idb(
            "ui",
            "describe-point",
            "--udid",
            actualUdid,
            "--json",
            // When passing user-provided values to a command, it's crucial to use `--`
            // to separate the command's options from positional arguments.
            // This prevents the shell from misinterpreting the arguments as options.
            "--",
            String(x),
            String(y)
          );
  
          if (stderr) throw new Error(stderr);
  
          return {
            isError: false,
            content: [{ type: "text", text: stdout }],
          };
        } catch (error) {
          return {
            isError: true,
            content: [
              {
                type: "text",
                text: errorWithTroubleshooting(
                  `Error describing point (${x}, ${y}): ${toError(error).message}`
                ),
              },
            ],
          };
        }
      }
    );
  }

• src/index.ts:132-141 (helper)

  The `isToolFiltered` helper function reads from the IOS_SIMULATOR_MCP_FILTERED_TOOLS environment variable and checks if the tool name is in the filter list, allowing selective disabling of tools.

  // Read filtered tools from environment variable
  const FILTERED_TOOLS =
    process.env.IOS_SIMULATOR_MCP_FILTERED_TOOLS?.split(",").map((tool) =>
      tool.trim()
    ) || [];
  
  // Function to check if a tool is filtered
  function isToolFiltered(toolName: string): boolean {
    return FILTERED_TOOLS.includes(toolName);
  }

• src/index.ts:122-130 (helper)

  The `idb` helper function wraps the `run` function to execute the idb binary with the provided arguments. It's used by the handler to call the `ui describe-point` subcommand.

  /**
   * Runs the idb command with the given arguments
   * @param args - arguments to pass to the idb command
   * @returns The stdout and stderr of the command
   * @see https://fbidb.io/docs/commands for documentation of available idb commands
   */
  async function idb(...args: string[]) {
    return run(getIdbPath(), args);
  }