ui_describe_point
Identifies the accessibility element at specific coordinates on the iOS Simulator screen, enabling precise UI element inspection for testing and debugging purposes.
Instructions
Returns the accessibility element at given co-ordinates on the iOS Simulator's screen
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| udid | No | Udid of target, can also be set with the IDB_UDID env var | |
| x | Yes | The x-coordinate | |
| y | Yes | The y-coordinate |
Implementation Reference
- src/index.ts:470-507 (handler)The asynchronous handler function for the 'ui_describe_point' tool. It resolves the device UDID if not provided, executes the 'idb ui describe-point' command with the given x and y coordinates, returns the JSON output on success, or an error message on failure.async ({ udid, x, y }) => { try { const actualUdid = await getBootedDeviceId(udid); const { stdout, stderr } = await idb( "ui", "describe-point", "--udid", actualUdid, "--json", // When passing user-provided values to a command, it's crucial to use `--` // to separate the command's options from positional arguments. // This prevents the shell from misinterpreting the arguments as options. "--", String(x), String(y) ); if (stderr) throw new Error(stderr); return { isError: false, content: [{ type: "text", text: stdout }], }; } catch (error) { return { isError: true, content: [ { type: "text", text: errorWithTroubleshooting( `Error describing point (${x}, ${y}): ${toError(error).message}` ), }, ], }; } }
- src/index.ts:461-469 (schema)Zod input schema for the tool, validating optional UDID (UUID regex), and required numeric x and y coordinates.{ udid: z .string() .regex(UDID_REGEX) .optional() .describe("Udid of target, can also be set with the IDB_UDID env var"), x: z.number().describe("The x-coordinate"), y: z.number().describe("The y-coordinate"), },
- src/index.ts:457-509 (registration)The registration of the 'ui_describe_point' tool using server.tool(), conditionally excluding if filtered via environment variable. Includes description, schema, and handler.if (!isToolFiltered("ui_describe_point")) { server.tool( "ui_describe_point", "Returns the accessibility element at given co-ordinates on the iOS Simulator's screen", { udid: z .string() .regex(UDID_REGEX) .optional() .describe("Udid of target, can also be set with the IDB_UDID env var"), x: z.number().describe("The x-coordinate"), y: z.number().describe("The y-coordinate"), }, async ({ udid, x, y }) => { try { const actualUdid = await getBootedDeviceId(udid); const { stdout, stderr } = await idb( "ui", "describe-point", "--udid", actualUdid, "--json", // When passing user-provided values to a command, it's crucial to use `--` // to separate the command's options from positional arguments. // This prevents the shell from misinterpreting the arguments as options. "--", String(x), String(y) ); if (stderr) throw new Error(stderr); return { isError: false, content: [{ type: "text", text: stdout }], }; } catch (error) { return { isError: true, content: [ { type: "text", text: errorWithTroubleshooting( `Error describing point (${x}, ${y}): ${toError(error).message}` ), }, ], }; } } ); }