ui_describe_point
Identify accessibility elements at specific screen coordinates in iOS Simulator to inspect UI components and verify interface behavior during development.
Instructions
Returns the accessibility element at given co-ordinates on the iOS Simulator's screen
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| udid | No | Udid of target, can also be set with the IDB_UDID env var | |
| x | Yes | The x-coordinate | |
| y | Yes | The y-coordinate |
Implementation Reference
- src/index.ts:470-507 (handler)Handler function that executes the tool: resolves UDID if needed, runs 'idb ui describe-point' with x,y coordinates, returns JSON output or error.async ({ udid, x, y }) => { try { const actualUdid = await getBootedDeviceId(udid); const { stdout, stderr } = await idb( "ui", "describe-point", "--udid", actualUdid, "--json", // When passing user-provided values to a command, it's crucial to use `--` // to separate the command's options from positional arguments. // This prevents the shell from misinterpreting the arguments as options. "--", String(x), String(y) ); if (stderr) throw new Error(stderr); return { isError: false, content: [{ type: "text", text: stdout }], }; } catch (error) { return { isError: true, content: [ { type: "text", text: errorWithTroubleshooting( `Error describing point (${x}, ${y}): ${toError(error).message}` ), }, ], }; } }
- src/index.ts:461-469 (schema)Zod input schema defining optional udid (validated UDID regex), required x and y numbers.{ udid: z .string() .regex(UDID_REGEX) .optional() .describe("Udid of target, can also be set with the IDB_UDID env var"), x: z.number().describe("The x-coordinate"), y: z.number().describe("The y-coordinate"), },
- src/index.ts:458-508 (registration)MCP server tool registration for 'ui_describe_point', conditional on not filtered, with description, schema, and handler.server.tool( "ui_describe_point", "Returns the accessibility element at given co-ordinates on the iOS Simulator's screen", { udid: z .string() .regex(UDID_REGEX) .optional() .describe("Udid of target, can also be set with the IDB_UDID env var"), x: z.number().describe("The x-coordinate"), y: z.number().describe("The y-coordinate"), }, async ({ udid, x, y }) => { try { const actualUdid = await getBootedDeviceId(udid); const { stdout, stderr } = await idb( "ui", "describe-point", "--udid", actualUdid, "--json", // When passing user-provided values to a command, it's crucial to use `--` // to separate the command's options from positional arguments. // This prevents the shell from misinterpreting the arguments as options. "--", String(x), String(y) ); if (stderr) throw new Error(stderr); return { isError: false, content: [{ type: "text", text: stdout }], }; } catch (error) { return { isError: true, content: [ { type: "text", text: errorWithTroubleshooting( `Error describing point (${x}, ${y}): ${toError(error).message}` ), }, ], }; } } );
- src/index.ts:141-154 (helper)Helper function used by the handler to resolve the target device UDID, falling back to booted simulator if not provided.async function getBootedDeviceId( deviceId: string | undefined ): Promise<string> { // If deviceId not provided, get the currently booted simulator let actualDeviceId = deviceId; if (!actualDeviceId) { const { id } = await getBootedDevice(); actualDeviceId = id; } if (!actualDeviceId) { throw new Error("No booted simulator found and no deviceId provided"); } return actualDeviceId; }