Click Screen
mobile_click_on_screen_at_coordinatesDestructive
Simulate screen taps at specific pixel coordinates on mobile devices for automated testing and interaction with iOS and Android applications.
Instructions
Click on the screen at given x,y coordinates. If clicking on an element, use the list_elements_on_screen tool to find the coordinates.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| device | Yes | The device identifier to use. Use mobile_list_available_devices to find which devices are available to you. | |
| x | Yes | The x coordinate to click on the screen, in pixels | |
| y | Yes | The y coordinate to click on the screen, in pixels |
Implementation Reference
- src/server.ts:340-354 (registration)Tool registration including schema definition (device, x, y) and thin handler that delegates to Robot.tap via getRobotFromDevice.
tool( "mobile_click_on_screen_at_coordinates", "Click Screen", "Click on the screen at given x,y coordinates. If clicking on an element, use the list_elements_on_screen tool to find the coordinates.", { device: z.string().describe("The device identifier to use. Use mobile_list_available_devices to find which devices are available to you."), x: z.number().describe("The x coordinate to click on the screen, in pixels"), y: z.number().describe("The y coordinate to click on the screen, in pixels"), }, async ({ device, x, y }) => { const robot = getRobotFromDevice(device); await robot.tap(x, y); return `Clicked on screen at coordinates: ${x}, ${y}`; } ); - src/server.ts:142-179 (helper)Helper function that selects and returns the appropriate Robot implementation (AndroidRobot, IosRobot, or MobileDevice) based on the device ID.
const getRobotFromDevice = (deviceId: string): Robot => { // from now on, we must have mobilecli working ensureMobilecliAvailable(); // Check if it's an iOS device const iosManager = new IosManager(); const iosDevices = iosManager.listDevices(); const iosDevice = iosDevices.find(d => d.deviceId === deviceId); if (iosDevice) { return new IosRobot(deviceId); } // Check if it's an Android device const androidManager = new AndroidDeviceManager(); const androidDevices = androidManager.getConnectedDevices(); const androidDevice = androidDevices.find(d => d.deviceId === deviceId); if (androidDevice) { return new AndroidRobot(deviceId); } // Check if it's a simulator (will later replace all other device types as well) const response = mobilecli.getDevices({ platform: "ios", type: "simulator", includeOffline: false, }); if (response.status === "ok" && response.data && response.data.devices) { for (const device of response.data.devices) { if (device.id === deviceId) { return new MobileDevice(deviceId); } } } throw new ActionableError(`Device "${deviceId}" not found. Use the mobile_list_available_devices tool to see available devices.`); }; - src/android.ts:437-439 (handler)Platform-specific handler for tap in AndroidRobot using ADB shell input tap command.
public async tap(x: number, y: number): Promise<void> { this.adb("shell", "input", "tap", `${x}`, `${y}`); } - src/ios.ts:189-192 (handler)Platform-specific handler for tap in IosRobot delegating to WebDriverAgent.
public async tap(x: number, y: number): Promise<void> { const wda = await this.wda(); await wda.tap(x, y); } - src/mobile-device.ts:180-182 (handler)Platform-specific handler for tap in MobileDevice using mobilecli runCommand.
public async tap(x: number, y: number): Promise<void> { this.runCommand(["io", "tap", `${x},${y}`]); }