mobile_swipe_on_screen
Perform swipe gestures on mobile device screens to navigate apps, scroll content, or trigger actions by specifying direction and coordinates.
Instructions
Swipe on the screen
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| device | Yes | The device identifier to use. Use mobile_list_available_devices to find which devices are available to you. | |
| direction | Yes | The direction to swipe | |
| x | No | The x coordinate to start the swipe from, in pixels. If not provided, uses center of screen | |
| y | No | The y coordinate to start the swipe from, in pixels. If not provided, uses center of screen | |
| distance | No | The distance to swipe in pixels. Defaults to 400 pixels for iOS or 30% of screen dimension for Android |
Implementation Reference
- src/server.ts:349-362 (handler)The handler function for the swipe_on_screen tool. It requires a device to be selected, then calls the appropriate swipe method on the Robot instance based on whether coordinates are provided.async ({ direction, x, y, distance }) => { requireRobot(); if (x !== undefined && y !== undefined) { // Use coordinate-based swipe await robot!.swipeFromCoordinate(x, y, direction, distance); const distanceText = distance ? ` ${distance} pixels` : ""; return `Swiped ${direction}${distanceText} from coordinates: ${x}, ${y}`; } else { // Use center-based swipe await robot!.swipe(direction); return `Swiped ${direction} on screen`; } }
- src/server.ts:343-348 (schema)Zod input schema for the swipe_on_screen tool parameters: required direction, optional x, y, distance.{ direction: z.enum(["up", "down", "left", "right"]).describe("The direction to swipe"), x: z.number().optional().describe("The x coordinate to start the swipe from, in pixels. If not provided, uses center of screen"), y: z.number().optional().describe("The y coordinate to start the swipe from, in pixels. If not provided, uses center of screen"), distance: z.number().optional().describe("The distance to swipe in pixels. Defaults to 400 pixels for iOS or 30% of screen dimension for Android"), },
- src/server.ts:341-363 (registration)Registration of the 'swipe_on_screen' tool (note: queried name was 'mobile_swipe_on_screen' but this is the matching swipe tool) using the custom 'tool' helper which calls server.tool."swipe_on_screen", "Swipe on the screen", { direction: z.enum(["up", "down", "left", "right"]).describe("The direction to swipe"), x: z.number().optional().describe("The x coordinate to start the swipe from, in pixels. If not provided, uses center of screen"), y: z.number().optional().describe("The y coordinate to start the swipe from, in pixels. If not provided, uses center of screen"), distance: z.number().optional().describe("The distance to swipe in pixels. Defaults to 400 pixels for iOS or 30% of screen dimension for Android"), }, async ({ direction, x, y, distance }) => { requireRobot(); if (x !== undefined && y !== undefined) { // Use coordinate-based swipe await robot!.swipeFromCoordinate(x, y, direction, distance); const distanceText = distance ? ` ${distance} pixels` : ""; return `Swiped ${direction}${distanceText} from coordinates: ${x}, ${y}`; } else { // Use center-based swipe await robot!.swipe(direction); return `Swiped ${direction} on screen`; } } );
- src/robot.ts:54-62 (helper)Robot interface defines the swipe(direction) and swipeFromCoordinate(x,y,direction,distance?) methods called by the tool handler.* Swipe in a direction. */ swipe(direction: SwipeDirection): Promise<void>; /** * Swipe from a specific coordinate in a direction. */ swipeFromCoordinate(x: number, y: number, direction: SwipeDirection, distance?: number): Promise<void>;
- src/android.ts:129-161 (helper)Implementation of swipe(direction) in AndroidRobot class, calculates start/end coordinates based on direction and screen size, executes via adb shell input swipe.public async swipe(direction: SwipeDirection): Promise<void> { const screenSize = await this.getScreenSize(); const centerX = screenSize.width >> 1; let x0: number, y0: number, x1: number, y1: number; switch (direction) { case "up": x0 = x1 = centerX; y0 = Math.floor(screenSize.height * 0.80); y1 = Math.floor(screenSize.height * 0.20); break; case "down": x0 = x1 = centerX; y0 = Math.floor(screenSize.height * 0.20); y1 = Math.floor(screenSize.height * 0.80); break; case "left": x0 = Math.floor(screenSize.width * 0.80); x1 = Math.floor(screenSize.width * 0.20); y0 = y1 = Math.floor(screenSize.height * 0.50); break; case "right": x0 = Math.floor(screenSize.width * 0.20); x1 = Math.floor(screenSize.width * 0.80); y0 = y1 = Math.floor(screenSize.height * 0.50); break; default: throw new ActionableError(`Swipe direction "${direction}" is not supported`); } this.adb("shell", "input", "swipe", `${x0}`, `${y0}`, `${x1}`, `${y1}`, "1000"); }