imeAction
Execute IME actions like done, next, search, send, go, and previous on Android or iOS devices to automate mobile interactions efficiently.
Instructions
Perform an IME action (e.g., done, next, search)
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | IME action to perform | |
| platform | Yes | Platform of the device |
Implementation Reference
- src/features/action/ImeAction.ts:12-51 (handler)Core handler logic: executes the IME action using observedInteraction wrapper, calls private executeImeActionasync execute( action: "done" | "next" | "search" | "send" | "go" | "previous", progress?: ProgressCallback ): Promise<ImeActionResult> { // Validate action input if (!action) { return { success: false, action: "", error: "No IME action provided" }; } return this.observedInteraction( async () => { try { await this.executeImeAction(action); return { success: true, action }; } catch (error) { const errorMessage = error instanceof Error ? error.message : String(error); return { success: false, action, error: `Failed to execute IME action: ${errorMessage}` }; } }, { changeExpected: true, tolerancePercent: 0.00, timeoutMs: 3000, // IME actions should be quick progress } ); }
- src/features/action/ImeAction.ts:53-85 (handler)Private method that maps IME actions to Android keyevents and executes via ADB shell input keyeventprivate async executeImeAction(imeAction: string): Promise<void> { logger.info("Executing IME action", { action: imeAction }); // Map IME actions to Android key codes const imeKeyCodeMap: { [key: string]: string } = { "done": "KEYCODE_ENTER", "next": "KEYCODE_TAB", "search": "KEYCODE_SEARCH", "send": "KEYCODE_ENTER", "go": "KEYCODE_ENTER", "previous": "KEYCODE_SHIFT_LEFT KEYCODE_TAB" // Shift+Tab for previous }; const keyCode = imeKeyCodeMap[imeAction]; if (!keyCode) { throw new Error(`Unsupported IME action: ${imeAction}`); } // Small delay to ensure any preceding text input is processed await new Promise(resolve => setTimeout(resolve, 100)); // Execute the key event(s) if (keyCode.includes(" ")) { // Handle multiple key combinations like Shift+Tab const keys = keyCode.split(" "); for (const key of keys) { await this.adb.executeCommand(`shell input keyevent ${key}`); } } else { await this.adb.executeCommand(`shell input keyevent ${keyCode}`); } } }
- src/server/interactionTools.ts:602-615 (handler)Tool handler function that instantiates ImeAction class and calls its execute methodconst imeActionHandler = async (device: BootedDevice, args: ImeActionArgs, progress?: ProgressCallback) => { try { const imeAction = new ImeAction(device); const result = await imeAction.execute(args.action, progress); return createJSONToolResponse({ message: `Executed IME action "${args.action}"`, observation: result.observation, ...result }); } catch (error) { throw new ActionableError(`Failed to execute IME action: ${error}`); } };
- Zod schema defining input parameters for the imeAction toolexport const imeActionSchema = z.object({ action: z.enum(["done", "next", "search", "send", "go", "previous"]).describe("IME action to perform"), platform: z.enum(["android", "ios"]).describe("Platform of the device") });
- src/server/interactionTools.ts:771-777 (registration)Registers the imeAction tool in the ToolRegistry with name, description, schema, and handlerToolRegistry.registerDeviceAware( "imeAction", "Perform an IME action (e.g., done, next, search)", imeActionSchema, imeActionHandler, true // Supports progress notifications );