AutoMobile

Instructions

Implementation Reference

• src/features/action/ImeAction.ts:12-51 (handler)

  Main handler logic for executing IME actions, including validation, ADB keyevent execution, and screen change observation.

  async execute(
    action: "done" | "next" | "search" | "send" | "go" | "previous",
    progress?: ProgressCallback
  ): Promise<ImeActionResult> {
    // Validate action input
    if (!action) {
      return {
        success: false,
        action: "",
        error: "No IME action provided"
      };
    }
  
    return this.observedInteraction(
      async () => {
        try {
          await this.executeImeAction(action);
  
          return {
            success: true,
            action
          };
        } catch (error) {
          const errorMessage = error instanceof Error ? error.message : String(error);
  
          return {
            success: false,
            action,
            error: `Failed to execute IME action: ${errorMessage}`
          };
        }
      },
      {
        changeExpected: true,
        tolerancePercent: 0.00,
        timeoutMs: 3000, // IME actions should be quick
        progress
      }
    );
  }

• src/features/action/ImeAction.ts:53-84 (handler)

  Private helper that maps IME actions to Android keycodes and dispatches them via adb shell input keyevent.

  private async executeImeAction(imeAction: string): Promise<void> {
    logger.info("Executing IME action", { action: imeAction });
  
    // Map IME actions to Android key codes
    const imeKeyCodeMap: { [key: string]: string } = {
      "done": "KEYCODE_ENTER",
      "next": "KEYCODE_TAB",
      "search": "KEYCODE_SEARCH",
      "send": "KEYCODE_ENTER",
      "go": "KEYCODE_ENTER",
      "previous": "KEYCODE_SHIFT_LEFT KEYCODE_TAB" // Shift+Tab for previous
    };
  
    const keyCode = imeKeyCodeMap[imeAction];
    if (!keyCode) {
      throw new Error(`Unsupported IME action: ${imeAction}`);
    }
  
    // Small delay to ensure any preceding text input is processed
    await new Promise(resolve => setTimeout(resolve, 100));
  
    // Execute the key event(s)
    if (keyCode.includes(" ")) {
      // Handle multiple key combinations like Shift+Tab
      const keys = keyCode.split(" ");
      for (const key of keys) {
        await this.adb.executeCommand(`shell input keyevent ${key}`);
      }
    } else {
      await this.adb.executeCommand(`shell input keyevent ${keyCode}`);
    }
  }

• src/server/interactionTools.ts:602-615 (handler)

  MCP tool handler wrapper that instantiates ImeAction class and invokes its execute method.

  const imeActionHandler = async (device: BootedDevice, args: ImeActionArgs, progress?: ProgressCallback) => {
    try {
      const imeAction = new ImeAction(device);
      const result = await imeAction.execute(args.action, progress);
  
      return createJSONToolResponse({
        message: `Executed IME action "${args.action}"`,
        observation: result.observation,
        ...result
      });
    } catch (error) {
      throw new ActionableError(`Failed to execute IME action: ${error}`);
    }
  };

• src/server/interactionTools.ts:216-219 (schema)

  Zod input schema defining the arguments for the imeAction tool: action enum and platform.

  export const imeActionSchema = z.object({
    action: z.enum(["done", "next", "search", "send", "go", "previous"]).describe("IME action to perform"),
    platform: z.enum(["android", "ios"]).describe("Platform of the device")
  });

• src/server/interactionTools.ts:771-777 (registration)

  Registration of the imeAction tool in the ToolRegistry with name, description, schema, and handler.

  ToolRegistry.registerDeviceAware(
    "imeAction",
    "Perform an IME action (e.g., done, next, search)",
    imeActionSchema,
    imeActionHandler,
    true // Supports progress notifications
  );

• src/models/ImeActionResult.ts:1-6 (schema)

  TypeScript interface defining the return type of the ImeAction execution.

  export interface ImeActionResult {
      success: boolean;
      action: string;
      error?: string;
      observation?: any;
  }