Skip to main content
Glama

ui-action

Automate Android app UI interactions: tap elements, input text, and scroll screens to test and debug applications directly through the development environment.

Instructions

Interact with app UI: tap, input, scroll.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
operationYes
xNo
yNo
elementIndexNo
textNo
directionNo
amountNoScroll fraction (0-1, default: 0.5)
deviceSpaceNoTreat x/y as device coordinates (skip scaling)

Implementation Reference

  • Main handler function for the 'ui-action' tool, which dispatches to specific operation handlers (tap, input, scroll).
    export async function handleUiActionTool(
      input: UiActionInput,
      context: ServerContext,
    ): Promise<Record<string, unknown>> {
      const device = await context.deviceState.ensureDevice(context.adb);
    
      const handler = operations[input.operation];
      if (!handler) {
        throw new ReplicantError(
          ErrorCode.INVALID_OPERATION,
          `Unknown operation: ${input.operation}`,
          "Valid operations: tap, input, scroll",
        );
      }
      return handler(input, context, device.id);
    }
  • Input schema definition for the 'ui-action' tool using Zod.
    export const uiActionInputSchema = z.object({
      operation: z.enum(["tap", "input", "scroll"]),
      x: z.number().optional(),
      y: z.number().optional(),
      elementIndex: z.number().optional(),
      text: z.string().optional(),
      direction: z.enum(["up", "down", "left", "right"]).optional(),
      amount: z.number().min(0).max(1).optional(),
      deviceSpace: z.boolean().optional(),
    });
    
    export type UiActionInput = z.infer<typeof uiActionInputSchema>;
  • Definition object for registering the 'ui-action' tool with the MCP server.
    export const uiActionToolDefinition = {
      name: "ui-action",
      description: "Interact with app UI: tap, input, scroll.",
      inputSchema: {
        type: "object",
        properties: {
          operation: {
            type: "string",
            enum: ["tap", "input", "scroll"],
          },
          x: { type: "number" },
          y: { type: "number" },
          elementIndex: { type: "number" },
          text: { type: "string" },
          direction: { type: "string", enum: ["up", "down", "left", "right"] },
          amount: { type: "number", minimum: 0, maximum: 1, description: "Scroll fraction (0-1, default: 0.5)" },
          deviceSpace: { type: "boolean", description: "Treat x/y as device coordinates (skip scaling)" },
        },
        required: ["operation"],
      },
      annotations: {
        readOnlyHint: false,
        destructiveHint: true,
        idempotentHint: false,
        openWorldHint: false,
      },
    };

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/thecombatwombat/replicant-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server