Android UI Assist MCP Server

Instructions

Implementation Reference

• src/server.ts:45-49 (registration)

  Registration of the 'take_android_screenshot' tool in the ListTools response, including name, description, and input schema reference.

  {
    name: 'take_android_screenshot',
    description: 'Capture a screenshot from an Android device or emulator',
    inputSchema: TakeScreenshotToolSchema,
  },

• src/server.ts:65-81 (handler)

  MCP CallTool handler switch case for 'take_android_screenshot', which parses input, calls takeScreenshot method, and formats the response as image and text content.

  case 'take_android_screenshot': {
    const input = TakeScreenshotInputSchema.parse(args);
    const result = await this.takeScreenshot(input);
    return {
      content: [
        {
          type: 'image',
          data: result.data,
          mimeType: 'image/png',
        },
        {
          type: 'text',
          text: `Android screenshot captured from ${result.deviceId}: ${result.width}x${result.height} pixels`,
        },
      ],
    };
  }

• src/server.ts:113-128 (handler)

  Private method that invokes captureScreenshotResponse from utils/screenshot and validates the output using TakeScreenshotOutputSchema.

  private async takeScreenshot(
    input: z.infer<typeof TakeScreenshotInputSchema>
  ): Promise<z.infer<typeof TakeScreenshotOutputSchema>> {
    const screenshot = await captureScreenshotResponse(input.deviceId);
  
    const result = {
      data: screenshot.data,
      format: screenshot.format,
      width: screenshot.width,
      height: screenshot.height,
      deviceId: screenshot.deviceId,
      timestamp: screenshot.timestamp,
    };
  
    return TakeScreenshotOutputSchema.parse(result);
  }

• src/utils/screenshot.ts:28-53 (helper)

  Helper function that captures the screenshot buffer via ADB, extracts PNG dimensions, converts to base64, and formats the response object.

  export async function captureScreenshotResponse(deviceId?: string): Promise<ScreenshotResponse> {
    try {
      // Capture screenshot as binary data (now returns Buffer directly)
      const buffer = captureScreenshot(deviceId);
  
      // Get image dimensions
      const { width, height } = getPNGDimensions(buffer);
  
      // Convert to base64
      const base64Data = binaryToBase64(buffer);
  
      // Return formatted response
      return {
        data: base64Data,
        format: 'png',
        width,
        height,
        deviceId: deviceId || 'default',
        timestamp: Date.now(),
      };
    } catch (error) {
      throw new Error(
        `Failed to capture screenshot: ${error instanceof Error ? error.message : String(error)}`
      );
    }
  }

• src/utils/adb.ts:153-201 (handler)

  Core handler function that validates the device, constructs and executes the ADB screencap command via execSync to retrieve the raw PNG screenshot buffer.

  export function captureScreenshot(deviceId?: string): Buffer {
    let targetDeviceId: string;
  
    try {
      if (deviceId) {
        // Verify the device exists
        const devices = getConnectedDevices();
        const device = devices.find(d => d.id === deviceId);
  
        if (!device) {
          throw new DeviceNotFoundError(deviceId);
        }
  
        if (device.status !== 'device') {
          throw new ADBCommandError(
            'DEVICE_NOT_AVAILABLE',
            `Device '${deviceId}' is not available (status: ${device.status})`,
            { device }
          );
        }
  
        targetDeviceId = deviceId;
      } else {
        // Use the first available device
        const device = getFirstAvailableDevice();
        targetDeviceId = device.id;
      }
  
      // Capture screenshot using binary command execution
      const command = targetDeviceId
        ? `-s ${targetDeviceId} exec-out screencap -p`
        : 'exec-out screencap -p';
      const screenshotData = executeADBCommandBinary(command);
  
      if (!screenshotData || screenshotData.length === 0) {
        throw new ScreenshotCaptureError(targetDeviceId);
      }
  
      return screenshotData;
    } catch (error) {
      if (error instanceof ADBCommandError) {
        throw error;
      }
      throw new ScreenshotCaptureError(
        deviceId || 'unknown',
        error instanceof Error ? error : undefined
      );
    }
  }

• src/types.ts:137-153 (schema)

  MCP tool schema definition for 'take_android_screenshot', used in tool registration and input validation.

  export const TakeScreenshotToolSchema = {
    type: 'object' as const,
    properties: {
      deviceId: {
        type: 'string' as const,
        description:
          'The ID of the Android device to capture a screenshot from. If not provided, uses the first available device.',
      },
      format: {
        type: 'string' as const,
        enum: ['png'],
        default: 'png',
        description: 'The image format for the screenshot. Currently only PNG is supported.',
      },
    },
    required: [] as string[],
  };