android_capture_frame_scrcpy

Instructions

Implementation Reference

• src/handlers.ts:641-671 (handler)

  Handler function that executes the android_capture_frame_scrcpy tool logic: processes input arguments, calls ADB wrapper's captureFrameScrcpy method, and returns the frame as base64 image or file path.

  export async function handleCaptureFrameScrcpy(adb: ADBWrapper, args: CaptureFrameScrcpyArgs): Promise<{ content: Array<{ type: string; text?: string; data?: string; mimeType?: string }> }> {
    const { outputPath, deviceSerial } = args;
  
    try {
      const result = await adb.captureFrameScrcpy(outputPath, deviceSerial);
  
      if (typeof result === 'string') {
        return {
          content: [
            {
              type: 'text',
              text: `Frame saved to: ${result}`,
            },
          ],
        };
      } else {
        // Return as base64-encoded image
        return {
          content: [
            {
              type: 'image',
              data: result.toString('base64'),
              mimeType: 'image/png',
            },
          ],
        };
      }
    } catch (error) {
      throw new Error(`Failed to capture frame with scrcpy: ${error instanceof Error ? error.message : String(error)}`);
    }
  }

• src/adb-wrapper.ts:943-1008 (helper)

  Core helper method in ADBWrapper that spawns scrcpy process for single frame capture (optimized, faster than ADB screencap), handles output as file or buffer, with fallback to screenshot.

  async captureFrameScrcpy(outputPath?: string, deviceSerial?: string): Promise<string | Buffer> {
    const device = await this.getTargetDevice(deviceSerial);
  
    // Ensure scrcpy is available
    if (!this.scrcpyInitialized) {
      try {
        await this.downloadScrcpy();
      } catch (err) {
        console.warn('Scrcpy not available, falling back to ADB screencap');
        return this.screenshot(outputPath, deviceSerial);
      }
    }
  
    return new Promise((resolve, reject) => {
      try {
        const tempPath = outputPath || join(tmpdir(), `scrcpy_frame_${Date.now()}.png`);
        
        // Use scrcpy to dump one frame directly
        const process = spawn(this.scrcpyPath, [
          '--serial', device,
          '--no-display',
          '--max-fps=1',           // Single frame
          '--video-codec=h264',
          '--video-bit-rate=2M',
          // Frame dump to raw output
        ], {
          stdio: ['ignore', 'pipe', 'pipe'],
          timeout: 5000,
        });
  
        let frameBuffer = Buffer.alloc(0);
        
        process.stdout.on('data', (chunk: Buffer) => {
          frameBuffer = Buffer.concat([frameBuffer, chunk]);
        });
  
        process.on('close', async (code) => {
          if (code !== 0) {
            // Fall back to screenshot
            return this.screenshot(outputPath, deviceSerial)
              .then(resolve)
              .catch(reject);
          }
  
          if (outputPath) {
            await writeFile(outputPath, frameBuffer);
            resolve(outputPath);
          } else {
            resolve(frameBuffer);
          }
        });
  
        process.on('error', (err) => {
          // Fall back to standard screenshot
          this.screenshot(outputPath, deviceSerial)
            .then(resolve)
            .catch(reject);
        });
      } catch (err) {
        // Fall back to standard screenshot
        this.screenshot(outputPath, deviceSerial)
          .then(resolve)
          .catch(reject);
      }
    });
  }

• src/index.ts:382-397 (schema)

  Input schema definition for the android_capture_frame_scrcpy tool, including parameters for output path and device serial.

    name: 'android_capture_frame_scrcpy',
    description: 'Capture a single frame via scrcpy (faster than ADB screencap)',
    inputSchema: {
      type: 'object',
      properties: {
        outputPath: {
          type: 'string',
          description: 'Local path to save the frame (optional). If not provided, returns base64 encoded image.',
        },
        deviceSerial: {
          type: 'string',
          description: 'Specific device serial number to target (optional)',
        },
      },
    },
  },

• src/index.ts:500-501 (registration)

  Tool registration in the CallToolRequestSchema switch statement, dispatching to the handler function.

  case 'android_capture_frame_scrcpy':
    return await handleCaptureFrameScrcpy(this.adb, args as any);

• src/index.ts:10-10 (registration)

  Import statement registering the handler function from handlers.ts.

  import { screenshotHandler, touchHandler, swipeHandler, launchAppHandler, listPackagesHandler, uiautomatorDumpHandler, uiautomatorFindHandler, uiautomatorClickHandler, uiautomatorWaitHandler, uiautomatorSetTextHandler, uiautomatorClearTextHandler, uiautomatorLongClickHandler, uiautomatorDoubleClickHandler, uiautomatorToggleCheckboxHandler, uiautomatorScrollInElementHandler, handleStartScrcpyStream, handleStopScrcpyStream, handleGetLatestFrame, handleCaptureFrameScrcpy, handleSendKeyEvent, handleInputText, handleExecuteCommand } from './handlers.js';