android_capture_frame_scrcpy
Capture a single Android device screen frame using scrcpy for faster performance than ADB screencap, saving locally or returning base64 encoded image.
Instructions
Capture a single frame via scrcpy (faster than ADB screencap)
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| outputPath | No | Local path to save the frame (optional). If not provided, returns base64 encoded image. | |
| deviceSerial | No | Specific device serial number to target (optional) |
Implementation Reference
- src/handlers.ts:641-671 (handler)Handler function that executes the android_capture_frame_scrcpy tool logic: processes input arguments, calls ADB wrapper's captureFrameScrcpy method, and returns the frame as base64 image or file path.export async function handleCaptureFrameScrcpy(adb: ADBWrapper, args: CaptureFrameScrcpyArgs): Promise<{ content: Array<{ type: string; text?: string; data?: string; mimeType?: string }> }> { const { outputPath, deviceSerial } = args; try { const result = await adb.captureFrameScrcpy(outputPath, deviceSerial); if (typeof result === 'string') { return { content: [ { type: 'text', text: `Frame saved to: ${result}`, }, ], }; } else { // Return as base64-encoded image return { content: [ { type: 'image', data: result.toString('base64'), mimeType: 'image/png', }, ], }; } } catch (error) { throw new Error(`Failed to capture frame with scrcpy: ${error instanceof Error ? error.message : String(error)}`); } }
- src/adb-wrapper.ts:943-1008 (helper)Core helper method in ADBWrapper that spawns scrcpy process for single frame capture (optimized, faster than ADB screencap), handles output as file or buffer, with fallback to screenshot.async captureFrameScrcpy(outputPath?: string, deviceSerial?: string): Promise<string | Buffer> { const device = await this.getTargetDevice(deviceSerial); // Ensure scrcpy is available if (!this.scrcpyInitialized) { try { await this.downloadScrcpy(); } catch (err) { console.warn('Scrcpy not available, falling back to ADB screencap'); return this.screenshot(outputPath, deviceSerial); } } return new Promise((resolve, reject) => { try { const tempPath = outputPath || join(tmpdir(), `scrcpy_frame_${Date.now()}.png`); // Use scrcpy to dump one frame directly const process = spawn(this.scrcpyPath, [ '--serial', device, '--no-display', '--max-fps=1', // Single frame '--video-codec=h264', '--video-bit-rate=2M', // Frame dump to raw output ], { stdio: ['ignore', 'pipe', 'pipe'], timeout: 5000, }); let frameBuffer = Buffer.alloc(0); process.stdout.on('data', (chunk: Buffer) => { frameBuffer = Buffer.concat([frameBuffer, chunk]); }); process.on('close', async (code) => { if (code !== 0) { // Fall back to screenshot return this.screenshot(outputPath, deviceSerial) .then(resolve) .catch(reject); } if (outputPath) { await writeFile(outputPath, frameBuffer); resolve(outputPath); } else { resolve(frameBuffer); } }); process.on('error', (err) => { // Fall back to standard screenshot this.screenshot(outputPath, deviceSerial) .then(resolve) .catch(reject); }); } catch (err) { // Fall back to standard screenshot this.screenshot(outputPath, deviceSerial) .then(resolve) .catch(reject); } }); }
- src/index.ts:382-397 (schema)Input schema definition for the android_capture_frame_scrcpy tool, including parameters for output path and device serial.name: 'android_capture_frame_scrcpy', description: 'Capture a single frame via scrcpy (faster than ADB screencap)', inputSchema: { type: 'object', properties: { outputPath: { type: 'string', description: 'Local path to save the frame (optional). If not provided, returns base64 encoded image.', }, deviceSerial: { type: 'string', description: 'Specific device serial number to target (optional)', }, }, }, },
- src/index.ts:500-501 (registration)Tool registration in the CallToolRequestSchema switch statement, dispatching to the handler function.case 'android_capture_frame_scrcpy': return await handleCaptureFrameScrcpy(this.adb, args as any);
- src/index.ts:10-10 (registration)Import statement registering the handler function from handlers.ts.import { screenshotHandler, touchHandler, swipeHandler, launchAppHandler, listPackagesHandler, uiautomatorDumpHandler, uiautomatorFindHandler, uiautomatorClickHandler, uiautomatorWaitHandler, uiautomatorSetTextHandler, uiautomatorClearTextHandler, uiautomatorLongClickHandler, uiautomatorDoubleClickHandler, uiautomatorToggleCheckboxHandler, uiautomatorScrollInElementHandler, handleStartScrcpyStream, handleStopScrcpyStream, handleGetLatestFrame, handleCaptureFrameScrcpy, handleSendKeyEvent, handleInputText, handleExecuteCommand } from './handlers.js';