analyze-video
Analyze mobile app bug videos to identify crashes, UI issues, and errors. Provides AI-powered analysis with code fix suggestions for React Native, iOS, and Android applications.
Instructions
Use this tool to analyze video files on the user's computer that show mobile app bugs. Reads local video files (MP4, MOV, WebM, AVI, MKV, FLV, MPEG, 3GP, WMV) and provides AI-powered analysis to identify errors, crashes, UI issues, and suggests code fixes. Works with React Native, iOS (Swift/Objective-C), and Android (Kotlin/Java) apps. Use this when the user asks you to analyze, examine, or debug a video file showing app behavior.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| videoPath | Yes | Absolute path to the video file on the user's computer (e.g., /Users/username/Desktop/bug.mp4). Max 500MB, max 10 minutes. | |
| question | No | Specific question about the bug (optional). Example: 'Why does the button not respond when tapped?' If not provided, a general analysis will be performed. |
Input Schema (JSON Schema)
{
"properties": {
"question": {
"description": "Specific question about the bug (optional). Example: 'Why does the button not respond when tapped?' If not provided, a general analysis will be performed.",
"type": "string"
},
"videoPath": {
"description": "Absolute path to the video file on the user's computer (e.g., /Users/username/Desktop/bug.mp4). Max 500MB, max 10 minutes.",
"type": "string"
}
},
"required": [
"videoPath"
],
"type": "object"
}
Implementation Reference
- src/client.ts:32-79 (handler)Core handler function that implements the analyze-video tool logic: file validation, presigned upload, job initiation, polling, and result formatting.async analyzeVideo( videoPath: string, question: string ): Promise<{ jobId: string; analysis: string }> { // Validate file size const stats = statSync(videoPath); if (stats.size > MAX_FILE_SIZE) { throw new Error(`Video file too large. Max size: 500MB (${stats.size} bytes provided)`); } // Step 1: Get presigned upload URL const { data: presignData } = await this.client.post("/upload/presign", { filename: basename(videoPath), content_type: this.getContentType(videoPath), file_size: stats.size, }); const { upload_url, video_key } = presignData; // Step 2: Upload video to Firebase Storage using PUT const fileStream = createReadStream(videoPath); await axios.put(upload_url, fileStream, { headers: { "Content-Type": this.getContentType(videoPath), }, timeout: 120000, // 2 minutes for upload maxBodyLength: MAX_FILE_SIZE, maxContentLength: MAX_FILE_SIZE, }); // Step 3: Start analysis job const { data: jobData } = await this.client.post("/analyze/start", { video_key: video_key, filename: basename(videoPath), question, analysis_type: "mobile_bug", }); const jobId = jobData.id; // Step 4: Poll for results const result = await this.pollJobStatus(jobId); return { jobId, analysis: this.formatAnalysis(result), }; }
- src/index.ts:54-67 (schema)Input schema definition for the analyze-video tool, specifying videoPath (required) and optional question.inputSchema: { type: "object", properties: { videoPath: { type: "string", description: "Absolute path to the video file on the user's computer (e.g., /Users/username/Desktop/bug.mp4). Max 500MB, max 10 minutes.", }, question: { type: "string", description: "Specific question about the bug (optional). Example: 'Why does the button not respond when tapped?' If not provided, a general analysis will be performed.", }, }, required: ["videoPath"], },
- src/index.ts:48-71 (registration)Registers the analyze-video tool in the MCP server's ListTools handler, including name, description, and schema.server.setRequestHandler(ListToolsRequestSchema, async () => { return { tools: [ { name: "analyze-video", description: "Use this tool to analyze video files on the user's computer that show mobile app bugs. Reads local video files (MP4, MOV, WebM, AVI, MKV, FLV, MPEG, 3GP, WMV) and provides AI-powered analysis to identify errors, crashes, UI issues, and suggests code fixes. Works with React Native, iOS (Swift/Objective-C), and Android (Kotlin/Java) apps. Use this when the user asks you to analyze, examine, or debug a video file showing app behavior.", inputSchema: { type: "object", properties: { videoPath: { type: "string", description: "Absolute path to the video file on the user's computer (e.g., /Users/username/Desktop/bug.mp4). Max 500MB, max 10 minutes.", }, question: { type: "string", description: "Specific question about the bug (optional). Example: 'Why does the button not respond when tapped?' If not provided, a general analysis will be performed.", }, }, required: ["videoPath"], }, }, ], }; });
- src/index.ts:75-104 (handler)MCP server handler that dispatches analyze-video tool calls to ClipSenseClient and formats the response.if (request.params.name === "analyze-video") { const { videoPath, question } = request.params.arguments as { videoPath: string; question?: string; }; try { // Upload video and analyze const result = await client.analyzeVideo(videoPath, question || "Analyze this bug video and identify the issue."); return { content: [ { type: "text", text: result.analysis, }, ], }; } catch (error: any) { return { content: [ { type: "text", text: `ā Error: ${error.message}\n\n${error.details || ""}`, }, ], isError: true, }; } }
- src/client.ts:84-107 (helper)Helper function to poll the analysis job status until completion or failure.private async pollJobStatus(jobId: string): Promise<any> { const maxAttempts = 120; // 10 minutes max (5s interval) let attempts = 0; while (attempts < maxAttempts) { const { data } = await this.client.get(`/analyze/jobs/${jobId}/status`); if (data.status === "completed") { // Get full job details const { data: fullJob } = await this.client.get(`/analyze/jobs/${jobId}`); return fullJob; } if (data.status === "failed") { throw new Error(`Analysis failed: ${data.error_message || "Unknown error"}`); } // Wait 5 seconds before next poll await new Promise((resolve) => setTimeout(resolve, 5000)); attempts++; } throw new Error("Analysis timeout after 10 minutes"); }