analyze_image
Analyze images by providing a URL or base64 data with a question to extract information using Gemini vision models.
Instructions
Analyze images using Gemini vision capabilities
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | Yes | Question or instruction about the image | |
| imageUrl | No | URL of the image to analyze | |
| imageBase64 | No | Base64-encoded image data (alternative to URL) | |
| model | No | Vision-capable Gemini model | gemini-2.5-flash |
Implementation Reference
- src/enhanced-stdio-server.ts:619-696 (handler)Main handler function that processes image analysis requests using Gemini vision models. Handles both image URLs and base64 data, validates inputs, prepares the image part for the API, calls generateContent, and returns the analysis text.private async analyzeImage(id: any, args: any): Promise<MCPResponse> { try { const model = args.model || 'gemini-2.5-flash'; // Validate inputs if (!args.imageUrl && !args.imageBase64) { throw new Error('Either imageUrl or imageBase64 must be provided'); } // Prepare image part let imagePart: any; if (args.imageUrl) { // For URL, we'd need to fetch and convert to base64 // For now, we'll just pass the URL as instruction imagePart = { text: `[Image URL: ${args.imageUrl}]` }; } else if (args.imageBase64) { // Log base64 data size for debugging console.error(`Image base64 length: ${args.imageBase64.length}`); // Extract MIME type and data const matches = args.imageBase64.match(/^data:(.+);base64,(.+)$/); if (matches) { console.error(`MIME type: ${matches[1]}, Data length: ${matches[2].length}`); imagePart = { inlineData: { mimeType: matches[1], data: matches[2] } }; } else { // If no data URI format, assume raw base64 console.error('Raw base64 data detected'); imagePart = { inlineData: { mimeType: 'image/jpeg', data: args.imageBase64 } }; } } const result = await this.genAI.models.generateContent({ model, contents: [{ parts: [ { text: args.prompt }, imagePart ], role: 'user' }] }); const text = result.text || ''; return { jsonrpc: '2.0', id, result: { content: [{ type: 'text', text: text }] } }; } catch (error) { console.error('Error in analyzeImage:', error); return { jsonrpc: '2.0', id, error: { code: -32603, message: `Image analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}` } }; } }
- src/enhanced-stdio-server.ts:273-304 (schema)Input schema definition for the analyze_image tool, including parameters for prompt, image source (URL or base64), and model selection. Specifies required fields and validation rules.{ name: 'analyze_image', description: 'Analyze images using Gemini vision capabilities', inputSchema: { type: 'object', properties: { prompt: { type: 'string', description: 'Question or instruction about the image' }, imageUrl: { type: 'string', description: 'URL of the image to analyze' }, imageBase64: { type: 'string', description: 'Base64-encoded image data (alternative to URL)' }, model: { type: 'string', description: 'Vision-capable Gemini model', enum: ['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.0-flash'], default: 'gemini-2.5-flash' } }, required: ['prompt'], oneOf: [ { required: ['imageUrl'] }, { required: ['imageBase64'] } ] } },
- src/enhanced-stdio-server.ts:477-478 (registration)Switch case in handleToolCall that routes analyze_image tool calls to the analyzeImage handler method.case 'analyze_image': return await this.analyzeImage(request.id, args);