analyze_image
Analyze images with Gemini AI to answer questions about visual content, identify objects, or extract information from photos using vision capabilities.
Instructions
Analyze images using Gemini vision capabilities
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | Yes | Question or instruction about the image | |
| imageUrl | No | URL of the image to analyze | |
| imageBase64 | No | Base64-encoded image data (alternative to URL) | |
| model | No | Vision-capable Gemini model | gemini-2.5-flash |
Implementation Reference
- src/enhanced-stdio-server.ts:660-737 (handler)Primary implementation of the analyze_image tool handler. Processes arguments, handles image data (URL or base64), calls Gemini vision API, and returns MCP response.private async analyzeImage(id: any, args: any): Promise<MCPResponse> { try { const model = args.model || 'gemini-2.5-flash'; // Validate inputs if (!args.imageUrl && !args.imageBase64) { throw new Error('Either imageUrl or imageBase64 must be provided'); } // Prepare image part let imagePart: any; if (args.imageUrl) { // For URL, we'd need to fetch and convert to base64 // For now, we'll just pass the URL as instruction imagePart = { text: `[Image URL: ${args.imageUrl}]` }; } else if (args.imageBase64) { // Log base64 data size for debugging console.error(`Image base64 length: ${args.imageBase64.length}`); // Extract MIME type and data const matches = args.imageBase64.match(/^data:(.+);base64,(.+)$/); if (matches) { console.error(`MIME type: ${matches[1]}, Data length: ${matches[2].length}`); imagePart = { inlineData: { mimeType: matches[1], data: matches[2] } }; } else { // If no data URI format, assume raw base64 console.error('Raw base64 data detected'); imagePart = { inlineData: { mimeType: 'image/jpeg', data: args.imageBase64 } }; } } const result = await this.genAI.models.generateContent({ model, contents: [ { parts: [{ text: args.prompt }, imagePart], role: 'user' } ] }); const text = result.text || ''; return { jsonrpc: '2.0', id, result: { content: [ { type: 'text', text } ] } }; } catch (error) { console.error('Error in analyzeImage:', error); return { jsonrpc: '2.0', id, error: { code: -32603, message: `Image analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}` } }; }
- src/utils/validation.ts:81-92 (schema)Zod schema for validating analyze_image tool inputs in ToolSchemasanalyzeImage: z .object({ prompt: z.string().min(1, 'Prompt is required'), imageUrl: CommonSchemas.imageUrl.optional(), imageBase64: CommonSchemas.base64Image.optional(), model: z.enum(['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.0-flash']).optional() }) .refine( data => data.imageUrl || data.imageBase64, 'Either imageUrl or imageBase64 must be provided' ),
- src/enhanced-stdio-server.ts:290-317 (registration)Tool registration entry returned by tools/list, including description and input schema{ name: 'analyze_image', description: 'Analyze images using Gemini vision capabilities', inputSchema: { type: 'object', properties: { prompt: { type: 'string', description: 'Question or instruction about the image' }, imageUrl: { type: 'string', description: 'URL of the image to analyze' }, imageBase64: { type: 'string', description: 'Base64-encoded image data (alternative to URL)' }, model: { type: 'string', description: 'Vision-capable Gemini model', enum: ['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.0-flash'], default: 'gemini-2.5-flash' } }, required: ['prompt'] } },
- src/enhanced-stdio-server.ts:490-491 (handler)Dispatch case in handleToolCall that routes analyze_image calls to the handler method.case 'analyze_image': return await this.analyzeImage(request.id, args);