analyze_image
Analyze images using Gemini AI vision models to answer questions, identify content, or extract information from images provided via URL or base64 data.
Instructions
Analyze images using Gemini vision capabilities
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| imageBase64 | No | Base64-encoded image data (alternative to URL) | |
| imageUrl | No | URL of the image to analyze | |
| model | No | Vision-capable Gemini model | gemini-2.5-flash |
| prompt | Yes | Question or instruction about the image |
Implementation Reference
- src/enhanced-stdio-server.ts:660-738 (handler)The primary handler function that implements the analyze_image tool logic, processing image inputs (URL or base64) and calling the Gemini vision API.private async analyzeImage(id: any, args: any): Promise<MCPResponse> { try { const model = args.model || 'gemini-2.5-flash'; // Validate inputs if (!args.imageUrl && !args.imageBase64) { throw new Error('Either imageUrl or imageBase64 must be provided'); } // Prepare image part let imagePart: any; if (args.imageUrl) { // For URL, we'd need to fetch and convert to base64 // For now, we'll just pass the URL as instruction imagePart = { text: `[Image URL: ${args.imageUrl}]` }; } else if (args.imageBase64) { // Log base64 data size for debugging console.error(`Image base64 length: ${args.imageBase64.length}`); // Extract MIME type and data const matches = args.imageBase64.match(/^data:(.+);base64,(.+)$/); if (matches) { console.error(`MIME type: ${matches[1]}, Data length: ${matches[2].length}`); imagePart = { inlineData: { mimeType: matches[1], data: matches[2] } }; } else { // If no data URI format, assume raw base64 console.error('Raw base64 data detected'); imagePart = { inlineData: { mimeType: 'image/jpeg', data: args.imageBase64 } }; } } const result = await this.genAI.models.generateContent({ model, contents: [ { parts: [{ text: args.prompt }, imagePart], role: 'user' } ] }); const text = result.text || ''; return { jsonrpc: '2.0', id, result: { content: [ { type: 'text', text } ] } }; } catch (error) { console.error('Error in analyzeImage:', error); return { jsonrpc: '2.0', id, error: { code: -32603, message: `Image analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}` } }; } }
- src/utils/validation.ts:81-92 (schema)Zod validation schema for analyze_image tool parameters, ensuring prompt is provided and either imageUrl or imageBase64 is present.analyzeImage: z .object({ prompt: z.string().min(1, 'Prompt is required'), imageUrl: CommonSchemas.imageUrl.optional(), imageBase64: CommonSchemas.base64Image.optional(), model: z.enum(['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.0-flash']).optional() }) .refine( data => data.imageUrl || data.imageBase64, 'Either imageUrl or imageBase64 must be provided' ),
- src/enhanced-stdio-server.ts:290-317 (registration)Tool registration in the getAvailableTools() method, defining the analyze_image tool name, description, and input schema for MCP tools/list.{ name: 'analyze_image', description: 'Analyze images using Gemini vision capabilities', inputSchema: { type: 'object', properties: { prompt: { type: 'string', description: 'Question or instruction about the image' }, imageUrl: { type: 'string', description: 'URL of the image to analyze' }, imageBase64: { type: 'string', description: 'Base64-encoded image data (alternative to URL)' }, model: { type: 'string', description: 'Vision-capable Gemini model', enum: ['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.0-flash'], default: 'gemini-2.5-flash' } }, required: ['prompt'] } },
- src/enhanced-stdio-server.ts:490-491 (registration)Dispatch/registration in the handleToolCall switch statement, routing analyze_image calls to the handler.case 'analyze_image': return await this.analyzeImage(request.id, args);