analyze_image
Analyze image content from URLs to extract descriptions and insights using AI vision models.
Instructions
Receives an image URL and analyzes the image content using GPT-4o-mini
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| imageUrl | Yes | URL of the image to analyze |
Implementation Reference
- src/index.ts:210-245 (handler)Core handler function that performs the image analysis using OpenAI's GPT-4o-mini model. Handles both URL and base64 image inputs by constructing appropriate image_url objects for the chat completion API.
private async analyzeImageWithGpt4( imageData: { type: 'url', data: string } | { type: 'base64', data: string, mimeType: string } ): Promise<string> { try { let imageInput: any; if (imageData.type === 'url') { imageInput = { type: 'image_url', image_url: { url: imageData.data } }; } else { // Construct data URI for OpenAI API imageInput = { type: 'image_url', image_url: { url: `data:${imageData.mimeType};base64,${imageData.data}` } }; } const response = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'Analyze the image content in detail and provide an explanation in English.', }, { role: 'user', content: [ { type: 'text', text: 'Please analyze the following image and explain its content in detail.' }, imageInput, // Use the constructed image input ], }, ], max_tokens: 1000, }); return response.choices[0]?.message?.content || 'Could not retrieve analysis results.'; } catch (error) { console.error('OpenAI API error:', error); throw new Error(`OpenAI API error: ${error instanceof Error ? error.message : String(error)}`); } } - src/index.ts:78-91 (registration)Registration of the 'analyze_image' tool in the ListToolsRequestSchema handler, including name, description, and input schema.
{ name: 'analyze_image', description: 'Receives an image URL and analyzes the image content using GPT-4o-mini', inputSchema: { type: 'object', properties: { imageUrl: { type: 'string', description: 'URL of the image to analyze', }, }, required: ['imageUrl'], }, }, - src/index.ts:119-129 (handler)Tool dispatch handler specific to 'analyze_image': validates arguments, checks image URL accessibility, and invokes the core analysis function.
if (toolName === 'analyze_image') { if (!isValidAnalyzeImageArgs(args)) { throw new McpError( ErrorCode.InvalidParams, 'Invalid arguments for analyze_image: imageUrl (string) is required' ); } const imageUrl = args.imageUrl; await this.validateImageUrl(imageUrl); // Validate URL accessibility analysis = await this.analyzeImageWithGpt4({ type: 'url', data: imageUrl }); - src/index.ts:81-90 (schema)Input schema for the 'analyze_image' tool, specifying required 'imageUrl' property.
inputSchema: { type: 'object', properties: { imageUrl: { type: 'string', description: 'URL of the image to analyze', }, }, required: ['imageUrl'], }, - src/index.ts:33-39 (helper)Type guard helper function to validate arguments for 'analyze_image' tool.
const isValidAnalyzeImageArgs = ( args: any ): args is { imageUrl: string } => typeof args === 'object' && args !== null && typeof args.imageUrl === 'string';