analyze_image
Extract insights from images using GPT-4-turbo. Submit an image URL to analyze and describe its content through natural language, enhancing AI-driven understanding of visual data.
Instructions
画像URLを受け取り、GPT-4-turboを使用して画像の内容を分析します
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| imageUrl | Yes | 分析する画像のURL |
Implementation Reference
- src/index.ts:210-245 (handler)Core handler function that executes the image analysis logic using GPT-4o-mini for the 'analyze_image' tool. Handles both URL and base64 inputs.private async analyzeImageWithGpt4( imageData: { type: 'url', data: string } | { type: 'base64', data: string, mimeType: string } ): Promise<string> { try { let imageInput: any; if (imageData.type === 'url') { imageInput = { type: 'image_url', image_url: { url: imageData.data } }; } else { // Construct data URI for OpenAI API imageInput = { type: 'image_url', image_url: { url: `data:${imageData.mimeType};base64,${imageData.data}` } }; } const response = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'Analyze the image content in detail and provide an explanation in English.', }, { role: 'user', content: [ { type: 'text', text: 'Please analyze the following image and explain its content in detail.' }, imageInput, // Use the constructed image input ], }, ], max_tokens: 1000, }); return response.choices[0]?.message?.content || 'Could not retrieve analysis results.'; } catch (error) { console.error('OpenAI API error:', error); throw new Error(`OpenAI API error: ${error instanceof Error ? error.message : String(error)}`); } }
- src/index.ts:119-129 (handler)Tool dispatch handler specifically for 'analyze_image': validates arguments, checks image URL accessibility, and invokes the core analysis function.if (toolName === 'analyze_image') { if (!isValidAnalyzeImageArgs(args)) { throw new McpError( ErrorCode.InvalidParams, 'Invalid arguments for analyze_image: imageUrl (string) is required' ); } const imageUrl = args.imageUrl; await this.validateImageUrl(imageUrl); // Validate URL accessibility analysis = await this.analyzeImageWithGpt4({ type: 'url', data: imageUrl });
- src/index.ts:78-91 (registration)Registration of the 'analyze_image' tool in the list of tools, including name, description, and input schema.{ name: 'analyze_image', description: 'Receives an image URL and analyzes the image content using GPT-4o-mini', inputSchema: { type: 'object', properties: { imageUrl: { type: 'string', description: 'URL of the image to analyze', }, }, required: ['imageUrl'], }, },
- src/index.ts:33-39 (schema)Type guard/schema validation for input arguments of the 'analyze_image' tool.const isValidAnalyzeImageArgs = ( args: any ): args is { imageUrl: string } => typeof args === 'object' && args !== null && typeof args.imageUrl === 'string';
- src/index.ts:194-207 (helper)Helper function to validate the accessibility and image content-type of the provided URL, used in 'analyze_image' handler.private async validateImageUrl(url: string): Promise<void> { try { const response = await axios.head(url); const contentType = response.headers['content-type']; if (!contentType || !contentType.startsWith('image/')) { throw new Error(`URL is not an image: ${contentType}`); } } catch (error) { if (axios.isAxiosError(error)) { throw new Error(`Cannot access image URL: ${error.message}`); } throw error; } }