OpenRouter MCP Multimodal Server

Instructions

Implementation Reference

• src/tool-handlers/analyze-image.ts:13-45 (handler)

  The handler function `handleAnalyzeImage` which processes the image analysis request by preparing the image and sending it to the OpenRouter/OpenAI API.

  export async function handleAnalyzeImage(
    request: { params: { arguments: AnalyzeImageToolRequest } },
    openai: OpenAI,
    defaultModel?: string,
  ) {
    const { image_path, question, model } = request.params.arguments;
  
    if (!image_path) {
      return { content: [{ type: 'text', text: 'image_path is required.' }], isError: true };
    }
  
    try {
      const imageUrl = await prepareImageUrl(image_path);
  
      const completion = await openai.chat.completions.create({
        model: model || defaultModel || DEFAULT_MODEL,
        messages: [
          {
            role: 'user',
            content: [
              { type: 'text', text: question || "What's in this image?" },
              { type: 'image_url', image_url: { url: imageUrl } },
            ],
          },
        ] as ChatCompletionMessageParam[],
      });
  
      return { content: [{ type: 'text', text: completion.choices[0].message.content || '' }] };
    } catch (error: unknown) {
      const msg = error instanceof Error ? error.message : String(error);
      return { content: [{ type: 'text', text: `Error: ${msg}` }], isError: true };
    }
  }

• src/tool-handlers/analyze-image.ts:7-11 (schema)

  The `AnalyzeImageToolRequest` interface defining the expected input structure for the `analyze_image` tool.

  export interface AnalyzeImageToolRequest {
    image_path: string;
    question?: string;
    model?: string;
  }

• src/tool-handlers.ts:73-85 (registration)

  Registration of the `analyze_image` tool in the `ListToolsRequestSchema` handler.

  {
    name: 'analyze_image',
    description: 'Analyze an image using a vision model',
    inputSchema: {
      type: 'object',
      properties: {
        image_path: { type: 'string', description: 'File path, URL, or data URL' },
        question: { type: 'string', description: 'Question about the image' },
        model: { type: 'string' },
      },
      required: ['image_path'],
    },
  },

• src/tool-handlers.ts:142-147 (handler)

  The `CallToolRequestSchema` handler routing the `analyze_image` request to the implementation function.

  case 'analyze_image':
    return handleAnalyzeImage(
      wrapToolArgs(args as AnalyzeImageToolRequest | undefined),
      this.openai,
      this.defaultModel,
    );