process_image_to_code
Convert images into code snippets by analyzing visual content and generating code in specified programming languages, streamlining development workflows.
Instructions
处理图像并生成代码
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| image_url | Yes | 图像URL | |
| instructions | No | 额外的指令或要求 | |
| language | No | 目标编程语言，例如：python, javascript, html, css等 | python |
Implementation Reference
- src/index.ts:212-249 (handler)Handler for 'process_image_to_code' tool: validates input, downloads image, builds prompt with language and instructions, calls Qwen VL model via helper, returns generated code or error.case "process_image_to_code": { const imageUrl = String(request.params.arguments?.image_url); const language = String(request.params.arguments?.language || "python"); const instructions = String(request.params.arguments?.instructions || ""); if (!imageUrl) { throw new Error("图像URL是必需的"); } try { // 下载图像 const imagePath = await downloadImage(imageUrl); // 构建提示词 let prompt = `请根据这张图片生成${language}代码。`; if (instructions) { prompt += ` ${instructions}`; } // 处理图像 const result = await processImageWithQwen(imagePath, prompt); return { content: [{ type: "text", text: result }] }; } catch (error: any) { return { content: [{ type: "text", text: `处理图像失败: ${error.message}` }], isError: true }; } }
- src/index.ts:157-176 (schema)Input schema definition for 'process_image_to_code' tool, specifying required image_url and optional language/instructions.inputSchema: { type: "object", properties: { image_url: { type: "string", description: "图像URL" }, language: { type: "string", description: "目标编程语言，例如：python, javascript, html, css等", default: "python" }, instructions: { type: "string", description: "额外的指令或要求", default: "" } }, required: ["image_url"] }
- src/index.ts:154-177 (registration)Registration of 'process_image_to_code' tool in the ListToolsRequestSchema handler, including name, description, and input schema.{ name: "process_image_to_code", description: "处理图像并生成代码", inputSchema: { type: "object", properties: { image_url: { type: "string", description: "图像URL" }, language: { type: "string", description: "目标编程语言，例如：python, javascript, html, css等", default: "python" }, instructions: { type: "string", description: "额外的指令或要求", default: "" } }, required: ["image_url"] } },
- src/index.ts:74-131 (helper)Helper function to process image with Qwen VL model: reads image to base64, sends to API with prompt, returns text response, cleans up temp file.async function processImageWithQwen(imagePath: string, prompt: string): Promise<string> { try { // 读取图像文件 const imageBuffer = fs.readFileSync(imagePath); const base64Image = imageBuffer.toString('base64'); // 准备请求数据 const requestData = { model: "qwen-vl-plus", input: { messages: [ { role: "user", content: [ { type: "text", text: prompt }, { type: "image", image: base64Image } ] } ] }, parameters: {} }; // 发送请求到API const response = await axios.post(API_ENDPOINT, requestData, { headers: { 'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json' } }); // 处理响应 if (response.data && response.data.output && response.data.output.text) { return response.data.output.text; } else { throw new Error("API响应格式不正确"); } } catch (error: any) { console.error("调用qwen2.5-vl模型API失败:", error); if (error.response) { console.error("API响应:", error.response.data); } throw new Error(`处理图像失败: ${error.message}`); } finally { // 清理临时文件 try { fs.unlinkSync(imagePath); } catch (e) { console.error("清理临时文件失败:", e); } } }
- src/index.ts:47-66 (helper)Helper function to download image from URL to a temporary file using streaming.async function downloadImage(imageUrl: string): Promise<string> { try { // 为图像生成唯一文件名 const filename = `${Date.now()}-${Math.random().toString(36).substring(2, 15)}.jpg`; const filePath = path.join(TEMP_DIR, filename); // 下载图像 const response = await axios({ method: 'GET', url: imageUrl, responseType: 'stream' }); await streamPipeline(response.data, fs.createWriteStream(filePath)); return filePath; } catch (error: any) { console.error("下载图像失败:", error); throw new Error(`下载图像失败: ${error.message}`); } }