process_image_to_description
Convert images into detailed descriptions tailored to specific focus areas and detail levels, enhancing accessibility and understanding of visual content.
Instructions
处理图像并生成描述
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| detail_level | No | 描述详细程度：简洁(brief)、标准(standard)或详细(detailed) | standard |
| focus | No | 描述重点，例如：物体、场景、人物、情感等 | |
| image_url | Yes | 图像URL |
Implementation Reference
- src/index.ts:251-294 (handler)Handler implementation for the 'process_image_to_description' tool. Parses input arguments, downloads the image from the provided URL, constructs a descriptive prompt based on detail_level and focus parameters, invokes the Qwen VL model via processImageWithQwen helper, and returns the generated description or an error response.case "process_image_to_description": { const imageUrl = String(request.params.arguments?.image_url); const detailLevel = String(request.params.arguments?.detail_level || "standard"); const focus = String(request.params.arguments?.focus || ""); if (!imageUrl) { throw new Error("图像URL是必需的"); } try { // 下载图像 const imagePath = await downloadImage(imageUrl); // 构建提示词 let prompt = "请描述这张图片"; if (detailLevel === "brief") { prompt += "，给出简洁的描述"; } else if (detailLevel === "detailed") { prompt += "，提供详细的描述，包括细节、背景和上下文"; } if (focus) { prompt += `，重点关注${focus}`; } // 处理图像 const result = await processImageWithQwen(imagePath, prompt); return { content: [{ type: "text", text: result }] }; } catch (error: any) { return { content: [{ type: "text", text: `处理图像失败: ${error.message}` }], isError: true }; } }
- src/index.ts:178-202 (registration)Registration of the 'process_image_to_description' tool in the ListToolsRequestSchema handler, including name, description, and input schema definition.{ name: "process_image_to_description", description: "处理图像并生成描述", inputSchema: { type: "object", properties: { image_url: { type: "string", description: "图像URL" }, detail_level: { type: "string", description: "描述详细程度：简洁(brief)、标准(standard)或详细(detailed)", enum: ["brief", "standard", "detailed"], default: "standard" }, focus: { type: "string", description: "描述重点，例如：物体、场景、人物、情感等", default: "" } }, required: ["image_url"] } }
- src/index.ts:74-131 (helper)Key helper function that reads the image file, encodes it to base64, constructs the API request to Qwen VL model with the prompt and image, sends the request, extracts the text response, and cleans up the temp file. Used by the tool handler.async function processImageWithQwen(imagePath: string, prompt: string): Promise<string> { try { // 读取图像文件 const imageBuffer = fs.readFileSync(imagePath); const base64Image = imageBuffer.toString('base64'); // 准备请求数据 const requestData = { model: "qwen-vl-plus", input: { messages: [ { role: "user", content: [ { type: "text", text: prompt }, { type: "image", image: base64Image } ] } ] }, parameters: {} }; // 发送请求到API const response = await axios.post(API_ENDPOINT, requestData, { headers: { 'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json' } }); // 处理响应 if (response.data && response.data.output && response.data.output.text) { return response.data.output.text; } else { throw new Error("API响应格式不正确"); } } catch (error: any) { console.error("调用qwen2.5-vl模型API失败:", error); if (error.response) { console.error("API响应:", error.response.data); } throw new Error(`处理图像失败: ${error.message}`); } finally { // 清理临时文件 try { fs.unlinkSync(imagePath); } catch (e) { console.error("清理临时文件失败:", e); } } }
- src/index.ts:47-66 (helper)Helper function to download image from URL to a temporary file using streaming, generates unique filename, returns the temp path. Called by the tool handler.async function downloadImage(imageUrl: string): Promise<string> { try { // 为图像生成唯一文件名 const filename = `${Date.now()}-${Math.random().toString(36).substring(2, 15)}.jpg`; const filePath = path.join(TEMP_DIR, filename); // 下载图像 const response = await axios({ method: 'GET', url: imageUrl, responseType: 'stream' }); await streamPipeline(response.data, fs.createWriteStream(filePath)); return filePath; } catch (error: any) { console.error("下载图像失败:", error); throw new Error(`下载图像失败: ${error.message}`); } }