extract_image_from_base64
Extract and process images from base64-encoded data for analysis. Use this tool to handle screenshots, clipboard images, or dynamically generated visuals without file system access, optimizing them for AI model interpretation.
Instructions
Extract and analyze images from base64-encoded data. Ideal for processing screenshots from clipboard, dynamically generated images, or images embedded in applications without requiring file system access.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| base64 | Yes | Base64-encoded image data to analyze (useful for screenshots, images from clipboard, or dynamically generated visuals) | |
| max_height | No | For backward compatibility only. Default maximum height is now 512px | |
| max_width | No | For backward compatibility only. Default maximum width is now 512px | |
| mime_type | No | MIME type of the image (e.g., image/png, image/jpeg) | image/png |
| resize | No | For backward compatibility only. Images are always automatically resized to optimal dimensions (max 512x512) for LLM analysis |
Implementation Reference
- src/image-utils.ts:333-433 (handler)The core handler function for the 'extract_image_from_base64' tool. It decodes the base64 input, validates and processes the image using Sharp (resize, compress), and returns an MCP tool response with image metadata and the processed base64 image.export async function extractImageFromBase64(params: ExtractImageFromBase64Params): Promise<McpToolResponse> { try { const { base64, mime_type, resize, max_width, max_height } = params; // Decode base64 let imageBuffer; try { imageBuffer = Buffer.from(base64, 'base64'); // Quick validation - valid base64 strings should be decodable if (imageBuffer.length === 0) { throw new Error("Invalid base64 string - decoded to empty buffer"); } } catch (e) { return { content: [{ type: "text", text: `Error: Invalid base64 string - ${e instanceof Error ? e.message : String(e)}` }], isError: true }; } // Check size if (imageBuffer.length > MAX_IMAGE_SIZE) { return { content: [{ type: "text", text: `Error: Image size exceeds maximum allowed size of ${MAX_IMAGE_SIZE} bytes` }], isError: true }; } // Process the image let metadata; try { metadata = await sharp(imageBuffer).metadata(); } catch (e) { return { content: [{ type: "text", text: `Error: Could not process image data - ${e instanceof Error ? e.message : String(e)}` }], isError: true }; } // Always resize to ensure the base64 representation is reasonable // This will help avoid consuming too much of the context window if (metadata.width && metadata.height) { // Use provided dimensions or fallback to defaults for optimal LLM context usage const targetWidth = Math.min(metadata.width, DEFAULT_MAX_WIDTH); const targetHeight = Math.min(metadata.height, DEFAULT_MAX_HEIGHT); // Only resize if needed if (metadata.width > targetWidth || metadata.height > targetHeight) { imageBuffer = await sharp(imageBuffer) .resize({ width: targetWidth, height: targetHeight, fit: 'inside', withoutEnlargement: true }) .toBuffer(); // Update metadata after resize metadata = await sharp(imageBuffer).metadata(); } } // Compress the image based on its format try { const format = metadata.format || mime_type.split('/')[1] || 'jpeg'; imageBuffer = await compressImage(imageBuffer, format); } catch (compressionError) { console.warn('Compression warning, using original image:', compressionError); // Continue with the original image if compression fails } // Convert back to base64 const processedBase64 = imageBuffer.toString('base64'); // Return both text and image content return { content: [ { type: "text", text: JSON.stringify({ width: metadata.width, height: metadata.height, format: metadata.format, size: imageBuffer.length }) }, { type: "image", data: processedBase64, mimeType: mime_type } ] }; } catch (error: unknown) { console.error('Error processing base64 image:', error); return { content: [{ type: "text", text: `Error: ${error instanceof Error ? error.message : String(error)}` }], isError: true }; } }
- src/index.ts:53-68 (registration)Registers the 'extract_image_from_base64' tool with the MCP server, including Zod input schema validation and the handler wrapper that calls the implementation.// Add extract_image_from_base64 tool server.tool( "extract_image_from_base64", "Extract and analyze images from base64-encoded data. Ideal for processing screenshots from clipboard, dynamically generated images, or images embedded in applications without requiring file system access.", { base64: z.string().describe("Base64-encoded image data to analyze (useful for screenshots, images from clipboard, or dynamically generated visuals)"), mime_type: z.string().default("image/png").describe("MIME type of the image (e.g., image/png, image/jpeg)"), resize: z.boolean().default(true).describe("For backward compatibility only. Images are always automatically resized to optimal dimensions (max 512x512) for LLM analysis"), max_width: z.number().default(512).describe("For backward compatibility only. Default maximum width is now 512px"), max_height: z.number().default(512).describe("For backward compatibility only. Default maximum height is now 512px") }, async (args, extra) => { const result = await extractImageFromBase64(args); return result; } );
- src/image-utils.ts:43-49 (schema)TypeScript type definition for the input parameters used by the extractImageFromBase64 handler function.export type ExtractImageFromBase64Params = { base64: string; mime_type: string; resize: boolean; max_width: number; max_height: number; };