analyze_figure_for_search
Analyze a scientific image or figure to extract key terms and automatically search for related biomedical literature.
Instructions
Analyze a scientific figure or image for literature search.
═══════════════════════════════════════════════════════════════════════ 🔬 VISION-TO-LITERATURE SEARCH (Experimental) ═══════════════════════════════════════════════════════════════════════
This tool enables searching for scientific literature based on images.
WORKFLOW (Agent MUST follow all steps automatically): ─────────────────────────────────────────────────────────
Provide an image (URL or base64-encoded)
This tool returns the image using MCP ImageContent protocol
YOU (the Agent) analyze the image using your vision capabilities
Extract relevant ENGLISH search terms from the image
IMMEDIATELY call
search_biomedical_images()orunified_search()with the extracted terms - DO NOT wait for user confirmationReturn both the analysis and search results to the user
⚠️ IMPORTANT RULES: ────────────────
ALL search queries must be in ENGLISH (Open-i requirement)
After analyzing the image, proceed to search automatically
Do NOT just describe the image and stop - complete the full workflow
If the image shows a medical condition, extract the medical term in English
SEARCH TYPES: ─────────────
"comprehensive": General analysis, extract all relevant terms (default)
"methodology": Focus on methods, equipment, techniques shown
"results": Focus on data, graphs, statistical findings
"structure": Focus on molecular/chemical structures
"medical": Focus on clinical/medical imaging findings
USE CASES: ──────────
📊 Scientific figures → Find papers with similar data/charts
🔬 Microscopy images → Find related research
🧬 Molecular structures → Find papers about the compound
📈 Graphs/plots → Find papers with similar analyses
🏥 Medical images → Find case reports or clinical studies
⚗️ Lab equipment → Find methodology papers
IMPORTANT: ────────── After analyzing the image, IMMEDIATELY search for related literature. Do NOT ask the user "do you want me to search?" — just do it. Use English medical terminology in all search queries.
Args: image: Base64-encoded image data OR data URI (data:image/png;base64,...) url: URL of the image to analyze context: Optional context about what to look for in the image search_type: Type of analysis focus (comprehensive/methodology/results/structure/medical)
Returns: List containing: - ImageContent: The image for you to analyze - TextContent: Instructions for next steps
Example: analyze_figure_for_search(url="https://example.com/figure1.png") analyze_figure_for_search(url="https://...", search_type="medical") analyze_figure_for_search(image="data:image/png;base64,iVBORw0...")
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| image | No | ||
| url | No | ||
| context | No | ||
| search_type | No | comprehensive |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |