Skip to main content
Glama

chat_with_vision

Analyze images and answer questions about visual content using AI vision capabilities. Upload images via paths or URLs and receive detailed descriptions or insights based on your prompts.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
promptYes
modelNogrok-4
image_pathsNo
image_urlsNo
detailNoauto

Implementation Reference

  • Main handler function for chat_with_vision tool. This async function takes a prompt, model name, optional image paths/URLs, and detail level. It creates a chat client, processes images (encoding them to base64 if needed), appends them as image content along with the prompt, and returns the AI response content.
    @mcp.tool() async def chat_with_vision( prompt: str, model: str = "grok-4", image_paths: Optional[List[str]] = None, image_urls: Optional[List[str]] = None, detail: str = "auto" ): client = Client(api_key=XAI_API_KEY) chat = client.chat.create(model=model, store_messages=False) user_content = [] if image_paths: for path in image_paths: ext = Path(path).suffix.lower().replace('.', '') if ext not in ["jpg", "jpeg", "png"]: raise ValueError(f"Unsupported image type: {ext}") base64_img = encode_image_to_base64(path) user_content.append(image(image_url=f"data:image/{ext};base64,{base64_img}", detail=detail)) if image_urls: for url in image_urls: user_content.append(image(image_url=url, detail=detail)) user_content.append(prompt) chat.append(user(*user_content)) response = chat.sample() client.close() return response.content
  • src/server.py:120-120 (registration)
    Tool registration using @mcp.tool() decorator on the chat_with_vision function, which registers it as an MCP tool with the FastMCP server.
    @mcp.tool()
  • Helper function encode_image_to_base64 that reads an image file from disk and converts it to a base64 encoded string, used by chat_with_vision to process local image files.
    def encode_image_to_base64(image_path: str): path = Path(image_path) if not path.exists(): raise FileNotFoundError(f"Image file not found: {image_path}") with open(image_path, "rb") as image_file: return base64.b64encode(image_file.read()).decode("utf-8")

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/merterbak/Grok-MCP'

If you have feedback or need assistance with the MCP directory API, please join our Discord server