Skip to main content
Glama

chat_with_vision

Analyze images from local files or public URLs using a Grok vision model to get answers about their content.

Instructions

Analyze one or more images with a Grok vision model.

Accepts local image paths and/or public URLs in the same call. Local images
are sent as base64 data URIs (JPG/JPEG/PNG only, max 20 MiB each).

Args:
    prompt: Question or instruction about the image(s).
    session: Optional session name for persistent history in `chats/{session}.json`.
    model: Vision-capable Grok model (default `grok-4.6`).
    image_paths: Local image file paths to analyze.
    image_urls: Public image URLs to analyze.
    detail: Image detail level. One of `"auto"`, `"low"`, or `"high"`.
    show_usage: Append a token usage and cost footer to the reply (default False).

Returns:
    The model's textual answer about the image(s).

Input Schema

NameRequiredDescriptionDefault
modelNogrok-4.6
detailNoauto
promptYes
sessionNo
image_urlsNo
show_usageNo
image_pathsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.0.4
  2. Removed
  3. Changed1 schema field changedv1.0.1
    • addedInput schema / properties / show_usage
      Added value: +{
      +  "default": false,
      +  "title": "Show Usage",
      +  "type": "boolean"
      +}
  4. Added
  5. Removed
  6. Added
  7. Removed
  8. Added
  9. Removed
  10. Changed1 schema field changed
    • changedInput schema / properties / model / default
      Previous value: -"grok-4"New value: +"grok-4-1-fast-reasoning"
  11. Changed1 schema field changed
    • addedInput schema / properties / session
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Session"
      +}
  12. Changed2 schema fields changed
    • changedInput schema / properties / model / default
      Previous value: -"grok-4-1-fast-non-reasoning"New value: +"grok-4"
    • changedOutput schema / (root)
      Previous value: -{
      -  "properties": {
      -    "result": {
      -      "title": "Result",
      -      "type": "string"
      -    }
      -  },
      -  "required": [
      -    "result"
      -  ],
      -  "title": "chat_with_visionOutput",
      -  "type": "object"
      -}New value: +null
  13. Changed2 schema fields changedv1.0.0
    • changedInput schema / properties / model / default
      Previous value: -"grok-4-0709"New value: +"grok-4-1-fast-non-reasoning"
    • addedInput schema / title
      Added value: +"chat_with_visionArguments"
  14. First observed

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose real behavioral facts: local images are converted to base64 data URIs, only JPG/JPEG/PNG up to 20 MiB each, and session enables persistent history at `chats/{session}.json`. It omits error behavior for invalid/oversized images, auth or rate-limit constraints, and how mixed local+URL batches are ordered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded summary sentence, then a constraints sentence, then a compact Args block and a one-line Returns. The Args list is not waste here because schema coverage is 0%; every line adds information the agent cannot get from structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with no output schema and no annotations, the description covers the return value ('textual answer about the image(s)'), input formats, size limits, and persistence. The main gap is routing guidance relative to sibling chat tools, but nothing needed to invoke the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there are 7 parameters, so the description must compensate — and it documents every one (prompt, session, model, image_paths, image_urls, detail, show_usage) with meaning beyond the bare titles. It adds real value the schema lacks, notably the `detail` enum values auto/low/high (schema has no enum) and the session file location, though it doesn't state constraints like mutual requirement or image-count limits.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Analyze one or more images with a Grok vision model') and the two input modalities (local paths, public URLs), which separates it from the generic chat/chat_with_files siblings. It never names an alternative sibling explicitly, so the differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the tool for image analysis, and the description notes local paths and URLs can be mixed in one call. There is no when-to-use vs when-not guidance, no mention of chat, chat_with_files, or stateful_chat as alternatives, and no statement that at least one image source is required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.