multivon-mcp

Official

Overview Schema Source Related Servers Discussions Admin

eval_vqa_faithfulness

Verifies whether an LLM's answer about an image is grounded in visible facts. Extracts up to three claims and checks each against the image, returning a score based on accurate claims.

Instructions

Check whether an LLM answer about an image is grounded in what's visible.

Image-grounded faithfulness. The vision judge extracts up to 3 factual claims from the answer, then verifies each one against the image. Score = fraction of claims that are accurate.

Use this for visual QA, image captioning, chart/diagram reading, and any LLM output that purports to describe an image.

Image input — exactly one of:

image: a local path, http(s) URL, or full data URI.
image_base64: raw base64 (no data: prefix); pair with mime_type (default "image/png").

Args: input: The question or prompt the LLM was answering. output: The LLM-generated answer to verify against the image. image: Path / URL / data URI for the image. image_base64: Alternative — raw base64 image bytes. mime_type: Mime type when using image_base64. Default "image/png". Other common values: "image/jpeg", "image/webp". judge_model: Provider:model for the vision judge. Must be vision-capable. Default "google:gemini-2.5-flash" (cheap). Other vision-capable options: "openai:gpt-4o-mini" or "anthropic:claude-sonnet-4-6". Haiku 4.5 also supports image input; verify task quality on your own held-out cases.

Returns: {"score": 0.0-1.0, "passed": bool, "reason": str, "threshold": float, "evaluator": "vqa_faithfulness"}.

Input Schema

TableJSON Schema

Name	Required	Default
`image`	No
`input`	Yes
`output`	Yes
`mime_type`	No	image/png
`judge_model`	No	google:gemini-2.5-flash
`image_base64`	No

Output Schema

TableJSON Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

Addedv0.3.2Jun 17, 2026

TDQS

A4.6/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the internal process (claim extraction, verification), the scoring logic, and the return format including score, passed, reason, threshold, and evaluator. It also details judge_model options and cautions about Haiku quality. It doesn't disclose edge cases (e.g., unreadable images) but covers the core behavior thoroughly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized into sections (purpose, usage, image input, args, returns) and front-loads the core purpose and method. Though moderately long, every sentence adds necessary information for a tool with six parameters. There is no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 params, no annotations, output schema present), the description is complete. It covers all parameters, defaults, alternatives, return format, and usage guidance. It even notes the threshold field in the output. An agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description fully compensates. The Args section provides detailed explanations for all six parameters, including input/output roles, image input alternatives (path/URL/data URI vs. base64), mime_type defaults and common values, and judge_model defaults and alternatives. This goes well beyond the bare schema definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise statement of what the tool does: 'Check whether an LLM answer about an image is grounded in what's visible.' It then details the method (extract up to 3 factual claims, verify each against the image, score = fraction accurate). This clearly distinguishes it from text-only siblings like eval_faithfulness and eval_hallucination.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use it: 'Use this for visual QA, image captioning, chart/diagram reading, and any LLM output that purports to describe an image.' This gives clear positive use cases, but it does not explicitly mention alternatives or when not to use it. It could be improved by naming text-only counterparts (e.g., eval_faithfulness) as not applicable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.