multimodal_eval_guide
Unified tool for multimodal AI evaluation: set action=guide for reference thresholds/interpretation (CLIP, FID, VQA), or set action=clip_score / fid_score / vqa_accuracy / pipeline to compute real metrics via HuggingFace Inference API and VLM BYOK calls. One tool for both reference and computation.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| fid | No | [pipeline] {real_images, generated_images} for FID. | |
| vqa | No | [pipeline] VQA config object (same inputs as vqa_accuracy). | |
| clip | No | [pipeline] {image_url, text} for CLIP. | |
| text | No | [clip_score only] Text description to compare against the image. | |
| model | No | [vqa_accuracy] VLM model ID (default: gpt-4o). | |
| score | No | [guide only] Optional score value to interpret. | |
| action | No | guide (default) = reference thresholds/interpretation. clip_score/fid_score/vqa_accuracy = compute that metric. pipeline = run all three. | |
| metric | No | [guide only] Metric to explain. | |
| api_key | No | [vqa_accuracy] Your API key for the provider (BYOK). | |
| image_url | No | [clip_score/vqa_accuracy] Public URL of the image. | |
| test_cases | No | [vqa_accuracy] Array of {question, accepted_answers} objects. | |
| real_images | No | [fid_score] Array of real image URLs. | |
| image_base64 | No | [clip_score/vqa_accuracy] Base64-encoded image data. | |
| system_prompt | No | [vqa_accuracy] Optional system prompt. | |
| image_mime_type | No | [clip_score/vqa_accuracy] MIME type for base64 image. | |
| generated_images | No | [fid_score] Array of generated image URLs. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| errors | No | ||
| metrics | No | ||
| results | No | ||
| web_tool | No | ||
| best_practices | No | ||
| comparison_table | No | ||
| score_interpretation | No |