IA-QA — 152 QA & Dev Tools for AI Agents

analyze_responses

Read-onlyIdempotent

Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a reference (ground truth): also ranks every output by closeness (token cosine + ROUGE-L composite) and names the closest. No LLM, no key, runs locally. The reference mode is gate-able in CI; the consensus/outlier mode is a signal (no reference to fail against). You bring the outputs (2+). A 2-way head-to-head with structural JSON diff is compare_responses.

Input Schema

Name	Required	Description	Default
`reference`	No	Optional ground-truth answer. If set, each output is also ranked by closeness to it and the closest one is named.
`responses`	Yes	The outputs to analyze (same task, N models/prompts/versions). Each item is a plain string or { "label": "GPT-4o", "text": "..." }. At least 2 required.

Output Schema

Name	Required	Description	Default
`count`	No
`summary`	No
`consensus`	No
`reference_ranking`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "reference": "Paris is the capital of France.",
+    "responses": [
+      {
+        "label": "A",
+        "text": "Lyon is the capital."
+      },
+      {
+        "label": "B",
+        "text": "Paris is the capital of France."
+      }
+    ]
+  }
+]

AddedJul 2, 2026

TDQS

Score is being calculated.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.