IA-QA — 152 QA & Dev Tools for AI Agents

compare_models

Read-onlyIdempotent

Put 2-5 AI models side by side: price per 1k tokens, context window, multimodal, reasoning, provider, pricing page. Facts only: names the cheapest and the largest context, never a "best model" — run your cases on each to choose.

Input Schema

Name	Required	Description	Default
`models`	Yes	Array of 2-5 model names (e.g. ["gpt-5","claude-sonnet-4-6","gemini-3.5-flash"])

Output Schema

Name	Required	Description	Default
`rows`	No
`cheapest`	No
`prices_as_of`	No
`largest_context`	No
`models_compared`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedNov 2, 2026

changedInput schema / examples

Previous value: -[
-  {
-    "models": [
-      "gpt-4o",
-      "claude-3-5-sonnet-20241022"
-    ]
-  }
-]New value: +[
+  {
+    "models": [
+      "gpt-6.1-sol",
+      "claude-sonnet-5-5"
+    ]
+  }
+]

Changed9 schema fields changedNov 2, 2026

changedInput schema / properties / models / description

Previous value: -"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-3.5-flash\"])"New value: +"Array of 2-5 model names (e.g. [\"gpt-5\",\"claude-sonnet-4-6\",\"gemini-3.5-flash\"])"

removedInput schema / properties / use_case

Removed value: -{
-  "description": "Optimize recommendation for this criterion",
-  "enum": [
-    "cost",
-    "context",
-    "reasoning",
-    "multimodal",
-    "speed"
-  ],
-  "type": "string"
-}

addedOutput schema / properties / cheapest
```
Added value: +{}
```
removedOutput schema / properties / cost_per_1k_total
```
Removed value: -{
-  "type": "string"
-}
```
addedOutput schema / properties / largest_context
```
Added value: +{}
```
removedOutput schema / properties / model
```
Removed value: -{}
```
addedOutput schema / properties / prices_as_of
```
Added value: +{}
```
removedOutput schema / properties / recommendation
```
Removed value: -{}
```
removedOutput schema / properties / use_case
```
Removed value: -{}
```

Changed1 schema field changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "models": [
+      "gpt-4o",
+      "claude-3-5-sonnet-20241022"
+    ]
+  }
+]

Changed1 schema field changedOct 26, 2026

changedInput schema / properties / models / description

Previous value: -"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-2.0-flash\"])"New value: +"Array of 2-5 model names (e.g. [\"gpt-4o\",\"claude-3.5-sonnet\",\"gemini-3.5-flash\"])"

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "cost_per_1k_total": {
+    "type": "string"
+  },
+  "model": {},
+  "models_compared": {
+    "type": "number"
+  },
+  "recommendation": {},
+  "rows": {},
+  "use_case": {}
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

A4.3/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the safety profile is covered. The description adds valuable behavioral context about output content: it reports facts only, names the cheapest and largest context, and avoids subjective recommendations. It does not mention data freshness or error handling, but that is a minor gap given the annotations and output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with zero waste. The core purpose and scope are front-loaded, followed by the key behavioral constraint. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, full annotation coverage, and a single well-documented parameter, the description provides everything an agent needs: what is compared, the input range, and the factual (non-recommendation) nature of the output. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the single 'models' parameter is fully documented in the schema, including the 2-5 constraint and examples. The description reinforces the 2-5 range but adds no new syntax or format details beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Put ... side by side') and resource ('AI models'), enumerates the exact attributes compared (price, context window, multimodal, reasoning, provider, pricing page), and scopes the input to 2-5 models. This clearly distinguishes it from single-model siblings like model_info or list_llm_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context for use ('run your cases on each to choose') and explicitly states what it will not do ('never a "best model"'), which helps an agent understand its role. However, it does not name specific alternative tools (e.g., model_info for single-model details) or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.