IA-QA — 152 QA & Dev Tools for AI Agents

mcp_server_evaluate

Read-only

Check that a live MCP server over Streamable HTTP speaks the protocol, 2026-07-28 or earlier: handshake or server/discover, tools/list and its definitions, error codes, one tools/call, structuredContent against outputSchema, latency. Returns PASS/FIX/BLOCK with a 0-100 score, or INCONCLUSIVE (score null) when authorization is required. The verdict covers the PROTOCOL, never whether tools answer correctly: what_this_does_not_prove lists what was not measured. Called: only test_tool_name, or a tool annotated readOnlyHint: true. A refusal of invented arguments is not scored (tool_behaviour: refused_placeholder); pass test_tool_args to exercise the tool. Private addresses and stdio are out of reach: paste tools/list into mcp_server_health_check.

Input Schema

Name	Required	Description
`url`	Yes	Public URL of the MCP endpoint (e.g. https://mcp.example.com/mcp). An origin alone (https://www.ia-qa.com) tries /mcp, then /.
`test_tool_args`	No	Real arguments, sent to the tested server. Without them, required ones are invented.
`test_tool_name`	No	Tool to call. Default: a read-only one, preferring no required arg.

Output Schema

Name	Required	Description	Default
`scope`	No
`score`	No
`checks`	No
`server`	No
`latency`	No
`verdict`	No
`endpoint`	No
`next_steps`	No
`tool_behaviour`	No
`what_this_proves`	No
`what_this_does_not_prove`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed7 schema fields changedNov 4, 2026

addedInput schema / properties / test_tool_args

Added value: +{
+  "additionalProperties": true,
+  "description": "Real arguments, sent to the tested server. Without them, required ones are invented.",
+  "type": "object"
+}

changedInput schema / properties / test_tool_name / description

Previous value: -"Tool to call in the tools/call step. Without it, the first tool annotated readOnlyHint: true is called, or none."New value: +"Tool to call. Default: a read-only one, preferring no required arg."

addedOutput schema / properties / next_steps
```
Added value: +{
+  "type": "array"
+}
```
addedOutput schema / properties / scope
```
Added value: +{
+  "type": "string"
+}
```
addedOutput schema / properties / tool_behaviour
```
Added value: +{
+  "type": "string"
+}
```
addedOutput schema / properties / what_this_does_not_prove
```
Added value: +{
+  "type": "array"
+}
```
addedOutput schema / properties / what_this_proves
```
Added value: +{
+  "type": "array"
+}
```

Changed1 schema field changedOct 31, 2026

addedInput schema / examples

Added value: +[
+  {
+    "url": "https://www.ia-qa.com/mcp"
+  }
+]

Changed8 schema fields changedOct 23, 2026

changedInput schema / properties / test_tool_name / description

Previous value: -"Specific tool name to use in the JSON-RPC test call (defaults to the first tool in the manifest)"New value: +"Tool to call in the tools/call step. Without it, the first tool annotated readOnlyHint: true is called, or none."

changedInput schema / properties / url / description

Previous value: -"Base URL of the MCP server (e.g. https://www.ia-qa.com or http://localhost:3001)"New value: +"Public URL of the MCP endpoint (e.g. https://mcp.example.com/mcp). An origin alone (https://www.ia-qa.com) tries /mcp, then /."

changedOutput schema / properties / checks / type
```
Previous value: -"object"New value: +"array"
```
addedOutput schema / properties / endpoint
```
Added value: +{
+  "type": "string"
+}
```

changedOutput schema / properties / latency / type

Previous value: -"object"New value: +[
+  "object",
+  "null"
+]

changedOutput schema / properties / score / type

Previous value: -"number"New value: +[
+  "number",
+  "null"
+]

addedOutput schema / properties / server
```
Added value: +{
+  "type": "object"
+}
```

removedOutput schema / properties / url

Removed value: -{
-  "type": "string"
-}

Changed1 schema field changedSep 11, 2026

changedInput schema / properties / url / description

Previous value: -"Base URL of the MCP server (e.g. https://ia-qa.com or http://localhost:3001)"New value: +"Base URL of the MCP server (e.g. https://www.ia-qa.com or http://localhost:3001)"

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "checks": {
+    "type": "object"
+  },
+  "latency": {
+    "type": "object"
+  },
+  "score": {
+    "type": "number"
+  },
+  "url": {
+    "type": "string"
+  },
+  "verdict": {
+    "type": "string"
+  }
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

A4.8/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses the verdict vocabulary (PASS/FIX/BLOCK with 0-100 score, INCONCLUSIVE with null score when authorization is required), the scope limit that the verdict covers the protocol and never tool correctness, and the presence of a what_this_does_not_prove field. It also explains that a refusal of invented arguments is not scored (tool_behaviour: refused_placeholder), which prevents misreading a FIX result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and scoping before the verdict semantics, and nearly every clause carries information. The single dense paragraph stacks many semicolon-separated constraints, which costs some scanability for an agent parsing it quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a live-network probe tool with an output schema present, the description supplies everything an agent needs: what is tested, what is not, the result vocabulary, the authorization edge case, and the safety constraint on which tool gets called. No gap remains that would cause a misinvocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines all three parameters, but the description still adds real meaning: why test_tool_args matters (without them required args are invented) and what happens with a bare origin URL. The read-only restriction on test_tool_name is reinforced here rather than left only in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — checks that a live MCP server over Streamable HTTP speaks the protocol (2026-07-28 or earlier) — and enumerates exactly what is probed: handshake/server/discover, tools/list, error codes, one tools/call, structuredContent vs outputSchema, latency. It also draws a sharp line against its nearest siblings, naming mcp_server_health_check for private addresses/stdio.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit calling constraint (only test_tool_name, or a tool annotated readOnlyHint: true) and tells the agent how to handle refused placeholder arguments by passing test_tool_args. It also states the out-of-scope conditions (private addresses, stdio) and routes those cases to mcp_server_health_check as the alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 152 QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.