IA-QA — 130+ QA & Dev Tools for AI Agents

mcp_server_evaluate

Read-only

Run a full compliance evaluation against a live MCP server URL. Tests: server reachability (ping), manifest discovery (GET /mcp), schema quality (snake_case names, descriptions, inputSchema), JSON-RPC 2.0 test call, and P50/P95 latency. Returns a PASS/FIX/BLOCK verdict with a 0-100 score and per-check details.

Input Schema

TableJSON Schema

Name	Required	Description	Default
`url`	Yes	Base URL of the MCP server (e.g. https://www.ia-qa.com or http://localhost:3001)
`test_tool_name`	No	Specific tool name to use in the JSON-RPC test call (defaults to the first tool in the manifest)

Output Schema

TableJSON Schema

Name	Required	Description	Default
`url`	No
`score`	No
`checks`	No
`latency`	No
`verdict`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedSep 11, 2026

changedInput schema / properties / url / description

Previous value: -"Base URL of the MCP server (e.g. https://ia-qa.com or http://localhost:3001)"New value: +"Base URL of the MCP server (e.g. https://www.ia-qa.com or http://localhost:3001)"

Changed1 schema field changedJun 21, 2026

addedOutput schema / properties

Added value: +{
+  "checks": {
+    "type": "object"
+  },
+  "latency": {
+    "type": "object"
+  },
+  "score": {
+    "type": "number"
+  },
+  "url": {
+    "type": "string"
+  },
+  "verdict": {
+    "type": "string"
+  }
+}

Changed1 schema field changedJun 10, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}

First observedMay 26, 2026

TDQS

A3.9/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, and destructiveHint=false, so the safety profile is covered. The description goes beyond them by disclosing the actual behavior: it performs an outbound GET /mcp manifest fetch, issues a JSON-RPC 2.0 test call to a live endpoint, measures P50/P95 latency, and emits a PASS/FIX/BLOCK verdict with a 0-100 score. It does not state permission/network prerequisites or whether the test call mutates anything on the target server.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and scope, then a compact list of checks and the return shape. No repetition of the title and no filler; each clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value detail is not the description's burden, yet the verdict format and score range are still summarized. Combined with annotations covering safety and open-world access, the definition gives an agent everything needed to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both url and test_tool_name are fully documented in the schema, giving a baseline of 3. The description mentions the live server URL and the JSON-RPC test call but adds no format, default, or constraint detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb and resource ('Run a full compliance evaluation against a live MCP server URL') followed by a concrete enumeration of the checks performed (ping, manifest discovery, schema quality, JSON-RPC call, latency). It never names a sibling, so an agent must infer that this is the superset of mcp_server_health_check / mcp_schema_lint from the test list alone rather than being told directly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'against a live MCP server URL' implies the context of use (a reachable server you want audited), but there is no explicit when-to-use guidance, no exclusions, and no routing to the closely related siblings mcp_server_health_check, mcp_schema_lint, or latency_benchmark. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All IA-QA — 130+ QA & Dev Tools for AI Agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?