Skip to main content
Glama

IA-QA Security — 30 tools

mcp_server_evaluate

Read-only

Run a compliance evaluation against a live MCP server over Streamable HTTP, the way a client meets it. Checks: initialize (protocol version, serverInfo, Mcp-Session-Id carried afterwards), notifications/initialized (202), tools/list with pagination, the spec's tool-definition rules, ping, an unknown method (-32601), one tools/call, GET on the endpoint (SSE or 405), and P50/P95 latency over pings. JSON and SSE answers are both read. Only a tool you name in test_tool_name, or one the server annotates readOnlyHint: true, is called — otherwise the call is skipped and not scored. Returns PASS/FIX/BLOCK with a 0-100 score and per-check details; INCONCLUSIVE (score null) when the server requires authorization, since no credentials are sent. A 301/302 on POST caps the verdict at FIX. The call runs from ia-qa.com: private and local addresses are refused, and stdio servers cannot be reached — for those, paste their tools/list answer into mcp_server_health_check. structuredContent is checked for presence, not validated against outputSchema.

Input Schema

NameRequiredDescriptionDefault
urlYesPublic URL of the MCP endpoint (e.g. https://mcp.example.com/mcp). An origin alone (https://www.ia-qa.com) tries /mcp, then /.
test_tool_nameNoTool to call in the tools/call step. Without it, the first tool annotated readOnlyHint: true is called, or none.

Output Schema

NameRequiredDescriptionDefault
scoreNo
checksNo
serverNo
latencyNo
verdictNo
endpointNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare it a safe, open-world read; the description adds substantial context beyond that: no credentials are sent so auth-required servers yield INCONCLUSIVE with a null score, a 301/302 on POST caps the verdict at FIX, private/local addresses are refused, and structuredContent is only checked for presence rather than validated against outputSchema. These are exactly the operational behaviors an agent could not infer from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, and the check inventory that follows is dense but each item carries information. It reads as a long comma-chained sentence rather than structured lines, which slightly hurts scanability, but there is little true filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with an output schema, the description still summarizes the return contract (PASS/FIX/BLOCK, 0-100 score, per-check details, INCONCLUSIVE with null score) and covers preconditions, network constraints, and score-affecting edge cases. Nothing an agent needs to invoke it correctly appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema for test_tool_name: if omitted, the first readOnlyHint:true tool is called, and otherwise the call is skipped and not scored. That clarifies the consequence of leaving the parameter out, which the schema alone does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Run a compliance evaluation against a live MCP server over Streamable HTTP' — and immediately enumerates the checks it performs (initialize, tools/list, ping, unknown method, tools/call, GET, latency). It is clearly separable from siblings like mcp_schema_lint and mcp_server_health_check, the latter of which it explicitly delegates stdio cases to.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing rules: stdio servers and private/local addresses cannot be reached here, and their tools/list output should be pasted into mcp_server_health_check instead. It also states when a tools/call is skipped versus executed (test_tool_name or a readOnlyHint:true tool), which is the key selection decision for this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.