mcp-judge

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It is exceptionally transparent: it discloses the $0.10 x402 payment flow, the first-call payment requirement and retry with X-PAYMENT header, the safe functional probe (paid tools never called), the objective nature of checks, and the output contents. This gives an agent a reliable model of side effects and prerequisites.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is information-dense yet every sentence earns its place: use case, pricing/payment flow, comparison method, output summary, cost comparison, limitations, and parameter instruction. It is front-loaded with the decision-oriented purpose rather than buried technical details. No filler or redundant phrasing is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still tells an agent exactly what to expect: a ranked list, recommended winner, plain-language explanation, and full report per server. It also covers the crucial operational context (payment handling, retry flow, safe probe, objective-only checks) and the input constraint of 2-5 endpoints. There are no critical gaps for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 even if the description added nothing about parameters. The description does reinforce that 'urls' is required, must contain 2-5 Streamable HTTP endpoints, and provides an example, but it largely repeats the schema's own parameter description. No additional semantic meaning is added beyond the structured schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear use-case question and states the tool 'compares 2-5 MCP servers head-to-head' with concrete checks. It explicitly distinguishes itself from evaluate_mcp by being cheaper for 3+ servers and emphasizes it performs objective checks only. This leaves no ambiguity about what the tool does or how it differs from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-to-use trigger ('Choosing between several MCP servers that do similar things?') and states a clear alternative ('cheaper than separate evaluate_mcp calls on 3+ servers'). It also clarifies what the tool does not do (does not judge real-world usefulness or safety), helping an agent decide when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full behavioral burden and does so thoroughly: it discloses the $0.05 x402 payment mechanism, first-call payment negotiation, that free tools are actually called while paid tools are only declared, and that it never pays for paid tools. It also states objectivity limits and the return payload.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average but every sentence serves a purpose: usage context, cost, behavior, outputs, routing tips, limitations, and parameter instruction. It opens with the high-priority usage directive and then flows logically. Slightly dense but well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single parameter, no output schema, and no annotations, the description covers all essentials: why to use it, when, cost, payment flow, checks performed, return format, limitations, and the required parameter with example. An agent has everything needed to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents 'url' as required Streamable HTTP endpoint. The description restates this ('Set 'url' (required) to the target's MCP endpoint (Streamable HTTP)') but adds no new semantics beyond what the schema offers, hence the 3 baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Returns a complete objective quality report') and resource (third-party MCP server), and distinguishes itself from siblings by referencing quick_check and list_criteria as lighter alternatives. It names exactly what it does without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Call this BEFORE integrating any third-party MCP server' and provides routing alternatives: 'run the FREE quick_check first to get just the grade, and list_criteria to see exactly how scoring works.' This gives the agent clear when-to-use and alternative conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full load. It discloses that the tool is FREE and that the evaluation is 'fully objective and automated — no human reviewer and no AI opinion', which are useful behavioral traits beyond what a schema would show. It also notes it takes no parameters, though that is already implicit in the empty schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is not overly long (~70 words) and front-loads the most important fact ('FREE, no payment required'). However, there is minor redundancy ('FREE' and 'no payment required') and some promotional phrasing ('Full transparency into what you pay for'), though it remains efficient overall.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description is complete. It explains what the tool returns (criteria, weights, grade scale), why to call it (before paying for scoring tools), and the nature of the evaluation (objective/automated). An agent has everything needed to invoke it correctly without further inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema description coverage is 100% (empty). According to the calibration guidance, 0 params warrants a baseline of 4. The description adds a note that it 'Takes no parameters' but otherwise does not need to explain parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('returns') and a clear resource ('the exact criteria MCP Judge uses to score an MCP server'), including what is returned (criteria, weights, grade scale). It distinguishes itself from siblings by positioning it as the transparency tool that explains the scoring of quick_check, evaluate_mcp, and compare_mcps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Call this to understand how quick_check, evaluate_mcp and compare_mcps score targets before paying', giving clear contextual guidance on when to use it (before paid tools). While it does not state exclusions or alternatives, the 'before paying' instruction is a strong usage signal.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: it discloses the return shape ('returns only...no detailed report'), cost ('FREE, no payment required'), and latency implications. It doesn't spell out failure/error behavior or whether the call is read-only, but for a lightweight check the key behavioral boundaries are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core result, then use case, then alternative routing, then parameter — a logical order. It is slightly promotional ('FREE, no payment required', 'wastes your tokens and can break your workflow'), but these factors are relevant to an agent deciding whether to call it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, but the description compensates by enumerating all returned fields and the lack of a detailed report. The single parameter is fully covered and siblings are routed. Only missing extras such as error/timeout behavior, which are not essential for first-line triage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the single 'url' parameter as the MCP endpoint. The description repeats this ('Set 'url' (required) to the target's MCP endpoint') and adds an example ('https://host/mcp'), which is a modest supplement but not new semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Description states an exact action — 'Instant trust check of any MCP server' — and names the exact return payload ('0-100 score, A-F grade, tool count, latency and a one-line verdict') while explicitly distinguishing itself from evaluate_mcp and compare_mcps. The resource and scope are unambiguous, and it cannot be confused with siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger: 'Use this FIRST, before integrating any third-party MCP server,' and explains why (unreliable servers waste tokens and break workflows). It also names the alternatives with their conditions: evaluate_mcp for a full report, compare_mcps for comparing alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.