Server Quality Checklist

Tool Scores

• search_mcp_serversA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears full responsibility for behavioral disclosure. It only states the search capability, missing details on read-only nature, authentication, rate limits, or any side effects. Minimal transparency beyond basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and purpose. Every word adds value. No redundancy or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with 3 parameters and no output schema, the description is adequate but has gaps. It does not explain result format, pagination, or search behavior beyond 'keyword + semantic'. Could be more complete given lack of annotations and output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the schema; it mentions keyword + semantic search but that is already implicit. No additional parameter guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool performs keyword + semantic search across the MCP server registry, with a specific use case ('when the user knows what tool category they want but not which server'). This distinguishes it from sibling tools like assess_server or get_install_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context for when to use the tool, but does not mention when not to use it or list alternatives. The guidance is clear and useful, but lacks exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_serversA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the output content (quality scores, install paths, etc.) but does not mention any side effects, auth requirements, or that it is a read-only operation. It assumes a non-destructive nature, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence encapsulates the core functionality without any wasted words. It is front-loaded with the key action 'Side-by-side comparison' and details the compared attributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter tool with no output schema, the description is relatively complete by listing the compared attributes. However, it does not mention output format, ordering, or whether results are aggregated, which would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'slugs', with min/max items already defined. The description does not add semantic detail beyond what the schema provides, so it meets the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares 2-5 MCP servers side-by-side, listing specific attributes (quality scores, install paths, etc.). This distinguishes it from siblings like assess_server (single server) and recommend_mcp_for_task (recommendation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for comparing multiple servers but does not explicitly state when to use it versus alternatives or any prerequisites. It could benefit from guidance on when to choose compare_servers over assess_server or search_mcp_servers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_install_commandA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description discloses the tool returns a JSON command ready to paste. It does not mention any side effects or authentication, but for a retrieval tool this is sufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words, perfectly front-loaded with the action and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-described parameters and no output schema, the description is complete: it explains what it does and what it returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have descriptions. The description adds marginal value beyond the schema, so baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the install command for a given server and client, and specifies the output format (JSON block). This distinguishes it from sibling tools like search or recommend.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the description, but there is no explicit guidance on when to use this tool versus alternatives, nor conditions to avoid usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_mcp_for_taskA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output structure: 'Returns top 3 ranked picks with reasoning, install commands, and quality scores.' This is sufficient for a read-only recommendation tool. It could mention ranking criteria or data sources for greater transparency, but the current description is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states purpose and output, the second gives usage guidance. It is front-loaded with key information and contains no superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description adequately covers what the tool does and returns. It mentions the top 3 picks, reasoning, install commands, and quality scores. It could be enhanced by referencing sibling tools for alternative uses, but it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'task' parameter. The tool description does not add additional examples or constraints beyond the schema's example. Thus, per the baseline, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Recommend the best MCP servers for a natural-language task.' It specifies the verb (recommend), resource (MCP servers), and context (natural-language task). This distinguishes it from sibling tools like search_mcp_servers (which lists servers) and assess_server (which evaluates a single server).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: 'Use this when the user asks for the right MCP server for a task they want to do.' It lacks explicit when-not-to-use or alternatives, but the context is clear enough for most agents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_serverA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses behavioral traits: advisory nature, conformance monitoring not enforced, and potential UNVERIFIED verdicts. This adds valuable context beyond a simple read operation, though no side effects or auth needs are mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences without redundancy: purpose, use, and behavioral notes. Front-loaded with the core action, efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage guidance, verdict shape, and advisory nature. For a simple one-parameter tool with no output schema, it provides sufficient context, though explicit mention of return format would be slightly better.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'server_id' is described in the schema as 'Server slug to assess.' The description does not add new meaning beyond usage context. With 100% schema coverage, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs an aggregated pre-flight trust assessment across all tools on a server, using the specific verb 'assess' and resource 'server'. It distinguishes from sibling 'check_tool_trust' by noting aggregation, and clarifies the verdict shape is identical.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the use case: 'is THIS server worth integrating?' decisions. It also provides context on when to be cautious with 'v1 advisory; conformance monitored not enforced; verdicts may be UNVERIFIED if not yet probed', guiding appropriate reliance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_tool_trustA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It accurately describes behavior: returns advisory verdict with specific fields, v1 only produces REVIEW or UNVERIFIED. Does not mention side effects, but none expected. Could add authentication requirements, but sufficient for a read-only check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no fluff. Front-loads purpose, then constraints, then usage guideline. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers purpose, return value, version limitations, and usage advice. No gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both parameters described in schema). Description adds no extra meaning beyond what schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: a pre-invocation advisory screen for a specific tool. It uses specific verb 'check' and resource 'tool trust', and distinguishes from siblings like assess_server and search_mcp_servers by focusing on a single tool's trust level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (pre-invocation advisory) and when not (not the in-path gate). Provides clear directive: treat UNVERIFIED as human review required. Distinguishes from sibling tools effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.