Server Quality Checklist

Tool Scores

• search_mcp_serversB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does convey the action and return fields, indicating a read-only search operation. However, it does not mention which fields the query matches against, ordering, authentication needs, or behavior when no results are found, leaving notable ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and then compactly lists the return fields. There is no redundant wording; it is appropriately sized and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a four-parameter search tool with full schema documentation, the description is largely sufficient: it states the action and the returned shape. It lacks explicit sibling differentiation and detail on search semantics, but these gaps are minor given the schema's completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and all four parameters have clear descriptions in the schema. The tool description adds no parameter-specific semantics beyond what the schema already provides, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ("Search the MCPVault directory for MCP servers") and lists the returned fields (name, grade, status, star count, description, listing URL), clearly identifying it as a list/search operation. However, it does not explicitly differentiate from sibling tools like get_mcp_server or submit_mcp_server, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives. The verb 'search' implies its purpose, but the description never states 'use get_mcp_server for a single server's details' or any other selection criteria. With three siblings present, this lack of routing leaves the agent to infer the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_mcp_serverA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. 'Submit... as a new MCP server listing' implies a mutation, but it does not state any side effects, prerequisites like authentication, whether this opens a PR or directly modifies the directory, whether a duplicate is rejected, or what the response will be. This is a significant gap for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The core action is front-loaded and the usage guidance is appended efficiently, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter, so the description covers purpose and usage adequately. However, with no output schema and no annotations, the agent is left without knowing what happens after submission or what the return value means. This incomplete behavioral context lowers the score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because the single parameter has a full description with an example. The tool description only repeats 'GitHub repository' and adds no meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Submit') and names the exact resource ('GitHub repository to the MCPVault directory as a new MCP server listing'). This clearly distinguishes it from the sibling read/search tools because it is the only one that creates a listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Use this when you find an MCP server on GitHub that is not yet in MCPVault' provides an explicit condition for use. The 'not yet in MCPVault' phrase implies the negative case, but it does not explicitly name alternative sibling tools such as search_mcp_servers or get_mcp_server for checking existence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_install_commandA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It communicates a read-only 'Get' action and discloses the return payload (command and listing URL), but it does not mention authentication, rate limits, or error behavior. This is adequate for a simple lookup but leaves some behavioral unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first states exactly what the tool does, and the second specifies the returned items. There is no redundant phrasing or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers the input (slug) and the output (command and listing URL). It does not explain what a MCPVault slug is or handle edge cases, but those are minor gaps given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema already describes slug as 'MCPVault slug for the server.' The description only repeats this concept ('by its MCPVault slug') without adding examples, format, or edge-case guidance, so it adds no real value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get the install command for a specific MCP server by its MCPVault slug.' This clearly distinguishes it from siblings like search_mcp_servers or submit_mcp_server, and the return content ('command... plus the listing URL') is explicitly stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes a clear usage context: when you have a specific MCPVault slug and need the install command. While it doesn't name alternatives or state when not to use it, the purpose is narrow enough that an agent can infer when to call it versus searching or submitting servers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mcp_serverA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It clearly communicates that this is a read-only retrieval operation and discloses the fields returned. It does not mention error behavior or auth requirements, but for a simple lookup tool this is a minor gap rather than a serious omission.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and resource, then efficiently lists the returned fields. Every element adds value and there is no filler or redundant restating of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter retrieval tool, the description is adequately complete: it names the input and the expected result fields. It does not specify behavior for a nonexistent slug, but that is not essential for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the slug parameter thoroughly with examples and a listing URL. The description adds no additional parameter-specific semantics, so the baseline score of 3 for high schema coverage is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve'), a specific resource ('a specific MCP server'), and the key identifier ('MCPVault slug'). It also enumerates the returned details, which clearly distinguishes it from the sibling search_mcp_servers, get_install_command, and submit_mcp_server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'By its MCPVault slug' gives clear context for when to use this tool: when you already know the slug and need full details. It does not explicitly say to use search_mcp_servers when the slug is unknown, so the when-not/alternative guidance is left implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.