Server Quality Checklist

Tool Scores

• execute_mcp_toolA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It adds useful context about on-demand server spawning, connection pooling, and configuration file locations, but it does not disclose error handling, potential latency, or permission requirements. This is moderate transparency but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with the main purpose front-loaded and two supplementary sentences about proxy mechanics. Every sentence adds value with no redundancy, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generic executor tool with an output schema and nested object parameters, the description covers the core purpose, operational model, and configuration sources. It could mention that arguments must conform to the target tool's schema, but the input schema already states that, so the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to detail parameters. The description adds no extra information about mcp_name, tool_name, or arguments, and the schema already provides sufficient descriptions. This matches the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes any available MCP tool through a proxy, using a specific verb and resource. It distinguishes itself from sibling tools (search, list, management) by calling itself the 'universal gateway' for invoking tools, which is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for calling MCP tools but does not explicitly state when to prefer it over alternatives like search_tools or list_available_mcps. It provides context about the proxy's behavior but no explicit 'when to use' or 'when not to use' guidance, which places it at an implied-usage level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reset_learned_weightsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly warns that the action erases all accumulated learning, disclosing its destructive nature. Since no annotations are provided, this is essential behavioral context beyond the schema. However, it does not mention the confirmation parameter or other side effects, though the schema covers confirmation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, with just two sentences: the primary action and a clear warning. It is front-loaded with purpose and includes a critical warning, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and an output schema, the description covers the core purpose and the critical destructive behavior. The confirmation requirement is documented in the schema, so the description need not repeat it. A minor gap is not stating what happens if confirm is false, but that is inferable from the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the confirm parameter with 'Must be True to confirm the reset', and the description adds no additional parameter-specific meaning. With 100% schema description coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it resets all learned weights, which is a specific verb+resource action. It distinguishes from sibling tools like get_learned_weights (which reads) and refresh_tool_index (which refreshes indexes), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use the tool versus alternatives, nor any exclusions or prerequisites. The warning implies caution but does not specify a usage context or compare with other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_tool_indexA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the mechanical behavior (connecting, retrieving, indexing) but omits side effects like whether the old index is replaced, network dependency, or failure modes. Since it's a refresh operation, some mutating behavior is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with the heading as the core summary, followed by a one-line elaboration and a clear usage note. Every sentence earns its place, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional param, output schema exists), the description covers purpose, usage timing, and basic behavior. It omits details about what happens during refresh, but the output schema likely provides return info. A slightly more explicit statement about side effects would make it a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the mcp_name parameter clearly documented as optional and defaulting to all. The description does not add further parameter-specific details, but the schema fully handles semantics, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: 'Refresh the tool index for semantic search' with explicit action of connecting to MCP servers and retrieving tool lists. It distinguishes itself from siblings like search_tools or execute_mcp_tool by focusing on index maintenance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use after adding new MCPs' and notes the index is auto-loaded on startup, which implies when manual refresh is unnecessary. Does not explicitly name alternatives but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_toolsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It mentions semantic similarity and the search intent but doesn't describe output ranking, result limits, or edge cases. There's an output schema that may cover returns, but the description itself adds only basic context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the core function and add a practical use case. Every word earns its place; no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with an output schema and clear parameter descriptions, the description provides sufficient context. It could mention what happens with no results or whether it uses learned weights (given sibling tools), but overall it's complete enough for the task.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with query, mcp_filter, and max_results all described in the schema. The description adds no extra meaning to parameters, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches tools by objective or description using semantic similarity, which is a specific verb+resource+method. It distinguishes from siblings like list_available_mcps (which lists all tools) and execute_mcp_tool (which runs tools).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes it's useful when you don't know the exact tool name but know what you want to do, providing clear context for when to use it. It doesn't explicitly state exclusions or alternatives, but the use case is clear enough to differentiate from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_mcpsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses the output (names and active connection status) but does not clarify whether the status is live or cached, nor does it mention authentication requirements or any side effects. For a read-only list tool, this is acceptable but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two sentences, with the primary purpose stated immediately and the return value elaborated concisely. There is no redundant or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (zero parameters, simple output) and the presence of an output schema, the description fully explains the functionality and return value. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (vacuously). The description correctly avoids parameter details, aligning with the baseline for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and the resource (configured MCP servers) with an additional specific detail (connection status). It is easily distinguished from sibling tools like execute_mcp_tool or close_mcp_connections, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied by the verb 'list' and the tool's role among siblings, but there are no explicit guidelines about when to use this tool instead of alternatives like search_tools or refresh_tool_index. The description does not provide any exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_learned_weightsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly states this is a 'View' operation, implying a non-destructive read, and describes the content it returns (tool selections and keywords). This gives enough transparency for an agent to understand the tool's behavior and side-effect profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and every clause adds value. No redundant information or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters and an output schema. The description explains the purpose and what is displayed, which is sufficient for an agent to know when and how to call it. Could mention that it is read-only explicitly, but the 'View' wording covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (top_n and tool_filter) with clear descriptions. The tool description does not add extra meaning to the parameters, but it doesn't need to because the schema is sufficient. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('View') and specifies the resource ('learned weights from the search system'). It further explains what the tool shows (which tools selected, for which keywords), clearly distinguishing it from the sibling reset_learned_weights and other actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states that this tool is for viewing learned weights and explaining the system's learned behavior, which gives clear context for when to use it. It does not explicitly exclude alternatives, but the read-only nature is evident and no other sibling tool would serve this purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_mcp_connectionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses that the tool closes all active MCP connections and adds the important reassurance that connections are re-established on next use, indicating the action is reversible and safe. This goes beyond a bare description and effectively communicates the tool's side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, with the primary action stated first. The second sentence gives usage guidance and the third explains the recovery behavior. There is no redundant information, and every sentence contributes meaningful value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no annotations), the description is fully complete. It covers what the tool does, when to use it, and the key behavior that connections are re-established, which addresses the main concerns an agent might have. The presence of an output schema means return value details are not required here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is empty and the schema description coverage is 100% by default. Per the rubric, a baseline score of 4 applies for 0-parameter tools, and no additional parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Close all active MCP connections' with a clear verb and resource, making the tool's purpose immediately obvious. This is distinct from the sibling tools which focus on execution, listing, and searching, so there is no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using the tool 'to clean up resources when done with MCP tools', providing a clear temporal context. It also notes connections will be re-established on next use, which helps the agent understand that this is a safe, non-permanent cleanup action. However, it does not explicitly mention any scenarios where the tool should not be used, though such exclusions are less critical for a cleanup utility.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.