Server Quality Checklist

Tool Scores

• suggest_minimal_setC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description only states it returns the smallest set of servers. It does not disclose side effects, idempotency, required input format, or behavior when no solution exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear purpose statement and a minimal Args block. No redundancy, but the structure could include more details without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Missing critical information: no output schema, no description of return value (e.g., list of server IDs?), no criteria for selection, and no edge cases. The tool's behavior is underspecified for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%. The description's Args section merely repeats 'task_description: Beschreibung der Aufgabe', adding no format, example, or constraints beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it recommends the minimal server combination for maximum efficiency, distinguishing it from siblings like optimize_server_set. However, it could be more specific about the output format or criteria for minimality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like optimize_server_set or analyze_task. Does not mention when not to use it or any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_context_usageA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden of behavioral disclosure. It mentions the tool 'estimates' but does not clarify if it is a read-only operation, whether it validates server names, or what happens if a server doesn't exist. The output format is also unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences plus an args line. It front-loads the purpose and immediately follows with a helpful elaboration. The only minor issue is that it’s in German, which might be inconsistent with the English tool name, but it’s still clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain what the tool returns (e.g., a numeric estimate, a percentage, or a range). It only states 'estimate context window usage' without mentioning output format, units, or success/error cases. This gap makes it less complete for an agent to understand post-invocation behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds essential meaning to the single parameter 'server_names'. It explains it as a list of server names and provides an example (['solana', 'weather']), which clarifies the input format. However, it does not specify constraints like valid server names or if the list should be from the available catalog.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to estimate the context window consumption for a combination of MCP servers. It uses a specific verb ('estimate') and resource ('context-window consumption'), and the sibling tools (analyze_task, get_server_catalog, etc.) are distinct in their functions, so this tool is well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: when you need to understand how much context budget a particular server combination consumes before using them. However, it does not explicitly state when not to use it or suggest alternative tools for different scenarios, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimize_server_setA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It states the tool 'analysiert' (analyzes) which servers can be removed, but it is unclear whether it actually performs removal or merely returns a suggestion. The lack of explicit side-effect disclosure is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description begins with a concise one-sentence summary, then includes a structured Args section. While slightly verbose, it is efficient and well-organized. Could trim the German repetition, but overall appropriate length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain what the tool returns. It describes analysis but does not mention the output format or return value. For a tool with two simple parameters, this is a notable omission, leaving the agent uncertain about the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description includes an Args section that adds meaning to both parameters: 'current_servers' is explained as 'Aktuell geladene Server-Namen' and 'task_description' as 'Was soll erledigt werden'. This provides context beyond the schema's minimal labels.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: optimizing a server set by analyzing which servers are necessary and which can be removed. The verb 'optimize' and resource 'server set' are specific, and the description distinguishes it from siblings like 'analyze_task' and 'suggest_minimal_set' by focusing on removal of unnecessary servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a task and a set of loaded servers, but it does not explicitly state when to use this tool vs. alternatives like 'suggest_minimal_set'. No exclusions or prerequisites are mentioned, leaving ambiguity about optimal context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_catalogA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It describes the output content (servers, categories, tool count, token consumption) but does not mention whether the operation is read-only, requires authentication, or has rate limits. The behavior is somewhat implied but not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, with the primary purpose stated first. Every sentence adds value, though it could be slightly more structured (e.g., listing output fields). It is appropriately sized for a simple parameterless tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description adequately explains the return content: all servers, categories, tool count, and estimated token consumption. It is complete enough for the agent to understand what it will receive, though it lacks details on possible pagination or ordering.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the input schema is empty. According to guidelines, a baseline of 4 is appropriate. The description does not need to add parameter information since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a complete catalog of all known MCP servers with categories, tool count, and estimated token consumption. It provides a specific verb (implied 'get') and resource (server catalog), and distinguishes itself from sibling tools that focus on analysis or optimization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. The description does not mention any conditions, prerequisites, or when-not-to-use scenarios, leaving the agent to infer usage solely from the tool's name and purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_taskA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool's behavior (determining relevant servers, estimating token consumption, giving optimization recommendations) but does not disclose whether it is read-only, requires authentication, or has side effects. The description is moderately transparent but lacks completeness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with the main purpose front-loaded. However, it repeats information in German and English, which adds redundancy. The structure is clear but could be more efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the one-parameter input and sibling tools, the description covers the tool's purpose and parameter. However, it does not describe the output format or return value (no output schema), leaving a gap. Additionally, it does not explain how this tool fits with siblings like 'optimize_server_set' or 'suggest_minimal_set'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description must add meaning. It explains that 'task_description' is a description of the task and provides an example ('Check SOL token safety'). This adds value beyond the schema, though it could specify format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool analyzes a task and recommends optimal server combinations, determines relevant MCP servers, estimates token consumption, and gives optimization recommendations. This clearly distinguishes it from siblings like 'estimate_context_usage' or 'get_server_catalog'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for analyzing tasks and getting server recommendations, but it provides no explicit guidance on when to use this tool versus alternatives like 'optimize_server_set' or 'suggest_minimal_set'. No when-not-to-use instructions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.