Server Quality Checklist

Tool Scores

• mcp_call_mac_system_profilerA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool's function and lists datatypes, but doesn't disclose behavioral traits like whether it's read-only, requires permissions, has rate limits, or what the output format is (though an output schema exists). The description adds some context but misses key operational details for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first two sentences, followed by a useful list of datatypes. Every sentence adds value, though the list is lengthy but necessary for clarity. It could be slightly more structured but remains efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, no annotations, and an output schema (which reduces the need to describe return values), the description is fairly complete. It explains the purpose, usage context, and details all parameter options. However, it lacks behavioral transparency details (e.g., safety, permissions), which holds it back from a perfect score despite the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage and only one parameter ('datatype') with no enum. The description compensates fully by listing all allowed datatypes with brief explanations (e.g., 'SPAirPortDataType - Airport/WiFi information'), adding significant meaning beyond the bare schema. This is excellent compensation for low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Call the system_profiler with the given datatype' and 'get the system information to help user to understand the system and potentially debug.' It specifies the verb ('call'), resource ('system_profiler'), and context ('system information'), though it doesn't explicitly differentiate from sibling tools like mcp_call_conda_info or mcp_call_gpu_available, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('to help user to understand the system and potentially debug') and lists allowed datatypes, which suggests when to use it for specific information types. However, it lacks explicit guidance on when to choose this tool over alternatives (e.g., sibling tools) or any exclusions, making it moderately helpful but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_call_gpu_availableA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it runs benchmarks, returns detailed diagnostic information, and helps determine GPU configuration status. However, it doesn't mention potential side effects, performance impact, or error conditions that might occur during execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It begins with the core purpose, explains parameters, details the return structure, and concludes with the tool's value. Every sentence adds necessary information without redundancy, and the return value documentation is efficiently organized in bullet points.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a diagnostic tool with no annotations and no output schema, the description provides comprehensive information about purpose, parameters, and return values. The detailed return structure documentation compensates for the lack of output schema. However, it doesn't cover potential error cases or execution constraints that might be relevant for a system diagnostic tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the schema's lack of parameter documentation. It successfully explains both parameters: 'env_name' is implied through 'specific conda environment,' and 'framework' is explicitly described with its default behavior ('if framework is not provided, it will default to torch'). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Check if GPU is available') and resource ('in torch for a specific conda environment'), with specific scope about Apple's Metal acceleration. It distinguishes from sibling tools like 'mcp_call_conda_info' and 'mcp_call_mac_system_profiler' by focusing on GPU availability testing rather than general conda or system information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('to determine if GPU acceleration via Apple's Metal is properly configured and functioning'), but doesn't explicitly state when NOT to use it or mention specific alternatives. The framework parameter guidance ('if framework is not provided, it will default to torch') offers some usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_call_conda_infoA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (detailed information across multiple categories), its optional parameter behavior ('If env_name is provided...'), and its practical use cases. It doesn't mention performance characteristics or error conditions, but covers the core behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose statement, parameter behavior explanation, detailed return value breakdown, and use case. Every sentence adds value, and the bulleted list efficiently communicates scope without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (system diagnostic tool with detailed output), the description provides comprehensive coverage: purpose, parameter semantics, detailed output breakdown, and use cases. With an output schema present, the description appropriately focuses on explaining what information is returned rather than technical return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only one parameter, the description adds significant value beyond the schema. It explains that 'env_name' is optional and specifies what happens when it's provided ('returns the information for the specified environment as well'), including that it affects the package list output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get comprehensive information about the Conda installation') and resource ('this system'), distinguishing it from sibling tools like GPU availability or system profiler calls. It provides a detailed scope of what information is retrieved.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('useful for diagnosing Conda-related issues or understanding the Python environment configuration'), providing clear context. However, it doesn't specify when NOT to use it or mention alternatives to this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.