Server Quality Checklist

Tool Scores

• execute_commandB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the allowed-list constraint and suggests simple commands, but lacks critical details: it doesn't specify security implications (e.g., permissions required), error handling, output format, or potential side effects (e.g., whether commands can modify the system). For a command execution tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the core purpose. The second sentence adds useful constraints and examples, but could be slightly more structured (e.g., separating constraints from examples). Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (command execution with security implications), lack of annotations, and no output schema, the description is incomplete. It mentions an allowed list and simple commands but omits critical context: what the output looks like, error conditions, safety warnings, or how to check allowed commands (via 'list_allowed_commands'). This leaves significant gaps for an AI agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters ('command', 'workingDir', 'timeout') with descriptions. The description adds no additional parameter semantics beyond what's in the schema, such as examples of allowed commands or timeout behavior. Baseline is 3 when schema coverage is high and no extra param info is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute a Windows command and return its output.' It specifies the verb ('execute'), resource ('Windows command'), and outcome ('return its output'). However, it doesn't explicitly differentiate from sibling tools like 'execute_powershell' or 'list_allowed_commands', which would be needed for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Only commands in the allowed list can be executed' and 'This tool should be used for running simple commands like 'dir', 'echo', etc.' This implies when to use it (for simple commands) and hints at constraints, but it doesn't explicitly state when NOT to use it or name alternatives (e.g., use 'execute_powershell' for PowerShell scripts), so it falls short of explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_powershellB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions that the tool returns output and has a prerequisite (PowerShell in allowed commands), but lacks details on security implications, error handling, execution environment, or potential side effects. For a tool that executes arbitrary scripts, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that each serve a purpose - the first states the core functionality, the second adds an important constraint. It's front-loaded with the main purpose and avoids unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a potentially dangerous script execution tool with no annotations and no output schema, the description is insufficient. It doesn't explain what kind of output to expect, error conditions, security implications, or execution limitations. For a tool that could have significant system impact, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema - it mentions script execution generally but provides no additional context about parameter usage, constraints, or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a PowerShell script and returns output, specifying the verb 'execute' and resource 'PowerShell script'. It distinguishes from sibling tools like execute_command by specifying PowerShell specifically, though it doesn't explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating 'PowerShell must be in the allowed commands list', suggesting a prerequisite. However, it doesn't provide explicit guidance on when to use this tool versus alternatives like execute_command or other system tools, leaving usage context somewhat implied rather than clearly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_system_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool can 'provide basic or full details,' which adds some context about output granularity, but fails to address critical aspects such as whether this is a read-only operation, potential performance impacts, or authentication requirements, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of just two sentences that efficiently convey the core functionality and parameter options without any wasted words. Every sentence earns its place by adding relevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is minimally adequate but incomplete. It covers the basic purpose and parameter hint, yet lacks details on behavioral traits, usage context, or output format, which are needed for a more comprehensive understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'detail' fully documented in the input schema (including enum values and default). The description adds minimal value by mentioning 'basic or full details,' which aligns with the schema but doesn't provide additional semantic context beyond what's already structured.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve') and resources ('system information including OS, hardware, and user details'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'get_network_info' or 'get_service_info', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_network_info' or 'execute_command', nor does it mention any prerequisites or exclusions. It only hints at usage through the 'detail' parameter options, but this is insufficient for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_network_infoB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it states what information is retrieved, it doesn't describe important behavioral aspects like whether this requires administrative privileges, what format the information is returned in, whether it's a read-only operation, or any rate limits or constraints. The description is insufficient for a tool that presumably accesses system-level network configuration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences that each serve clear purposes: the first establishes the core functionality, and the second adds the filtering capability. There's no wasted language or redundancy, making it appropriately sized and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there are no annotations and no output schema, the description is incomplete for a tool that retrieves system network configuration. It doesn't explain what format the information is returned in, what permissions are required, whether it's safe to use, or what specific data structures to expect. For a system-level information retrieval tool, this leaves significant gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single optional parameter. The description adds marginal value by mentioning the filtering capability ('Can be filtered to a specific interface'), which aligns with the schema's description of 'Optional interface name to filter results.' No additional parameter semantics beyond what the schema provides are included.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve') and resources ('network configuration information'), and lists concrete examples of what information is retrieved (IP addresses, adapters, DNS settings). It distinguishes itself from siblings like get_system_info by focusing specifically on network configuration rather than general system information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning filtering capability ('Can be filtered to a specific interface'), but doesn't explicitly state when to use this tool versus alternatives like get_system_info or other sibling tools. No guidance is provided about when NOT to use this tool or what specific scenarios warrant its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scheduled_tasksB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions retrieving information, which implies a read-only operation, but doesn't disclose behavioral traits such as permissions required, potential rate limits, whether it returns real-time or cached data, or error conditions. For a system tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by key usage details. Every word earns its place without redundancy, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is minimally adequate for a read-only tool with well-documented parameters. It covers the basic purpose and parameter implications but lacks details on return values, error handling, or system-specific constraints. For a tool interacting with system tasks, more context would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters (e.g., 'Action to perform' for 'action' and 'Name of the specific task' for 'taskName'). The description adds some value by explaining that 'query' retrieves all tasks and 'status' gets detailed info for a specific task, but this mostly reiterates what the enum and schema imply. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Retrieve information') and resource ('scheduled tasks on the system'), making the purpose unambiguous. It distinguishes between querying all tasks and getting detailed status of a specific task, which is helpful. However, it doesn't explicitly differentiate from sibling tools like 'get_service_info' or 'list_running_processes' that also retrieve system information, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage guidelines by mentioning 'all tasks' vs. 'specific task', which correlates with the 'action' parameter. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get_service_info' for services or 'list_running_processes' for processes. No exclusions or prerequisites are stated, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the tool can 'retrieve information' and 'query all services or get detailed status,' it doesn't disclose important behavioral aspects like whether this requires administrative privileges, what format the information returns in, potential rate limits, or error conditions. The description provides basic functionality but lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with just two sentences that efficiently convey the tool's purpose and main functionality. Every word earns its place, and the information is front-loaded with the core purpose stated immediately. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only information retrieval tool with 2 parameters and 100% schema coverage but no output schema, the description provides adequate basic context about what the tool does. However, it doesn't explain what information is returned, the format of the response, or how to interpret the results. Given the lack of output schema and no annotations, more detail about the return value would be helpful for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds some semantic context by explaining that the tool can 'query all services or get detailed status of a specific service,' which maps to the action parameter's enum values. However, with 100% schema description coverage where both parameters are well-documented in the schema, the description doesn't provide significant additional value beyond what's already in the structured schema. The baseline of 3 is appropriate when the schema does most of the parameter documentation work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve information', 'query', 'get detailed status') and identifies the resource ('Windows services'). It distinguishes between querying all services and getting status for a specific service, but doesn't explicitly differentiate from sibling tools like get_system_info or get_network_info that might also retrieve system information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage scenarios (query all services vs. get status for specific service) through the action parameter explanation, but doesn't provide explicit guidance on when to use this tool versus alternatives like get_system_info or get_scheduled_tasks. No when-not-to-use guidance or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_running_processesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions filtering capability, it doesn't describe what information is returned about each process, whether the list is real-time or cached, permission requirements, or potential system impact. For a system query tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear, efficient sentences with zero waste. The first states the core purpose, the second adds the optional filtering capability. Perfectly front-loaded and appropriately sized for this simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only query tool with one well-documented parameter, the description is minimally adequate. However, with no annotations and no output schema, it should ideally provide more context about what process information is returned and any system considerations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single optional filter parameter. The description adds no additional parameter semantics beyond what's in the schema, maintaining the baseline score for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all running processes') and resource ('on the system'), with explicit mention of filtering capability. It distinguishes from siblings like execute_command or get_system_info by focusing specifically on process enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for viewing running processes, but provides no explicit guidance on when to use this tool versus alternatives like get_service_info or list_allowed_commands. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_allowed_commandsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates this is a read operation ('List') for informational purposes, which is clear but lacks details on rate limits, authentication needs, or output format. No contradiction exists, but it's minimal beyond basic intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that front-load the core purpose and follow with a brief utility statement. Every word earns its place with no redundancy or fluff, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, no output schema), the description is adequate but minimal. It covers the basic purpose and utility but lacks details on behavioral traits like response format or system-specific constraints, which could be helpful for an agent in this server context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the lack of inputs. The description doesn't need to add parameter details, and it correctly aligns with this by not mentioning any parameters, earning a baseline score for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('all commands that are allowed to be executed'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'execute_command' or 'get_system_info' beyond the general scope of permissions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating 'This helps understand what operations are permitted,' suggesting it's for checking permissions before executing commands. However, it lacks explicit guidance on when to use this versus alternatives like 'execute_command' or other system-info tools, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.