Server Quality Checklist

Tool Scores

• add_to_whitelistC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool adds to a whitelist, implying a write operation, but doesn't cover critical aspects like permissions required, whether it overwrites existing entries, error conditions, or side effects. This leaves significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that efficiently conveys the core action without any wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a write operation with security implications), lack of annotations, and no output schema, the description is insufficient. It doesn't explain what happens after adding (e.g., success response, error handling), how it integrates with the security system, or prerequisites, leaving the agent with incomplete context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for all three parameters, including an enum for 'securityLevel'. The description adds no additional parameter information beyond what's in the schema, so it meets the baseline score of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add') and resource ('command to the whitelist'), making the purpose understandable. However, it doesn't distinguish this tool from sibling tools like 'update_security_level' or 'approve_command', which might have overlapping functionality in a security context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't clarify if this is for initial whitelisting versus updating existing entries, or how it relates to siblings like 'update_security_level' or 'remove_from_whitelist'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_commandC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a mutation ('Approve') but doesn't specify permissions required, whether the action is reversible, or what happens after approval (e.g., does it trigger execution?). This leaves critical behavioral traits unclear for a tool that likely changes system state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It lacks details on behavioral outcomes, error conditions, or integration with sibling tools (e.g., how approval relates to 'execute_command'), leaving gaps in understanding the tool's role in the broader context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'commandId' well-documented in the schema. The description adds no additional meaning about the parameter beyond what the schema provides, such as format examples or sourcing guidance, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Approve') and the target resource ('a pending command'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'deny_command' beyond the opposite action, missing explicit differentiation that would warrant a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'deny_command' or 'execute_command', nor does it mention prerequisites such as needing a pending command from 'get_pending_commands'. This lack of contextual direction leaves the agent without usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deny_commandC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. 'Deny' implies a mutation that changes command status, but the description doesn't disclose behavioral traits like required permissions, whether denial is reversible, what happens to the denied command, or any side effects. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a command denial operation (a mutation with potential security implications), no annotations, no output schema, and sibling tools like 'approve_command', the description is incomplete. It lacks context on prerequisites, consequences, alternatives, or return values, leaving significant gaps for an AI agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (commandId and reason) adequately. The description doesn't add any meaning beyond what the schema provides, such as explaining what constitutes a valid reason or how the commandId is obtained. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Deny a pending command' clearly states the action (deny) and target resource (pending command). It's specific and unambiguous, though it doesn't explicitly differentiate from sibling tools like 'approve_command' or 'execute_command' beyond the verb choice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a pending command ID), when denial is appropriate, or how it differs from 'approve_command' or other command-handling tools in the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_commandC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but lacks critical details such as security implications, permission requirements, whether it's destructive, rate limits, or output format. This is a significant gap for a tool that executes shell commands, which can have high-risk behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded and appropriately sized for its purpose, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing shell commands (potentially high-risk with no annotations) and lack of output schema, the description is incomplete. It fails to address security, permissions, or return values, leaving the agent with insufficient context for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for 'command' and 'args'. The description doesn't add any parameter-specific details beyond what the schema provides, such as examples or constraints, so it meets the baseline for high schema coverage without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute') and resource ('a shell command on the current platform'), making the purpose understandable. However, it doesn't distinguish this tool from its siblings like 'approve_command' or 'deny_command', which appear to be related to command management but have different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. Given siblings like 'approve_command' and 'deny_command', it's unclear if 'execute_command' requires approval, operates independently, or has specific prerequisites, leaving the agent without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_from_whitelistC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool removes a command from a whitelist, implying a mutation operation, but doesn't describe side effects (e.g., whether this affects pending commands or security levels), permissions required, or error conditions. This leaves significant gaps in understanding how the tool behaves beyond its basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the core action and resource, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's mutation nature (removing from a whitelist), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like what happens after removal, error handling, or integration with sibling tools (e.g., 'get_whitelist'), leaving the agent with insufficient context for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'command' documented as 'The command to remove from whitelist'. The description doesn't add any meaning beyond this, such as format examples or constraints, but the schema provides adequate baseline information, justifying a score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('remove') and target resource ('a command from the whitelist'), making the purpose immediately understandable. It doesn't explicitly differentiate from siblings like 'deny_command' or 'update_security_level', which might have overlapping security functions, but the verb+resource combination is specific enough for basic understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'deny_command' or 'update_security_level', nor does it mention prerequisites (e.g., whether the command must already be in the whitelist). It simply states what the tool does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_security_levelC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an update operation but doesn't clarify whether this requires admin permissions, if changes are reversible, what happens to existing security settings, or potential side effects. For a security-related mutation tool, this lack of transparency is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a security-level update tool with no annotations and no output schema, the description is incomplete. It doesn't address behavioral aspects like permissions, side effects, or return values, nor does it help differentiate from sibling tools. The agent would lack sufficient context to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters clearly documented in the schema. The description doesn't add any meaningful parameter semantics beyond what the schema already provides (e.g., it doesn't explain what 'whitelisted command' means or provide examples). The baseline score of 3 reflects adequate but unenhanced parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update') and the target resource ('security level of a whitelisted command'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'approve_command' or 'deny_command', which might have overlapping functionality in a security context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'add_to_whitelist' or 'remove_from_whitelist'. It doesn't mention prerequisites (e.g., whether the command must already be whitelisted) or contextual constraints, leaving the agent with insufficient usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_platform_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves information, implying a read-only operation, but doesn't specify what information is returned (e.g., OS version, shell type, environment details), whether it requires permissions, or if there are rate limits. This leaves significant gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action ('Get information'), making it easy to parse, and every part of the sentence contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that retrieves system information. It doesn't explain what specific data is returned (e.g., platform details, shell version) or the format of the output, which is critical for an agent to use the tool effectively. This leaves too much ambiguity for practical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any. A baseline of 4 is applied as per the rules for tools with no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('information about the current platform and shell'), making it immediately understandable. However, it doesn't explicitly differentiate this tool from its siblings (like 'execute_command' or 'get_pending_commands'), which focus on command execution and management rather than platform metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context (e.g., use for system diagnostics or compatibility checks), or exclusions, leaving the agent to infer usage based on the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_whitelistB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does without behavioral details. It doesn't disclose whether this is a read-only operation, if it requires authentication, rate limits, or what format the returned list has. This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose with no wasted words. It's perfectly front-loaded and appropriately sized for a simple retrieval tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely returns structured data. It doesn't explain what 'whitelisted commands' entails, the return format, or any behavioral context, leaving significant gaps for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter semantics, but that's appropriate here, warranting a baseline score above minimum viable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('list of whitelisted commands'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_pending_commands' or 'get_platform_info' beyond the resource name, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_pending_commands' or 'add_to_whitelist'. There's no mention of prerequisites, context, or exclusions, leaving usage entirely implicit from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pending_commandsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool retrieves pending commands but doesn't mention whether this requires special permissions, how results are formatted, if there are rate limits, or what happens if no commands are pending. For a security/approval-related tool, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a simple retrieval tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description provides the minimum viable information about what it does. However, given the security/approval context and sibling tools that suggest this is part of a command management system, more information about permissions, return format, or typical usage patterns would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't mention parameters since none exist, which is correct. Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('list of commands pending approval'), making the purpose immediately understandable. It doesn't specifically differentiate from siblings like 'execute_command' or 'approve_command', but the verb+resource combination is unambiguous in context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives like 'execute_command' or 'approve_command'. The description only states what it does, not when it should be used in relation to the sibling tools that manage command approval workflows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.