Server Quality Checklist

Tool Scores

• get_terminal_infoC
  2.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Gets terminal information', implying a read-only operation, but doesn't specify what information is returned (e.g., format, structure), whether it requires permissions, or if there are rate limits. The description is too vague to provide meaningful behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description 'Gets terminal information' is concise with a single sentence, but it's under-specified rather than efficiently informative. It lacks front-loaded detail that could clarify the tool's purpose, making it too brief to be truly helpful. While not verbose, it fails to earn its place by providing sufficient value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain what 'terminal information' entails, the return format, or any behavioral aspects. For a tool with no structured data to rely on, the description should provide more context to be fully usable by an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the input schema has 100% description coverage (though empty). With no parameters to document, the description doesn't need to add parameter semantics. A baseline score of 4 is appropriate as the schema fully covers the parameterless nature, and the description doesn't introduce confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Gets terminal information' is a tautology that essentially restates the tool name 'get_terminal_info'. It provides a verb ('Gets') and resource ('terminal information'), but lacks specificity about what information is retrieved (e.g., terminal type, dimensions, capabilities) and doesn't differentiate from sibling tools like 'execute_command' or 'file_modify'. This makes it vague and minimally informative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions, and fails to distinguish it from sibling tools such as 'execute_command' (for running commands) or 'file_modify' (for file operations). This absence of usage instructions leaves the agent without direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_commandC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without disclosing behavioral traits. It doesn't mention permissions needed, side effects (e.g., file changes, system impacts), error handling, or output format, which is critical for a command execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, though this brevity contributes to underspecification rather than clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with no annotations, no output schema, and 0% schema coverage, the description is incomplete. It lacks essential details about execution context, safety, and results, making it inadequate for informed tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but adds no parameter information. It doesn't explain what 'command' should contain, how 'wait_for_output' affects behavior, or what 'timeout' units are (seconds?), leaving all three parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Executes a terminal command' states the basic action (verb+resource) but is vague about scope and lacks differentiation from siblings like 'get_terminal_info'. It doesn't specify what kind of commands or execution environment, making it minimally adequate but unclear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'file_modify' or 'get_terminal_info'. The description implies execution but doesn't mention prerequisites, safety considerations, or typical use cases, leaving the agent with no contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_modifyC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'Writes content to a file' indicates a mutation operation but fails to detail critical aspects such as permissions required, error handling (e.g., if the file doesn't exist), side effects, or response format. It doesn't contradict annotations (none exist), but offers minimal behavioral insight beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with 'Writes content to a file', a single sentence that front-loads the core action without unnecessary words. It's appropriately sized for the basic purpose, though this brevity contributes to gaps in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, mutation tool, no annotations, no output schema), the description is incomplete. It doesn't explain parameter roles, behavioral traits, or return values, leaving the agent with insufficient information to use the tool effectively. The conciseness comes at the cost of essential details needed for proper invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning parameters like 'filepath', 'content', 'mode', 'position', and 'create_dirs' are undocumented in the schema. The description adds no meaning beyond the basic action, failing to explain what these parameters do, their formats, or how they affect the write operation. For a tool with 5 parameters, this is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Writes content to a file' clearly states the verb ('writes') and resource ('a file'), making the basic purpose understandable. However, it lacks specificity about what kind of writing (e.g., overwriting, appending) and doesn't differentiate from potential sibling tools like 'execute_command' or 'get_terminal_info', which are unrelated to file operations. It's not tautological but remains somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are no explicit instructions on scenarios for application, prerequisites, or comparisons with sibling tools (e.g., 'execute_command' for command execution vs. file writing). Usage is implied by the action but lacks contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.