Server Quality Checklist

Tool Scores

• dbQueryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'execute a SQL query' which implies read/write operations but doesn't disclose behavioral traits like whether it supports transactions, what happens with DDL vs DML queries, error handling, or security implications. For a database tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized and front-loaded with the core functionality, making it easy for an agent to quickly understand what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, but does have an output schema (which means return values are documented elsewhere), the description is minimally adequate. It covers the basic purpose but lacks behavioral context and usage guidance that would be helpful for a database operation tool. The existence of an output schema reduces the need to explain return values, but other gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema (profile, query, parameters, timeoutMs, rowLimit). Baseline 3 is appropriate when the schema does the heavy lifting, though the description could have explained parameter relationships or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute a SQL query') and target resource ('PostgreSQL database using a configured profile'), providing specific verb+resource combination. However, it doesn't differentiate from sibling tools like sshExecute or trainClassifier, which operate on different systems entirely, so it doesn't need sibling differentiation but could mention it's for database operations specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this tool is appropriate (e.g., for database queries vs. other operations) or when not to use it (e.g., for non-SQL operations). With sibling tools like sshExecute for shell commands and trainClassifier for ML tasks, some basic differentiation would be helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trainClassifierB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions remote execution via SSH but lacks critical details: whether this is a read-only or destructive operation (training typically modifies models), authentication requirements beyond the 'profile' parameter, potential side effects (e.g., file system changes on remote host), rate limits, or error handling. The description is insufficient for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary elaboration. Every word earns its place by specifying the action, target, and mechanism concisely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (remote execution, 6 parameters, mutation likely required for training) and the presence of an output schema (which reduces need to describe return values), the description is minimally adequate. However, with no annotations and a mutation-oriented task, it should provide more behavioral context (e.g., safety warnings, prerequisites) to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema (e.g., it doesn't explain relationships between parameters like how 'commandTemplate' interacts with 'subclasses'). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run classifier training commands') and the mechanism ('on a remote host via SSH'), which is specific and actionable. It distinguishes from sibling tools like 'sshExecute' by focusing specifically on classifier training rather than general SSH execution. However, it doesn't explicitly differentiate from 'dbQuery' beyond the SSH context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'sshExecute' or 'dbQuery'. It doesn't mention prerequisites (e.g., SSH setup, classifier framework availability), nor does it specify scenarios where this tool is preferred over general SSH execution for training tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sshExecuteB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action. It fails to mention critical aspects like security implications, permission requirements, potential side effects (e.g., command execution risks), or error handling, which are essential for a tool that executes commands remotely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (remote command execution with 6 parameters) and the presence of an output schema (which handles return values), the description is minimally adequate. However, it lacks details on behavioral traits and usage guidelines, which are crucial for safe and effective use, leaving gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema fully documents all parameters. The description does not add any additional meaning or context beyond what the schema provides, such as examples or usage notes for parameters like 'profile' or 'timeoutMs'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Execute a command') and resource ('on a remote server via SSH using a configured profile'), distinguishing it from sibling tools like dbQuery and trainClassifier which involve database operations and machine learning tasks respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives or any prerequisites. The description lacks context about suitable scenarios or exclusions, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.