Server Quality Checklist

Tool Scores

• swmm_continuityC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It only states the action (parse) without revealing side effects, error conditions, or performance implications. The agent cannot know if the operation is read-only, what happens on missing data, or any authentication needs. The minimal description leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and front-loaded with the verb. However, it omits critical parameter semantics, which makes it less effective despite its brevity. A conciseness score of 3 reflects that it is not wasteful but fails to deliver essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter with 0% schema coverage, no annotations, and no output schema, the description is severely incomplete. It fails to specify the parameter's meaning, return format, or edge cases. The agent lacks sufficient information to invoke the tool correctly without external knowledge of SWMM .rpt files.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining the 'rpt' parameter. The description mentions 'SWMM .rpt' but does not clarify whether the parameter expects a file path, file content, or a reference. The agent cannot infer the expected format (string could be path, base64, or raw text) from the description alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: parsing Runoff Quantity and Flow Routing continuity tables from a SWMM .rpt file. The verb 'Parse' combined with the specific resource 'continuity tables' makes the purpose unambiguous. It distinguishes itself from sibling tools (swmm_compare, swmm_peak, swmm_run) which handle different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description only explains what the tool does, without mentioning prerequisites, context, or scenarios where this tool is preferred over siblings. Without such guidance, the agent must infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swmm_compareC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose all behavioral traits. It states 'compare continuity error (%)' but does not explain what the comparison entails (e.g., subtraction, percentage difference), whether the files are read or modified, or any side effects. This is insufficient for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, but it lacks necessary details. It is front-loaded with the action and resource, but the brevity comes at the cost of completeness. A longer description with parameter mapping and output info would be more useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and two required parameters, the description should explain what the tool returns (e.g., difference in continuity error) and clarify the role of each parameter. The current description does not provide enough context for an agent to use the tool correctly, especially when sibling tools exist but are not differentiated beyond basic purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so parameters have no descriptions. The tool description mentions 'two SWMM .rpt files' but does not map them to the parameters 'rpt' and 'rpt2'. The parameter names themselves are ambiguous (could be file paths or content). The example 'GUI vs CLI' does not clarify which parameter is which. This leaves the agent without enough information to correctly assign file paths.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'Compare', the resource 'continuity error (%)', and the files 'SWMM .rpt files', with an example 'GUI vs CLI'. It effectively distinguishes from sibling tools like swmm_continuity (one file) and swmm_peak (peaks) by focusing on comparison of two files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'two SWMM .rpt files' and the example 'GUI vs CLI', but it does not explicitly state when to use this tool versus alternatives (e.g., when comparing consistency) or when not to use it. No exclusions or alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swmm_peakC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states the tool parses from a .rpt file, but does not reveal error behavior (e.g., node not found), whether it modifies files, or required file structure. Critical missing details for a parse tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise at two sentences, front-loaded with the main action. Every word adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parse tool with 2 parameters and no output schema, the description is adequate but could include details on expected .rpt format (e.g., SWMM 5 output) or behavior on missing node. It covers essentials but lacks robustness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning: 'rpt' is the SWMM .rpt file, 'node' is the node/outfall name, and it must be supplied. This clarifies the purpose beyond the bare parameter names, though format or constraints are not specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states that the tool parses peak flow and time-of-peak from a SWMM .rpt file for a specific node/outfall. It is clear and specific, but does not differentiate from sibling tools like swmm_compare or swmm_run beyond the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions that the node name must be supplied, but provides no guidance on when to use this tool vs alternatives (e.g., swmm_run for simulation, swmm_compare for comparisons) or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swmm_runA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses writing output files and auto-detecting outfall node when omitted, but lacks detail on side effects like file overwriting, error handling, or runtime. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the primary action and adding key auto-detection detail. Every word is purposeful, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity, no output schema, and no annotations, the description omits important context such as return value format, error conditions, typical runtime, and details about output file structure. It is insufficient for fully understanding tool behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is low (33%), so description must compensate. It explains the 'node' parameter's auto-detection behavior and mentions 'inp' and 'runDir' by context. However, parameters like 'rptName' and 'outName' are not described beyond their presence in file names, leaving some semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs swmm5 on an INP file and writes output files into runDir. It mentions specific outputs (rpt, out, manifest.json) and distinguishes from siblings like swmm_compare, swmm_continuity, and swmm_peak by focusing on simulation execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for running a simulation but does not explicitly state when to use this tool versus its siblings. No guidance on prerequisites or when not to use it is provided, though the sibling names offer indirect context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.