Server Quality Checklist

Tool Scores

• poem_translateA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states that the tool parses and validates, and returns a structured brief, which is useful. However, it does not disclose behaviors like what happens on validation failure or whether the operation is read-only, though parsing/validating implies no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, using three sentences plus a target list. The 'YOU' and 'The tool handles...' lines are valuable because they clarify the division of labor between the tool and the LLM. Minor redundancy ('Parses...validates' vs 'handles parsing and validation') is acceptable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the overall workflow well, but with no output schema, it does not describe the structure or content of the translation brief, which is critical for the LLM to use it effectively. It also omits error-handling behavior for invalid POEM content. More detail would make it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds a supported-targets list that supplements the 'target' parameter, but it does not add meaningful detail about the parameters beyond what the schema already provides, such as the structure of the translation brief.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Prepare a .poem spec for translation') and elaborates that it parses, validates, and returns a translation brief. This distinguishes it from sibling tools poem_read and poem_validate, which focus on reading and validation alone.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this tool to prepare a translation brief, then the LLM uses that brief to generate code. It does not explicitly exclude alternatives like poem_validate, but it clarifies the tool handles validation within the translation workflow, implying the correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• poem_readA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the parse-and-return behavior and mentions the output ('parsed structure'), implying a non-destructive read operation. However, it omits details like error handling or what happens with malformed input, leaving room for more explicit behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences: the first defines the tool's function and return value, the second gives usage context. No redundancy or fluff; the description is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read/parse tool with no output schema, the description adequately covers the return value and primary use case. It could mention that it does not validate or translate (which siblings do), but its context of understanding/discussing/translating already hints at that. The coverage is solid given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already fully documented. The description does not add extra meaning beyond the schema; it merely restates the 'source' as .poem file content, which matches the schema description. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Read and parse a .poem file') and resource (structured elements: constants, structs, functions, etc.), clearly distinguishing it from sibling tools like poem_validate and poem_translate by emphasizing parsing for understanding/discussion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: 'Use this when someone shares a .poem file or asks about its contents.' This signals appropriate usage without explicitly excluding alternatives or naming them, but the context alone is sufficient for a read/parse tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• poem_validateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It explains what is checked (brace balance, naming conventions, structural completeness) and what is returned (errors, warnings, stats), providing useful context beyond a simple 'validates' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, specific checks, return types. Every sentence adds value, no filler, and the structure is logical and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter validation tool with no output schema or annotations, the description covers the input, validation criteria, and return categories. It could specify the output structure in more detail, but it is adequate for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% because the only parameter 'source' has a clear description. The tool description adds little beyond the schema, but it is consistent with the parameter's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a .poem file for syntax correctness and convention compliance, listing specific checks. It is distinct from siblings poem_read and poem_translate, which handle reading and translation respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to validate .poem files, which is clear in context. It does not explicitly mention when not to use it or name alternatives, but the sibling tools make the use case obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.