Server Quality Checklist

Tool Scores

• get_error_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that it retrieves common error messages, omitting important behaviors like default sorting, date filtering, pagination limits, or whether this is a read-only operation. The schema hints at options like sort by userCount or lastSeen, but the description does not contextualize these behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no extraneous words. It is front-loaded with the main purpose. However, it is arguably too terse for a tool with six parameters, but this dimension rewards efficiency, and the description is appropriately sized without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (six parameters, no annotations, no output schema), the description is inadequate. It fails to provide context about how this tool differs from siblings, what the results look like, or when to use it. The description is a minimal stub that leaves the agent without essential decision-making information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is fully documented within the schema itself. The description adds little beyond the schema—it does not explain how parameters map to the 'most common' concept beyond what the sort enum already provides. Baseline 3 is appropriate since the schema already handles parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get most common error messages' uses a specific verb and resource, clearly indicating a retrieval tool focused on aggregated error messages. It distinguishes itself from siblings like get_errors by emphasizing 'most common', suggesting a frequency-based aggregation. However, it could be more explicit about what fields are returned and how 'most common' is determined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. Sibling tools get_errors and get_error_details are not mentioned, and there are no hints about appropriate contexts for using this tool, such as identifying top error messages for triage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_errorsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states 'Retrieve' and 'optional filtering' without disclosing default behavior (e.g., default limit, date range defaults), pagination, return format, or any operational nuances. This is minimal disclosure for a retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words, making it concise and well-structured. However, it omits important context that could be included compactly, so it is not a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 optional parameters, no output schema, and no annotations, the description is too sparse. It does not explain what the default behavior is, how filters combine, or what the return object looks like, leaving an agent under-informed for selecting and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides descriptions for 100% of parameters, so the description does not need to add parameter-level detail. The description's 'optional filtering' is generic and does not add meaning beyond the schema's explicit parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource ('Retrieve errors from TrackJS') and mentions optional filtering. However, it does not differentiate this tool from siblings like get_error_details or get_error_messages, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The phrase 'with optional filtering' refers to parameters, not usage context, and no exclusions or alternative tool references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_error_detailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It does not state that the operation is read-only, whether any authorization is needed, what 'detailed information' includes, or what the response format looks like. The description only asserts it retrieves details, leaving key behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's purpose without filler. It front-loads the action and resource clearly, and every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter schema and no output schema, the description should at least clarify what 'detailed information' means or what the response will include. It does not, and it also fails to differentiate from sibling tools. The tool is minimally described but not complete for an agent to know what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the only parameter (errorId) with a clear description, achieving 100% schema description coverage. The description adds no extra meaning beyond 'specific error', so the baseline of 3 applies; the schema already provides the necessary semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get') and resource ('detailed information about a specific error'), which clearly identifies it as a read tool for a single error. It is implicitly distinguished from siblings 'get_errors' (plural) and 'get_error_messages' (messages), but it does not explicitly name alternatives or differentiate the 'detailed information' from message-level data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied by the word 'specific' and the singular 'errorId' parameter, suggesting use when retrieving details for one error. However, there is no explicit guidance on when to use this tool versus alternatives like 'get_errors' or 'get_error_messages', and no exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.