Server Quality Checklist

Tool Scores

• resolve_glitchtip_issueA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It correctly identifies this as a mutation operation ('Mark as resolved'), but doesn't address important behavioral aspects like whether this requires specific permissions, whether the resolution is reversible, what happens to the issue after resolution, or potential rate limits. The description adds some context about timing but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with just two sentences that each serve a clear purpose: the first states the action, and the second provides usage context. There's no wasted language or redundancy, and the information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description provides adequate basic information about what the tool does and when to use it. However, it lacks important context about what 'resolved' means in the system, what the response looks like, or any error conditions. The description is complete enough for basic understanding but has clear gaps for a mutation operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'issue_id' clearly documented in the schema. The description doesn't add any additional parameter information beyond what the schema already provides, such as format examples or constraints. With complete schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Mark as resolved') and target resource ('GlitchTip issue'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this from sibling tools like 'get_glitchtip_issue' or 'get_glitchtip_issues' beyond the obvious action difference, so it doesn't reach the highest tier of sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual guidance about when to use this tool ('after you've fixed the bug causing the error'), which helps the agent understand the appropriate timing. However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools, preventing a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_glitchtip_issueA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what information is returned ('complete stacktrace, error counts, and timing information'), which adds useful context beyond the basic read operation implied by 'Get'. However, it doesn't cover aspects like authentication needs, rate limits, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with three concise sentences that each add value: stating the purpose, providing usage guidance, and listing returned information. There is no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is reasonably complete. It covers purpose, usage, and return details, though it could be enhanced with more behavioral context (e.g., error cases) or explicit sibling differentiation to achieve a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single parameter 'issue_id' as a numeric string. The description doesn't add any parameter-specific details beyond what the schema provides, such as format examples or constraints, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get detailed information') and resources ('GlitchTip issue'), including key details like 'full stacktrace'. It distinguishes from the sibling 'get_glitchtip_issues' by focusing on a single issue, though it doesn't explicitly name that sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('when you need to investigate a specific error in detail'), which implicitly distinguishes it from the sibling 'get_glitchtip_issues' (for listing issues). However, it doesn't explicitly mention when not to use it or name alternatives like 'resolve_glitchtip_issue'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_glitchtip_issuesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that this is a read operation (listing) and describes the return format (issue titles, counts, status, first/last seen timestamps). However, it doesn't mention pagination behavior, rate limits, authentication requirements, or error conditions that would be important for a production monitoring tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three focused sentences: one stating the purpose, one providing usage context, and one describing the return format. Every sentence adds value without redundancy, and the information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema, the description provides adequate context about what the tool does and what it returns. However, it lacks details about behavioral aspects like pagination, error handling, or authentication that would be valuable given this is a production monitoring tool with sibling mutation tools available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single optional parameter. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for adequate but not enhanced parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all issues from GlitchTip'), specifies the scope ('for the configured project'), and distinguishes from sibling tools by focusing on listing all issues rather than getting a specific issue (get_glitchtip_issue) or resolving one (resolve_glitchtip_issue).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use it ('to see all current errors and exceptions in production'), but doesn't explicitly state when not to use it or name alternatives. It implies this is for monitoring production issues but doesn't contrast with the sibling tools beyond their different functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.