Server Quality Checklist

Tool Scores

• sentry_pattern_analysisA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses the merging of near-duplicates by stripping IDs, ranking by event volume, and that it's for incident triage. Does not explicitly state side effects, but as a read/analysis tool, it's sufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with use cases front-loaded and explanation following. Slightly verbose but every sentence adds value; could be trimmed slightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately describes the high-level output (patterns and event volumes). For a tool with 7 parameters, it covers the what, why, and how-to-use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 7 parameters with descriptions (100% coverage). The tool description adds no extra semantic value beyond the schema; baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it ranks issue groups by event volume and merges near-duplicates into patterns. It distinguishes from siblings by explicitly carving out use cases for pattern analysis vs. individual issue details or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit trigger phrases ('group similar errors', etc.) and states it's essential for incident triage. Lacks explicit 'when not to use', but the use cases are well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sentry_trace_lookupA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return structure (spans, errors, grouped by service with timeline). No annotations exist, so this adds useful behavioral context, though more details on limitations or edge cases could enhance transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is somewhat verbose with emoji and all-caps emphasis. While front-loaded, it could be more concise and structured without sacrificing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking output schema and annotations, description covers key aspects: purpose, usage, expected output. Missing details on performance implications or error handling, but adequate for a well-scoped tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description does not add significant meaning beyond the schema's own parameter descriptions; it only reiterates context already present in schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool is for Sentry trace ID lookup, provides example user queries, and explicitly distinguishes from the sibling 'search_issues' tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use scenarios (mention of trace ID, specific user phrases) and a critical exclusion ('not search_issues'), guiding correct agent selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sentry_get_issue_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It details what information is returned (summary, stack trace, breadcrumbs, etc.) and explains breadcrumbs as 'chronological events leading UP TO the error'. It does not mention potential limitations like rate limits or output size, but covers core behavioral traits well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-structured with an opening emoji marker, clear sections, and a list of triggers. Every sentence contributes to understanding. Could be slightly more concise, but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is provided, but the description lists the return components (summary, stack trace, breadcrumbs, etc.). With 5 parameters fully described in schema, the description provides enough context for an agent to know what the tool does and what to expect. Lacks details on output structure but sufficient given no schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the purpose of breadcrumbs and frames (e.g., 'how many stack frames to show per exception (innermost first)') and emphasizing that 'reasoning' is for explanation. This extra context helps the agent understand parameter usage beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'ROOT CAUSE TOOL' and lists specific user queries that trigger it. It explicitly describes what is fetched (summary, stack trace, breadcrumbs, etc.) and distinguishes from sibling tools like sentry_search_issues and sentry_trace_lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit triggers for usage (e.g., 'what happened before this error', 'show the stack trace') and suggests it as a follow-up after sentry_search_issues. It implicitly contrasts with sibling tools but does not explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sentry_search_issuesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden of behavioral transparency. It discloses key behaviors: automatically searches across all projects unless a project slug is provided, uses Sentry search syntax, default queries and limits, and that count mode returns counts/trends with an ASCII chart. It also notes that in count mode 'is:*' tokens are ignored. Missing details about pagination or rate limits, but overall transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, starting with a bold purpose statement followed by details. It is somewhat lengthy but every sentence adds value, covering usage, syntax, and special modes. It could be slightly more concise, but the current length is justified by the richness of information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no output schema, and moderate complexity, the description provides a strong understanding of how to use the tool. It explains default behaviors, query syntax, count mode, and filters. While it doesn't detail the output format of issue lists or error handling, it covers the essential aspects for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, yet the description adds substantial value beyond the schema. It provides concrete examples for 'query', explains the return type for 'count', directs users to 'sentry://projects' for project slugs, and explains the statsPeriod format for 'time_window'. The 'reasoning' parameter is explained as a justification for tool usage. This greatly enhances parameter understandability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it is 'THE PRIMARY SENTRY SEARCH TOOL' and lists numerous user intents (e.g., 'check sentry', 'look at errors', 'search sentry'). It clearly explains that it searches grouped issues across the whole org or filtered by project, uses Sentry search syntax, and has a count mode. This effectively differentiates it from sibling tools like sentry_get_issue_details or sentry_pattern_analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use this tool by listing example user requests ('check sentry', 'look at errors', etc.). It also explains the count mode for getting counts/trends. However, it does not explicitly state when not to use it or name alternative tools, though the context implies it is for issue searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.