Server Quality Checklist

Tool Scores

• list_jira_issuesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It does not state that the tool is read-only, mention authentication needs, rate limits, or any side effects. The return type is vague ('Dictionary containing issues list and metadata'), providing minimal behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured as an Args list followed by Returns. It covers all 14 parameters adequately but lacks a front-loaded purpose statement. The length is justified by the number of parameters, but the lack of a summary or hierarchical grouping reduces conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 14 parameters (all optional), the description covers each filter. However, it omits high-level purpose, usage context, and return structure details. The output schema exists but its content is not described beyond a vague statement, leaving the agent uncertain about response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds meaningful explanations for each parameter (e.g., project: 'e.g., SMQE, OSIM', timeframe: 'default: 0 = disabled'). This adds value beyond the schema, though some descriptions are terse.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The tool name 'list_jira_issues' clearly indicates its function, and the parameter list implies filtering and listing issues. However, the description lacks an explicit high-level statement of purpose, relying solely on parameter explanations. A concise purpose summary would improve clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings (e.g., get_jira_issue_details, get_jira_issues_by_sprint). The description does not mention alternatives or context-specific usage, leaving the agent without decision-making support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jira_issue_detailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavioral traits. It mentions 'from Snowflake' and return includes comments, but lacks details on error handling, rate limits, or behavior for invalid keys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Succinct with clear purpose and structured Args/Returns sections. Every sentence adds value, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple tool with one parameter and an output schema, but lacks guidance on error scenarios or limitations like handling non-existent keys.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the Args section clearly defines 'issue_keys' with example format and type, compensating well for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches detailed info for multiple JIRA issues by keys, distinguishing it from sibling tools like listing or getting links. However, it does not explicitly differentiate itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., get_jira_issue_links). No when-not or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jira_issue_linksA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states the tool 'get's links from Snowflake' but does not clarify side effects, permissions required, error handling for invalid keys, or whether it is read-only. The minimal disclosure is insufficient for a mutation-free assurance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one line for purpose, then structured Args and Returns sections. Every sentence is informative with no waste. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 1 parameter and an output schema (not shown), the description adequately covers the input format and hints at the output type. However, it lacks error handling details and full return structure, but given the presence of output schema, this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage. The description adds an example ('SMQE-1280') and clarifies the expected format (JIRA issue key), which meaningfully supplements the schema. This is well above the baseline of 3 for high-coverage cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get', the resource 'issue links', and the specific identifier 'by its key' and source 'from Snowflake'. It is distinct from siblings like get_jira_issue_details (details vs links) and list_jira_issues (all issues vs one link).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving links of a specific issue, but provides no explicit guidance on when to use this over sibling tools like get_jira_issue_details or list_jira_issues. No when-not-to-use or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jira_project_summaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description states it returns a dictionary of statistics, implying a read-only operation, but does not disclose potential side effects or caching behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear, concise sentences with no unnecessary words; front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with an output schema, the description is adequate, though it could briefly hint at the output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the schema covers all needs; baseline 4 applies as per guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets a summary of all projects from JIRA in Snowflake, distinguishing it from siblings that focus on individual issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the tool name and description, but no explicit guidance is given on when to use this over sibling tools like list_jira_issues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jira_issues_by_sprintA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Mentions source (Snowflake) and return format (dictionary with issues and metadata), but does not disclose rate limits, data freshness, authentication, or side effects. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise, front-loaded purpose, clear Args/Returns structure. Every sentence is useful with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool with 3 params and output schema present, description covers purpose, parameters, and return type. Lacks prerequisites or error handling, but adequate for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It provides examples for sprint_name and project, and explains limit. Adds value beyond schema by clarifying format and usage, though could include more detail like valid project key patterns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Get all JIRA issues in a specific sprint by sprint name from Snowflake', providing a specific verb, resource, and scope. Differentiates from siblings like get_jira_issue_details (single issue) and list_jira_issues (all issues).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly tells when to use: when you have a sprint name and want its issues. Does not explicitly state when not to use or name alternatives, but the context of siblings and the clear purpose make it sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.