Server Quality Checklist

Tool Scores

• jira_get_issueA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that optional parameters cause 'extra API call(s)', which is useful behavioral context. However, with no annotations, it doesn't explicitly state read-only behavior, error handling, or permissions. It partially carries the transparency burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a concise purpose statement followed by Args and Returns sections. Every sentence earns its place, and the format makes it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with three parameters and no output schema, the description covers functionality, optional behaviors, and return contents. It lacks details on errors or permissions, but these are not critical for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for parameters (0% coverage), so the description fully compensates. It explains issue_key with an example format and describes the boolean flags include_subtasks and include_transitions, adding clear meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get detailed information about a specific JIRA issue' with the specific resource (issue key) and what details are included (comments and links). It is distinguished from sibling tools like search, create, update, and transition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool's purpose of retrieving a single issue by key is evident, but there is no explicit when-to-use vs. alternatives statement. The description implies usage for getting detailed info on a known issue, which is clear yet not as explicit as naming alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_create_issueA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the creation behavior, required parameters, the return value (JSON with issue key), and a Cloud-specific nuance for assignee_name. It lacks details on permissions, validation, or side effects, but covers the key behavioral aspects for a create tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an opening summary, a caveat, an Args list, and a Returns line. The Args list is justified because the schema lacks descriptions. The 'Optional fields include...' sentence is slightly redundant with the Args list, but it provides a quick overview, so it still earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, no output schema, and no annotations, the description covers all parameters, the return value, and a platform-specific behavior. It does not address potential errors, permissions, or custom field formatting, but for a creation tool this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates by documenting all 8 parameters with meaningful explanations, examples (e.g., project_key example, issue type examples), and the Cloud versus Server/DC distinction for assignee_name. This is far beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Creates a new issue in a specified Jira project' — a specific verb+resource statement that clearly distinguishes it from sibling tools like jira_update_issue or jira_search_issues. It also lists required vs optional parameters, reinforcing the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on required fields and optional fields, plus a specific platform caveat for Cloud instances (accountId vs username). However, it does not explicitly state when to prefer this tool over alternatives or mention when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_transition_issueA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavior. It explicitly mentions the state change and return confirmation, and points out the source of valid transition IDs. However, it omits permissions, failure modes, or reversibility, leaving some burden unmet for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with Args and Returns sections. Slightly longer than minimal but every sentence adds necessary information; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity and no output schema, the description provides the essential steps: how to obtain the transition ID and what is returned. It is complete enough for correct invocation, though it could mention error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. Description defines each parameter with examples (issue_key) and explains that transition_id comes from jira_get_issue transitions, and comment is optional. Fully compensates for schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States 'Transition a JIRA issue to a new workflow state' – clear verb, resource, target. Distinct from siblings (get, create, update) and references jira_get_issue for transition IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly directs to 'Use jira_get_issue with include_transitions=True to see available transitions and their IDs', giving a concrete prerequisite and pointing to the correct sibling tool. Clarifies when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_get_create_metaA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavior disclosure. It explains the two distinct output modes based on the issue_type parameter and summarizes the return content ('Issue types list, or field details (name, required, allowed values)'). This goes beyond a mere restatement of the tool name, though it does not mention potential errors or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a one-sentence summary followed by clearly labeled Args and Returns sections. Every sentence adds value: the first explains the purpose, the second explains the two call modes, the Args list clarifies parameter formats, and the Returns section sets expectations. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the return values and the two call modes. It could be more detailed about the exact structure of 'field details' (e.g., whether IDs are included), but for a metadata retrieval tool with only two parameters, the description is sufficiently complete for an agent to invoke it correctly. It also covers both parameter modes, which is the main complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description fully compensates for the 0% schema coverage. It explains both parameters with examples (project_key: 'OCPBUGS', issue_type: 'Bug') and describes the effect of omitting issue_type. This gives an agent complete understanding of how to fill the arguments, which is far more than the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a clear resource 'metadata for creating issues', and further specifies the two modes of operation (listing issue types vs. retrieving field details). This distinguishes it from sibling tools like jira_create_issue, which perform actual creation, and jira_get_issue, which retrieves existing issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to call with and without the issue_type parameter: 'Call without issue_type to list available types. Call with issue_type to get field requirements.' While it doesn't name alternative tools, the purpose is clear enough that an agent would know this is the metadata lookup companion to jira_create_issue. It could have explicitly stated 'Use this before creating an issue' but the context is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_update_issueA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It explains partial-update behavior ('only provided fields are updated'), that labels replace existing ones, and that assignee format differs by platform (accountId on Cloud vs username on Server/DC). It does not mention potential destructive implications or permission requirements, but adds meaningful behavioral context beyond 'Update fields'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently organized: a one-sentence purpose, a few context notes, an Args list with per-parameter explanations, and a Returns line. It avoids fluff, though the parameter list is lengthy; however, this length is justified given the 0% schema coverage. It is front-loaded with the main behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given high complexity (8 parameters, no annotations, no output schema), this description is remarkably complete. It covers the purpose, all parameters, return value, and critical edge cases like Cloud/Server assignee differences and label replacement. It leaves little ambiguity for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates fully by listing each parameter with a meaningful explanation: issue_key with format example, assignee with platform-specific format, labels with replacement semantics, comment with timing, and custom_fields as a dictionary. This adds substantial meaning beyond the schema's bare type/title fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Update fields on an existing JIRA issue' — a specific verb+resource that clearly distinguishes it from sibling tools like jira_create_issue, jira_search_issues, or jira_transition_issue. It further clarifies scope by noting all field parameters are optional and that a comment can be added in the same call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use it to update fields on an existing issue, with all fields optional, and notes the Cloud vs Server/DC distinction for assignee. However, it does not explicitly state when to use an alternative (e.g., jira_transition_issue for status changes) or provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_search_issuesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It discloses pagination behavior, total count, and the return dict structure. It also explains the role of start_at and max_results. While it doesn't explicitly state that this is a read-only operation, 'search' implies no side effects, and no annotations contradict this. This is good transparency for a read-oriented tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a short purpose statement, a note on pagination, and clear Args/Returns sections. It front-loads the main purpose and remains appropriately sized for the tool's simplicity, with no redundant sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description explains the return dict structure (issues array, total, start_at, max_results). All three parameters are documented with defaults and usage. For a simple tool with 3 parameters and no nested objects, the description covers all necessary information for an agent to invoke it correctly, including JQL format and pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so effectively: jql is described with an example, max_results explains its purpose and default, and start_at is described in the context of pagination. This adds meaning well beyond the schema's types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search for JIRA issues using JQL,' which clearly identifies the action (search) and resource (JIRA issues). This distinguishes it from sibling tools like jira_get_issue and jira_create_issue. The JQL specification further clarifies the search capability and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by explaining that JQL is used for querying and explicitly advises 'Use start_at to page through large result sets.' It gives a JQL example and explains pagination. However, it does not explicitly state when to use this tool versus alternatives like jira_get_issue, though the name and purpose make this obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.