Server Quality Checklist

Tool Scores

• jira_create_issueC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Create a new JIRA issue', which implies a write operation, but doesn't cover critical behaviors: whether it requires specific permissions, what happens on success/failure (e.g., returns an issue ID), if there are rate limits, or if it's idempotent. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste: 'Create a new JIRA issue'. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary details. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a mutation tool with 6 parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain return values (e.g., issue ID), error conditions, or behavioral nuances. For a creation tool with no structured output, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with all 6 parameters well-documented in the schema (e.g., 'Project key', 'Issue type'). The description adds no parameter information beyond what's in the schema. According to the rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a new JIRA issue' clearly states the verb ('Create') and resource ('JIRA issue'), which is adequate. However, it doesn't distinguish this tool from its sibling 'jira_update_issue' (which also modifies issues) or specify what makes creation unique versus updating. The purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a project key), when not to use it (e.g., for existing issues), or refer to siblings like 'jira_update_issue' for modifications. Without any usage context, the agent must infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_add_commentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states the action is 'Add' (implying a write/mutation), but doesn't disclose permission requirements, whether comments are editable/deletable, rate limits, or response format. For a mutation tool with zero annotation coverage, this leaves critical gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero wasted words. It front-loads the core purpose efficiently, making it easy to parse. Every word earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens on success/failure, return values, error conditions, or side effects. Given the complexity of adding data to JIRA, more context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (issueKey, comment) adequately. The description adds no additional meaning about parameter usage, constraints, or examples beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add') and target resource ('comment to a JIRA issue'), making the purpose immediately understandable. It distinguishes from siblings like jira_get_issue (read) and jira_update_issue (modify fields), though it doesn't explicitly differentiate from jira_create_issue which creates a different resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing issue), when to choose this over jira_update_issue for communication, or any constraints. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_get_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states what the tool does ('Get details') without describing the return format, error handling, authentication needs, rate limits, or whether it's a read-only operation. For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded with the core purpose. There is no wasted language or unnecessary elaboration, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what details are returned, error conditions, or behavioral aspects like read-only nature. For a tool with no structured data beyond the input schema, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'issueKey' fully documented in the schema. The description doesn't add any additional meaning beyond what the schema provides (e.g., it doesn't explain the format of issue keys beyond the schema's example). With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get details') and resource ('specific JIRA issue'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'jira_search' which might also retrieve issue information, missing the opportunity to specify this tool is for single-issue retrieval by key.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention that this is for retrieving a single issue by its key, as opposed to 'jira_search' for multiple issues or filtering, or when to prefer this over other read operations. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action ('Search issues') but lacks behavioral details: it doesn't disclose that this is a read-only operation (implied but not explicit), doesn't mention authentication needs, rate limits, pagination behavior beyond schema hints, or what the output looks like. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action ('Search issues') and method ('using JQL'), making it easy to parse. Every word earns its place, and there's no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (search with pagination), lack of annotations, and no output schema, the description is incomplete. It doesn't explain the return format, error handling, or behavioral traits like read-only nature. For a tool with 3 parameters and no structured safety hints, more context is needed to guide an AI agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters (jql, startAt, maxResults). The description adds no additional meaning beyond the schema—it mentions 'using JQL' but doesn't explain JQL syntax or usage. With high schema coverage, the baseline is 3, as the description doesn't compensate but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Search') and resource ('JIRA Issues') with the specific method ('using JQL'). It distinguishes from siblings like 'jira_get_issue' (single issue retrieval) and 'jira_create_issue' (creation), but doesn't explicitly contrast with other search-related tools since none are listed. The purpose is specific but could be more distinctive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a JQL query), exclusions, or comparisons to siblings like 'jira_get_issue' for single issues. The description assumes the user knows when search is appropriate, offering no explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_transition_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('transition') but doesn't cover critical aspects like required permissions, whether the transition is reversible, rate limits, or what happens to the issue's history. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for the tool's complexity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with 3 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain behavioral traits, usage context, or return values, leaving significant gaps for an agent to understand how to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (issueKey, transitionId, comment). The description adds no additional meaning beyond what the schema provides, such as explaining what a 'transitionId' represents or how to obtain it, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Transition an issue to a new status' clearly states the verb ('transition') and resource ('issue'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'jira_update_issue', which might also change issue states, so it lacks sibling distinction for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing issue access), exclusions, or compare to siblings like 'jira_update_issue' for status changes, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_update_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool updates issues but doesn't mention critical behaviors: whether it requires specific permissions, if updates are reversible, what happens to unspecified fields, or error handling. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for what it conveys, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 5 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns, error conditions, or behavioral nuances like partial updates. Given the complexity and lack of structured data, more context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so parameters are well-documented in the schema itself. The description doesn't add any semantic context beyond implying that parameters correspond to updatable fields, which is already clear from the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Update an existing JIRA issue' clearly states the verb ('Update') and resource ('JIRA issue'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like jira_transition_issue (which also modifies issues) or specify what aspects can be updated, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing issue key), contrast with jira_create_issue for new issues or jira_transition_issue for status changes, or indicate any constraints like permission requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.