Server Quality Checklist

Tool Scores

• jira_check_user_issuesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions checking membership and listing issues but doesn't disclose authentication requirements (though schema hints at apiToken), rate limits, error conditions, or what happens if the user isn't a member. For a tool with 5 parameters and no annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality. It wastes no words but could be slightly more structured (e.g., separating the two main actions). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format (e.g., structured list of issues, membership boolean), error handling, or how the two actions (check membership + list issues) relate. For a tool with authentication and data retrieval complexity, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing good parameter documentation. The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain relationships between parameters (e.g., how email/userName interact) or usage nuances. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('checks', 'lists') and resources ('user', 'project', 'assigned issues'). It distinguishes from siblings like 'jira_list_project_members' by focusing on a specific user's membership and issues rather than listing all members. However, it doesn't explicitly differentiate from 'jira_search_issues' which might also find user issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'jira_list_project_members' for membership checking or 'jira_search_issues' for finding user issues. No prerequisites or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_create_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool creates an issue but doesn't describe what happens upon creation (e.g., issue key generation, notifications, permissions required, error handling, or rate limits). For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero waste. It's appropriately sized and front-loaded, clearly stating the core action without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., issue key, success status), error conditions, or behavioral nuances. For a 10-parameter tool that creates resources, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description adds no additional meaning beyond stating 'with specified details', which is redundant. Baseline 3 is appropriate when the schema does the heavy lifting, though the description doesn't compensate or enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Creates') and resource ('new issue in a Jira project'), making the purpose unambiguous. It distinguishes from siblings like 'jira_get_issue' (read) or 'jira_search_issues' (query), though it doesn't explicitly name alternatives. The description is specific but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'jira_check_user_issues' or 'jira_search_issues'. It doesn't mention prerequisites (e.g., authentication setup) or contextual constraints (e.g., project access). Usage is implied by the action but without explicit when/when-not statements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_get_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it 'retrieves details' but doesn't specify what details are returned, whether it's a read-only operation, authentication requirements beyond the schema, or potential rate limits. This is inadequate for a tool with authentication parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part of the sentence ('retrieves details', 'specific Jira issue', 'by key') contributes essential information, making it optimally concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving authentication and issue retrieval), lack of annotations, and no output schema, the description is insufficient. It doesn't explain what 'details' are returned, error conditions, or how authentication parameters interact, leaving significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema fully documents all four parameters. The description doesn't add any parameter-specific information beyond what's in the schema, such as clarifying the format of 'issueKey' or explaining the relationship between authentication parameters. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieves details') and resource ('specific Jira issue by key'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'jira_search_issues' or 'jira_check_user_issues', which might also retrieve issue information but with different scopes or filters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention that this is for single-issue lookup by key, as opposed to 'jira_search_issues' for broader queries or 'jira_check_user_issues' for user-specific issues, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_list_project_membersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation, implying read-only behavior, but doesn't mention authentication requirements, rate limits, pagination, or what the output format looks like. This is a significant gap for a tool with authentication parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero waste. It's appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of authentication parameters and no output schema, the description is incomplete. It doesn't explain authentication requirements, return values, or error handling. For a tool with 4 parameters (including auth) and no annotations, more context is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description doesn't add any additional meaning beyond what's in the schema, such as explaining how parameters interact or providing examples. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Lists') and resource ('all members of a specific Jira project'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'jira_list_projects' or 'jira_check_user_issues', which might also involve listing project-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites like authentication, nor does it compare with siblings like 'jira_check_user_issues' for user-specific data or 'jira_list_projects' for broader project info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_list_projectsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but provides minimal behavioral context. It doesn't disclose whether this is a read-only operation (implied by 'Lists'), authentication requirements beyond the parameters, rate limits, pagination behavior, or what the output format looks like. For a tool with authentication parameters, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized for a simple list operation and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has authentication parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain how authentication works, what the return data includes, or any behavioral constraints. For a tool interacting with an external API like Jira, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description adds no additional parameter semantics beyond what's in the schema. The baseline score of 3 reflects adequate coverage through the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Lists') and resource ('all Jira projects'), specifying scope with 'the user has access to'. It distinguishes from siblings like 'jira_list_project_members' by focusing on projects rather than members, but doesn't explicitly differentiate from other list tools like 'jira_list_sprints' beyond the resource name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description doesn't mention prerequisites like authentication setup, nor does it suggest when to use this versus 'jira_search_issues' for project-related queries. Usage context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_list_sprintsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'filtering options' but doesn't explain key behaviors: whether this is a read-only operation (implied by 'Lists'), what authentication is required (hinted by parameters but not stated), how results are returned (e.g., pagination, format), or any rate limits. For a tool with authentication parameters and no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Lists current sprints in Jira') and adds a brief qualifier ('with filtering options'). There is no wasted language or redundancy, making it appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no output schema, no annotations), the description is incomplete. It doesn't address authentication requirements, result format, error handling, or how filtering parameters interact. Without annotations or output schema, the agent lacks sufficient context to use this tool effectively beyond basic parameter passing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal value beyond the input schema, which has 100% coverage. It mentions 'filtering options,' which aligns with parameters like 'boardId,' 'projectKey,' and 'state,' but doesn't provide additional semantics (e.g., how filters combine or default behaviors). With high schema coverage, the baseline is 3, as the schema already documents parameters well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Lists') and resource ('current sprints in Jira'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'jira_list_projects' or 'jira_list_project_members', which reduces specificity. The mention of 'filtering options' adds some context but doesn't fully distinguish it from other list operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication setup), compare it to sibling tools like 'jira_search_issues' for sprint-related queries, or specify scenarios where filtering by board or project is appropriate. This leaves the agent without clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jira_search_issuesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the search action but lacks critical details: it doesn't specify if this is a read-only operation (likely, but not confirmed), mention rate limits, describe the return format (e.g., list of issues with fields), or note any constraints like pagination. This leaves significant gaps for a tool with 5 parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and filtering criteria, making it easy to parse quickly, which is ideal for conciseness in tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with 5 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return values (e.g., what data the search yields), behavioral traits like safety or performance, or how it integrates with sibling tools, leaving the agent with insufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions filtering 'by project and assignee', which aligns with the 'projectKey' and 'assigneeName' parameters in the schema. However, with 100% schema description coverage, the schema already fully documents all 5 parameters (including authentication details like 'jiraHost', 'email', and 'apiToken'), so the description adds minimal value beyond restating what's in the structured data, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Searches for') and resource ('Jira issues') with specific filtering criteria ('by project and assignee'), which provides a concrete purpose. However, it doesn't explicitly differentiate from sibling tools like 'jira_check_user_issues' or 'jira_list_projects', which might also involve issue-related queries, leaving some ambiguity about its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites like authentication (implied by parameters but not stated), nor does it compare to siblings such as 'jira_check_user_issues' for user-specific queries or 'jira_list_projects' for broader project info, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.