Server Quality Checklist

Tool Scores

• create_jira_issueC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it creates an issue from Markdown. It doesn't disclose behavioral traits like authentication needs, rate limits, error handling, or what happens on success/failure. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's core function without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavior, parameter meanings, and expected outcomes, making it inadequate for an AI agent to use the tool effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but only mentions 'Markdown description' for the 'description' parameter. It doesn't explain the meaning of other 6 parameters (e.g., 'project', 'summary', 'issue_type'), leaving most semantics undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Creates') and resource ('new JIRA issue'), specifying it's from Markdown description. It distinguishes from sibling 'update_jira_issue' by focusing on creation rather than modification, though it doesn't explicitly contrast with 'search_jira_issues' or 'echo'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'search_jira_issues' or 'update_jira_issue'. The description implies usage for creating issues from Markdown, but lacks explicit context, prerequisites, or exclusions for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_jira_issuesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions JQL syntax but doesn't cover critical aspects like authentication needs, rate limits, pagination behavior, or error handling. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration. Every word earns its place, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a Jira search tool with 3 parameters, 0% schema coverage, no annotations, and no output schema, the description is incomplete. It lacks details on parameter meanings, behavioral traits, and return values, making it inadequate for the agent to use the tool effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only mentions the 'query' parameter indirectly via JQL, leaving 'site_alias' and 'basic_only' completely unexplained. The description adds minimal value beyond the schema, failing to adequately address the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Search for') and resource ('Jira issues'), specifying the method ('using JQL syntax'). It distinguishes from siblings like create_jira_issue and update_jira_issue by focusing on retrieval rather than modification. However, it doesn't explicitly differentiate from potential other search tools, keeping it at 4 instead of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context for JQL usage, or comparisons to other tools like echo. This leaves the agent without explicit usage instructions, scoring a 2 for minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_jira_issueC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'Only provided fields will be updated,' which adds useful context about partial updates. However, it lacks critical behavioral details: it doesn't specify if this is a mutation requiring permissions, what happens on success/failure, rate limits, or authentication needs. For a mutation tool with 7 parameters, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences that are front-loaded and waste-free. Every word earns its place by stating the core action and a key behavioral constraint. No unnecessary details or redundancy are present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given high complexity (7 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It lacks details on return values, error handling, permissions, and parameter meanings. For a mutation tool with significant schema complexity, this minimal description leaves too many gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only vaguely references 'fields' without explaining what fields are updatable or their semantics (e.g., 'issue_key' as identifier, 'additional_fields' for custom fields). This adds minimal value beyond the schema's property names, failing to adequately address the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Updates') and resource ('an existing JIRA issue'), making the purpose immediately understandable. It distinguishes from 'create_jira_issue' by specifying 'existing' issue. However, it doesn't explicitly differentiate from potential sibling tools like 'search_jira_issues' in terms of update vs. read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_jira_issue' or 'search_jira_issues'. There's no mention of prerequisites (e.g., needing an existing issue key), appropriate contexts, or when not to use it. The only implied usage is updating fields, but no explicit alternatives or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• echoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool echoes input text with optional transformation, but doesn't disclose behavioral traits like whether it's read-only, has side effects, rate limits, or error handling. The description is minimal and lacks necessary operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence) and front-loaded with the core purpose. Every word earns its place, with no wasted text. It efficiently communicates the essential functionality without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't explain what the tool returns, how transformations work, or any behavioral aspects. For a tool with two parameters and no structured documentation, this minimal description leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'input text' and 'optional case transformation,' which map to the two parameters (text and transform), but doesn't explain what 'transform' accepts (e.g., uppercase, lowercase) or provide examples. It adds some meaning but doesn't fully compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Echo back the input text' specifies the verb (echo) and resource (input text), and 'with optional case transformation' adds detail about functionality. However, it doesn't distinguish from sibling tools (Jira-related tools), which is expected since this is a simple utility tool in a different domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'optional case transformation' but doesn't specify what transformations are available or when to apply them. There's no context about prerequisites, limitations, or relationship to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.