Server Quality Checklist

Tool Scores

• jitbit_list_ticketsB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover key behavioral traits (read-only, non-destructive, idempotent, open-world). The description adds minimal context beyond this, such as mentioning pagination via 'offset' and the return format, but doesn't disclose rate limits, authentication needs, or detailed behavioral constraints. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose. The Args and Returns sections are structured but slightly verbose, as some details (e.g., parameter defaults) are redundant with the schema. Most sentences earn their place, though minor trimming could improve efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, rich annotations, and 100% schema coverage, the description is mostly complete. It explains the return format (list of tickets with fields), which compensates for the lack of an output schema. However, it could better address sibling tool differentiation and usage scenarios to enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents parameters. The description repeats parameter details (e.g., mode options, limit range) without adding significant meaning beyond the schema, such as explaining how 'status' interacts with 'mode' or providing usage examples. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'List and filter tickets in Jitbit Helpdesk,' which specifies the verb (list/filter) and resource (tickets). However, it doesn't explicitly differentiate from sibling tools like 'jitbit_search_tickets,' which likely has overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (jitbit_get_ticket, jitbit_search_tickets). It lacks explicit context, exclusions, or alternatives, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jitbit_search_ticketsB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations cover key behavioral traits (read-only, non-destructive, idempotent, open-world), so the description's burden is lower. It adds value by specifying that the search matches against ticket subjects and bodies, which isn't in the annotations. However, it doesn't disclose other behaviors like rate limits, authentication needs, or error handling, keeping the score at a baseline level.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, with a clear purpose statement followed by parameter and return details. It avoids unnecessary fluff, though the parameter section slightly repeats schema information, which is minor but keeps it from a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, rich annotations, and high schema coverage, the description is reasonably complete. It explains the search scope and return format, though without an output schema, it could benefit from more detail on the ticket structure. However, it covers the essentials for a search operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning the input schema already documents all parameters thoroughly. The description repeats some parameter details (e.g., query matches subjects and bodies) but doesn't add significant new meaning beyond the schema. This meets the baseline score when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for tickets in Jitbit Helpdesk by keyword or phrase, providing a specific verb ('search') and resource ('tickets'). However, it doesn't explicitly differentiate from sibling tools like jitbit_list_tickets, which might also retrieve tickets but potentially without search capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like jitbit_get_ticket or jitbit_list_tickets. It mentions searching by keyword or phrase but doesn't specify scenarios where this is preferred over other methods, leaving the agent to infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jitbit_get_ticketA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide key behavioral hints (read-only, non-destructive, idempotent, open-world), so the bar is lower. The description adds valuable context by specifying that it retrieves 'full ticket details' and 'complete comment history,' which clarifies the scope of data returned beyond what annotations indicate. No contradictions with annotations are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by structured Args and Returns sections. Each sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema) and rich annotations, the description is mostly complete. It covers purpose, parameters, and return details. However, it could benefit from more explicit usage guidelines compared to siblings, slightly reducing completeness for optimal agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'ticketId' fully documented in the schema. The description mentions 'ticketId' in the Args section but does not add significant meaning beyond what the schema provides, such as format examples or constraints. Baseline score of 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get a single ticket') and resource ('from Jitbit Helpdesk'), including the scope ('with its full conversation thread'). It distinguishes from sibling tools like 'jitbit_list_tickets' and 'jitbit_search_tickets' by focusing on retrieving a single ticket rather than listing or searching multiple tickets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'a single ticket,' suggesting it's for detailed retrieval rather than bulk operations. However, it does not explicitly state when to use this tool versus alternatives like 'jitbit_list_tickets' or 'jitbit_search_tickets,' nor does it mention any exclusions or prerequisites for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.