Server Quality Checklist

Tool Scores

• getChangesetC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions retrieving diff content but doesn't disclose behavioral traits such as rate limits, authentication needs, error handling, or response format. For a read operation with no annotations, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and key details (commit message, author, diff). Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no annotations, no output schema), the description is incomplete. It lacks information on output format, error cases, or how results are structured (e.g., JSON fields). For a tool retrieving detailed commit data, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (revision, includeDiff, diffLimit) with details like defaults and constraints. The description adds minimal value by implying diff content is included, but doesn't provide additional semantics beyond what the schema offers. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get information about') and resource ('WordPress code changeset/commit'), and specifies what information is retrieved ('commit message, author, and diff'). It distinguishes from siblings like getTicket or searchTickets by focusing on code changesets rather than tickets or timelines. However, it doesn't explicitly differentiate from getTimeline or getTracInfo, which might overlap in some contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like getTimeline or getTracInfo, nor does it mention prerequisites or exclusions. It implies usage for retrieving commit details but lacks explicit context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getTicketC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions what information is returned (description, comments, metadata), it doesn't cover important behavioral aspects like whether this is a read-only operation (implied but not stated), error handling for invalid IDs, rate limits, authentication requirements, or response format. For a tool with no annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently communicates the core functionality. It's appropriately sized for a simple retrieval tool and front-loads the essential information. There's no wasted language, though it could potentially be more comprehensive given the lack of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description is incomplete. It doesn't explain what the return value looks like (structure, format), error conditions, or important behavioral constraints. For a tool that retrieves potentially complex ticket data with comments and metadata, more context is needed to help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't add any parameter-specific information beyond what's already in the schema. Since schema description coverage is 100%, all parameters (id, includeComments, commentLimit) are fully documented in the schema itself. The description mentions 'including description, comments, and metadata' which aligns with the parameters but doesn't provide additional semantic context. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed information about a specific WordPress Trac ticket including description, comments, and metadata.' It specifies the verb ('Get'), resource ('WordPress Trac ticket'), and scope of information returned. However, it doesn't explicitly differentiate from sibling tools like 'getTracInfo' or 'searchTickets' which might also retrieve ticket information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'getTicket' over 'searchTickets' for finding tickets, or how it differs from 'getTracInfo' which might provide broader Trac information. There's no context about prerequisites, typical use cases, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getTimelineB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'gets' activity, implying a read-only operation, but doesn't specify aspects like rate limits, authentication needs, or what happens if parameters exceed limits. It adds minimal context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action and resource, with no wasted words. It directly conveys the tool's purpose without unnecessary elaboration, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose but lacks details on behavioral traits, usage guidelines, and output format, which are important for a read operation with configurable parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, detailing both parameters with defaults and limits. The description adds no additional parameter semantics beyond what the schema provides, such as explaining how 'days' and 'limit' interact or the format of returned events. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get recent activity') and resources ('WordPress Trac timeline'), including the types of events covered ('recent tickets, commits, and other events'). However, it doesn't explicitly differentiate from sibling tools like getChangeset or getTicket, which might handle similar data but with different scopes or formats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like getChangeset or getTicket, nor does it mention any prerequisites or exclusions. It implies usage for retrieving recent activity but lacks explicit context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchTicketsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the return format ('ticket summaries with basic info') but lacks critical behavioral details: it doesn't specify if results are paginated, sorted, or limited beyond the 'limit' parameter; doesn't mention authentication needs, rate limits, or error handling; and doesn't clarify what 'basic info' includes. For a search tool with no annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and key details. Every word earns its place: it specifies the action, target, method, and return format without redundancy or fluff. This is optimally concise for a search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete for a search tool: it covers the basic purpose and return format. However, it lacks details on behavioral traits (e.g., pagination, errors) and doesn't fully compensate for the missing output schema by explaining what 'ticket summaries' entail. This is adequate but has clear gaps, fitting a score of 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds minimal value beyond the schema by implying the query can use 'keyword or filter expression' and that results include 'ticket summaries', but it doesn't provide additional syntax examples, format details, or constraints. This meets the baseline of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Search for') and resource ('WordPress Trac tickets') with the method ('by keyword or filter expression'). It distinguishes from siblings like getTicket (single ticket retrieval) and getChangeset (code changes) by focusing on multi-ticket search. However, it doesn't explicitly contrast with getTimeline or getTracInfo, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like getTicket for single tickets or getTimeline for history. It mentions the search capability but offers no context about prerequisites, typical use cases, or exclusions. This leaves the agent without explicit direction for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getTracInfoA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves metadata but lacks details on permissions, rate limits, response format, or error handling. For a read operation without annotations, this leaves significant gaps in understanding how the tool behaves beyond its basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and lists the retrievable metadata types without unnecessary words. Every part of the sentence contributes directly to understanding the tool's function, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter with full schema coverage) and lack of annotations or output schema, the description is adequate for basic understanding but incomplete. It covers the purpose and parameters but misses behavioral details like response format or error cases, which are important for a tool without structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, documenting the single parameter 'type' with an enum. The description adds value by listing the specific enum values (components, milestones, priorities, severities), which clarifies what 'Trac metadata' includes, though it doesn't provide additional syntax or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get') and resource ('WordPress Trac metadata'), listing the exact types of information retrievable (components, milestones, priorities, severities). It distinguishes this tool from siblings like getTicket or searchTickets by focusing on metadata rather than tickets or changesets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when metadata about Trac is needed, but provides no explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or specific contexts that would help an agent choose this tool over siblings like getTicket for ticket-related data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.