Server Quality Checklist

Tool Scores

• add_issue_to_projectC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but lacks critical behavioral details. It mentions the tool adds an existing issue but doesn't disclose permissions needed, rate limits, whether it's idempotent, or what happens if the issue is already in the project. The return value is vaguely described as 'a formatted string confirming the addition' without specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The Args and Returns sections are structured clearly, though the return description could be more specific. No wasted sentences, but minor improvements in detail could enhance clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 5 parameters with 0% schema coverage, the description is incomplete. It covers the basic action and parameters but misses behavioral context (e.g., auth, errors), output details, and usage guidelines relative to siblings. For a mutation tool with multiple parameters, this leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all 5 parameters with brief explanations (e.g., 'owner: The GitHub organization or user name that owns the project'), adding basic semantics beyond the schema's titles. However, it doesn't clarify relationships (e.g., 'owner' vs 'issue_owner') or provide examples, leaving gaps in understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add an existing GitHub issue to a Project V2') with specific resources (GitHub issue, Project V2), distinguishing it from siblings like 'create_issue' (creates new) or 'delete_project_item' (removes). However, it doesn't explicitly differentiate from 'update_project_item_field' which might modify project items, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_draft_issue' or 'update_project_item_field', nor does it mention prerequisites (e.g., issue must exist, user must have permissions). It only states what the tool does without context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_draft_issueC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is 'create' (implying a write/mutation operation) and mentions a confirmation message return, but doesn't cover important aspects like required permissions, whether the operation is idempotent, rate limits, error conditions, or what happens if the project doesn't exist. For a mutation tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for Args and Returns, making it easy to parse. It's appropriately sized with no redundant information. The only minor improvement would be integrating the purpose statement more seamlessly with the parameter documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It covers basic parameter semantics but lacks critical behavioral context (permissions, error handling), doesn't explain the return value structure beyond 'confirmation message', and provides no guidance on when to use this versus sibling tools. The agent would need to make assumptions about important operational aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly lists all 4 parameters with brief explanations, adding meaning beyond the schema which has 0% description coverage. It clarifies that 'body' is optional and provides context for 'owner' and 'project_number'. However, it doesn't explain parameter formats (e.g., what constitutes a valid project number) or constraints beyond what's obvious from the schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('create a draft issue') and resource ('directly in a GitHub Project V2'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'create_issue' or 'add_issue_to_project', which could create confusion about when to use this specific draft creation tool versus other issue-related tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_issue' or 'add_issue_to_project'. It mentions the target (GitHub Project V2) but doesn't specify prerequisites, appropriate contexts, or exclusions. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists projects and returns a formatted string, but it doesn't cover critical aspects like authentication requirements, rate limits, pagination behavior, error handling, or whether it's read-only or has side effects. For a tool with no annotations, this leaves significant gaps in understanding its operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose in the first sentence. The 'Args' and 'Returns' sections are structured but could be more integrated; overall, it's efficient with minimal waste, though it could be slightly more polished in flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is somewhat complete but lacks depth. It covers the basic purpose and parameter semantics but misses behavioral details and usage guidelines. For a simple list tool, this is adequate but not comprehensive, aligning with a minimum viable score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds some meaning beyond the input schema by explaining that 'owner' refers to 'The GitHub organization or user name,' which clarifies the parameter's purpose. However, with 0% schema description coverage and only one parameter, the baseline is 4, but the description doesn't fully compensate by providing details like format examples or constraints, so it scores slightly lower at 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List GitHub Projects V2 for a given organization or user.' It specifies the verb ('List'), resource ('GitHub Projects V2'), and scope ('organization or user'), which is specific and actionable. However, it doesn't explicitly differentiate from sibling tools like 'get_project_items' or 'get_project_fields', which might have overlapping functionality, so it doesn't reach a perfect score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the 'owner' parameter but doesn't clarify scenarios where this tool is preferred over siblings like 'get_project_items' or 'get_project_fields', nor does it mention prerequisites or exclusions. This lack of contextual usage advice limits its effectiveness for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_project_itemB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action is 'Delete', implying a destructive mutation, but doesn't specify if this is reversible, what permissions are required, or any rate limits. The mention of 'Returns: A confirmation message' hints at output but lacks detail on error handling or side effects, leaving significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose followed by parameter explanations. The 'Args' and 'Returns' sections are structured clearly, but the 'Returns' part is slightly redundant as it doesn't add detail beyond 'confirmation message'. Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical information such as authentication requirements, error conditions, whether deletion is permanent, and how it differs from sibling tools. The return value is vague ('confirmation message'), and without annotations, more behavioral context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value beyond the input schema by explaining each parameter's purpose: 'owner' as the GitHub organization/user name, 'project_number' as the project number, and 'item_id' as the ID to delete. Since schema description coverage is 0% (no titles or descriptions in schema), this compensates well, though it could include format examples (e.g., 'item_id' as a string format).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('an item from a GitHub Project V2'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'update_project_item_field' or 'get_project_items', which would require more nuance about when deletion versus modification or retrieval is appropriate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication needs, permissions), exclusions (e.g., cannot delete certain item types), or compare to siblings like 'update_project_item_field' for modifications instead of deletions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_fieldsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is a 'Get' operation, implying read-only behavior, but doesn't clarify authentication needs, rate limits, error conditions, or what 'formatted string' entails. This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the purpose stated first followed by parameter and return details in a structured format. Every sentence adds value, though minor improvements in flow could enhance readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It covers purpose and parameters but lacks details on behavioral traits, usage context, and output specifics, leaving gaps that could hinder effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining 'owner' as 'The GitHub organization or user name' and 'project_number' as 'The project number', which clarifies beyond the schema's basic titles. However, it doesn't detail format constraints or examples, preventing a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'fields available in a GitHub Project V2', with specific mention of SingleSelect fields. It distinguishes from siblings like get_project_items (which retrieves items, not fields) and update_project_item_field (which modifies fields). However, it doesn't explicitly contrast with all siblings, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description doesn't mention prerequisites, context for usage, or comparisons to other field-related operations that might exist. This leaves the agent without explicit direction on application scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_project_item_fieldB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is an update operation but doesn't cover critical aspects like required permissions, whether changes are reversible, rate limits, or error handling. The description adds minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear parameter explanations in a bullet-like format. Every sentence adds value: the first states the action, and the subsequent lines clarify parameter semantics efficiently. There's no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, no annotations, no output schema), the description is moderately complete. It covers parameter meanings but lacks behavioral details (e.g., permissions, side effects) and output specifics beyond 'A confirmation message'. For a mutation tool with no structured support, this leaves gaps in guiding the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the input schema, which has 0% description coverage. It explains the purpose of each parameter (e.g., 'owner: The GitHub organization or user name') and provides crucial details like 'field_value' accepting 'text, date, or option ID for single select'. This compensates well for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Update a field value for a project item.' It specifies the verb ('update') and resource ('field value for a project item'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'delete_project_item' or 'get_project_fields' beyond the 'update' action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing project item), exclusions, or comparisons to siblings like 'add_issue_to_project' or 'create_issue'. The agent must infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_issueA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool creates an issue (implying a write/mutation operation) and mentions the return format, but doesn't disclose critical behavioral traits like required permissions (e.g., write access to repo), whether the operation is idempotent, rate limits, error conditions, or what happens on duplicate titles. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns) and uses bullet points for readability. It's appropriately sized for a 4-parameter tool, though the 'Args' and 'Returns' labels are slightly redundant given the schema. Every sentence adds value, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with 4 params), lack of annotations, and no output schema, the description is moderately complete. It covers purpose and parameters well but lacks behavioral context (permissions, errors, side effects) and detailed return value explanation beyond 'formatted string'. For a GitHub API tool, more context about authentication or API constraints would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides clear semantic meaning for all 4 parameters: 'owner' as organization/user name, 'repo' as repository name, 'title' as issue title, and 'body' as optional issue body. This adds substantial value beyond the bare schema, though it doesn't specify format constraints (e.g., character limits) or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new GitHub issue') and identifies the resource (GitHub issue). It distinguishes from siblings like 'create_draft_issue' by specifying it creates a regular issue, not a draft, and from 'add_issue_to_project' by focusing on creation rather than project association.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for creating GitHub issues, but doesn't explicitly state when to use this vs. alternatives like 'create_draft_issue' or 'add_issue_to_project'. It provides basic context (GitHub platform) but lacks explicit guidance on prerequisites, exclusions, or comparative scenarios with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_itemsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses some behavioral traits: filtering logic (state OR custom field), pagination via cursor, case-insensitive matching for custom fields, and automatic fetching of more items when filtering. However, it doesn't cover important aspects like authentication requirements, rate limits, error conditions, or what happens with invalid parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by organized 'Args' and 'Returns' sections. Every sentence adds value, though the 'When filtering...' note could be more concise. It's appropriately sized for a 7-parameter tool with complex filtering logic.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, filtering logic, pagination) and lack of annotations/output schema, the description does a good job covering essential information. It explains parameters thoroughly, describes the filtering approach, mentions pagination, and specifies the return format. However, it could better address authentication, error handling, or relationship to sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides excellent parameter semantics beyond the schema. With 0% schema description coverage, it fully compensates by explaining all 7 parameters: their purposes (e.g., 'owner: The GitHub organization or user name'), constraints (e.g., 'Currently supports SingleSelect and Iteration fields'), default values (e.g., 'limit: Maximum number of items to return (default: 50)'), and behavioral implications (e.g., 'When filtering, the system automatically fetches more items to improve efficiency').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get items in a GitHub Project V2' with filtering capabilities. It specifies the resource (GitHub Project V2 items) and action (get/filter), but doesn't explicitly differentiate from sibling tools like 'list_projects' or 'get_project_fields'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through filtering options ('Can filter by state OR a single custom field=value') and mentions 'When filtering, the system automatically fetches more items to improve efficiency.' However, it doesn't provide explicit guidance on when to use this tool versus alternatives like 'list_projects' or 'get_project_fields', nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.