Server Quality Checklist

Tool Scores

• gh_issue_editA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the destructive 'Replace' behavior and reveals an implementation detail ('Body is passed via a temp file'), which is useful. However, it does not mention permissions, error behavior, or reversibility, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no fluff. Each sentence earns its place: the first states the purpose, the second adds a critical behavioral detail about the temp file. This is an exemplar of minimal effective structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple 3-param tool and the presence of an output schema, the description plus schema is mostly adequate. However, it lacks usage guidance and edge-case context (e.g., behavior for non-existent issues or permission requirements), which prevents a higher score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is already documented. The description adds a small note about the body being passed via a temp file, but this does not significantly enhance understanding beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Replace an issue's body' using a specific verb and resource. It distinguishes this tool from siblings like gh_issue_create, gh_issue_comment, and gh_pr_create by focusing on editing an existing issue's body rather than creating or commenting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. It does not mention that one should use gh_issue_create for new issues or that this tool only handles body edits, leaving the agent to infer the context from the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gh_issue_commentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses a useful behavioral detail: the body is passed via a temp file, which is not evident from the schema. However, it does not mention other behaviors like authentication, rate limits, or return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the purpose, and the second adds a useful implementation detail. It is front-loaded, concise, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple tool, the description covers the essential purpose and a key behavioral detail. The output schema exists (though not shown), so return values need not be described. It lacks explicit usage boundaries but is otherwise complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all three parameters. The description adds a note about the body being passed via a temp file but does not enrich the semantics of repo, number, or body beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Add a comment') and target ('to an issue (or PR)'). This specific verb+resource combination distinguishes it from sibling tools like gh_issue_create and gh_issue_edit, which handle creation and editing rather than commenting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool—when adding a comment to an issue or PR—but does not explicitly mention alternatives or when not to use it. The naming and sibling list imply differentiation, but no direct comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gh_issue_createA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds a meaningful implementation detail ('Body is passed via a temp file') that helps the agent understand shell quoting is unnecessary. However, it does not mention permissions, side effects, or response behavior, which are relevant for a mutation tool. The added detail is useful but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: the first states the primary action, the second highlights a critical implementation detail. Every word earns its place, and the structure is front-loaded with purpose. No redundancy with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the 100% schema coverage, and the presence of an output schema, the description is largely sufficient. It covers the key quirk (temp file) and leaves parameter details to the schema. It does not address usage exclusions or alternative tools, which would push it to 5, but the core information needed to invoke it correctly is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description's temp-file note adds practical meaning to the 'body' parameter beyond the schema's 'issue body Markdown' — it explains how the value is handled and why shell quoting is not an issue. This enriches parameter understanding, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Create a GitHub issue' — a specific verb and resource that clearly distinguishes this tool from siblings like gh_issue_comment, gh_issue_edit, and gh_pr_create. The scope is unambiguous and immediately actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: use this to create a GitHub issue. While no explicit 'when not to use' or alternative tools are named, the verb 'create' and the sibling names make the usage context evident. The schema note about not relying on the default repo adds a practical prerequisite, but this lives in the schema rather than the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gh_pr_createA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It meaningfully reveals that the body is passed via a temp file to avoid shell quoting and that it wraps only `gh pr create`, not the push. This provides useful behavioral context beyond the obvious 'create' action, though it does not mention authentication or output details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and front-loaded. The first sentence states the core purpose, and the subsequent sentences add essential context without unnecessary fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters and an output schema, so the description appropriately focuses on behavior and prerequisites rather than repeating schema details. It covers the temp-file mechanism and the push requirement. It is largely complete, though it could mention potential error conditions or authentication dependencies.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all five parameters, so the baseline is 3. The description adds minimal extra semantics by noting the body is passed via a temp file, which slightly clarifies body handling, but otherwise relies on the schema for parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Create a pull request,' which is a specific verb with a clear resource. It distinguishes itself from sibling issue-related tools by focusing on pull requests, and the statement 'this wraps ``gh pr create`` only, not the push' further clarifies its exact scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: 'The current branch must already be pushed to the remote' and explicitly states the tool does not perform the push. It implies the user should use this only after pushing, though it does not explicitly name alternatives or when-not-to-use scenarios beyond the push limitation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.