Server Quality Checklist

Tool Scores

• gp_callersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Mentions sub-millisecond speed and no false positives, but lacks explicit statement of read-only nature or side effects; no annotations to compensate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences plus note; front-loaded with core action and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers core functionality and use cases, but lacks description of output format (list structure) given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds no extra parameter info beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists callers or callees of a symbol, distinguishes from sibling gp_impact by noting multi-hop analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (e.g., before rename) and when not to use (full blast-radius), even recommends alternative tool gp_impact.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gp_impactA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It describes the computed outputs and scope (depth up to 3), but does not explicitly state whether the tool is read-only or has side effects. It mentions replacing a grep command, implying safety, but direct side-effect disclosure is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: start with purpose, then when to use, then special case, then exclusion. Every sentence adds value, no fluff. It is front-loaded with the most important information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains what the tool returns (direct callers, transitive callers, affected tests, export status) and how parameters like 'since' affect results. It covers the key aspects for an agent to invoke and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. The description adds value by explaining default depth (3), the purpose of 'since' for PR review, and path resolution strategy. This context enriches understanding beyond the schema's minimal descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's purpose: 'Compute the blast radius of a rename or signature change'. It lists specific outputs (direct callers, transitive callers up to depth 3, affected tests, export status) and distinguishes it from sibling gp_callers by stating it is not for direct callers only.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance: 'ALWAYS call before proposing a rename, signature change, or behavior change' and 'Do NOT use just to see direct callers; use gp_callers for that.' Also includes a special case for the 'since' parameter to scope to changed files, ideal for PR review.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gp_indexA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that indexing is slow and only needed after source changes. No annotations exist, so description carries behavioral burden. Could mention what happens if index is stale, but current detail is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with action, then usage guidelines. Every word serves a purpose; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, return values are not described, but for a simple re-index action this is acceptable. Covers when, why, and side effects (slowness). Minor gap: no mention of success/failure feedback.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by detailing the resolution order for the optional path parameter: GRAPHPILOT_ROOT, MCP workspace roots, parent walk, or a single ~/.graphpilot index.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb "re-index" and resource "repo", and distinguishes from sibling tools (gp_callers, gp_impact, gp_recall) by stating when to call (after batch edits) and when not to (not before every query).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: "after batch edits" and when not to: "Do NOT call before every query". Also provides rationale: "indexing is slow; only needed when source files changed."

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gp_recallA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses pre-indexed nature, no false positives, sub-millisecond performance, and path resolution behavior. Does not cover error handling (e.g., symbol not found), but overall good context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Highly concise and front-loaded: starts with purpose, then guidelines, then parameter hints. Every sentence adds value with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description specifies return values (kind, file:line, signature). All 4 parameters are explained. Sibling tools are listed. Complete for the intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (baseline 3). The description adds meaning by explaining substring usage ('Pass substring:true for partial-name searches') and path resolution behavior, going beyond the simple schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds a symbol definition by name, returning kind, file:line, and signature. It distinguishes from sibling tools by explicitly stating to use instead of grep and not for 'who calls X?' (use gp_callers).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when to use ('ALWAYS use instead of grep -rn...') and when not to use ('Do NOT use for "who calls X?" — use gp_callers for that'), along with hints for substring parameter usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.