Server Quality Checklist

Tool Scores

• inspect_dangling_refsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint:true, and the description reinforces this with 'Read-only.' Beyond that, it specifies the exact audit categories (dangling refs, unreachable entities, normalization gaps), adding meaningful context beyond the annotation. It does not discuss edge cases or error behavior, but given the annotation coverage, this is a strong but not perfect disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, concise, and front-loaded with the core action and target. It contains zero wasted words and efficiently conveys the tool's purpose and read-only nature. Excellent structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the tool has four parameters and no schema descriptions, making the description the primary source of guidance. The description is too sparse: it omits parameter semantics, default behavior, or any usage nuance. It also fails to differentiate from siblings. Overall, it is incomplete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides no explanation of any parameters (cache, rootIds, includeUnreachable, includeNormalizationGaps). The agent has no guidance beyond parameter names, which is insufficient for correct invocation. The description completely fails to compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Audit a serialized Apollo InMemoryCache... for dangling __refs, unreachable entities, and objects Apollo could not normalize.' It uses a specific verb (audit), a well-defined resource (cache.extract() output), and distinct detectable issues that differentiate it from siblings like patch_cache (which modifies) and diagnose_cache_graph (which likely diagnoses broader graph issues). No ambiguity remains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (audit a cache.extract() output) and sets clear input context, but it does not explicitly state when to use this tool over the sibling tools or provide exclusion criteria. With siblings present, the lack of explicit differentiation means the agent must infer usage context, which is a moderate gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patch_cacheA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate write (readOnlyHint false) and idempotent. The description adds dryRun behavior and optional gc, but it does not clarify whether the input cache is mutated in place or a new store is returned, which is a behavioral gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the action and outcome.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and annotations, the description is too sparse for a tool with a complex operations structure. It does not explain the operation types (modify/evict) in enough detail to construct valid operations, and the gc behavior is only mentioned as optional.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description only minimally addresses parameters: it mentions gc and dryRun, but does not explain the 'operations' structure or the 'cache' parameter beyond being a serialized cache. This is insufficient for the complex nested operations schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as applying declarative repairs (modify/evict, optional gc) to a serialized cache, which is distinct from the sibling diagnostic tools. The verb 'apply' and resource 'cache' are specific, and it mentions the core actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear hint about using dryRun for validation, but it does not explicitly compare with sibling tools or specify when to use this over diagnostic tools. The usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagnose_cache_graphA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already set readOnlyHint=true, and the description reinforces this with 'never mutates' and 'Plans only'. It adds valuable context about the outputs (findings, patch ops, narration) and the flow. No contradictions with annotations; the description enhances behavioral understanding beyond the hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise—two sentences—and front-loaded with the core action. Every clause adds value: the first sentence states what it does and what it returns, the second clarifies safety and the relationship to patch_cache. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex with a nested parameter and an output schema. The description covers the main outputs and the non-mutating nature, and it mentions the sibling patch_cache to complete the workflow. While it doesn't dive into cache format details or edge cases, the presence of an output schema and the clear workflow make it sufficiently complete for selection and basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% for parameter descriptions, so the description must compensate. However, it only says 'serialized cache' without detailing the required structure, expected format, or any constraints beyond the schema's generic object shape. This is insufficient for an agent to correctly construct the 'cache' parameter, especially given its nested complexity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a full inspect -> reason -> plan graph over a serialized cache, and distinctly distinguishes from siblings by mentioning the output is patch operations to feed into patch_cache. It specifies the resource (cache) and the action (diagnose), making it unambiguous and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use this tool: to get findings and proposed patch operations without mutating anything. It says 'Plans only; never mutates' and points to patch_cache for the actual application, giving clear workflow context. It doesn't explicitly say when not to use it (e.g., for targeted inspection with inspect_dangling_refs), but the intent is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.