Server Quality Checklist

Tool Scores

• how_to_use_forgetful_toolB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description implies a read-only operation returning documentation, with no mention of side effects, authentication, or rate limits. It is adequate but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description includes an Args and Returns section that, while structured, contains some redundant phrasing. Could be more concise, but is still reasonably clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the return values need not be detailed. However, the tool is simple (1 param, read-only), so the description is mostly complete, though usage guidance is lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains the single parameter 'tool_name' as 'Name of the tool to get documentation for', which adds meaning beyond the bare schema that lacks a description. Schema coverage is 0%, but the description compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns documentation for a specific tool, but does not explicitly distinguish from the sibling 'discover_forgetful_tools' which likely lists all tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus its siblings 'discover_forgetful_tools' or 'execute_forgetful_tool'. The agent is left to infer the use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_forgetful_toolA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. Does not explicitly state mutation, side effects, errors, or preconditions. Examples imply create/update/delete actions but no warnings about irreversible operations or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very verbose (many lines of examples and best practices). While structured with sections, it lacks conciseness; not every sentence earns its place. Could be shortened by referencing sibling tools more.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a dynamic dispatcher, covers many sub-operations and linking strategies. Points to discover_forgetful_tools for parameter details. Lacks explanation of error handling or invalid tool_name behavior, but otherwise fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description compensates with extensive examples for each sub-tool (memory, project, entity, etc.), showing exact argument structures. Adds significant semantic value beyond the minimal schema, though not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool executes any registered tool dynamically. Differentiates from sibling tools (discover_forgetful_tools for discovery, how_to_use_forgetful_tool for help) by focusing on execution and providing a quick-start guide.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage as the executor after discovering tools, but does not explicitly state when to use vs alternatives. Points to discover_forgetful_tools for details, but lacks exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_forgetful_toolsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source for behavioral traits. It explains the return structure (dictionary with keys like tools_by_category, total_count) and states it returns enough info for direct tool calls. However, it does not explicitly state that the tool is read-only or non-destructive, though that is implied. It could also mention any limitations like caching or performance, but for a discovery tool, the disclosure is fairly adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is overly long (over 300 words) and includes a full catalog of all available tools, which is redundant as the tool's output will contain that information. It is structured with sections and bullet points, but conciseness is compromised due to the inclusion of extensive listing that goes beyond the tool's own functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema (context signal), the description adequately covers the return structure and usage scenario. It explains the single parameter and provides a workflow. The inclusion of the full catalog, while not strictly necessary, adds context for understanding the tool ecosystem. Overall, it is complete enough for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one optional parameter 'category' with no description (0% schema coverage). The description adds meaning by listing the allowed categories (user, memory, project, code_artifact, document, entity, linking) and explaining the filtering behavior. It does not specify case sensitivity or format, but the provided list is sufficient for practical use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover available tools, optionally filtered by category' and explicitly distinguishes it from siblings by mentioning that the returned information is sufficient for LLMs to call tools directly without needing how_to_use. It is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a step-by-step workflow that explicitly tells when to use this tool ('1. Call discover_forgetful_tools() to see this catalog') and when to use alternatives ('2. Call execute_forgetful_tool', '3. If needed, call how_to_use_forgetful_tool'). It also explains the optional category filter, making usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.