Server Quality Checklist

Tool Scores

• handbook_create_entryA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false, which are consistent with a creation operation. The description adds the return type ('Returns: The created HandbookEntry') but does not disclose other behavioral traits like required permissions or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a clear first sentence and a parameter list, and includes a return statement. It is not overly verbose, though the parameter list is redundant with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose and parameters. With no output schema, it mentions the return type but lacks detail on the returned object's structure. For a creation tool, this is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%; all parameters have descriptions in the schema. The description enumerates the parameters with types and optionality but adds little semantic value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Create a new handbook entry,' clearly stating the verb and resource. It distinguishes from siblings such as handbook_update_entry (update) and handbook_list_entries (list), as those are separate tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The sibling tools are listed, but the description does not indicate prerequisites, context, or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handbook_update_entryA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description correctly indicates a mutation (update) but adds no additional behavioral context beyond the annotations; no information about preconditions, error behavior, or effects on related data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a clear statement and Args/Returns sections, but the Args section essentially duplicates the schema, making it slightly longer than necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks details about the return value format and potential errors; while annotations and schema cover basics, the description could be more complete for proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are already well-documented in the input schema; the description merely lists them without additional semantics, providing no added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (update) and the resource (handbook entry), and references the identifier (ID), making it easy to distinguish from create and list tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus the sibling tools; it only implies update through the verb, but lacks comparison or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handbook_search_tagsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the tool's safety profile is clear. The description adds return structure ('HandbookTagsResponse with array of matching tag strings') but does not disclose other behaviors like case sensitivity, pagination, or limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the main purpose. The Args/Returns section adds some redundancy but is not excessively long. Could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers the essential aspects. It lacks details on pagination or result limits, but given the low complexity, it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and includes a description for searchQuery. The description repeats the param info without adding new meaning, so baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search for tags across all handbook entries.' It uses a specific verb ('Search') and resource ('tags'), and the scope is explicit. Sibling tools deal with handbook entries (list, get, create, etc.), so this tool is well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for tag searching but does not explicitly state when to use this tool versus alternatives (e.g., filtering entries instead). No guidance on when not to use it or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handbook_get_overviewA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, idempotent, non-destructive. The description adds the lightweight nature and specifies which fields are returned (title, ID, app, tags), enhancing transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Highly concise: two short paragraphs covering purpose, arguments, and return type with zero waste. Front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains the return structure (array of overview entries). For a simple overview tool with one parameter, it is completely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with an adequate parameter description. The tool description repeats the parameter example, adding marginal value; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets a lightweight overview of handbook entries for a specific app, returning only title, ID, app, and tags. It distinguishes from siblings like handbook_get_entry (which likely returns full content) and other CRUD tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a lightweight overview is needed rather than full content, but does not explicitly contrast with siblings or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handbook_get_entryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already confirm read-only, idempotent, non-destructive. Description adds that content is in markdown and returns a full object, which is useful context. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, well-structured sentences. First line states purpose, then docstring-style breakdown. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, parameter, and return value well. Simple tool with one param and no output schema, so description is adequate. Could mention error handling briefly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage, baseline 3. Description adds 'MongoDB ObjectId' detail not in schema, providing format guidance beyond the schema's generic 'ID'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Retrieve a single handbook entry by its ID' with a specific verb and resource. Distinguishes from siblings like handbook_list_entries and handbook_create_entry.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. While it's implied for retrieving a specific entry, it doesn't contrast with siblings like handbook_search_tags or handbook_list_entries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handbook_list_entriesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, so the safety profile is clear. The description adds that it returns a list of objects with specific fields, but does not disclose any additional behavioral traits (e.g., pagination, rate limits).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, with two paragraphs. The first sentence states the purpose, the second lists return fields, and the third gives usage guidance. Every sentence is valuable and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and rich annotations, the description is adequate. It explains the return fields and when to use it. However, it could mention whether results are paginated or ordered, but for a simple list tool, it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (0), so the schema coverage is 100%. The baseline for 0 parameters is 4. The description does not need to add parameter semantics, and it correctly omits any.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve all handbook entries' with a specific verb and resource. It distinguishes itself from siblings like handbook_get_entry (single entry) and handbook_search_tags by focusing on listing all entries, and explicitly mentions its use for an overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use when: you need an overview of all existing entries or want to find an entry ID.' This provides clear guidance, though it does not mention when not to use it or list alternatives directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.