Server Quality Checklist

Tool Scores

• get_sectionB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool returns 'full text and summary', implying a read-only operation. However, it doesn't specify behavior for invalid indices or any error conditions. For a simple fetch, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main action. Every sentence adds value: the first defines the function, the second explains how to locate the section. There is no unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (fetch with three integer parameters), but there is no output schema. The description mentions 'full text and summary' but does not elaborate on the response structure or what 'summary' entails. It is complete enough for basic use but leaves some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning no parameter descriptions. The parameter names (book_id, chapter_index, section_index) are self-explanatory, but the description adds minimal meaning beyond the schema. It only indirectly references chapter_index and section_index. With zero coverage, the description should compensate more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches one section's full text and summary. It mentions how sections are located via search_book or get_book_outline, but does not explicitly differentiate from sibling tools like get_book_outline, which returns the outline structure. The purpose is clear but not fully differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use this tool: after obtaining indices from search_book or get_book_outline. It does not explicitly state when not to use it or provide alternatives. The usage context is clear but lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_booksA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only lists output fields but does not disclose behavioral details such as pagination, ordering, or performance characteristics. Agent cannot assess if results are paginated or if there are limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is front-loaded with the action ('list every book') and immediately describes the output fields. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are documented. For a simple list tool, the description is mostly complete, but missing potential details like pagination or data limits that an agent might need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the input schema, so schema description coverage is 100%. The description adds no parameter info, but given zero parameters, a baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every book and specifies exactly which fields are returned (id, slug, name, author, type, page count). It is easily distinguishable from sibling tools like book_status, get_book_outline, and search_book.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving a complete list of all books, but does not explicitly state when to use this tool over alternatives like search_book. No when-not or exclusion criteria are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden of transparency. It accurately states the tool returns status data for all books, with no side effects. However, it could be more explicit about potential performance implications (e.g., large result set) or data freshness, but overall it's sufficiently clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence (12 words) that conveys the core purpose efficiently. Every word earns its place; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description is complete. It provides enough context to understand what the tool does without needing elaboration on return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so the description adds no parameter information. Baseline 4 is appropriate since no parameters exist and the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports ingest status (chapter/section counts, chunk counts per stage) for every book. The verb 'report' and resource 'ingest status for every book' are specific. It distinguishes from siblings which handle outlines, sections, listing, and searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like get_book_outline or list_books. The description lacks explicit context for when to invoke or not invoke, and does not mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_book_outlineA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses behavior for auto-split sections (synthesized title) and confirms no section text is returned. With no annotations, the description adequately covers behavioral traits, though it omits explicit read-only indication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with purpose first, then details, then usage flow. However, the paragraph about auto-split sections is slightly verbose for the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers all necessary context: what it returns, what it doesn't, and how it fits into the typical workflow alongside sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter book_id is not described in schema (0% coverage). The description mentions it in the flow but does not explain what it is or how to obtain it, adding minimal meaning beyond the name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it returns the chapter/section outline (the map for get_section). Distinguishes from get_section by specifying it returns metadata like chapter_index, title, page_start, and section details, not the text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides typical flow: list_books -> get_book_outline -> get_section. Also advises using search_book when the target section is unknown, offering clear when-to-use and when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_bookA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses hybrid search, scoring, and return format. Lacks details on rate limits, error handling, or pagination, but is adequate for expected behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two effici

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, return type, and filters. Minor gap on top_k parameter, but overall complete given the presence of an output schema and sibling differentiation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage; the description compensates by explaining book_id, author, source_type (including allowed values). Missing explanation for top_k, but four of five parameters are adequately described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the library and returns cited, scored excerpts. It distinguishes from sibling tools like get_book_outline and get_section by specifying the output type and optional filters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use (searching with excerpts and scoring, optional scoping/filtering) and implies when not to (e.g., use get_section for exact section). It lacks explicit exclusions or alternative naming but provides sufficient context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.