Server Quality Checklist

Tool Scores

• book_infoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose behavioral traits such as permissions, side effects, error conditions, or whether the operation is read-only. It only lists output fields without explaining their semantics or any potential limitations, leaving key behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loads the core purpose. It is a single sentence with a list of returned fields, and every word earns its place. It loses a point because the reference to `lib info` may be unclear to an agent without external context, but overall it is well-structured and brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description is the only source of context, but it fails to define the meaning of 'metrics', 'subscores', or 'hard_triggers' and does not explain how the output is structured. The presence of sibling tools suggests a need to differentiate usage, yet no such context is provided. The simple parameter list does not compensate for the missing depth.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required parameter (`book_id`) with no schema description, and the parameter description coverage is 0%. The tool description does not mention the parameter at all, so no additional meaning is added beyond the parameter name. The agent has no guidance on the expected format, source, or constraints of the book ID, making this a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (a book) and the type of data returned (metadata and quality metrics). It lists specific fields and references `lib info`, which distinguishes it from sibling tools that focus on listings, chapters, or quote verification. However, it lacks an explicit verb like 'retrieves' or 'returns', making it slightly less direct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as list_books or get_chapters. The mention of `lib info` is a reference to an external command, not an alternative tool, and no exclusions or conditions are provided. The agent is left to infer the usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_chaptersB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the output fields (chapter number, title, token count, summary) which is useful, but does not mention permissions, limitations, or any side-effect behavior. This is adequate but not comprehensive for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the tool's purpose and output. Every word contributes to understanding, with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and an output schema exists, the description sufficiently covers the essential behavior. It lacks usage alternatives and edge-case details, but for a straightforward list operation, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description's phrase 'by its id' adds minimal semantic meaning to book_id, clarifying it is the book identifier. However, it does not explain the id format or any constraints, leaving the parameter only slightly enhanced over the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing a table of contents for a book by its id, listing chapter number, title, token count, and summary. This distinguishes it from list_books and book_info, though it does not explicitly contrast with the similarly named get_chapters sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like get_chapters. The description merely states what it does, leaving the agent to infer usage context without any exclusions or preference rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_booksA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return field names, which is useful, but does not explicitly state that it is a safe read-only operation or mention any limitations. For a simple list operation, this is adequate but not rich in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that delivers all necessary information without fluff. Every phrase earns its place, making it highly concise and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with an output schema, the description sufficiently covers the purpose and return fields. It does not mention ordering or filtering, but these are likely optional for a simple list-all operation. Overall, the description is complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter documentation needed. The description correctly avoids adding unnecessary detail, matching the baseline for tools without parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all books in the library', specifying the exact fields returned. It naturally distinguishes itself from sibling tools like list_chapters and verify_quote, which target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a complete list of all books is needed, but does not explicitly mention alternatives or when not to use it. The context makes the basic use case clear, but there is no exclusions or comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• findA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It goes beyond the bare minimum by mentioning 'bm25 ranking, snippets, RU/EN stemming' and how book_id restricts the search. This adds meaningful context about how the tool operates internally.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences), front-loaded with the main purpose, and every word adds value. It efficiently packs scope, ranking method, language support, and parameter behavior without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the search scope, algorithm, snippets, stemming, and book_id restriction, which is substantial for a search tool with no output schema. It lacks explicit details on return format beyond 'snippets', but this is a minor gap given the sibling context and simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description explains book_id ('restricts the search to one book') and implies query's role via 'full-text search'. However, the limit parameter is not described at all, leaving a gap in understanding its purpose and behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Full-text search across chapters and book titles/authors in the whole library'. This specific verb ('search') and resource ('chapters, titles, authors') distinguish it from siblings like list_books or verify_quote, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the search scope ('whole library') and the optional restriction to one book via book_id. It does not explicitly mention alternatives, but the context is clear: use this tool when you need to search content, while siblings handle listing or verification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_chaptersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden, and it does well: it discloses the default 12000-token budget, the error condition for passing both parameters, and the graceful fallback of returning empty text with an explanation instead of throwing an error. It does not disclose return format details or behavior of the undocumented from_ parameter, which keeps it from a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tightly packed sentences deliver the core purpose, usage modes, default behavior, error condition, and fallback behavior without any filler. The most important information is front-loaded in the first sentence, and every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description covers the essential behavioral context: modes, default, error handling, and fallback. The main gaps are the undocumented from_ parameter and the absence of any description of the returned data shape beyond 'text' and 'message,' so it is strong but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does a good job explaining spec (with an example) and budget (token-based), plus the default and mutual-exclusivity constraints. However, schema description coverage is 0%, and the from_ parameter is completely undocumented in both the schema and the description, leaving a meaningful gap for an integer parameter with a default that is not self-evident.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Chapter text of a book,' a specific verb-plus-resource statement that clearly distinguishes get_chapters from sibling tools like list_chapters (which likely lists chapter metadata). It further clarifies the two retrieval modes (by spec or budget), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use spec vs. budget, the default behavior when neither is provided, and that passing both is an error. It also gives retry guidance when the budget is too small. However, it does not mention when to use this tool over sibling tools like find or book_info, so it stops short of full alternative-tool guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_quoteA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses execution behavior (full scan vs FTS5 candidates) and the expected return content (verdict, location, similarity, passage, diff), which goes beyond a basic summary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and outputs, then usage notes. No redundant wording; every clause adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description details return values and both scanning modes. It gives an agent enough context to invoke the tool correctly, with only the 'limit' parameter lightly documented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates: it thoroughly explains book_id's role and implies quote's meaning. However, 'limit' is not described, leaving its effect (e.g., number of FTS5 candidates) ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a precise action with a specific verb and resource: 'Verify a quote against the library' and enumerates the verdict types and return fields. It also distinguishes from sibling tools by declaring it is the tool to use when quoting a book.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this whenever you quote a book to the user' and explains two usage modes (with book_id vs omit). However, it does not name alternative tools or state when not to use it, though the distinction is clear from the sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.