Server Quality Checklist

Tool Scores

• search_book_contentC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return format ('list of hits with text snippets') and pagination behavior via offset, but lacks critical details like authentication requirements, rate limits, error conditions, or whether this is a read-only operation. For a search tool with 5 parameters, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with three sentences that each serve a purpose: stating the core functionality, describing the return format, and explaining two parameters. However, the structure could be improved by front-loading the most critical information about all parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, 0% schema description coverage, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'library_name' does, doesn't clarify the relationship between 'book_id' and 'library_name', and provides minimal behavioral context. The agent would struggle to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate but only partially does so. It explains 'hit_limit' and 'offset' parameters, but doesn't mention 'book_id', 'query', or 'library_name' at all. The three undocumented parameters remain unexplained, leaving significant gaps in understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for text within a book and returns hits with snippets, which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'search_books' or 'fts_search', which likely have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_books' or 'fts_search'. The description only states what the tool does, not when it's appropriate or what distinguishes it from similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fts_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the return format (list of hits with book_id and snippet) but lacks details on permissions, rate limits, pagination, or error handling. For a search tool with zero annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of two sentences that efficiently convey the core functionality and output. There is no wasted language, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It explains the return format but misses behavioral aspects like search scope, performance hints, or error cases. For a tool with 2 parameters and no structured support, more context is needed to guide effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The tool description does not mention any parameters explicitly, failing to compensate for the coverage gap. However, with only 2 parameters (one required, one optional with a default), the baseline is moderate as the schema structure is simple, but no value is added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Full-Text Search for a string' specifies the verb and resource, and 'Returns a list of hits that include a book_id and a short snippet of text' explains the output. It distinguishes from siblings like 'search_book_content' by focusing on full-text search across books rather than content within a specific book, though the distinction could be more explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_book_content' or 'search_books'. The description implies usage for finding text snippets across books but does not specify contexts, exclusions, or prerequisites, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_book_contentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions a 'limit' parameter with a default and 'sentence_aware' behavior, but doesn't disclose critical behavioral traits: whether this is a read-only operation, potential rate limits, authentication needs, error conditions, or what happens if book_id is invalid. For a tool with 5 parameters and no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—four sentences total, with the first stating the purpose and the next three explaining key parameters. Every sentence adds value, and it's front-loaded with the core function. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with 0% schema coverage, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format (e.g., text structure, error responses), doesn't cover all parameters, and lacks behavioral context. For a content retrieval tool with multiple parameters, this leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains three parameters (limit, offset, sentence_aware) with some semantics (e.g., 'adjusts to nearest sentence boundary'), but doesn't cover 'book_id' (required) or 'library_name'. Since it documents 3 of 5 parameters partially, it adds value but doesn't fully compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and resource 'text content of a book', making the purpose evident. However, it doesn't explicitly differentiate from sibling tools like 'get_book_details' or 'search_book_content', which likely retrieve different aspects of books. The purpose is clear but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_book_details' (likely metadata) and 'search_book_content' (likely search within content), there's no indication of when this retrieval tool is appropriate versus those. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_book_detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool returns metadata, but doesn't specify permissions required, rate limits, error handling, or the format of the returned data (e.g., JSON structure). For a read operation with no annotation coverage, this leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, starting with the core purpose in the first sentence. The second sentence efficiently explains the 'fields' parameter. There's no wasted text, making it easy to parse quickly, though it could be slightly more structured with bullet points or clearer separation of ideas.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (3 parameters, no annotations, no output schema), the description is incomplete. It covers the tool's purpose and one parameter but misses details on behavior, usage context, and the other parameters. For a tool that retrieves metadata, more information on what metadata is available, error cases, or example outputs would be beneficial to fully guide an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds some meaning for the 'fields' parameter by explaining it as 'an optional list of metadata fields to return' with examples, which is helpful since schema description coverage is 0%. However, it doesn't address the 'book_id' (required) or 'library_name' parameters, leaving their purposes unclear. With 3 parameters and low schema coverage, the description partially compensates but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed metadata for a book.' It specifies the verb ('Get') and resource ('detailed metadata for a book'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from siblings like 'get_book_content' or 'search_books', which might also retrieve book information but with different scopes or methods.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_book_content' (likely for content retrieval), 'search_books' (likely for searching), and 'get_field_values' (possibly for specific fields), there's no indication of when this tool is preferred, such as for comprehensive metadata versus filtered results or other use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_library_schemaB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool is 'useful for understanding available metadata fields,' which implies a read-only, informational operation, but doesn't explicitly state behavioral traits like whether it requires authentication, has rate limits, returns structured data, or handles errors. For a tool with no annotations, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second adds practical context without redundancy. Both sentences earn their place by providing essential information efficiently, with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is somewhat complete but has gaps. It explains what the tool does and its utility, but without annotations or output schema, it should ideally cover more behavioral aspects (e.g., return format, error handling). The description is adequate for basic understanding but could be more comprehensive for a tool that might return complex schema data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, and the tool description doesn't mention any parameters. Since schema_description_coverage is low (<50%), the description should compensate but doesn't add any parameter information. However, with 0 parameters effectively (since 'library_name' has a default of null and is not required), the baseline is 4, but the description doesn't clarify the optionality or purpose of 'library_name', so it's scored lower at 3 for missing this nuance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the schema of the library, including standard and custom columns.' This specifies the verb ('Get') and resource ('schema of the library') with additional detail about content ('standard and custom columns'). However, it doesn't explicitly differentiate this from sibling tools like 'get_field_values' or 'search_books', which might also involve library metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance: 'Useful for understanding available metadata fields for search and update.' This suggests when to use it (to understand metadata for search/update operations) but doesn't explicitly state when not to use it or name alternatives among sibling tools. For example, it doesn't clarify if this should be used instead of 'get_field_values' for schema information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_booksA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context about pagination behavior (limit/offset defaults), fallback behavior when query is missing, and metadata field filtering. However, it doesn't cover important aspects like rate limits, authentication needs, error conditions, or what the response structure looks like (e.g., format of returned results).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. Each sentence adds specific value: query behavior, pagination details, field selection, and text field limits. There's minimal redundancy, though the structure could be slightly more organized with bullet points or clearer separation of concepts.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema), the description provides adequate coverage for most parameters and basic behavior. However, it lacks details about the response format, error handling, and the undocumented 'library_name' parameter. For a search tool with multiple siblings, more guidance on tool selection would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 6 parameters, the description compensates well by explaining 5 out of 6 parameters: query (with fallback behavior), limit and offset (pagination with defaults), fields (optional metadata selection), and text_field_limit (max length for text fields). Only 'library_name' is undocumented, leaving a minor gap in coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Search for books matching a metadata query' with the verb 'search' and resource 'books', making it immediately understandable. It distinguishes from siblings like 'get_book_details' (specific book) and 'search_book_content' (content search), though it doesn't explicitly name these alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by stating 'If query is not provided, lists all books,' which helps understand the tool's behavior with missing parameters. However, it doesn't explicitly guide when to use this tool versus alternatives like 'fts_search' or 'search_book_content,' leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_field_valuesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the core functionality well but doesn't mention important behavioral aspects like whether this is a read-only operation, potential performance implications for large datasets, or what happens with invalid field names. The description adds value by explaining the facet-building use case but misses key operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured - a clear purpose statement followed by parameter explanations. Every sentence earns its place, with no wasted words. The front-loaded purpose statement immediately communicates the tool's function, followed by essential parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, no annotations, and no output schema, the description does a good job with parameters but lacks information about return format, error conditions, or performance characteristics. It's complete enough for basic understanding but misses important contextual details an agent would need for robust implementation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic explanations for all 6 parameters. Each parameter is described with its purpose and constraints (e.g., 'Optional list of book IDs to restrict the search to', 'Optional regex to filter values', 'Max number of values to return (default 50)'), adding significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('unique values and their counts for a specific metadata field'), distinguishing it from siblings like search or content retrieval tools. It explicitly mentions the use case for 'building facets' which helps differentiate its analytical function from data-fetching siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for building facets'), but doesn't explicitly state when not to use it or name specific alternatives among the sibling tools. It implies usage for metadata analysis rather than content retrieval, but lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.