Server Quality Checklist

Tool Scores

• read_pdf_pageC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the basic action but fails to cover critical aspects like error handling (e.g., invalid page numbers), performance implications (e.g., large PDFs), or output format (e.g., text extraction vs. raw data), leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no wasted words, front-loading the core functionality. It efficiently communicates the tool's purpose without unnecessary elaboration, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a 4-parameter tool with no annotations, 0% schema coverage, and no output schema, the description is incomplete. It lacks details on parameter usage, error conditions, and return values, which are essential for an agent to invoke the tool correctly in varied contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters. It mentions 'page or range of pages', hinting at 'page', 'startPage', and 'endPage', but doesn't clarify their relationships (e.g., that 'page' is for single pages, while 'startPage' and 'endPage' define ranges) or the 'path' parameter's role, failing to add sufficient meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read') and resource ('a specific page or range of pages from a PDF file'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'read_pdf' or 'search_pdf', which would be needed for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'read_pdf' or 'search_pdf'. It lacks context about scenarios where reading specific pages is preferred over other operations, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pdfC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'Search for specific text' but does not describe what happens if the text is not found, whether it returns matches or just a boolean, or any performance or error-handling traits. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly, with no wasted information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search operation with 3 parameters, no annotations, and no output schema, the description is incomplete. It does not cover parameter meanings, behavioral outcomes, or return values, making it inadequate for the agent to fully understand how to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description does not mention any parameters, failing to add meaning beyond the schema. It does not explain what 'path', 'query', or 'caseSensitive' mean in context, leaving all three parameters undocumented and unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search for specific text') and resource ('within a PDF file'), making the purpose immediately understandable. However, it does not differentiate from sibling tools like 'read_pdf' or 'read_pdf_page', which might also involve text extraction, so it lacks explicit sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_pdf_metadata' or 'read_pdf'. It implies usage for text searching but offers no context on prerequisites, exclusions, or specific scenarios, leaving the agent to infer usage without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pdf_metadataB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'without reading all content', which hints at performance characteristics, but doesn't disclose important behavioral traits like what metadata fields are returned, whether file access permissions are required, error conditions, or response format. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with a single sentence that efficiently communicates the core purpose and key constraint. Every word earns its place, and it's front-loaded with the main functionality. There's zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and 0% schema description coverage, the description is incomplete. It doesn't explain what metadata is returned, how errors are handled, or provide sufficient parameter guidance. While the purpose is clear, the description lacks the contextual information needed for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for its single parameter 'path', and the tool description provides no information about this parameter. The description doesn't explain what 'path' represents (file path, URL, etc.), expected format, or constraints. With low schema coverage and no parameter information in the description, it fails to compensate adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get metadata information from a PDF file' with the specific verb 'Get' and resource 'metadata information from a PDF file'. It distinguishes from sibling tools by specifying 'without reading all content', which differentiates it from read_pdf and read_pdf_page. However, it doesn't explicitly contrast with search_pdf, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'without reading all content', suggesting this tool is for lightweight metadata extraction rather than full content reading. However, it doesn't provide explicit guidance on when to use this tool versus alternatives like search_pdf or when not to use it. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pdfA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that the tool 'Returns the full text content and metadata,' which gives some behavioral insight into output. However, it lacks details on error handling (e.g., invalid paths, corrupted files), performance (e.g., large file handling), or side effects (e.g., file locking). For a tool with zero annotation coverage, this is a significant gap in behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by return information. Every word earns its place with no redundancy or fluff, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (reading PDFs) and lack of annotations or output schema, the description is adequate but incomplete. It covers the basic purpose and output types but misses details on parameter semantics, error conditions, and behavioral nuances. For a tool with no structured support, it provides a foundation but leaves gaps in full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It does not add any meaning beyond the schema—no explanation of what 'path' represents (e.g., file path, URL, format) or constraints. However, with only one parameter, the baseline is higher, but the description fails to provide additional semantic context, resulting in a minimal viable score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Read and extract text content') and resource ('from a PDF file'), distinguishing it from sibling tools like get_pdf_metadata (metadata only), read_pdf_page (single page), and search_pdf (search within). It precisely defines the tool's function beyond just restating the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for extracting full text and metadata from PDFs, but does not explicitly state when to use this tool versus alternatives like read_pdf_page for specific pages or search_pdf for targeted content. No exclusions or prerequisites are mentioned, leaving usage context somewhat open to interpretation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.