Server Quality Checklist

Tool Scores

• list_pdfsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's read-only nature (implied by 'Find' and returns data) and output format ('Returns name, path, pages'), but lacks details on error handling, performance limits, or permissions required for directory access. It adds value beyond the schema by explaining the return structure and integration with 'read_pdf'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, with three sentences that each serve a clear purpose: stating the tool's function, providing a parameter example, and explaining the output and integration. There is no wasted text, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (searching files with filtering), no annotations, and no output schema, the description is largely complete. It covers purpose, usage, and output structure, but lacks details on error cases or performance constraints (e.g., large directories). The integration hint with 'read_pdf' enhances context, though some behavioral gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description adds minimal semantic value beyond the schema, only mentioning 'name_pattern' with an example ('e.g., '*report*''), which is already covered in the schema description. No additional parameter insights are provided, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Find PDF files'), target resource ('in a directory'), and scope ('Returns name, path, pages for each PDF'). It distinguishes from sibling tools by explicitly mentioning 'read_pdf' for a different purpose, avoiding overlap.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use this tool ('Find PDF files in a directory') and when to use an alternative ('Use the returned 'path' directly with read_pdf'), clearly differentiating from the sibling 'read_pdf' tool. It also includes a practical example ('e.g., '*report*'') to illustrate usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pdfA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the 10-page limit per request, compatibility with both text and scanned documents, and the distinction between extraction modes. However, it doesn't mention error handling, performance characteristics, or authentication requirements, which would be valuable for a tool with 8 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with four concise sentences that each serve a distinct purpose: stating the core function, providing usage preference, specifying limits/compatibility, and explaining mode options. It's front-loaded with the most important information. The only minor improvement would be integrating the page limit more naturally with the parameter explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no output schema, no annotations), the description does a good job covering the essential context. It explains the tool's purpose, when to use it, key limitations, and high-level mode options. However, for a tool with this many parameters and no output schema, it could benefit from mentioning what the return format looks like or providing more guidance on parameter combinations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema - it mentions the 'image_only' and 'text_only' modes which are already in the schema's enum, but doesn't provide additional context about parameter interactions or usage patterns. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Read PDF content') and distinguishes it from alternatives ('Always prefer this over cat or file read for PDF files'). It identifies the resource (PDF files) and differentiates from sibling tools like 'grep_pdf' and 'list_pdfs' by focusing on content extraction rather than searching or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs. alternatives ('Always prefer this over cat or file read for PDF files') and offers specific usage contexts ('Use 'image_only' to see actual page layout, or 'text_only' for pure text'). It clearly directs the agent away from generic file operations for PDFs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grep_pdfA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully describes key behavioral traits: that it returns 'matching lines with page numbers' and has 'No page limit (unlike read_pdf's 10-page limit).' However, it doesn't mention performance characteristics, error conditions, or what happens with invalid PDFs. For a tool with no annotations, this is good but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (3 sentences) and front-loaded with the core purpose. Every sentence earns its place: first states what it does, second explains why to use it over alternatives, third distinguishes from sibling tool. Zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (10 parameters, no output schema, no annotations), the description does well by explaining the core behavior and key differentiators. However, it doesn't describe the return format in detail (beyond 'matching lines with page numbers') or error handling. For a search tool with rich parameters but no output schema, this is good but could be more complete about results structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. According to scoring rules, with high schema coverage (>80%), the baseline is 3 even with no param info in description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search text in PDFs' (verb+resource). It explicitly distinguishes from standard grep/rg tools that don't work on PDFs, and distinguishes from sibling 'read_pdf' by noting the lack of page limit. This is specific and clearly differentiates from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Standard grep/rg does NOT work on PDFs (binary format). Use this tool instead.' It also distinguishes from sibling 'read_pdf' by noting 'No page limit (unlike read_pdf's 10-page limit).' This gives clear when-to-use and when-not-to-use guidance with named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.