Server Quality Checklist

Tool Scores

• pdf-metadataB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool extracts metadata but doesn't mention error handling (e.g., for invalid files), performance characteristics, or what happens if metadata fields are missing. For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and provides examples of metadata fields. There is no wasted verbiage, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is minimally adequate. However, it lacks details on return values (since no output schema exists) and behavioral context, which are important for a tool performing extraction. It's complete enough for basic understanding but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'file' clearly documented as 'Path to the PDF file to get metadata from'. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Extract') and resource ('metadata from a PDF file'), and lists example metadata fields. It distinguishes from the 'read-pdf' sibling (likely for content extraction) and 'search-pdf' (likely for text search), though not explicitly. A 5 would require explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools 'read-pdf' or 'search-pdf'. It also lacks information about prerequisites (e.g., file accessibility) or alternative approaches. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read-pdfA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's behavior by stating it 'returns the full text content' and mentions optional features like page filtering and text cleaning. However, it lacks details on error handling, performance characteristics, or limitations (e.g., file size constraints, supported PDF formats), which would be valuable for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose ('Extract text from a PDF file') followed by additional context about return values and optional features. Every sentence adds value without redundancy, making it efficiently structured and appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema, no annotations), the description is reasonably complete. It covers the main purpose and key behavioral aspects (returning text, optional features). However, without an output schema, it could benefit from more detail on the return format (e.g., structure of the text output, handling of metadata), slightly reducing completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear documentation for all 4 parameters. The description adds minimal value beyond the schema by mentioning 'optional page filtering and text cleaning', which loosely corresponds to the 'pages' and 'clean_text' parameters but does not provide additional semantic context. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Extract text from a PDF file') and resource ('PDF file'), distinguishing it from sibling tools like 'pdf-metadata' (which likely extracts metadata) and 'search-pdf' (which likely searches within PDFs). The verb 'extract text' precisely defines the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for text extraction from PDFs but does not explicitly state when to use this tool versus alternatives like 'pdf-metadata' or 'search-pdf'. It mentions optional features (page filtering, text cleaning) which provide some context, but lacks explicit guidance on tool selection scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-pdfA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the return format ('Returns matching text with context and page numbers'), which is valuable behavioral information. However, it doesn't mention potential limitations like maximum file size, search performance, error conditions, or authentication requirements that would help an agent understand operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first states the purpose, the second specifies the return format. No wasted words, front-loaded with core functionality, and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, search operation), no annotations, and no output schema, the description provides good coverage of what the tool does and what it returns. However, it lacks details about error handling, performance characteristics, or limitations that would make it fully complete for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. This meets the baseline expectation when schema coverage is complete, but provides no additional semantic context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for specific text'), target resource ('within a PDF file'), and distinguishes from siblings by focusing on text search rather than metadata extraction (pdf-metadata) or full content reading (read-pdf). It provides a complete purpose statement with verb+resource+differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (searching within PDFs) but doesn't explicitly state when to use this tool versus alternatives like read-pdf for full content or pdf-metadata for file information. No explicit when-not-to-use guidance or named alternatives are provided, leaving usage decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.