Server Quality Checklist

Tool Scores

• batch_convertC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses batch conversion and a summary return, but omits side effects (e.g., file modifications), error handling, and dependencies. The behavioral disclosure is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences are concise, but the brevity sacrifices needed detail. While not verbose, missing parameter explanations and behavioral context reduce effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists but is not explained (summary with per-file results is vague). Sibling tools provide contrast, but the description lacks completeness on error handling, output format, and parameter details for quality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description only mentions 'directory' implicitly and does not explain the 'quality' parameter (e.g., standard vs high). No added meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Convert all PDFs in a directory to Markdown', specifying the verb (convert), resource (all PDFs in a directory), and output format (Markdown). This distinguishes it from sibling tools like convert_pdf (single file) and extract_streaming (streaming extraction).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs alternatives, no prerequisites (e.g., directory existence, permissions), and no exclusions. It simply states what it does without context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_pdfA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description transparently discloses key behaviors: automatic PDF type detection, best extraction method selection, and return of confidence scores and warnings. This adds value beyond the schema, though it doesn't mention potential side effects or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted words. Essential information is front-loaded: action, resource, then additional features. Highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While purpose and output details are covered, the lack of parameter descriptions (especially for 'format' and 'quality') leaves the tool incomplete for effective use. An output schema exists but is not shown; description should compensate but does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, placing full burden on the description to explain parameters like 'format' and 'quality'. The description fails to provide any details about these parameters, offering no guidance on valid values or behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Convert a PDF to AI-readable Markdown') and resource, distinguishing it from siblings like analyze_pdf or extract_streaming. The verb 'Convert' and target format 'Markdown' make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for general PDF-to-Markdown conversion with automatic detection, but does not explicitly state when to use alternatives like extract_structured or batch_convert. No when-not or exclusions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_streamingA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden. It details the streaming format (NDJSON), event types (classified, page, warning, complete), and ordering. It also mentions OCR re-extraction in standard/high quality. However, it does not cover error handling or authorization, which would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose and includes a detailed event format example. While the code block is somewhat lengthy, it provides essential context. Every sentence adds value, making it appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a streaming tool, the description covers event types, ordering, and use case. It implicitly defines the output schema through examples. It does not cover error scenarios or timeouts, but given the tool's nature, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions quality modes but does not explicitly explain the file_path parameter or possible quality values beyond 'standard/high'. The description adds little meaning beyond the schema's names and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool streams extraction events for a PDF as NDJSON, with a specific use case for large documents. However, it does not explicitly distinguish itself from sibling tools like extract_structured, which would have earned a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends use for large documents (100+ pages) where waiting is impractical, providing clear context. However, it does not mention when NOT to use it or suggest alternatives, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_structuredA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions auto-normalization and return types but does not disclose read-only behavior, permissions, or error handling. It provides moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is front-loaded with key actions and outputs, containing no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description sufficiently explains return values. It covers main features but lacks details on error conditions or prerequisites. Overall, it is fairly complete for a data extraction tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (no parameter descriptions in schema). The description adds context for the schema parameter (optional mapping) but does not explain file_path or quality. It partially compensates for the gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structured data from PDFs, specifying outputs like tables as JSON and key-value pairs with auto-normalization. It distinguishes from sibling tools by focusing on structured extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for structured data extraction but lacks explicit guidance on when to use this tool versus alternatives like extract_streaming or analyze_pdf. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_pdfA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description is sole source. Discloses return values and non-destructive nature, but lacks details on prerequisites, errors, or limitations. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose and key differentiator. No unnecessary words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (1 param, has output schema), and description covers purpose, returned data, and cost comparison. Minor gaps (e.g., file limitations) but overall sufficient for a triage tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has one parameter (file_path) with 0% description coverage. Description does not add detail about the parameter type, format, or constraints beyond what the schema already shows, missing opportunity to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states tool does PDF triage, classifying type and auditing page quality without full extraction. It explicitly differentiates from sibling convert_pdf by noting it is cheaper and for initial assessment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends use over convert_pdf for initial assessment and notes it avoids full extraction, providing clear context. Does not cover all sibling tools but adequately guides when to use this one.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pdf_metadataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Clearly states no extraction, implying read-only behavior, and lists metadata returned. While it doesn't cover all behavioral details (e.g., file existence requirements), it is transparent enough for a simple metadata tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences covering purpose, output, and usage guidelines with zero wasted words; information is front-loaded and clearly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description adequately covers what the tool returns and how to use it in a workflow. Lacks error handling details but is sufficient for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'file_path' has no schema description (0% coverage), and the description does not add details about valid file types, path format, or access requirements, leaving interpretation to the field name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states it retrieves PDF metadata (page count, file size, document type, table presence) and distinguishes from siblings by clarifying no extraction is performed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Directs to use this first and lists specific alternatives (convert_pdf, analyze_pdf, extract_structured) based on the need, providing clear decision guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_extractionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses behavioral traits such as returning per-page audit statuses (usable/silently-empty/recovered/review/unverifiable), a headline with count of silently dropped pages, and an overall verdict with tamper-evident signature. It also explains the ground truth method (reusing pdfmux's audit pass). No annotations exist, so the description carries full burden and meets it well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that introduces the core concept first ('Audit an extraction of a PDF...') and then adds detail. It is efficient with no wasted words, though slightly dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to detail return values but still summarizes them. It covers main use cases and parameter semantics. Complexity is moderate and all key aspects are addressed, with minor gaps like error scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds meaning: it explains that extracted_text can be from another engine or omitted, and names the engine as a parameter. It could be more specific about fmt and engine values, but overall it compensates for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool audits a PDF extraction for silently-dropped pages, specifying the verb 'audit', resource 'extraction of a PDF', and unique failure mode. It distinguishes from sibling tools like convert_pdf or extract_structured by focusing on verification rather than conversion or extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool: pass extracted_text from another engine for certification, or omit it for self-certification. It provides clear context but does not explicitly state when not to use or list alternatives beyond the two modes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.