Server Quality Checklist

Tool Scores

• read_documentC
  2.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It only lists file types and mentions 'portion', but fails to disclose behavior like error handling, page start, or output format. Missing essential behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very short, but one sentence is an instruction for user presentation rather than tool functionality. Could be more concise by removing extraneous guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite high parameter count (8) and multiple file formats, the description provides minimal context. Output schema exists but is not used in description. Missing return value details and usage scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no explanation for any of the 8 parameters. It does not help the agent understand defaults, options, or relationships between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Read a portion' and specifies multiple document types (PDF, Excel, etc.). However, it does not differentiate from sibling tools like document_info or visual_evaluate_document, missing a chance to clarify distinct purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The IMPORTANT note instructs on presenting results but does not provide usage context, such as prerequisites or when to avoid the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist. The description does not disclose recursion behavior, filtering logic, or performance characteristics. The phrase 'configured root directory' is ambiguous without context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with no unnecessary words, but it is under-specified. Conciseness is positive, but the content is too minimal to be fully useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description lacks details on parameter usage, default behaviors, and output structure. For a tool with 3 parameters and no schema descriptions, the description is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description fails to mention any of the three parameters (root, pattern, limit), leaving the agent to infer from schema alone. This does not add value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and resource (documents under root directory). However, it does not differentiate from sibling tools like document_info or read_document, which could be inferred but not explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use vs alternatives like document_info (single doc) or search. No exclusions or prerequisites provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• document_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must explicitly disclose behavioral traits. It only says 'Return metadata' without mentioning permissions, rate limits, or what is included in 'metadata'. For a tool with no safety annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no wasted words. However, it omits necessary details, making it too brief to be fully effective. Conciseness is good, but completeness suffers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 param, no nested objects) and has an output schema, so the description may not need to detail return values. However, it fails to clarify what 'metadata' constitutes, leaving some ambiguity. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It does not add any context about the 'file_path' parameter (e.g., format, allowed protocols). The parameter name is self-explanatory, but without additional guidance, the score is low.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return metadata') and the resource ('a specific document'). It effectively distinguishes from siblings: list_documents (listing), read_document (content), and visual_evaluate_document (visual evaluation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: use when needing metadata for a known document. However, no explicit when-to-use or when-not-to-use guidance is provided, nor are alternatives compared despite sibling tools being named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• visual_evaluate_documentA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description reveals key behaviors: it returns content blocks (text/images) and can run local OCR. It implies a read-only operation by using 'extract', but does not clarify if the document is modified or detail potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences that front-load the core purpose and key feature (inline images). No unnecessary details or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and no output schema, the description provides a general overview but lacks detail on parameter defaults, supported file formats, and error handling. Sibling differentiation is not explicitly addressed, leaving some gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It hints at file_path (document), enable_ocr (optional OCR), page (page number), but does not describe max_pages or ocr_lang explicitly. This partial coverage leaves some parameters ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts visual content (charts, tables, diagrams) from PDFs/images and returns inline images for AI analysis. This distinguishes it from siblings like read_document (text extraction) and document_info (metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions optional OCR usage but does not explicitly state when to use this tool over alternatives or any constraints (e.g., file size limits). Usage context is implied but not clearly delineated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.