Server Quality Checklist

Tool Scores

• document_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states what metadata is returned (file type, size, modification time) and notes variations (PDF metadata, line counts for text files), which is helpful. However, it does not mention error conditions (e.g., missing file, insufficient permissions), time-out behavior, or any side effects, leaving some behavioral uncertainty for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the core purpose, the second lists the main outputs, and the Args section cleanly describes the parameter. Every sentence contributes value with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one param) and an output schema exists, so return values are covered externally. The description logically describes the metadata returned and file-type-specific additions. It lacks guidance on edge cases like unsupported file types or empty results, but completeness is adequate for a metadata retrieval tool. A minor gap is not explicitly stating what happens for non-PDF binary files, but the description's inclusive language ('Includes file type...') gives sufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must add meaning to the single 'path' parameter. It does add the constraint 'File path inside the allowed root', which is valuable. However, it does not specify path format (absolute vs relative), whether directories are valid, or any path syntax details. This is a partial compensation but not comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'metadata for a document', distinguishing it from sibling tools like read_document (which reads full content) and list_documents (which lists documents). The explicit 'without reading its full content' clarifies the scope and separates it from content-reading tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool—when you need metadata rather than content—by explicitly stating it returns metadata and not content. It does not name specific alternatives or exclusions, but the context of the sibling tools and the phrasing 'without reading its full content' gives clear contextual guidance without explicitly saying 'use read_document instead'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_structuredA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It does well by disclosing that parent directories are created, CSV data must be flat objects, JSON is pretty-printed, and it specifies the return shape. However, it does not mention overwrite behavior or what happens when a path falls outside the allowed root, which are relevant for a file-writing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-line purpose statement followed by Args and Returns blocks. Every sentence carries useful information, and there is no filler or repetition of schema fields without added context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and only a sparse schema, the description covers the essential usage: all parameters are explained, constraints are provided, and the return value is documented, including the rows edge case. The only notable gap is overwrite behavior and error handling for invalid or out-of-root paths, which keeps it from being fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, so the description must compensate. It fully does: it explains that path is relative to the allowed root, that data is a JSON string with format-specific shape requirements, and that format controls pretty-printing. This adds meaning far beyond the schema's bare names, types, and enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear action: 'Write extracted data to a file inside the allowed root.' It names the resource (file), the action (write), and a key constraint. The sibling tools are all read/list/validation operations, so this write tool is clearly differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the verb 'Write extracted data' and by the surrounding sibling tools, but the description never explicitly states when to prefer this tool over alternatives or when not to use it. There is no exclusions or alternative routing, so it relies on the agent inferring that this is the write/save operation among read-only siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the read-only nature (listing), the security constraint (never leaving root), and the return format. It doesn't mention permissions or rate limits, but for a listing operation the disclosed traits are adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a one-sentence summary, parameter explanations, and return details. It's slightly verbose but every sentence provides value, and the critical scoping constraint is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the operation, parameters, and return contract clearly. It lacks only minor details like sorting order or error behavior, which are not critical for a listing tool. The presence of an output schema (indicated by context) further reduces the need to document return fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates. It explains directory (relative or absolute, with '.' for root) and glob_pattern (format, defaults, recursive matching, and security constraints), giving complete meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (list files) and resource (directory), with clear scoping to the allowed root. It distinguishes from siblings like read_document (reading content) and document_info (metadata) by focusing on listing directory entries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies usage for listing directory contents and even notes that returned paths are accepted by other tools, guiding workflow. It doesn't explicitly contrast with alternatives or state when not to use, but the context is sufficient for most cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chunk_documentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full behavioral burden. It discloses PDF-specific page markers and page_hint behavior, the overlap constraint (must be smaller than max_chars), and the return structure. It could note that it is read-only, but that is implied. Overall it is transparent about key behaviors without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose sentence, a usage note, a behavior note, an Args section, and a Returns statement. It is lean with no redundant wording and front-loads the core purpose, making it easy for an agent to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, parameters, behavior, and return format. It even describes the output structure (list of {index, start_char, page_hint, text}) despite an output schema being available. There are no obvious gaps for an agent to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain the parameters. It does: 'path' is the file path inside the allowed root, 'max_chars' is the maximum chunk size, and 'overlap' repeats characters between chunks, with the constraint that overlap must be smaller than max_chars. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (split) and resource (document's text), producing ordered, overlapping chunks. It also notes the tool is for long documents that don't fit one context read, implicitly distinguishing it from read_document and other siblings. This is more than adequate for purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Useful for long documents that do not fit one context read,' giving a clear when-to-use scenario. However, it does not name alternative tools or state when not to use it, so it misses the full 5-level guidance of explicit exclusions and named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_documentA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden, and it excels: PDFs are extracted per page with '--- page N ---' markers, CSV is rendered as an aligned table, and unsupported types error. It also explains the pages argument's 1-indexed syntax and examples, offering far more context than the bare schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a one-sentence summary, a compact list of supported formats, and an Args block that maps directly to the schema. Every sentence adds value, with format details and page syntax front-loaded before parameter specifics.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter read tool with no annotations, the description covers all operational essentials: accepted types, per-format rendering rules, page selection syntax, and error behavior. An output schema exists, so return-value documentation is unnecessary, and nothing an agent needs for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, making the description the sole source of parameter meaning. It defines 'path' as a file path inside the allowed root and explains 'pages' with concrete examples like '3', '1-5', and '1-3,7', plus the default of empty string meaning all pages.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair ('Read a document and return its text content'), immediately clarifying the tool's function. It further specifies supported formats and states that other types return an error, which distinguishes it from sibling tools like list_documents or document_info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by enumerating supported file types and noting that unsupported types return a clear error, giving an implicit when-not-to-use. However, it does not explicitly name alternative tools for metadata or chunking tasks, such as document_info or chunk_document, so the guidance relies on inference rather than direct exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_jsonA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses that the tool returns every validation error with JSON Pointer paths, not just the first, and details the exact return structure including the meaning of 'pointer'. It also specifies the schema draft version, providing comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a clear purpose line, a behavioral note, then parameter and return documentation. Every sentence adds value, and the core action is front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters and an output schema, the description is complete. It covers the input formats, the validation behavior, and the output structure, including the meaning of 'pointer'. No critical information is missing, so an agent can call it correctly without further lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description's explanation of 'data' as the JSON document and 'json_schema' as the schema string provides essential semantic meaning beyond the bare parameter names. It fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('validate') and resource ('a JSON string against a JSON Schema'), and specifies the schema draft (2020-12). This clearly distinguishes it from the document management siblings, as it describes a validation operation rather than document handling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case by noting it reports every error 'so extraction mistakes can be fixed in one pass', which guides when to use it. It does not explicitly name alternatives, but the sibling tools are all document-oriented, so the context is sufficient. A slight gap is the absence of explicit 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.