Server Quality Checklist

Tool Scores

• list_supported_document_typesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It weakly implies a read-only operation (listing) but does not explicitly state side effects, authentication requirements, rate limits, or response behavior. The existence of an output schema covers return format, but not behavioral traits. A simple 'List' implies read-only, but for a complete definition more context is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that leads with the action and then clarifies with concrete examples. There is no filler, and every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, an output schema exists, and the purpose is simple, the description covers the essentials. It explains what the tool does and gives representative examples. It could add a note about usage direction (e.g., that these types can be used with extract_fields or validate_against_schema), but such cross-referencing is not required for basic invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is an empty object. According to the rubric, a parameterless tool gets a baseline 4. The description adds no parameter-specific information, but none is needed; the schema is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and a clear resource 'document types with pre-built schemas', including helpful examples (invoice, resume, purchase_order). It clearly conveys the tool's function, though it does not explicitly distinguish it from siblings like extract_fields or validate_against_schema, so it falls just short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools. The description does not mention alternatives, typical usage scenarios, or any conditions that would make this tool the right choice. This is a clear gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_fieldsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It does disclose failure modes (corrupt PDF, bad schema), the exact return contract ({ok: True/False}), and the non-empty properties requirement. However, it omits the side-effect profile (e.g., read-only disk access) and edge behaviors such as empty extraction results, so disclosure is solid but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Purpose is front-loaded in a single line, followed by a clean Args/Returns structure that is scannable. Slightly padded by trailing ellipses and the '...' after the failure list, but overall efficient with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description covers the return format, failure modes, and semantic details of both parameters—enough to call the tool correctly. Gaps are minor: no sibling-routing guidance and the meaning of the returned 'text_length' field is undefined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must supply meaning. It adds substantial context: schema may be a JSON Schema object, a built-in name like 'invoice', or a .json file path, and must have a non-empty 'properties'; pdf_path is explained as a path to the PDF. This goes well beyond the bare type declarations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence states a specific verb (extract), a resource (structured fields), and a source/method (unstructured PDF using regex/heuristics). This clearly distinguishes extraction behavior from list_supported_document_types, though it doesn't explicitly name the differentiation from validate_against_schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to choose this tool versus its siblings. It doesn't mention when validate_against_schema or list_supported_document_types would be the better pick, and states no preconditions or exclusions beyond the inline schema-constraint note in Args. The agent must infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_against_schemaA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behavior: never raises exceptions, invalid schemas are reported via an 'error' key, and the return is a structured report with passed/failed/missing lists and human-readable reasons. This goes beyond a generic 'validates' and gives the agent a clear expectation of failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently formatted with an Args and Returns section, front-loaded with the purpose sentence. It is not verbose—every line contributes to the agent's understanding. It could be tightened (the Returns section is slightly redundant with the behavioral note), but it's well-structured and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two required parameters, no output schema, and no annotations, the description covers the essential context: what the tool does, what each parameter is, how errors are handled, and what the response contains. It references the sibling workflow (extract_fields) which aids routing. The only minor gap is that it doesn't specify the expected shape of 'data' beyond being an object, but the example mitigates this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain the parameters. It does: 'data' is explicitly the output of extract_fields, and 'schema' is a JSON Schema. This adds meaning beyond the bare 'object' type in the schema and gives the agent the necessary context for both arguments. It could add more detail about expected structures, but it is sufficient for a two-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource: 'Validate extracted data against a JSON Schema.' It names the specific tool and its input, and the example referencing extract_fields ties it to the workflow. It is clearly distinct from the siblings (list_supported_document_types and extract_fields) because validation is an entirely separate operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use it: after extraction, with the 'data' from extract_fields. It implies the pipeline (extract → validate) without explicitly saying 'use this when...' but the example makes the usage obvious. It does not mention when not to use it or name alternatives, so it lacks an explicit exclusion but is not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.