Server Quality Checklist

Tool Scores

• list_docsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and non-destructive behaviors; the description adds context about reading from INCLUSIO_CONTENT_DIR and returning specific fields, enhancing transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary purpose, followed by usage context, technical details, and return format, all in a concise, well-organized structure without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, rich annotations, and the existence of an output schema, the description covers the tool's full behavior including data source, return format, and edge case (empty list). Sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline 4 applies. The description does not need to elaborate on parameters, and schema coverage is 100% (vacuous).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists documents from data/meta.yaml, and distinguishes itself from siblings like render, audit_pdf, and doc_count by specifying its role as a discovery step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool first to get doc_id and class before calling render or audit_pdf, and suggests doc_count for just the count. Provides clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doc_countA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, destructiveHint, openWorldHint. Description adds that it's a cheap probe and confirms server sees valid manifest without transferring full list. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise; uses two clear blocks. Main action front-loaded. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, good annotations, and presence of an output schema, the description sufficiently covers all necessary contextual information for a simple count operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters defined; schema covers all. Baseline for 0 parameters is 4. Description does not need to add parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Count the documents registered in data/meta.yaml' with a specific verb and resource. It also distinguishes from sibling tool list_docs by noting that list_docs provides individual document details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: as a cheap connectivity/health probe. Also tells when not to use: if you need each document's id/class/flags, call list_docs instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_pdfA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly and idempotent. Description adds important context: shells out to verapdf read-only, never modifies PDF/manifest, requires 'verapdf' on PATH, and includes 'verapdf_present' field in response. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear first sentence, usage paragraph, parameter descriptions, and return info. No redundant or extraneous content; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, description sufficiently covers return type. Also documents external dependency and failure flag, making it complete for a tool that shells out.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters fully. Description adds value for 'strict' parameter by explaining caller decides error handling, which enhances semantic clarity beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'audit' and specific resource 'built PDFs for accessibility conformance with veraPDF' with explicit standards. Distinguishes from siblings by referencing 'render' and 'list_docs' for related tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (before publishing) and what it does not do (does not build or modify). References alternative tools ('render' for production, 'list_docs' for listing) for proper workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• renderA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses write behavior to build/.cache/rendered/, non-mutation of data/meta.yaml, and idempotent nature. Consistent with annotations (readOnlyHint=false, destructiveHint=false, idempotentHint=true). Adds context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  All sentences serve a purpose: purpose, usage, args, returns. Front-loaded with action and context. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully covered: purpose, usage, parameters, output schema, sibling relationships. Output schema is described in text. Annotations present. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Each parameter is described in the description with additional context (e.g., doc_id discovery via list_docs). Schema coverage is 100%, but description adds value by linking parameters to tool usage workflow.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it renders a template-driven document to disk. Distinguishes itself as the only write tool among siblings, using specific verb 'render' and explicit resource 'registered template-driven document'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (materialise before compile/audit), what it does not do (never mutates data/meta.yaml), and suggests discovery with list_docs and follow-up with audit_pdf. Also contrasts with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.