Server Quality Checklist

Tool Scores

• get_document_metadataA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It conveys a read-only intent via 'Returns metadata' and 'verify before reading', but lacks details on error handling, file existence checks, or permissions. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the return value and followed by use cases. No wasted words, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter metadata tool with no output schema, the description fully covers what is returned (listing all metadata fields) and when to use it. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the schema's 'path' parameter; it only repeats that it applies to .docx files, which is already in the schema. No extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'Returns' and resource 'metadata about a .docx file', listing concrete fields (file name, size, dates, counts). This distinguishes it from siblings like read_document (content) or search_document (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear use cases: 'for document statistics or to verify a document before reading'. This implies when to use over reading, but does not explicitly name alternatives or exclusions, so slightly below a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format (full paths) and the specific file type (.docx), which goes beyond the tool name. It implies a read-only listing operation, though it does not explicitly mention limitation such as recursive vs. non-recursive behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and includes a purpose statement plus a return-value detail. Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description is complete: it defines what is listed, why to use it, and what it returns (full paths). There is no ambiguity that would prevent an agent from invoking it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'directory' parameter, so the baseline is 3. The description adds minimal new meaning beyond the schema, simply restating that the operation occurs within a directory/folder without providing extra syntax or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (Lists), the resource (.docx files), and the scope (a directory). It is distinct from sibling tools like read_document, search_document, get_document_metadata, and resolve_document, focusing on discovery rather than content operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use this to discover what Word documents are available in a specific folder.' It does not explicitly mention exclusions or name alternative tools, but the context is unambiguous and appropriate for selecting this tool over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals key behaviors: the search is case-insensitive, and it returns matching lines with line numbers and context, which is more informative than a generic search description. It does not discuss error handling or edge cases, but for a read-only search tool, the essential behaviors are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of three concise sentences, each serving a clear purpose: statement of function, behavioral detail (case-insensitivity), and usage context. There is no redundancy or extraneous information, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool without an output schema, the description is complete. It covers the tool's purpose, return format ('matching lines with line numbers and context'), the case-insensitive behavior, and when to use it. The absence of error details is acceptable for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides complete descriptions for both 'path' and 'query', with 100% coverage. The description adds 'case-insensitive' but this is also noted in the schema's query property. Therefore, the description does not add significant meaning beyond what the schema already provides, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Searches for text within a .docx file and returns matching lines with line numbers and context.' This specifies the operation, resource, and output format, and it distinguishes the tool from siblings like read_document (which likely returns the full document) and get_document_metadata (which provides metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use for finding specific content within a document.' This clearly indicates when the tool is appropriate. However, it does not explicitly mention alternatives or contrast with sibling tools, so it falls short of the highest tier of guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_documentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool reads a file and returns the full text content, and notes the path requirement. It doesn't detail error cases or binary handling, but for a simple read operation, the key behaviors are transparently described without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. It front-loads the core action ('Reads a .docx file'), then provides usage and prerequisite context. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description is complete: it states the function, the output (full text), the path requirement, and a reference to resolve_document for common scenarios. It is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for the single 'path' parameter, describing it as 'Absolute or relative path to a .docx file'. The description adds a pointer to resolve_document for name-only inputs, but this is more usage guidance than parameter semantics. Baseline 3 is appropriate as the description adds marginal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Reads a .docx file and returns the full text content', which is a specific verb and resource, clearly distinguishing it from sibling tools like search_document, get_document_metadata, list_documents, and resolve_document. It also states common use cases ('read, summarize, or analyze a Word document'), reinforcing its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to read, summarize, or analyze a Word document', providing clear usage guidance. It also gives a practical exclusion: 'use resolve_document first if you only have a document name', naming a specific alternative tool for a prerequisite scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_documentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even without annotations, the description discloses key behavioral outcomes: resolved absolute path, candidate list for ambiguous names, and helpful error for not found. This explains the tool's return behavior and implies a non-destructive read operation. Missing details like case sensitivity or permissions are minor for a resolver.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, followed by usage context and output behavior. Every sentence earns its place, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and lack of output schema, the description adequately covers return values (path, candidates, error), usage triggers, and downstream integration with sibling tools. It is complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both 'name' and 'location'. The description adds contextual examples ('the report on my desktop') and confirms the optional nature of location, but does not substantially extend beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Finds') and resource ('.docx files'), clearly stating the tool resolves friendly names and optional locations to absolute paths. It distinguishes itself from siblings like read_document and search_document by focusing on name-to-path resolution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'when a user refers to a document by name ... rather than by full path.' It also provides a clear follow-up instruction to use read_document or search_document with the resolved path, guiding the agent on alternatives and next steps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.