Server Quality Checklist

Tool Scores

• word_analyzeC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It mentions returning a text index but does not explain the output structure or behavior beyond that. It also references a 'fmt_hints' parameter that is not part of the input schema, which is a behavioral inconsistency and leaves the agent uncertain about expected inputs and outputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is not concise; it spends significant space detailing an addressing scheme and format hints that pertain to word_replace, not word_analyze. This irrelevant content detracts from the core message and makes the description longer than necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description fails to explain how the tool is used or what the returned text index represents. It also introduces a non-existent parameter, making the description incomplete and inconsistent with the structured data. For a simple one-parameter tool, the description should be clearer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single 'path' parameter with zero description coverage, and the description does not clarify what 'path' means. Worse, the description introduces a 'fmt_hints' parameter that does not exist in the schema, actively misleading the agent about the available parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'Analyze a Word document structure' and that it 'Returns text index for word_replace', which is a specific verb+resource and distinguishes it from siblings like word_replace (which modifies) and word_read_document (which reads content). However, the subsequent addressing scheme and format hints sections are tangential to word_analyze and may confuse the reader.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool is a precursor to word_replace by stating it returns a text index for that tool, giving some context. However, it does not explicitly say when to use this versus alternatives like word_read_document, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• word_replaceB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions preserving formatting, which is a useful trait, but omits critical side effects such as whether the original file is modified, whether output_path is overwritten, or any error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. Every component (action, resource, key behavior) earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a complex replacement format with keys like p:0 and c:0:3:0, and optional fmt_hints with run-level formatting, yet the description provides no explanation of these or how formatting preservation actually works. It relies entirely on schema examples, leaving significant gaps for an agent to correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All four parameters have schema descriptions with examples, including the complex replacement map and fmt_hints. The tool description itself adds no additional parameter semantics, so a baseline score of 3 is appropriate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Replace') and resource ('text in a Word document'), plus a distinctive behavior ('preserving all formatting') that separates it from siblings like word_analyze and word_read_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as word_analyze, word_read_document, or word_clone_template. It simply states the action without any context, exclusions, or prerequisite conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• word_read_documentA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It discloses the scope of reading (paragraphs and tables), indicating it does not return other elements like headers or images. However, it does not explicitly state that the operation is non-destructive, safe, or what happens on missing files, though the verb 'read' implies no modification.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence that front-loads the core purpose and scope. No wasted words; it contains exactly the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description provides adequate context: it identifies the target document and the specific elements read. It doesn't need to detail return values since an output schema exists. Minor omissions like error handling or explicit read-only confirmation slightly reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single parameter document_path with 0% schema description coverage. The description does not explicitly define the parameter, though the tool name and description make it evident that document_path is the file path to the Word document. This minimal inference does not fully compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (read), the resource (Word document), and the specific scope (paragraphs + tables). It distinguishes itself from sibling tools like word_analyze and word_replace by focusing purely on reading content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for retrieving the raw content (paragraphs and tables) from a Word document. However, it doesn't explicitly contrast with sibling tools like word_analyze (which might analyze content) or provide when-to-use vs alternatives, leaving usage largely inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• word_clone_templateA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool clones a template, preserves formatting, and is restricted to table-templates. However, it does not mention overwrite behavior for output_path or any other side effects, so it is not fully comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states core function, second provides legacy status and alternatives. Every sentence earns its place, and the information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has seven parameters and no annotation, but the description provides no parameter semantics, and only a high-level purpose. Although it mentions legacy status and alternatives, the lack of parameter guidance means the agent cannot correctly construct a call, making it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description completely ignores all seven parameters, including optional fields like date, title, main_topic. It does not explain how parameters map to replacement fields, leaving the agent without necessary invocation details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'Clone a table-template Word doc and replace fields' with a specific verb+resource, and notes it preserves formatting. This clearly distinguishes from siblings like word_replace, which modifies an existing doc rather than cloning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly labels itself as 'Legacy tool' and instructs 'for new templates, prefer word_analyze + word_replace'. This directly tells the agent when to avoid this tool and which alternatives to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.