Server Quality Checklist

Tool Scores

• describe_templateA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains the return format and input constraints, but does not explicitly state whether the operation is read-only or has side effects. However, the description is sufficiently transparent about what the tool does.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear first sentence stating purpose, followed by usage guidelines, input instructions, and a list of return values. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description provides complete context including inputs, outputs (matching the output schema), and when to use it. It adequately differentiates from sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains that either template_path or template_b64+template_filename can be used, adding meaning beyond the schema. However, it does not define each parameter's semantics, so the compensation is partial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as a one-call summary of a template's overall shape, listing specific outputs like page count and target counts. It also differentiates from sibling tools by explicitly mentioning when to use list_template_targets instead.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'USE WHEN' and 'DO NOT USE WHEN' sections, providing clear guidance on when to use the tool and when to use the alternative list_template_targets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_templateA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly identifies the tool as read-only ('Read'), explains pagination (start, limit, next_start), and describes two methods for providing a template (path or base64+filename). Minor omission: no mention of what happens if both are provided or error handling, but still adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a purpose sentence, then usage guidelines, then parameter explanation. No unnecessary words, and important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (so return format is covered), the description provides sufficient context: purpose, usage, parameter semantics. It doesn't discuss error conditions or edge cases, but for a read tool with moderate complexity, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the purpose of template_path vs. template_b64+template_filename as alternative inputs, and describes start and limit for pagination. It could be more explicit about default values, but overall adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a template's paragraphs in a paginated way to understand structure before editing. It uses a specific verb ('Read') and resource ('template's paragraphs'), and provides context for its use. It also distinguishes itself from the sibling tool 'list_template_targets'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly includes a 'USE WHEN' and 'DO NOT USE WHEN' section, providing clear guidance on when to use this tool and when to use an alternative (list_template_targets). This is exemplary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_template_targetsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description must carry behavioral info fully. It describes pagination, filtering, deprecated parameters, and returned fields. It is non-destructive but doesn't explicitly state read-only nature; still, transparency is high.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections, front-loaded purpose, and concise sentences. Slightly verbose in listing returned fields but each line adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description covers all necessary context: usage scenario, parameter details, pagination behavior, and returned fields. Complete and actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds rich semantics: explains alternative ways to provide template (path vs b64+filename), allowable target kinds, pagination start/limit with note on next_start, and deprecation alias for limit. Greatly aids correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists editable targets in a template for building an edit list, distinguishing it from siblings like inspect_template (human-readable overview) and fill_and_save (later step).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'USE WHEN' and 'DO NOT USE WHEN' conditions are given, directing to use as first call before fill_and_save and to avoid if a human-readable overview suffices, recommending inspect_template instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fill_and_saveA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: input/output modes, edit type requirements, style targeting rules, built-in robustness (conflict skipping, duplicate checks, type coercion), output format limitations, and an exhaustive list of possible status values and return fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized with sections (USE WHEN, INPUT modes, OUTPUT modes, edit dict structure, STYLE EDIT TARGETING, BUILT-IN ROBUSTNESS). Every sentence adds value, though it could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, edit types, multiple modes) and lack of annotations, the description covers all necessary aspects: preconditions, input/output options, error handling, and return format. The presence of an output schema (not shown but flagged) makes the return description somewhat redundant but still complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (only titles), but the description adds comprehensive meaning for all 7 parameters, including the complex edits structure with required fields per edit type, and alternative input/output modes. It fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Apply a batch of edits to the template and write/return the result,' which is a specific verb+resource. It distinguishes itself from sibling tools like list_template_targets, describe_template, and inspect_template by focusing on writing/returning the modified template.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'USE WHEN' and 'DO NOT USE WHEN' sections, advising to call list_template_targets first and warning that fill_and_save will fail without proper target_ids. It also details when to use different input/output modes, making usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.