Server Quality Checklist

Tool Scores

• create_templateA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a creation/mutation operation and mentions the output (template ID for use with generate_document). However, it doesn't disclose important behavioral traits like permission requirements, whether templates are editable after creation, rate limits, or what happens on duplicate slugs. The description adds value but leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence establishes the core purpose with technical specifics, and the second provides crucial downstream usage context. There's zero wasted language or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with 11 parameters, no annotations, and no output schema, the description is somewhat incomplete. While it clearly states the purpose and provides usage guidance, it lacks important contextual information about permissions, error conditions, template lifecycle, and what the actual return value contains beyond 'template ID'. For a creation tool in a template management system, more behavioral context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly with descriptions, enums, and defaults. The description adds minimal parameter semantics beyond the schema - it mentions HTML/Handlebars markup and JSON schema, which are already covered in the schema properties. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new reusable document template'), identifies the resource ('document template'), and distinguishes it from siblings by mentioning the output ('use the returned template ID with `generate_document`'). It provides concrete technical details (HTML/Handlebars markup with JSON schema) that differentiate it from other tools like list_templates or delete_template.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('Create a new reusable document template') and explicitly mentions a downstream usage ('use the returned template ID with `generate_document`'). However, it doesn't specify when NOT to use it or explicitly compare it to alternatives like preview_template for testing templates before creation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_documentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the core functionality (PDF generation, download URL return) and mentions supported features (custom paper sizes, orientation, filenames). However, it lacks details on error conditions, rate limits, authentication requirements, or whether the generated document is stored permanently.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and output, followed by essential usage details. Every sentence adds value without redundancy, and it efficiently covers key aspects (template options, data injection, supported features) in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, nested objects) and lack of annotations or output schema, the description does a good job of covering the main functionality and parameters. However, it could be more complete by addressing error handling, storage implications, or linking to sibling tools like preview_template for validation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by mentioning the mutual exclusivity of template_id and markup and the injection of data into templates, but does not provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Generate a PDF document'), resource ('from a template and data'), and output ('Returns a download URL for the generated PDF'). It distinguishes this tool from siblings like create_template (which creates templates) and get_document (which retrieves documents) by focusing on document generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool ('Generate a PDF document from a template and data') and offers guidance on parameter usage ('Provide either template_id or markup'). However, it does not explicitly state when NOT to use it or mention alternatives like preview_template for testing templates before generation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the return values (status, download URL, file size, page count, creation date), which adds valuable context beyond the input schema. However, it lacks details on error handling, authentication needs, or rate limits, leaving minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of two concise sentences that directly state the tool's purpose and return values. Every sentence earns its place by providing essential information without unnecessary elaboration or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is mostly complete. It explains what the tool does and what it returns, but it could improve by mentioning error cases or prerequisites. Without an output schema, the return value details are helpful, though not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'document_id' fully documented in the schema as 'The document ID returned from generate_document'. The description adds no additional parameter details beyond this, so it meets the baseline of 3 where the schema does the heavy lifting without compensating for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get details about' and the resource 'a previously generated document by its ID', making the purpose specific and unambiguous. It distinguishes this from sibling tools like generate_document (which creates documents) and list_templates (which lists templates), avoiding tautology by not just restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'previously generated document', suggesting this tool is for retrieving details after creation, but it does not explicitly state when to use this versus alternatives like get_usage or list_templates. No exclusions or clear alternatives are provided, leaving some ambiguity in context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usageA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns specific data (plan name, document/page counts, remaining quota), which is helpful behavioral context. However, it doesn't mention potential limitations like rate limits, authentication needs, or whether the data is real-time vs cached.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose ('Get current API usage statistics') and efficiently lists the return data. Every word adds value with zero waste, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is largely complete—it explains what the tool does and what it returns. A minor gap is the lack of output format details (e.g., JSON structure), but this is acceptable for a straightforward read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, earning a baseline 4 for not adding unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get current API usage statistics') and resource ('for this billing month'), distinguishing it from sibling tools like get_document or list_templates that handle documents/templates rather than usage metrics. It provides a complete picture of what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'current API usage statistics for this billing month,' suggesting it's for monitoring quota/limits. However, it doesn't explicitly state when to use it versus alternatives or provide exclusions, though no obvious alternatives exist among siblings for usage tracking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_templateA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully communicates critical behavioral traits: the irreversible nature of the action ('This action is irreversible') and ownership requirements ('Only templates you own can be deleted'). These are essential safety and permission details that the agent needs to know.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three short sentences that each earn their place: the core action, the irreversible warning, and the ownership constraint. It's front-loaded with the primary purpose and wastes no words while covering essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description does well by warning about irreversibility and ownership requirements. However, it doesn't mention what happens after deletion (e.g., whether it returns confirmation, error handling for non-existent templates, or cascading effects on related documents), leaving some behavioral gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'template_id' fully documented in the schema. The description doesn't add any additional parameter information beyond what the schema already provides (it only mentions 'by its ID' which the schema covers). According to the rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete') and resource ('a template by its ID'), distinguishing it from siblings like create_template, list_templates, and preview_template. It provides a precise verb+resource combination that leaves no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('Only templates you own can be deleted'), which helps the agent understand prerequisites. However, it doesn't explicitly mention when NOT to use it or name specific alternatives (like using update_template instead for modification), so it doesn't reach the highest score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_templatesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: it returns specific data fields (template names, slugs, categories, descriptions, paper sizes, and IDs) and mentions the optional filtering capability. However, it doesn't cover aspects like pagination, rate limits, authentication needs, or error handling, leaving some gaps for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, the second adds filtering details and return values, and the third provides usage context. Every sentence earns its place by adding essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is mostly complete. It covers purpose, filtering, return values, and usage with siblings. However, without annotations or an output schema, it could benefit from more behavioral details (e.g., response format, error cases), but it adequately serves the tool's simple listing function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'category' fully documented in the schema (including description, type, and enum values). The description adds value by explaining the optional nature ('Optionally filter by category') and the effect of omitting it ('Omit to list all'), which complements the schema's 'Omit to list all' note, but doesn't provide significant additional semantics beyond what's already in the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'available document templates', making the purpose specific and unambiguous. It distinguishes from siblings like 'create_template' (creation), 'delete_template' (deletion), 'generate_document' (generation), 'get_document' (retrieval), 'get_usage' (usage stats), and 'preview_template' (preview) by focusing solely on listing with optional filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('List available document templates') and provides clear alternatives by naming specific sibling tools: 'Use the returned ID with `generate_document` or `preview_template`.' It also implies when not to use it (e.g., for creation, deletion, or retrieval of specific documents), though it doesn't explicitly list all exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_templateA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (markup, styles, schema, sample data) and its purpose (inspection before generation). However, it doesn't mention potential limitations like whether previews are cached, if there are rate limits, or authentication requirements. For a read-only tool with no annotations, this is good but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three sentences that each serve a distinct purpose: stating what the tool returns, when to use it, and what the output components mean. There is no wasted language, and the information is front-loaded with the core functionality first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no annotations and no output schema, the description provides excellent context about what the tool does and when to use it. However, it doesn't fully describe the return format (e.g., structure of markup/styles/schema) or potential error conditions. Given the complexity is low, this is nearly complete but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single parameter. The description adds meaningful context by explaining that the schema 'shows which data fields the template expects' and that 'sample data (if available) shows example values,' which helps the agent understand what to expect from the output. This goes beyond the basic parameter documentation in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get', 'inspect') and resources ('full markup, styles, and data schema of a template'). It distinguishes from siblings like list_templates (which lists IDs) and generate_document (which creates documents). The description explicitly mentions what the tool returns: markup, styles, and schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Use this to inspect a template before generating a document.' It also references the sibling tool list_templates as the source for template IDs, creating a clear workflow connection. This gives the agent clear context for when this tool is appropriate versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.