Server Quality Checklist

Tool Scores

• compile_mjmlC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Compile') but doesn't describe what happens during compilation (e.g., error handling, performance implications, or output format). It mentions 'various options' but doesn't explain their behavioral impact beyond what the schema covers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. However, it could be more structured by explicitly listing key options or use cases, but it avoids redundancy and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, nested objects) and lack of annotations or output schema, the description is insufficient. It doesn't explain the compilation process, error handling, or what the output looks like (e.g., HTML string or file). For a tool with rich parameters and no structured output info, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 8 parameters. The description adds no additional meaning beyond implying 'various options' exist, which doesn't enhance understanding of individual parameters. The baseline score of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Compile') and resource ('MJML to HTML'), specifying the core transformation. It distinguishes from siblings like 'validate_mjml' (validation) and 'generate_template' (template creation) by focusing on compilation. However, it doesn't explicitly differentiate from 'get_component_info' in terms of resource scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'validate_mjml' for validation or 'generate_template' for template generation. It mentions 'various options' but doesn't specify contexts or prerequisites for choosing this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'generate' but doesn't clarify if this creates new files, requires specific permissions, has rate limits, or what the output looks like (e.g., HTML code, file saved). This is inadequate for a tool with multiple parameters and potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for the tool's complexity, making it easy for an agent to parse quickly without unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like what happens when 'outputPath' is omitted or how 'customColors' and 'customFonts' are applied, leaving significant gaps for the agent to handle.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters well. The description adds no additional meaning beyond the schema, such as explaining how 'variables' map to template placeholders or what 'outputPath' defaults to. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'generate' and the resource 'pre-built email templates', making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'compile_mjml' or 'validate_mjml', which might also involve template processing, so it misses the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'compile_mjml' or 'validate_mjml'. It lacks context about prerequisites, such as whether it's for creating new templates from scratch or modifying existing ones, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_component_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't describe behavioral traits like whether it's read-only (implied by 'Get'), what happens if parameters are omitted, error conditions, or response format. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'Get MJML component reference and documentation'. It's front-loaded with the core purpose, has zero waste, and is appropriately sized for a simple tool. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain what the tool returns (e.g., documentation text, examples, or structured data), how to interpret results, or any limitations. Without annotations or output schema, the description should provide more context about behavior and outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't add any meaning beyond what the input schema provides. The schema has 100% description coverage, with clear documentation for both parameters ('component' and 'category'), including enum values for 'category'. With high schema coverage, the baseline is 3, as the schema does the heavy lifting and the description doesn't compensate with additional context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get MJML component reference and documentation' - a specific verb ('Get') and resource ('MJML component reference and documentation'). It distinguishes from siblings like 'compile_mjml', 'generate_template', and 'validate_mjml' which focus on different operations. However, it doesn't explicitly differentiate itself from potential similar documentation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to use it (e.g., for learning about MJML components) versus when to use sibling tools like 'compile_mjml' for processing MJML code. There's no context about prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_mjmlB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool validates MJML but doesn't describe what happens during validation (e.g., returns errors/warnings, checks against a schema), whether it's read-only or has side effects, or any performance or permission considerations. For a validation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just four words ('Validate MJML syntax and structure'), with zero wasted language. It's front-loaded with the core purpose and appropriately sized for a simple validation tool, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (validation with three parameters) and no output schema, the description is minimally adequate but incomplete. It states what the tool does but lacks details on behavior, usage context, or return values. With no annotations and no output schema, the agent must rely heavily on the schema and tool name, leaving gaps in understanding the full context of validation results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter semantics beyond what the input schema provides. Since schema description coverage is 100%, the baseline score is 3. The schema fully documents all three parameters (input, filePath, validationLevel) with descriptions and defaults, so the description doesn't need to compensate, but it also doesn't add any extra context about parameter usage or interactions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Validate MJML syntax and structure', which is a specific verb ('validate') applied to a specific resource ('MJML syntax and structure'). It distinguishes from sibling tools like 'compile_mjml' (which transforms MJML) and 'generate_template' (which creates templates), though it doesn't explicitly mention these distinctions in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when validation is needed (e.g., before compilation), what scenarios it's best for, or how it differs from sibling tools like 'compile_mjml' (which might include validation implicitly). The agent must infer usage from the tool name and context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.