Server Quality Checklist

Tool Scores

• create_styleD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but provides none. 'Define' suggests a creation/mutation operation, but there's no information about permissions needed, whether this is idempotent, what happens on failure, rate limits, or what the tool actually does beyond the vague 'define' action. No behavioral traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three words. While this represents severe under-specification, from a pure conciseness perspective, there's zero wasted language. Every word earns its place, and the description is front-loaded with the core concept.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 complex parameters (including nested objects), 0% schema description coverage, no annotations, no output schema, and no sibling tool differentiation, the description is completely inadequate. It provides minimal context for what is clearly a sophisticated styling/visualization tool that requires understanding of canvas dimensions and rule structures.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 2 complex parameters (canvas and rules), the description provides no parameter information whatsoever. It doesn't explain what 'canvas' represents, what 'rules' contain, or how these relate to 'visual representation rules.' The description fails to compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Define visual representation rules' is vague and tautological - it essentially restates the tool name 'create_style' in different words. While it suggests something about visual rules, it doesn't specify what kind of style is being created, for what purpose, or what resource it operates on. It doesn't distinguish this from sibling tools like create_domain or create_substance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides absolutely no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, appropriate contexts, or when to choose this over sibling tools like create_domain, create_substance, or generate_diagram. The agent receives no usage direction whatsoever.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_diagramC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Generate diagram' but doesn't explain what this entails—e.g., whether it's a read-only operation, if it modifies data, requires authentication, has rate limits, or what the output looks like. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's appropriately sized and front-loaded, clearly stating the tool's basic function without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description is incomplete. It lacks details on behavior, parameter meanings, output format, and usage context, making it inadequate for an agent to understand how to effectively invoke the tool beyond a superficial level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description mentions 'domain/substance/style' as inputs, which maps to three of the four parameters (domain, substance, style), but it omits 'variation' and doesn't explain what these parameters mean, their formats, or how they influence diagram generation, failing to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Generate diagram from domain/substance/style' states the basic action (generate) and resources (diagram from three inputs), but it's vague about what kind of diagram or how it's generated. It doesn't distinguish from siblings like 'create_domain', which suggests different operations, but the purpose lacks specificity beyond the basic verb+resource combination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or relationships to sibling tools like 'create_domain', 'create_style', or 'create_substance', leaving the agent with no context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_substanceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool defines objects and relationships, implying a write operation, but doesn't disclose critical traits like whether it's idempotent, requires specific permissions, handles errors, or what happens on success/failure. For a tool with 3 required parameters and no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single, front-loaded sentence: 'Define mathematical objects and relationships.' It wastes no words and directly states the core purpose, making it easy for an agent to parse quickly. Every part of the sentence earns its place by conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 required parameters, no annotations, no output schema, and low schema coverage), the description is incomplete. It doesn't explain what the tool returns, how to interpret results, or provide enough context for safe and effective use. For a definition tool with significant parameter details, more information is needed to guide the agent adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low at 33%, with only the 'domain' parameter having a description ('Reference to domain'). The description 'Define mathematical objects and relationships' adds minimal semantic context, hinting that 'declarations' might define objects and 'statements' might define relationships, but it doesn't explain parameter formats, constraints, or examples. This insufficiently compensates for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Define mathematical objects and relationships' clearly states the tool's purpose with a specific verb ('define') and resource ('mathematical objects and relationships'). It distinguishes from siblings like 'create_domain' or 'generate_diagram' by focusing on mathematical definitions rather than domains, styles, or diagrams. However, it doesn't explicitly differentiate from 'create_style' which might also involve definitions, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, such as whether it's for initial setup or ongoing updates, or how it relates to sibling tools like 'create_domain' or 'create_style'. This lack of usage context leaves the agent to infer appropriate scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_domainC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Create' implies a write/mutation operation, it doesn't specify permissions needed, whether the operation is idempotent, what happens on conflicts, or any rate limits. For a creation tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a tool with 2 parameters and gets straight to the point with zero wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations, no output schema, and incomplete parameter documentation (50% schema coverage), the description is inadequate. It doesn't explain what the tool returns, what happens after creation, or provide enough context about the DSL system to guide proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'DSL definitions' which hints at the purpose of the parameters, but doesn't explain what 'name' and 'types' represent in this context. With 50% schema description coverage (only 'name' has a description), the description adds minimal value beyond what the schema provides, meeting the baseline for moderate schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and the resource ('domain-specific language (DSL) definitions'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like create_style or create_substance, which likely create different types of definitions in the same system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like create_style or create_substance. There's no mention of prerequisites, appropriate contexts, or exclusions, leaving the agent with no usage direction beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.