Server Quality Checklist

Tool Scores

• explain_dialC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It implies a read-only action ('explain') but does not explicitly state that it is non-destructive, idempotent, or requires no special permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence, 10 words) and front-loaded with the verb. However, it could be more informative without increasing length, e.g., by stating the return type.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and minimal description, the tool definition lacks completeness. It does not explain what the tool returns (e.g., text, object), how errors are handled, or that 'dial' likely refers to a key from 'get_dials' output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the single required parameter 'dial'. The description adds that the parameter identifies the dial to explain, but does not specify allowed values, format, or provide examples. The description partially compensates but lacks sufficient detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'explain what a dial means and how each value band changes behavior', which is a distinct verb+resource compared to siblings like 'get_dials' or 'set_dial'. However, it assumes domain knowledge about what a 'dial' is without further definition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_dials' (which likely returns current values) or 'apply_preset'. The description does not specify contextual usage conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_presetB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It only states 'Apply' without revealing side effects (e.g., overwrites all dials?), reversibility, or permissions. Lacks disclosure of any behavioral traits beyond the action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence, no fluff. Lists values immediately. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter enum tool without output schema, the description covers the basic function and options. Missing context on what happens after applying the preset (e.g., modifies current dials). Adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description names the parameter (preset) via the listed values. However, the schema already provides the enum; description adds minimal extra context (labeling them as 'named presets'). Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool applies a named preset, with a specific verb (Apply) and resource (preset). It distinguishes from siblings like list_presets or set_dial by indicating it applies a full preset rather than listing or modifying individual settings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Does not mention exclusions, prerequisites, or compare with siblings like reset_dials or set_dial. Usage context is purely implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_dialA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adds one behavioral trait: persistence across sessions. However, it does not disclose whether setting overrides existing values, whether the effect is immediate, or any side effects. For a mutation tool, more detail would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise—two short phrases that quickly convey core info. No wasted words. However, it could be slightly more structured (e.g., front-loading the verb) without significant length increase.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no output schema and 100% parameter coverage, the description is adequate. It covers persistence and value range. However, it lacks context on what a dial controls or how this setting interacts with other dials and presets, which would help an agent understand when to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters with descriptions (dial name, value 0–10). The description adds no further semantic meaning beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action (set) and resource (one behavior dial) with constraints (value 0–10, persists across sessions). It effectively distinguishes this tool from siblings like get_dials (read-only), reset_dials (unset), and apply_preset (batch set).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context for setting a single dial, but does not explicitly guide when to use this tool versus alternatives like apply_preset, get_dials, or reset_dials. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_presetsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It clearly states it lists presets and their descriptions, implying a read-only operation, but does not explicitly disclose safety traits, side effects, or data sources.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resource, no wasted words. Perfectly concise and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description is adequately complete: it explains what the tool returns (list of presets with descriptions). However, it could mention that no filtering or pagination is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description is not required to add parameter info; baseline 4 is appropriate as it adds no value beyond the schema but does not need to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and resource 'presets', clearly stating the action and output. It distinguishes from siblings like apply_preset and explain_dial, which have different verbs and resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. The description simply states what it does, leaving the agent to infer appropriate usage without context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reset_dialsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It clearly states the action (reset all dials) and outcome (balanced default preset), but doesn't disclose details like reversibility or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence with no extraneous words. Every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a simple reset operation with no parameters. It lacks details on edge cases (e.g., already at default) but overall covers core behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has no parameters, so schema coverage is 100%. Description does not need to add parameter info; it correctly describes a parameterless operation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the verb 'Reset' and resource 'all dials' with a clear target 'balanced default preset', distinguishing it from siblings like set_dial or apply_preset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for resetting all dials to default but lacks explicit when-not-to-use or alternatives. Sibling tools like apply_preset could be alternatives but are not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dialsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses that the tool loads behavioral dials (rigor, verification, etc.) and warns 'Higher is not always better', adding helpful context. However, it does not describe the output format or side effects, though for a parameterless read tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The first sentence provides the imperative purpose, the second adds a caution. Ideal length for immediate comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description fully covers what the agent needs to know: when to call it (start of task) and what it does (load dials). It also lists example dials, making the context complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters and schema coverage is 100%. The description adds meaning beyond the schema by explaining what the dials represent and the mandatory usage pattern, justifying a baseline score of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it loads the user's binding behavior dials and explicitly marks it as required at the start of every task, distinguishing it from siblings that apply, explain, list, reset, or set dials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'REQUIRED at the start of every task', providing a strong usage directive. It also instructs the agent to follow the dials exactly, with no ambiguity about when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.