Server Quality Checklist

Tool Scores

• create_scatter_plotB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits. It mentions output options (svg/png) but fails to address side effects, permissions, data validation, or processing behavior. The core creation action is stated, but additional important behavioral details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured as a clean parameter list with concise explanations. While slightly lengthy due to nine parameters, it is front-loaded with the purpose and efficiently uses bullet-like syntax. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description need not detail return values. However, it lacks information on requirements, error conditions, or data constraints. For a complex tool with many options, it feels slightly incomplete, but covers the basics adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description provides critical meaning for each parameter (e.g., 'X data points', 'Optional point labels (for hover)'). It also documents allowed values for 'palette' and 'output', which are absent from the schema. This fully compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a scatter plot,' which is a specific verb and resource. The tool name and description immediately distinguish it from sibling tools like create_bar_chart, create_histogram, etc., making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as create_line_chart or create_histogram. There is no mention of prerequisites, context, or exclusions, leaving the agent to infer usage without support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_pie_chartB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It lacks details on side effects (e.g., file creation for PNG output), data validation (e.g., values must be positive? sum to 100?), and error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured as a clear list with the purpose first. Some redundancy exists (e.g., 'Show percentage labels on slices' is obvious from parameter name 'show_percent'), but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers parameter details and output options. However, lacks usage guidance, behavioral traits, and constraints (e.g., label-value alignment). Incomplete for a tool with 6 parameters and no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by explaining each parameter: 'values: Numeric values (proportions)', 'labels: Slice labels (optional)', and enumerating palette options. Adds meaning beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb and resource: 'Create a pie / donut chart.' This is specific and distinguishes it from sibling chart tools like create_bar_chart or create_line_chart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use a pie chart versus alternatives (e.g., bar chart for comparisons). Missing context about suitability for proportional data vs. other chart types.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_bar_chartA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It explains output formats (svg vs. png) and horizontal option, but does not detail file saving behavior, error handling, or data validation. Some transparency exists, but gaps remain about side effects and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured as a clear parameter list, front-loading the purpose. However, it is somewhat verbose with 'Args:' formatting; each sentence earns its place but could be more succinct.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a chart creation tool, the description covers all parameters and output options, making it fairly complete. It does not mention error behavior or performance, but given the tool's simplicity and explicit parameters, it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description fully compensates. It explains each parameter's purpose, defaults, and options (e.g., palette choices, output formats). This adds essential meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a bar chart.' and lists parameters, indicating the specific resource and action. However, it does not differentiate from sibling tools like create_histogram or create_line_chart, missing an opportunity to clarify when to use this tool over alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus sibling tools (e.g., bar chart vs. histogram). The description lacks context about preferred use cases, prerequisites, or scenarios where this tool is unsuitable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_histogramA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It explains output options (svg vs png) and cumulative distribution, but does not mention side effects, permissions, or handling of edge cases like missing values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with a clear opening statement followed by parameter list. Slightly lacking in structure (e.g., no bullet points), but no unnecessary content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters and presence of output schema, description covers parameter semantics and output format choices. Could be enhanced with error handling or value constraints, but largely sufficient for a plotting tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description adds meaningful explanations for each parameter (e.g., 'Numeric data points', 'Number of bins (default: 10)'), compensating well beyond schema titles and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Create a histogram,' a specific verb and resource. It distinguishes from sibling tools which are other chart types (bar, line, pie, scatter).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use histogram versus alternative chart types. The description only lists parameters, not usage context or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_line_chartA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It mentions output format options (svg/png) and lists parameters with defaults, which adds some transparency. However, it does not disclose side effects, authentication requirements, or error handling behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear purpose statement followed by a compact list of parameters. Every sentence is informative, and the information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (8 parameters, output options), the description covers key aspects: purpose, parameters, and output format. The presence of an output schema reduces the need to document return values. Minor gaps include lack of error conditions or performance notes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides semantic explanations for all 8 parameters (e.g., 'X-axis data points', 'Color palette: default, vibrant, pastel, monochrome'), which adds value beyond the schema's titles and types. With 0% schema description coverage, the description compensates adequately, though details are somewhat brief.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Create a line chart', which is a specific verb+resource that clearly indicates the tool's function. It distinguishes itself from sibling tools like create_bar_chart or create_scatter_plot by specifying 'line chart'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., bar chart, scatter plot). The description simply states what it does without context on appropriate use cases or when to avoid it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.