Server Quality Checklist

Tool Scores

• quick_plotB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It explains that the server validates the figure and returns a JSON string, and justifies the string format for robustness. However, it omits details on authorization, rate limits, side effects, or error states.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, front-loaded with the core purpose. It is efficient but includes a slightly redundant justification about robustness. All sentences add value; no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description adequately covers return format. However, with three siblings listed, it lacks comparative guidance, making it less complete for an agent choosing among tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds a high-level usage statement ('pass data as a list of records and choose a chart kind') but does not enrich parameter semantics beyond what the schema already provides. No additional nuance for x, y, color, or title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds an interactive Plotly figure and returns JSON, distinguishing it from siblings that output HTML (render_figure_html) or describe plots (describe_plot). The use of 'return it as JSON' and 'frontend can parse' differentiates the output format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells how to use the tool (pass data as records, choose kind) but provides no guidance on when to use this tool versus siblings like create_figure or render_figure_html. No exclusions or contexts are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_figureA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the server builds the figure, surfaces schema errors, and returns it as a JSON string. This gives the agent a clear picture of what happens during execution, including validation and return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: purpose, usage guideline, behavioral note. It is concise, front-loaded, and every sentence adds essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter with nested structure, an output schema, and siblings, the description is fairly complete. It covers purpose, usage, and behavior. It could potentially mention that the output is a JSON string of the validated figure, which it does. No critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage of the single parameter, and the description's phrasing about the figure spec largely echoes the schema. It adds no new semantic detail beyond what the schema already states, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Construct and validate a Plotly figure from a full figure spec' with specific verb and resource. It distinguishes from sibling tools by emphasizing 'full control over traces and layout', implying the other tools are for simpler or different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a direct usage guideline: 'Use this when you want full control over traces and layout.' This helps the agent select this tool over siblings for full control tasks, though it does not explicitly mention when not to use or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_figure_htmlA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses output format (HTML string), portability (save/embed), and the inline_plotlyjs option's tradeoff. Lack of mention of edge cases or constraints prevents a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, output usage, usage guidance. No redundant information. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, output schema exists), the description covers purpose, usage, parameters, and behavior comprehensively. It is adequate for correct agent selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds value by relating figure param to create_figure's shape and explaining the inline_plotlyjs tradeoff (file size vs dependency). This exceeds mere schema repetition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Render' and the resource 'Plotly figure spec to HTML string'. It distinguishes from 'raw figure JSON' and implies contrast with siblings like create_figure and quick_plot.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when a consumer wants a portable, viewable artifact rather than the raw figure JSON.' Provides clear context but doesn't explicitly list when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_plotA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool runs a code-generation loop using an AI agent (Gemini, opencode), which is a key behavioral trait. It mentions slower performance but does not cover data privacy or timeout risks. With no annotations, this is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loading the main purpose, then adding context about speed and return format. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers the input (data and description), the process (code generation), the output (JSON same as siblings), and comparisons to related tools. It is complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining data as 'rows as JSON objects' and description as 'natural-language description', going beyond the schema's minimal wording.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it builds a Plotly figure from data and a natural-language description, and returns JSON. It distinguishes from siblings by naming quick_plot and create_figure and noting it is slower.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this for open-ended chart requests and that it is slower than other tools, guiding when to use or avoid it. It names alternative tools for similar tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.