Server Quality Checklist

Tool Scores

• read_metadataA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden. It discloses that only first rows are read for efficiency, provides output details (file info, per-column profile, warnings, suggestions), and implies read-only behavior. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two paragraphs, front-loading the main purpose. Every sentence adds value, though it could be slightly tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description efficiently covers what's returned (file info, column profile with stats, warnings, suggested operations) and ties to sibling tools. It's complete for a simple profiling tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter file_path is fully described in the schema. The description adds context about reading first rows but doesn't augment parameter meaning beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool profiles a data file, listing specific outputs (structure, types, quality warnings, suggested next steps) and explicitly positions it as the recommended first call for unknown datasets, distinguishing it from siblings like run_pandas_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends this as the first call when exploring an unknown dataset, and mentions efficiency (reads only first rows). While it doesn't specify when not to use it, the context is clear relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_chartjsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the HTML file is written to a configurable directory (MCP_CHARTS_DIR, default ./charts) and returns the path. It also notes the behavioral quirk about pie charts using only the first series. This is adequate for behavioral expectations, though more details on overwriting or authentication would improve it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only five sentences, each essential. No redundant or verbose language. It starts with the core purpose, then elaborates on supported types, output location, and usage hints. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, nested data object, output schema exists), the description covers the key aspects: purpose, input format, output file generation, and special behavior. It omits details like error handling or file naming, but these are reasonable gaps for a straightforward tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 67% schema description coverage, the description adds value by explaining that data should be aggregated and that pie charts only use the first series. The chart_type enum is reinforced, and the title parameter is straightforward. The description supplements the schema's missing description for chart_type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Generate an interactive Chart.js HTML file from series data,' specifying the verb and resource. It distinguishes from siblings (read_metadata, interpret_column_data, run_pandas_code) by focusing on visualization output. Examples of chart types and input source are given.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises to 'Feed it aggregated data — e.g. the output of a run_pandas_code group-by,' providing clear when-to-use guidance. It also notes 'Pie charts use the first series only,' which is a usage caveat. However, it does not explicitly state when not to use or mention alternatives, keeping it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_pandas_codeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It describes the sandbox, restricted operations (import, open, exec, etc.), and result format (records for DataFrames/Series). It does not mention timeouts or error handling, but the critical safety behaviors are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (~100 words) and well-structured, with the main action first, followed by scope, file loading, assignment rule, safety restrictions, and result format. Every sentence provides necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code execution) and no annotations, the description covers allowed operations, restricted actions, file integration, and output format. It is mostly complete but could mention error behavior or limits for better agent guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the scoped libraries (pd, np), the preloading of file_path into a DataFrame df, and the assignment requirement. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it executes pandas code in a restricted sandbox and returns the result. It distinguishes itself from sibling tools like generate_chartjs, read_metadata, and interpret_column_data by being the only tool that runs arbitrary code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool (available libraries, file loading, assignment to result) but does not explicitly state when not to use it or mention alternatives. It is adequate but lacks exclusions or comparison hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• interpret_column_dataA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses key behaviors: it scans the whole file, reports dtype/total/null/unique counts, value frequencies sorted most-common first, and caps frequencies at 200 distinct values. It does not mention performance implications but covers essential behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two paragraphs. The first sentence immediately states the purpose. The second paragraph provides essential details without superfluous words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three parameters and an output schema, the description explains the output components (dtype, counts, frequencies) and behavior (full scan, 200 cap). It adequately informs an AI agent for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all parameters are described. The description adds value by mentioning file format support (CSV/TSV/Excel), sheet_name applicability, and the cap on distinct values. This enriches the schema's bare descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'complete value distribution' of columns, specifying verb 'return' and resource 'value distribution'. It distinguishes itself from sibling read_metadata by noting it scans the whole file rather than a sample, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states this tool is 'ideal for understanding categorical columns' and contrasts with read_metadata ('Unlike read_metadata, this scans the whole file'). It does not explicitly list when not to use or name other alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.