Server Quality Checklist

Tool Scores

• preview_dataB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds that the operation is local and returns a Markdown table. It does not disclose offset behavior, behavior on missing files, or whether large files are fully loaded, but for a simple read-only preview the annotations and description together are adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. Every phrase contributes meaning: return, first N rows, local data file, Markdown table.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the return format and core operation, but it omits offset semantics and does not position the tool relative to the sibling tools. Since there is no output schema, a bit more context about edge cases or usage would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents path and limit, but offset lacks a description. The phrase 'first N rows' adds little beyond the schema's 'Rows to show (default 20)', and no parameter-level meaning is added in the description for offset.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return'), the resource ('first N rows of a local data file'), and the output format ('Markdown table'). This is specific enough to distinguish it from siblings like query_data, which implies a more general row-returning operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use this tool versus alternatives such as query_data or describe_data, and it does not state exclusions or prerequisites. The use case is only implied by the word 'preview' in the title and 'first N rows' in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_data_filesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true and openWorldHint=false, so the description does not need to cover the read-only or local nature. It adds useful behavioral context by stating that only tabular formats are returned and that output includes file sizes. It does not discuss subtle behaviors like hidden files, sorted order, or invalid-directory errors, but the annotation coverage lowers the bar.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that puts the action, target, and output detail first, then packs the format list into a clean parenthetical. There is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with only two well-documented parameters and read-only annotations, the description is nearly complete. It tells the agent what is returned (tabular data files with sizes) and the supported formats. Minor omissions, such as default behavior for 'recursive' and error handling for nonexistent directories, do not significantly hamper correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for both parameters: 'dir' and 'recursive' are each described. The description adds some context by specifying 'local' and listing supported file formats, but it does not add meaningful parameter-level details beyond the schema. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('List') and a precise resource ('tabular data files CSV/TSV/JSON/NDJSON/Parquet/Excel in a local directory'), and adds the output detail 'with sizes'. This clearly distinguishes it from sibling tools like preview_data, describe_data, and query_data, which operate on the file contents rather than listing files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit when-to-use or when-not-to-use guidance relative to the siblings. However, the description strongly implies this is the discovery step before analyzing or previewing data files. Clear alternative guidance would improve selection reliability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_dataA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reinforces the readOnlyHint annotation and adds concrete behavioral detail: 'the file is never modified' and 'Returns a Markdown table'. It also constrains the query language with 'No SQL'. These details go beyond the annotation and give the agent a clearer picture of side effects and output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no waste. The core behavior is front-loaded, every clause adds information, and the return format and safety guarantee are stated tersely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately complex tool with no output schema, the description covers the important decision factors: read-only, file never modified, Markdown return, and supported query operations. It falls short only on file-format/error behavior and explicit sibling tool placement, but these are secondary given the schema and sibling names.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description maps many schema parameters to high-level concepts: where -> filter, select -> columns, order_by, limit/offset, group_by with aggregations. However it does not detail sub-fields like op operators, value formats, or order_by.dir. Schema description coverage is 63%, so the schema already handles some burden, but the description does not fully compensate for the remaining uncovered parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Run a read-only structured query over a local data file' and enumerates concrete operations (filter, select, order_by, limit/offset, group_by with aggregations). It does not explicitly differentiate from sibling tools like describe_data or preview_data, but the querying scope is clear enough to understand its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to choose this tool over siblings such as describe_data or preview_data. The description implies use for structured querying via its operation list, and 'No SQL, no writes' sets boundaries, but it does not state when-not or alternatives, leaving routing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_dataA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already marks this as safe, and the description adds meaningful behavioral insight: it reads a sample for very large files, implying approximate or sampled statistics rather than exact full-file calculations. This goes beyond the annotation and helps set expectations about result accuracy.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The core output is listed first, and the token-efficiency/sampling note is a single concise second sentence. Every part contributes to understanding the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a profiling tool with no output schema, the description enumerates what will be returned (schema, row/column counts, per-column statistics) and mentions sampling behavior. Minor gaps include supported file formats and whether 'local' excludes remote paths, but overall the definition is sufficient for an agent to understand scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for both path and limit. The tool description reinforces that limit controls sampling, but adds little beyond the schema's existing parameter documentation. Baseline 3 is appropriate since the schema handles parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a specific resource (a local data file's schema, counts, per-column statistics). This clearly distinguishes it from siblings like preview_data (raw rows), query_data (querying), visualize_data (charts), and diff_data (comparisons). No ambiguity about what the tool produces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: for getting a summary of a data file's structure and statistics, particularly for very large files where token-efficient sampling is valuable. It does not explicitly name alternatives or say when not to use it, but the context of profiling and sampling is clear enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diff_dataA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal (readOnlyHint=false, openWorldHint=false), so the description carries the disclosure burden. It adds valuable behavioral context: 'Both files stay local', it reports added/removed/changed/unchanged counts, and it can optionally write an HTML report. This covers privacy and side-effect expectations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tight sentences with no filler. It front-loads the core purpose, then details outputs and the local-only guarantee. Every sentence contributes essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four parameters and no output schema, the description covers the main outcome (counts plus optional report path) and the local-processing guarantee. It could be slightly more explicit about behavior when no --key is supplied, but overall it gives enough context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is already individually documented. The description adds some context by explaining the role of --key in row matching and the out_path report, but it does not substantially go beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Compare two local data files' with a useful metaphor ('a git-style diff for data'). It clearly distinguishes this tool from siblings like query_data or visualize_data by focusing on comparison and diff reporting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this when comparing two local data files, especially when row-level change counts or an offline HTML report are needed. It does not explicitly name alternatives or state when not to use it, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• visualize_dataA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only say readOnlyHint is false; the description goes further and discloses the write behavior (file on disk), the return value (path), and important behavioral constraints (fully offline, zero external requests, data never leaves the machine). This is exactly the kind of context an agent needs beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core transformation and remains compact, but there is mild redundancy between 'fully-offline,' 'zero external requests,' and 'data never leaves the machine.' All sentences earn their place, though one could be trimmed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters, nested objects, and no output schema, the description covers the high-level purpose and output but omits operational details such as supported input formats, default title/limit behavior, and whether an existing out_path is overwritten. It is adequate for selection but not fully complete for invocation edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 70%, so the schema already explains most parameters such as path, where, select, group_by, out_path, and aggregate. The description adds only high-level framing ('optionally after a query') and does not document the undocumented params (limit, title, offset, order_by), but the missing semantics are largely inferable from names and schema positions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Turn ... into') with a concrete resource and result: a local data file becomes a self-contained interactive HTML explorer. It clearly distinguishes this from siblings by emphasizing a shareable, explorable offline artifact rather than a plain text table.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear context: use when the user wants a shareable, explorable artifact instead of a plain text table, implying this over preview/query output. It does not explicitly name alternatives or exclusion conditions relative to list_data_files, describe_data, or diff_data, so it falls short of explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.