Server Quality Checklist

Tool Scores

• list_datasetsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the responsibility of behavioral disclosure. It clearly states the tool lists tables and views with type and column count, which is the primary behavior. However, it does not explicitly mention that it is read-only or describe any permissions or side effects, though the verb 'List' implies a non-mutating operation. This is a minor gap, making the transparency adequate but not fully explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first sentence stating the exact purpose and return fields, and the second providing context on when to call it. There is no redundant information; every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description covers the essential information: what is listed, what is returned, and when to use it. It also provides a rationale that ties into the sibling tool run_sql_query. Minor gaps like handling of empty databases or system schemas are not critical for this simple tool, so it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is empty. Per guidelines, the baseline for 0 parameters is 4. The description adds no parameter-specific semantics because none exist, but it also doesn't need to. Thus, it meets the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with a clear resource ('every table and view available in the database') and explicitly states the output fields ('type and column count'). It distinguishes itself from sibling tools by positioning this as a discovery tool, and the note that run_sql_query needs real table names reinforces its role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage timing: 'Call this first when exploring an unknown database' and explains why, noting that run_sql_query depends on these names. This is clear guidance for when to use it. It doesn't explicitly say when not to use it, but for a listing tool, the directive to call it first is sufficient. It also implies an alternative (run_sql_query) but frames it as a dependent tool rather than a direct alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_data_profileA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden; it compensates by disclosing output fields, the approximate nature of distinct counts, and the DuckDB SUMMARIZE implementation with large-table efficiency. It does not explicitly state that the operation is read-only, but the 'get' verb and summary-statistics framing strongly imply no mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: one purpose sentence, one implementation/performance sentence, and a one-item Args entry. Every sentence earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one parameter and no output schema, the description gives essential output details and input provenance, making selection and invocation feasible. It lacks a precise return-shape description and explicit alternative guidance, but the listed statistics and sibling tool names provide sufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only 'dataset' with a title and no description (0% coverage), while the description's Args section adds actionable semantics: 'Exact table or view name, as returned by list_datasets.' This fully clarifies the value format and provides the source for valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action ('Generate summary statistics') and explicitly enumerates the output fields (type, null percentage, approximate distinct count, min/max, mean/stddev/quartiles). This clearly distinguishes it from siblings like get_schema and get_row_count by covering full-column profiling rather than just schema or row count.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description supplies a concrete prerequisite ('dataset: Exact table or view name, as returned by list_datasets') and notes performance suitability for large tables via DuckDB's SUMMARIZE. It stops short of explicitly naming sibling alternatives or stating when not to use this tool, but the intended context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_row_countA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It clarifies that the tool returns an exact count for a single dataset, but it does not disclose whether this is a full scan or metadata lookup, nor any performance or failure characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: a purpose statement, a usage tip, and an Args section. Every sentence adds value, and the structure is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers purpose, context, and parameter semantics. It omits the exact return type, but given the name 'get_row_count' this is likely obvious; however, it could explicitly state that it returns a non-negative integer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description explains the 'dataset' parameter as 'Exact table or view name, as returned by list_datasets,' giving agents a precise source and format. This adds substantial meaning to the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and object — 'Get the exact row count for one dataset' — which clearly defines its function. It also hints at its targeted use case, distinguishing it from siblings like run_sql_query and list_datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use before run_sql_query on large tables to understand sampling scope. This provides clear contextual guidance and implies when not to use it (as a substitute for querying).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses what the tool does (check if server is running, can reach DuckDB, report guardrail config) and implies read-only behavior through 'Verify' and 'report'. It does not explicitly state 'does not modify data', but the health-check nature and reported fields are transparent enough. Could add explicit read-only confirmation, but the description is informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first gives specific technical details, the second gives direct usage guidance. Every sentence carries weight; there is no fluff or redundancy. It is front-loaded with the most important purpose first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema, simple health check), the description is complete. It explains what is verified (server, DuckDB connection), what is reported (guardrails: row limits, timeout, DB path), and when to use it. No additional context is necessary for an agent to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already covers this completely (100% coverage with an empty properties object). There is nothing for the description to add about parameters, so a baseline of 4 is appropriate. The description mentions the report fields, which indirectly clarifies what the tool returns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Verify' and the specific resource (OmniData database server, DuckDB reachability, guardrail configuration). It is highly specific and distinguishes itself from sibling tools like list_datasets or run_sql_query by focusing on connection health and configuration status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this first to confirm the MCP connection is alive.' This gives explicit usage timing (before other tools). It does not explicitly exclude alternatives or name when-not-to-use, but the instruction 'first' is strong guidance. Missing explicit alternative names prevents a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schemaA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It explicitly states the output content (column names, types, nullability) and that the input must be an exact name from list_datasets. The read-only nature is implied by 'Get', and no side effects or permission requirements are mentioned, but for a metadata read this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a brief Args section. Every sentence contributes value, and the front-loaded purpose statement ensures immediate understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read-only tool, the description provides the essential information: purpose, input requirement, and prerequisite. No output schema is needed because the return type is described as column names, types, and nullability, which fully specifies the result expected.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a string parameter with no description. The tool description fully compensates by explaining that 'dataset' must be the exact table or view name returned from list_datasets, and that it is required. This gives the agent precise guidance on how to populate the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving column names, types, and nullability for a single dataset. It distinguishes this from sibling tools like list_datasets (which lists available datasets) and get_row_count (which counts rows), and specifies both tables and views are covered.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Use list_datasets first to get valid names' provides a clear prerequisite and sequence of use. It does not explicitly list when not to use the tool, but the context of sibling tools makes the alternative use cases clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sql_queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job: it discloses read-only enforcement, single-statement restriction, automatic LIMIT injection, server-side row cap, and query timeout cancellation. This is far beyond minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a short intro, bullet-pointed guardrails, and a compact Args section. Every sentence adds relevant information and there is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description thoroughly covers safety guardrails, limits, and the row-count caveat. However, it does not describe the exact shape of the 'structured results' return value, and with no output schema, a bit more detail about the return format would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description fully compensates by explaining each parameter: sql is a single read-only statement, and row_limit is a desired cap that may be ignored if the query already has LIMIT. This adds meaning the schema lacks entirely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it executes a read-only SQL query and returns structured results, with explicit allowed statement types (SELECT/WITH/EXPLAIN/DESCRIBE/SHOW). It is easily distinguished from sibling tools like get_row_count, get_schema, and list_datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends using get_row_count when true table size is needed, providing a clear alternative. However, it does not explicitly contrast with other data exploration tools like get_data_profile or generate_chart, so the guidance is good but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_chartA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description fully carries the burden. It discloses read-only behavior, success returns an inline image, failure returns an error dict (with specific failure cases like bad SQL, missing column, empty result, oversized render), and explains the lack of a static return-type annotation due to SDK limitations. This is thorough and adds significant behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: an overview sentence, a return/error contract, and a clear argument list. While it is long, the length is justified by the number of parameters and the need to explain error handling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers input semantics, parameter defaults, failure modes, and return format. Since there is no output schema, it appropriately explains what the caller should expect on both success and failure. This is complete for a chart-generation tool with 6 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description fully compensates by explaining each of the 6 parameters: sql with aggregation advice, chart_type enum, x_column, y_column, optional series_column, and title default. This adds meaning far beyond the bare parameter names in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a read-only SQL query and renders the result as a chart image, specifying chart types (bar, line, scatter). It distinguishes itself from sibling tools like run_sql_query by focusing on chart rendering rather than raw query results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives useful usage context, such as read-only safety guardrails and advising the caller to aggregate/group data for cleaner charts. However, it does not explicitly state when to prefer this over alternatives like run_sql_query for non-chart needs, so it lacks an explicit exclusion or alternative recommendation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_pyspark_jobA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so exceptionally. It discloses that steps run in order, an unconditional final row cap is applied, and row_limit is capped at server's max_row_limit. It also emphasizes the fixed op set and validation step, which prevents misuse.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place. It is front-loaded with the core purpose and limitation, then systematically covers operations, ordering, row cap, and arguments. No fluff or redundancy; the structure is logical and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with no output schema, the description is remarkably complete. It details the full set of supported operations with examples, explains ordering and row cap behavior, and ties arguments to other tools (list_datasets). An agent has everything needed to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates fully. Each parameter is explained: source_dataset is an exact table name from list_datasets, operations is an ordered list with a detailed enumeration of supported ops and examples, and row_limit is a desired max rows capped by server. This adds meaning far beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (run), resource (declarative PySpark transformation pipeline against DuckDB dataset), and scope (heavier aggregations/transformations). It explicitly distinguishes itself from run_sql_query, and the phrase 'NOT arbitrary code execution' further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use it: 'for heavier aggregations/transformations than run_sql_query is meant for'. It also explains constraints: each step must be one of a fixed set of operations, validated before running. This frames appropriate usage and sets expectations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.