Server Quality Checklist

Tool Scores

• sqlbot_describe_tableC
  2.6/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It does not mention that the tool is read-only, the nature of the result, or any side effects. The only clue is 'Describe checked fields', which is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise, but it does not make efficient use of that brevity. It could be restructured to include more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and 4 parameters (2 required), the description lacks critical context about what the tool returns. The phrase 'Describe checked fields' is ambiguous; it does not specify the format or extent of information provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides descriptions for all 4 parameters (100% coverage), so the description adds no additional meaning. The tool name and description already imply the role of `table_name` and `datasource_id`, but the optional `access_key` and `secret_key` are not elaborated beyond their schema defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Describe') and resource ('checked fields for a SQLBot metadata table'), which distinguishes it from sibling tools like sqlbot_list_tables. However, it is somewhat vague as it does not clarify what 'checked fields' means or how it differs from listing tables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as sqlbot_ask_data or sqlbot_generate_report. The description does not indicate prerequisites or non-applicable scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sqlbot_ask_dataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. The description only states the tool processes natural-language questions via a pipeline. It does not reveal whether the tool modifies data, authentication requirements (though parameters hint at keys), rate limits, or side effects. The finish_step parameter suggests multi-step behavior, but this is not explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) and front-loaded with the core purpose. However, given the tool's complexity (9 parameters), it could be slightly longer to include essential context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is incomplete for an agent to fully understand the tool's capabilities. It does not mention the output format, the meaning of the finish_step enum, or how the access_key/secret_key are used. Given no output schema and low schema coverage, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 44% (4 of 9 parameters have descriptions). The description adds no information about parameters, failing to compensate for the low coverage. Parameters like lang, return_img, finish_step, include_analysis, and data_preview_limit are not explained, leaving an agent without guidance on their purpose or defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool allows asking a natural-language data question via a Text-to-SQL pipeline. It uses a specific verb ('Ask') and resource ('SQLBot'), and the mention of 'existing Text-to-SQL pipeline' distinguishes it from sibling tools like sqlbot_describe_table or sqlbot_list_datasources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios, prerequisites, or when not to use it. Sibling tools exist for specific tasks like describing tables or listing sources, but no comparison or context is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sqlbot_list_tablesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the basic function, omitting details like read-only nature, pagination, error handling, or side effects. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no wasted words. However, it could be expanded slightly to include more behavioral context without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description should provide more context about return values, expected behavior, and potential errors. It currently lacks this, making it incomplete for a listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All three parameters are fully described in the input schema (100% coverage), so the description adds no additional meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists metadata tables for a datasource, using a specific verb and resource. It distinguishes from sibling tools like sqlbot_list_datasources (lists datasources) and sqlbot_describe_table (describes a specific table). However, the term 'checked' is slightly ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as sqlbot_list_datasources or sqlbot_describe_table. The description lacks any context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sqlbot_list_datasourcesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavior beyond listing, such as read-only nature, authentication requirements, or pagination; it does not.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no extraneous information, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should clarify return values or structure; it does not, leaving the agent without expected output context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter descriptions; the tool description adds no additional meaning beyond the schema, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists SQLBot datasources and specifies the API key context, distinguishing it from sibling tools that deal with tables, queries, and reports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool over alternatives like sqlbot_list_tables or sqlbot_describe_table; usage context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sqlbot_generate_reportA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description effectively discloses key behavior: generation with analysis, conclusion, chart, and automatic fallback to local SVG on failure or timeout. This goes beyond the schema, but could mention required auth or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two sentences that cover purpose, output, and fallback behavior without any redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 params, no output schema), the description explains the report content (analysis, conclusion, chart) and fallback mechanism, but lacks specifics on output format or how the report is returned (e.g., file, data). This leaves some ambiguity for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 63% of parameters with descriptions, meeting the baseline. The tool description adds no additional parameter-level information, so it neither improves nor detracts from the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a SQLBot data report with analysis, conclusion, and chart, distinguishing it from sibling tools that focus on querying, describing, or listing. The fallback behavior is also mentioned, adding specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the purpose is clear, there is no explicit guidance on when to use this tool versus alternatives like sqlbot_ask_data. The description implies usage for report generation but does not provide when-not or contextual triggers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.