Server Quality Checklist

Tool Scores

• count_rowsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'matching given filters' which implies filtering behavior, but doesn't disclose authentication needs (though 'access_token' parameter hints at it), rate limits, performance characteristics, error conditions, or what the return value looks like (just a count number or structured response).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a counting operation and front-loads the core functionality immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain the return format (e.g., integer count, JSON structure), error handling, authentication requirements, or how filters work with the table context. The schema handles parameter documentation, but behavioral aspects are largely missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds minimal value beyond the schema by implying that 'filters' parameter is used for matching rows, but doesn't provide additional syntax, format details, or examples beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Count') and resource ('rows'), specifying that it counts rows matching given filters. It distinguishes from siblings like 'query_table' or 'sql_search' by focusing on counting rather than retrieving data, but doesn't explicitly differentiate from them in the description text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'query_table' or 'sql_search', nor does it mention prerequisites or context for usage. It simply states what the tool does without indicating appropriate scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schema_fieldsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Get') but doesn't describe what 'schema fields' include (e.g., column names, types, constraints), whether authentication is required (though 'access_token' is optional in schema), or any rate limits or errors. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain the return format (e.g., JSON structure of schema fields), potential errors, or usage context relative to siblings. For a tool with no structured output or behavioral hints, more detail is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all parameters well. The description doesn't add meaning beyond this, such as explaining relationships between parameters (e.g., how 'network', 'collection_slug', and 'table_name' combine to identify the table). Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('schema fields for a specific table'), making the purpose evident. However, it doesn't distinguish this tool from sibling tools like 'list_tables' or 'query_table', which might also involve schema-related operations, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing to know the table name from 'list_tables', or contrast it with siblings like 'query_table' for data retrieval versus schema inspection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_collectionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it lists collections but doesn't mention if this is a read-only operation, requires authentication, has rate limits, or what the output format looks like (e.g., pagination, error handling). This leaves significant gaps for an agent to understand how to invoke it safely and effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It is front-loaded and appropriately sized, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a list operation with authentication parameters and no output schema, the description is incomplete. It doesn't address behavioral aspects like authentication needs, response format, or error handling, which are crucial for an agent to use this tool correctly in context with its siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('network' and 'access_token') with examples and requirements. The description adds no additional meaning beyond this, such as explaining parameter interactions or constraints, so it meets the baseline for adequate but not enhanced parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'collections available in an Omics AI Explorer network', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_tables', which might list different resources in the same network, leaving some ambiguity about uniqueness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_tables' or 'query_table'. It lacks context about prerequisites, such as network selection, or exclusions, making it unclear how it fits into the broader toolset.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but lacks details on permissions (e.g., whether 'access_token' is required for certain networks), output format (e.g., list of table names or full metadata), pagination, or error handling. This leaves significant gaps for a tool with authentication parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It directly states what the tool does, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not explain what the output looks like (e.g., a list of table names, JSON objects), how authentication works with the 'access_token', or potential limitations (e.g., rate limits, network-specific behaviors). For a tool with authentication and multiple parameters, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters ('network', 'collection_slug', 'access_token') with clear descriptions. The description adds no additional meaning beyond implying the tool operates within a collection, which is redundant with the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('tables in a specific collection'), making the purpose immediately understandable. However, it does not explicitly differentiate from sibling tools like 'list_collections' or 'get_schema_fields', which might list different resources or provide schema details instead of table names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it does not mention that 'list_collections' should be used first to find collections, or that 'get_schema_fields' is for table details rather than listing tables. There is no context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_tableC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions optional filters and pagination but fails to cover critical aspects like authentication needs (implied by 'access_token' parameter), rate limits, error handling, or return format. For a query tool with 8 parameters, this leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Query data from a table') and briefly mentions key features ('optional filters and pagination'). There is no wasted verbiage, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no annotations, no output schema), the description is inadequate. It doesn't explain authentication requirements, result format, error conditions, or how to interpret parameters like 'filters' or 'order_by'. For a data query tool, this leaves too many operational questions unanswered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by hinting at filters and pagination but doesn't provide additional context beyond what's in the schema (e.g., how filters work, pagination defaults). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Query') and resource ('data from a table'), specifying the core action. It distinguishes from siblings like 'count_rows' or 'get_schema_fields' by focusing on data retrieval rather than metadata or aggregation. However, it doesn't explicitly differentiate from 'sql_search', which might also query data, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'sql_search' or 'list_tables'. It mentions optional filters and pagination but doesn't specify use cases, prerequisites, or exclusions. This lack of context makes it challenging for an AI agent to choose between sibling tools effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sql_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only mentions the Trino syntax constraint. It omits critical behavioral details such as authentication needs (implied by access_token parameter but not explained), rate limits, query execution time, error handling, or what happens on failure. This is inadequate for a tool with multiple parameters and potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary details. Every word earns its place, making it easy to parse quickly while avoiding redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing SQL queries (with 6 parameters, no output schema, and no annotations), the description is insufficient. It lacks information on return values, error conditions, security implications, or how results are formatted, leaving significant gaps for an AI agent to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds minimal value beyond the schema by noting Trino syntax and double quotes for identifiers, but doesn't elaborate on parameter interactions or usage examples. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute a SQL query') and target ('against a collection'), specifying the syntax ('using Trino syntax'). It distinguishes from siblings like count_rows or get_schema_fields by focusing on query execution rather than metadata retrieval, though it doesn't explicitly contrast with query_table which might be similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like query_table or other siblings. The description implies usage for SQL queries on collections but lacks context on prerequisites, performance considerations, or exclusions, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.