Server Quality Checklist

Tool Scores

• aggregate_dataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'simple aggregation' but doesn't clarify permissions needed, rate limits, whether it's read-only or has side effects, or what the output format looks like. For a data querying tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence fragment that gets straight to the point. Every word earns its place by specifying the core functionality without any fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a data aggregation tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns, how results are formatted, error conditions, or provide enough context for an agent to understand the full scope of this operation beyond the basic function listing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds minimal value by listing the aggregation functions but doesn't provide additional context about parameter interactions or usage patterns beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs aggregation operations (count, sum, avg, min, max) on data, which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'preview_table_data' or 'run_graphql_query' that might also access data, leaving some ambiguity about when to choose this specific aggregation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'run_graphql_query' that might handle similar data operations, nor does it specify prerequisites or contexts where aggregation is preferred over other data retrieval methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_graphql_typeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. The description only states what the tool does ('Provides details'), but doesn't disclose any behavioral traits such as whether it's a read-only operation, what format the details are provided in, error handling, performance characteristics, or authentication requirements. For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence that efficiently communicates the core function. It's front-loaded with the main purpose and includes useful enumeration of type categories without unnecessary elaboration. Every word earns its place with no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that presumably returns structured data about GraphQL types. It doesn't indicate what 'details' are provided (fields, descriptions, interfaces, etc.), the format of the response, or any limitations. For a tool that likely returns complex type information, this leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter 'typeName'. The description doesn't add any parameter semantics beyond what's in the schema - it doesn't explain what constitutes a valid type name beyond 'specific GraphQL type', provide examples, or clarify the relationship between type categories and the parameter. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Provides details about a specific GraphQL type' with explicit enumeration of the type categories (Object, Input, Scalar, Enum, Interface, Union). It uses a specific verb ('Provides details') and identifies the resource ('GraphQL type'), but doesn't explicitly differentiate from sibling tools like 'list_root_fields' or 'run_graphql_query' which might also provide type-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_root_fields' (which might list types) or 'run_graphql_query' (which might return type information indirectly), nor does it specify prerequisites or appropriate contexts for usage beyond the basic function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions checking reachability but doesn't specify what 'reachable' entails (e.g., HTTP status codes, timeouts, authentication requirements). For a diagnostic tool with zero annotation coverage, this leaves critical behavioral traits like error handling or performance implications unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary details. It's appropriately sized and front-loaded, though it could be slightly more structured by including usage hints. Overall, it earns its place without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's diagnostic nature and lack of annotations or output schema, the description is incomplete. It doesn't explain what the check entails (e.g., network connectivity, endpoint response validation) or what results to expect (e.g., success/failure indicators). For a tool with no structured output documentation, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'healthEndpointUrl' with its description and format. The description doesn't add any meaning beyond what the schema provides, such as explaining default behavior when the parameter is omitted. Baseline 3 is appropriate when the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Checks if the configured Hasura GraphQL endpoint is reachable...' This specifies the verb ('checks') and resource ('Hasura GraphQL endpoint'), though it doesn't explicitly differentiate from siblings like 'run_graphql_query' which might also test connectivity indirectly. The purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing a configured endpoint, or compare it to siblings like 'run_graphql_query' that might serve similar diagnostic purposes. Without such context, users must infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_root_fieldsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions listing fields but doesn't disclose behavioral traits such as whether this is a read-only operation, if it requires authentication, rate limits, or what the output format looks like (e.g., list of strings, structured data). This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of GraphQL operations and lack of annotations or output schema, the description is incomplete. It doesn't explain what 'available' means (e.g., based on permissions), how results are returned, or provide context for the sibling tools, leaving gaps in understanding the tool's full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single optional parameter 'fieldType' with its enum values. The description implies filtering by field type but doesn't add any syntax, format, or usage details beyond what the schema provides, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Lists') and the resource ('available top-level query, mutation, or subscription fields'), making the purpose understandable. However, it doesn't explicitly differentiate from siblings like 'describe_graphql_type' or 'run_graphql_query', which might also involve GraphQL operations, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios like exploring GraphQL schema, comparing to 'describe_graphql_type' for detailed field info, or using 'run_graphql_query' for actual queries, leaving the agent with no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_graphql_mutationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool performs data modifications (insert, update, delete), which implies mutation operations, but fails to address critical aspects like authentication requirements, error handling, side effects, or response format. This leaves significant gaps for a tool that modifies data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary elaboration. It's appropriately sized for its purpose, though it could be slightly more structured by explicitly contrasting with the query sibling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a GraphQL mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns, error conditions, or behavioral nuances, making it incomplete for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters ('mutation' and 'variables') adequately. The description adds no additional semantic context beyond what's in the schema, such as examples of mutation strings or variable usage, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as executing GraphQL mutations for data operations (insert, update, delete). It specifies the action ('executes') and resource ('GraphQL mutation'), though it doesn't explicitly differentiate from its sibling 'run_graphql_query' beyond the mutation vs query distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'run_graphql_query' or other data manipulation tools. It lacks context about prerequisites, appropriate scenarios, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but lacks behavioral details. It doesn't disclose whether this is a read-only operation, what permissions are required, how errors are handled, or the format of the returned structure. The description is minimal beyond stating the output content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part of the sentence contributes directly to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that returns structural metadata. It doesn't explain the return format, error conditions, or behavioral traits, leaving significant gaps for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, such as examples or constraints. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'shows' and the resource 'structure of a table', specifying what information is included (columns with types and descriptions). It distinguishes from siblings like list_tables (which lists table names) and preview_table_data (which shows actual data), though not explicitly named.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing table metadata rather than data or schema listings, but doesn't explicitly state when to use this tool versus alternatives like list_tables or preview_table_data. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_table_dataB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions the default limit of 5 rows, which is useful behavioral context. However, it lacks details on permissions needed, error handling (e.g., if table doesn't exist), or response format (e.g., structure of returned rows), leaving gaps for a tool that fetches data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('fetch esa limited sample of rows') and includes key details (default limit, table specification). There is no wasted text, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is adequate for a simple data-fetching tool with two parameters. It covers the basic purpose and default behavior, but lacks details on output (e.g., what 'esa' means, row format) and error conditions, which could be important for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (tableName and limit). The description adds minimal value by mentioning the default limit of 5, but doesn't provide additional context beyond what the schema already covers, such as examples or constraints not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetch') and resource ('rows from a specified table'), specifying it's a 'limited sample' with a default of 5 rows. It distinguishes from siblings like 'describe_table' (metadata) or 'run_graphql_query' (full queries), but doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for previewing table data with a sample, suggesting it's for quick inspection rather than full data retrieval. However, it doesn't explicitly state when to use this versus alternatives like 'run_graphql_query' for complete data or 'list_tables' for table discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions organization by schema with descriptions, which adds context beyond a basic list. However, it lacks details on behavioral traits such as pagination, rate limits, authentication needs, or what 'available' means (e.g., permissions). The description does not contradict annotations (none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Lists available data tables') and adds useful context ('organized by schema with descriptions'). There is no wasted verbiage or redundancy, making it appropriately sized for its function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is moderately complete. It covers the purpose and organization but lacks details on output format (e.g., structure of returned data), error handling, or integration with siblings. Without an output schema, more guidance on return values would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'schemaName' fully documented in the schema as an optional filter. The description does not add any parameter-specific details beyond what the schema provides, such as format examples or default behavior when omitted. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Lists') and resource ('available data tables (or collections) managed by Hasura'), specifying organization by schema with descriptions. It distinguishes from siblings like 'describe_table' (detailed metadata) and 'preview_table_data' (data viewing), though not explicitly named. However, it could be more specific about what 'lists' entails (e.g., names, counts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for browsing tables by schema, but does not explicitly state when to use this tool versus alternatives like 'describe_table' (for detailed info) or 'list_root_fields' (for GraphQL endpoints). No guidance on prerequisites or exclusions is provided, leaving usage context inferred rather than defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_graphql_queryA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the 'read-only' behavioral trait, which is crucial for safety, but lacks details on rate limits, authentication needs, error handling, or response format. The description adds some value but doesn't fully compensate for the missing annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads key information ('Executes a read-only GraphQL query'). It wastes no words and clearly communicates the core purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a mutation sibling, the description is adequate but has clear gaps. It covers the read-only constraint and endpoint, but doesn't explain return values, error cases, or how it differs from 'run_graphql_mutation'. For a tool with behavioral complexity and sibling tools, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any meaning beyond what's in the schema (e.g., no examples or constraints on query structure). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'executes' and resource 'GraphQL query' with the specific constraint 'read-only' and target 'Hasura endpoint'. However, it doesn't explicitly distinguish this from its sibling 'run_graphql_mutation', which is a notable gap since both tools involve GraphQL operations against the same endpoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the 'read-only' constraint, suggesting this tool is for queries rather than mutations, but it doesn't explicitly state when to use this vs. 'run_graphql_mutation' or other siblings like 'list_tables'. No explicit alternatives, exclusions, or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.