Server Quality Checklist

Tool Scores

• execute_graphqlA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions that the tool 'returns actual data' and includes error details, but does not explicitly state whether it is read-only, any authorization requirements, or potential side effects. Additional behavioral details would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for args, returns, workflow, and an important note. While slightly verbose, each sentence adds value and the critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter and no output schema, the description covers purpose, usage prerequisites, workflow, and error handling. It is sufficient for an agent to use the tool effectively, though mentioning read-only nature would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a 'query' string with no description. The description adds meaning by specifying it must be a valid GraphQL query validated by validate_query, and explains return values. This compensates for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute your GraphQL query and retrieve data from the Gen3 data commons,' specifying the verb (execute), resource (GraphQL query), and context (Gen3 data commons). It also distinguishes this tool from siblings by positioning it as the final execution step after validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends running validate_query first and outlines a workflow sequence, guiding when to use this tool. It does not specify when not to use it, but the context is clear enough for the agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_query_templateA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries the burden. It explains that the output is a template with useful fields and relationships, and mentions the output structure (data.template). However, it lacks details on error cases (e.g., invalid entity) or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It starts with the main purpose, then lists parameter descriptions, and ends with the workflow placement. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and no annotations, the description sufficiently covers purpose, parameters, return value, and context within the sibling workflow. It enables an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description provides clear explanations for all three parameters: entity_name with examples, include_relationships stating its purpose, and max_fields describing its control over template size. This adds value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it generates a ready-to-use GraphQL query template for any data type, with specific verb+resource. It explicitly places itself in a workflow sequence (get_schema_summary → get_schema_entity → here → validate_query → execute_graphql), distinguishing it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides the intended workflow and prerequisite (entity name from get_schema_summary). While it doesn't explicitly exclude alternatives, the workflow gives clear context on when to use this tool in the pipeline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schema_entityA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It declares a read operation with no destructive hints, but lacks details on limitations, authentication, or performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, front-loaded purpose, and the workflow list provides clear structure without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Provides workflow context and sibling relationship, but lacks details on the return format or any constraints. Adequate for its role in the pipeline.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'entity_name' is clearly described with its source (from get_schema_summary), adding meaning beyond the bare schema which has no description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get detailed information for a specific entity' and distinguishes from siblings by positioning it between get_schema_summary and generate_query_template.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow and instructs to use after get_schema_summary and before generate_query_template. However, no explicit when-not-to-use guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schema_summaryA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns an overview with entity names, relationships, and metadata, and that field details are omitted. It does not mention side effects or auth requirements, but for a read-only schema discovery tool, this is adequate. Slightly more could be said about potential performance or caching, but it's not critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using only three sentences plus a returns line and workflow. Every sentence serves a purpose: stating the action, clarifying scope, and providing workflow guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (0 parameters, no output schema), the description is complete. It explains the return value (overview with entities, relationships, metadata) and explicitly states what is omitted (field details). The workflow provides full context for integration with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100%. The description adds value by explaining what the output contains (entity names, relationships, metadata) and the workflow context. Since there are no parameters, the baseline is 4, and the description meets that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states that the tool provides an overview of the data model structure, including nodes, relationships, and metadata. It differentiates from sibling tools by mentioning that field details are omitted and that get_schema_entity should be used for specifics. The workflow clearly positions it as the starting point.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'This is your starting point!' and advises against using it for detailed field exploration, directing to get_schema_entity. It also includes a complete workflow sequence: start here, then use get_schema_entity, generate_query_template, validate_query, execute_graphql.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_queryA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses behavior: validates syntax and schema existence, returns detailed error messages and fix suggestions, and states valid query is safe to execute. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with paragraphs, bullet points, and a workflow. Every sentence adds value without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool with one param and no output schema, description covers purpose, usage, input format, and output description. Workflow contextualizes it among siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'query'; description adds 'The GraphQL query string to validate' beyond the schema's type string, clarifying the expected input. Schema coverage 0% but description compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates GraphQL queries, specifying syntax and schema existence checks. It distinguishes from siblings like execute_graphql and generate_query_template.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Always validate before executing' and provides a workflow showing its place before execute_graphql. Gives context on when to use (before execution) and implies not to use otherwise.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.