Server Quality Checklist

Tool Scores

• graphql_mutateA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the mutating nature and the server-side permission requirement, which are essential. However, it does not describe side effects, idempotency, or other behavioral details beyond the write nature and the environment variable condition.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, front-loaded with the core purpose, and includes a useful example. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, return values need no explanation. The description provides the key permission context and clarifies the write nature, but it omits guidance on error handling, authentication specifics beyond the env var, and explicit differentiation from sibling tools. It is minimally adequate for a simple two-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the 'mutation' or 'variables' parameters beyond an example. The parameter names are self-explanatory, but the description adds no explicit semantic detail about expected formats, required variables, or how variables map to the mutation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute a GraphQL mutation (write operation)', using a specific verb and resource, and explicitly labels it as a write operation, which distinguishes it from sibling tools like graphql_query. The example mutation reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It indicates when the tool is permitted (server must have GRAPHMCP_ALLOW_MUTATIONS=1) and labels it as a write operation, implying it should be used for mutations rather than queries. However, it does not explicitly name alternatives like 'use graphql_query for read operations'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphql_schemaA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states the tool returns the schema, implying a read-only action, but doesn't disclose any additional behavioral details such as authentication requirements or side effects. Adequate for the simple operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core action and a usage hint. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (no parameters), output schema exists, and the description provides both the purpose and a usage trigger. Complete for its scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and with schema coverage at 100% (vacuously), the description doesn't need to explain parameters. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and identifies the exact resource ('the GraphQL schema (SDL)'), clearly distinguishing it from sibling tools that validate, query, or mutate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to 'Call this before writing queries,' providing clear contextual guidance for when to use the tool, though it doesn't explicitly state exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphql_validateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It honestly states that the query is not executed and provides the return structure (valid, errors, depth), plus a hint about low cost. While it does not detail error semantics or auth requirements, the essential behavior is transparent for a validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately front-loaded with the core purpose, followed by return format and usage guidance. No extraneous words; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and an output schema, the description is quite complete: it explains the operation, non-execution, return type, and when to use it in relation to a sibling. It does not allude to potential limitations (e.g., query size or schema dependence), but these are not critical for basic validation usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It identifies the parameter as 'a GraphQL query', giving meaning beyond the generic schema title 'Query', but it does not provide format examples, constraints, or additional details. This is moderate compensation for a single simple parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('validate') and resource ('GraphQL query') while explicitly noting it does not execute the query. This distinguishes it from sibling tools like graphql_query and graphql_mutate, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this to catch typos and invalid fields cheaply before calling graphql_query.' It names an alternative tool and specifies the exact scenario (pre-execution validation), fulfilling the when-to-use criterion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphql_queryA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses critical behavioral details: it is read-only, returns both data and errors, and explicitly rejects mutations. It also gives a concrete example of query and variables, adding useful context. It could mention auth requirements or rate limits, but for a simple read-only query tool, this is above average.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence purpose, a clear guidance to select only needed fields, and a single example that covers both parameters. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no annotations, the description provides all necessary context for an agent to invoke the tool correctly: purpose, read-only guarantee, mutation rejection, parameter format, and return shape. The sibling tools are distinguishable, and the output schema is mentioned via the return {'data', 'errors'}. It is complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage, but the description fully compensates by providing a complete example showing both the 'query' string and 'variables' object with a role variable. This demonstrates the exact format for both parameters and their relationship, adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute a read-only GraphQL query' with a specific verb and resource, and explicitly distinguishes it from the sibling graphql_mutate by noting 'Mutations are rejected here.' This leaves no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly indicates read-only usage and rejects mutations, which tells the agent when not to use this tool. However, it does not explicitly name alternatives like graphql_schema or graphql_validate, though the 'read-only' and 'mutations rejected' framing implies the correct context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.