Server Quality Checklist

Tool Scores

• inspectC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Inspect this server' implies a read-only operation, but it does not specify what is inspected (e.g., configuration, status, logs), whether it requires authentication, any rate limits, or the format of results. The description adds minimal behavioral context beyond the implied action, failing to compensate for the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words: 'Inspect this server'. It is front-loaded and wastes no words, though this brevity contributes to its lack of detail. Every word serves a purpose, even if that purpose is minimal, making it structurally efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's name suggests potential complexity (e.g., server inspection could involve multiple aspects), the description is incomplete. With no annotations, no output schema, and a vague purpose, it fails to provide enough context for an agent to understand what the tool does, how to use it effectively, or what to expect in return. The high schema coverage for a trivial parameter does not compensate for these broader gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 100% description coverage, and the parameter '__ignore__' is documented as 'This does not do anything' with a default value. The description does not mention parameters, but since schema coverage is high and there is only one parameter that is non-functional, the baseline is elevated. The description does not add meaning beyond the schema, but the parameter's trivial nature justifies a score above the minimum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Inspect this server' is a tautology that essentially restates the tool name 'inspect' with minimal additional information. It specifies the resource ('this server') but lacks a clear, specific verb or action that distinguishes what 'inspect' entails compared to siblings like 'introspect' and 'query'. This makes the purpose vague and under-specified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'introspect' or 'query'. There is no mention of context, prerequisites, or exclusions, leaving the agent with no information to make an informed choice among sibling tools. This absence of usage guidelines is a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool queries a GraphQL endpoint but lacks critical details: authentication requirements, rate limits, error handling, response format, or whether it's read-only or mutative. This leaves significant gaps for an agent to understand operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It front-loads the core purpose ('Query a GraphQL endpoint') and succinctly specifies inputs, making it easy to parse and understand quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a GraphQL query tool with 2 parameters, 0% schema coverage, no annotations, and no output schema, the description is incomplete. It lacks details on authentication, error handling, response structure, and usage distinctions from siblings, leaving the agent under-informed for reliable tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'query' and 'variables' as inputs, aligning with the schema's two parameters, but adds no semantic context—such as what constitutes a valid GraphQL query, how variables should be formatted (e.g., JSON string), or examples. This minimal addition fails to adequately clarify parameter usage beyond naming them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Query') and target ('a GraphQL endpoint'), specifying the required inputs ('with the given query and variables'). It distinguishes from sibling tools 'inspect' and 'introspect' by focusing on execution rather than schema exploration. However, it doesn't explicitly contrast with those siblings, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools 'inspect' or 'introspect'. The description implies usage for executing GraphQL queries but offers no context about alternatives, prerequisites, or constraints, leaving the agent to infer appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• introspectA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions the tool's purpose and usage timing, it lacks details on behavioral traits such as whether this is a read-only operation, potential performance impacts, error conditions, or what the output format looks like (though there's no output schema). For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with essential information in a single, efficient sentence. It states the action ('Introspect the GraphQL schema'), the purpose ('to get the schema information'), and the usage context ('before doing a query... if you do not have it available'). Every word earns its place with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (introspecting a GraphQL schema), no annotations, no output schema, and 100% schema coverage for a single trivial parameter, the description is minimally adequate. It covers the basic purpose and usage context but lacks details on behavioral aspects, output format, or error handling that would be helpful for an AI agent. A score of 3 reflects this being the minimum viable description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 100% description coverage (the '__ignore__' parameter is documented as 'This does not do anything'). The tool description adds no parameter-specific information beyond what the schema provides. According to scoring rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Introspect the GraphQL schema' (verb+resource). It specifies this is to 'get the schema information' for use before querying. However, it doesn't explicitly differentiate from sibling tools like 'inspect' or 'query', which might have overlapping functionality in a GraphQL context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: 'use this tool before doing a query to get the schema information if you do not have it available as a resource already.' This gives a specific scenario (pre-query when schema info is unavailable). However, it doesn't mention when NOT to use it or explicitly compare to alternatives like the 'inspect' sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.