Server Quality Checklist

Tool Scores

• searchSubgraphsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool returns 'a concise list of matching subgraphs with their IDs, names, networks, and signal, or an error message,' which gives some output context. However, it lacks details on rate limits, authentication needs, pagination, error conditions beyond generic 'error message,' or whether it's read-only (implied but not stated). For a search tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by Args and Returns sections. It uses clear, efficient language with no redundant information. However, the Returns section could be more concise by integrating details into the main description, and some sentences (e.g., 'or an error message') are slightly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and output format but lacks depth in usage guidelines, behavioral details, and parameter semantics. For a search tool interacting with a network, more context on limitations or typical use cases would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantic value beyond the input schema. It states 'searchQuery (str): The search term to find matching subgraphs,' which clarifies the parameter's purpose but doesn't provide format examples, constraints (e.g., length, special characters), or search behavior (e.g., partial matches, case sensitivity). With 0% schema description coverage and only one parameter, this meets the baseline but doesn't compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for subgraphs on The Graph Network by name or description.' It specifies the action (search), resource (subgraphs), and scope (The Graph Network). However, it doesn't explicitly differentiate from sibling tools like 'getSubgraphSchema' or 'querySubgraph' beyond the search focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions searching by name or description but doesn't specify scenarios where this is preferred over sibling tools like 'getSubgraphSchema' (likely for schema retrieval) or 'querySubgraph' (likely for querying data). No prerequisites, exclusions, or comparative context are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getSubgraphSchemaB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool fetches schema via GraphQL introspection and returns data in JSON or text format, but doesn't cover critical aspects like error handling (e.g., what happens if the subgraphId is invalid), authentication needs, rate limits, or whether it's a read-only operation. This leaves significant gaps for an agent to understand its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear sections for arguments and returns. Every sentence adds value: the first explains the tool's function, and the subsequent lines detail parameters and output without redundancy. It's appropriately sized for a tool with two parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose and parameters effectively, but lacks details on behavioral aspects like error conditions or usage context. Without annotations or output schema, it doesn't fully equip an agent for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains that 'subgraphId' identifies the subgraph to query and 'asText' controls the output format (JSON vs. GraphQL text), clarifying the purpose and effect of each parameter. This compensates well for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetch the schema of a specified subgraph using GraphQL introspection.' It specifies the verb ('fetch'), resource ('schema of a specified subgraph'), and method ('GraphQL introspection'), but doesn't explicitly differentiate from sibling tools like 'querySubgraph' or 'searchSubgraphs' beyond the schema focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'querySubgraph' or 'searchSubgraphs'. It lacks context about prerequisites, such as whether the subgraph must exist or be accessible, and doesn't mention any exclusions or specific scenarios for its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• querySubgraphA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool returns 'Query result in JSON format, or an error message,' which adds some context about output behavior. However, it lacks details on authentication needs, rate limits, performance characteristics, or what happens on invalid inputs, which are important for a mutation-like query execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the main purpose in the first sentence. The Args and Returns sections are structured clearly, but the formatting with markdown-like sections could be slightly more concise if integrated into a single paragraph.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing GraphQL queries, no annotations, no output schema, and 2 parameters, the description is moderately complete. It covers the basic purpose and parameters but lacks details on error types, response structure beyond 'JSON format,' and integration with sibling tools, leaving gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by explaining that 'subgraphId' is 'The ID of the subgraph to query' and 'query' is 'The GraphQL query string to execute.' With 0% schema description coverage, this compensates well by clarifying the purpose of each parameter, though it doesn't specify format constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute a GraphQL query against a specified subgraph.' This includes a specific verb ('Execute'), resource ('GraphQL query'), and target ('specified subgraph'), which distinguishes it from sibling tools like getSubgraphSchema (which retrieves schema) and searchSubgraphs (which searches for subgraphs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying it's for executing GraphQL queries, but it doesn't explicitly state when to use this tool versus alternatives like getSubgraphSchema or searchSubgraphs. There's no guidance on prerequisites, error handling, or specific contexts where this tool is preferred over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.