Server Quality Checklist

Tool Scores

• run_opencypher_queryC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the basic action without details on permissions, rate limits, error handling, or what 'executes' entails (e.g., read-only vs. mutating queries). This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It is appropriately sized and front-loaded, clearly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a query execution tool with no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks details on return values, error cases, and behavioral traits, making it inadequate for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'provided openCypher' which hints at the 'query' parameter but does not explain the 'parameters' parameter or provide any syntax, format, or examples. This adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'Executes the provided openCypher against the graph,' which clearly identifies the action (executes) and resource (openCypher query on a graph). However, it lacks specificity about what openCypher is or how it differs from sibling tools like run_gremlin_query, making it somewhat vague rather than fully distinguishing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools like run_gremlin_query for different query languages or get_graph_schema for schema inspection, leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_gremlin_queryC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool executes a query but doesn't describe what that entails—e.g., whether it's read-only or mutative, if it requires specific permissions, potential side effects, or response format. This leaves significant gaps for a query execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for the tool's complexity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, and no output schema, the description is incomplete. It lacks details on behavioral traits, parameter usage, and expected outputs, which are crucial for a query execution tool with potential complexity and side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description mentions 'the provided Tinkerpop Gremlin' which hints at the 'query' parameter's content, but it doesn't explain the query format, syntax, or any constraints, offering minimal semantic value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Executes') and the resource ('Tinkerpop Gremlin against the graph'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'run_opencypher_query' beyond mentioning Gremlin specifically, which is implied but not contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'run_opencypher_query' for OpenCypher queries or when to prefer one query language over another, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_graph_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a read operation ('Get'), implying it's non-destructive, but doesn't mention authentication requirements, rate limits, error conditions, or what 'status' entails (e.g., health metrics, configuration details). The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action ('Get the status'), making it immediately clear. Every word earns its place, and there's no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what 'status' includes (e.g., uptime, version, connectivity) or the return format, leaving the agent guessing. For a status-checking tool, this lacks necessary operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100% (empty schema). The description doesn't need to explain parameters, and it correctly implies no inputs are required by focusing on the 'currently configured' graph. This meets the baseline of 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('status of the currently configured Amazon Neptune graph'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get_graph_schema' or query tools, but the focus on 'status' rather than schema or query execution provides implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_graph_schema' or the query tools. It doesn't mention prerequisites (e.g., whether a graph must be configured first) or typical use cases (e.g., health checks, monitoring). The agent receives no explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_graph_schemaB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (schema information) but lacks details on permissions, rate limits, error handling, or response format. For a tool with zero annotation coverage, this is insufficient, scoring a 2 due to missing critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, using two sentences that efficiently convey the tool's purpose. It front-loads the main action ('Get the schema') and adds specific details without waste, making it easy to parse. Every sentence earns its place, justifying a score of 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is adequate but has clear gaps. It explains what the tool retrieves but lacks output details or behavioral context, which is needed for full completeness. This meets the minimum viable standard, scoring a 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately focuses on the tool's purpose without redundant parameter details. This aligns with the baseline of 4 for zero-parameter tools, as it adds value by explaining what the tool does without unnecessary repetition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get the schema for the graph') and resources ('vertex and edge labels', 'combinations'), making it easy to understand what it does. It doesn't explicitly distinguish from sibling tools like 'get_graph_status' or query tools, but the focus on schema retrieval is distinct enough for a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_graph_status' for status checks or query tools for data retrieval, leaving the agent without context for tool selection. This lack of comparative usage information results in a score of 2.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.