Server Quality Checklist

Tool Scores

• relationshipsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description must carry full burden. It only discloses that circular CONTAINS dependencies are prevented, but doesn't explain side effects of delete/create, permission requirements, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no filler. Front-loaded purpose, then supported operations, then key behavioral safeguard.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values are covered. But the description doesn't clarify conditional parameter usage based on operation, despite the schema providing some hints. Could be improved with examples or when-to-use conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description doesn't need to add much. It does mention relationship types which align with relationship_type parameter, but adds no operational nuance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool manages relationships between nodes with specific relationship types (CONTAINS, RELATE_TO) and enumerates operations (create, delete, list). This distinguishes it from sibling tools like nodes or query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. It does not mention exclusions, prerequisites, or alternative tools for similar operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• envA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It reveals that variables matching secret patterns are automatically hidden from LLM responses and that files are stored at a specific path. These are meaningful behavioral details that a user would need to know, though it does not mention side effects like deletion permanence.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short paragraphs, front-loaded with the core purpose. Every sentence provides useful information: the main function, secret detection behavior, and storage location. There is no redundant phrasing or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are already covered. The description supplies important context about secret handling and file storage. It does not address error cases or operation-specific effects, but for a tool of this complexity (6 params, 5 operations) it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are described in the schema (100% coverage), so the baseline is 3. The description adds little direct parameter context, but it does explain the secret detection patterns that relate to variable keys. However, it does not elaborate on operation-specific parameter usage beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'Manage environment variables', clearly identifying the resource and function. It distinguishes from sibling tools (nodes, relationships, execute, query) by focusing on environment variables. However, 'manage' is somewhat generic and does not enumerate the specific CRUD operations, though the schema's operation parameter clarifies this.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions. The description only states what it does, not when it should be preferred over other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nodesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses important behaviors: __main__ blocks are automatically stripped from SCRIPT nodes, ENV variables are auto-loaded via CONTAINS relationships, and data parameters accept dicts or JSON strings. These specifics go beyond a generic CRUD description, though it does not cover all edge behaviors (e.g., update merge/replace semantics).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (overview, args, returns, best practices, examples) and front-loaded purpose. Although lengthy, the SCRIPT best practices and examples add significant value and avoid redundancy, earning their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and complex SCRIPT-node handling, the description is nearly complete. It includes all parameter semantics, best practices, and examples for create and list. Since an output schema exists, the vague 'Returns: Operation result' is acceptable, but missing examples for read/update/delete and potential update semantics are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate—and it does thoroughly. It explains each parameter's purpose, enumerates allowed values for operation and node_type, specifies which parameters apply to which operations, and clarifies that data/filters can be dicts or JSON strings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Manage graph nodes' and explicitly lists the node types (SKILL, KNOWLEDGE, SCRIPT, ENV) and supported operations (create, read, update, delete, list). This makes the tool's purpose unmistakable and distinct from siblings like 'relationships' or 'env'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides rich operational details (Args, SCRIPT best practices, examples) but never explicitly states when to choose this tool over alternatives. The usage context is implied by the tool name and node-type scope, but there is no direct when/when-not guidance or mention of sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the read-only nature, the exact set of allowed operations, and a key safety feature: 'Results are automatically sanitized to hide secret values.' This goes beyond the schema and gives critical behavioral context, though it does not address error handling, pagination, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the purpose, the second lists constraints. Every sentence adds value, and it is front-loaded with the core action. No waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is quite complete given the rich schema (full parameter coverage), an output schema (handles return values), and its own behavioral notes. It covers the essential safety and constraint aspects, but could have explicitly mentioned that this is a graph exploration tool versus node/relationship access, though that is implied by 'knowledge graph' and sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all parameters with 100% description coverage, so the baseline is 3. The description adds a little extra meaning to the 'cypher' parameter by enumerating allowed operations, but it does not explain other parameters (limit, parameters) beyond what the schema already says.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Execute read-only Cypher queries to explore the knowledge graph.' It uses a specific verb (execute), a resource (Cypher queries), and a domain (knowledge graph), while the read-only qualifier distinguishes it from sibling 'execute' or 'nodes' tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool (to explore the knowledge graph through Cypher) and explicitly restricts allowed operations ('Only MATCH, RETURN, WITH, WHERE, ORDER BY operations are allowed'), which helps the agent avoid invalid or write queries. However, it does not mention any alternatives or exclusions relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• executeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full responsibility. It discloses several behavioral traits: automatic loading of SCRIPT nodes, merging PEP 723 dependencies, loading ENV variables, executing via 'uv run', and sanitizing output to remove secrets. This gives a good sense of what happens during execution, though it doesn't address side effects or security boundaries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact with a clear main line and bullet-pointed automatic behaviors. Every sentence adds value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a code executor with multiple automatic steps. The description covers the key aspects of loading, dependency merging, env handling, execution method, and output sanitization. With an output schema present, the lack of return-value detail is acceptable. However, it could mention error behavior or access to imported names for added completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, so baseline is 3. The description adds meaning by explaining that 'imports' are SCRIPT nodes with PEP 723 dependency merging, and that 'envs' are directly specified ENV nodes in addition to connected ones, enriching the schema's brief parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as executing Python code and importing functions from SCRIPT nodes, using a specific verb ('Execute') and resource, distinguishing it from sibling tools like 'query' or 'env'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for using this tool (when you need to run Python code with SCRIPT node imports), though it doesn't explicitly state when to avoid it or mention alternatives. This fits 'clear context, no exclusions'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.