Server Quality Checklist

Tool Scores

• create_relationsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a creation operation, implying mutation, but does not cover critical aspects like permissions needed, whether it's idempotent, error handling, or what happens on success/failure. The 'active voice' note is trivial and does not add meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, and the first sentence front-loads the core purpose effectively. The second sentence ('Relations should be in active voice') is arguably unnecessary but does not significantly detract from clarity. Overall, it is well-structured and avoids verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with no annotations, 0% schema description coverage, and no output schema, the description is incomplete. It lacks details on behavioral traits, parameter meanings, return values, and error conditions. For a tool that creates multiple relations, this level of documentation is inadequate for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the input schema provides no parameter descriptions. The tool description mentions 'relations' but does not explain the structure or semantics of the 'relations' array (e.g., what 'from', 'to', 'relationType' mean beyond the schema's basic types). It adds minimal value over the bare schema, insufficient to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create multiple new relations') and the resource ('between entities in the knowledge graph'), which is specific and informative. However, it does not explicitly differentiate this tool from its sibling 'create_entities' or 'delete_relations', which would be needed for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_entities' or 'delete_relations'. It mentions 'relations should be in active voice', which is a stylistic note but not a usage guideline. Without explicit when/when-not instructions or named alternatives, this falls to a minimal score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_observationsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Add' implies a write/mutation operation, it doesn't specify permissions needed, whether changes are reversible, rate limits, or what happens if entities don't exist. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with one main parameter and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, 0% schema description coverage, and no output schema, the description is inadequate. It doesn't explain what constitutes valid observations, how they're stored, what the response looks like, or error conditions. More context is needed given the complexity of modifying a knowledge graph.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter documentation. The description mentions 'observations' and 'entities' but doesn't explain the structure, format, or constraints of the 'observations' array parameter. It adds minimal semantic context beyond what's implied by the tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add new observations') and target ('to existing entities in the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'create_entities' or 'delete_observations', which would require more specific scope definition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., entities must already exist), exclusions, or comparisons to sibling tools like 'create_entities' (for new entities) or 'delete_observations' (for removal).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is a creation operation, implying mutation, but lacks details on permissions, idempotency, error handling, or rate limits. For a batch creation tool with zero annotation coverage, this is insufficient to inform the agent about critical behavioral traits beyond the basic 'create' intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Create multiple new entities') and specifies the context ('in the knowledge graph'). There is no wasted verbiage, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a batch creation tool with no annotations, no output schema, and low schema description coverage (0%), the description is incomplete. It fails to address behavioral aspects like side effects, return values, or error conditions, and doesn't clarify usage relative to siblings, leaving significant gaps for the agent to operate effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'multiple new entities', which aligns with the 'entities' array parameter in the schema, but adds no semantic details beyond what the schema implicitly suggests. With 0% schema description coverage, the schema lacks descriptions for the 'entities' parameter itself, but the description doesn't compensate by explaining what entities consist of or how they're structured, leaving the agent reliant on the nested schema properties alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'create' and the resource 'entities in the knowledge graph', with the additional detail 'multiple new' indicating batch creation. However, it doesn't explicitly differentiate from sibling tools like 'create_relations' or 'add_observations', which would require more specific language about what entities are versus relations or observations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether entities must be unique), exclusions (e.g., when to use 'add_observations' instead for existing entities), or comparisons to siblings like 'create_relations' for different resource types, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Delete' clearly indicates a destructive operation, the description doesn't specify whether this deletion is permanent or reversible, what permissions are required, whether there are rate limits, what happens to orphaned data, or what confirmation/response to expect. For a destructive tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized for a tool with one parameter, though it could potentially be more front-loaded with critical behavioral information given the destructive nature of the operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is insufficiently complete. It doesn't address critical context like whether deletions are permanent, what permissions are needed, what happens to associated data, or what the tool returns. The mention of 'associated relations' being deleted is helpful but doesn't compensate for the broader behavioral gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'entityNames' clearly documented as 'An array of entity names to delete'. The description adds that these are 'multiple entities' and mentions 'associated relations' will also be deleted, providing some additional context about side effects. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete multiple entities and their associated relations') and the resource ('from the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly distinguish this tool from sibling tools like 'delete_observations' or 'delete_relations', which handle different resource types in the same knowledge graph system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this deletion operation is appropriate, what prerequisites might exist, or how it differs from other deletion tools like 'delete_observations' or 'delete_relations' that operate on different graph components.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_observationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Delete' implies a destructive mutation, it doesn't specify whether this operation is reversible, requires specific permissions, has side effects on related data, or provides confirmation feedback. For a destructive tool with zero annotation coverage, this is a significant gap in safety and operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and target, making it easy to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations, no output schema, and low schema description coverage, the description is inadequate. It lacks critical information about behavioral traits, error handling, return values, and differentiation from siblings, leaving the agent under-informed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'specific observations' and 'entities', which aligns with the 'deletions' parameter structure in the schema. However, with 0% schema description coverage, the description doesn't add meaningful details about parameter format, constraints, or examples beyond what's minimally implied. It compensates slightly but not fully for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target ('specific observations from entities in the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'delete_entities' or 'delete_relations', which would require more specific language about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'delete_entities' or 'delete_relations'. It lacks any context about prerequisites, appropriate scenarios, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_relationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but lacks critical behavioral details. It implies a destructive operation ('Delete') but doesn't specify if deletions are permanent, require permissions, have side effects, or provide confirmation. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, clearly front-loading the core action. It's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens upon deletion, error conditions, or return values, leaving significant gaps in understanding the tool's behavior and outcomes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'relations' parameter and its nested properties. The description adds no additional meaning beyond stating it deletes 'multiple relations', which is implied by the schema. Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('multiple relations from the knowledge graph'), making the purpose understandable. It distinguishes from siblings like 'delete_entities' and 'delete_observations' by specifying relations, though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'delete_entities' or 'create_relations', nor does it mention prerequisites such as existing relations. The description only states what it does without context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_nodesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action ('open') but doesn't disclose behavioral traits such as whether this is a read-only operation, if it requires specific permissions, what happens if nodes don't exist (e.g., errors or empty results), or the format of returned data. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and resource. It wastes no words and is appropriately sized for a simple retrieval tool, with every part contributing to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a retrieval tool with no annotations and no output schema), the description is incomplete. It doesn't explain what 'open' entails (e.g., returns node details, errors on missing nodes), the return format, or behavioral constraints. For a tool interacting with a knowledge graph, more context on outcomes and limitations is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'names' fully documented in the schema as 'An array of entity names to retrieve'. The description adds minimal value beyond this, only reiterating 'by their names' without providing additional context like name format, case sensitivity, or handling of duplicates. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('open') and target resource ('nodes in the knowledge graph'), with specificity about the mechanism ('by their names'). It distinguishes from siblings like 'read_graph' (general reading) and 'search_nodes' (search-based retrieval), though it doesn't explicitly name these alternatives. The purpose is unambiguous but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., nodes must exist), exclusions (e.g., not for creating or deleting nodes), or direct comparisons to siblings like 'read_graph' or 'search_nodes'. The description implies usage for retrieval by name but offers no contextual boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_graphC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'Read' but doesn't disclose behavioral traits such as whether this is a safe operation, potential rate limits, authentication needs, or what format the output takes (e.g., JSON, size limits). This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded and efficiently conveys the core action, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity implied by 'knowledge graph' and lack of annotations or output schema, the description is incomplete. It doesn't explain what 'read' returns (e.g., structure, size), how it interacts with other tools, or any prerequisites. This is inadequate for a tool that might handle large or complex data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. A baseline of 4 is applied as it adequately handles the lack of parameters without introducing confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Read') and target ('the entire knowledge graph'), but it's vague about what 'read' entails—does it return all nodes, edges, metadata, or something else? It doesn't distinguish from siblings like 'search_nodes' or 'open_nodes', which might also involve reading graph data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For example, it doesn't explain if this is for bulk retrieval versus targeted searches (like 'search_nodes') or if it has performance implications. The description lacks context for choosing among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nodesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. While 'search' implies a read operation, it doesn't specify whether this is paginated, rate-limited, permission-dependent, or what format results take. For a search tool with zero annotation coverage, this leaves significant behavioral gaps unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core functionality without unnecessary elaboration. Every word earns its place, and the structure is front-loaded with the essential information. No wasted verbiage or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what constitutes a 'node', what fields are searched, how results are ranked, what the return format looks like, or any limitations. Given the complexity of knowledge graph search and the lack of structured metadata, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'query' parameter thoroughly. The description adds no additional parameter semantics beyond what's in the schema - it mentions the query but doesn't elaborate on syntax, examples, or search behavior. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('search for nodes') and the target ('knowledge graph'), providing a specific verb+resource combination. However, it doesn't distinguish this tool from potential sibling alternatives like 'open_nodes' or 'read_graph', which might also retrieve node information but through different mechanisms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'open_nodes' and 'read_graph' that might also access node data, there's no indication of when search-based retrieval is preferred over direct access or full graph reading. No prerequisites, exclusions, or comparative context are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.