Server Quality Checklist

Tool Scores

• delete_observationsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool deletes observations, implying a destructive mutation, but doesn't cover critical aspects like permissions needed, whether deletions are permanent or reversible, error handling, or rate limits. This leaves significant gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a destructive tool with no annotations, 0% schema coverage, and no output schema, the description is insufficient. It doesn't explain what 'observations' are, how deletions affect the graph, or what the response looks like, leaving the agent poorly equipped to use this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning parameters are undocumented in the schema. The description adds no information about the 'deletions' parameter or its nested structure (entityName, observations), failing to compensate for the schema's lack of descriptions. This leaves the agent with no semantic understanding of the inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target ('specific observations from entities in the knowledge graph'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'delete_entities' or 'delete_relations', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'delete_entities' or 'delete_relations', nor does it mention prerequisites such as existing entities or observations. It lacks explicit usage context, leaving the agent to infer from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_entitiesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Create multiple new entities' which implies a write/mutation operation, but it doesn't disclose critical traits like whether this requires specific permissions, if it's idempotent, what happens on conflicts, or if there are rate limits. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's action without unnecessary words. It's front-loaded with the key verb and resource, making it easy to parse quickly. Every part of the sentence earns its place by conveying the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with 1 parameter (but nested objects), 0% schema description coverage, no annotations, and no output schema, the description is inadequate. It doesn't explain return values, error conditions, or behavioral nuances, leaving the agent with insufficient information to use the tool effectively in context with siblings like 'delete_entities' or 'read_graph'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning parameter details like 'entities', 'entityType', 'name', and 'observations' are undocumented in the schema. The description only mentions 'multiple new entities' without explaining what an entity consists of or how parameters map to the creation process. It adds minimal value beyond the schema, failing to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the resource 'multiple new entities in the knowledge graph', making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'create_relations' or 'add_observations', which would require more specificity about what constitutes an 'entity' versus other graph elements.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_relations' or 'add_observations'. It mentions 'multiple new entities' but doesn't clarify prerequisites, such as whether entities must be unique or if this is for batch creation versus single operations. Without context on alternatives or exclusions, usage is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_observationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but lacks behavioral details. It states the action ('Add') but doesn't disclose permissions needed, whether it's idempotent, rate limits, error handling, or what happens if entities don't exist. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (mutation tool with nested parameters), no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks details on behavior, error cases, return values, and usage context, making it insufficient for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but adds minimal param semantics. It implies 'observations' and 'entities' are involved but doesn't explain parameter structure, data formats, or constraints beyond what the schema defines. Baseline 3 applies as it hints at the general purpose without detailed compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Add') and resource ('observations to existing entities in the knowledge graph'), making the purpose understandable. It distinguishes from siblings like 'create_entities' (new entities) and 'delete_observations' (removal), but doesn't explicitly contrast with 'search_nodes' or 'read_graph' for observation retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., entities must exist), exclusions (e.g., cannot add to non-existent entities), or compare to siblings like 'create_entities' for new entities with observations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_relationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Create multiple new relations', implying a write/mutation operation, but doesn't disclose critical behaviors like whether this requires specific permissions, if relations are immutable after creation, what happens on duplicate relations, or any rate limits. The active voice note is minor and doesn't compensate for the lack of essential operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences that directly state the action and a constraint. It's front-loaded with the core purpose, and every word earns its place without redundancy. No extraneous information is included, making it efficient for quick parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a write operation with nested parameters), lack of annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't explain what 'entities' are, how relations are validated, what the tool returns, or error conditions. For a mutation tool in a knowledge graph context, this leaves significant gaps for an AI agent to operate safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The tool description mentions 'relations' and 'entities' but doesn't explain the parameter structure (e.g., that 'relations' is an array of objects with 'from', 'to', 'relationType'). It adds minimal meaning beyond the bare schema, failing to compensate for the coverage gap, but at least hints at the domain (knowledge graph).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('multiple new relations between entities in the knowledge graph'), making the purpose understandable. It distinguishes from siblings like 'create_entities' (which creates entities, not relations) and 'delete_relations' (which deletes, not creates). However, it doesn't specify what 'entities' or 'knowledge graph' refer to in this context, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_entities' or 'add_observations'. It mentions 'relations should be in active voice', which is a stylistic constraint but not a usage guideline. There are no explicit when/when-not instructions or prerequisites, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Delete' implies a destructive operation, the description doesn't specify whether deletions are permanent, reversible, require specific permissions, have confirmation steps, or affect system state beyond the stated entities and relations. This leaves significant behavioral gaps for a destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded with the core functionality, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is insufficiently complete. It doesn't address important contextual aspects like deletion permanence, error conditions, confirmation requirements, or what happens to orphaned relations. The combination of destructive operation with minimal behavioral disclosure creates significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'entityNames' clearly documented as 'An array of entity names to delete'. The description adds no additional parameter semantics beyond what the schema already provides, which is adequate but not additive given the comprehensive schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target ('multiple entities and their associated relations from the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like delete_observations or delete_relations, which handle different resource types in the same system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like delete_observations or delete_relations, nor does it mention prerequisites or conditions for use. It simply states what the tool does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_relationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Delete' implies a destructive mutation, the description doesn't address whether deletions are permanent, reversible, require specific permissions, have side effects on connected entities, or what happens if relations don't exist. For a destructive operation with zero annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a tool with one parameter and good schema documentation, with every word contributing essential information about the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after deletion (success/failure responses), whether the operation is atomic, or how it interacts with the knowledge graph structure. Given the complexity of graph operations and lack of structured safety information, more context about behavioral expectations is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single 'relations' parameter well-documented in the schema as an array of objects with 'from', 'to', and 'relationType' properties. The description adds no additional parameter semantics beyond what's already in the schema, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target resource ('multiple relations from the knowledge graph'), making the purpose immediately understandable. However, it doesn't explicitly distinguish this tool from sibling tools like 'delete_entities' or 'delete_observations', which would require specifying that this tool specifically removes relationship connections rather than entities or observations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'delete_entities' or 'create_relations'. It doesn't mention prerequisites, consequences, or typical scenarios for deleting relations versus other deletion operations, leaving the agent without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_nodesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Open' implies a read operation, but the description doesn't specify what 'open' entails (e.g., retrieving node details, expanding in UI, or returning metadata), whether it's idempotent, has rate limits, or requires permissions. For a tool with no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for the tool's apparent simplicity. Every word earns its place without being overly terse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a single parameter with good schema coverage, the description is incomplete. It doesn't explain what 'open' means operationally, what data is returned, or how it differs from sibling read tools. For a knowledge graph tool with multiple related siblings, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'names' well-documented in the schema as 'An array of entity names to retrieve'. The description adds minimal value beyond this, mentioning 'by their names' which echoes the schema. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('open') and target resource ('specific nodes in the knowledge graph by their names'), which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'read_graph' or 'search_nodes', which appear to be related read operations. The purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'read_graph' and 'search_nodes' available, there's no indication of whether this is for retrieving specific known nodes versus broader queries. No prerequisites, exclusions, or alternative recommendations are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_graphC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action ('Read') without disclosing behavioral traits like permissions needed, rate limits, pagination, or what 'entire' entails (e.g., size limits, performance impact). It's minimal and leaves critical operational details unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded and appropriately sized for a simple tool, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that presumably returns complex graph data. It lacks context on output format, size, or structure, which is essential for an agent to use it effectively, especially compared to siblings with more specific functions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate, earning a baseline score of 4 for adequately handling the parameter-free case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Read the entire knowledge graph' states a clear verb ('Read') and resource ('knowledge graph'), but lacks specificity about what 'entire' means and doesn't differentiate from sibling tools like 'search_nodes' or 'open_nodes'. It's vague about scope and boundaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_nodes' or 'open_nodes'. The description implies a broad read operation but doesn't specify use cases, prerequisites, or exclusions compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nodesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool searches nodes but doesn't describe behavioral traits such as search scope (e.g., partial vs. exact matches), performance characteristics (e.g., speed, limitations), or output format (e.g., list of nodes, pagination). This is a significant gap for a search tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search operation, lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the search returns (e.g., node details, IDs), how results are structured, or any limitations (e.g., result count, sorting). For a tool with no structured output information, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'query' parameter documented as matching 'entity names, types, and observation content.' The description adds no additional parameter semantics beyond this, as it only repeats 'based on a query.' With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for nodes in the knowledge graph based on a query.' It specifies the verb ('search'), resource ('nodes in the knowledge graph'), and mechanism ('based on a query'). However, it doesn't explicitly differentiate from sibling tools like 'open_nodes' or 'read_graph,' which might also retrieve node information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'open_nodes' or 'read_graph,' nor does it specify use cases, prerequisites, or exclusions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.