Server Quality Checklist

Tool Scores

• add_observationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an 'add' operation (implying mutation) but doesn't cover critical aspects like required permissions, whether changes are reversible, rate limits, or what happens on success/failure. For a mutation tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for the tool's apparent complexity and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, no output schema, and 0% schema description coverage, the description is inadequate. It doesn't explain what 'observations' are in this context, how they're structured, what happens when added to non-existent entities, or what the tool returns. More context is needed given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'observations' and 'entities' which aligns with the single parameter 'observations' in the schema, but provides no additional semantic context beyond what's implied by the parameter name. With 0% schema description coverage, the description doesn't compensate by explaining what constitutes valid observations or entities, maintaining a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add new observations') and target ('to existing entities in the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'create_entities' or 'delete_observations', which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_entities' (for new entities) or 'delete_observations'. It mentions 'existing entities' which implies a prerequisite, but offers no explicit when/when-not instructions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates entities, implying a write operation, but doesn't cover permissions, idempotency, error handling, or what happens on partial failures (e.g., if some entities fail). This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with 1 parameter (an array of nested objects), no annotations, and no output schema, the description is insufficient. It doesn't explain return values, error cases, or behavioral nuances, leaving the agent with inadequate information for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'multiple new entities' which hints at the 'entities' array parameter, but schema description coverage is 0%, meaning all parameter details are undocumented. The description adds minimal value beyond the schema's structure, so it meets the baseline for moderate coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create multiple new entities') and the target resource ('in the knowledge graph'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'create_relations' or 'add_observations', which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'add_observations' or 'create_relations'. It lacks context about prerequisites, such as whether entities must exist before adding observations or relations, or when batch creation is preferred over individual operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_relationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a creation operation ('Create multiple new relations'), implying mutation, but fails to address critical aspects like permissions needed, whether changes are reversible, error handling, or rate limits. The active voice note is trivial and doesn't compensate for the lack of substantive behavioral information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and to the point with two sentences, avoiding unnecessary verbosity. However, the second sentence ('Relations should be in active voice') feels somewhat misplaced and doesn't contribute significantly to tool understanding, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with no annotations, 0% schema description coverage, and no output schema, the description is insufficient. It doesn't explain return values, error conditions, or provide enough context to use the tool effectively, leaving significant gaps in understanding for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate, but it only vaguely references 'relations' without explaining the structure or semantics of the 'relations' array parameter. It adds minimal value beyond the schema, which already defines the properties ('from', 'to', 'relationType'), resulting in a baseline score due to inadequate parameter clarification.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create multiple new relations') and the resource ('between entities in the knowledge graph'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'create_entities' or 'delete_relations', which would require more specific context about when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance with the phrase 'Relations should be in active voice', which is a stylistic hint rather than usage advice. It lacks explicit when-to-use instructions, prerequisites, or alternatives compared to siblings like 'create_entities' or 'delete_relations', leaving the agent without clear contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that associated relations are also deleted, which adds some context beyond the basic 'delete' action. However, it doesn't address critical aspects like whether the deletion is permanent/irreversible, permission requirements, error handling, or what happens if entities don't exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized for a tool with one parameter and no annotations, with every word earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive operation with no annotations and no output schema, the description is insufficient. It doesn't explain what 'associated relations' means in practice, what confirmation or feedback the tool provides, or the implications of bulk deletion. Given the complexity and risk of a delete operation, more behavioral context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'entityNames' clearly documented as 'An array of entity names to delete'. The description doesn't add any additional parameter semantics beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target resources ('multiple entities and their associated relations from the knowledge graph'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like delete_observations or delete_relations, which handle different resource types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like delete_observations or delete_relations, nor does it mention prerequisites, dependencies, or scenarios where this tool is appropriate. It simply states what the tool does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_observationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Delete' implies a destructive mutation, the description doesn't specify whether deletions are permanent, reversible, require specific permissions, or have side effects on related data. For a destructive operation with zero annotation coverage, this represents a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized for a tool with one primary parameter and gets straight to the point with zero wasted language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations, no output schema, and 0% schema description coverage, the description is insufficiently complete. It doesn't address critical context like what happens after deletion, error conditions, permissions required, or how this differs from other deletion tools. The agent would need to guess about important behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the tool has only 1 parameter ('deletions') which is an array of objects. The description mentions 'specific observations from entities' which aligns with the parameter structure, but doesn't provide additional semantic context about what constitutes valid observations, deletion constraints, or format requirements beyond what's minimally implied.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and target ('specific observations from entities in the knowledge graph'), providing a specific verb+resource combination. However, it doesn't explicitly distinguish this tool from sibling tools like 'delete_entities' or 'delete_relations', which would require more specific differentiation about what types of deletions each performs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'delete_entities' or 'delete_relations'. There's no mention of prerequisites, constraints, or specific scenarios where this deletion method is appropriate versus other deletion tools in the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_relationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action is 'Delete' (implying mutation/destruction) but lacks critical details: it doesn't specify if deletions are permanent, require specific permissions, have side effects (e.g., cascading deletions), or include error handling. This is a significant gap for a destructive tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately scannable. Every word earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens post-deletion (e.g., confirmation, error messages, or return values), behavioral constraints, or integration with sibling tools. The agent lacks sufficient context to use this tool safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'relations' parameter and its nested structure (from, to, relationType). The description adds no additional parameter semantics beyond implying multiple deletions via 'multiple relations'. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('multiple relations from the knowledge graph'), making the purpose immediately understandable. It distinguishes from sibling tools like delete_entities and delete_observations by specifying the target resource type. However, it doesn't specify the exact scope or mechanism (e.g., batch deletion vs. individual), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., existing relations), exclusions (e.g., when not to delete), or comparisons with siblings like delete_entities or create_relations. The agent must infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_graphC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose performance implications (e.g., large data returns), rate limits, authentication needs, or what 'entire' entails (e.g., all nodes/relations). The description is minimal and lacks critical context for a read operation on a knowledge graph.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for a simple tool, making it easy for an agent to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and the complexity of reading an entire knowledge graph, the description is incomplete. It doesn't explain return values, data format, pagination, or scope limitations, leaving significant gaps for the agent to infer behavior in a potentially data-intensive context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. A baseline of 4 is applied as it correctly avoids redundancy, though it doesn't compensate for any gaps (none exist).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Read the entire knowledge graph' clearly states the action (read) and resource (knowledge graph), but lacks specificity about what 'entire' means compared to sibling tools like 'search_nodes' or 'open_nodes'. It distinguishes from obvious write/delete siblings but not from other read-like tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_nodes' or 'open_nodes'. It implies usage for reading all graph data but doesn't specify scenarios, prerequisites, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_nodesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the behavior of returning 'complete node information including subdomain and all metadata', which adds value beyond the input schema. However, it doesn't cover error handling (e.g., invalid names), performance aspects (e.g., rate limits), or mutation effects (implied read-only but not stated). The description is adequate but misses key operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the action and input, the second specifies the output. Every word earns its place with no redundancy or fluff. It's front-loaded with the core purpose and efficiently structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 1 parameter with 100% schema coverage and no output schema, the description is moderately complete. It covers purpose and output but lacks details on errors, permissions, or performance. For a read operation with no annotations, it should ideally mention safety (non-destructive) or limitations, but it's adequate for basic use. The absence of an output schema increases the burden slightly, which isn't fully met.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the 'names' parameter fully documented in the schema as 'An array of entity names to retrieve, returning full entity information including subdomain'. The description adds marginal value by reinforcing 'by their names' and linking to the output, but doesn't provide additional syntax, constraints, or examples beyond the schema. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Open specific nodes in the knowledge graph by their names' (verb+resource) and 'Returns the complete node information including subdomain and all metadata' (outcome). It distinguishes from siblings like 'search_nodes' (search vs. open by name) and 'read_graph' (read entire graph vs. specific nodes), though not explicitly named. However, it doesn't fully differentiate from 'search_nodes' in edge cases where names are known.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you know exact node names ('by their names') and want complete information. It doesn't explicitly state when to use alternatives like 'search_nodes' (for fuzzy/partial matches) or 'read_graph' (for entire graph), nor does it mention prerequisites or exclusions. The context is clear but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_nodesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the search covers multiple fields (entity names, types, subdomains, observation content), uses OR logic for multiple keywords, and matches keywords anywhere in fields. However, it lacks details on response format, pagination, or error handling, which are important for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey purpose, scope, and behavior without redundancy. Every sentence adds value, such as specifying search fields and keyword logic, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (search operation with one parameter) and no annotations or output schema, the description is moderately complete. It covers what the tool does and how keywords are handled, but lacks details on return values, limitations, or error cases, which would be helpful for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'query' parameter thoroughly. The description adds minimal value beyond the schema by reiterating the OR condition and field coverage, but does not provide additional syntax, examples, or constraints. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('search for nodes') and resource ('knowledge graph'), and distinguishes it from siblings by specifying it searches based on keywords across multiple entity fields. It explicitly mentions what fields are searched (entity names, types, subdomains, observation content), which differentiates it from tools like 'open_nodes' or 'read_graph'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: when searching for nodes by keywords across specific entity fields. It does not explicitly state when not to use it or name alternatives among siblings, but the context is sufficient for an agent to infer this is for keyword-based searches rather than other operations like creation or deletion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.