Server Quality Checklist

Tool Scores

• evaluateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions actions like scoring and generating critiques but lacks details on permissions, side effects (e.g., whether evaluation modifies the graph), rate limits, or output format. For a tool with 4 parameters and no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and uses efficient language without redundancy. However, the second sentence ('Provides detailed analysis...') could be integrated more tightly, and it slightly repeats 'evaluate' from the first sentence, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (evaluating thinking processes with 4 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'graph health' entails, how scores are calculated, or what the output looks like (e.g., structured report vs. simple score). This leaves critical gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters with clear descriptions. The description adds no additional meaning about parameters beyond implying evaluation of 'thinking process' and 'graph health', which aligns with the schema but doesn't enhance understanding of parameter usage or interactions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with specific verbs ('score confidence', 'generate critiques', 'assess overall graph health') and identifies the resource ('thinking process'). It distinguishes from potential siblings like 'graph' (which might visualize) or 'think' (which might generate thoughts) by focusing on evaluation. However, it doesn't explicitly contrast with 'metacog' or 'prune', which could have overlapping evaluation aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'metacog' (which might involve meta-cognition) or 'prune' (which might remove weak nodes). It implies usage for analyzing thinking processes but doesn't specify contexts, prerequisites, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions querying and visualizing but doesn't address critical traits like whether it's read-only or mutative, authentication needs, rate limits, or output format. For a tool with multiple actions and no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two concise sentences that front-load the core purpose. Each phrase ('Query and visualize the thought graph', 'View the DAG structure...') earns its place by outlining functionality without redundancy. Minor improvements could include more structured formatting, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple actions, 3 parameters) and lack of annotations or output schema, the description is incomplete. It doesn't explain return values, error conditions, or behavioral constraints needed for effective use. The description should compensate for missing structured data but falls short, leaving gaps in contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing good documentation for all parameters. The description adds minimal value beyond the schema by listing general capabilities ('view the DAG structure, find paths, inspect branches, and get statistics') that loosely map to action enum values. This meets the baseline for high schema coverage but doesn't enhance parameter understanding significantly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Query and visualize the thought graph' with specific verbs and resources. It distinguishes from siblings like 'evaluate', 'metacog', 'prune', 'reset', and 'think' by focusing on graph exploration rather than modification or analysis. However, it doesn't explicitly contrast with each sibling tool, keeping it at a 4 instead of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists capabilities but doesn't specify contexts, prerequisites, or exclusions relative to sibling tools like 'evaluate' or 'metacog'. This lack of comparative guidance leaves the agent without clear decision-making criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pruneB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions actions like 'remove dead ends' and 'consolidate redundant branches,' which imply destructive changes, but doesn't clarify the permanence of these changes, potential side effects, or error handling. For a tool with multiple actions including 'prune' (which suggests deletion), this is a significant gap in transparency about its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: it starts with the core purpose ('Prune and optimize the thought graph'), lists specific actions, and ends with a usage hint. Both sentences earn their place by clarifying functionality and context, with no redundant or vague phrasing. A slight deduction because the second sentence could be more tightly integrated, but overall it's efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple actions including potentially destructive ones like 'prune'), no annotations, and no output schema, the description is incomplete. It doesn't cover what the tool returns (e.g., success status, optimized graph details), error conditions, or detailed behavioral traits. For a tool with 3 parameters and varied operations, this leaves significant gaps in understanding how to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (action, nodeId, reason) with descriptions and enums for 'action.' The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain parameter interactions (e.g., when nodeId is required) or provide examples. Baseline 3 is appropriate as the schema handles the heavy lifting, but the description doesn't compensate with extra insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Prune and optimize the thought graph' with specific actions like 'remove dead ends, consolidate redundant branches, and optimize reasoning paths.' It distinguishes from siblings like 'evaluate' or 'graph' by focusing on maintenance and optimization rather than evaluation or visualization. However, it doesn't explicitly differentiate from all siblings (e.g., 'metacog' or 'reset'), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: 'Helps maintain graph efficiency during deep reasoning,' suggesting it should be used for ongoing optimization in reasoning processes. However, it lacks explicit guidance on when to use this tool versus alternatives like 'reset' (which might clear the graph) or 'think' (which might generate new nodes), and it doesn't specify prerequisites or exclusions, leaving usage somewhat ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• thinkB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that thoughts become nodes in a DAG with confidence scoring, edges, and metadata, which adds behavioral context beyond basic creation. However, it doesn't cover important aspects like whether this is a write operation, if it's idempotent, error conditions, or how the graph persists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently convey core functionality and key features. The first sentence states the main purpose, while the second adds important structural context about the DAG. No wasted words, though it could be slightly more front-loaded with the most critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 13 parameters, nested objects, and no output schema or annotations, the description provides adequate but incomplete context. It explains the cognitive graph concept and reasoning strategies but doesn't address return values, error handling, or how this integrates with sibling tools. The schema compensates for parameter documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 13 parameters thoroughly. The description mentions reasoning strategies that map to some parameters (strategy, dialectic, parallel, analogical, abductive) but doesn't add significant meaning beyond what the schema provides. Baseline 3 is appropriate when schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Add') and resource ('thought to the cognitive graph') with specific context about reasoning strategies and node structure. It distinguishes from potential siblings like 'evaluate' or 'graph' by focusing on creation rather than assessment or visualization, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'using the current strategy' and mentions multiple reasoning strategies, suggesting when different approaches might apply. However, it lacks explicit guidance on when to choose this tool over siblings like 'evaluate' or 'metacog', or any prerequisites for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• metacogA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It describes key traits: the tool can view states, get suggestions, switch strategies, and receive guidance, with automatic detection of stuck states. However, it lacks details on permissions, rate limits, side effects (e.g., whether switching strategies affects ongoing processes), or response format. The description adds value but is incomplete for a tool with behavioral complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, listing core operations in the first sentence and adding context in the second. Every sentence earns its place by explaining functionality and automatic features without redundancy or fluff. It efficiently conveys the tool's scope in two clear sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (3 parameters, no annotations, no output schema), the description is somewhat complete but has gaps. It covers what the tool does and hints at usage, but lacks details on behavioral traits like side effects or response format. Without an output schema, the description should ideally explain return values, but it doesn't. It's adequate for basic understanding but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters well with enums and descriptions. The description doesn't add specific parameter semantics beyond what's in the schema, but it provides high-level context that aligns with the parameters (e.g., 'switch strategies' relates to the 'action' and 'strategy' parameters). With 3 parameters and full schema coverage, the baseline is 3, but the description's alignment with parameters justifies a slight bump.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs (view, get, switch, receive) and resources (thinking state, strategy suggestions, guidance). It distinguishes metacognitive operations from likely siblings like 'think' or 'evaluate' by focusing on monitoring and adjusting reasoning processes rather than direct thinking or evaluation. However, it doesn't explicitly differentiate from all siblings like 'reset' or 'prune'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'automatically detects stuck states and recommends actions,' suggesting this tool is for overcoming reasoning blocks. However, it doesn't explicitly state when to use this versus alternatives like 'think' for direct reasoning or 'reset' for restarting processes. No clear exclusions or named alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resetA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool resets state (destructive behavior) and starts a new session, but lacks details on permissions, side effects, or what 'fresh' entails (e.g., does it clear all history?). It adds some context but is not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action ('reset') and purpose ('start a fresh reasoning session'). Every word earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (state resetting), no annotations, and no output schema, the description is reasonably complete—it covers the main action and intent. However, it could benefit from more behavioral details (e.g., confirmation of reset, error cases) to fully guide usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'problem' parameter. The description does not add meaning beyond the schema (e.g., it doesn't explain how the problem statement integrates with the reset). Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('reset') and the resources ('thought graph and metacognitive state'), and distinguishes it from siblings by specifying 'start a fresh reasoning session'—implying a clean slate versus tools like 'think', 'evaluate', or 'graph' which likely operate on existing content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use this tool ('start a fresh reasoning session'), but does not explicitly mention when not to use it or name alternatives among siblings (e.g., 'prune' might be for partial cleanup). The guidance is implied but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.