Server Quality Checklist

Tool Scores

• emit_eventC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'emits' an event and returns a 'reactive suggestion', but lacks details on side effects (e.g., if it modifies state), authentication needs, rate limits, or error handling. This is inadequate for a tool that likely triggers downstream processing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, starting with the core action ('emit a consultation event') followed by key details. Both sentences are informative, though the second sentence could be slightly more structured (e.g., separating the enum list from the return statement).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of an event-emitting tool with no annotations and no output schema, the description is incomplete. It doesn't explain what 'reactive processing' entails, what a 'reactive suggestion' looks like, or potential side effects, leaving significant gaps for the agent to understand the tool's behavior and outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting all three parameters. The description adds minimal value by listing the valid event types (which are already in the schema's enum) and mentioning the optional 'data' payload, but doesn't provide additional semantics beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('emit') and resource ('consultation event'), and it specifies the valid event types. However, it doesn't explicitly differentiate this tool from its siblings like 'log_pattern_assessment' or 'get_events', which might have overlapping domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'reactive processing' but doesn't clarify when this emission is appropriate compared to other event-related tools like 'get_events' or 'log_pattern_assessment', leaving the agent without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_stateA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns one entry if a key is specified or all entries if omitted, which adds behavioral context beyond the input schema. However, it doesn't cover aspects like error handling, performance, or authentication needs, leaving gaps for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded: the first part states the core purpose, the second explains parameter behavior, and the third provides usage context. Every sentence earns its place with no wasted words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description partially compensates by explaining the return behavior based on the key parameter. However, for a tool that reads shared state in a consultation system, it lacks details on output format, error cases, or coordination specifics, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters thoroughly. The description adds marginal value by implying the 'key' parameter's effect on output (single vs. all entries), but doesn't provide additional syntax or format details beyond what the schema specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads shared state from a consultation, specifying the resource (shared state) and action (read). It distinguishes from the sibling 'write_state' by indicating this is a read operation, though it doesn't explicitly contrast with other siblings like 'get_events' or 'get_subgraph' that might also retrieve data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance: 'Use for subagent coordination and progress tracking' suggests a specific context. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_events' or 'get_subgraph', nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_architectureA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: deterministic behavior ('Same consultation always produces same results'), prerequisites ('Requires pattern_assessment steps to have been logged'), and the computational process. However, it doesn't cover aspects like error handling, performance, or output format details, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. It uses bullet-like phrasing ('computes: maturity level...') to list outputs efficiently. However, the sentence structure is slightly dense, and some phrasing ('Same consultation always produces same results') could be more streamlined, though all content earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (scoring tool with 3 parameters, no annotations, no output schema), the description is moderately complete. It covers the purpose, prerequisites, and outputs but lacks details on the return format (e.g., structure of maturity scores or roadmap), which is critical since there's no output schema. This leaves gaps for an AI agent to fully understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain 'consultation_id' further or provide examples). This meets the baseline of 3, as the schema does the heavy lifting, but the description doesn't compensate with extra insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Deterministic architecture scoring from stored pattern assessments' and details what it computes (maturity level, pattern status, gap analysis, etc.). It distinguishes itself from siblings by focusing on scoring based on logged pattern assessments, though it doesn't explicitly name alternatives. The description is specific about the verb ('computes') and resource ('pattern assessments'), but lacks direct sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Requires pattern_assessment steps to have been logged during step 3 (traverse graph).' This implies when to use it (after logging assessments) but doesn't explicitly state when not to use it or name alternative tools for similar tasks. The guidance is helpful but incomplete, as it doesn't compare to siblings like 'consultation_report' or 'critique_consultation'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_eventsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the polling behavior and optional filtering, which is useful, but it doesn't cover aspects like rate limits, authentication needs, error handling, or what the return format looks like (e.g., list of events, pagination). For a polling tool with zero annotation coverage, this leaves gaps in understanding its full behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured, consisting of two sentences that efficiently convey the tool's purpose and parameter usage. Every sentence earns its place: the first states the core function, and the second explains key parameters. There is no wasted text, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (polling with filters), no annotations, and no output schema, the description is somewhat complete but has gaps. It covers the basic purpose and parameter guidance, but it doesn't explain the return values (e.g., event structure, error responses) or behavioral details like polling intervals. For a tool with no output schema, more context on what to expect would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already documents all parameters ('consultation_id', 'since_id', 'event_type') with clear descriptions. The description adds marginal value by reinforcing the use of 'since_id' for new events and 'event_type' for filtering, but it doesn't provide additional syntax, format details, or examples beyond what the schema states. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Poll consultation events with optional filters.' It specifies the verb ('poll') and resource ('consultation events'), distinguishing it from siblings like 'emit_event' (which creates events) or 'consultation_report' (which generates reports). However, it doesn't explicitly contrast with all siblings, such as 'read_state', which might also retrieve event-related data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use certain parameters ('Use since_id to get only new events since a previous poll. Use event_type to filter by type.'), but it doesn't explicitly state when to use this tool versus alternatives like 'read_state' or 'consultation_report'. It implies usage for polling events with filters, yet lacks explicit exclusions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_bookA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by explaining what the tool returns ('full text, chapter, page numbers, section title, suggested_questions'), the deterministic nature of suggested questions, and the logging capability. However, it doesn't mention potential limitations like rate limits, error conditions, or whether this is a read-only operation (though implied by 'search').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences that each serve a distinct purpose: explaining the core functionality, providing usage guidance, and describing logging. It's front-loaded with the most important information. The 'DEEP CONTEXT' prefix is slightly verbose but doesn't significantly detract from overall efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description provides adequate coverage of the tool's purpose and usage context. However, it doesn't fully compensate for the lack of output schema by describing the exact structure of returned passages or the format of suggested_questions. The description is complete enough for basic understanding but leaves some implementation details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3 even without additional parameter information in the description. The description does add some context about 'concept_ids from get_subgraph' and 'consultation_id to log retrieval steps', but doesn't provide significant semantic value beyond what's already documented in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('RAG search', 'embeds', 'returns') and resources ('book sections', 'passages'). It distinguishes from siblings by mentioning 'concept_ids from get_subgraph' and 'consultation_id from match_concepts', showing awareness of related tools. The description goes beyond the name 'ask_book' to explain the retrieval-augmented generation mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('ALWAYS scope with concept_ids from get_subgraph for precision') and mentions prerequisites ('Pass consultation_id to log retrieval steps'). However, it doesn't explicitly state when NOT to use it or name specific alternatives among the many sibling tools, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• supervise_consultationA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the return format ('workflow phase progress, recommended next tool call, step summary, recent event alerts, shared state entries') and the tool's role in workflow guidance. However, it doesn't mention error conditions, performance characteristics, or whether this is a read-only operation (though 'Track' implies reading). For a tool with no annotations, this provides useful context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences. The first sentence clearly states purpose and return values. The second sentence provides usage timing. Every element earns its place with no redundant information. It's appropriately sized for a single-parameter tool with clear functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 1 parameter with 100% schema coverage and no output schema, the description provides good contextual completeness. It explains what the tool returns (progress metrics, recommendations, summaries, alerts, state entries) which compensates for the missing output schema. For a workflow guidance tool, this covers the essential context about what information agents will receive. However, it doesn't mention error handling or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'consultation_id' with its description. The description doesn't add any parameter-specific information beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Track consultation progress and suggest the next action.' It specifies the verb ('Track', 'suggest') and resource ('consultation progress'), and distinguishes it from siblings like 'plan_consultation' or 'consultation_report' by focusing on progress tracking and guidance rather than planning or reporting. However, it doesn't explicitly differentiate from 'get_events' or 'read_state', which could also track progress-related data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Call after each major step for guided workflow.' This gives explicit timing guidance. It doesn't specify when NOT to use it or name alternatives among siblings, but the context implies it's for ongoing consultation tracking rather than initial planning or final reporting, which helps differentiate from tools like 'plan_consultation' and 'consultation_report'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• critique_consultationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it's a read-only analysis tool (implied by 'critique' and 'analyzes'), deterministic rather than generative, returns structured issues with severity levels and suggestions, and explicitly states it makes no LLM calls. However, it doesn't mention performance characteristics, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: purpose statement, analysis dimensions, and behavioral constraints. Every sentence adds value with zero wasted words, and key information is front-loaded with the core function stated immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter analysis tool with no output schema, the description provides good context about what the tool does, how it works (deterministic structural analysis), and what it returns (issues with severity, categories, suggestions). The main gap is lack of output format details, but given the tool's relatively simple function and good behavioral disclosure, it's mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a single parameter clearly documented in the schema. The description doesn't add any parameter-specific information beyond what the schema provides (consultation_id as the session to critique), so it meets the baseline for high schema coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a 'deterministic quality critique of a consultation session' with specific analysis dimensions (workflow completeness, traversal depth, pattern assessment coverage, passage diversity, critical edge checks). It distinguishes from siblings by focusing on structural analysis rather than generation, reporting, or planning functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('analyzes logged steps') and distinguishes from LLM-based approaches ('No LLM calls — pure structural analysis'), but doesn't explicitly state when to use this versus alternatives like consultation_report or supervise_consultation. It provides some guidance but lacks explicit when/when-not comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_subagentA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (structural validation of specific fields), what it returns (validation result with errors and warnings), and important behavioral constraints ('No LLM calls — pure structural validation'). However, it doesn't mention error handling, performance characteristics, or other operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences that each earn their place: first states the tool's purpose and scope, second specifies what it checks, third describes the return value and important constraint. No wasted words, front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (validation with specific field requirements), no annotations, and no output schema, the description provides good coverage of what the tool does, what it validates, and its behavioral constraints. However, without an output schema, it could more explicitly describe the structure of the validation result it returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'response' parameter. The description adds context that this is 'The JSON object returned by a graph-analysis subagent' and mentions the specific fields being validated, but doesn't provide additional syntax or format details beyond what the schema provides. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('validate', 'checks', 'returns') and identifies the resource ('subagent responses from scatter-gather graph traversal'). It distinguishes from siblings by specifying it validates subagent responses rather than performing other operations like asking, consulting, or generating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('for subagent responses from scatter-gather graph traversal') but doesn't explicitly state when to use this tool versus alternatives. It mentions 'No LLM calls — pure structural validation' which provides some guidance on its scope, but doesn't name specific sibling tools or provide explicit when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_pattern_assessmentA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the tool stores assessments for later use by 'score_architecture', which implies persistence and data recording behavior. However, it lacks details on potential side effects, error handling, or performance characteristics (e.g., rate limits, idempotency).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey purpose and usage. Every sentence adds value without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, including nested objects) and the absence of annotations and output schema, the description is reasonably complete. It explains the tool's role in a workflow and its relationship to other tools, but could benefit from more detail on behavioral aspects like error cases or data persistence guarantees.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description does not add any additional meaning or context beyond what the schema provides (e.g., it doesn't explain parameter interactions or provide examples). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Record a pattern assessment') and resources ('for a consultation session'), and distinguishes it from sibling tools by explicitly mentioning its relationship to 'score_architecture' (a sibling tool). It goes beyond restating the name by explaining the action and context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines, specifying when to use it ('Call this during graph traversal (step 3) for each architectural pattern you identify in the user's codebase or confirm is missing') and linking it to another tool ('score_architecture'). It clearly defines the context and purpose without being misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_failure_scenariosA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and discloses key behavioral traits: it's deterministic ('same consultation always produces same scenarios'), includes specific outputs (walkthroughs with file:line references, coverage mapping, warnings), and mentions cascading failure analysis. It doesn't cover rate limits or auth needs, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with 'STRESS TEST' and key purpose, but could be slightly more concise by integrating some details (e.g., 'Deterministic' note) more seamlessly. Most sentences earn their place by adding value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description does well to explain the tool's behavior, outputs (scenarios with specific elements), and prerequisites. It could improve by hinting at return format or error handling, but covers complexity adequately for a stress-test tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description doesn't add meaning beyond what the schema provides for 'consultation_id' or 'max_scenarios', though it implies 'consultation_id' relates to analysis sessions mentioned in the usage guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('generate concrete failure scenario walkthroughs') and resources ('missing/partial patterns'), distinguishing it from siblings like 'score_architecture' or 'health_check' by focusing on stress testing and failure analysis rather than evaluation or monitoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool ('Requires pattern_assessment steps from step 3') and provides context for its application ('STRESS TEST'), with no misleading guidance. This helps differentiate it from alternatives like 'consultation_report' or 'critique_consultation' by specifying prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conceptsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the tool returns compact output by default, supports filtering via search, and can include definitions. However, it doesn't mention pagination, rate limits, or error handling, leaving some behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by usage details and alternatives in three concise sentences. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is mostly complete. It covers purpose, usage, and key behaviors, but lacks details on output structure beyond compact format, which could be improved since there's no output schema to rely on.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value beyond the schema by mentioning the default behavior for include_definitions and the purpose of search, but doesn't provide additional syntax or format details. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all 138 concepts in the knowledge graph'), specifying the scope and output format. It distinguishes from sibling tools by contrasting with 'match_concepts' for consultation workflows, providing clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this to browse the catalogue; for consultation workflows, prefer match_concepts as the entry point.' This clearly states when to use this tool versus an alternative, with named context and exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• match_conceptsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: deterministic matching ('same description always produces the same concept ranking'), embedding similarity method, ranked output with scores, consultation_id creation for session tracking, and fingerprint generation. It doesn't mention rate limits or auth needs, but covers core behavior thoroughly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with three sentences that each serve distinct purposes: stating the core function, explaining deterministic behavior, and providing usage guidance. It's front-loaded with the most important information and contains no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 100% schema coverage, but no annotations or output schema, the description provides strong context about behavior, workflow role, and deterministic nature. It could benefit from mentioning output format details (since no output schema exists), but otherwise covers the essential context well given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add meaningful semantic context beyond what the schema provides about project_description, max_results, or similarity_threshold. It meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('match', 'returns', 'creates') and resources ('project description', 'knowledge graph concepts', 'consultation_id'). It distinguishes from siblings by mentioning its role as an 'ENTRY POINT' and explicitly naming related tools (get_subgraph, ask_book) for subsequent steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('ENTRY POINT'), when to use alternatives (pass consultation_id to get_subgraph and ask_book for step logging), and distinguishes it from siblings like list_concepts. It establishes clear sequencing in a workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_stateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it performs an upsert operation (implying mutation), logs a state_write step, and handles JSON-serializable values. However, it doesn't mention potential side effects like overwriting existing data or error conditions, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured: it starts with a clear purpose, immediately provides usage guidelines, and includes essential behavioral details without unnecessary elaboration. Every sentence adds value, and there is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation tool with no annotations or output schema), the description does a good job covering purpose, usage, and key behaviors. However, it lacks details on return values or error handling, which would be helpful for completeness, though not strictly required without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema, only reinforcing that the value is 'JSON-serializable' (implied by the schema's description). This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Upsert a key-value pair') and resource ('in consultation shared state'), distinguishing it from sibling tools like read_state (which presumably reads rather than writes). The purpose is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Use for subagent coordination') and provides concrete examples ('store discovered concepts, current phase, conflict markers, or any JSON-serializable value'), clearly differentiating it from alternatives like read_state for retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consultation_reportA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (computes metrics, shows checks, identifies gaps) and its optional diffing capability. However, it lacks details on output format, error handling, or performance characteristics, which would be needed for a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by specific metrics and usage guidelines in a logical flow. Every sentence adds value—none are redundant or vague—making it efficiently structured and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, and no output schema, the description does well by covering purpose, usage, and key metrics. However, it lacks details on the output structure (e.g., what the coverage metrics look like) and any limitations or prerequisites, which would be needed for full completeness in this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the purpose of the optional 'compare_to' parameter ('to see diffs') and contextualizing 'consultation_id' as part of coverage evaluation. However, it doesn't provide additional syntax or format details beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('compute coverage metrics') and resource ('for a consultation session'), distinguishing it from siblings like 'score_architecture' or 'validate_subagent' by focusing on coverage analysis rather than scoring or validation. It enumerates specific metrics like concept coverage, relationship type coverage, and gap identification, making the purpose highly specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('Call before synthesizing to ensure thorough coverage') and includes an alternative usage scenario ('Optionally compare two sessions with the same project fingerprint to see diffs'). This clearly distinguishes it from tools like 'critique_consultation' or 'supervise_consultation' by focusing on pre-synthesis coverage checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_subgraphA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it performs BFS traversal, returns nodes and edges, includes relationship types for discovery (alternative_to, requires, etc.), and mentions logging for coverage tracking. However, it doesn't specify rate limits, error handling, or performance characteristics, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with core functionality, uses bullet-like structure for relationship types, and every sentence adds value (e.g., traversal method, usage context, logging). No wasted words, efficiently conveying necessary information in three focused sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, 100% schema coverage, no output schema, and no annotations, the description is largely complete: it covers purpose, usage, behavior, and parameters context. However, it lacks details on output format (nodes/edges structure) and error cases, which could be important for a tool with complex traversal logic.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the purpose of traversal (to discover what users are missing via relationship types) and linking concept_ids to match_concepts/list_concepts, providing context beyond schema. It doesn't detail individual parameters but enhances overall parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'bounded graph traversal from seed concepts' using BFS up to max_hops, returning reachable nodes and edges. It specifies the exact operation (traversal), resource (graph/concepts), and distinguishes from siblings like list_concepts (which lists concepts) or match_concepts (which matches concepts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Given one or more concept IDs (from match_concepts or list_concepts)' and provides guidance on relationship types to discover what users are missing (e.g., alternative_to, requires). It also mentions passing consultation_id for logging, indicating integration with other tools like match_concepts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (database connection status, graph statistics, pipeline status) and its diagnostic purpose. However, it doesn't mention potential side effects, rate limits, or authentication requirements, leaving some behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and well-structured in two sentences. The first sentence states what the tool does and returns, while the second provides crucial usage guidance. Every word earns its place with no redundancy or wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's diagnostic nature, 0 parameters, no output schema, and no annotations, the description provides good contextual completeness. It explains what information is returned and when to use it. The main gap is the lack of output format details, but for a health check tool, the described return categories are reasonably sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the empty input. The description appropriately doesn't add parameter information, maintaining focus on the tool's purpose and usage. A baseline of 4 is appropriate for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check', 'returns') and resources ('server health', 'graph scope', 'database connection status', 'graph statistics', 'pipeline status'). It distinguishes from siblings by focusing on system diagnostics rather than knowledge graph operations like 'list_concepts' or 'get_subgraph'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Call this first to understand how large the knowledge graph is and whether the database is reachable.' This provides clear guidance on its initial diagnostic role versus alternatives like data retrieval or analysis tools among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_consultationA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a generative tool that creates a plan based on complexity assessment (using concept count, keywords, relationship density), returns structured output (step-by-step plan with tool names/parameters), and has a specific call pattern (once after match_concepts). It doesn't mention error handling or performance characteristics, but covers core operational behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: purpose statement, complexity assessment details, and usage instructions. Every sentence adds essential information with zero waste. It's appropriately sized and front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (generative planning with assessment logic), no annotations, and no output schema, the description does well by explaining the assessment criteria, output format, and sequencing requirements. However, it doesn't detail the plan structure or potential edge cases, leaving some gaps for a tool with behavioral complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining that the consultation_id comes 'from match_concepts,' providing context about the parameter's origin and relationship to another tool. This semantic context goes beyond the schema's basic type/requirement documentation, though it doesn't elaborate on format or validation details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('generate an adaptive consultation plan') and resources ('after match_concepts'), explicitly distinguishing it from siblings by mentioning its dependency on match_concepts. It specifies what it assesses (project complexity) and what it returns (step-by-step plan with tool names and parameters).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions: 'Call once after match_concepts, then follow the generated plan.' It clearly states when to use it (after match_concepts) and what to do next (follow the plan), distinguishing it from alternatives like consultation_report or supervise_consultation by specifying its unique sequencing role.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.