Server Quality Checklist

Tool Scores

• generate_codeA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors beyond annotations: automatic retries on gate rejection, returning the last attempt marked validated:false with findings if all attempts are vetoed, and that generation runs on infrastructure without executing in the user's environment. This richly complements the readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long but each sentence carries substantive information: the process, the failure behavior, and the execution environment. There is no redundant structure, though it is slightly dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description addresses the most important contextual factors: what happens on repeated rejections, the safety of the execution model, and the relation to validate_ai_output. Given the presence of an output schema, it does not need to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for all three parameters with detailed descriptions. The description adds no additional parameter-specific meaning beyond stating the generation context, so a baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Generate code and gate it in one step', clearly stating the tool's combined generation and validation function. It explicitly differentiates from sibling validate_ai_output by referencing the same protection engine, making the scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for generating code with built-in validation, and mentions validate_ai_output as the source of the protection engine, suggesting that tool handles validation alone. This gives clear context, though it doesn't explicitly list when not to use alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_codeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond the annotations: it states the tool is read-only and never executes code (aligning with readOnlyHint), deliberately has no pass/fail verdict, and produces concrete latency/throughput arithmetic. The description enriches the agent's understanding of what the tool will and won't do.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, and every clause earns its place. It packs the key behavioral constraints and usage alternatives without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers what the tool returns (scores/findings, no verdict), its scope (existing code), its safety characteristics (read-only, never executes), and when to use it. The presence of an output schema and complete input schema coverage means the description doesn't need to explain structured fields; it provides all necessary contextual guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already fully documents all three parameters (code, language, analysis_type) with descriptions. The tool description does not directly add parameter-level semantics, but it does reinforce the high-level purpose (e.g., 'scores and findings') that maps to the analysis_type enum. Baseline of 3 is appropriate given complete schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs an 'Advisory deep-dive on existing code' that 'scores and findings', with a specific scope (existing code, performance, scalability, reliability, tech debt). It explicitly distinguishes itself from validate_ai_output by noting it deliberately returns no pass/fail verdict, so an agent can tell which tool fits.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use it to understand a validate_ai_output rejection or review inherited code; use validate_ai_output when you need an accept/reject decision.' This clearly contrasts with sibling tools and sets the context for choosing this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ai_outputA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive, but the description goes further by stating 'Read-only: the code is analyzed, never executed,' explaining deterministic gates with veto power that 'cannot be overridden,' and describing the frontier-model scoring dimensions. This is rich behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact but information-dense. The core verdict mechanism is front-loaded; subsequent sentences add behavior, use-case guidance, and alternatives without redundancy. Every sentence contributes distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description doesn't need to explain return values. It covers purpose, when to use, safety behavior, parameter nuances, and examples of rejection logic. The tool is fully contextualized for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all three parameters. The description adds value by explaining that context 'language sharpens SDK-reality checks' and by distinguishing validation_type options ('code_generation' vs 'text'), deepening meaning beyond enum labels.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'The merge gate for AI-written CODE: returns a binary approve/reject verdict.' It immediately distinguishes the tool from siblings by naming validate_plan and analyze_code as alternatives for different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit invocation guidance is given: 'Call it on every AI-generated diff before accepting it.' It also provides clear when-not/when-else direction: 'use validate_plan for plans, analyze_code for an advisory report without a verdict.' This fully covers selection vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_planA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false. The description reinforces and extends this by stating 'Read-only: nothing is executed or stored beyond the verdict' and adding evaluative details (completeness, feasibility, performance, security) plus a concrete example of a rejected plan. It provides context beyond the annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (four sentences), front-loaded with the primary purpose, then adds behavior, side effects, and an alternative tool. Every sentence adds relevant information with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Combined with the rich annotations, full parameter schema, and output schema, the description covers purpose, usage timing, behavior, side effects, and alternatives. It does not need to explain return structure because output schema exists. It is complete for a validation tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both 'plan' and 'context', so baseline is 3. The description adds value by giving a concrete rejection example (polling every 100ms) and enumerating evaluation criteria (completeness, feasibility, etc.), which helps the agent craft an appropriate plan. This goes slightly beyond schema descriptions, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a gate for PLANS, designs, and specs, with a specific verb implied ('run BEFORE any code is written'). It distinguishes itself from validate_ai_output by explicitly stating that validate_ai_output is for the code that follows, making the resource and timing unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit timing ('BEFORE any code is written'), positions it as the 'cheapest place to catch a bad design,' and directly names the alternative for code validation ('Use validate_ai_output for the code that follows'). This gives the agent clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.