Server Quality Checklist

Tool Scores

• implement_codeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds a useful behavioral cue with 'Always review the returned code before accepting it,' implying output may need validation. It does not richly describe side effects, output handling, or failure modes, but given the annotations, a mid score is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: a one-sentence purpose, a one-sentence usage condition, a one-sentence advice line, and a one-sentence warning. The structure front-loads the core action and keeps every sentence functional. It could be slightly tighter but is well-organized and not padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a full input schema, output schema, and annotations covering safety, thedescription covers purpose, usage context, best-practice input guidance, and an important review warning. It does not mention pagination, side effects, or error cases, but those are less relevant for a code-generation tool and are partially covered by the output schema. The description is complete enough for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all six parameters including context, constraints, task_description, language, temperature, and max_tokens. The description reinforces that context and constraints should be provided, but adds no new parameter-level meaning beyond the schema, so it earns the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Generate') and resource ('implementation code from a specification'), then enumerates concrete artifact types (function,class,module,API endpoint, CRUD logic, glue code), making it obvious what the tool does and how it differs from siblings like generate_tests, refactor_code, and fix_code. The purpose is precise and immediately actionable for an agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this tool when you need the local model to write a function, class...', giving clear usage context. It advises providing context and constraints for best results. However, it does not state when not to use it or name alternative siblings explicitly, so it stops short of a fully explicit routing rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_docsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as readOnly, and the description adds a genuinely useful behavioral warning: 'Always review the returned documentation for accuracy.' This warns the agent that generated docs may be imperfect, which is valuable beyond the structured annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, front-loaded sentences with no filler. The purpose comes first, the style options follow, and the important accuracy warning is isolated for emphasis.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, full schema coverage, and readOnly annotation, the description is complete. It establishes the input (existing code), the style choices, and the need for output review, leaving no critical gap for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema carries the full parameter documentation. The description summarizes the style enum values, but this largely duplicates the schema rather than adding new semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Generate documentation for existing code.' It names the distinct deliverables (docstrings, README, API reference), which clearly separates it from sibling tools like generate_tests, implement_code, or refactor_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context—use it for existing code and choose a documentation style—but it never explicitly states when to prefer this tool over siblings like generate_tests or refactor_code. There are no exclusions or alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_testsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, and the description does not contradict this—generating test code produces output without mutating state. The description adds value beyond annotations by disclosing that the output is a complete, runnable file and by warning to always review returned tests before accepting, implying outputs may be imperfect. This is useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—two concise sentences plus an IMPORTANT warning—with every sentence earning its place: it states the action, describes the output, guides input configurability, and advises on output review. No redundancy or irrelevant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generative tool with 6 parameters and an output schema, the description adequately covers purpose, output nature, and optional input guidance. The review warning addresses output reliability risk. It doesn't discuss all potential failure modes, but full schema coverage and the output schema mitigate that gap, making the definition sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal parameter-level meaning by telling users to specify framework and scenarios, but the schema already documents framework, requirements, code, language, max_tokens, and temperature. No additional parameter semantics are provided beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb-object pair ('Generate test code') and clearly defines the deliverable as a 'complete, runnable test file' covering happy paths and edge cases. This distinguishes it from sibling tools like generate_docs and implement_code, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: generate tests for a given implementation, and explicitly directs the agent to specify the testing framework and particular scenarios via the framework and requirements parameters. It lacks explicit exclusions or comparisons to alternatives, but the context is sufficient for an agent to know when to select it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refactor_codeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint=true, and the description adds useful behavioral context: the model preserves external API/behavior and the result must be reviewed before acceptance. The warning 'Always review the returned code before accepting it' communicates uncertainty quality without contradicting the read-only annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: action first, then context, examples, and a critical safety warning. Every sentence contributes and there is no repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a full input schema, output schema, and annotations; the description supplies the missing practical context—when to use it, what to preserve, and the need for review. Nothing essential for safe and correct invocation is absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by giving concrete refactoring examples and framing refactor_description within preserving behavior, which helps an agent formulate meaningful transformation requests.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Apply a specific refactoring to existing code.' It clarifies the scope with examples like renaming and extracting methods, and distinguishes from bug-fixing or implementation by emphasizing preservation of external API and behavior, though it does not name a sibling explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states that the tool is useful for mechanical refactors, giving clear context for when it applies. However, it does not explicitly say when not to use it or name alternatives such as fix_code or implement_code, leaving exclusion to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fix_codeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already provide readOnlyHint=true, and the description adds useful behavioral context by noting that the local model 'will return a corrected version' and that the agent should 'Always review the returned fix before accepting it.' This warns about the model's fallibility without contradicting the read-only annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short, purposeful paragraphs: what the tool does, what inputs to provide, and an important caution. The 'IMPORTANT' warning earns its place and is not excessive. There is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, its output schema availability, and the annotations, the description is complete. It specifies required inputs, the expected output behavior, and a review caution. Nothing essential is missing for an agent deciding whether and how to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already describes code, errors, language, review_comments, and optional overrides. The description echoes the input requirements but does not add new parameter-level details beyond what the schema provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fix bugs in code based on errors or review feedback.' This clearly distinguishes fix_code from siblings like refactor_code (which implies improving structure rather than fixing bugs) and implement_code (which implies creating new code). The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear usage context: 'Provide the broken code along with compiler/runtime errors and/or reviewer comments.' It tells the agent what inputs are needed and what will be returned. However, it does not explicitly name sibling tools or state when not to use it, so it stops short of full alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_local_model_healthA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description aligns with readOnlyHint=true by describing a check rather than a mutation, and it adds useful behavioral context by saying it returns availability status and model metadata. This is transparent for a read-only health-check tool with no side effects to disclose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences with no filler. The core purpose comes first, the return summary second, and the usage guidance third. Every sentence adds distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only health check that already has an output schema, the description covers purpose, returned information, and when to invoke it. There is no missing detail an agent would need to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema coverage is 100%, so the description has no parameter documentation burden. The no-parameter baseline of 4 applies because there is nothing missing or ambiguous to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the exact verb and resource: 'Check if the local llama.cpp server is reachable and loaded.' It also states the return value ('availability status and model metadata'), which makes the tool's purpose unmistakable and distinguishes it from the sibling implementation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear trigger conditions: call before a batch of implementation tasks, or after a previous tool call fails with a connection error. It also explains the decision being made ('whether to retry or fall back to direct cloud-model implementation'). It does not explicitly state when not to use it, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.