Server Quality Checklist

Tool Scores

• explain_ruleA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits itself. It only states the purpose and parameter format but does not describe return behavior, error handling, or any side effects. For a read-only explanation tool, some mention of expected output or failure modes would be helpful; the absence leaves the agent with unmet expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: a single purpose sentence followed by a clear 'Args' section. It avoids unnecessary words and is appropriately sized for a simple one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema available), the description adequately covers purpose and parameter semantics. It does not elaborate on behavior or usage contexts, but the output schema likely documents return values. A small decrement is warranted due to the lack of usage guidance and behavioral transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the tool description compensates fully by documenting the 'code' parameter with format and examples ('WPS432' or '432'). This adds meaning beyond the raw schema and ensures the agent knows exactly what to pass.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Explain a wemake-python-styleguide rule by code'. It uses a specific verb (explain) and resource (wemake-python-styleguide rule), and the scope is unambiguous. This distinguishes it from sibling tools like lint_file or update_rule_db, which operate on files or the rule database rather than providing rule explanations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description and parameter documentation do not mention any conditions, prerequisites, or exclusions relative to sibling tools such as lint_code or lint_project. The usage context is implied by the name but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_rule_dbA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It does reveal that the tool scrapes and updates, implying a write operation and network access, but it does not specify whether the database is overwritten, whether it is destructive, or any side effects like schema changes or rate limits. For a mutation tool, this is minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the agent action and target resource without any wasted words or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With zero parameters and an output schema present, the description covers the core action sufficiently. However, it omits important contextual details such as network requirements, idempotency, potential impact on the existing database, or when to execute this tool. It is adequate for the tool's simplicity but lacks thorough behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema has no fields, so there is nothing for the description to add about parameter semantics. The description does not mention any parameters, which is consistent. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verbs 'Scrape and update' and names the exact resource ('offline wemake-python-styleguide rules database'), making the action and target clear. This also distinguishes it from sibling tools (lint_file, lint_project, lint_code, explain_rule), which are all about linting or explaining rules rather than updating a database.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternative guidance is provided, but the purpose is clear enough to imply that this tool is for refreshing the offline rules database. The description does not state prerequisites, frequency, or scenarios where it should be preferred or avoided, so it only meets the 'implied usage' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_codeA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It mentions the ruleset but does not disclose whether the operation is read-only, whether it executes the code, or how results are returned. This lack of safety and return-state info is a significant gap for a tool with no other metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—two short sentences—with no redundant wording or filler. Every word contributes useful information about the purpose and the single parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the presence of an output schema (so return format need not be explained), the description is mostly complete. It captures the core function and input semantics. Minor omissions like explicit read-only assurance or behavioral disclaimers are not critical here but slightly reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds 'Python source code string to lint,' clarifying that the input is raw source text rather than a file path, which adds value beyond the schema's minimal type/title. However, it omits any constraints, examples, or format details that could further aid usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (lint), the resource (raw Python code string), and the specific ruleset (wemake-python-styleguide). This distinctly separates it from siblings like lint_file and lint_project, which presumably operate on files or projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'raw Python code string' implicitly signals that this tool is for ad-hoc code snippets rather than files or projects, aiding selection among siblings. However, it does not explicitly mention alternatives or exclusion criteria, so a small gap remains.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_fileA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not mention read-only behavior, error handling, output format, or any side effects. Even though an output schema exists, the description itself offers no transparency beyond stating the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, containing only two sentences plus the argument documentation. It front-loads the core purpose and includes no redundant information, earning a perfect score for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with a single parameter and an output schema, so the description does not need to explain return values. It covers the essential purpose and parameter semantics. However, the lack of usage guidance and behavioral details slightly reduces completeness, but given the simplicity, it remains adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions (0% coverage), but the description compensates by explaining the parameter 'path' as a path to a Python (.py) file, adding file-type specificity and intent. This is valuable and goes beyond the schema's bare 'string' type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Lint'), a specific resource ('a single Python file'), and the applicable rule set ('wemake-python-styleguide rules'). This distinguishes it from sibling tools like 'lint_project' and 'lint_code', which likely handle different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for a single Python file, which differentiates it from project-level linting. However, it does not explicitly state when to prefer this tool over alternatives or provide exclusions, leaving the decision partially implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_projectA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that linting is recursive and uses a specific rule set, and mentions the default directory. However, it does not state whether the operation is read-only, what the output looks like, or how errors are handled, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the main purpose and then documenting the argument. There is no extraneous information, and every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with one optional parameter and an output schema present, so the description need not detail return values. It covers the tool's purpose, scope, and the parameter adequately. Minor gaps like error handling or prerequisites are not critical for this level of complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only provides a string type with a default for 'directory', while the description explains it is 'Path to the target project directory' and that it defaults to the current directory. This adds meaningful semantic context beyond the bare schema, fully covering the single parameter despite 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Lint') and resource ('an entire Python project directory recursively') and names the rule set (wemake-python-styleguide). This clearly differentiates it from sibling tools like lint_file and lint_code, which likely target individual files or code snippets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies its use case: linting a whole project directory recursively. It does not explicitly list alternatives or exclusions, but the scope is well-defined and distinct from the sibling tools, providing enough context for an agent to choose appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.