Server Quality Checklist

Tool Scores

• get_failuresB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. However, it does not clarify whether the tool runs tests itself or simply reports failures from a previous run. It also fails to disclose any side effects or the role of the 'test_dir' parameter, leaving significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with two short sentences. Both sentences serve a purpose: the first states the core action, the second gives the rationale. It is well-structured and front-loaded, though it sacrifices necessary detail for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description is minimal but incomplete. It does not explain how 'test_dir' affects behavior, nor does it clarify the relationship with sibling tools or the format of the returned failures. This is insufficient for reliable usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one optional parameter ('test_dir') with a default value, but the description does not mention or explain it. With 0% schema description coverage, the description should compensate, but it completely ignores the parameter, leaving its meaning unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Devolver solo las pruebas que fallaron' (return only failed tests). This distinct verb+resource combination differentiates it from sibling tools like 'run_all_tests' and 'run_test_file', which execute tests rather than retrieve failures.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Para corregir rápido' implies a usage context (quick fixing), but there is no explicit guidance on when to use this tool versus alternatives, nor any mention of prerequisites or workflow. The usage is implied rather than clearly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_all_testsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only states the action and summary output, but does not mention side effects, environment requirements, test_dir usage, or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that clearly communicates the tool's purpose. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks context about the test_dir parameter, output summary format, and when to use this tool versus siblings. Given the simplicity (1 optional param) it is still incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 1 parameter (test_dir) with 0% description coverage, and the description does not mention it. The parameter name is somewhat self-explanatory, but the description adds no meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs all tests in the Godot project and returns a summary, using a specific verb and indicating the outcome. This distinguishes it from siblings like run_test_file (single file) and get_failures (retrieves failures).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when all tests need to be run, but it does not explicitly mention when to prefer this over alternatives or provide exclusions. Sibling tools exist but are not referenced.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_test_fileA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavioral traits. It only states the action ('run') and provides an example, but fails to mention side effects, output, or any changes to the system. The description is minimal and leaves the actual execution behavior (e.g., displaying results, modifying files) undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence followed by an example. It is concise, front-loaded, and every element adds value. There is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and no output schema, the description is adequate but not complete. It tells the user what the tool does and an example, but does not explain what happens after execution (e.g., test results, error reporting). More detail about the execution outcome would improve completeness, especially given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required string parameter with no description. The description partially compensates by providing an example 'res://test_test_player.gd', which clarifies the expected resource-path format. However, it does not fully explain the parameter semantics (e.g., constraints on file type or path structure), relying on the example to convey meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Ejecutar un archivo de pruebas' (Run a test file). It includes a concrete example of the file path, making the purpose unambiguous. It also distinguishes itself from siblings like run_all_tests by specifying a single test file rather than all tests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the example (run a specific test file) but does not explicitly contrast with alternatives like run_all_tests or get_failures. No clear 'when to use' or 'when not to use' guidance is provided, relying on the user to infer from the name and example.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.