Server Quality Checklist

Tool Scores

• buildC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Build') but does not describe what this entails—e.g., whether it's a compilation process, if it requires specific dependencies, potential side effects like file generation, or error handling. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, though it could be slightly more structured by including key details like behavioral traits.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a build operation, lack of annotations, and no output schema, the description is incomplete. It fails to explain what the build process involves, what outputs or errors might occur, or how it interacts with the Android project, leaving the agent with insufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'folder' parameter clearly documented. The description adds no additional meaning beyond the schema, such as explaining what constitutes a valid Android project folder or any constraints. Given the high schema coverage, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Build') and the target resource ('the Android project in the folder'), making the purpose understandable. However, it does not explicitly differentiate from sibling tools like 'instrumentedTest' or 'test', which likely involve testing rather than building, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as the sibling tools 'instrumentedTest' or 'test'. It lacks context about prerequisites, timing, or exclusions, leaving the agent without clear usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• instrumentedTestC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but reveals nothing about execution characteristics: whether it's read-only or mutating, what permissions are needed, how long it runs, what happens on failure, or what output to expect. For a test execution tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with no wasted words. It's appropriately sized for a single-parameter tool and front-loads the essential information. Every word earns its place in this concise formulation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a test execution tool with no annotations, no output schema, and behavioral unknowns, the description is insufficiently complete. It doesn't explain what 'instrumented test' means in practice, what the tool actually does during execution, what results to expect, or how it differs from the sibling 'test' tool. For a potentially complex operation like Android testing, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'folder' well-documented in the schema as 'The full path of the current folder that the Android project sits'. The description adds no additional parameter information beyond what's already in the schema. With complete schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run instrumented test') and the target ('for the Android project in the folder'), making the purpose understandable. It distinguishes from sibling tools 'build' and 'test' by specifying 'instrumented test', which suggests a more specific type of testing than generic 'test'. However, it doesn't fully explain what an 'instrumented test' entails compared to other test types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling 'test' tool. It implies usage for Android projects in a folder, but doesn't specify prerequisites, when instrumented tests are appropriate, or any exclusions. This leaves the agent with little context for choosing between available testing options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• testC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions running tests but doesn't disclose execution details (e.g., test environment, output format, side effects, or error handling). For a tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and target, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficient. It lacks details on test execution behavior, output expectations, error conditions, and differentiation from sibling tools, leaving significant gaps for an agent to understand proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'folder' well-documented in the schema. The description adds no additional parameter semantics beyond implying the folder contains an Android project, which is already inferred from the schema's description. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run test') and target ('for the Android project in the folder'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'build' or 'instrumentedTest', which likely have overlapping testing-related functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'instrumentedTest'. The description implies it runs tests for Android projects, but doesn't specify test types, prerequisites, or exclusions, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.