Server Quality Checklist

Tool Scores

• aitest_runA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It mentions that the AI engine plans, executes, analyzes, and returns a report, but does not disclose potential behavioral traits such as being asynchronous, duration, or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus an example, front-loaded with purpose. Every sentence earns its place; no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No annotations or output schema, and nested objects exist. Description is adequate for a simple tool but lacks details about result format, async behavior, or prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds minimal value beyond schema, just reiterating that description is a natural-language test description. No explanation of nested object parameters beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool submits a browser test job to aiTest, taking a URL and natural-language description. It distinguishes itself from sibling tools (aitest_report, aitest_status) by focusing on running tests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an example and mentions what to provide, but does not explicitly state when not to use the tool or suggest alternatives for querying test status or results. Sibling names imply alternatives but are not referenced.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aitest_reportA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the tool returns a comprehensive test report for finished jobs, including specific details. No mention of side effects, but as a read operation, it is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second lists contents. No unnecessary words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema or annotations, the description is fairly complete for a simple 1-param tool. It covers what the tool returns and its prerequisite (finished job). Could mention that the job must exist, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter job_id with 100% schema coverage. The schema already describes it as 'The job ID returned by aitest_run.' The description adds context about the report contents but not about the parameter itself, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get the complete test report for a finished job,' specifying the verb, resource, and scope. It distinguishes from siblings (aitest_run and aitest_status) by focusing on completed jobs and detailed results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finished jobs and lists report contents (step-by-step results, pass/fail summary, etc.), making it clear when to use this tool vs. alternatives. No explicit exclusions but context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aitest_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description clearly states it returns status and timing info, implying a read-only operation. It is transparent about what to expect, though it doesn't disclose potential side effects (none expected).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose and quickly mentions return values. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose and return values (status, timing info) but lacks details on the exact structure of 'timing info' or whether additional fields are returned. Without an output schema, slightly more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'job_id' is fully described in the schema with 100% coverage. The description adds no additional meaning beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks the status of a submitted test job, using a specific verb ('Check') and resource ('status of test job'). It distinguishes from siblings 'aitest_run' (submits) and 'aitest_report' (likely retrieves full report).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use after submitting a job via 'previously submitted', but lacks explicit when-not-to-use or comparison with sibling tools like 'aitest_report'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.