Server Quality Checklist

Tool Scores

• robot_create_testC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses that the result is a reviewable draft, which suggests no immediate execution, but it does not mention file creation, overwrite behavior, side effects, or input expectations. For a creation tool this is a substantial gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant wording. It is appropriately short, though 'nearby resources, libraries, and reusable keywords' is somewhat vague and could be replaced with more precise behavioral content without hurting conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a creation tool with six parameters, no annotations, and no schema descriptions, the description is too thin to let an agent invoke it correctly. It conveys the high-level purpose but omits essential information about required inputs, output behavior, and overwrite semantics. The presence of an output schema helps but does not make up for the missing parameter and behavior guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the six parameters. 'Nearby resources, libraries, and reusable keywords' hints at context but does not clarify the meaning or relationship of request, target_path, workspace, tags, test_name, or overwrite. The description fails to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Create') and resource ('reviewable Robot draft'), and the modifier 'reviewable' plus 'draft' distinguishes it from execution-oriented siblings like robot_run and robot_dry_run. It stops short of explicitly naming the output artifact (e.g., a Robot file) or target path, so it is clear but not fully precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool instead of robot_run, robot_dry_run, or robot_list_tests. The 'reviewable draft' phrasing implies it is for authoring rather than executing, but no explicit conditions or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• robot_runB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose the core behavior: running suites and returning JSON with status, failures, and output location. However, it does not mention side effects like writing output artifacts, executing arbitrary test code, or consuming significant time/resources beyond the implied execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It is concise and places the primary action first, though the brevity contributes to under-specification elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return-value details are not the main burden. Still, the description lacks context for tag filtering, workspace semantics, timeout meaning, and when to choose this over robot_dry_run or robot_parse_results, leaving the definition adequate but incomplete for a six-parameter runner tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the six parameters. It only hints at 'selected Robot suites' for paths and 'output location' for output_dir, leaving timeout, workspace, exclude_tags, and include_tags without meaningful explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Run') and resource ('selected Robot suites'), and it names the return payload. However, it does not explicitly distinguish this from the sibling robot_dry_run, so an agent must infer that 'run' means actual execution rather than dry-run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives such as robot_dry_run, robot_parse_results, or robot_list_tests. The description does not mention exclusions, prerequisites, or conditions that would route an agent to a sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• robot_list_testsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden; it does disclose the key non-execution guarantee, which is valuable for an agent weighing side effects. However, it does not mention file/path expectations, default workspace behavior, or any failure modes, so behavioral coverage remains partial rather than complete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One compact sentence with no filler; every phrase ('List', 'test cases, tags, source files, and line numbers', 'without executing them') earns its place. The verb and object are front-loaded, making the intent immediately visible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has an output schema, so return values need not be re-described, but the description still lacks parameter semantics and explicit usage guidance for 2 defaults-only parameters. An agent can infer basic behavior from the name and 'without executing,' yet important invocation details (what path/workspace mean, when this is preferable to parsing results) are absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description never discusses the 'path' or 'workspace' parameters, leaving their meaning, relationship, and constraints entirely to the sparse schema field names and defaults. The description also doesn't clarify whether 'path' refers to a file, directory, or suite pattern, so it fails to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List') and concrete resource ('Robot test cases, tags, source files, and line numbers'), and adds 'without executing them' to set it apart from execution-oriented siblings like robot_run and robot_dry_run. This makes the tool's intent immediately actionable and differentiates it from robot_list_resources by naming test cases rather than resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without executing them' implies a safe inspection use case and distinguishes it from run/dry_run, but it never names an alternative or states conditions for choosing this tool over robot_parse_results or robot_list_resources. The guidance is therefore implied rather than explicit, leaving the agent to infer selection rules from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• robot_parse_resultsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. 'Parse... and return' signals a read-only analysis operation and states the shape of the result, but it does not mention error behavior, workspace requirements, or side effects (even though side effects are unlikely).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no filler. Every word earns its place: the verb, subject, and output are all stated directly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has an output schema, so return format does not need to be explained. However, the missing workspace semantics and lack of usage guidance leave a moderate gap for an agent deciding whether and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It clarifies that output_xml refers to an output.xml file, but it says nothing about the workspace parameter, leaving half of the parameters under-documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Parse'), a specific resource ('output.xml'), and the return content ('compact test statistics and failure messages'). This lets an agent distinguish it from sibling tools like robot_run, robot_list_tests, or robot_dry_run, which have clearly different verbs and purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool instead of a sibling, or when not to use it. It implies post-run analysis of a Robot Framework output file, but the agent is left to infer that from the name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• robot_list_resourcesA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and explicitly discloses the key behavioral trait: no execution occurs. It clearly signals a safe, read-only operation. It does not describe file discovery behavior or error conditions, but the main side-effect concern is addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence with no filler. It front-loads the core action and scope, then adds the important non-execution qualifier.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return-value details are unnecessary. However, with zero schema parameter descriptions and no mention of path/workspace roles, the description is not fully complete for correct invocation, even though both parameters are optional with defaults.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented 'path' and 'workspace' parameters. It does not mention either parameter, leaving their meaning and relationship entirely to the agent's guess.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a clear resource ('Robot resource files, keywords, and imports'), and an explicit qualifier ('without executing them'). This distinguishes it from siblings like robot_run, robot_dry_run, and especially robot_list_tests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without executing them' implies this is for static inspection rather than execution, but the description does not explicitly state when to prefer this over robot_parse_results or robot_list_tests. Sibling differentiation is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• robot_dry_runA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It usefully warns that 'Robot libraries and variable files may still execute code while importing', which is a critical non-obvious behavior. It does not describe how validation results are returned, but the output schema likely covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, purposeful sentences. The first states the tool's function and scope; the second adds an important caveat. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's core purpose and a key behavioral warning, and the existence of an output schema reduces the need to explain return values. However, it omits parameter semantics and does not explicitly guide an agent on when to choose this over robot_run or other siblings, leaving some operational context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter-level guidance. It does not explain what paths should point to, what timeout controls, or how workspace is used. The parameter names are somewhat self-explanatory, but the description adds no semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Validate'), a clear resource ('Robot imports, variables, and keywords'), and an explicit boundary ('without test execution'). This clearly distinguishes it from sibling tools like robot_run and robot_list_tests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without test execution' provides clear usage context: this is a pre-flight validation tool rather than a test runner. It does not explicitly name alternatives, but the execution boundary and the validate-focused wording make the primary use case clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.