Server Quality Checklist

Tool Scores

• scan_workflow_fileA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It communicates a scan (non-destructive, read-operation) intent and specifies the categories analyzed, which gives the agent a solid picture of the tool's behavior. It does not mention auth requirements or what happens if the path is invalid, but for a read-only analysis tool the disclosed scope is reasonably complete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence front-loads the core action and then enumerates the scan categories tersely. Every clause earns its place; there is no filler, redundancy, or restating of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The input side is fully covered given a single, well-contextualized parameter. However, with no output schema and no annotation coverage, the description does not convey what the scan returns—findings, severity levels, or error behavior—which an agent would reasonably want before invoking a security-scanning tool. That return-format gap is the main omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The single parameter 'path' is well-named, and the 'workflow file on disk' phrasing reinforces that it is a filesystem path to a YAML/JSON workflow file. This partial compensation covers the parameter's intent, though the description omits specifics like whether the path should be relative or absolute, and whether the file must exist prior to the call.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Scan') with a clear resource ('a GitHub Actions workflow file on disk') and enumerates the exact checks performed (dangerous triggers, template injection, unpinned actions, excessive permissions, secrets interpolated into shell commands). The 'on disk' qualifier cleanly separates it from the sibling scan_workflow_content, which presumably scans content strings rather than files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'on disk' phrase implies this tool is for file paths, giving implicit context about when to reach for it versus scan_workflow_content. However, there is no explicit statement that scan_workflow_content should be used when workflow content is available as a string or inline text, nor any exclusion or alternative named directly. The guidance exists but is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_workflow_contentA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full disclosure burden. It clarifies that the input is raw content passed directly (not a file path) and that it is for content not yet on disk, implying a read-only scan. However, it does not explicitly state that the tool has no side effects or what it returns. This is a moderate gap for a non-annotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, and the core action and scoping constraint are front-loaded. Every word adds value, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no annotations and no output schema, the description provides the essential information: what input to provide and when to use it. It lacks details about return formats or error behavior, but for a scanning action the intent is clear. It is complete enough for an agent to invoke correctly in the stated scenario.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter (yaml_content) with no description coverage (0%). The description adds meaning by specifying it should be 'raw GitHub Actions workflow YAML content' and clarifies that it is passed directly, not as a file reference. This compensates for the schema's lack of detail, giving an agent sufficient understanding of what to supply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Scan raw GitHub Actions workflow YAML content directly') and clearly distinguishes from its sibling by emphasizing 'raw content directly' for 'a workflow being drafted that isn't written to disk yet.' This makes the tool's purpose unambiguous and differentiates it from scan_workflow_file without needing to inspect the sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context for when to use this tool: 'for a workflow being drafted that isn't written to disk yet.' It implies the alternative (scan_workflow_file) is for when the workflow exists on disk, though it does not name it or provide explicit exclusions. The guidance is clear enough for an agent to decide correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.