Server Quality Checklist

Tool Scores

• dry_run_workflowA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and destructiveHint=false. The description adds that only ephemeral logs are written to a temporary workspace, and confirms no side effects occur. This adds valuable behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the purpose. It uses two focused sentences with additional detail in a second sentence. No unnecessary repetition, though the second sentence could be split for scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, but the description explicitly states the return structure: {status, outputs, preview} and explains 'preview' as a human-readable list of steps. It covers both parameters adequately and addresses the tool's safety profile. Complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so baseline is 3. The description adds that 'workflow' can be a name from list_workflows or a file path, and 'params' are optional with defaults used when omitted. This adds modest value beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: 'Preview what a workflow would do without executing any task.' It specifies the verb (Preview), the resource (workflow), and distinguishes from siblings like run_workflow and list_workflows by emphasizing non-execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises use for inspecting side effects before real execution, and clarifies that no tasks are executed. It implicitly distinguishes from run_workflow but does not explicitly state when not to use or name alternatives beyond the context. A clear usage context is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_workflowA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description details that a workflow may run arbitrary shell commands, Python, write files, and make HTTP requests, explaining the nature of destructiveness beyond the annotations (destructiveHint=true, openWorldHint=true). There is no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main action and output, followed by a concise warning. Every sentence adds necessary information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description provides the return structure ({status, workflow, outputs}) and covers safety warnings. It mentions dry_run_workflow for preview. It could also reference validate_workflow for completeness, but it is adequately thorough given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description only paraphrases the schema (e.g., workflow is a name or path, params are optional) and adds no substantive extra meaning beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool executes a workflow and returns results, specifying inputs (workflow name/path and optional params) and output structure ({status, workflow, outputs}). This is distinct from siblings like list_workflows, validate_workflow, and dry_run_workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly warns that the tool is destructive and advises calling dry_run_workflow first to preview side effects, and only running trusted workflows. This provides clear guidance on when to use the tool vs. the alternative, though it could also mention validate_workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_workflowsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description reinforces and expands: 'Read-only: it only reads YAML files and never executes anything.' It also details the return structure (object with count and workflows) and clarifies that it takes no arguments. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place. The first sentence states the action and return structure, the second provides usage guidance, the third confirms read-only and no arguments. No fluff, front-loaded, and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is fully complete. It covers purpose, return format, usage context, behavioral traits, and references siblings. There is no missing information for an agent to correctly select and invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4 per the instructions. The description states 'Takes no arguments,' which adds no new information beyond the empty schema, but that is acceptable for a no-parameter tool. The schema coverage is trivially 100%, but the explicit mention of no arguments is clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource: 'List the workflows available in this server's workflow directory.' It further distinguishes itself from siblings by stating 'Call this first to discover which workflows exist and what inputs each accepts before calling dry_run_workflow or run_workflow.' This makes the purpose unambiguous and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use the tool: 'Call this first to discover which workflows exist... before calling dry_run_workflow or run_workflow.' It also notes it is read-only and takes no arguments. While it doesn't explicitly list when not to use it or mention alternatives like validate_workflow, the context with sibling tools and the sequencing advice is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_workflowA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description reinforces with 'Read-only: no tasks run and nothing is written.' Additionally, it describes the full return object including issues with level, message, line, step, hint. No contradiction; adds useful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with no wasted words. First sentence gives core function, second explains invocation, third details return value, fourth states safety. Well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully documents the return structure. Annotations cover safety. The tool has low complexity (one param, no nested objects). The description is sufficient for correct selection and invocation, including guidance on when to use it relative to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (one parameter 'path' described). The description restates 'Give the file `path`' but adds no new semantics or constraints beyond what the schema already provides. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool validates a workflow YAML file without running it, and gives the return structure. It distinguishes from siblings by contrasting 'without running it' and provides specific use cases like checking before running or explaining malformations. This is a specific verb+resource with clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to check a workflow the agent authored or edited before running it, or to explain why a workflow is malformed.' This gives two clear scenarios and implies it is for validation only, contrasting with the sibling tools (list, dry_run, run). No ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.