Server Quality Checklist

Tool Scores

• generate_test_fixturesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose side effects. It describes the output content in detail but fails to clarify whether the tool writes files to the given path or returns the scaffolds as a response. This ambiguity about filesystem modification is a significant omission.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single information-dense sentence with no filler. It could be slightly better structured to separate the two main outputs, but overall it is concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generation tool with no annotations or output schema, the description should state whether it modifies the target directory, what happens to existing files, and any prerequisites. It only describes the product, not the process, leaving critical operational details unknown.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value beyond the schema by explaining how the path is used to derive authData from auth fields and inputData from component input fields, and how the component parameter scopes the generation. This contextualizes the parameters more deeply than the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool's output: 'Ready-to-run test scaffolds' with specific components (bundle mock, appTester test skeleton). It distinguishes from sibling tools (inspect, lint, validate) which are analytical, not generative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the purpose but not explicitly stated. The 'Optionally target one component' hint gives some scoping guidance, but there are no explicit alternatives or when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_integrationA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It does disclose that the tool evaluates the project's own entry module 'exactly as the Zapier CLI does', which is a useful behavioral trait. However, it does not explicitly state whether any modification occurs (though 'inspect' implies read-only), nor does it mention potential errors or dependencies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence for the main purpose and a second clarifying note. Both are front-loaded and every word serves a purpose, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description appropriately lists the key components of the summary, giving the agent a good idea of the return value. The note about the entry module provides important execution context. It is complete enough for a simple single-parameter inspect tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a description for the single `path` parameter, giving 100% schema coverage. The tool description adds no further semantic detail about the parameter, so it provides no value beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Structured summary of a zapier-platform-cli project' and lists the specific components it covers (triggers, creates, searches, authentication, platform version, request hooks). This distinctively separates it from siblings like lint or validate, which have differing objectives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose is clear enough to imply when to use it (when you need an overview of a project's structure), but there is no explicit mention of when not to use it or how it compares to alternative sibling tools. The note about evaluating the entry module adds some context but no direct usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_integrationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden. It discloses the execution method (npx), the dependency requirement, and the error behavior (install guidance if CLI missing). This is good, but it does not describe the output or behavior on validation success/failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action verb. No wasted words; every sentence adds value (what, how, prerequisite/error behavior).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description adequately covers the core purpose, execution method, and a key prerequisite/failure mode. It could mention expected output, but is sufficient for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the single parameter (path to the integration project directory) at 100% coverage. The description does not add further parameter semantics, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it runs the project's `zapier validate` command and specifies the checks (schema + style). This distinguishes it from sibling tools like inspect_integration and lint_integration, which imply different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for validating a Zapier integration project via zapier validate, and gives a prerequisite (requires CLI, returns guidance if missing). However, it does not explicitly state when to use this over lint_integration or other alternatives, nor when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_integrationA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It lists specific checks performed, giving a clear behavioral picture. However, it doesn't disclose whether the operation is read-only or what the output format is, so it's not a perfect 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver the core purpose, specific checks, and relationship to validation. No wasted words, and the first sentence is front-loaded with an action verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (one parameter) and no output schema, the description covers purpose, checks, and relationship to validate. Minor gap: it doesn't mention the expected output or return format, so a small part of the behavior is unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter is fully documented by the schema. The description adds no additional parameter-specific meaning, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Checks a Zapier integration against common app-review blockers' with specific examples (e.g., missing sample data, unlabeled input fields, version mismatches), and distinguishes it from `zapier validate` by noting it complements rather than replaces, differentiating it from the sibling validate_integration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (check for review blockers) and provides an exclusion ('Complements (does not replace) `zapier validate`'), giving clear guidance on when not to rely on it as a substitute for validation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.