Enables natural language interaction with the QIT CLI for testing WordPress/WooCommerce plugins, including running security, E2E, PHPStan, and activation tests, managing test environments, handling test packages, and validating plugin ZIP files.
Enables natural language interaction with the QIT CLI for testing WordPress/WooCommerce plugins, including running security, E2E, PHPStan, and activation tests, managing test environments, handling test packages, and validating plugin ZIP files.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@QIT MCP Server run security tests on my woocommerce plugin"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP (Model Context Protocol) server that wraps the QIT CLI, enabling natural language interaction and agent-based workflows for WordPress/WooCommerce plugin testing.
Natural Language Testing: Run QIT tests using conversational commands
Agent Integration: Enable AI agents to run tests as part of automated workflows
Full CLI Coverage: Access all essential QIT CLI functionality through MCP tools
Smart CLI Detection: Automatically finds QIT CLI in PATH, local vendor, or custom location
Related MCP server: WordPress Developer MCP Server
Node.js 18+
QIT CLI installed and accessible
git clone https://github.com/woocommerce/qit-mcp
cd qit-mcp
npm install
npm run buildAdd to your Claude Code MCP settings (~/.claude.json):
{
"mcpServers": {
"qit": {
"type": "stdio",
"command": "node",
"args": ["/path/to/qit-mcp/dist/index.js"],
"env": {
"QIT_CLI_PATH": "/path/to/qit-cli/qit"
}
}
}
}The MCP server automatically detects QIT CLI in this order:
QIT_CLI_PATH environment variable
qit in system PATH
./vendor/bin/qit (local Composer installation)
Tool | Description |
| Connect to WooCommerce.com Partner Developer account |
| Check current authentication status |
Tool | Description |
| Run any test type (security, e2e, phpstan, activation, etc.) |
| Run a predefined test group from qit.json |
Tool | Description |
| Get test result(s) by ID |
| List test runs with filters |
| Get detailed test report |
| Open result in browser |
Tool | Description |
| Fetch status of a registered test group |
Tool | Description |
| Start a local test environment |
| Stop a running environment |
| List running environments |
| Execute command in environment container |
| Reset environment database |
Tool | Description |
| Manage test packages (publish, download, scaffold, delete, show) |
| List available test packages with filtering |
Tool | Description |
| Manage backends, partners, and tunneling configuration |
Tool | Description |
| List extensions you can test |
| Validate a plugin ZIP file |
| Manipulate QIT cache |
| Get QIT config directory path |
| Re-sync with QIT Manager |
"Run security tests on my-plugin"
→ run_test(type="security", plugin="my-plugin")
"Start a test environment with PHP 8.2"
→ start_environment(php_version="8.2")
"Show me my recent failed tests"
→ list_tests(status="failed")
"What plugins can I test?"
→ list_extensions()The MCP server communicates via stdio using the Model Context Protocol. See the MCP documentation for integration details.
# Install dependencies
npm install
# Build
npm run build
# Watch mode
npm run dev
# Run tests
npm test
# Lint
npm run lintMIT
Authenticate with WooCommerce.com Partner Developer account. Required before running managed tests.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| user | No | WooCommerce.com username (will prompt if not provided) | |
| application_password | No | Application password (will prompt if not provided) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that authentication is required for managed tests and includes installation instructions for QIT CLI, but it does not disclose key behavioral traits such as what happens on successful authentication (e.g., stores credentials, returns a token), error handling, or rate limits. The QIT CLI warnings are context but not core behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is not appropriately sized or front-loaded. The first two sentences are relevant, but the remaining content about QIT CLI installation methods and environment variables is lengthy and not directly about the tool's core functionality. This extraneous information reduces conciseness, as it includes troubleshooting steps that could be better placed elsewhere.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of an authentication tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral outcomes (e.g., what authentication returns, how errors are handled), does not explain the relationship with sibling tools like 'get_auth_status', and omits information on session management or token usage. The QIT CLI instructions add context but do not compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear documentation for 'user' and 'application_password' parameters. The description does not add any meaning beyond what the schema provides regarding parameters, as it focuses on authentication purpose and QIT CLI setup. With high schema coverage, the baseline score of 3 is appropriate, as the schema adequately handles parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Authenticate with WooCommerce.com Partner Developer account. Required before running managed tests.' It specifies the verb (authenticate) and resource (WooCommerce.com Partner Developer account), and provides context about its necessity for managed tests. However, it does not explicitly differentiate from sibling tools like 'get_auth_status', which might check authentication status rather than perform authentication.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'Required before running managed tests.' This implies it should be used as a prerequisite for tools like 'run_test' or 'run_test_group'. However, it does not explicitly state when not to use it (e.g., if already authenticated) or name alternatives, such as using 'get_auth_status' to check authentication status first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute a command inside a running QIT test environment's PHP container.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| command | Yes | Command to execute inside the container | |
| env_id | No | Environment ID. If not provided, uses the most recent environment. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that QIT CLI must be installed (a prerequisite) and mentions the 'env_id' parameter's default behavior ('uses the most recent environment'). However, it lacks critical behavioral details: what happens if the environment isn't running, whether commands are persistent, error handling, or output format. For a command execution tool with no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. The first sentence states the purpose, but the majority is a lengthy error/warning message about QIT CLI installation (7 lines), which belongs in error handling or prerequisites, not the core description. This wastes space and obscures the tool's functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It covers the basic purpose and prerequisites but misses behavioral context (e.g., execution safety, output format, error conditions) and doesn't leverage sibling tool context. For a command execution tool in a test environment suite, this leaves significant gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('command' and 'env_id') fully. The description adds minimal value: it implies 'env_id' is optional (matching schema) and notes the default behavior for 'env_id'. No additional syntax, format, or constraints are provided beyond the schema. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Execute a command inside a running QIT test environment's PHP container.' This specifies the verb ('execute'), resource ('command'), and context ('running QIT test environment's PHP container'). However, it doesn't explicitly differentiate from sibling tools like 'run_test' or 'reset_environment', which also involve execution in test environments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions prerequisites (QIT CLI installation) but doesn't compare to sibling tools like 'run_test' (for running tests) or 'start_environment' (for environment management). There's no explicit 'when-to-use' or 'when-not-to-use' context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check if QIT CLI is authenticated and show current authentication status.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool checks and shows authentication status, but fails to describe key behavioral traits: what the output looks like (e.g., success/failure, user details), whether it has side effects (likely none, but not stated), or any error handling. The CLI installation note is context but not core behavior. This leaves significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not concise. The first sentence states the purpose clearly, but the rest is an error message about CLI installation that doesn't belong in the tool description—it should be handled elsewhere (e.g., in error responses or documentation). This adds unnecessary length and distracts from the core functionality, reducing clarity and efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is incomplete. It lacks details on output format (e.g., what 'show current authentication status' entails), error conditions, or dependencies. The CLI installation note is misplaced and doesn't compensate for these gaps, making it inadequate for effective agent use despite the low complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100% (as there are no parameters to describe). The description doesn't need to add parameter semantics, so a baseline score of 4 is appropriate. It correctly doesn't mention any parameters, aligning with the empty input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Check if QIT CLI is authenticated and show current authentication status.' This specifies the verb ('check' and 'show') and resource ('authentication status'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'authenticate' beyond the read-only nature implied by 'check' and 'show'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating the tool checks authentication status, which suggests it should be used to verify authentication before operations that require it. However, it lacks explicit guidance on when to use this tool versus alternatives like 'authenticate' (for logging in) or other status-checking tools, and doesn't specify prerequisites or exclusions beyond the CLI installation note.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch the status of a registered test group from QIT Manager.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| group | Yes | Name of the test group to fetch status for | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions a CLI dependency issue but fails to describe what the tool actually does behaviorally (e.g., what 'status' includes, whether it's a read-only operation, error handling, or output format beyond the JSON parameter). The CLI warning is context but not core behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured: the first sentence is useful, but the rest is verbose CLI installation instructions that don't belong in a tool description. This wastes space and buries the purpose, making it less front-loaded and concise than ideal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It lacks details on what 'status' entails, how results are returned, or error conditions. The CLI warning adds noise but not completeness for the tool's functionality, leaving significant gaps for a tool with two parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents both parameters ('group' and 'json'). The description adds no parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for adequate coverage without extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'fetch' and the resource 'status of a registered test group from QIT Manager', which is specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_test_result' or 'get_test_report', which might also retrieve status-related information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions QIT CLI installation issues but doesn't explain the tool's context relative to siblings like 'get_test_result' or 'run_test_group', leaving the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the QIT configuration directory path.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool requires QIT CLI installation and may fail if not detected, which is useful behavioral context. However, it lacks details on what the tool returns (e.g., a string path, error codes), performance characteristics, or side effects, leaving significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and verbose. The first sentence states the purpose clearly, but the rest is dominated by installation instructions and error messages that don't belong in a tool description. This wastes space and dilutes the core information, making it less effective for an AI agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It explains prerequisites but fails to describe the return value (e.g., what the path looks like, error formats) or behavioral nuances. For a tool with zero structured data, this leaves the agent under-informed about how to interpret results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, maintaining focus on the tool's purpose and prerequisites. This meets the baseline for tools with no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get the QIT configuration directory path.' This is a specific verb ('Get') and resource ('QIT configuration directory path'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'manage_config' or 'list_environments', which might also involve configuration or environment paths.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It focuses entirely on installation prerequisites and error handling for missing CLI, without mentioning context, prerequisites beyond CLI installation, or comparisons to sibling tools like 'manage_config' that might handle configuration differently.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a detailed local test report for a specific test run. Only available for tests run locally (not managed tests).
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| test_run_id | Yes | Test run ID to get report for |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions the tool is only for local tests, it fails to describe what 'detailed report' contains, whether it's read-only or has side effects, authentication requirements, or error handling. The CLI installation warning is procedural rather than behavioral context about the tool itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured with significant waste. The first sentence is useful, but the extensive CLI installation warning (7 lines) belongs in error handling or prerequisites, not the core description. This creates noise and buries the actual tool purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the 'detailed report' contains, its format, or what happens on failure. The CLI warning addresses setup but not tool behavior, leaving significant gaps in understanding how to properly use this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with one parameter clearly documented in the schema. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline expectation but doesn't provide additional semantic context about the test_run_id parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'detailed local test report for a specific test run', making the purpose understandable. It distinguishes from siblings like 'get_test_result' by specifying 'detailed report' and 'local test run', but doesn't explicitly contrast with all similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage context by stating 'Only available for tests run locally (not managed tests)', which helps differentiate from potential managed test alternatives. However, it doesn't explicitly mention when to use this versus siblings like 'get_test_result' or 'open_test_result', leaving some ambiguity about tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get test result(s) by test run ID.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| test_run_id | Yes | Single test run ID or array of test run IDs | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. However, it fails to describe any behavioral traits—such as whether this is a read-only operation, what permissions are needed, how results are returned (e.g., format, pagination), or error handling. Instead, it's dominated by irrelevant CLI installation instructions, offering no useful context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. The first line is useful, but it's buried under lengthy, irrelevant CLI installation warnings that don't belong in a tool description. This wastes space and distracts from the tool's purpose, making it inefficient and cluttered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It lacks essential details like return values, error conditions, or behavioral context. The CLI installation text is irrelevant to the tool's functionality, failing to provide the necessary information for an agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for 'test_run_id' (single ID or array) and 'json' (boolean for JSON output). The description adds no parameter semantics beyond this, but the schema adequately covers the parameters, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool 'Get test result(s) by test run ID' which clarifies the verb (get) and resource (test result(s)), but it's vague about what 'test result(s)' entails compared to sibling tools like 'get_test_report' or 'open_test_result'. It doesn't distinguish itself from these alternatives, making the purpose somewhat generic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives like 'get_test_report' or 'open_test_result'. The description focuses entirely on installation errors for QIT CLI, which is irrelevant to usage decisions. This leaves the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all running QIT test environments with compact output.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool lists environments with 'compact output', which hints at the return format, but doesn't describe behavioral traits such as whether it's read-only, potential errors, or how it interacts with the system. The warning about QIT CLI installation adds some context but is more about prerequisites than tool behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is not appropriately sized or front-loaded. The first sentence states the purpose, but the majority of the text is a lengthy warning about QIT CLI installation that repeats information and includes installation instructions, which doesn't earn its place in a tool description. This reduces clarity and efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (a tool with no parameters but no output schema), the description is incomplete. It lacks details on what the output looks like (e.g., format of 'compact output'), error handling, or dependencies beyond the CLI warning. Without annotations or output schema, more context is needed for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, which is efficient. A baseline of 4 is applied since it avoids redundancy and focuses on other aspects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'all running QIT test environments' with the qualifier 'with compact output', which specifies what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'get_test_report' or 'list_tests', which might also list related resources, so it doesn't fully distinguish from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions prerequisites (QIT CLI installation) but doesn't specify contexts or exclusions relative to sibling tools like 'list_tests' or 'get_test_report'. This lack of comparative usage advice limits its effectiveness.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List WooCommerce extensions you have access to test with QIT. Use 'search' parameter to filter results and reduce response size.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| search | No | Search/filter extensions by name or slug (recommended to reduce response size) | |
| limit | No | Maximum number of extensions to return (default: 20, use higher values only when needed) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions filtering with the 'search' parameter to reduce response size, which adds some behavioral context, but it doesn't disclose other traits like rate limits, authentication needs, or what the output looks like. The CLI installation warning is irrelevant to tool behavior and doesn't compensate for missing transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. The first sentence is relevant, but it's followed by a lengthy, irrelevant CLI installation warning that doesn't belong in a tool description. This wastes space and distracts from the tool's purpose, making it inefficient and cluttered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a simple read operation with 2 parameters, the description is incomplete. It lacks details on authentication requirements, output format, or error handling. The CLI warning adds noise instead of useful context, failing to compensate for the missing structured information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('search' and 'limit') with descriptions. The description adds minimal value by mentioning the 'search' parameter for filtering, but it doesn't provide additional meaning beyond what the schema states. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List WooCommerce extensions you have access to test with QIT', which provides a clear verb ('List') and resource ('WooCommerce extensions'), but it doesn't distinguish this from sibling tools like 'list_environments', 'list_packages', or 'list_tests'. The purpose is understandable but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a brief note to 'Use 'search' parameter to filter results and reduce response size', which gives some implied usage guidance, but it doesn't explicitly state when to use this tool versus alternatives like 'list_packages' or 'list_tests'. No exclusions or clear context are provided, making it minimal guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List available QIT test packages with compact output. Use 'search' to filter and 'limit' to control response size.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| search | No | Search/filter packages by name or namespace | |
| type | No | Filter by package type | |
| limit | No | Maximum number of packages to return (default: 20) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'compact output' which is useful behavioral information, but doesn't describe pagination behavior, rate limits, authentication requirements, or what happens when QIT CLI is not detected (the CLI warning appears to be static text rather than behavioral description). For a listing tool with zero annotation coverage, this is insufficient disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured with 80% of the text being a CLI installation warning that doesn't belong in a tool description. The actual tool description is only the first sentence, followed by irrelevant installation instructions. This violates front-loading principles and includes substantial waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a listing tool with 3 parameters, 100% schema coverage, but no annotations and no output schema, the description is incomplete. It doesn't explain the return format, pagination, authentication requirements, or error conditions. The CLI warning text is irrelevant noise that doesn't help the agent understand tool behavior. The description fails to compensate for missing structured data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description mentions 'search' and 'limit' parameters by name but adds no additional semantic context beyond what's in the schema. The 'type' parameter isn't mentioned at all. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'List available QIT test packages with compact output' which clearly states the verb (list) and resource (QIT test packages). However, it doesn't distinguish this tool from sibling tools like 'list_tests' or 'list_environments' - all appear to be listing operations for different resources. The purpose is clear but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions using 'search' to filter and 'limit' to control response size, but this is parameter guidance rather than when-to-use guidance. There's no indication of when to use this tool versus alternatives like 'list_tests' or 'search' operations. No prerequisites, exclusions, or comparison to sibling tools are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List test runs with optional filters. Default per_page is 10 to reduce response size.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| status | No | Filter by test status | |
| test_type | No | Filter by test type (e.g., 'security', 'e2e') | |
| extension | No | Filter by extension slug | |
| per_page | No | Maximum number of results to return (default: 10) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'Default per_page is 10 to reduce response size' which is useful behavioral context about pagination defaults. However, it fails to describe important behaviors like whether this is a read-only operation, what the response format looks like, whether results are paginated beyond the per_page setting, or any rate limits. The QIT CLI warning is installation guidance, not tool behavior disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured with wasted content. The first sentence is relevant tool description, but the remaining 80% is QIT CLI installation troubleshooting that doesn't belong in a tool description. This is not conciseness - it's inappropriate content inclusion that dilutes the actual tool information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter listing tool with no annotations and no output schema, the description is incomplete. It doesn't explain what 'test runs' are in this context, doesn't describe the return format or structure, and doesn't provide context about how this fits with the sibling tools. The QIT CLI warning is irrelevant to the tool's actual functionality and doesn't compensate for the missing contextual information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'optional filters' which is already clear from the schema, and mentions the default per_page value which the schema also documents. No additional parameter semantics are provided beyond what's in the structured schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'List test runs with optional filters' which clearly states the verb (list) and resource (test runs). However, it doesn't distinguish this from other list-related tools like 'list_environments' or 'list_extensions' - it's generic about what makes this listing operation unique. The purpose is understandable but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance about when to use this tool versus alternatives. There's no mention of when this should be used instead of 'get_test_report' or 'get_test_result', nor does it explain the relationship to 'run_test' or 'run_test_group'. The only usage information is about QIT CLI installation issues, which isn't actual usage guidance for the tool itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Low-level QIT cache manipulation. For refreshing cache data, use sync_cache instead.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Cache action: get (retrieve), set (store), or delete (remove) | |
| key | Yes | The cache key to operate on | |
| value | No | The value to store (required for 'set' action) | |
| expiration | No | Expiration time in seconds (optional for 'set' action) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'low-level manipulation' but doesn't explain what that means operationally (e.g., direct key-value operations vs higher-level abstractions). The CLI warning adds installation context but doesn't describe tool behavior like error handling, performance characteristics, or side effects. For a mutation tool (set/delete actions) with zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured with significant wasted content. The first sentence is useful, but the lengthy CLI installation warning (7 lines) belongs in documentation or error messages, not in a tool description for AI agents. This creates noise and buries the actual usage guidance. The description is not appropriately sized for its purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a cache manipulation tool with 4 parameters, no annotations, and no output schema, the description is incomplete. While it provides some usage guidance, it lacks crucial information about what the tool returns, error conditions, or behavioral details needed for proper invocation. The CLI warning doesn't compensate for these gaps in tool semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all parameters well-documented in the schema itself (action with enum values, key, value, expiration). The description adds no parameter information beyond what's already in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Low-level QIT cache manipulation' which identifies the resource (QIT cache) and general action (manipulation), but 'manipulation' is vague compared to the specific actions in the schema (get, set, delete). It distinguishes from sibling 'sync_cache' for refreshing, but doesn't fully clarify what 'low-level manipulation' entails versus other cache-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides when-not-to-use guidance: 'For refreshing cache data, use sync_cache instead.' This clearly distinguishes from a sibling tool and helps the agent choose between alternatives. The warning about QIT CLI installation also implies prerequisites for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Manage QIT configuration (backends, partners, tunneling). Actions: add_backend, remove_backend, switch_backend, current_backend, add_partner, remove_partner, setup_tunnel, set_default_tunnel
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | The configuration action to perform | |
| name | No | Backend or partner name (required for add/remove/switch) | |
| url | No | Backend URL (for add_backend) | |
| tunnel_method | No | Tunnel method (for tunnel actions: ngrok, cloudflare, etc.) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but lacks behavioral details. It lists actions but doesn't disclose effects (e.g., whether changes are persistent, require authentication, or have side effects). The CLI warning is helpful for setup but doesn't describe tool behavior during execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured: it mixes tool purpose with installation instructions and a warning. The first sentence is useful, but the CLI warning (6 lines) is extraneous for tool selection. It's not front-loaded and includes unnecessary content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a configuration management tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on return values, error conditions, or operational context (e.g., how it interacts with other tools like 'authenticate'). The CLI warning doesn't compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description lists action names but doesn't add meaning beyond the schema's enum values. It doesn't explain parameter dependencies (e.g., 'name' is required for specific actions) or provide examples, so it adds minimal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool manages QIT configuration for backends, partners, and tunneling, with specific actions listed. It distinguishes itself from siblings like 'manage_cache' or 'manage_package' by focusing on configuration management. However, it doesn't explicitly contrast with all siblings, so it's not a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It lists actions but doesn't explain prerequisites (e.g., QIT CLI installation), appropriate contexts, or when to choose other tools like 'authenticate' or 'exec_in_environment'. The CLI warning is diagnostic, not usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Manage QIT test packages. Actions: publish, download, scaffold, delete, show
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | The package management action to perform | |
| package_name | No | Full package ID with version (e.g., 'namespace/name:version'). Required for download, show, delete actions. | |
| path | No | Path for scaffold output or package location | |
| type | No | Package type for scaffold action | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It mentions that QIT CLI is required and gives installation steps, which is useful context about dependencies. However, it doesn't describe what each action does (e.g., what 'publish' entails, if 'delete' is destructive), the tool's permissions, rate limits, or output format, leaving significant gaps for a multi-action tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. It starts with a brief purpose, then devotes most space to CLI installation warnings and instructions unrelated to tool usage. This wastes sentences that don't help an agent invoke the tool, making it inefficient and cluttered despite moderate length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, multiple actions) and lack of annotations and output schema, the description is incomplete. It misses critical details like what each action does, behavioral implications (e.g., destructive operations), and output expectations, leaving the agent under-informed for proper tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all five parameters well. The description adds no additional meaning about parameters beyond listing the actions, which are covered by the 'action' enum. This meets the baseline of 3, as the schema does the heavy lifting without description compensation needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool 'Manage[s] QIT test packages' and lists five specific actions, which provides a clear general purpose. However, it doesn't specify what a 'QIT test package' is or how this differs from sibling tools like 'list_packages' or 'validate_zip', leaving some ambiguity about its exact scope and differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'list_packages' or other package-related tools. It includes installation instructions for QIT CLI, but these are prerequisites rather than usage context, failing to help an agent decide between this and sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Open a test result in the default web browser and return the report URL.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| test_run_id | Yes | Test run ID to open in browser |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It mentions the tool opens a browser and returns a URL, but doesn't disclose important behavioral traits: whether this is a read-only operation, if it modifies any state, what happens if the browser can't be opened, error conditions, or what format the returned URL takes. The QIT CLI warning is about prerequisites, not tool behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured with the core purpose buried after a lengthy warning about QIT CLI. The warning occupies 80% of the text but is about prerequisites, not tool behavior. The actual tool description is just one sentence. This is not appropriately front-loaded and contains excessive prerequisite information that doesn't help the agent understand the tool itself.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the returned URL looks like, what happens if opening fails, or how this differs from simply retrieving test results. The heavy focus on installation prerequisites doesn't compensate for missing behavioral context. Given the complexity of interacting with browsers and test systems, more complete guidance is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single parameter 'test_run_id' well-described in the schema. The description adds no additional parameter information beyond what's in the schema. According to scoring rules, with high schema coverage (>80%), the baseline is 3 even with no param info in description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Open a test result') and resource ('in the default web browser'), and specifies the outcome ('return the report URL'). It distinguishes from siblings like 'get_test_result' by emphasizing the browser opening action rather than just retrieving data. However, it doesn't explicitly contrast with all similar siblings like 'get_test_report'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_test_result' or 'get_test_report'. It mentions QIT CLI requirements but doesn't explain the specific use case for opening in browser versus other ways of accessing test results. The only contextual information is about prerequisites, not tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reset a QIT test environment's database to the post-setup state.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| env_id | No | Environment ID to reset. If not provided, resets the most recent environment. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the tool resets a database to a specific state, implying a destructive mutation, but doesn't disclose critical behavioral traits: whether this requires specific permissions, if changes are irreversible, what happens to ongoing processes, error handling, or typical response format. The installation warning adds operational context but not behavioral transparency about the tool's execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. The first sentence states the purpose, but the remaining ~80% is installation troubleshooting unrelated to tool functionality. This wastes space and buries the operational intent. Every sentence after the first fails to earn its place in a tool description meant for AI agents.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a mutation tool with potential side effects, the description is incomplete. It lacks information on prerequisites, behavioral outcomes, error conditions, and relationship to sibling tools. The installation warning addresses a setup issue but doesn't compensate for missing operational context. For a destructive reset tool, this is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with one parameter ('env_id') fully documented in the schema. The description adds no parameter-specific information beyond what the schema provides (e.g., no examples of environment IDs, no clarification on 'most recent environment' logic). With high schema coverage, the baseline is 3, and the description doesn't compensate with additional semantic context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Reset a QIT test environment's database to the post-setup state.' This specifies the verb ('reset'), resource ('QIT test environment's database'), and target state ('post-setup state'). It distinguishes from siblings like 'start_environment' or 'stop_environment' by focusing on database reset rather than environment lifecycle management. However, it doesn't explicitly differentiate from potential data-clearing operations in other tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., environment must be running), exclusions (e.g., don't use during active tests), or compare to siblings like 'manage_cache' or 'sync_cache' that might affect environment state. The bulk of the text is installation instructions, not usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a QIT test on a plugin or theme. Local tests (activation, e2e, performance, woo-api, woo-e2e) run in Docker locally. Managed tests (api, compatibility, malware, phpcompatibility, phpstan, plugin-check, security, validation) are enqueued on GitHub Actions. Version flags - Full (php/wp/woo): activation, e2e, performance, compatibility, woo-api, woo-e2e; WP/Woo only: phpstan, phpcompatibility; None: security, malware, validation, api, plugin-check. Local-only flags (env_vars, volumes, additional_plugins/themes, object_cache, tunnel, ui, test_packages): activation, e2e, performance.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| type | Yes | The type of test to run | |
| plugin | Yes | Plugin slug, path to ZIP file, or path to plugin directory | |
| zip | No | Custom source for the plugin/theme (local ZIP, local directory, or URL to a .zip file) | |
| php_version | No | PHP version to use (e.g., '8.1', '8.2', '8.3') | |
| wp_version | No | WordPress version to use (e.g., '6.4', '6.5', 'stable', 'rc') | |
| wc_version | No | WooCommerce version to use (e.g., '8.5', '9.0', 'latest') | |
| additional_plugins | No | Additional plugins to install (slug or slug:version format) | |
| additional_themes | No | Additional themes to install (slug or slug:version format) | |
| test_packages | No | Test packages to include | |
| volumes | No | Docker volumes to mount (host:container format) | |
| env_vars | No | Environment variables to set (key-value pairs) | |
| object_cache | No | Enable Redis object cache | |
| tunnel | No | Enable tunnelling method (cloudflare, ngrok) | |
| ui | No | Run tests in Playwright UI mode (for e2e tests) | |
| config | No | Path to qit.json configuration file | |
| wait | No | Wait for managed test to complete before returning (default: true) | |
| async | No | Enqueue test and return immediately without waiting | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It describes execution methods (local Docker vs. GitHub Actions) and test type categorizations, but fails to cover critical behavioral aspects such as error handling, output format, runtime implications, or side effects. For a complex tool with 18 parameters, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and verbose. It mixes tool functionality with installation instructions and error messages ('⚠️ QIT CLI not detected...'), which are irrelevant to the tool's purpose. The first paragraph is dense with technical details but lacks clear organization, making it hard to parse efficiently. Sentences do not earn their place as they include extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's high complexity (18 parameters, no annotations, no output schema), the description is incomplete. It omits explanations of return values, error conditions, and practical usage examples. While it covers test type distinctions, it fails to provide a holistic understanding needed for effective tool invocation, especially for a mutation tool with significant behavioral implications.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all 18 parameters thoroughly. The description adds some context by linking certain parameters to specific test types (e.g., 'Local-only flags...: activation, e2e, performance'), but this is minimal value beyond the schema. The baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Run a QIT test on a plugin or theme.' It specifies the verb ('Run') and resource ('QIT test on a plugin or theme'), making the action clear. However, it doesn't explicitly differentiate this tool from sibling tools like 'run_test_group' or 'list_tests', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance by detailing which test types are local vs. managed and which flags apply to which tests (e.g., 'Local tests... run in Docker locally. Managed tests... are enqueued on GitHub Actions.'). However, it lacks explicit when-to-use instructions compared to alternatives like 'run_test_group' or prerequisites for invoking the tool, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a group of tests defined in qit.json configuration file.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| group | Yes | Name of the test group defined in qit.json | |
| config | No | Path to qit.json configuration file | |
| wait | No | Wait for all tests to complete before returning | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool runs test groups from a configuration file, but fails to describe what 'running' entails (e.g., execution environment, side effects, timeouts, output format). The CLI installation troubleshooting is behavioral context but not about the tool's operation itself. Significant gaps remain for a tool that presumably executes tests.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded. The first sentence is useful, but the remaining 80% is installation troubleshooting that doesn't belong in a tool description. This wastes space and buries the actual purpose. While concise in word count, the structure fails to prioritize tool functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a test execution tool with no annotations and no output schema, the description is incomplete. It mentions the configuration file but doesn't explain what happens during execution, what outputs to expect, error conditions, or how it relates to sibling tools. The CLI troubleshooting is contextually misplaced rather than helpful for understanding the tool's operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents all 4 parameters. The description adds no parameter-specific information beyond implying the 'config' parameter references 'qit.json'. This meets the baseline of 3 when schema coverage is high, but adds minimal extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Run a group of tests defined in qit.json configuration file' which provides a clear verb ('run') and resource ('group of tests'), but it doesn't distinguish this tool from sibling tools like 'run_test' or explain how test groups differ from individual tests. The purpose is understandable but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'run_test' or 'get_group_status'. Instead, it's dominated by installation instructions for missing CLI dependencies, which are irrelevant to usage decisions. There's no mention of prerequisites, when-not-to-use scenarios, or comparisons with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start a local QIT test environment. Creates a temporary, ephemeral environment for testing.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| environment_type | No | Type of environment to create: 'e2e' or 'performance' | |
| php_version | No | PHP version to use (e.g., '8.1', '8.2', '8.3') | |
| wp_version | No | WordPress version to use (e.g., '6.4', '6.5', 'stable', 'rc') | |
| wc_version | No | WooCommerce version to use (e.g., '8.5', '9.0', 'latest') | |
| plugins | No | Plugins to install (slug, path, or URL) | |
| themes | No | Themes to install (slug, path, or URL) | |
| test_packages | No | Test packages to set up environment from | |
| utilities | No | Utility packages for environment setup | |
| volumes | No | Docker volumes to mount (host:container format) | |
| php_extensions | No | PHP extensions to enable | |
| env_vars | No | Environment variables to set (key-value pairs) | |
| object_cache | No | Enable Redis object cache | |
| tunnel | No | Enable tunnelling method (cloudflare, ngrok) | |
| config | No | Path to qit.json configuration file | |
| skip_setup | No | Skip running setup phases even if qit-test.json is found | |
| skip_activating_plugins | No | Skip activating plugins during environment setup | |
| skip_activating_themes | No | Skip activating themes during environment setup | |
| json | No | Return output in JSON format |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the environment is 'temporary, ephemeral' which is useful context, but doesn't describe what happens when starting fails, whether this requires specific permissions, what resources are consumed, or what the typical runtime behavior looks like. For a complex 18-parameter tool with no annotations, this is insufficient behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not appropriately sized. The first two sentences describe the tool's purpose, but the remaining 80% is installation troubleshooting that doesn't belong in a tool description. This violates front-loading principles and includes content that should be in error messages or documentation links rather than the core description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (18 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what happens after starting the environment, how to interact with it, what success/failure looks like, or how this integrates with the broader testing workflow. The installation troubleshooting doesn't compensate for these gaps in explaining the tool's role and behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 18 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. According to scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in description, which applies here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Start a local QIT test environment. Creates a temporary, ephemeral environment for testing.' This specifies the verb ('Start'), resource ('local QIT test environment'), and key characteristics ('temporary, ephemeral'). However, it doesn't explicitly differentiate from sibling tools like 'reset_environment' or 'stop_environment' beyond the 'start' action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. While it mentions prerequisites (QIT CLI installation), it doesn't explain when to start an environment versus using existing ones, or how this relates to siblings like 'list_environments', 'reset_environment', or 'run_test'. The installation instructions are helpful but don't constitute usage guidelines.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Stop a running QIT test environment.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| env_id | No | Environment ID to stop. If not provided, stops all environments. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions that the tool requires QIT CLI installation, which is useful context about prerequisites. However, it doesn't describe what 'stop' actually does behaviorally - whether it gracefully shuts down services, preserves data, or destroys the environment. For a mutation tool with zero annotation coverage, this is a significant gap in behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured and not front-loaded with essential information. The first sentence is good, but then it devotes 90% of the content to installation instructions and error messages that don't belong in a tool description. This is wasted space that should instead explain tool behavior, usage context, or parameter details. The description fails the 'every sentence should earn its place' test.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It focuses on installation problems rather than explaining what the tool does, when to use it, what happens when you stop an environment, or what the expected outcome is. Given the complexity of environment management and the lack of structured documentation, the description should provide much more contextual information about the operation's effects and appropriate usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (the env_id parameter is fully documented in the schema), so the baseline is 3 even though the description adds no parameter information. The description doesn't mention the parameter at all, nor does it provide additional context about environment IDs or the default behavior when no ID is provided. The schema carries the full parameter documentation burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('stop') and resource ('a running QIT test environment'), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'reset_environment', which could be a similar operation. The first sentence effectively communicates the core function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'reset_environment' or 'list_environments'. While it mentions prerequisites (QIT CLI installation), it doesn't explain the appropriate context for stopping an environment versus other environment management operations. The focus is on setup requirements rather than usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Re-sync local cache with QIT Manager.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. While it mentions the tool requires QIT CLI installation, it doesn't describe what the sync actually does behaviorally - what gets updated, whether it's destructive, how long it takes, or what happens on success/failure. The description is more about prerequisites than behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured - it leads with the actual purpose in one sentence, then devotes 90% of the content to installation instructions that don't belong in a tool description. The warning about QIT CLI not being detected is particularly problematic as it appears to be runtime diagnostic output rather than descriptive documentation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations, no output schema, and 0 parameters, the description should focus on what the tool does and what it returns. Instead, it's dominated by installation troubleshooting. The actual behavioral context - what 're-sync' means, what gets synchronized, what the expected outcome is - is barely addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0 parameters and 100% schema description coverage, the baseline is 4. The description correctly indicates this is a parameterless operation that performs a sync, which aligns with the empty input schema. No additional parameter information is needed or provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool 'Re-sync local cache with QIT Manager', which provides a clear verb ('re-sync') and resource ('local cache'), but it doesn't distinguish from sibling tools like 'manage_cache'. The purpose is understandable but lacks specificity about what distinguishes this sync operation from other cache-related operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'manage_cache'. It focuses entirely on installation prerequisites rather than usage context. There's no mention of when this sync is needed or what triggers it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate a local plugin or theme ZIP file's content.
⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:
Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH
For more information, visit: https://github.com/woocommerce/qit-cli
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Path to the ZIP file to validate |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions QIT CLI dependency and installation methods, which is useful context about prerequisites. However, it doesn't describe what validation entails (e.g., checks performed, success/failure criteria, output format), making it incomplete for a validation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is poorly structured - it starts with the tool's purpose but immediately devotes most content to installation instructions and error messages. The warning about QIT CLI not being detected is front-loaded but doesn't belong in the core description, making it inefficient and cluttered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what validation means in this context, what criteria are checked, or what the tool returns. The QIT CLI dependency information is helpful but doesn't compensate for the missing functional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'path' parameter clearly documented. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3 for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool validates a local plugin or theme ZIP file's content, which is a clear purpose. However, it doesn't distinguish this from sibling tools like 'list_extensions' or 'manage_package', and the description is dominated by installation instructions rather than tool functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions QIT CLI requirements but doesn't explain the tool's role in the broader workflow or how it relates to sibling tools like 'run_test' or 'manage_package'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
authenticateexec_in_environmentget_auth_statusget_group_statusget_qit_dirget_test_reportget_test_resultlist_environmentslist_extensionslist_packageslist_testsmanage_cachemanage_configmanage_packageopen_test_resultreset_environmentrun_testrun_test_groupstart_environmentstop_environmentsync_cachevalidate_zipMost tools have distinct purposes, such as authentication (authenticate, get_auth_status), environment management (start_environment, stop_environment, reset_environment), and test operations (run_test, run_test_group). However, some overlap exists between get_test_report and get_test_result, which could cause confusion about their specific functions, and manage_cache vs. sync_cache might be ambiguous without careful reading of descriptions.
All tool names follow a consistent snake_case pattern with clear verb_noun structures, such as list_environments, get_test_result, and manage_config. The naming is predictable and readable throughout the set, with no deviations in style or convention.
With 22 tools, the count is on the higher side for a testing tool server, bordering on heavy but still manageable. It covers various aspects like authentication, environment control, test execution, and reporting, which justifies the number, but it might feel slightly overwhelming compared to more streamlined servers.
The tool set provides comprehensive coverage for QIT testing workflows, including setup (authenticate, manage_config), environment lifecycle (start, stop, reset, exec), test execution (run_test, run_test_group), result retrieval (get_test_result, open_test_result), and auxiliary functions (validate_zip, sync_cache). No obvious gaps are present for the domain of WooCommerce extension testing.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
AI QA tester — real browsers scan sites for bugs, SEO, perf, and accessibility issues via chat.
Plan Salesforce deploys, open pull requests and trigger pipelines from your AI client.
AI pentesting: run scans, triage vulnerabilities, review PRs, manage schedules and assets.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/woocommerce/qit-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
