The Autotest AI MCP Server enables AI assistants to manage and execute automated testing workflows with the following capabilities:
Project & Structure Management
List all projects, screens, and test cases (optionally filtered by screen)
Create new projects (with name, base URL, description), screens, and test cases (with priority and description)
Test Case Details & Steps
Get detailed information and steps for a specific test case
Add test steps to a test case, supporting types: ASSERT, ACTION, INPUT, INFO, or GET_OTP, with options for selectors, values, URLs, and expected results
Test Execution & Reporting
Trigger execution of specific test cases (run_test_case)
Retrieve recent test run results and statuses
Generate Puppeteer test files from test case steps (generate_test_file)
Project Data Configuration
View, update, add, and delete key-value configuration variables for a project (get_project_data_config, update_project_data_config, set_project_data_config_key, delete_project_data_config_key)
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Autotest AI MCP Server list all projects"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Tài liệu này hướng dẫn cách cài đặt, cấu hình và tích hợp Autotest AI MCP Server vào các công cụ AI Editor/Client (như Claude Desktop, Cursor) để AI có thể tự động đọc, tạo, và chạy các kịch bản kiểm thử trên hệ thống Autotest AI của bạn.
Để MCP Server hoạt động được, bạn cần có 2 thông tin quan trọng sau:
AUTOTEST_AI_URL (URL Server Backend):
Địa chỉ đường dẫn đến server Autotest AI đang chạy.
AUTOTEST_AI_API_KEY (Mã API Key/Token):
Đây là mã token xác thực quyền truy cập.
Cách lấy:
Đăng nhập vào trang giao diện web Autotest AI.
Truy cập trang Profile & Settings (bằng cách click vào góc dưới bên trái menu hoặc truy cập trực tiếp đường dẫn /profile.html).
Nhấn nút Generate Key để tạo mã khóa API Key của riêng bạn.
Copy mã này để cấu hình.
Related MCP server: Testmo MCP Server
Do repository này đã được chuyển sang chế độ Public, bạn không cần tải mã nguồn về máy mà có thể khởi chạy trực tiếp thông qua lệnh npx của Node.js.
Mở Cursor.
Truy cập vào Settings (biểu tượng bánh răng góc trên bên phải) -> Chọn tab Features -> Cuộn xuống mục MCP.
Nhấn nút + Add New MCP Server.
Điền các thông số cấu hình như sau:
Name: autotest-ai-mcp
Type: Chọn stdio
Command:
npx -y github:dongtrieuit/autotest-ai-mcpNhấn nút + Add Environment Variable để thêm 2 biến môi trường sau:
Biến 1: Tên AUTOTEST_AI_URL, Giá trị: https://autotest.bbtech.io.vn .
Biến 2: Tên AUTOTEST_AI_API_KEY, Giá trị: <Mã API Key đã copy ở https://autotest.bbtech.io.vn/profile.html>.
Nhấn Save. Cursor sẽ tự động khởi chạy và hiển thị chấm xanh lá cây báo trạng thái Connected.
Tìm file cấu hình claude_desktop_config.json trên máy tính của bạn:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
Mở file bằng một công cụ soạn thảo text bất kỳ và thêm cấu hình dưới đây vào (nếu file trống thì bọc ngoài bằng dấu {}):
{
"mcpServers": {
"autotest-ai-mcp": {
"command": "npx",
"args": ["-y", "github:dongtrieuit/autotest-ai-mcp"],
"env": {
"AUTOTEST_AI_URL": "https://autotest.bbtech.io.vn",
"AUTOTEST_AI_API_KEY": "mã_api_key_của_bạn_ở_đây"
}
}
}
}Khởi động lại ứng dụng Claude Desktop. Bạn sẽ nhìn thấy biểu tượng phích cắm (plug) xuất hiện ở khung chat hiển thị danh sách các tools khả dụng.
Sau khi kết nối thành công, bạn có thể yêu cầu AI tự động gọi các tool sau bằng ngôn ngữ tự nhiên:
list_projects: Lấy danh sách toàn bộ các dự án trên hệ thống.
list_screens: Lấy danh sách các màn hình (screens) của một dự án (Yêu cầu nhập: projectId).
list_test_cases: Lấy danh sách kịch bản test (Có thể lọc theo màn hình qua screenId).
get_test_case_details: Lấy thông tin chi tiết và toàn bộ các bước kiểm thử của một kịch bản test (Yêu cầu nhập: testCaseId).
get_latest_test_runs: Lấy danh sách kết quả lịch sử chạy test gần đây.
run_test_case: Kích hoạt hệ thống thực thi chạy tự động một kịch bản test (Yêu cầu nhập: testCaseId).
generate_test_file: Biên dịch danh sách các bước kiểm thử của một test case thành mã nguồn Puppeteer hoàn chỉnh và lưu lên hệ thống lưu trữ S3 (Yêu cầu nhập: testCaseId, testName, steps).
create_project: Tạo dự án mới (Yêu cầu nhập: name, baseUrl (tùy chọn), description (tùy chọn)).
create_screen: Tạo màn hình mới (Yêu cầu nhập: projectId, name, description (tùy chọn)).
create_test_case: Tạo kịch bản test mới (Yêu cầu nhập: screenId, name, priority (tùy chọn), description (tùy chọn)).
create_test_step: Thêm một bước kiểm thử vào kịch bản test (Yêu cầu nhập: testCaseId, stepOrder, stepType (ASSERT/ACTION/INPUT/INFO/GET_OTP) và các chi tiết hành động như action, selector, value, expected, url).
get_project_data_config: Xem toàn bộ danh sách các biến dữ liệu được thiết lập cho dự án (Yêu cầu nhập: projectId).
update_project_data_config: Cập nhật/ghi đè toàn bộ đối tượng JSON chứa các biến dữ liệu của dự án (Yêu cầu nhập: projectId, dataConfig).
set_project_data_config_key: Thêm mới hoặc cập nhật một cặp biến key-value cụ thể của dự án (Yêu cầu nhập: projectId, key, value).
delete_project_data_config_key: Xóa một khóa biến khỏi cấu hình của dự án (Yêu cầu nhập: projectId, key).
Bạn có thể mở khung chat với Cursor hoặc Claude và thử các câu lệnh sau:
"Liệt kê danh sách các dự án đang có trên hệ thống"
"Lấy thông tin chi tiết các bước chạy của test case ID 12"
"Hãy tạo một dự án mới tên là 'Dự án Thanh toán' có Base URL là https://pay.site.vn"
"Thêm bước kiểm thử nhập giá trị 'admin' vào ô input #username cho test case ID 5"
"Chạy thử test case ID 5 đi"
"Xem tất cả các biến môi trường/biến test của dự án ID 2"
"Thiết lập biến 'OTP_SECRET' với giá trị '123456' cho dự án ID 2"
"Xóa biến 'PASSWORD' khỏi cấu hình của dự án ID 2"
"Kiểm tra kết quả chạy test gần đây nhất xem có lỗi nào không"
Create a new project.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | The name of the project. | |
| baseUrl | No | The base URL for tests (optional). | |
| description | No | Optional project description. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits such as required permissions, side effects, or idempotency. For a creation tool, more details are needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence, which is appropriate for a simple tool. However, it could benefit from a bit more context without adding significant length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should hint at return values or success indicators. It also lacks information about uniqueness constraints or post-creation behavior, making it incomplete for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are documented well. The description adds no additional meaning beyond the schema, hence baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new project' clearly states the action and resource, but does not differentiate from sibling tools like create_screen or create_test_case, which also use 'create' verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives; usage is implied by the function but lacks context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new screen under a specific project.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | The name of the screen. | |
| projectId | Yes | The ID of the parent project. | |
| description | No | Optional screen description. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description lacks any behavioral details beyond the action. It does not disclose potential side effects, required permissions, idempotency, or success/failure responses, which are critical for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundant information. However, it could include a bit more context without becoming verbose, such as linking to prerequisites.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description should at least hint at the response (e.g., returns the created screen object). It also omits necessary context like the requirement that the projectId must refer to an existing project.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers all three parameters with clear descriptions (100% coverage). The description adds no additional parameter meaning beyond what the schema provides, meriting a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('new screen') with the context 'under a specific project', distinguishing it from sibling tools like create_project or create_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_project, create_test_case). The description does not specify prerequisites like the project must exist or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new test case under a screen.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | The name of the test case. | |
| priority | No | Optional priority level. | |
| screenId | Yes | The ID of the parent screen. | |
| description | No | Optional test case description. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It only states the action without disclosing side effects, authentication needs, rate limits, or what happens upon creation (e.g., ID generation). The description is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is front-loaded and efficiently communicates the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 4 parameters and no output schema. The description is minimal but covers the basic purpose. Given the simplicity, it is adequate but lacks details on return value or behavior beyond creation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. The tool description adds no extra meaning beyond the schema, so the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create) and the resource (test case) with context (under a screen). It effectively distinguishes from sibling tools like create_test_step or create_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., creating a test step or project). There is no mention of prerequisites or context for when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add a step to a test case.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Optional URL for navigation or verification. | |
| value | No | Optional value input (e.g., typed text). | |
| action | No | The action to perform (e.g., 'click', 'type'). | |
| expected | No | Optional expected assertion string. | |
| selector | No | Optional DOM selector target. | |
| stepType | Yes | The type of the step. | |
| stepOrder | Yes | The execution order index (1-based). | |
| testCaseId | Yes | The ID of the parent test case. | |
| description | No | Optional step description. | |
| expectedJson | No | Optional expected JSON object for assertion. | |
| otpProviderId | No | Optional OTP provider ID. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only says 'Add a step', implying mutation but lacks details on side effects, required permissions, or validation behavior (e.g., stepOrder insertion).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with no waste, but given the tool's complexity (11 parameters, nested objects), it may be too brief to be maximally useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and 11 parameters, the description is insufficient. It does not explain required vs optional fields, step types, or behavioral effects like insertion vs append.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no extra meaning beyond 'Add a step', which does not compensate for the lack of parameter context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add' and the resource 'step' with context 'to a test case', effectively distinguishing from siblings like create_project or create_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or when not to use this tool. The context of 'adding a step to a test case' is implied but no alternatives or prerequisites (e.g., test case must exist) are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get recent test execution runs and their status.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Number of runs to return (default is 5). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It correctly implies a read-only operation but doesn't detail ordering, data freshness, or whether results are project-scoped. The description is minimally adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core functionality without any extraneous words. It is highly efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 optional parameter, no output schema), the description is adequate but incomplete. It fails to explain what 'recent' means, how runs are ordered, or what fields are included in the status. An agent may need to infer or test.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter 'limit' described). The tool description adds no parameter information beyond what the schema provides, so it meets the baseline of 3 for high coverage. No extra clarity or context is added.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('recent test execution runs'), and what is returned ('their status'). It distinguishes itself from sibling tools like 'run_test_case' or 'get_test_case_details' by focusing on runs rather than test cases or creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when not to use it, nor does it reference sibling tools or compare their purposes. An agent would need to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get all steps and detailed definitions of a specific test case.
| Name | Required | Description | Default |
|---|---|---|---|
| testCaseId | Yes | The unique ID of the test case. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description indicates a read-only operation with no side effects, but lacks details on permissions, rate limits, or behavior for invalid IDs. Adequate for a simple fetch but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-formed sentence that immediately conveys the tool's purpose with no redundancy or unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 param, no output schema), the description covers the core purpose but does not detail the structure of the output (e.g., steps, definitions). Adequate for a minimal tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter testCaseId, which is described in the schema. The tool description does not add extra meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'all steps and detailed definitions of a specific test case', distinguishing it from siblings like list_test_cases which likely return summaries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving details of a test case but does not explicitly state when not to use it or mention alternatives like list_test_cases for browsing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all projects configured in the test automation system.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read-only listing operation with no side effects. Given the absence of annotations, the description adequately conveys the behavioral trait of simply retrieving data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundant information. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with no parameters or output schema, the description is sufficiently complete. It does not explain the return format, but that is acceptable for a straightforward list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the schema coverage is 100%, so the description does not need to add parameter details. The baseline for zero parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and the resource ('all projects'), directly indicating what the tool does. It is specific and unambiguous, distinguishing it from sibling tools like create_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is given on when to use this tool versus alternatives like create_project or list_screens. However, the purpose is intuitive given the resource name and context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all screens under a specific project by project ID.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | The unique ID of the project. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only implies a read-only operation. It does not disclose any potential limitations, pagination behavior, or performance characteristics, leaving a gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded. It efficiently states the purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with one parameter and no output schema, the description is minimally adequate. However, it lacks any mention of output format or potential edge cases, making it slightly incomplete for full contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 100% of the parameters with a clear description for projectId. The description adds 'by project ID' which aligns with the schema but does not provide additional meaning beyond what the schema already conveys.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list', the resource 'screens', and the constraint 'under a specific project by project ID', which distinguishes it from siblings like list_projects and list_test_cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as list_projects or list_test_cases. The description only states what it does without context about prerequisites or excluded scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List test cases. Can filter by screen ID.
| Name | Required | Description | Default |
|---|---|---|---|
| screenId | No | Optional screen ID to filter by. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose behavioral traits such as pagination, rate limits, authentication requirements, or return format. Only states the basic action and optional filter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, purpose stated first. Efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and no output schema, the description is adequate but fails to mention scope (e.g., project context) or whether results are paginated. Could provide more context about what is returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and description adds no new meaning beyond the schema's own description of 'screenId'. Baseline of 3 applies as description does not compensate for any lack of schema detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'List' and resource 'test cases', with optional filter by screen ID. Distinguishes from sibling 'get_test_case_details' but does not explicitly differentiate from 'list_screens' or 'list_projects'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_test_case_details' or 'list_screens'. Does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Trigger the execution queue for a specific test case ID.
| Name | Required | Description | Default |
|---|---|---|---|
| testCaseId | Yes | The ID of the test case to execute. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits but only says 'trigger execution queue', leaving ambiguity about side effects, asynchronicity, idempotency, or error states.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no unnecessary words. However, it could benefit from slightly more detail without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 1 parameter and no output schema, the description is functional but lacks information on return values or what triggering the queue entails.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds no extra meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Trigger the execution queue') and the specific resource ('a specific test case ID'). It distinguishes itself from sibling tools that create, list, or get details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no prerequisites or conditions mentioned. The description is too minimal to guide appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0create_projectcreate_screencreate_test_casecreate_test_stepget_latest_test_runsget_test_case_detailslist_projectslist_screenslist_test_casesrun_test_caseScored across 10 tools
Each tool targets a distinct entity (project, screen, test case, test step, test runs) with clear actions (create, list, get, run). No ambiguity among tool purposes.
All tools consistently use verb_noun pattern in snake_case (e.g., create_project, list_screens), with verbs like create, list, get, run and nouns matching domain entities.
10 tools is well-scoped for a test automation system, covering project setup, test case management, and execution without excess or deficiency.
Missing update and delete operations for all entities (project, screen, test case, test step) and no tool for individual test run details or cancellation, creating significant gaps for agent workflows.
Give your AI agents the tools to build, manage, and run automation workflows.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
AI-callable tools for API mocking, testing, monitoring, security, and automation.
Drive OctoPerf load testing from any AI agent: import, edit, validate, run scenarios, read metrics.
