TestRail MCP Server
Provides tools for managing TestRail projects, suites, test cases, test runs, and test results, enabling AI-assisted test management workflows.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TestRail MCP Serverlist all test cases in suite 5"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TestRail MCP Server
A Model Context Protocol (MCP) server that integrates TestRail with Claude Code, enabling AI-assisted test management workflows.
Features
Project Management: List and retrieve TestRail projects
Suite Management: Access test suites and their configurations
Test Case Operations: Retrieve, update, and manage test cases
Test Run Creation: Create and manage test runs
Results Tracking: Add test results and retrieve execution history
Automation Integration: Link automated tests to TestRail cases
Related MCP server: TestRail MCP Server
Prerequisites
Node.js 18.x or higher
npm or pnpm
TestRail instance with API access enabled
TestRail user account with appropriate permissions
Installation
Option 1: Using Pre-built Container (Recommended)
The easiest way to use the TestRail MCP server is via the pre-built container. See CONTAINER.md for detailed instructions.
Quick start:
# Pull the container
podman pull ghcr.io/yshpyluk/mcp-testrail:latest
# Configure in .mcp.json (see CONTAINER.md for full setup)Option 2: Local Installation
Clone and Install Dependencies
cd mcp-testrail
npm installBuild the Project
npm run buildConfigure Environment Variables
Create a .env file in the project root:
TESTRAIL_BASE_URL=https://your-instance.testrail.io
TESTRAIL_USERNAME=your-email@example.com
TESTRAIL_API_KEY=your-api-key
TESTRAIL_PROJECT_ID=1 # REQUIRED: The TestRail project to work withGetting Your TestRail API Key:
Log in to TestRail
Go to My Settings (top-right corner)
Click on API Keys tab
Click Add Key to generate a new API key
Copy the generated key (you won't be able to see it again)
Configuring Your Project
TESTRAIL_PROJECT_ID is required - this MCP server is designed to work with a single TestRail project. All operations will be performed on the configured project. To find your project ID, check the URL when viewing your project in TestRail (e.g., .../index.php?/projects/overview/1 - the ID is 1).
Configuration for Claude Code
Add the TestRail MCP server to your Claude Code configuration:
Option 1: Global Configuration (~/.config/claude/config.json)
{
"mcpServers": {
"testrail": {
"command": "node",
"args": ["/absolute/path/to/mcp-testrail/dist/index.js"],
"env": {
"TESTRAIL_BASE_URL": "https://your-instance.testrail.io",
"TESTRAIL_USERNAME": "your-email@example.com",
"TESTRAIL_API_KEY": "your-api-key",
"TESTRAIL_PROJECT_ID": "1"
}
}
}
}Option 2: Project-Level Configuration (.mcp.json in project root)
{
"mcpServers": {
"testrail": {
"command": "node",
"args": ["/absolute/path/to/mcp-testrail/dist/index.js"],
"env": {
"TESTRAIL_BASE_URL": "https://your-instance.testrail.io",
"TESTRAIL_USERNAME": "your-email@example.com",
"TESTRAIL_API_KEY": "your-api-key",
"TESTRAIL_PROJECT_ID": "1"
}
}
}
}Security Note: For production use, consider using environment variables instead of hardcoding credentials:
{
"mcpServers": {
"testrail": {
"command": "node",
"args": ["/absolute/path/to/mcp-testrail/dist/index.js"],
"env": {
"TESTRAIL_BASE_URL": "${TESTRAIL_BASE_URL}",
"TESTRAIL_USERNAME": "${TESTRAIL_USERNAME}",
"TESTRAIL_API_KEY": "${TESTRAIL_API_KEY}",
"TESTRAIL_PROJECT_ID": "${TESTRAIL_PROJECT_ID}"
}
}
}
}Available Tools
Project Operations
list_projects
Get all TestRail projects.
Example:
// No parameters neededResponse:
[
{
"id": 1,
"name": "Akrochem ERP",
"is_completed": false,
"suite_mode": 1,
"url": "https://your-instance.testrail.io/index.php?/projects/overview/1"
}
]get_project
Get the configured TestRail project details.
Parameters: None
Suite Operations
list_suites
Get all test suites for the configured project.
Parameters: None
get_suite
Get a specific test suite by ID.
Parameters:
suite_id(number): The ID of the suite
Test Case Operations
list_test_cases
Get test cases for the configured project, optionally filtered by suite.
Parameters:
suite_id(number, optional): Filter by suite ID
Example:
{
"suite_id": 5
}get_test_case
Get a specific test case by ID.
Parameters:
case_id(number): The ID of the test case
update_test_case
Update a test case with new information.
Parameters:
case_id(number): The ID of the test casetitle(string, optional): New titlecustom_automation_id(string, optional): Automation identifierrefs(string, optional): Reference IDs (e.g., "JIRA-123")priority_id(number, optional): Priority (1=Low, 2=Medium, 3=High, 4=Critical)
Example:
{
"case_id": 100,
"custom_automation_id": "tests/specs/ui/purchase-order/new-po-page.spec.ts",
"refs": "AK-1234"
}Test Run Operations
create_test_run
Create a new test run in the configured project.
Parameters:
suite_id(number): The ID of the test suitename(string): Name of the test rundescription(string, optional): Descriptioncase_ids(number[], optional): Specific test cases to include (omit for all cases)
Example:
{
"suite_id": 5,
"name": "Automated Regression - 2025-01-08",
"description": "Nightly automated test run",
"case_ids": [100, 101, 102]
}list_test_runs
Get test runs for the configured project.
Parameters:
suite_id(number, optional): Filter by suite ID
get_test_run
Get a specific test run by ID.
Parameters:
run_id(number): The ID of the test run
close_test_run
Close a test run (mark as completed).
Parameters:
run_id(number): The ID of the test run
Test Results Operations
add_test_results
Add test results for multiple test cases.
Status IDs:
1= Passed2= Blocked3= Untested4= Retest5= Failed
Parameters:
run_id(number): The ID of the test runresults(array): Array of test result objects
Result Object:
case_id(number): Test case IDstatus_id(number): Status (1-5)comment(string, optional): Test result commentversion(string, optional): Version/build testedelapsed(string, optional): Time elapsed (e.g., "1m 30s")defects(string, optional): Comma-separated defect IDs
Example:
{
"run_id": 50,
"results": [
{
"case_id": 100,
"status_id": 1,
"comment": "Test passed successfully",
"version": "v2.5.0",
"elapsed": "45s"
},
{
"case_id": 101,
"status_id": 5,
"comment": "Timeout waiting for element",
"defects": "JIRA-456"
}
]
}get_test_results
Get all test results for a test run.
Parameters:
run_id(number): The ID of the test run
Usage Examples with Claude Code
Example 1: Create Test Run from Playwright Execution
Create a test run in TestRail for the upcoming automated test execution:
- Suite: UI Tests (ID: 5)
- Name: "Automated Regression - [TODAY'S DATE]"
- Include all test cases from the suiteExample 2: Push Test Results from CI/CD
Add test results to TestRail run #50:
- Case 100: Passed (45s)
- Case 101: Failed - "Timeout waiting for selector" (link to JIRA-456)
- Case 102: Passed (1m 20s)Example 3: Sync Automation IDs
Update test cases with automation IDs based on our Playwright test files:
- Case 100 -> tests/specs/ui/purchase-order/new-po-page.spec.ts
- Case 101 -> tests/specs/ui/purchase-order/uom-validation.spec.tsExample 4: Generate Test Coverage Report
List all test cases in suite 5 and compare with our Playwright test files.
Create a coverage report showing which TestRail cases have automated tests.Integration Patterns
Pattern 1: CI/CD Integration
Use the TestRail MCP server in GitHub Actions or other CI/CD pipelines:
# .github/workflows/playwright-testrail.yml
- name: Run Tests and Report to TestRail
run: |
# Run Playwright tests
npx playwright test --reporter=json > test-results.json
# Use Claude Code to parse results and push to TestRail
claude-code "Parse test-results.json and create TestRail run with results"Pattern 2: Manual Test Run Creation
Create a test run for Sprint 23 smoke tests:
- Include only priority 4 (Critical) test cases
- Name: "Sprint 23 - Smoke Tests"Pattern 3: Test Case Management
Update all purchase order test cases to link to Jira epic AK-1000Pattern 4: Results Analysis
Get test results for the last 5 runs and identify flaky tests
(cases that have inconsistent pass/fail status)Development
Build
npm run buildWatch Mode (for development)
npm run watchProject Structure
mcp-testrail/
├── src/
│ ├── index.ts # Main MCP server implementation
│ └── testrail-client.ts # TestRail API client wrapper
├── dist/ # Compiled JavaScript (generated)
├── package.json
├── tsconfig.json
└── README.mdTroubleshooting
"Error: Missing required environment variables"
Ensure TESTRAIL_BASE_URL, TESTRAIL_USERNAME, and TESTRAIL_API_KEY are set in your MCP configuration.
"Error: 401 Unauthorized"
Check that:
Your API key is correct
Your TestRail username is correct
API access is enabled in your TestRail instance (Admin > Site Settings > API)
"Error: Cannot find module"
Run npm run build to compile TypeScript to JavaScript.
"Connection refused"
Verify your TESTRAIL_BASE_URL is correct and accessible from your network.
Security Best Practices
Never commit API keys to version control
Use environment variables for sensitive credentials
Limit API key permissions to only what's needed
Rotate API keys regularly
Use project-level .mcp.json for team-specific configurations
TestRail API Reference
For more details on TestRail API capabilities:
License
MIT
Available Tools
13 toolsadd_test_resultsA
Add test results for multiple test cases in a test run. Status IDs: 1=Passed, 2=Blocked, 3=Untested, 4=Retest, 5=Failed
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run | |
| results | Yes | Array of test results to add |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the add action and repeats status IDs from the schema. It does not disclose whether existing results are overwritten, whether the test run must be in a certain state, or what the function returns on success.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with an important status ID mapping appended. It is concise, front-loaded, and every part serves a purpose without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with two parameters and a complete schema, the description is functional. However, it lacks critical context for a write operation: no information about side effects, prerequisites, or response behavior, which makes it incomplete for an agent deciding whether to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed descriptions for all parameters, including the status ID mapping. The description adds minimal extra value by mentioning 'multiple test cases' and listing status IDs, but the schema already documents these details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Add' plus resource 'test results' and clarifies scope 'multiple test cases in a test run'. This clearly distinguishes from sibling tools such as get_test_results, which reads results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for adding test results, but it does not explicitly state when to use it over alternatives or provide any exclusions. Sibling tools like get_test_results exist for reading, but no comparison is made.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
close_test_runA
Close a test run (marks it as completed)
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run to close |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states the effect ('marks it as completed') but does not disclose potential side effects, error conditions (e.g., if the run is already closed), or permissions required. For a state-changing tool, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the key action and outcome. Every word earns its place, with no redundancy or irrelevant detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has low complexity (one parameter, no output schema) and the description covers the basic purpose. However, it lacks information about what happens on success or failure, and whether there are any constraints. It is adequate but minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for run_id ('The ID of the test run to close'), so the schema fully explains the parameter. The tool description adds no additional parameter information, but the baseline of 3 is appropriate given the schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('close') and resource ('test run'), and clarifies that closing marks it as completed. It clearly distinguishes from sibling tools like create_test_run, list_test_runs, and get_test_run, which involve different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a test run should be marked as completed, but it does not explicitly state when to use this tool versus alternatives, nor mention any prerequisites such as the run needing to be open or results being added first. There is no explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_test_runB
Create a new test run in the configured TestRail project
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Name of the test run | |
| case_ids | No | Optional: Specific test case IDs to include. If omitted, includes all cases in suite. | |
| suite_id | Yes | The ID of the test suite | |
| description | No | Optional description of the test run |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. 'Create' implies a write operation, but no details are given about required permissions, side effects, idempotency, or what 'configured project' means. The description is too sparse to cover these aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence of eight words. It conveys the essential purpose without unnecessary elaboration, earning a high score for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is a simple create operation with full schema documentation, but the description lacks guidance on return values (no output schema exists) and clarifies the ambiguous 'configured project'. It is minimally viable but has clear gaps in contextual information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all four parameters (name, case_ids, suite_id, description) adequately described. The description itself adds no parameter-level information, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Create') and names the resource ('test run') within a clear context ('configured TestRail project'). It clearly distinguishes from sibling tools like list_test_runs or close_test_run by indicating a creation action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives is provided. The description does not mention related operations like add_test_results or update_test_case, leaving the agent to infer usage solely from the tool name and generic description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_projectB
Get the configured TestRail project details
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior, but it only states a simple getter. It does not mention whether the operation is read-only, requires authentication, or what 'configured' means, nor does it describe the return format or any potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the action and object with no unnecessary words. It is highly scannable and effective for a simple getter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal and omits essential context: what 'configured project' refers to, what 'details' includes, and any return value structure (no output schema exists). For a tool with no parameters and no annotations, the description should clarify these points to allow correct invocation and interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description cannot add parameter meaning. Baseline is 4 for no parameters, which is appropriate here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get' and a clear resource 'TestRail project details', clearly indicating the tool retrieves project information. However, it does not explicitly distinguish itself from sibling tools like list_projects, leaving some ambiguity about what 'configured' means in contrast to listing all projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_projects or get_suite. The description simply states what it does without mentioning any exclusions, prerequisites, or preferred contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suiteA
Get a specific test suite by ID
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | Yes | The ID of the suite |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of conveying safety. The verb 'Get' implies a read-only operation, but the description does not disclose error behavior (e.g., 404 if not found), response format, or any other behavioral traits. This is minimal but acceptable for a simple retrieval.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no filler. It is efficient and front-loaded with the verb and object.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID operation with one parameter and no output schema, the description is adequate. It tells the agent what the tool does and what input is required. It could benefit from a note about what is returned or how to handle missing IDs, but overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully describes the one parameter, suite_id, as 'The ID of the suite.' The description repeats this without adding additional meaning. Since schema coverage is 100%, the baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('a specific test suite'), with the scope ('by ID'). This distinguishes it from list_suites, which retrieves multiple suites. It follows the same pattern as get_project and get_test_case, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like list_suites or get_test_case. There is no mention of prerequisites or exclusions. The usage is implied by the get-by-ID pattern, but no direct guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_caseA
Get a specific test case by ID
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | The ID of the test case |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only states the action 'Get' (read), but does not mention potential errors (e.g., if ID not found), authentication requirements, or return format. For a simple getter, this is insufficient transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded with the verb and resource. It contains no redundant words or unnecessary details, earning a perfect score for conciseness and structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is a simple get-by-ID operation, and the schema fully documents the parameter. However, with no output schema or annotations, the description does not explain the return value or error behavior. It is adequate but leaves gaps that could affect an agent's confidence in using it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with parameter 'case_id' clearly described as 'The ID of the test case'. The description's 'by ID' adds no new meaning beyond what the schema provides, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a specific test case by ID' uses a specific verb ('Get') and resource ('test case') with a clear identifier ('by ID'). It clearly distinguishes from sibling tools such as list_test_cases (which lists multiple) and update_test_case (which modifies).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when a single test case is needed by ID, but it does not explicitly state when to use this tool versus alternatives. No exclusions or alternative tool names are mentioned, leaving the usage context implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_resultsA
Get all test results for a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the operation is a read ('Get') and that it returns all results for a run, but does not mention behaviors such as empty result handling, potential large payloads, or ordering. It is not misleading but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence efficiently conveys the tool's purpose with no redundant words. It is appropriately sized and front-loaded, earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one fully described parameter and no output schema. The description is sufficient for basic usage, though it could benefit from context about when results are available (e.g., after test run completion) or their structure. Still, it is complete for a straightforward read operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully documents the single parameter 'run_id' with a description, and the tool description does not add additional semantics beyond referencing the test run. The high schema coverage (100%) warrants the baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves all test results for a test run, using the specific verb 'Get' and resource 'test results'. It distinguishes itself from sibling tools like get_test_run (which retrieves run metadata) and add_test_results (which adds results).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is provided on when to use this tool versus alternatives. The purpose implies it is for retrieving results after a run, but there are no direct comparisons, exclusions, or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_runB
Get a specific test run by ID
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the action 'Get a specific test run by ID' without mentioning read-only nature, return format, error handling, or permissions, leaving the agent without critical context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with zero filler or redundancy. It is appropriately sized for a simple get-by-ID tool and is front-loaded with the action verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool with no output schema, the description is minimal but adequate in stating the core operation. However, it omits any detail about return value, error conditions, or usage context, leaving some ambiguity given the lack of output schema and annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already fully describes run_id as 'The ID of the test run' (100% schema description coverage). The description's mention of 'by ID' adds no additional parameter semantics beyond the schema, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a specific test run by ID' uses a specific verb (Get) and clearly identifies the resource (test run) and selection method (by ID). This distinguishes it from sibling list_test_runs, which retrieves a collection, and other tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs. alternatives like list_test_runs. It does not state that this is for retrieving a single known run while list_test_runs is for browsing multiple runs, nor does it mention any prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_projectsA
Get all TestRail projects
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does not disclose pagination, ordering, permissions, or any side effects (though likely none). 'Get all' is simple, but the absence of additional behavioral context leaves the agent uncertain about response shape or system limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, directly front-loaded sentence with no filler or repetition. Every word adds meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 0-param list tool with no output schema, the description is largely sufficient. It clearly states resource and scope. It could mention return type or hidden constraints, but the simplicity makes it near-complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema is trivially complete. The description adds no parameter info because none is needed. Baseline 4 is appropriate given the 0-param context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all TestRail projects' clearly states a specific action (get) on a specific resource (projects) with a scope ('all'). It strongly distinguishes from siblings like get_project (singular) and list_suites (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: call this to retrieve all projects. However, it does not explicitly contrast with get_project or mention when to choose this over other listing tools. The guidance is inferred from the resource naming but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_suitesA
Get all test suites for the configured project
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the operation without mentioning potential return structure, ordering, pagination, or any assumptions about the 'configured project'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. Every word contributes to understanding the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list operation with no output schema, the description is reasonably complete. It could clarify what 'configured project' means, but the core operation is unambiguously stated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so per rubric the baseline is 4. The description adds no parameter-specific meaning because none exist; the schema confirms an empty property set.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb ('Get') and resource ('all test suites'), scoped to 'the configured project'. It distinguishes from sibling tools like get_suite (singular) and list_test_cases (different resource type).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as get_suite for a single suite or list_test_cases for test cases. The description implies usage but does not specify exclusions or preferred contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_casesA
Get test cases for the configured project. Optionally filter by suite_id.
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | No | Optional: Filter by suite ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It does not explicitly state that the operation is read-only, mention pagination or ordering, or describe what the response contains. 'Get' implies a read, but the description lacks details beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences, front-loaded with the core purpose and followed by the optional filter. Every word earns its place; no unnecessary details or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and no output schema or annotations, the description is fairly complete. It states the scope and filter, and the siblings provide context for when to use it. It could mention return format or pagination, but these are minor gaps given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single parameter (suite_id with description), so baseline is 3. The description repeats 'Optionally filter by suite_id' without adding any extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get test cases') and the resource ('for the configured project'), distinguishing it from siblings like get_test_case (singular) and list_suites (suites). The optional filter by suite_id adds specificity without confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool (listing test cases for the configured project) and mentions the optional filter. However, it does not explicitly exclude alternatives like get_test_case for individual retrieval, though the naming implies the distinction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_runsA
Get test runs for the configured project. Optionally filter by suite_id.
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | No | Optional: Filter by suite ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the basic list/filter behavior but does not mention whether it is read-only, how pagination works, what fields are returned, or any authentication/prerequisite expectations. For a list operation, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded: it states the primary function in the first sentence and the optional parameter in the second. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description is minimally adequate but lacks important context: what the returned test runs look like, any default behavior, and when to prefer this over get_test_run. Sibling information is available but not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the description restates the only parameter ('Optionally filter by suite_id') without adding new meaning. The schema already provides the type and description, so the description adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action ('Get test runs') and the resource ('for the configured project'). It distinguishes the tool from siblings like list_projects and list_suites, and the optional filter by suite_id adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving multiple test runs, but does not explicitly contrast with get_test_run or other siblings. The mention of 'configured project' gives some context, but no exclusions or alternative recommendations are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_test_caseD
Update a test case with new information
| Name | Required | Description | Default |
|---|---|---|---|
| refs | No | Optional: References (e.g., Jira ticket IDs) | |
| title | No | Optional: New title for the test case | |
| case_id | Yes | The ID of the test case to update | |
| priority_id | No | Optional: Priority ID (1=Low, 2=Medium, 3=High, 4=Critical) | |
| custom_automation_id | No | Optional: Automation ID (e.g., test file path or identifier) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of explaining behavioral traits. It does not disclose whether updates are partial or full replacement, how non-existing case IDs are handled, or any error/response behavior. The description is purely declarative with no transparency into side effects or limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, but it is under-specified and does not earn its place. It adds no information beyond the tool name. This is under-specification rather than concise, valuable content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description should explain update semantics, error conditions, and response handling. It provides none of this. The schema covers parameters but not behavioral context, leaving the tool significantly incomplete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents each parameter (e.g., optional nature, priorities, automation ID). The description adds no extra parameter information. Baseline is 3 because the schema does the heavy lifting, and the description neither enhances nor contradicts it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a test case with new information' essentially restates the tool name 'update_test_case' without adding specific scope or behavioral detail. It does not mention which fields can be updated or how it differs from other tools. While it is a clear verb+resource, the lack of any additional context makes it tautological.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as creating a test run or getting test case details. There is no mention of prerequisites, typical use cases, or when not to use it. The sibling list is not leveraged to differentiate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
13 tool updates
v1.0.0- First observed
add_test_results - First observed
close_test_run - First observed
create_test_run - First observed
get_project - First observed
get_suite - First observed
get_test_case - First observed
get_test_results - First observed
get_test_run - First observed
list_projects - First observed
list_suites - First observed
list_test_cases - First observed
list_test_runs - First observed
update_test_case
TDQS
Scored across 13 tools
Each tool targets a distinct resource (project, suite, test case, test run, test result) with a clear action (list, get, create, update, close, add). There is no overlap between list/get variants or between action verbs.
Tool names consistently follow a list_/get_ prefix for read operations and specific action verbs (create, add, update, close) for mutations, all in lowercase snake_case. The pattern is uniform and predictable.
13 tools is well within the ideal range for a domain-specific server, covering all major TestRail entities (projects, suites, test cases, runs, results) without unnecessary redundancy or bloat.
The core workflow of creating runs, adding results, and closing runs is covered, but there are notable gaps: no create/delete for test cases or suites, and no generic get_project by ID. This limits full lifecycle management of test assets.
Maintenance
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Persistent context for Claude. Your AI always knows your projects and next actions across sessions.
Persistent memory for Claude Code and Cursor. Stop re-explaining your project every session.
- platform7nOAuthtech.p7n
Connect Claude to your Platform7n workspaces — chat, links, and tasks. One-click OAuth.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.1812 npmMIT
- AlicenseBqualityDmaintenanceEnables AI assistants to interact directly with TestRail instances for managing test projects, suites, cases, runs, results, plans, milestones, and attachments through the TestRail API with secure authentication.77137 npm1MIT
- AlicenseAqualityFmaintenanceEnables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.42743 npm44MIT
- AlicenseAqualityAmaintenanceConnect your AI coding assistant to TestRail to manage test cases, runs, results, and more directly from VS Code, Cursor, Claude Desktop, or Claude Code.519 npm1MIT