Skip to main content
Glama
jithinjosejacob

Xray MCP Server

by jithinjosejacob
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
TypeScript
Remote

Xray MCP Server

A Model Context Protocol (MCP) server for integrating with Xray Test Management. This server enables AI assistants like Claude to interact with Xray, supporting test execution and importing test results from various formats.

Features

  • Dual Deployment Support: Works with both Xray Cloud and Xray Server/Data Center

  • Test Execution: Create and execute test runs directly from AI conversations

  • Multiple Import Formats: Import test results from JUnit, Cucumber, Xray JSON, Robot Framework, and TestNG

  • Query & Management: Query test executions, retrieve test information, and manage test plans

  • Type-Safe: Built with TypeScript for robust type checking and IDE support

Related MCP server: Xray MCP Server

Installation

# Clone the repository
git clone <repository-url>
cd xray-mcp-server

# Install dependencies
npm install

# Build the project
npm run build

Configuration

Environment Variables

Copy .env.example to .env and configure based on your Xray deployment:

For Xray Cloud

XRAY_DEPLOYMENT=cloud
XRAY_CLOUD_CLIENT_ID=your_client_id
XRAY_CLOUD_CLIENT_SECRET=your_client_secret

Get your API credentials from: https://xray.cloud.getxray.app/api-keys

For Xray Server/Data Center

Using Personal Access Token (Recommended)

XRAY_DEPLOYMENT=server
XRAY_JIRA_BASE_URL=https://your-jira-instance.com
XRAY_AUTH_TYPE=token
XRAY_TOKEN=your_personal_access_token

Using Basic Authentication

XRAY_DEPLOYMENT=server
XRAY_JIRA_BASE_URL=https://your-jira-instance.com
XRAY_AUTH_TYPE=basic
XRAY_USERNAME=your_username
XRAY_PASSWORD=your_password

MCP Client Configuration

Add this server to your MCP client configuration (e.g., Claude Desktop):

MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "xray": {
      "command": "node",
      "args": ["/absolute/path/to/xray-mcp-server/dist/index.js"],
      "env": {
        "XRAY_DEPLOYMENT": "cloud",
        "XRAY_CLOUD_CLIENT_ID": "your_client_id",
        "XRAY_CLOUD_CLIENT_SECRET": "your_client_secret"
      }
    }
  }
}

Or if you want to use the .env file, ensure it's in the working directory:

{
  "mcpServers": {
    "xray": {
      "command": "node",
      "args": ["/absolute/path/to/xray-mcp-server/dist/index.js"],
      "cwd": "/absolute/path/to/xray-mcp-server"
    }
  }
}

Available Tools

1. import_test_results

Import automated test execution results from various formats.

Supported Formats:

  • junit - JUnit XML format

  • cucumber - Cucumber JSON format

  • xray-json - Xray native JSON format

  • robot - Robot Framework XML format

  • testng - TestNG XML format

Example:

Import JUnit test results for project DEMO with test plan DEMO-100:
- Format: junit
- Project: DEMO
- Test Plan: DEMO-100
- Results: <xml content here>

2. execute_tests

Create and execute a test run in Xray for specified test cases.

Example:

Execute tests DEMO-1, DEMO-2, and DEMO-3 with:
- Test Plan: DEMO-100
- Environments: Chrome, Production
- Summary: Regression Test Run - Sprint 5

3. query_test_executions

Query and filter test executions in Xray.

Example:

Find all test executions in project DEMO:
- Created after: 2024-01-01
- Status: FAIL
- Limit: 20

4. get_test_info

Retrieve detailed information about a specific test case.

Example:

Get information for test case DEMO-123

5. create_test_execution

Create a new test execution container in Xray.

Example:

Create a test execution in project DEMO:
- Summary: API Regression Test - Release 2.0
- Description: Testing all API endpoints for release 2.0
- Test Plan: DEMO-100
- Environments: QA, Staging

6. update_test_execution

Update status and details of a test within an execution.

Example:

Update test DEMO-5 in execution DEMO-200:
- Status: PASS
- Comment: All assertions passed successfully

7. get_test_plans

List test plans in a project.

Example:

Get test plans for project DEMO, limit 10

8. associate_tests_to_execution

Add test cases to an existing test execution.

Example:

Associate tests DEMO-10, DEMO-11, DEMO-12 to execution DEMO-200

Usage Examples

Importing JUnit Results

<!-- example-junit.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="Test Suite" tests="2" failures="1" errors="0" time="5.123">
  <testcase classname="com.example.LoginTest" name="testSuccessfulLogin" time="2.5">
  </testcase>
  <testcase classname="com.example.LoginTest" name="testInvalidPassword" time="2.623">
    <failure message="Expected error message not displayed">
      AssertionError: Expected error message not displayed
    </failure>
  </testcase>
</testsuite>

Then in your AI conversation:

Import these JUnit test results to project DEMO:
- Format: junit
- Project: DEMO
- Test Plan: DEMO-100
- Results: <paste XML content>

Executing Tests

Execute the following tests:
- DEMO-1: Login with valid credentials
- DEMO-2: Login with invalid credentials
- DEMO-3: Password reset flow

Associate with test plan DEMO-100 and run in Chrome environment

Querying Test Executions

Find all failed test executions in project DEMO from the last week

API Coverage

Feature

Xray Cloud

Xray Server/DC

Import JUnit

✅

✅

Import Cucumber

✅

✅

Import Xray JSON

✅

✅

Import Robot

✅

✅

Execute Tests

✅

✅

Query Executions

✅ (GraphQL)

✅ (JQL)

Get Test Info

✅

✅

Create Execution

✅

✅

Update Test Run

✅

✅

Get Test Plans

✅

✅

Associate Tests

✅

✅

Architecture

The server uses a multi-client architecture:

┌─────────────────┐
│   MCP Server    │
└────────┬────────┘
         │
         ├──────────────┐
         │              │
    ┌────▼────┐    ┌────▼────┐
    │  Cloud  │    │ Server  │
    │ Client  │    │ Client  │
    └────┬────┘    └────┬────┘
         │              │
         │              │
    Xray Cloud    Xray Server/DC

  • XrayClient Interface: Common interface for both clients

  • CloudClient: Implements Xray Cloud API v2 with GraphQL

  • ServerClient: Implements Xray Server/DC REST API v1

  • Factory Pattern: Automatically creates the correct client based on configuration

Development

Running in Development Mode

npm run dev

Building

npm run build

Project Structure

xray-mcp-server/
├── src/
│   ├── index.ts                 # Entry point
│   ├── server.ts                # MCP server setup
│   ├── config/                  # Configuration management
│   ├── xray/                    # Xray API clients
│   │   ├── client.ts            # Client interface
│   │   ├── cloud-client.ts      # Cloud implementation
│   │   ├── server-client.ts     # Server implementation
│   │   ├── factory.ts           # Client factory
│   │   └── types.ts             # Type definitions
│   ├── tools/                   # MCP tool definitions
│   └── utils/                   # Utilities
├── dist/                        # Compiled output
├── package.json
├── tsconfig.json
└── .env                         # Configuration (not in git)

Troubleshooting

Authentication Errors

Error: Authentication failed. Please check your credentials.

Solution:

Connection Errors

Error: Network error: Unable to reach Xray API

Solution:

  • Check your XRAY_JIRA_BASE_URL is correct (for Server deployment)

  • Verify network connectivity to Xray

  • Check if a proxy or firewall is blocking the connection

Invalid Test Key Format

Error: Invalid test key format

Solution: Test keys must follow the format PROJECT-123 where PROJECT is the project key and 123 is the issue number.

Import Failures

Error: Import fails with validation errors

Solution:

  • Verify the XML/JSON format is correct for the specified format type

  • Check that the project key exists in Jira

  • Ensure test keys referenced in results exist (or use autoCreateTests: true for Cloud)

Missing Dependencies

Error: TypeScript errors or missing modules

Solution: Run npm install to ensure all dependencies are installed.

Contributing

Contributions are welcome! Please ensure:

  • Code follows the existing style

  • TypeScript types are properly defined

  • Error handling is comprehensive

  • Documentation is updated for new features

License

MIT

Resources

Support

For issues and questions:

  • Check the troubleshooting section above

  • Review Xray API documentation

  • Open an issue in the repository

Available Tools

8 tools
associate_tests_to_executionC

Add test cases to an existing test execution

ParametersJSON Schema
NameRequiredDescriptionDefault
executionKeyYesTest execution key
testKeysYesArray of test keys to add

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It states the action is 'Add test cases' which implies a mutation operation, but doesn't address permissions, whether this is idempotent, what happens if test keys already exist in the execution, or any rate limits. The description is minimal and lacks behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's appropriately sized for this tool and front-loads the core purpose immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after adding tests (e.g., execution status changes), what the response looks like, or potential error conditions. Given the complexity of modifying test executions, more context would be helpful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters ('executionKey' and 'testKeys'). The description adds no additional meaning beyond what the schema provides - it doesn't explain format expectations, constraints, or relationships between parameters. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add test cases') and target resource ('to an existing test execution'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'update_test_execution' or 'execute_tests', which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'update_test_execution' or 'execute_tests'. It mentions 'existing test execution' as a prerequisite but doesn't specify exclusions or contextual usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_executionC

Create a new test execution container in Xray

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyYesJira project key
summaryYesExecution summary/title
descriptionNoOptional: Detailed description
testPlanKeyNoOptional: Test plan key to associate with
testEnvironmentsNoOptional: Execution environments

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Create a new test execution container', implying a write operation, but doesn't mention permissions, side effects, error handling, or response format. This is inadequate for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and appropriately sized, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a creation tool with no annotations and no output schema, the description is incomplete. It lacks information on behavioral traits, return values, and usage context, which are critical for an agent to invoke the tool correctly in a real-world scenario.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning all parameters are documented in the schema. The description doesn't add any additional meaning beyond what's in the schema, such as explaining relationships between parameters or usage examples. Baseline 3 is appropriate when the schema handles parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Create' and the resource 'new test execution container in Xray', which specifies what the tool does. However, it doesn't differentiate from sibling tools like 'update_test_execution' or 'associate_tests_to_execution', which would require mentioning creation versus modification or association.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't specify if this is for initial setup versus updates, or when to choose this over 'update_test_execution' or 'associate_tests_to_execution'. This lack of context leaves the agent without clear usage instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

execute_testsC

Create and execute a test run in Xray for specified test cases

ParametersJSON Schema
NameRequiredDescriptionDefault
testKeysYesArray of test issue keys (e.g., ["PROJ-123", "PROJ-124"])
testPlanKeyNoOptional: Test plan key to associate execution with
testEnvironmentsNoOptional: Test environments (e.g., ["Chrome", "Production"])
summaryNoOptional: Summary/title for the test execution
descriptionNoOptional: Detailed description of the test execution

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Create and execute' implies a write/mutation operation, the description doesn't address critical aspects like required permissions, whether this is an atomic operation, what happens on failure, rate limits, or what the output looks like (especially since there's no output schema). This leaves significant gaps for a tool that performs creation operations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded with the core functionality, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool that creates and executes test runs (a mutation operation) with 5 parameters and no output schema, the description is insufficient. It lacks information about behavioral traits (permissions, side effects), doesn't differentiate from sibling tools, and provides no guidance on usage context. The 100% schema coverage helps with parameters, but overall completeness is poor for this type of operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter information beyond what's already in the schema descriptions (e.g., it doesn't explain relationships between parameters or provide examples beyond the schema). This meets the baseline expectation when schema coverage is complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create and execute a test run') and the resource ('in Xray for specified test cases'), providing a specific verb+resource combination. However, it doesn't distinguish this tool from sibling tools like 'create_test_execution' or 'associate_tests_to_execution', which likely have overlapping functionality in the Xray test management context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'create_test_execution' and 'associate_tests_to_execution' available, there's no indication of the specific scenarios, prerequisites, or differences that would help an agent choose this tool over others in the test execution workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_infoC

Retrieve detailed information about a specific test case

ParametersJSON Schema
NameRequiredDescriptionDefault
testKeyYesTest issue key (e.g., "PROJ-123")

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Retrieve' implies a read-only operation, it doesn't specify aspects like authentication requirements, rate limits, error handling (e.g., what happens if the test key is invalid), or the format of the returned information. For a tool with zero annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose ('Retrieve detailed information about a specific test case') with zero wasted words. It's appropriately sized for a simple tool with one parameter and no complex behavioral nuances to explain.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete for a tool that retrieves 'detailed information'. It doesn't hint at what information is returned (e.g., test status, steps, attachments) or potential side effects. While the schema covers the single parameter well, the overall context for the agent to use this tool effectively is lacking.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the 'testKey' parameter clearly documented as a 'Test issue key (e.g., "PROJ-123")'. The description adds no additional parameter details beyond what the schema provides, such as examples of valid keys or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Retrieve') and resource ('detailed information about a specific test case'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_test_plans' or 'query_test_executions', which might also retrieve test-related information but with different scopes or filters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid test key), exclusions, or comparisons to siblings like 'get_test_plans' (which might retrieve broader test plan info) or 'query_test_executions' (which might filter executions). This leaves the agent to infer usage from context alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_plansC

List test plans in a project

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyYesJira project key
limitNoOptional: Maximum number of results (default: 50)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'List test plans' but does not mention any behavioral traits like pagination, sorting, error handling, or rate limits. This is a significant gap for a tool with no annotation coverage, making it unclear how the tool behaves beyond basic listing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action ('List test plans') and context ('in a project'), making it easy to parse quickly. Every part of the description earns its place by conveying essential information succinctly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It does not explain what the tool returns (e.g., list format, fields included) or any behavioral aspects like pagination or errors. For a tool with no structured data beyond the input schema, this leaves critical gaps in understanding how to use it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, fully documenting both parameters ('projectKey' and 'limit'). The description does not add any meaning beyond what the schema provides, such as explaining the format of 'projectKey' or typical use cases for 'limit'. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and resource ('test plans in a project'), making the tool's purpose understandable. However, it does not differentiate from sibling tools like 'get_test_info' or 'query_test_executions', which might also retrieve test-related data, leaving some ambiguity about its specific scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It lacks context such as whether this is for browsing all test plans, if it's the primary retrieval method, or how it differs from siblings like 'get_test_info' or 'query_test_executions', leaving the agent without usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

import_test_resultsC

Import automated test execution results from various formats (JUnit, Cucumber, Xray JSON, Robot Framework, TestNG)

ParametersJSON Schema
NameRequiredDescriptionDefault
formatYesFormat of the test results to import
resultsYesTest results content as XML or JSON string
projectKeyYesJira project key (e.g., "PROJ")
testPlanKeyNoOptional: Test plan key to associate results with (e.g., "PROJ-123")
testEnvironmentsNoOptional: Test environments (e.g., ["Chrome", "Production"])
testExecKeyNoOptional: Existing test execution key to update

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a write operation ('import') but doesn't specify permissions needed, whether it creates or updates records, error handling, or side effects. This is a significant gap for a tool with multiple parameters and no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action and lists formats without unnecessary elaboration. Every word earns its place, making it easy to scan and understand quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It lacks behavioral context, usage differentiation from siblings, and details on what happens after import (e.g., does it create test executions?). More information is needed to guide effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning beyond listing format names, which are already in the enum. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('import') and resource ('automated test execution results'), and lists specific formats (JUnit, Cucumber, etc.), making the purpose explicit. However, it doesn't distinguish this tool from sibling tools like 'create_test_execution' or 'update_test_execution', which might handle similar test-related operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'create_test_execution' or 'update_test_execution'. The description only lists supported formats without indicating context or prerequisites, leaving the agent to guess based on parameter names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_test_executionsC

Query and filter test executions in Xray

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyYesJira project key
testPlanKeyNoOptional: Filter by test plan key
statusNoOptional: Filter by execution status
startDateNoOptional: Filter executions created after this date (ISO 8601 format)
endDateNoOptional: Filter executions created before this date (ISO 8601 format)
limitNoOptional: Maximum number of results to return (default: 50)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It mentions 'query and filter' which implies a read-only operation, but doesn't address pagination behavior (beyond the limit parameter), rate limits, authentication requirements, or what happens when no results match filters. For a tool with 6 parameters and no annotation coverage, this is inadequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise at just 6 words ('Query and filter test executions in Xray'). Every word earns its place by establishing the core action and resource. There's no wasted language, repetition, or unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a query tool with 6 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what information is returned, how results are structured, whether the tool supports pagination beyond the limit parameter, or typical response formats. The agent would need to guess about the tool's behavior and outputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so all parameters are well-documented in the schema itself. The description adds no additional parameter semantics beyond the generic 'query and filter' context. This meets the baseline expectation when schema documentation is comprehensive, but doesn't provide extra value like explaining relationships between parameters or special constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('query and filter') and resource ('test executions in Xray'), making it immediately understandable. However, it doesn't explicitly differentiate this query/filter tool from sibling tools like 'get_test_info' or 'get_test_plans', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_test_info' and 'get_test_plans' that might retrieve similar information, the agent receives no help in choosing between them. There's no mention of prerequisites, typical use cases, or exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_executionC

Update status and details of a test within an execution

ParametersJSON Schema
NameRequiredDescriptionDefault
executionKeyYesTest execution key (e.g., "PROJ-456")
testKeyYesTest key to update (e.g., "PROJ-123")
statusYesTest status
commentNoOptional: Comment about the execution
defectsNoOptional: Associated defect keys

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions updating 'status and details' but does not cover critical aspects like required permissions, whether the update is reversible, potential side effects (e.g., if it triggers notifications or workflows), or error handling. This leaves significant gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It is front-loaded with the core action and target, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a mutation tool with no annotations and no output schema, the description is insufficient. It lacks information on behavioral traits (e.g., auth needs, side effects), usage context relative to siblings, and what the tool returns. This makes it incomplete for safe and effective agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents all parameters, including descriptions and enum values for 'status'. The description adds no additional parameter semantics beyond implying that 'details' might include 'comment' and 'defects', but this is already clear from the schema. Baseline score of 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update') and the target ('status and details of a test within an execution'), which is specific and actionable. However, it does not explicitly differentiate from sibling tools like 'execute_tests' or 'import_test_results', which might also involve test execution updates, leaving some ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. For example, it does not specify if this is for manual updates, post-execution status changes, or how it differs from 'execute_tests' or 'import_test_results'. This lack of context makes it harder for an agent to choose correctly among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 8 tool updatesv1.0.0
    • First observedassociate_tests_to_execution
    • First observedcreate_test_execution
    • First observedexecute_tests
    • First observedget_test_info
    • First observedget_test_plans
    • First observedimport_test_results
    • First observedquery_test_executions
    • First observedupdate_test_execution
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.4/5.0
Disambiguation4/5

Most tools have distinct purposes focused on different aspects of test management (creation, querying, execution, import), but there is some potential overlap between 'create_test_execution' and 'execute_tests' where the latter might be confused as a superset of the former. The descriptions help clarify that 'execute_tests' specifically creates and runs tests, while 'create_test_execution' is just a container, but the naming could cause mild confusion.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with snake_case (e.g., 'associate_tests_to_execution', 'create_test_execution', 'get_test_info'). There are no deviations in naming conventions, making the set predictable and easy to parse for an agent.

Tool Count5/5

With 8 tools, the server is well-scoped for test management in Xray, covering key operations like creation, querying, execution, and import without being overly complex or sparse. Each tool appears to serve a specific function that contributes to the domain's workflow.

Completeness4/5

The tool set provides strong coverage for test execution and management, including CRUD-like operations (create, query, update) and import capabilities. A minor gap is the lack of tools for deleting or archiving test executions or plans, which might limit full lifecycle management, but agents can likely work around this with existing tools.

Maintenance

ActivityInactive
ResponsivenessUnresponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • TestRail MCP Server

    Derrbal
    A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.
    18
    12
    MIT

  • Xray MCP Server

    c4m3lblue-star
    A
    license
    B
    quality
    D
    maintenance
    Enables integration with Xray Cloud APIs for comprehensive test management including creating and managing test cases, test executions, test plans, and test sets. Supports CI/CD automation and test result tracking through GraphQL APIs.
    23
    67
    3
    ISC

  • TestRail MCP Server

    denys-ispaniuk
    A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact with TestRail test management system, supporting full CRUD operations on projects, suites, sections, test cases, runs, results, plans, and milestones.
    35
    2,326
    1
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/jithinjosejacob/xray-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server