TestDino MCP
The TestDino MCP server connects AI agents to the TestDino test management platform, enabling natural language interaction with test data and management functions.
Connection & Account
health: Verify your PAT, view account info, and list available organizations and projects.
Test Execution & Results
list_testruns: Browse and filter test runs by branch, time interval, author, commit, or environment with pagination.get_run_details: Retrieve details for one or more test runs (batch up to 20), including statistics, suites, and metadata.list_testcase: Find test cases filtered by status, browser, error category, tags, runtime, artifacts, spec file, and more.get_testcase_details: Fetch full details of a test case including error messages, stack traces, steps, console logs, and artifacts (screenshots, videos, traces).
AI-Assisted Debugging
debug_testcase: Aggregate historical failure data across multiple executions to identify failure patterns, flakiness, and browser-specific issues โ with a pre-formatted AI debugging prompt.
Manual Test Case Management
list_manual_test_cases: Search and filter manual test cases by status, priority, severity, type, layer, behavior, tags, automation status, and more.get_manual_test_case: Retrieve full details including steps, preconditions, postconditions, and custom fields.create_manual_test_case: Create new manual test cases with steps, metadata, preconditions, and postconditions.update_manual_test_case: Modify any fields of an existing manual test case.
Test Suite Organization
list_manual_test_suites: Browse the test suite hierarchy to navigate and find suite IDs.create_manual_test_suite: Create new test suite folders (including nested suites) to organize test cases.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TestDino MCPshow me the latest test run results for the staging environment"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A Model Context Protocol (MCP) server that connects TestDino to AI agents. This server enables you to interact with your TestDino test data directly through natural language commands.
Quick Start
Add this to your MCP client config (~/.cursor/mcp.json for Cursor), then restart the client:
{
"mcpServers": {
"TestDino": {
"command": "npx",
"args": ["-y", "@testdino/mcp"],
"env": {
"TESTDINO_PAT": "your-pat-here"
}
}
}
}Get your PAT from TestDino โ User Settings โ Personal Access Tokens, then ask your assistant: "Check TestDino health." You should see your account, organizations, and projects. Full setup for Cursor and Claude Desktop is in Integration.
Related MCP server: Testmo MCP Server
What is This?
This MCP server bridges the gap between your TestDino test management platform and AI agents. Instead of manually navigating the TestDino dashboard, you can ask your AI assistant to:
Check test run results
Analyze test failures
Get detailed test case information
Manage manual test cases and suites
Plan releases, spin up manual test runs, and assign testers
Run exploratory testing sessions
Record per-case verdicts (passed / failed / blocked / etc.) from chat
All through simple conversational commands.
Features
๐ Health Check: Verify your connection and validate your TestDino PAT. Get account information and list all available organizations and projects.
๐ Test Run Management: List and retrieve detailed information about your test runs with filtering options (branch, time, author, commit, environment).
๐งช Test Case Analysis: Get comprehensive details about individual test cases, including errors, logs, execution steps, and artifacts (screenshots, videos, traces).
๐ AI-Assisted Debugging: Debug test case failures with historical data aggregation, failure pattern analysis, and AI-friendly debugging prompts. Analyze patterns across multiple executions to identify root causes.
๐งญ Test Quality Audit: Fetch a server-curated audit prompt plus branch signals, analyze your local test code, and store the completed report back in TestDino without uploading raw source files.
๐ Test Case Management: Create, update, list, and retrieve manual test cases with comprehensive filtering and organization (status, priority, severity, type, layer, behavior, tags).
๐ Test Suite Organization: Create and manage test suite hierarchies to organize your manual test cases.
๐ Release Planning: Create, browse, and update releases (a.k.a. milestones), nest them up to 3 levels deep, and track rolled-up progress stats across all runs in a release.
โถ๏ธ Manual Test Runs: Spin up runs scoped to specific suites or the whole project, attach them to a release, and update workflow state without leaving chat.
โ Per-case Assignment & Verdicts: Inside a run, assign each test case to a tester (by User _id or email) and set the result โ
passed,failed,blocked,skipped,retestโ exactly as the UI does.๐ญ Exploratory Sessions: Create exploratory testing sessions with mission/charter, assign a tester, link to a release, and track state.
๐งฉ Failure Triage at Scale: Group a run's failing tests by their underlying error signature to see which reds are one root cause versus many, so you fix the real problem first.
๐ Issue Tracker Integrations: Connect Jira, Linear, Asana, or monday.com and file a tracked issue straight from a failing test or run, then check its status later without leaving chat.
๐ MCP Compatible: Built on the Model Context Protocol standard. You can configure TestDino MCP with any MCP-compatible IDEs or AI agents (Cursor, Claude Desktop, etc.).
โก Easy Setup: Install and configure in minutes with npx. No installation required!
๐ Secure: PAT stored securely in your local configuration. One PAT provides access to all organizations and projects you have permissions for.
Available Tools
The server provides 35 tools across nine domains:
Test Execution & Results:
health- Verify your connection and validate your PAT. Shows account information, available organizations, and projects with access permissions.list_testruns- Browse test runs with filters (branch, time interval, author, commit, environment). Supports pagination and batch operations.get_run_details- Get comprehensive details about a specific test run including statistics, test suites, test cases, and metadata. Supports batch operations (comma-separated IDs, max 20).list_testcase- List test cases with comprehensive filtering (by test run, status, browser, error category, branch, environment, commit, author, spec file, tags, runtime, artifacts, and more). Can filter by test run criteria or directly by test case properties.get_testcase_details- Get detailed information about a specific test case including error messages, stack traces, test steps, console logs, and artifacts. Can identify by testcase_id alone or by testcase_name with testrun_id/counter.debug_testcase- Debug a test case by aggregating historical failure data across multiple executions. Returns failure patterns, error categories, common error messages, error locations, browser-specific issues, and a pre-formatted debugging prompt for AI analysis. Perfect for root-cause analysis and identifying flaky test behavior.get_audit_report- Fetch the audit context (server-curated prompt + top failing / flaky / slow tests for the branch) to start a new audit, browse historical reports, or retrieve one byreportId.submit_audit_report- Submit a completed audit report (score, findings, recommendations, markdown) to TestDino.
Test Case Management:
list_manual_test_cases- Search and list manual test cases with comprehensive filtering (project, suite, status, priority, severity, type, layer, behavior, automation status, tags, flaky status).get_manual_test_case- Get detailed information about a specific manual test case including steps, custom fields, preconditions, postconditions, and all metadata.create_manual_test_case- Create new manual test cases with steps, preconditions, postconditions, and metadata (priority, severity, type, layer, behavior).update_manual_test_case- Update existing manual test cases (title, description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions).list_manual_test_suites- List test suite hierarchy to find suite IDs for organization. Supports filtering by parent suite.create_manual_test_suite- Create new test suite folders to organize test cases. Supports nested suites by providing parentSuiteId.
Releases (a.k.a. Milestones):
list_releases- Browse releases for a project with filters (search, type, completion status, parent release). Releases group runs + sessions and can nest up to 3 levels deep.get_release- Get full details for one release including dates, status, parent/root hierarchy, and rolled-up progress stats across all runs in this release and its descendants. Accepts internal_idor counter-style ID likeMS-12.create_release- Create a new release with name, type, dates, and optional parent for nesting.update_release- Modify an existing release โ name, dates, completion flags, type, linked issues. Closed releases are still editable.
Manual Test Runs:
list_manual_runs- Browse manual runs in a project. Filter by status, state, environment, release, tags, or free-text name search.get_manual_run- Get full details for one run โ test stats (total/passed/failed/blocked/untested), contributors, attachments, linked release. Accepts internal_idor counter-style ID likeRUN-12.create_manual_run- Create a new manual run. ChooseselectionMode='all'for every case in the project, or'selected'with suite/case IDs to scope it. Attach to a release withreleaseId.update_manual_run- Modify run metadata โ name, environment, state, release attachment, tags. Closed runs are read-only except forreleaseId.list_run_test_cases- Get the per-case execution rows inside a run โ exactly what the UI shows in the run's test-case table. Each row includes the current assignee and current result. Filter by assignee (email or _id) or result.update_run_test_case- Set the assignee and/or result for one test case inside a run โ same as clicking "Assign to" + the result pill in the UI. Works even on untested cases (auto-creates the per-case row on first edit). Accepts caseKey (TC-156), test case _id, or the internaltcm_rtc_โฆID.
Exploratory Sessions:
list_sessions- Browse exploratory sessions in a project. Filter by status, state, sessionType, assignee (email or _id), release, tags.get_session- Get full details for one session โ name, mission, status, assignee, linked release, findings. Accepts internal_idor counter-style ID likeSES-12.create_session- Create a new exploratory session with mission/charter, sessionType, assignee, estimate, and optional release attachment.update_session- Modify session metadata โ name, mission, assignee, state, estimate, linked issues, attachments.
Error Analysis:
get_run_error_clusters- Group a run's failing tests by error signature to triage at scale. Surfaces clusters of tests that share the same root-cause error, with an optionalstatusfilter (all,failed,flaky). Use it afterlist_testrunsto understand why a run failed.
Integrations (Issue Trackers):
connect_integration- Return an OAuth connect URL for a provider (Jira, Linear, Asana, monday.com, GitHub). Show the URL to the user to authorize; do not open it programmatically.get_integration_status- Report whether a provider is connected for a project. SetincludeCreateOptionsto also fetch the fields available for issue creation. Call this beforecreate_external_issue.create_external_issue- File an issue in a connected tracker (Jira, Linear, Asana, monday.com) from a TestDino source entity such as a failing test case or run. Supportspreviewand idempotent retries viaidempotencyKey.get_external_issue- Fetch previously created issues by their IDs or keys (one or many) and return their current status in the provider (Jira, Linear, Asana).
Provider support: All providers can be connected and status-checked. Issue creation works with Jira, Linear, Asana, and monday.com. Issue read-back works with Jira, Linear, and Asana. GitHub is supported as a PR/CI integration, not an issue tracker.
AI Insights:
get_ai_insights- TestDino's AI analysis at three levels: project overview (per-category failure counts + top offenders over a date range), run (AI failure categorization, failure clusters, error-analysis table, LLM-written summary), and test case (recommendations + quick fixes). Returns adisabledstatus when AI features are turned off for the project (Settings โ AI).get_trace_analysis- Resolve a failing test's hosted Playwright trace to a short-lived download URL and return a runbook for local trace-CLI debugging (open โ actions โ snapshot โ close).
AI Insights are also available inline: pass
include_ai_insights: truetoget_run_details(attaches the run's AI Insights underai_insights) ordebug_testcase(attaches recommendations + quick fixes underai_fixes).
Installation Options
There are mainly 3 options to use TestDino MCP:
Option 1: Via npx (Recommended - No Installation)
No installation required
Automatically downloads and runs when needed
Always uses the latest version
Configured in any MCP compatible platform
Option 2: Global Installation
npm install -g @testdino/mcpInstall once, use in any project
Requires Node.js 20+ and npm
Use command:
testdino-mcp
Option 3: Project Installation
npm install @testdino/mcpInstalled in your project's
node_modulesUse command:
npx @testdino/mcp
Integration
A simple Integration guide for Cursor IDE:
Step 1: Get Your Personal Access Token (PAT)
Log in to your TestDino account
Navigate to User Settings โ Personal Access Tokens
Generate a new PAT from the Personal Access Tokens section.
Important: This PAT provides access to all organizations and projects you have permissions for
Step 2: Configure Cursor
Open or create the MCP configuration file:
Windows:
%APPDATA%\Cursor\mcp.jsonmacOS/Linux:
~/.cursor/mcp.jsonProject-specific:
.cursor/mcp.jsonin your project root
If you are using npx or installing inside project, Add the configuration:
{
"mcpServers": {
"TestDino": {
"command": "npx",
"args": ["-y", "@testdino/mcp"],
"env": {
"TESTDINO_PAT": "Your PAT here"
}
}
}
}Important: Replace Your PAT here with your actual Personal Access Token (PAT) from Step 1.
Step 3: Restart and Verify
Completely close and restart Cursor
Verify the connection by asking: "Check TestDino health"
You should see your account name, available organizations, and projects!
Alternative: Global Installation
If you prefer to install globally instead of using npx or project installation:
npm install -g @testdino/mcpThen use this configuration:
{
"mcpServers": {
"TestDino": {
"command": "testdino-mcp",
"env": {
"TESTDINO_PAT": "Your PAT here"
}
}
}
}The server uses the standard MCP protocol, so it will work with other MCP-compatible clients as well.
Claude Desktop
Claude Desktop uses the same config shape, in a different file:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"TestDino": {
"command": "npx",
"args": ["-y", "@testdino/mcp"],
"env": {
"TESTDINO_PAT": "your-pat-here"
}
}
}
}Restart Claude Desktop, then ask "Check TestDino health" to verify.
Local development
By default the server talks to https://mcp.testdino.com. To point it at a local API while developing, set TESTDINO_API_URL (see .env.example):
TESTDINO_API_URL=http://localhost:3001Usage
Once configured, simply talk to your AI assistant in natural language. Important: Tools require your Personal Access Token (PAT) configured as TESTDINO_PAT in mcp.json. The PAT automatically provides access to all organizations and projects you have permissions for.
Example Commands
Try these natural language commands in Cursor or Claude Desktop (or other MCP-compatible clients):
Connection & Setup:
"Check if my TestDino connection is working"
"Validate my TestDino PAT"
Exploring Test Runs:
"Show me my last 5 test runs"
"What test runs are on the develop branch?"
"List test runs from the last 3 days"
"Show me test runs by author john"
"Find test runs for commit abc123"
"List all test runs in production environment"
Analyzing Test Results:
"Get details for test run test_run_6901b2abc6b187e63f536a6b"
"Get details for test run counter 42"
"What test cases failed in test run test_run_6901b2abc6b187e63f536a6b?"
"Show me all flaky tests from the last test run"
"List all failed test cases in production environment"
"Show me test cases from the main branch that took more than 100 seconds"
"Find all timeout issues in test cases from commit abc123"
Debugging Test Failures:
"Debug test case 'Verify user login' in project proj_123"
"Debug 'Verify that User Can Complete the Journey from Login to Order Placement @webkit' from testdino reports"
"Analyze failures for 'Checkout flow' test case in project proj_123"
"What are the failure patterns for 'API authentication' test?"
"Why is test case 'User registration' failing?"
"Debug test case 'Verify that user can login and logout successfully @chromium'"
Managing Manual Test Cases:
"List all manual test cases in project proj_123"
"Search for manual test cases with tag 'smoke' in project proj_123"
"Show me all critical priority manual test cases in project proj_123"
"Get details for manual test case TC-123 in project proj_123"
"Create a new manual test case for login feature in suite Authentication Tests"
"Update test case TC-123 to mark it as deprecated"
"List all test suites in project proj_123"
"Create a new test suite called 'Authentication Tests' in project proj_123"
Releases:
"List all releases in project proj_123"
"Show me the open iterations on this project"
"Get details for release MS-12 and show the rolled-up test progress"
"Create a release called 'Sprint 42' as an iteration from May 12 to May 26"
"Mark release MS-12 as completed"
Manual Test Runs:
"List manual runs in release MS-12"
"Show me all in-progress runs on staging"
"Get details for run RUN-7"
"Create a manual run called 'Sprint 42 Smoke' linked to release MS-12 on staging, covering the Authentication suite"
"Set run RUN-7's state to Done"
"Re-attach run RUN-7 to release MS-13"
Assigning Cases & Recording Verdicts in a Run:
"List the test cases in run RUN-1 with their current assignees and results"
"Assign TC-156 in run RUN-1 to alice@company.com and mark it Passed"
"Mark TC-157 in run RUN-1 as Failed"
"Assign TC-158 in run RUN-1 to bob@company.com โ leave the result untested"
"Show me all cases in run RUN-1 assigned to alice@company.com that are still untested"
Exploratory Sessions:
"List active exploratory sessions for project proj_123"
"Show me sessions assigned to tester@company.com"
"Get details for session SES-3"
"Create an exploratory session called 'Auth charter โ May 12' with mission 'find session-handling bugs around 2FA' assigned to tester@company.com, 60 minute estimate"
"Update session SES-3 to mark it Done"
Documentation
Installation Guide: Detailed setup instructions for Cursor, Claude Desktop, and other MCP-compatible clients
Tools Documentation: Comprehensive guide to all 33 available tools with examples, parameters, and use cases
AI Agent Skills Guide: Guide for AI agents on tool selection patterns, decision trees, and best practices
Requirements
Node.js: Version 20.0.0 or higher
NPM: Latest version recommended (for package management)
TestDino Account: Valid account with Test Runs and/or Test Case Management access
Personal Access Token (PAT): Required for authentication. Get it from TestDino Settings โ Personal Access Tokens
Key Features Explained
AI-Assisted Debugging with debug_testcase
The debug_testcase tool helps you understand why tests are failing by analyzing historical execution data:
Historical Analysis: Aggregates data from multiple test runs to identify patterns
Failure Patterns: Identifies common error categories, messages, and locations
Browser-Specific Issues: Detects browser-specific failure patterns
Flaky Test Detection: Analyzes retry patterns and flakiness indicators
AI-Friendly Output: Returns pre-formatted debugging prompts for AI analysis
Code Correlation: Provides file and line numbers for error locations, enabling AI to correlate with source code
Example Workflow:
Ask: "Debug test case 'Verify user login' from testdino reports"
AI calls
debug_testcasewith projectId and testcase_nameTool returns historical data with failure patterns
AI analyzes the data and correlates with your test code (if accessible)
AI provides specific fix suggestions based on patterns and code analysis
Support
Documentation: See docs/TOOLS.md for complete tool documentation
Installation Help: See docs/INSTALLATION.md for detailed setup instructions
TestDino Support: support@testdino.com
TestDino Documentation: https://docs.testdino.com
Available Tools
35 toolsconnect_integrationA
Returns an OAuth connect URL for the requested integration provider (Jira, Linear, Asana, monday.com, GitHub). Show the returned URL to the user โ do NOT open it programmatically. The user must visit it in their browser to authorize the connection. Call get_integration_status first to check whether the provider is already connected; if it is, this returns status already_connected instead of a URL.
| Name | Required | Description | Default |
|---|---|---|---|
| orgId | No | Organization ID. Optional โ derived from your PAT scopes when omitted; pass it explicitly if your PAT spans multiple organizations. | |
| provider | Yes | Integration provider to connect (Required). | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the OAuth URL return, the need for user browser authorization, and the already_connected behavior. Slight gaps remain on side effects or permissions, but the core behavior is well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: three sentences with the main purpose front-loaded, and every sentence adds useful context without fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return value (URL), the user workflow, and the alternative outcome (already_connected). No output schema exists, but for a simple URL-returning tool, this is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds little beyond what's in the schema, though it does list providers matching the enum. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Returns an OAuth connect URL for the requested integration provider' with a specific list of providers. This distinguishes it from sibling tools like get_integration_status and create_external_issue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided: 'Call get_integration_status first to check whether the provider is already connected' and 'Show the returned URL to the user โ do NOT open it programmatically.' This clearly instructs when and how to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_external_issueA
Files an issue in a connected provider (Jira, Linear, Asana, monday.com, GitHub) from a TestDino source entity. source.type and source.id are required โ they identify the TestDino entity (e.g. a test case or test run) the issue is about; the server resolves them into the issue draft. Pass preview: true to see what would be created without creating it. Idempotent when idempotencyKey is supplied โ safe to retry with the same key. Call get_integration_status first (with includeCreateOptions: true to discover the provider fields available for target). If the provider is not connected, this returns INTEGRATION_NOT_CONNECTED with a connect URL โ show that URL to the user, do not open it programmatically.
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | TestDino source entity the issue is about (Required). Both fields are required. | |
| target | No | Provider-specific destination fields (e.g. Jira project key and issue type, monday board ID). Discover available fields via get_integration_status with includeCreateOptions: true. | |
| preview | No | When true, returns the draft that would be created (wouldCreate: false) without filing the issue. | |
| summary | No | Issue title/summary. Optional โ derived from the source entity when omitted. | |
| linkBack | No | When true, the created issue is linked back to the TestDino source entity (currently supported for Jira). | |
| provider | Yes | Integration provider to file the issue in (Required). | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| description | No | Issue description body. Supports plain text. | |
| idempotencyKey | No | Unique key to prevent duplicate issues on retry. Use a stable identifier such as the source entity ID. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for disclosing behavior. It does so thoroughly: preview mode returns a draft without creating (wouldCreate: false), idempotencyKey makes it safe to retry, and the tool returns a specific error code with a connect URL when the provider is not connected. These are meaningful behavioral details beyond simple write semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense without bloat. Every sentence adds value: purpose, required identifiers, preview behavior, idempotency, prerequisite call, and error-handling guidance. It is front-loaded with the main purpose and then logically expands into usage details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 9 parameters and nested objects, the description covers the essential context: what the source is, how to discover target fields, what preview does, idempotency, and error behavior. There is no output schema, so the description partially explains return values (draft, error code). It could be slightly more explicit about the full success response shape, but it is sufficiently complete for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds context that enriches parameter understanding, particularly source.type/source.id being required and resolved server-side into the issue draft, and the purpose of target being discoverable via get_integration_status. It doesn't rename or redefine every parameter, but it clarifies the relationships and usage of key fields enough to justify a score above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with an action verb and specific resource: "Files an issue in a connected provider (Jira, Linear, Asana, monday.com, GitHub) from a TestDino source entity." It clearly differentiates from sibling tools like get_external_issue and connect_integration by specifying the action (file/create) and the source context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs when to use it: "Call get_integration_status first (with includeCreateOptions: true to discover the provider fields available for target)." It also provides guidance for a specific error case (INTEGRATION_NOT_CONNECTED) and tells the agent to show the URL to the user rather than opening it programmatically. This is clear, actionable usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_manual_runA
Create a new manual test run. Requires write permission. selectionMode controls which test cases are included: 'all' (default โ every case in the project) or 'selected' (use testCaseIds and/or suiteIds to scope). releaseId attaches the run to a release. note accepts rich HTML. IMPORTANT: tags must be a JSON array of strings here โ e.g. ["smoke","regression"] โ NOT the comma-separated form that list_manual_runs accepts as a filter.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Run name (required). | |
| note | No | Rich HTML note. | |
| tags | No | Array of tag strings, e.g. ["smoke","regression"]. NOT a comma-separated string. | |
| links | No | Array of link objects. | |
| state | No | Workflow state (default 'new'). Either canonical ('in_progress') or display ('In Progress') form โ server normalizes to lowercase+underscored so UI colors render correctly. | |
| forecast | No | ||
| suiteIds | No | ||
| projectId | Yes | Project ID (required). | |
| releaseId | No | Attach run to this release. | |
| attachments | No | Array of attachment objects or URLs. | |
| environment | No | Environment label, e.g. 'Staging'. | |
| testCaseIds | No | ||
| linkedIssues | No | Array of linked-issue objects (same shape list_manual_runs returns). | |
| selectionMode | No | Default 'all'. | |
| includeUnsorted | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description discloses important behaviors: requires write permission, explains selectionMode options, warns about tags format mismatch with list_manual_runs, and notes state normalization. This adds good context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with 5 sentences covering essential points. It is front-loaded with the primary purpose and permission. While not bulleted, the information density is good and no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 15 parameters and no output schema, the description covers key aspects (permission, selectionMode, tags, state, note) but omits details about return value, some less common parameters (forecast, attachments), and potential error conditions. It is adequate for typical use but incomplete for edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaningful insights for several parameters: explains selectionMode's effect with 'all' vs 'selected', clarifies tags format (JSON array not comma-separated), and describes state normalization. This goes beyond the schema's enum and type info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new manual test run.' with a specific verb and resource. It distinguishes from sibling tools like update_manual_run or list_manual_runs by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool over alternatives (e.g., when to use create_manual_run vs create_manual_test_case). It only mentions a write permission prerequisite, which is necessary but not comparative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_manual_test_caseA
Create a new manual test case. Use this to document new test scenarios, features, or requirements. Supports adding test steps, preconditions, postconditions, and metadata like priority, severity, and type.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Tags to add to your test cases. | |
| type | No | Test case type. | |
| flags | No | Automation flags/checklist options. | |
| layer | No | Test layer. | |
| steps | No | Array of test steps. For Classic format: action, expectedResult, and optional data. For Gherkin format: event and stepDescription. Each top-level step can include attachments as URLs or local file paths. | |
| title | Yes | Test case title (Required). A clear, descriptive title for the test case. | |
| status | No | Test case status. | |
| behavior | No | Test behavior type. | |
| priority | No | Test case priority level. | |
| severity | No | Test case severity level. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| suiteName | Yes | Test suite name (Required). The suite where this test case will be created. Use list_manual_test_suites to find suite names. | |
| attachments | No | Array of attachment URLs or file paths (up to 10MB each). | |
| description | No | Detailed description of what this test case validates. | |
| customFields | No | Custom fields as key-value pairs. Only available if custom fields are configured in test case management settings. | |
| preconditions | No | Prerequisites or setup required before executing this test case. | |
| postconditions | No | Expected state or cleanup actions after executing this test case. | |
| automationStatus | No | Automation status of the test case. | |
| testStepsDeclarationType | No | Type of test steps declaration format. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It states the tool creates a test case, but does not mention any side effects, required permissions, rate limits, or error handling. For a creation tool, this minimal transparency is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the core purpose. Every sentence adds value, and there is no redundancy or wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 19 parameters and no output schema. The description lacks information about what the tool returns upon success (e.g., ID of created test case) or potential errors. Given the complexity, the description is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with detailed parameter descriptions. The description adds marginal value by summarizing supported fields ('test steps, preconditions, postconditions, and metadata'), but does not provide additional meaning beyond what the schema already conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Create a new manual test case') and resource ('manual test case'). It distinguishes from sibling tools like create_manual_run or create_manual_test_suite by specifying it's for documenting test scenarios, features, or requirements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to document new test scenarios, features, or requirements,' which provides clear usage context. However, it does not mention when not to use it or suggest alternatives (e.g., use create_manual_run for executing test cases), so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_manual_test_suiteA
Create a new test suite folder to organize test cases. Use this to create a logical grouping for related test cases. Suites can be nested by providing a parentSuiteId.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Suite name (Required). A descriptive name for the test suite. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| description | No | Description of the test suite. | |
| parentSuiteId | No | Optional parent suite ID to create this suite as a child of another suite. If not provided, creates a root-level suite. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose side effects. It does not mention what the tool returns (no output schema), whether it requires permissions, or if names must be unique. Behavioral context like idempotency or error handling is absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, clear sentences with no wasted words. Front-loaded with the primary purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main purpose and nesting, but fails to explain the return value or any behavioral constraints (e.g., uniqueness of names). Given the lack of output schema and annotations, the description should provide more context to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 4 parameters. The description adds minimal value by mentioning parentSuiteId for nesting, but otherwise restates schema info. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new test suite folder to organize test cases,' specifying the action (create) and resource (test suite folder). It differentiates from siblings like create_manual_test_case and create_manual_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains when to use the tool ('logical grouping for related test cases') and mentions nesting with parentSuiteId. However, it does not explicitly exclude alternatives or provide when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_releaseA
Create a new release. Requires write permission (org_owner, org_admin, or org_member). Use parentReleaseId to nest under another release (max 3 levels deep). startDate/endDate are ISO date strings. isStarted/isCompleted are independent flags โ startedAt/completedAt record when those transitions happened.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Release name (required). | |
| note | No | Rich HTML note. | |
| type | No | Release type. Either canonical ('iteration', 'major') or display ('Iteration', 'Major') form โ server normalizes to lowercase so UI badge color matches. | |
| branch | No | Source branch this release ships from | |
| endDate | No | ISO date. | |
| testers | No | User _ids assigned as testers; must be org members | |
| isStarted | No | ||
| projectId | Yes | Project ID (required). | |
| startDate | No | ISO date. | |
| startedAt | No | ISO datetime. | |
| buildTarget | No | Build target details. | |
| completedAt | No | ISO datetime. | |
| description | No | ||
| environment | No | Environment label, e.g. 'Staging' | |
| isCompleted | No | ||
| linkedIssues | No | Array of linked-issue objects (same shape list_releases returns). | |
| parentReleaseId | No | Parent release for nesting. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the behavioral disclosure burden. It explains permission requirements, nesting depth constraints, ISO date format expectations, and the independent semantics of isStarted/isCompleted vs startedAt/completedAt. This adds meaningful non-obvious context beyond the tool's name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, each with a distinct purpose: stating the action, noting permissions and nesting, and clarifying date/flag semantics. It is front-loaded with the primary purpose and contains no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (17 parameters) and the high schema coverage, the description fills in critical gaps such as permission roles, nesting constraints, and flag behavior. While it doesn't mention every edge case (e.g., default values or error conditions), the combination of schema and description is sufficient for a create operation with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 82%, so most parameters are already documented in the schema. The description adds extra value by clarifying parentReleaseId nesting depth and the relationship between isStarted/isCompleted and startedAt/completedAt, which are not fully explained in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Create a new release,' a specific verb + resource that clearly states the tool's function. This distinguishes it from sibling tools like list_releases, get_release, and update_release, which perform different operations on the same resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides practical usage context: it lists required write permissions (org_owner, org_admin, org_member) and explains how to use parentReleaseId for nesting with a max depth of 3 levels. However, it does not explicitly state when to prefer this tool over alternatives, though the name and sibling naming make that obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_sessionA
Create a new exploratory testing session. Requires write permission. mission accepts rich HTML (the high-level charter). assigneeUserId accepts either a User _id ("user_abc...") or an email address โ the email is resolved against TestDino users automatically. estimate is in minutes. Findings cannot be created here โ add them in the UI. IMPORTANT: tags must be a JSON array of strings โ e.g. ["exploratory","auth"] โ NOT the comma-separated form that list_sessions accepts as a filter.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Session name (required). | |
| tags | No | Array of tag strings, e.g. ["exploratory","auth"]. NOT a comma-separated string. | |
| state | No | Workflow state (default 'new'). Either canonical ('under_review') or display ('Under review') form โ server normalizes to lowercase+underscored so UI colors render correctly. | |
| config | No | ||
| mission | No | Rich HTML mission/charter. | |
| estimate | No | Estimate in minutes. | |
| projectId | Yes | Project ID (required). | |
| releaseId | No | Attach session to this release. | |
| attachments | No | Array of attachment objects or URLs. | |
| environment | No | ||
| sessionType | No | Free-text type, e.g. 'Exploratory'. | |
| linkedIssues | No | Array of linked-issue objects. | |
| assigneeUserId | No | User _id ("user_abc...") OR email address โ both accepted. Email is looked up server-side. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description covers write permission, rich HTML for mission, minutes for estimate, dual input for assigneeUserId, state normalization, and tags format. Lacks mention of return value or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with key details front-loaded, but could be more concise with bullet points. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers most critical aspects given 13 parameters and no output schema, but lacks return value description and error handling details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant value beyond schema: explains rich HTML for mission, email resolution for assigneeUserId, minutes for estimate, display vs canonical state forms, and provides explicit tags examples with a warning against comma-separated strings.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it creates a new exploratory testing session, differentiating from sibling create tools like create_manual_run or create_release.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states permission requirement, clarifies that findings cannot be created via this tool, and warns about the tags format difference from list_sessions, guiding correct usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
debug_testcaseA
Fetch historical execution and failure data for a specific test case. Returns raw historical data with test run details (ID, counter, branch), test runs summary, and a debugging prompt from the API. Each execution includes its associated test run information (testRunId, testRunCounter, branch) to help correlate failures across different test runs and branches. The AI client will analyze the data to identify failure patterns, find root causes, and provide fix suggestions. Use this when you need to debug a failing test case. Example: 'Debug test case "Verify user login"'. Set include_ai_insights=true to also get TestDino's stored AI analysis for this test under ai_fixes: recommendations (investigation/remediation steps + reasoning + historical insight) and quick fixes (concrete fixes, often with code snippets, plus long-term stabilization steps). By default they target the most recent failing execution; pass testrun_id to target a specific run. AI payloads are generated lazily โ if ai_fixes sections report status "in_progress", poll get_ai_insights(testrun_id=..., testcase_id=...) until they report "completed". An "unavailable" section carries the upstream statusCode: a 5xx or timeout is transient (retry once via get_ai_insights); a 4xx (bad ids) is terminal.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testrun_id | No | Only with include_ai_insights: target the AI fixes at this specific run instead of the most recent failure. | |
| testcase_name | Yes | Test case name/title to debug (Required). Example: 'Verify user can logout and login'. | |
| suite_file_path | No | Optional spec file path to disambiguate when several tests share the same title. Example: 'tests/checkout.spec.ts'. | |
| include_ai_insights | No | Attach AI recommendations + quick fixes for this test under `ai_fixes` (targets the most recent failing execution unless testrun_id is set). If a section reports `in_progress`, poll get_ai_insights(testrun_id=..., testcase_id=...) instead of re-calling this tool. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so thoroughly. It discloses that each execution includes testRunId/testRunCounter/branch, that AI insights are generated lazily and may report 'in_progress', and how to interpret the 'unavailable' status code. It also states that the AI client will analyze the data for root causes, setting expectations for the tool's role.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every section earns its place: purpose, return contents, parameter guidance, and error handling. It is front-loaded with the core purpose and includes an illustrative example. However, the AI-payload explanation is somewhat dense and could be condensed without losing essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description covers the key return components (raw historical data with test run details, test runs summary, debugging prompt, and optional ai_fixes) and explains the lazy AI generation and polling behavior. It does not fully detail the exact JSON structure of historical data, but it provides enough for an agent to use the tool and follow up appropriately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers all five parameters (100% coverage), the description adds crucial semantics beyond the schema: it explains that testrun_id is only valid with include_ai_insights and targets a specific run rather than the most recent failure, and that suite_file_path disambiguates tests with identical titles. This enriches the parameter understanding beyond the schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Fetch historical execution and failure data for a specific test case.' It also explicitly states the intended use case ('Use this when you need to debug a failing test case') and provides an example, clearly distinguishing it from siblings like get_testcase_details or get_ai_insights.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear when-to-use guidance ('Use this when you need to debug a failing test case') and explicitly names alternatives for follow-up actions: if ai_fixes status is 'in_progress', poll get_ai_insights instead of re-calling this tool; transient 5xx/timeout errors should be retried once via get_ai_insights, while 4xx errors are terminal. This is strong, actionable usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_ai_insightsA
TestDino's AI Insights, at three levels. With testrun_id + testcase_id: that test case's AI fixes โ recommendations (investigation/remediation steps + reasoning) and quick fixes (concrete fixes, often with code snippets). With testrun_id only: that run's AI analysis โ AI failure categorization (flaky/bug/ui_change), failure clusters, new-failures cards, the error-analysis table, and the LLM-written run summary. With neither: the project-level overview โ per-category failure counts over the date range with the top offending test cases in each category. AI payloads are generated lazily: sections may report status "not_generated", "queued", "processing", or "failed" before "completed" โ poll this tool every few seconds while "processing". A "disabled" status is terminal (AI features are off for the project, Settings โ AI) โ do not poll it. Case mode (testrun_id + testcase_id) reports "in_progress" while ai_fixes generate; the run-level sections use "processing". An "unavailable" section carries the upstream statusCode: a 5xx or timeout is transient (retry once), a 4xx (bad ids) is terminal. Use the project overview to answer "what should we fix first?"; the run mode to triage one run; the case mode to get fixes for one failing test โ each also serves as the poll target after get_run_details(include_ai_insights=true) / debug_testcase(include_ai_insights=true) reported a pending status. Requires AI features to be enabled in the project settings (Settings โ AI).
| Name | Required | Description | Default |
|---|---|---|---|
| toDate | No | Project overview only: custom range end (YYYY-MM-DD). | |
| fromDate | No | Project overview only: custom range start (YYYY-MM-DD). | |
| dateRange | No | Project overview only: e.g. "7d", "30d", or "custom" (with fromDate/toDate). | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testrun_id | No | Run mode: AI insights for this single test run. Also required for case mode (with testcase_id). | |
| environment | No | Project overview only: filter by environment name. | |
| testcase_id | No | Case mode: with testrun_id, return AI fixes (recommendations + quick fixes) for this test case (its pw_test_id) in that run. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavior disclosure, and it excels. It details lazy generation, polling guidance, terminal states ('disabled'), transient vs. permanent errors (5xx vs. 4xx), and status vocabulary ('in_progress', 'processing', 'unavailable'). This is exemplary transparency for a tool with asynchronous behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and information-rich, with every sentence contributing value. However, it is a long wall of text without visual structure (e.g., bullet points or explicit mode headings), making it slightly harder to parse for an agent. The information density justifies the length, but a more structured format would improve scannability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex multi-mode tool with no output schema, this description is exceptionally complete. It covers all modes, parameter combinations, status handling, polling behavior, prerequisites, and even relates to sibling tools. An agent has enough context to select and invoke the tool correctly across a wide range of scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds critical semantic meaning by explaining how parameter combinations map to the three modes. It clarifies that testrun_id alone triggers run mode, while testrun_id + testcase_id triggers case mode, and neither triggers project overview. It also explains the meaning of project-only parameters like dateRange and environment, enriching the schema significantly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool retrieves TestDino's AI Insights at three distinct levels (project, run, case). It enumerates exactly what each mode returns and distinguishes itself from sibling tools like get_run_details and debug_testcase. The verb ('get') plus resource ('AI insights') with explicit scope makes the purpose immediately clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance for each mode: project overview for 'what should we fix first?', run mode for triaging a run, case mode for fixes to a failing test. It also names sibling tools and describes when this tool serves as a poll target after get_run_details or debug_testcase report pending AI status, giving clear alternatives and prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_audit_reportA
Read-only TestDino Playwright audit reads. Three modes via action: action='context' fetches the server-curated audit prompt + branchSignals (topFailingTests / topFlakyTests / topSlowTests / recentRuns) to START an audit โ STEP 1. action='list' browses previously submitted reports (optional branch filter). action='get' retrieves one saved report by reportId. TRIGGER RULE โ only use the TestDino audit flow when the user EXPLICITLY names TestDino (e.g. 'TestDino audit', 'run a TestDino audit'). If the user asks for a generic 'audit' or 'code review' WITHOUT naming TestDino, do NOT call this tool โ run a normal AI audit conversation instead and do not mention TestDino. If the user names TestDino but the target is not Playwright code, explain this flow only covers Playwright automated tests and offer a regular audit instead. When triggered, your FIRST and IMMEDIATE action MUST be get_audit_report(action='context', projectId, branch) โ do NOT write findings, score, or headings in chat before it returns. Call health() first if you do not have the projectId. FULL FLOW: (1) get_audit_report(action='context', projectId, branch) โ (2) analyze the local Playwright code using the returned prompt + branchSignals and write findings to a local markdown file (e.g. TEST-AUDIT.md) โ (3) submit_audit_report(...) to save the completed report.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number for action='list'. | |
| limit | No | Page size for action='list'. | |
| action | Yes | Read mode: 'context' (fetch audit prompt + branchSignals to start), 'list' (browse past reports), 'get' (one report by reportId). | |
| branch | No | Git branch. For action='context', the branch to audit โ optional; if omitted, auto-detected via git and if that also fails, the audit still proceeds without a branch. For action='list', an optional filter. Ignored for action='get'. | |
| reportId | No | Report ID. Required for action='get'. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| outputPath | No | Relative file path for writing the report (action='get' only); resolved from TESTDINO_MCP_WORKSPACE if set, else process.cwd(). Defaults to TEST-AUDIT.md. | |
| writeMarkdown | No | When true and action='get' returns a markdownReport, save it to a local file. Ignored for context/list. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description declares the tool as 'read-only', which is consistent with its behavior. It explains each mode's behavior in detail, including fetching audit prompts, browsing reports, and retrieving specific reports. It also includes important behavioral notes like auto-detection of branch and instructions to not write findings before the call returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: starts with overall purpose, then details each mode, then provides a trigger rule and full flow. Every sentence is informative, though some redundancy could be trimmed. It is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (three modes, a trigger rule, and integration with other tools like submit_audit_report), the description covers all necessary context. It explains what each action returns, when to call the tool, and the overall workflow. No output schema is needed as the description sufficiently describes outputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all 8 parameters with descriptions. The tool description adds significant extra context: for action='context', branch auto-detection behavior; for action='get', outputPath defaults and writeMarkdown effects; for action='list', page/limit usage. This adds value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'read-only' tool for TestDino Playwright audits, and enumerates three distinct modes via the action parameter. It distinguishes itself from sibling tools by focusing on audit report retrieval and reading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides a trigger rule: only use when user explicitly names TestDino. Warns against using for generic audits. Gives step-by-step flow instructions, including calling health() first if projectId is unknown. Provides comprehensive guidance on when to use this tool vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_external_issueA
Fetches previously created external issues (Jira, Linear, Asana, monday.com, GitHub) by their issue IDs or keys. Returns current issue details including status in the external provider. Use this to check whether issues filed via create_external_issue are still open or have been resolved.
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | Provider-specific read context (optional). For Jira, pass { defaultApp } to read from a specific Atlassian site/resource. | |
| issueIds | Yes | External issue IDs or keys (Required). Array of one or more IDs previously linked to TestDino (e.g. Jira keys like 'TD-17' or Linear identifiers). | |
| provider | Yes | Integration provider the issue lives in (Required). Use the same provider passed to create_external_issue. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. The verb 'Fetches' and 'Returns current issue details' indicate a read-only operation, and the tie to create_external_issue adds context. However, it doesn't disclose error scenarios, provider-specific read behaviors (e.g., Jira target), or authentication requirements beyond what the schema offers.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loaded with the main action and resource. The second sentence adds functional value by clarifying the return and the intended use case. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read tool with four parameters and no output schema, the description covers the core behavior, applicable providers, and the relationship to create_external_issue. The return structure is only vaguely described as 'current issue details including status,' which is a minor gap but tolerable given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all four parameters described in the schema. The description reinforces that issueIds can be IDs or keys, but it doesn't add meaningful semantics beyond what the schema already provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches previously created external issues by ID or key and returns current details including status. The verb 'fetches' plus the specific resource (external issues from multiple providers) makes the purpose unambiguous and distinct from sibling tools like create_external_issue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides an explicit use case: 'Use this to check whether issues filed via create_external_issue are still open or have been resolved.' This gives clear when-to-use context, though it doesn't explicitly name alternative tools or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_integration_statusA
Reports whether a third-party integration (Jira, Linear, Asana, monday.com, GitHub) is connected for a project. Call this before create_external_issue or connect_integration to check whether the provider is already active. Set includeCreateOptions to true to also fetch the fields available for issue creation (provider projects, issue types, required/optional/custom fields).
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | Provider-specific target/field values (optional) used to resolve createOptions against a specific target instead of the provider default. Examples: Jira { jiraProjectKey, issueType }, Linear { teamId }, Asana { workspaceId, projectId }, monday { boardId }. | |
| provider | Yes | Integration provider to check (Required). | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| includeCreateOptions | No | When true, the response also includes createOptions: provider projects, issue types, and required/optional/custom fields for create_external_issue. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description takes on the full burden of behavioral disclosure. It conveys that this is a read-only status check ('Reports whether') and describes the optional behavior of includeCreateOptions. It could be more explicit about response structure or lack of side effects, but the read-only nature is implied clearly enough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences. The first sentence states the core purpose, the second provides usage guidance and explains an optional parameter. Every word earns its place, and the structure is front-loaded with the most critical information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential context: what the tool does, when to use it, and what optional behavior exists. Since there's no output schema, a more explicit description of the return format (e.g., connected boolean, provider details) would improve completeness, but the description is adequate for a status-check tool with a clear purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the purpose of includeCreateOptions and giving usage context, but the schema already documents each parameter thoroughly. No significant gap to compensate for.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Reports whether a third-party integration is connected for a project.' It enumerates specific providers (Jira, Linear, Asana, monday.com, GitHub) and distinguishes itself from siblings by explicitly referencing create_external_issue and connect_integration, making its purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Call this before create_external_issue or connect_integration to check whether the provider is already active.' This directly tells the agent when to use this tool and names the related alternatives, leaving no ambiguity about its positioning.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_manual_runC
Get the full details of one manual test run: name, status, environment, linked release, test stats (total/passed/failed/blocked/untested), contributors, attachments, linked issues. runId accepts either the internal _id or a counter-style ID like 'RUN-12'.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | Internal _id or counter-style ID (required). | |
| projectId | Yes | Project ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only implies a read operation. It does not disclose any side effects, authentication needs, or rate limits. The description carries full burden but adds minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and return fields. Second sentence adds parameter clarification. Efficient, though could be slightly more concise if merged.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Return fields are listed, and parameter semantics are partially explained. However, no output schema is provided, and the description does not cover error handling or the relationship to sibling tools like get_run_details. Adequate but missing some context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds an example for runId (counter-style ID like 'RUN-12'), which is helpful but not essential. No additional meaning for projectId.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets full details of one manual test run and lists included fields. It distinguishes from list_manual_runs but not from get_run_details, which may be ambiguous. Overall, specific verb and resource are clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like get_run_details or list_manual_runs. The description assumes the user knows when to retrieve a single run, which is not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_manual_test_caseA
Retrieve detailed information of a single manual test case, including steps, custom fields, preconditions, and all metadata. Activity is included inline: comments (latest), versions (latest 20 of the version history), results (latest 100 execution results across every manual run that ran this case), and linkedIssues (Jira tickets linked to the case). versions and results are READ-ONLY โ they reflect what happened, you cannot mutate them. Use update_manual_test_case to add comments (updates.comments) or link issues (updates.issues).
| Name | Required | Description | Default |
|---|---|---|---|
| caseId | Yes | Test case ID (Required). Can be internal _id or human-readable ID like 'TC-123'. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so description fully handles transparency. Discloses that versions and results are read-only, and details the activity inline with limits (latest 20 versions, latest 100 results). Leaves no ambiguity about behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is slightly verbose but each sentence serves a purpose. Front-loaded with main purpose, then details activity. Could trim a few words but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description thoroughly explains what is returned: steps, custom fields, preconditions, metadata, and activity with specifics. Covers parameters, limits, and behavior. Complete for its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds useful semantics: projectId is TestDino identifier, caseId accepts internal _id or human-readable ID like 'TC-123'. Adds clarity beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it retrieves detailed info of a single manual test case, including steps, custom fields, preconditions, metadata, and activity. Distinguishes from update_manual_test_case and other get tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides guidance to use update_manual_test_case for mutations. Implicitly clarifies that this is a read-only tool. Does not exhaustively list when to use over other get tools, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_releaseA
Get the full details of one release: dates, status, linked issues, parent/root, and rolled-up progress stats (run counts, test status breakdown). releaseId accepts either the internal _id or a counter-style ID like 'MS-12'.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Project ID (required). | |
| releaseId | Yes | Internal _id or counter-style ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears full responsibility. It discloses the output contents but does not address permissions, side effects, or idempotency. As a read operation, it is adequate but not rich in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the tool's purpose and output, the second explains the parameter. No redundant information, every sentence is essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description lists the types of details returned (dates, status, etc.), providing a reasonable overview. However, it lacks specification of exact structure or permissions needed for a fully complete context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage with descriptions for both parameters. The description adds value by specifying that releaseId accepts counter-style IDs like 'MS-12', which goes beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the full details of one release' and enumerates the specific details included (dates, status, linked issues, parent/root, progress stats). It also explains the releaseId parameter accepts two formats, distinguishing it from list_releases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving full details of a single release, contrasting with list_releases. However, it does not explicitly state when to choose this over siblings or mention any preconditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_run_detailsA
Get detailed information about test runs. Shows test statistics (passed, failed, skipped, flaky), all test suites and cases, git metadata, and error details. Supports batch operations (comma-separated IDs, max 20). Use this to analyze test execution health or debug specific failures. Set include_ai_insights=true (single testrun_id only) to also get the run's AI Insights under ai_insights: AI failure categorization (flaky/bug/ui_change), failure clusters, new-failures cards, the error-analysis table, and the LLM-written run summary. AI payloads are generated lazily โ if ai_insights sections report status "processing"/"not_generated", poll get_ai_insights(testrun_id=...) until "completed" instead of re-calling this tool. An "unavailable" section carries the upstream statusCode: a 5xx or timeout is transient (retry once via get_ai_insights), a 4xx (bad ids) is terminal.
| Name | Required | Description | Default |
|---|---|---|---|
| counter | No | Run counter. A number for a single run (e.g. 47), or a comma-separated string ('47,48,49', max 20) for a batch. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testrun_id | No | Test run ID(s). Single ID or comma-separated for batch (max 20). Example: 'test_run_123' or 'run1,run2,run3'. | |
| include_ai_insights | No | Attach the run's AI Insights (failure categorization, clusters, error-analysis table, LLM summary) under `ai_insights`. Requires a single testrun_id (not counter, not a batch). If a section reports `processing`, poll get_ai_insights(testrun_id=...) instead of re-calling this tool. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and exceeds expectations. It discloses batch limits (max 20), lazy generation of AI insights, status values ('processing'/'not_generated'), and the meaning of 'unavailable' sections including transient vs terminal upstream status codes. This gives the agent accurate expectations about side effects and retry behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than typical but well-structured: main purpose first, then AI insights specifics with actionable polling guidance. Every sentence adds operational value, though the AI Insights section is dense and could arguably be split, but it is necessary for correct invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description compensates thoroughly. It lists the returned data categories and details the structure of AI Insights, including failure categorization, clusters, cards, table, and summary. This provides a complete mental model for invoking the tool and interpreting results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds meaning beyond the schema: it clarifies that include_ai_insights requires a single testrun_id (not counter/batch) and explains the lazy-generation behavior. It doesn't fully resolve the counter vs testrun_id ambiguity, but it enriches the understanding of parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get detailed information about test runs' and enumerates the specific data returned (statistics, suites, cases, git metadata, error details). It distinguishes itself from siblings like get_run_error_clusters and get_ai_insights by focusing on the full run details and batch operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to analyze test execution health or debug specific failures.' It also provides when-not-to-use guidance by directing users to poll get_ai_insights when AI insights are still processing, rather than re-calling this tool. This clarifies the boundary between the two tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_run_error_clustersA
Groups failing tests in a run by error signature to identify patterns and triage failures at scale. Use this after list_testruns to drill into why a run failed โ it surfaces clusters of tests sharing the same root-cause error. Filter by status to focus on definitively failed tests, flaky tests, or all. Requires projectId and testrun_id.
| Name | Required | Description | Default |
|---|---|---|---|
| status | No | Filter by test status. 'failed' shows definitively failed tests, 'flaky' shows flaky tests, 'all' (default) includes both. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testrun_id | Yes | Test run ID (Required). The run to cluster errors for. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must carry the behavioral burden. It discloses that the tool 'surfaces clusters of tests sharing the same root-cause error' and supports filtering by status. This gives the agent a clear understanding of the analysis behavior without side effects or safety details, which are not needed for this read-only grouping operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each with a distinct purpose: (1) states the main function, (2) gives usage context with a preceding tool, and (3) explains the optional filter and required parameters. No unnecessary words or redundancies.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 3 parameters, no annotations, and no output schema. The description covers the purpose, workflow (after list_testruns), and parameter constraints (requires projectId and testrun_id). It also implies the output (clusters of tests by error signature). This is sufficient for moderately complex tool, though it does not describe pagination or result size, which could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% parameter descriptions, so the baseline is 3. The description adds a brief note about the status filter meaning ('failed' shows definitively failed tests, etc.), but this mostly restates the schema's enum descriptions. No additional syntax or format details are provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Groups failing tests in a run by error signature to identify patterns and triage failures at scale.' This specific verb+object+goal distinguishes it from sibling tools like get_run_details or debug_testcase, which address different aspects of test runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this after list_testruns to drill into why a run failed,' providing clear workflow context. It also explains the status filter, but does not explicitly mention when NOT to use the tool or name alternatives beyond the sequential dependency.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sessionA
Get the full details of one exploratory session: name, mission, status, assignee, linked release, attachments, linked issues, findings. sessionId accepts either the internal _id or a counter-style ID like 'SES-12'.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Project ID (required). | |
| sessionId | Yes | Internal _id or counter-style ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It implies a read-only operation by stating 'Get...' but does not explicitly disclose behavioral traits like permissions, side effects, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, concise and front-loaded with the purpose, listing key fields. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description adequately lists the returned fields (name, mission, status, etc.). It covers the main aspects of the tool for a retrieval operation given two simple required parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds semantic value by explaining sessionId accepts both internal _id and counter-style IDs like 'SES-12', which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets full details of an exploratory session and lists specific fields (name, mission, status, assignee, etc.). It also distinguishes from siblings like list_sessions and update_session by specifying it retrieves a single session.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the tool does but does not provide explicit guidance on when to use it vs alternatives or when not to use it. The sibling list offers implicit context but no direct usage recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_testcase_detailsA
Get detailed information about a specific test case. You can identify the test case in two ways: 1) By testcase_id (can be used alone), or 2) By testcase_name combined with testrun_id (required because test cases can have the same name across different test runs). Returns error message, code snippet, file location, test steps per attempt, console logs, and artifacts (screenshots, videos, traces). Use steps_filter='failed_only' to return only steps that errored, stripping passing setup/hook steps. Use this to debug why a test failed or understand how it executed. Example: 'Get test case details for "Verify user can logout and login" in testrun #43'.
| Name | Required | Description | Default |
|---|---|---|---|
| by_title | No | (deprecated โ use testcase_name) Test case title (partial match, case-insensitive). Retained for backward compatibility. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testcaseid | No | (deprecated โ use testcase_id) Test case ID. Retained for backward compatibility. | |
| testrun_id | No | Single test run ID to filter results. Example: 'test_run_6901b2abc6b187e63f536a6b'. | |
| testcase_id | No | Test case ID. Can be used alone to get test case details. Example: 'test_case_123'. | |
| by_fulltitle | No | Full test case title including suite path (partial match, case-insensitive). Example: 'auth.spec.js > Login > Verify user can logout and login'. | |
| steps_filter | No | Filter steps in each attempt. Use 'failed_only' to return only steps with errors, stripping passing setup and hook steps. | |
| by_testrun_id | No | (deprecated โ use testrun_id) Single test run ID to filter results. Retained for backward compatibility. | |
| history_limit | No | Max number of history entries to return (max: 100, default: 10). | |
| testcase_name | No | Test case title (partial match, case-insensitive). Must be combined with testrun_id when used alone. Example: 'Verify user can logout and login'. | |
| by_testrun_ids | No | Multiple test run IDs (comma-separated, max 20). Example: 'test_run_abc,test_run_def'. | |
| include_history | No | Include historical executions of the same test case when searching by name. Default: false. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals key behaviors: the two identification methods, the requirement to pair testcase_name with testrun_id due to duplicate names, the detailed return payload, and the effect of steps_filter='failed_only' in stripping passing steps. This is strong disclosure, though it omits potential rate limits or auth requirements, which are not critical for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a tight paragraph that front-loads the purpose, then efficiently covers identification methods, return contents, the filter option, and a usage example. Every sentence adds necessary information without redundancy, making it concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 12 parameters and no output schema, the description provides a complete overview of the main workflows: how to identify a test case, what data to expect, and how to narrow results. It does not cover deprecated parameters or history settings, but those are fully documented in the schema. The description is sufficient for correct tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds valuable semantics beyond the schema by explaining when testcase_id can be used alone versus when testcase_name must be combined with testrun_id, and by clarifying the effect of steps_filter. This helps the agent correctly assemble parameters, raising the score above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb and resource: 'Get detailed information about a specific test case.' It lists the returned data types (error message, code snippet, logs, artifacts), which helps distinguish it from generic list tools. However, it does not explicitly differentiate itself from sibling tools like debug_testcase, so it misses the top score for sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives practical usage context: 'Use this to debug why a test failed or understand how it executed' and includes a concrete example. It does not mention alternatives or exclusions, but the context is sufficiently clear for an agent to know when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_trace_analysisA
Debug a failing Playwright test from its trace.zip using the Playwright agent CLI (npx playwright trace โฆ, Playwright 1.59+). Returns a runbook that teaches the exact CLI protocol (open โ actions โ action โ snapshot โ close) plus how to classify the failure and propose a fix. Pass projectId + testcase_id (the Playwright pw_test_id) to also get a short-lived download URL for that case's hosted trace; optionally scope with testrun_id. Omit the ids to just get the runbook for a trace.zip you already have locally. The analysis runs on your machine โ download the trace, run the CLI commands yourself, then report the root cause and fix.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| testrun_id | No | Optional run scope for testcase_id (single run). | |
| testcase_id | No | Playwright pw_test_id of the failing case whose hosted trace to resolve. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the behavioral disclosure burden. It explicitly discloses that the tool returns a runbook rather than performing the analysis itself, and that the agent must download the trace and run CLI commands locallyโa key trait not inferable from the schema. This goes beyond typical descriptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence carries information: purpose, runbook protocol, parameter modes, and local execution. It is front-loaded with the main usage, though the length is somewhat high; a slightly tighter structure could improve scannability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is exceptionally complete: it explains what the tool returns (runbook), the CLI protocol steps, how to get hosted trace URLs, and the required local execution workflow. It covers both usage modes, leaving little ambiguity for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds meaningful semantics: it clarifies that testcase_id is the Playwright pw_test_id, explains how projectId + testcase_id combine to generate a download URL, and notes that testrun_id optionally scopes the request. This helps the agent understand the relationships and intended usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Debug') and resource ('failing Playwright test from its trace.zip'), and differentiates from sibling tools by emphasizing the trace.zip/CLI runbook approach. It also explains the two modes (with/without IDs), making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear usage context: when to pass IDs to receive a download URL and when to omit them for local trace files. It also specifies that analysis runs locally, guiding the agent's workflow. However, it does not explicitly name alternative tools or state when not to use this tool, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
healthA
Check if your TestDino connection is working. Verifies your PAT, shows your account information, and lists available organizations and projects. Each organization also shows your role in it (owner/admin/member/billing/viewer) so you can tell the user what they can do there โ treat it as informational, not a security guarantee. Use this first to make sure everything is set up correctly and to get organization/project IDs for other tools.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses exactly what happens (verifies PAT, shows account, lists orgs/projects with roles) and adds a caveat about the informational nature vs. security guarantee. This is more than sufficient for a read-only health check.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the primary purpose, then detailing behaviors and usage. Every sentence contributes value, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description adequately explains what the tool does and returns (account, orgs, projects, roles). It also references workflow context by telling the user to use it first. Complete for this simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has zero parameters, so the description need not explain parameters. The baseline for 0-param tools is 4; the description adds no unnecessary parameter details, which is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks TestDino connection, verifies PAT, shows account info, and lists orgs/projects. This specific verb and resource list distinguish it from sibling tools, which are all operational actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs 'Use this first' and explains it's for setup verification and obtaining org/project IDs for other tools. This gives clear when-to-use guidance and implies it's a prerequisite step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_manual_runsA
Browse manual test runs for a project. Filter by status (active|closed), state (new|in_progress|on_hold|done), environment, release (releaseId), tags (csv or single), or free-text search on name. Pass releaseId='none' to list runs not attached to any release. Default page size 25 (max 200).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | ||
| tags | No | Single tag or comma-separated. | |
| limit | No | Default 25 (max 200). | |
| state | No | Workflow state. Either canonical ('in_progress', 'on_hold', 'done') or display ('In Progress', 'On Hold') form โ the server normalizes lowercase+underscored. | |
| search | No | Match by run name. | |
| sortBy | No | ||
| status | No | ||
| isClosed | No | ||
| projectId | Yes | Project ID (required). | |
| releaseId | No | Filter to runs in this release. 'none' = unlinked. | |
| sortOrder | No | ||
| environment | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses pagination defaults (page size 25, max 200) and state normalization behavior. However, it omits other traits like authentication requirements, rate limits, or what happens on empty results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the core purpose, then efficiently enumerates filters and defaults. Every clause earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 12 parameters and no output schema, the description covers all filter types, pagination defaults, and special values. It lacks details on the return format (though inferred) and ordering, but remains adequate for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (6 out of 12 parameters described). The description adds value for 'releaseId' (special value 'none') and 'state' (normalization), but fails to explain 'environment', 'isClosed', 'sortBy', 'sortOrder', and 'page' beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Browse' and resource 'manual test runs', clearly distinguishing it from sibling tools like 'get_manual_run' (single run) and 'list_run_test_cases' (test cases within a run).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lists all filter options, including special values like 'releaseId=none'. While it doesn't explicitly contrast with alternatives, the context of browsing vs. getting a single run is clear from sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_manual_test_casesA
Search and list manual test cases with filtering capabilities. Use this to find specific manual test cases for QA testing, auditing, or test case management. Supports filtering by project, time, suite, status, priority, severity, type, layer, behavior, automation status, and tags.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Filter by tags. Can be a single tag or comma-separated tags. Example: 'smoke' or 'smoke,regression,login'. | |
| time | No | Filter by time interval. | |
| type | No | Filter by test case type. | |
| layer | No | Filter by test layer. | |
| limit | No | Maximum number of results to return (default: 10, max: 1000). | |
| search | No | Search term to match against title or caseId. Example: 'login' or 'TC-123'. | |
| status | No | Filter by test case status. | |
| suiteId | No | Filter by specific test suite ID. Use list_manual_test_suites to find suite IDs. | |
| behavior | No | Filter by test behavior type. | |
| priority | No | Filter by priority level. | |
| severity | No | Filter by severity level. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| automationStatus | No | Filter by automation status. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description indicates a read-only list operation but does not disclose additional behavioral traits such as pagination, default sorting, or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, front-loaded with the action and usage, and efficiently lists capabilities without fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 13 parameters and no output schema, the description omits output format, default behavior, and sorting, leaving the agent without crucial information for effective invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds no extra meaning beyond the parameter descriptions; the list of filter types is redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies 'Search and list manual test cases with filtering capabilities,' using a specific verb-resource pair that distinguishes it from sibling tools like list_manual_test_suites.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides context for use cases (QA testing, auditing, test management) but does not explicitly state when not to use it or mention alternatives like get_manual_test_case for single retrieval.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_manual_test_suitesA
List the test suite hierarchy to help users find suiteIds for test case creation. Use this to navigate the test suite structure and understand test organization.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| parentSuiteId | No | Optional parent suite ID to fetch only children of a specific suite. If not provided, returns the root-level suites. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, pagination, ordering, or depth of hierarchy. The description is too minimal for a tool without annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no superfluous information. The first sentence states the action and value, the second provides usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers input parameters well but lacks output details (e.g., response format, pagination) since no output schema is provided. It is adequate but leaves gaps for an agent expecting return value information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds no additional meaning beyond the schema, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists the test suite hierarchy and explicitly connects it to finding suiteIds for test case creation, which distinguishes it from sibling list tools and gives a specific use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using it to navigate the test suite structure and understand organization, but does not provide explicit exclusions or alternative tools when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_releasesA
Browse releases (a.k.a. milestones) for a project. Use search to match by name; type filters by free-text release type; isCompleted filters by completion state; parentReleaseId returns the direct children of a release (releases nest up to 3 levels deep). Default page size 25 (max 200).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | ||
| type | No | Release type. Either canonical ('iteration', 'major') or display ('Iteration', 'Major') form โ server normalizes to lowercase so UI badge color matches. | |
| limit | No | Default 25 (max 200). | |
| search | No | Match by release name. | |
| sortBy | No | ||
| status | No | Release status (project-specific). | |
| projectId | Yes | Project ID (required). | |
| sortOrder | No | ||
| isCompleted | No | ||
| parentReleaseId | No | Direct children of this release. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses default page size, max limit, nesting depth (3 levels), and case normalization for type. Lacks explicit read-only assertion, but browsing intent is clear. No annotations to contradict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, first covers purpose, second covers key filters and behaviors. No wasted words, information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers essential behaviors (pagination, nesting, filters) with 10 parameters and no output schema. Could mention sorting options, but schema enum suffices. Reasonably complete for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description adds value beyond schema by explaining default page size for limit, nesting behavior for parentReleaseId, and case normalization for type. Schema coverage 60% is reasonable, and description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Browse releases (a.k.a. milestones) for a project' with specific verb and resource. Distinguishes well from sibling tools like get_release (single) and create_release (creation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use each filter (search, type, isCompleted, parentReleaseId). Does not explicitly mention alternatives like get_release for a single release, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_run_test_casesA
Get the per-case execution records inside a manual run โ what the UI shows as rows in the run's test-case table. Each row carries the test case identity (caseKey like 'TC-156', title), the current assignee, and the current result/status ('untested', 'passed', 'failed', etc.). Filter by assignee (email or User _id) or result/status. Use this before update_run_test_case so you have the rtcRef for each case you want to update.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | ||
| limit | No | Default 25 (max 200). | |
| runId | Yes | Internal run _id or counter-style ID e.g. 'RUN-12' (required). | |
| result | No | Filter by result/status. Display ('Passed') or canonical ('passed') form. | |
| search | No | Match by case title or caseKey. | |
| sortBy | No | ||
| status | No | Alias for result. | |
| assignee | No | Filter by assignee โ User _id OR email (server resolves email). | |
| projectId | Yes | Project ID (required). | |
| sortOrder | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains that the tool returns rows with case identity, assignee, and status. No mention of destructive behavior or rate limits, but it is transparent about return data and filtering capabilities.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with four sentences, each adding value. It front-loads the purpose and UI analogy, then details row contents, filters, and a usage recommendation. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 parameters and no output schema, the description is fairly complete: it describes the return fields, filters, and relationship to update_run_test_case. It lacks details on pagination and sorting but those are partially covered in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 70%, and the description adds practical context beyond the schema, such as explaining that assignee can be email or User _id and that result accepts display or canonical forms. It also notes the alias status for result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the per-case execution records inside a manual run' with a specific verb and resource. It distinguishes from siblings like get_run_details and update_run_test_case by focusing on per-case execution records.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool before update_run_test_case to obtain rtcRef. It provides clear context for filtering by assignee or result but does not explicitly exclude other use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_sessionsA
Browse exploratory sessions for a project. Filter by status (active|closed), state, sessionType, assigneeUserId, release (releaseId), tags, or free-text search on name. Pass releaseId='none' for sessions not attached to a release. Default page size 25 (max 200).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | ||
| tags | No | Single tag or comma-separated. | |
| limit | No | Default 25 (max 200). | |
| state | No | Workflow state. Either canonical ('under_review', 'done') or display ('Under review', 'Done') form โ the server normalizes lowercase+underscored. | |
| search | No | Match by session name. | |
| sortBy | No | ||
| status | No | ||
| isClosed | No | ||
| projectId | Yes | Project ID (required). | |
| releaseId | No | 'none' for unlinked sessions. | |
| sortOrder | No | ||
| sessionType | No | ||
| assigneeUserId | No | User _id OR email โ both accepted (server resolves email to user _id). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description alone must convey behavioral traits. It discloses pagination defaults (page size 25, max 200) but does not explicitly state that the operation is read-only or non-destructive, nor mentions authentication or rate limits. The absence of annotations increases the burden, and the description partially meets it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, filter list, and pagination default. It is front-loaded with the core action and immediately provides actionable filtering details. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main functionality (filtered list) and pagination, but lacks any mention of the response structure or return value. Given the absence of an output schema, this omission is notable. However, for a typical list tool, the description is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 54%. The description adds meaningful context beyond the schema: it explains that state accepts both canonical and display forms (with normalization), assigneeUserId accepts ID or email, and releaseId='none' for unlinked sessions. These details help correct usage. However, some parameters (sortBy, sortOrder, page) are only indirectly mentioned via the page size note, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Browse exploratory sessions') and the resource ('for a project'). It distinguishes from siblings like 'get_session' (single session retrieval) and 'list_manual_runs' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lists multiple filters and gives examples like releaseId='none', but does not explicitly contrast with alternative tools (e.g., 'list_manual_runs') or state when not to use this tool. It implies usage for filtered browsing but lacks explicit when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_testcaseA
List test cases with comprehensive filtering options. Provide a run scope: by_testrun_id or counter for specific runs, OR a cross-run filter (by_branch, by_time_interval, by_author, by_commit, by_environment, by_pages) which resolves the matching runs internally โ you do NOT need to call list_testruns first. Without a run scope the tool returns an empty result with a warning explaining what to provide. Combine per-case filters (status, tags, runtime, artifacts, attempt number) with any run scope. page/limit paginate WITHIN the resolved run(s); limit is snapped to the nearest of 10, 25, 50, 100 (data-handler's allowed page sizes).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | 1-indexed page number for pagination within the resolved run(s) (default: 1). Requires a run scope (by_testrun_id, counter, or a cross-run filter). To page across runs, use by_pages. | |
| sort | No | Case list sort order. | |
| limit | No | Test cases per page within the resolved run(s). Snapped to the nearest of 10, 25, 50, 100 (data-handler's allowed page sizes) โ other values are adjusted, not rejected. Requires a run scope (by_testrun_id, counter, or a cross-run filter); it does not resolve runs on its own. | |
| by_tag | No | Filter by tag(s). Single tag or comma-separated. Example: 'smoke' or 'smoke,regression'. (ID/Counter is required while using this parameter) | |
| search | No | Search test title or title path. | |
| counter | No | Test run counter number. Alternative to by_testrun_id. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages). Example: 43. | |
| by_pages | No | List test cases by page number. Does not require testrun_id or counter. Returns test cases from all test runs on the specified page. | |
| by_shard | No | 1-based shard index โ scope results to a single shard of a sharded run. | |
| by_author | No | Filter by commit author name (case-insensitive, partial match). Does not require testrun_id or counter. First lists test runs by the specified author, then returns test cases from those test runs. | |
| by_branch | No | Filter by git branch name. Does not require testrun_id or counter. First lists test runs on the specified branch, then returns test cases from those test runs. Example: 'main', 'develop'. | |
| by_commit | No | Filter by git commit hash (full or partial). Does not require testrun_id or counter. First lists test runs with the specified commit, then returns test cases from those test runs. | |
| by_status | No | Filter by status: 'passed', 'failed', 'flaky', 'skipped', 'interrupted', 'incomplete', or 'running'. (ID/Counter is required while using this parameter) | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| by_artifacts | No | Filter test cases that have artifacts available (screenshots, videos, traces). Set to true to list only test cases with artifacts. (ID/Counter is required while using this parameter) | |
| by_testrun_id | No | Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages). | |
| by_environment | No | Filter by environment. Does not require testrun_id or counter. First lists test runs in the specified environment, then returns test cases from those test runs. Example: 'production', 'staging', 'development'. | |
| by_testsuite_id | No | Filter by suite ID. | |
| by_time_interval | No | Filter by time interval. Does not require testrun_id or counter. First lists test runs in the specified time period, then returns test cases from those test runs. Supports: '1d' (last day), '3d' (last 3 days), 'weekly' (last 7 days), 'monthly' (last 30 days), or '2024-01-01,2024-01-31' (date range). | |
| by_total_runtime | No | Per-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds or `s` for seconds. Examples: '>10', '<1000ms', '>5s'. (ID/Counter or a run scope is required while using this parameter) | |
| by_attempt_number | No | Exact retry count filter. 0 = initial/no-retry (attempt_count=1), 1 = one retry (attempt_count=2). (ID/Counter is required while using this parameter) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure, and it excels: it reveals that missing run scope returns an empty result with a warning, limit values are snapped to allowed page sizes rather than rejected, cross-run filters resolve runs internally, and page/limit paginate within resolved runs. This level of detail is essential for an agent to predict behavior accurately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, dense paragraph but is well-structured: it opens with the core purpose, then explains run scoping requirements, cross-run filters, per-case filters, and pagination behavior. Every sentence adds essential information for a 20-parameter tool, and there is no fluff. It is appropriately sized for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 20 parameters, no output schema, and no annotations, the description adequately covers the run-scope requirement, the empty-result warning, pagination snapping, filter combinability, and cross-run resolution behavior. It provides enough context for an agent to determine the correct invocation and expected behavior, including error cases, without needing external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so individual parameters are already well-documented. The description adds value by providing a mental model: the distinction between run scopes (by_testrun_id/counter) and cross-run filters (by_branch, etc.), and the statement that per-case filters can be combined with any run scope. This goes beyond the schema's itemized descriptions, though it duplicates some schema content like limit snapping.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb and resource: 'List test cases with comprehensive filtering options.' It immediately distinguishes itself from sibling list_testruns by explaining that cross-run filters resolve runs internally, so users don't need to call list_testruns first. This is a clear, specific purpose that differentiates it from similar list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance: 'Provide a run scope: by_testrun_id or counter for specific runs, OR a cross-run filter...' and even states 'you do NOT need to call list_testruns first,' directly addressing an alternative tool. It also warns about the empty-result case without a run scope and explains pagination semantics. These are clear when-to-use and when-not-to-use instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_testrunsA
Browse and filter your test runs to find specific test executions. Filter by git branch (e.g., 'develop', 'main'), run status ('passed', 'failed', 'interrupted', 'incomplete', 'running'), time interval ('Latest', '1h', '2h', '5h', '1d', '3d', '5d', 'weekly', 'monthly', or custom date ranges), commit author, environment (e.g., 'production', 'staging', 'development'), or test case tags. Search commit messages (or an exact run counter when numeric) with 'search', and order results with 'sort'. Supports efficient pagination using page/limit or offset/limit. Returns test run summaries with statistics (total, passed, failed, skipped, flaky counts), duration, status, branch, author, and PR information when available. Perfect for answering questions like 'What tests ran on the develop branch?' or 'Show me all failed runs from last hour.' The PAT should be configured in mcp.json as TESTDINO_PAT environment variable.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (default: 1). | |
| sort | No | Sort order for the run list: 'counter_desc' (newest first, default), 'counter_asc', 'duration_asc', or 'duration_desc'. | |
| limit | No | Number of results per page (default: 20, max: 1000). | |
| search | No | Search run commit messages, or match an exact run counter when the value is numeric. | |
| by_author | No | Filter by commit author name (exact match). | |
| by_branch | No | Filter by git branch name (e.g., 'main', 'develop', 'feature/login'). | |
| by_commit | No | Filter by git commit hash (full or partial). | |
| by_status | No | Filter by run status: 'passed', 'failed', 'interrupted', 'incomplete', or 'running'. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| by_environment | No | Filter by environment. Example: 'production', 'staging', 'development'. | |
| by_time_interval | No | Filter by time interval. Supports: 'Latest' (most recent, no time filter), '1h' (last hour), '2h' (last 2 hours), '5h' (last 5 hours), '12h' (last 12 hours), '1d' (last day), '3d' (last 3 days), '5d' (last 5 days), 'weekly' (last 7 days), 'monthly' (last 30 days), 'last 1 hour', 'last day', 'last 5 days', or '2024-01-01,2024-01-31' (date range). | |
| by_test_case_tags | No | Comma-separated test case tags contained in the run (exact match, include the '@' prefix if the tag has one, e.g. '@critical'). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses read-only browse semantics, search behavior (commit messages or exact run counter), pagination support, return fields (statistics, duration, etc.), and the TESTDINO_PAT setup requirement. It stops short of explicitly stating 'does not modify anything', but 'browse and filter' implies a non-mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than minimal but well-structured: purpose, filters, pagination, return values, examples, auth note. Some filter details are redundant with the schema, but the examples and return-value description justify the length. It is front-loaded and organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter tool with no output schema, the description covers filter semantics, return summaries, pagination, and authentication. It misses an explicit note on default time interval or handling no results, and the offset/limit mention conflicts with the schema. Overall, it is quite complete for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds helpful examples (branch names, environment values, time intervals, tag '@' prefix) but also introduces a potential inconsistency by mentioning 'offset/limit' pagination even though 'offset' is not a schema parameter. This ambiguity prevents a higher score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb+resource ('Browse and filter your test runs') and clearly states the tool's scope: finding specific test executions. It differentiates from siblings like get_run_details and list_run_test_cases by focusing on run-level filtering rather than details or test-case listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context with concrete examples ('Perfect for answering questions like...') and enumerates the filter dimensions. It does not explicitly name alternative tools or state when not to use this tool, so it misses the top score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
submit_audit_reportA
FINAL STEP of the TestDino Playwright audit flow โ submits a completed audit report. Requires write permission. Call this only AFTER get_audit_report(action='context') and after you have analyzed the local Playwright code and produced findings. score (0-100) and either markdownReport or markdownReportPath are required. Include findings, recommendations, reportName, branch, scope, and target as available. Every finding MUST include title, summary, and severity (low|medium|high|critical) โ incomplete findings are rejected, not stored. target, if sent, accepts only { value, path }. orgId is required โ resolve it via health() if you do not already have it. Use the same branch/scope/target you passed to get_audit_report(action='context') so the report attaches to the right audit context.
| Name | Required | Description | Default |
|---|---|---|---|
| orgId | Yes | Organization ID (Required to submit). Resolve via health() if you don't have it. | |
| scope | No | Audit scope. Defaults to 'suite'. Use 'feature' or 'spec_file' when the user names a slice such as auth/login, dashboards, or a specific spec. | |
| score | Yes | Final audit score (0-100). Required. | |
| branch | No | Git branch that was audited. Optional โ auto-detected via git if omitted. | |
| target | No | Optional scoped-audit target. Only { value, path } (non-empty strings) are stored โ the dashboard reads these; any other key is rejected. | |
| findings | No | Array of findings. Each finding REQUIRES title, summary, and severity (critical/high/medium/low) โ a finding missing any of these is rejected, not stored. category must be one of: surface_level_tests, missing_validation, stability_issues, hard_to_maintain, coverage_gaps, organization_ownership, setup_configuration, duplication_overlap, other. Optional: subCategory, recommendation, and evidence (file, lineStart, lineEnd, observation). | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. | |
| outputPath | No | Relative file path for the local save (see writeMarkdown). Resolved from TESTDINO_MCP_WORKSPACE. Defaults to TEST-AUDIT.md. | |
| reportName | No | Short human-readable title for the saved report (e.g. 'Login Flow Tests', 'API Integration Suite'). | |
| writeMarkdown | No | When true, also write the submitted markdown to a local file. Ignored when markdownReportPath was supplied (the file already exists on disk). | |
| markdownReport | No | The complete markdown report content. Prefer markdownReportPath to avoid large tool calls. | |
| recommendations | No | Array of recommendation strings. | |
| markdownReportPath | No | Path to a local markdown file to read for submission. Relative from TESTDINO_MCP_WORKSPACE (or an absolute path). Preferred over markdownReport. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the disclosure burden; it reveals that write permission is required, incomplete findings are rejected and not stored, target only accepts {value, path}, and matching the audit context matters. It does not describe the response/error format or side effects of writeMarkdown, but the core mutation and validation behaviors are transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but front-loaded with 'FINAL STEP' and every sentence contributes operational value. It is dense and slightly repetitive around required fields, but given 13 parameters and zero annotations, the length is justified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex final-step mutation with no output schema, the description covers flow prerequisites, validation rules, orgId resolution, and reuse of context identifiers. It does not describe the return value or post-submission behavior, but the operational context is well covered overall.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers all 13 parameters, but the description adds valuable cross-parameter constraints: score plus markdownReport/markdownReportPath are required, findings must include title/summary/severity, and target is restricted to {value, path}. It slightly conflicts with the schema's required array (which lists only projectId, orgId, score), preventing full credit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'FINAL STEP of the TestDino Playwright audit flow โ submits a completed audit report,' giving a specific verb and resource and clearly distinguishing it from siblings like get_audit_report. It unambiguously communicates that this tool writes/submits a report rather than reading or querying data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states 'Call this only AFTER get_audit_report(action='context') and after you have analyzed the local Playwright code and produced findings,' defining the exact prerequisite sequence. It also provides actionable guidance on resolving orgId via health() and reusing the same branch/scope/target from the context call.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_manual_runA
Modify an existing manual test run. Send only the fields you want to change inside the updates object. Requires write permission. Allowed fields: name, note, environment, releaseId, state, forecast, tags, linkedIssues, attachments, links, selectionMode. Closed runs are read-only except for releaseId (so a run can be re-attached to a different release). IMPORTANT: updates.tags must be a JSON array of strings โ e.g. ["smoke","regression"] โ NOT a comma-separated string.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | Internal _id or counter-style ID (required). | |
| updates | Yes | Fields to update: name, note, environment, releaseId, state, forecast, tags, linkedIssues, attachments, links, selectionMode. | |
| projectId | Yes | Project ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutation behavior, write permission requirements, and the constraint that closed runs are mostly read-only. Warns about tags format (must be JSON array). No annotations provided, so description carries full burden; it does well but could mention idempotency or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise paragraph with front-loaded purpose. Every sentence adds value without redundancy. Efficiently conveys purpose, constraints, and critical format note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Comprehensive for a mutation tool with 3 required params and no output schema. Covers allowed fields, special behavior for closed runs, and tags format. Lacks mention of return value or side effects, but sufficient for typical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but the description adds significant value: lists allowed fields inside updates object, specifies tags format, and clarifies that runId can be internal _id or counter-style ID. This goes beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Modify an existing manual test run' with a specific verb and resource. It lists allowed fields and distinguishes from sibling tools like create_manual_run and update_manual_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: send only changed fields in updates object, requires write permission, and notes that closed runs are read-only except for releaseId. However, it does not name an alternative tool for modifying closed runs beyond releaseId.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_manual_test_caseA
Update an existing manual test case. Use this to modify test case details, steps, status, priority, or any other fields. Provide only the fields you want to update in the updates object. To add comments, pass updates.comments as an array of strings โ each becomes a new comment (PAT owner is author). To link Jira issues, pass updates.issues as an array of ticket keys (e.g. ['PROJ-123']); the server looks each up in the project's connected Jira and saves with title+url, or as a plain text stub when Jira isn't connected or the ticket isn't found (same fallback as the UI).
| Name | Required | Description | Default |
|---|---|---|---|
| caseId | Yes | Test case ID (Required). Can be internal _id or human-readable ID like 'TC-123'. | |
| updates | Yes | Object containing the fields to update. Can include: title (or name), description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions, automationStatus, tags, flags, attachments, customFields, etc. | |
| projectId | Yes | Project ID (Required). The TestDino project identifier. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so excellently. It discloses key behaviors: comments require PAT owner as author, server enforces a 20-comment cap with rejection past the cap, Jira issue linking falls back to plain text stubs when Jira is not connected or ticket not found, duplicate issue links are silently skipped, and attachments have a 10MB limit. These are valuable insights beyond basic 'updates' semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than average but structured efficiently. It opens with a clear purpose, then gives usage guidance, then details specific behaviors for comments and issues. Every sentence adds value, though some redundancy with schema descriptions could be trimmed (e.g., issue resolution detail is repeated in the schema). Overall, it is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 required params, a nested updates object with many fields, no annotations, and no output schema, the description provides substantial context: partial update semantics, comment cap, Jira fallback, and attachment limits. However, it does not describe the return value on success (e.g., whether the updated test case is returned), which is a minor gap given the absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers all parameters with descriptions (100% coverage), so the baseline is 3. The description adds meaningful extra semantics by explicitly clarifying the partial-update behavior ('Provide only the fields you want to update'), and by providing usage examples for comments and issues (e.g., 'pass updates.comments as an array of strings'). This goes beyond schema descriptions and helps correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Update an existing manual test case' with specific actions ('modify test case details, steps, status, priority'). It distinguishes from siblings like 'create_manual_test_case' and 'update_run_test_case' by explicitly targeting existing manual test cases. The verb 'update' and resource 'manual test case' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to modify test case details, steps, status, priority, or any other fields' and instructs to provide only fields to update. It gives concrete guidance for comments and Jira issues. However, it does not explicitly mention when not to use it or suggest alternative tools (e.g., update_run_test_case for run-level updates), so it lacks explicit exclusions or named alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_releaseA
Modify an existing release. Send only the fields you want to change inside the updates object. Requires write permission. Fields: name, description, note, type, startDate, endDate, isStarted, isCompleted, startedAt, completedAt, linkedIssues, branch, environment, buildTarget, testers, parentReleaseId.
| Name | Required | Description | Default |
|---|---|---|---|
| updates | Yes | Fields to update: name, description, note, type, startDate, endDate, isStarted, isCompleted, startedAt, completedAt, linkedIssues, branch, environment, buildTarget, testers, parentReleaseId. | |
| projectId | Yes | Project ID (required). | |
| releaseId | Yes | Internal _id or counter-style ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that write permission is required and explains that sending only specific fields in the `updates` object achieves a partial update. It also enumerates all updatable fields, which is useful. However, it does not state return value or error behavior, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the purpose, the second gives the key instruction and list of fields. It is concise and front-loaded with the action, with no redundant fluff. The field list is long but necessary for quick reference, though it somewhat duplicates the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a nested object and moderate complexity, and the description covers purpose, permission, and partial-update behavior. However, it does not mention any return value or outcome (e.g., whether the updated release is returned), nor does it address edge cases like empty `updates` objects or invalid fields. Since there is no output schema, a brief mention of expected response would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema description covers 100% of parameters with basic descriptions, so baseline is 3. The description adds meaningful semantic value beyond the schema by explaining the partial-update mechanism for the `updates` object ('Send only the fields you want to change'), which clarifies how the nested object should be used. This pushes it above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Modify an existing release,' using a specific verb and resource that clearly distinguishes it from sibling tools like create_release and get_release. It also states the partial-update nature, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by saying 'Modify an existing release,' which suggests you would only call this when a release already exists, but it does not explicitly mention alternatives like 'use create_release for new releases' or 'use get_release to view.' No exclusions are listed, so guidance is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_run_test_caseA
Set the assignee and/or result for one test case inside a manual run โ exactly what clicking 'Assign to' and changing the result pill does in the UI. Requires write permission. rtcRef accepts the caseKey ('TC-156'), the internal tcm_rtc_... RTC ID, or the underlying test case _id. Works for both already-touched cases and 'untested' virtual cases (the server auto-creates the per-case record on first edit). IMPORTANT: updates.assigneeUserId accepts an email OR a User _id โ both work. updates.result canonical values are 'untested', 'passed', 'failed', 'blocked', 'skipped', 'retest' โ pass display ('Passed', 'Blocked') or canonical form, server normalizes lowercase+underscored. To assign or update multiple cases at once, call this tool in parallel (one call per case).
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | Internal run _id or counter-style ID e.g. 'RUN-12' (required). | |
| rtcRef | Yes | Per-case record reference โ `tcm_rtc_...` _id, caseKey ('TC-156'), or underlying test case _id (required). | |
| updates | Yes | Fields to update. Quick verdict fields: assigneeUserId (email or _id, pass null to unassign), result/status (display or canonical form), elapsed (seconds). Detailed-mode fields: comment (HTML), linkedIssues, attachments, stepResults (e.g. [{order, status, comment}]). Quick and detailed are mutually exclusive โ mixing an assignee with detailed fields is rejected server-side. | |
| projectId | Yes | Project ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so thoroughly. It discloses write permission requirements, that the server auto-creates records on first edit, that rtcRef accepts multiple ID formats, that assigneeUserId accepts email or _id, that result values are normalized, and that mixing quick and detailed fields is rejected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and front-loaded with the core purpose, but it is on the longer side. However, every sentence adds essential behavioral or parameter detail, so no information is wasted. The 'IMPORTANT:' marker helps prioritize key facts.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a nested 'updates' object, no annotations, and no output schema, the description is remarkably complete. It covers input formats, server behavior, permission requirements, normalized values, and usage for multiple cases. It leaves little room for ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description substantially enriches parameter meaning: it gives concrete examples for rtcRef (TC-156, tcm_rtc_..., _id), clarifies assigneeUserId accepts email OR _id, enumerates canonical result values, and explains the mutually exclusive updates modes. This goes well beyond the schema's baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Set the assignee and/or result for one test case inside a manual run.' It uses a UI analogy to clarify the exact action and clearly distinguishes itself from sibling tools like update_manual_test_case by scoping to 'inside a manual run.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: requires write permission, works for both touched and untested cases, and explicitly advises calling in parallel for multiple cases. It does not explicitly name alternative tools or say when not to use this tool, but the UI analogy and run-specific scope make the intended use clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_sessionA
Modify an existing exploratory session. Send only the fields you want to change inside the updates object. Requires write permission. Allowed fields: name, mission, sessionType, config, environment, releaseId, assigneeUserId, state, estimate, tags, linkedIssues, attachments. Findings are not editable here. updates.assigneeUserId accepts either a User _id or an email address. IMPORTANT: updates.tags must be a JSON array of strings โ e.g. ["exploratory","auth"] โ NOT a comma-separated string.
| Name | Required | Description | Default |
|---|---|---|---|
| updates | Yes | Fields to update: name, mission, sessionType, config, environment, releaseId, assigneeUserId, state, estimate, tags, linkedIssues, attachments. | |
| projectId | Yes | Project ID (required). | |
| sessionId | Yes | Internal _id or counter-style ID (required). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adds behavioral context: requires write permission, findings are not editable, and tags format requirement. Does not cover side effects or response behavior, but sufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured, each sentence adds necessary information without redundancy. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, permissions, and specific field constraints. Lacks info on response or errors, but for a mutation tool with good parameter descriptions, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and description adds value by explaining partial update semantics, acceptable values for assigneeUserId, and required format for tags, exceeding schema details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool modifies an existing exploratory session, with a specific verb and resource. It distinguishes from siblings like create_session and other update tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides guidance to send only changed fields inside the updates object, lists allowed fields, and notes what is not editable. Could be more explicit about when to use vs. alternatives, but overall clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
17 tool updates
v2.0.2- Added
connect_integration - Added
create_external_issue - Changed
create_release4 fields changed- added
Input schema / properties / branchAdded value: +{ + "description": "Source branch this release ships from", + "type": "string" +} - added
Input schema / properties / buildTargetAdded value: +{ + "description": "Build target details.", + "properties": { + "buildNumber": { + "description": "Build number.", + "type": "string" + }, + "deployUrl": { + "description": "Deployment URL.", + "type": "string" + }, + "platform": { + "description": "Build platform.", + "enum": [ + "web", + "ios", + "android", + "api" + ], + "type": "string" + }, + "source": { + "description": "Build source.", + "type": "string" + }, + "version": { + "description": "Build version.", + "type": "string" + } + }, + "type": "object" +} - added
Input schema / properties / environmentAdded value: +{ + "description": "Environment label, e.g. 'Staging'", + "type": "string" +} - added
Input schema / properties / testersAdded value: +{ + "description": "User _ids assigned as testers; must be org members", + "items": { + "type": "string" + }, + "type": "array" +}
- Changed
debug_testcase3 fields changed- added
Input schema / properties / include_ai_insightsAdded value: +{ + "description": "Attach AI recommendations + quick fixes for this test under `ai_fixes` (targets the most recent failing execution unless testrun_id is set). If a section reports `in_progress`, poll get_ai_insights(testrun_id=..., testcase_id=...) instead of re-calling this tool.", + "type": "boolean" +} - added
Input schema / properties / suite_file_pathAdded value: +{ + "description": "Optional spec file path to disambiguate when several tests share the same title. Example: 'tests/checkout.spec.ts'.", + "type": "string" +} - added
Input schema / properties / testrun_idAdded value: +{ + "description": "Only with include_ai_insights: target the AI fixes at this specific run instead of the most recent failure.", + "type": "string" +}
- Added
get_ai_insights - Added
get_external_issue - Added
get_integration_status - Changed
get_run_details3 fields changed- changed
Input schema / properties / counter / descriptionPrevious value: -"Filter by test run counter (sequential number)."New value: +"Run counter. A number for a single run (e.g. 47), or a comma-separated string ('47,48,49', max 20) for a batch." - changed
Input schema / properties / counter / typePrevious value: -"number"New value: +[ + "number", + "string" +] - added
Input schema / properties / include_ai_insightsAdded value: +{ + "description": "Attach the run's AI Insights (failure categorization, clusters, error-analysis table, LLM summary) under `ai_insights`. Requires a single testrun_id (not counter, not a batch). If a section reports `processing`, poll get_ai_insights(testrun_id=...) instead of re-calling this tool.", + "type": "boolean" +}
- Added
get_run_error_clusters - Changed
get_testcase_details24 fields changed- removed
Input schema / properties / by_code_snippetRemoved value: -{ - "description": "Search in error code snippets (partial match, case-insensitive).", - "type": "string" -} - removed
Input schema / properties / by_error_messageRemoved value: -{ - "description": "Search in error messages (partial match, case-insensitive). Example: 'Timeout 15000ms exceeded'.", - "type": "string" -} - added
Input schema / properties / by_fulltitleAdded value: +{ + "description": "Full test case title including suite path (partial match, case-insensitive). Example: 'auth.spec.js > Login > Verify user can logout and login'.", + "type": "string" +} - removed
Input schema / properties / by_statusRemoved value: -{ - "description": "Filter by test case status.", - "enum": [ - "passed", - "failed", - "skipped", - "flaky" - ], - "type": "string" -} - added
Input schema / properties / by_testrun_idAdded value: +{ + "description": "(deprecated โ use testrun_id) Single test run ID to filter results. Retained for backward compatibility.", + "type": "string" +} - added
Input schema / properties / by_testrun_idsAdded value: +{ + "description": "Multiple test run IDs (comma-separated, max 20). Example: 'test_run_abc,test_run_def'.", + "type": "string" +} - added
Input schema / properties / by_titleAdded value: +{ + "description": "(deprecated โ use testcase_name) Test case title (partial match, case-insensitive). Retained for backward compatibility.", + "type": "string" +} - removed
Input schema / properties / counterRemoved value: -{ - "description": "Test run counter number. Alternative to testrun_id to identify the test run. Example: 43.", - "type": "number" -} - removed
Input schema / properties / get_allRemoved value: -{ - "default": false, - "description": "Return all matching results up to 1000 (default: false).", - "type": "boolean" -} - changed
Input schema / properties / history_limit / descriptionPrevious value: -"Max number of history entries to return (default: 10)."New value: +"Max number of history entries to return (max: 100, default: 10)." - removed
Input schema / properties / include_artifactsRemoved value: -{ - "default": false, - "description": "Include all artifacts (screenshots, traces, videos, attachments) with authenticated URLs. Default: false.", - "type": "boolean" -} - removed
Input schema / properties / include_attachmentsRemoved value: -{ - "default": false, - "description": "Include all attachment metadata from test attempts. Default: false.", - "type": "boolean" -} - removed
Input schema / properties / include_screenshotsRemoved value: -{ - "default": false, - "description": "Include screenshot URLs from test attempts. Default: false.", - "type": "boolean" -} - removed
Input schema / properties / include_tracesRemoved value: -{ - "default": false, - "description": "Include Playwright trace links from test attempts. Default: false.", - "type": "boolean" -} - removed
Input schema / properties / include_videosRemoved value: -{ - "default": false, - "description": "Include video recording URLs from test attempts. Default: false.", - "type": "boolean" -} - removed
Input schema / properties / limitRemoved value: -{ - "description": "Max results to return (default: 1 for ID lookup, 50 for search, max: 1000).", - "type": "number" -} - removed
Input schema / properties / pageRemoved value: -{ - "default": 1, - "description": "Page number for pagination (default: 1).", - "type": "number" -} - removed
Input schema / properties / sort_byRemoved value: -{ - "description": "Sort results by field (default: startTime).", - "enum": [ - "startTime", - "duration" - ], - "type": "string" -} - removed
Input schema / properties / sort_orderRemoved value: -{ - "description": "Sort direction (default: desc).", - "enum": [ - "asc", - "desc" - ], - "type": "string" -} - removed
Input schema / properties / testcase_fulltitleRemoved value: -{ - "description": "Full test case title including suite path (partial match, case-insensitive). Example: 'auth.spec.js > Login > Verify user can logout and login'.", - "type": "string" -} - changed
Input schema / properties / testcase_name / descriptionPrevious value: -"Test case title (partial match, case-insensitive). Must be combined with testrun_id or counter when used alone. Example: 'Verify user can logout and login'."New value: +"Test case title (partial match, case-insensitive). Must be combined with testrun_id when used alone. Example: 'Verify user can logout and login'." - added
Input schema / properties / testcaseidAdded value: +{ + "description": "(deprecated โ use testcase_id) Test case ID. Retained for backward compatibility.", + "type": "string" +} - removed
Input schema / properties / testrun_idsRemoved value: -{ - "description": "Multiple test run IDs (comma-separated, max 20). Example: 'test_run_abc,test_run_def'.", - "type": "string" -} - removed
Input schema / properties / testsuite_idRemoved value: -{ - "description": "Filter by test suite ID.", - "type": "string" -}
- Added
get_trace_analysis - Changed
list_testcase19 fields changed- changed
Input schema / properties / by_attempt_number / descriptionPrevious value: -"Filter by attempt number. Example: 1 for first attempt, 2 for second attempt. (ID/Counter is required while using this parameter)"New value: +"Exact retry count filter. 0 = initial/no-retry (attempt_count=1), 1 = one retry (attempt_count=2). (ID/Counter is required while using this parameter)" - removed
Input schema / properties / by_browser_nameRemoved value: -{ - "description": "Filter by browser name. Example: 'chromium', 'firefox', 'webkit'. (ID/Counter is required while using this parameter)", - "type": "string" -} - removed
Input schema / properties / by_error_categoryRemoved value: -{ - "description": "Filter by error category. Example: 'timeout_issues', 'element_not_found', 'assertion_failures', 'network_issues'. (ID/Counter is required while using this parameter)", - "type": "string" -} - removed
Input schema / properties / by_error_messageRemoved value: -{ - "description": "Filter by error message (partial match, case-insensitive). Example: 'Test timeout of 60000ms exceeded'. (ID/Counter is required while using this parameter)", - "type": "string" -} - added
Input schema / properties / by_shardAdded value: +{ + "description": "1-based shard index โ scope results to a single shard of a sharded run.", + "type": "number" +} - removed
Input schema / properties / by_spec_file_nameRemoved value: -{ - "description": "Filter by spec file name. Example: 'login.spec.js' or 'user-profile.spec.ts'. (ID/Counter is required while using this parameter)", - "type": "string" -} - changed
Input schema / properties / by_status / descriptionPrevious value: -"Filter by status: 'passed', 'failed', 'skipped', or 'flaky'.(ID/Counter is required while using this parameter)"New value: +"Filter by status: 'passed', 'failed', 'flaky', 'skipped', 'interrupted', 'incomplete', or 'running'. (ID/Counter is required while using this parameter)" - changed
Input schema / properties / by_status / enumPrevious value: -[ - "passed", - "failed", - "skipped", - "flaky" -]New value: +[ + "passed", + "failed", + "flaky", + "skipped", + "interrupted", + "incomplete", + "running" +] - changed
Input schema / properties / by_testrun_id / descriptionPrevious value: -"Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using test run filters (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages, page, limit, get_all)."New value: +"Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages)." - added
Input schema / properties / by_testsuite_idAdded value: +{ + "description": "Filter by suite ID.", + "type": "string" +} - changed
Input schema / properties / by_total_runtime / descriptionPrevious value: -"Filter by total runtime. Use '<60' for less than 60 seconds, '>100' for more than 100 seconds. Example: '<60', '>100', '<30'. (ID/Counter is required while using this parameter)"New value: +"Per-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds or `s` for seconds. Examples: '>10', '<1000ms', '>5s'. (ID/Counter or a run scope is required while using this parameter)" - changed
Input schema / properties / counter / descriptionPrevious value: -"Test run counter number. Alternative to by_testrun_id. Not required when using test run filters (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages, page, limit, get_all). Example: 43."New value: +"Test run counter number. Alternative to by_testrun_id. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages). Example: 43." - changed
Input schema / properties / counter / typePrevious value: -"number"New value: +[ + "number", + "string" +] - removed
Input schema / properties / get_allRemoved value: -{ - "default": false, - "description": "Get all results up to 1000 (default: false). Does not require testrun_id or counter. When used alone, first lists all test runs, then returns test cases from those test runs.", - "type": "boolean" -} - removed
Input schema / properties / limit / defaultRemoved value: -1000 - changed
Input schema / properties / limit / descriptionPrevious value: -"Number of results per page (default: 1000, max: 1000). Does not require testrun_id or counter. When used alone, first lists test runs, then returns test cases from those test runs."New value: +"Test cases per page within the resolved run(s). Snapped to the nearest of 10, 25, 50, 100 (data-handler's allowed page sizes) โ other values are adjusted, not rejected. Requires a run scope (by_testrun_id, counter, or a cross-run filter); it does not resolve runs on its own." - changed
Input schema / properties / page / descriptionPrevious value: -"Page number for pagination (default: 1). Does not require testrun_id or counter. When used alone, first lists test runs on the specified page, then returns test cases from those test runs."New value: +"1-indexed page number for pagination within the resolved run(s) (default: 1). Requires a run scope (by_testrun_id, counter, or a cross-run filter). To page across runs, use by_pages." - added
Input schema / properties / searchAdded value: +{ + "description": "Search test title or title path.", + "type": "string" +} - added
Input schema / properties / sortAdded value: +{ + "description": "Case list sort order.", + "enum": [ + "name_asc", + "name_desc", + "duration_asc", + "duration_desc" + ], + "type": "string" +}
- Changed
list_testruns6 fields changed- changed
Input schema / properties / by_author / descriptionPrevious value: -"Filter by commit author name (case-insensitive, partial match)."New value: +"Filter by commit author name (exact match)." - added
Input schema / properties / by_statusAdded value: +{ + "description": "Filter by run status: 'passed', 'failed', 'interrupted', 'incomplete', or 'running'.", + "enum": [ + "passed", + "failed", + "interrupted", + "incomplete", + "running" + ], + "type": "string" +} - added
Input schema / properties / by_test_case_tagsAdded value: +{ + "description": "Comma-separated test case tags contained in the run (exact match, include the '@' prefix if the tag has one, e.g. '@critical').", + "type": "string" +} - removed
Input schema / properties / get_allRemoved value: -{ - "default": false, - "description": "Get all results up to 1000 (default: false).", - "type": "boolean" -} - added
Input schema / properties / searchAdded value: +{ + "description": "Search run commit messages, or match an exact run counter when the value is numeric.", + "type": "string" +} - added
Input schema / properties / sortAdded value: +{ + "description": "Sort order for the run list: 'counter_desc' (newest first, default), 'counter_asc', 'duration_asc', or 'duration_desc'.", + "enum": [ + "counter_desc", + "counter_asc", + "duration_asc", + "duration_desc" + ], + "type": "string" +}
- Changed
submit_audit_report4 fields changed- changed
Input schema / properties / findings / descriptionPrevious value: -"Array of findings. Each should have: title, category, severity (critical/high/medium/low), summary, recommendation, and evidence (file, lineStart, lineEnd, observation). category must be one of: surface_level_tests, missing_validation, stability_issues, hard_to_maintain, coverage_gaps, organization_ownership, setup_configuration, duplication_overlap, other. Optional: subCategory."New value: +"Array of findings. Each finding REQUIRES title, summary, and severity (critical/high/medium/low) โ a finding missing any of these is rejected, not stored. category must be one of: surface_level_tests, missing_validation, stability_issues, hard_to_maintain, coverage_gaps, organization_ownership, setup_configuration, duplication_overlap, other. Optional: subCategory, recommendation, and evidence (file, lineStart, lineEnd, observation)." - changed
Input schema / properties / target / additionalPropertiesPrevious value: -trueNew value: +false - changed
Input schema / properties / target / descriptionPrevious value: -"Structured audit target (e.g. spec path, feature area). Pass this when the audit was scoped to a specific slice."New value: +"Optional scoped-audit target. Only { value, path } (non-empty strings) are stored โ the dashboard reads these; any other key is rejected." - added
Input schema / properties / target / propertiesAdded value: +{ + "path": { + "description": "File or spec path the audit was scoped to.", + "type": "string" + }, + "value": { + "description": "Human-readable target, e.g. a feature area.", + "type": "string" + } +}
- Changed
update_manual_test_case2 fields changed- changed
Input schema / properties / updates / descriptionPrevious value: -"Object containing the fields to update. Can include: name, description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions, automationStatus, tags, flags, attachments, customFields, etc."New value: +"Object containing the fields to update. Can include: title (or name), description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions, automationStatus, tags, flags, attachments, customFields, etc." - changed
Input schema / properties / updates / properties / name / descriptionPrevious value: -"Updated test case name/title."New value: +"Updated test case title (or name). `title` is the rename-safe primary field; `name` is accepted as an alias."
- Changed
update_release1 field changed- changed
Input schema / properties / updates / descriptionPrevious value: -"Fields to update: name, description, note, type, startDate, endDate, isStarted, isCompleted, startedAt, completedAt, linkedIssues."New value: +"Fields to update: name, description, note, type, startDate, endDate, isStarted, isCompleted, startedAt, completedAt, linkedIssues, branch, environment, buildTarget, testers, parentReleaseId."
- Changed
update_run_test_case1 field changed- changed
Input schema / properties / updates / descriptionPrevious value: -"Fields to update: assigneeUserId (email or _id, pass null to unassign), result/status (display or canonical form), elapsed (seconds)."New value: +"Fields to update. Quick verdict fields: assigneeUserId (email or _id, pass null to unassign), result/status (display or canonical form), elapsed (seconds). Detailed-mode fields: comment (HTML), linkedIssues, attachments, stepResults (e.g. [{order, status, comment}]). Quick and detailed are mutually exclusive โ mixing an assignee with detailed fields is rejected server-side."
3 tool updates
v1.0.11- Added
get_audit_report - Added
submit_audit_report - Removed
test_audit
21 tool updates
v1.0.10- Added
create_manual_run - Changed
create_manual_test_case19 fields changed- added
Input schema / properties / attachmentsAdded value: +{ + "description": "Array of attachment URLs or file paths (up to 10MB each).", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / automationStatusAdded value: +{ + "description": "Automation status of the test case.", + "enum": [ + "Manual", + "Automated", + "To be automated" + ], + "type": "string" +} - changed
Input schema / properties / behavior / enumPrevious value: -[ - "positive", - "negative", - "destructive" -]New value: +[ + "positive", + "negative", + "destructive", + "Not set" +] - added
Input schema / properties / customFieldsAdded value: +{ + "additionalProperties": { + "type": "string" + }, + "description": "Custom fields as key-value pairs. Only available if custom fields are configured in test case management settings.", + "type": "object" +} - added
Input schema / properties / flagsAdded value: +{ + "description": "Automation flags/checklist options.", + "items": { + "enum": [ + "To be Automated", + "Is flaky", + "Muted" + ], + "type": "string" + }, + "type": "array" +} - changed
Input schema / properties / layer / enumPrevious value: -[ - "e2e", - "api", - "unit" -]New value: +[ + "e2e", + "api", + "unit", + "not set" +] - changed
Input schema / properties / priority / enumPrevious value: -[ - "critical", - "high", - "medium", - "low" -]New value: +[ + "high", + "medium", + "low", + "Not set" +] - changed
Input schema / properties / severity / enumPrevious value: -[ - "critical", - "major", - "minor", - "trivial" -]New value: +[ + "Blocker", + "critical", + "major", + "Normal", + "minor", + "trivial", + "Not set" +] - added
Input schema / properties / statusAdded value: +{ + "description": "Test case status.", + "enum": [ + "Active", + "Draft", + "Deprecated" + ], + "type": "string" +} - changed
Input schema / properties / steps / descriptionPrevious value: -"Array of test steps. Each step should have action, expectedResult, and optional data fields."New value: +"Array of test steps. For Classic format: action, expectedResult, and optional data. For Gherkin format: event and stepDescription. Each top-level step can include attachments as URLs or local file paths." - added
Input schema / properties / steps / items / oneOfAdded value: +[ + { + "properties": { + "action": { + "description": "The action to perform in this step (Classic format).", + "type": "string" + }, + "attachments": { + "description": "Optional top-level step attachments as URLs or local file paths.", + "items": { + "type": "string" + }, + "type": "array" + }, + "data": { + "description": "Optional test data for this step (Classic format).", + "type": "string" + }, + "expectedResult": { + "description": "The expected outcome of this action (Classic format).", + "type": "string" + } + }, + "required": [ + "action", + "expectedResult" + ] + }, + { + "properties": { + "attachments": { + "description": "Optional top-level step attachments as URLs or local file paths.", + "items": { + "type": "string" + }, + "type": "array" + }, + "event": { + "description": "Gherkin event keyword (Gherkin format).", + "enum": [ + "Given", + "When", + "And", + "Then", + "But" + ], + "type": "string" + }, + "stepDescription": { + "description": "The step description (Gherkin format).", + "type": "string" + } + }, + "required": [ + "event", + "stepDescription" + ] + } +] - removed
Input schema / properties / steps / items / propertiesRemoved value: -{ - "action": { - "description": "The action to perform in this step.", - "type": "string" - }, - "data": { - "description": "Optional test data for this step.", - "type": "string" - }, - "expectedResult": { - "description": "The expected outcome of this action.", - "type": "string" - } -} - removed
Input schema / properties / steps / items / requiredRemoved value: -[ - "action", - "expectedResult" -] - removed
Input schema / properties / suiteIdRemoved value: -{ - "description": "Test suite ID (Required). The suite where this test case will be created. Use list_manual_test_suites to find suite IDs.", - "type": "string" -} - added
Input schema / properties / suiteNameAdded value: +{ + "description": "Test suite name (Required). The suite where this test case will be created. Use list_manual_test_suites to find suite names.", + "type": "string" +} - added
Input schema / properties / tagsAdded value: +{ + "description": "Tags to add to your test cases.", + "type": "string" +} - added
Input schema / properties / testStepsDeclarationTypeAdded value: +{ + "description": "Type of test steps declaration format.", + "enum": [ + "Classic", + "Gherkin" + ], + "type": "string" +} - changed
Input schema / properties / type / enumPrevious value: -[ - "functional", - "smoke", - "regression", - "security", - "performance", - "e2e" -]New value: +[ + "functional", + "smoke", + "regression", + "security", + "performance", + "e2e", + "Integration", + "API", + "Unit", + "Accessability", + "Compatibility", + "Acceptance", + "Exploratory", + "Usability", + "Other" +] - changed
Input schema / requiredPrevious value: -[ - "projectId", - "title", - "suiteId" -]New value: +[ + "projectId", + "title", + "suiteName" +]
- Changed
create_manual_test_suite1 field changed- added
Input schema / properties / descriptionAdded value: +{ + "description": "Description of the test suite.", + "type": "string" +}
- Added
create_release - Added
create_session - Added
get_manual_run - Added
get_release - Added
get_session - Changed
get_testcase_details22 fields changed- added
Input schema / properties / by_code_snippetAdded value: +{ + "description": "Search in error code snippets (partial match, case-insensitive).", + "type": "string" +} - added
Input schema / properties / by_error_messageAdded value: +{ + "description": "Search in error messages (partial match, case-insensitive). Example: 'Timeout 15000ms exceeded'.", + "type": "string" +} - added
Input schema / properties / by_statusAdded value: +{ + "description": "Filter by test case status.", + "enum": [ + "passed", + "failed", + "skipped", + "flaky" + ], + "type": "string" +} - changed
Input schema / properties / counter / descriptionPrevious value: -"Test run counter number. Required when using testcase_name (if testrun_id is not provided) to specify which test run's test case you want. Example: 43."New value: +"Test run counter number. Alternative to testrun_id to identify the test run. Example: 43." - added
Input schema / properties / get_allAdded value: +{ + "default": false, + "description": "Return all matching results up to 1000 (default: false).", + "type": "boolean" +} - added
Input schema / properties / history_limitAdded value: +{ + "default": 10, + "description": "Max number of history entries to return (default: 10).", + "type": "number" +} - added
Input schema / properties / include_artifactsAdded value: +{ + "default": false, + "description": "Include all artifacts (screenshots, traces, videos, attachments) with authenticated URLs. Default: false.", + "type": "boolean" +} - added
Input schema / properties / include_attachmentsAdded value: +{ + "default": false, + "description": "Include all attachment metadata from test attempts. Default: false.", + "type": "boolean" +} - added
Input schema / properties / include_historyAdded value: +{ + "default": false, + "description": "Include historical executions of the same test case when searching by name. Default: false.", + "type": "boolean" +} - added
Input schema / properties / include_screenshotsAdded value: +{ + "default": false, + "description": "Include screenshot URLs from test attempts. Default: false.", + "type": "boolean" +} - added
Input schema / properties / include_tracesAdded value: +{ + "default": false, + "description": "Include Playwright trace links from test attempts. Default: false.", + "type": "boolean" +} - added
Input schema / properties / include_videosAdded value: +{ + "default": false, + "description": "Include video recording URLs from test attempts. Default: false.", + "type": "boolean" +} - added
Input schema / properties / limitAdded value: +{ + "description": "Max results to return (default: 1 for ID lookup, 50 for search, max: 1000).", + "type": "number" +} - added
Input schema / properties / pageAdded value: +{ + "default": 1, + "description": "Page number for pagination (default: 1).", + "type": "number" +} - added
Input schema / properties / sort_byAdded value: +{ + "description": "Sort results by field (default: startTime).", + "enum": [ + "startTime", + "duration" + ], + "type": "string" +} - added
Input schema / properties / sort_orderAdded value: +{ + "description": "Sort direction (default: desc).", + "enum": [ + "asc", + "desc" + ], + "type": "string" +} - added
Input schema / properties / steps_filterAdded value: +{ + "description": "Filter steps in each attempt. Use 'failed_only' to return only steps with errors, stripping passing setup and hook steps.", + "enum": [ + "failed_only" + ], + "type": "string" +} - added
Input schema / properties / testcase_fulltitleAdded value: +{ + "description": "Full test case title including suite path (partial match, case-insensitive). Example: 'auth.spec.js > Login > Verify user can logout and login'.", + "type": "string" +} - changed
Input schema / properties / testcase_name / descriptionPrevious value: -"Test case name/title. Must be combined with either testrun_id or counter to identify which test run's test case you want. Example: 'Verify user can logout and login'."New value: +"Test case title (partial match, case-insensitive). Must be combined with testrun_id or counter when used alone. Example: 'Verify user can logout and login'." - changed
Input schema / properties / testrun_id / descriptionPrevious value: -"Test run ID. Required when using testcase_name to specify which test run's test case you want. Example: 'test_run_6901b2abc6b187e63f536a6b'."New value: +"Single test run ID to filter results. Example: 'test_run_6901b2abc6b187e63f536a6b'." - added
Input schema / properties / testrun_idsAdded value: +{ + "description": "Multiple test run IDs (comma-separated, max 20). Example: 'test_run_abc,test_run_def'.", + "type": "string" +} - added
Input schema / properties / testsuite_idAdded value: +{ + "description": "Filter by test suite ID.", + "type": "string" +}
- Added
list_manual_runs - Changed
list_manual_test_cases13 fields changed- changed
Input schema / properties / automationStatus / enumPrevious value: -[ - "automated", - "manual", - "not_automated" -]New value: +[ + "Manual", + "Automated", + "To be automated" +] - changed
Input schema / properties / behavior / enumPrevious value: -[ - "positive", - "negative", - "destructive" -]New value: +[ + "positive", + "negative", + "destructive", + "Not set" +] - removed
Input schema / properties / isFlakyRemoved value: -{ - "description": "Filter test cases marked as flaky. Set to true to show only flaky tests, false for non-flaky.", - "type": "boolean" -} - changed
Input schema / properties / layer / enumPrevious value: -[ - "e2e", - "api", - "unit" -]New value: +[ + "e2e", + "api", + "unit", + "not set" +] - changed
Input schema / properties / limit / defaultPrevious value: -50New value: +10 - changed
Input schema / properties / limit / descriptionPrevious value: -"Maximum number of results to return (default: 50, max: 1000)."New value: +"Maximum number of results to return (default: 10, max: 1000)." - changed
Input schema / properties / priority / enumPrevious value: -[ - "critical", - "high", - "medium", - "low" -]New value: +[ + "high", + "medium", + "low", + "Not set" +] - changed
Input schema / properties / search / descriptionPrevious value: -"Search term to match against title, description, or caseId. Example: 'login' or 'TC-123'."New value: +"Search term to match against title or caseId. Example: 'login' or 'TC-123'." - changed
Input schema / properties / severity / enumPrevious value: -[ - "critical", - "major", - "minor", - "trivial" -]New value: +[ + "Blocker", + "critical", + "major", + "Normal", + "minor", + "trivial", + "Not set" +] - changed
Input schema / properties / status / enumPrevious value: -[ - "actual", - "draft", - "deprecated" -]New value: +[ + "active", + "draft", + "deprecated" +] - changed
Input schema / properties / tags / descriptionPrevious value: -"Filter by tags (comma-separated list). Example: 'smoke,regression' or 'critical'."New value: +"Filter by tags. Can be a single tag or comma-separated tags. Example: 'smoke' or 'smoke,regression,login'." - added
Input schema / properties / timeAdded value: +{ + "description": "Filter by time interval.", + "enum": [ + "last 1 hour", + "Last 5 hours", + "Yesterday", + "last 7 days" + ], + "type": "string" +} - changed
Input schema / properties / type / enumPrevious value: -[ - "functional", - "smoke", - "regression", - "security", - "performance", - "e2e" -]New value: +[ + "functional", + "smoke", + "regression", + "security", + "performance", + "e2e", + "Integration", + "API", + "Unit", + "Accessability", + "Compatibility", + "Acceptance", + "Exploratory", + "Usability", + "Other" +]
- Added
list_releases - Added
list_run_test_cases - Added
list_sessions - Changed
list_testruns1 field changed- changed
Input schema / properties / by_time_interval / descriptionPrevious value: -"Filter by time: '1d' (last day), '3d' (last 3 days), 'weekly' (last 7 days), 'monthly' (last 30 days), or '2024-01-01,2024-01-31' (date range)."New value: +"Filter by time interval. Supports: 'Latest' (most recent, no time filter), '1h' (last hour), '2h' (last 2 hours), '5h' (last 5 hours), '12h' (last 12 hours), '1d' (last day), '3d' (last 3 days), '5d' (last 5 days), 'weekly' (last 7 days), 'monthly' (last 30 days), 'last 1 hour', 'last day', 'last 5 days', or '2024-01-01,2024-01-31' (date range)."
- Added
test_audit - Added
update_manual_run - Changed
update_manual_test_case20 fields changed- changed
Input schema / properties / updates / descriptionPrevious value: -"Object containing the fields to update. Can include: title, description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions, etc."New value: +"Object containing the fields to update. Can include: name, description, steps, status, priority, severity, type, layer, behavior, preconditions, postconditions, automationStatus, tags, flags, attachments, customFields, etc." - added
Input schema / properties / updates / properties / attachmentsAdded value: +{ + "description": "Add or remove attachments (up to 10MB each).", + "properties": { + "add": { + "description": "Array of attachment URLs or file paths to add.", + "items": { + "type": "string" + }, + "type": "array" + }, + "remove": { + "description": "Array of attachment IDs or URLs to remove.", + "items": { + "type": "string" + }, + "type": "array" + } + }, + "type": "object" +} - added
Input schema / properties / updates / properties / automationStatusAdded value: +{ + "description": "Updated automation status.", + "enum": [ + "Manual", + "Automated", + "To be automated" + ], + "type": "string" +} - changed
Input schema / properties / updates / properties / behavior / enumPrevious value: -[ - "positive", - "negative", - "destructive" -]New value: +[ + "positive", + "negative", + "destructive", + "Not set" +] - added
Input schema / properties / updates / properties / commentsAdded value: +{ + "description": "Array of comment bodies to append to the test case. Each string becomes a new comment with the PAT owner as the author. The server enforces a cap of 20 comments per case; entries past the cap are rejected.", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / updates / properties / customFieldsAdded value: +{ + "additionalProperties": { + "type": "string" + }, + "description": "Updated custom fields as key-value pairs. Only available if custom fields are configured in test case management settings.", + "type": "object" +} - added
Input schema / properties / updates / properties / flagsAdded value: +{ + "description": "Updated automation flags/checklist options.", + "items": { + "enum": [ + "To be Automated", + "Is flaky", + "Muted" + ], + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / updates / properties / issuesAdded value: +{ + "description": "Array of Jira ticket keys to link to the test case (e.g. ['PROJ-123', 'ENG-45']). The server resolves each key against the project's connected Jira: tickets that are found are saved with title and URL; tickets that aren't found (or if no Jira is connected) are saved as plain text stubs โ same fallback as the UI's add-issue flow. Duplicate links for the same ticket are silently skipped.", + "items": { + "type": "string" + }, + "type": "array" +} - changed
Input schema / properties / updates / properties / layer / enumPrevious value: -[ - "e2e", - "api", - "unit" -]New value: +[ + "e2e", + "api", + "unit", + "not set" +] - added
Input schema / properties / updates / properties / nameAdded value: +{ + "description": "Updated test case name/title.", + "type": "string" +} - changed
Input schema / properties / updates / properties / priority / enumPrevious value: -[ - "critical", - "high", - "medium", - "low" -]New value: +[ + "high", + "medium", + "low", + "Not set" +] - changed
Input schema / properties / updates / properties / severity / enumPrevious value: -[ - "critical", - "major", - "minor", - "trivial" -]New value: +[ + "Blocker", + "critical", + "major", + "Normal", + "minor", + "trivial", + "Not set" +] - changed
Input schema / properties / updates / properties / status / enumPrevious value: -[ - "actual", - "draft", - "deprecated" -]New value: +[ + "Active", + "Draft", + "Deprecated" +] - changed
Input schema / properties / updates / properties / steps / descriptionPrevious value: -"Updated test steps array."New value: +"Updated test steps array. For Classic format: action, expectedResult, and optional data. For Gherkin format: event and stepDescription. Each top-level step can include attachments as URLs or local file paths." - added
Input schema / properties / updates / properties / steps / items / oneOfAdded value: +[ + { + "properties": { + "action": { + "description": "The action to perform in this step (Classic format).", + "type": "string" + }, + "attachments": { + "description": "Optional top-level step attachments as URLs or local file paths.", + "items": { + "type": "string" + }, + "type": "array" + }, + "data": { + "description": "Optional test data for this step (Classic format).", + "type": "string" + }, + "expectedResult": { + "description": "The expected outcome of this action (Classic format).", + "type": "string" + } + }, + "required": [ + "action", + "expectedResult" + ] + }, + { + "properties": { + "attachments": { + "description": "Optional top-level step attachments as URLs or local file paths.", + "items": { + "type": "string" + }, + "type": "array" + }, + "event": { + "enum": [ + "Given", + "When", + "And", + "Then", + "But" + ], + "type": "string" + }, + "stepDescription": { + "type": "string" + } + }, + "required": [ + "event", + "stepDescription" + ] + } +] - removed
Input schema / properties / updates / properties / steps / items / propertiesRemoved value: -{ - "action": { - "type": "string" - }, - "data": { - "type": "string" - }, - "expectedResult": { - "type": "string" - } -} - added
Input schema / properties / updates / properties / tagsAdded value: +{ + "description": "Updated tags.", + "type": "string" +} - added
Input schema / properties / updates / properties / testStepsDeclarationTypeAdded value: +{ + "description": "Updated test steps declaration type.", + "enum": [ + "Classic", + "Gherkin" + ], + "type": "string" +} - removed
Input schema / properties / updates / properties / titleRemoved value: -{ - "description": "Updated test case title.", - "type": "string" -} - changed
Input schema / properties / updates / properties / type / enumPrevious value: -[ - "functional", - "smoke", - "regression", - "security", - "performance", - "e2e" -]New value: +[ + "functional", + "smoke", + "regression", + "security", + "performance", + "e2e", + "Integration", + "API", + "Unit", + "Accessability", + "Compatibility", + "Acceptance", + "Exploratory", + "Usability", + "Other" +]
- Added
update_release - Added
update_run_test_case - Added
update_session
12 tool updates
- First observed
create_manual_test_case - First observed
create_manual_test_suite - First observed
debug_testcase - First observed
get_manual_test_case - First observed
get_run_details - First observed
get_testcase_details - First observed
health - First observed
list_manual_test_cases - First observed
list_manual_test_suites - First observed
list_testcase - First observed
list_testruns - First observed
update_manual_test_case
TDQS
Scored across 35 tools
Some tools have overlapping purposes or very similar names, such as list_testcase vs list_manual_test_cases and get_testcase_details vs get_manual_test_case, which differ only by 'manual'. Additionally, get_run_details and get_ai_insights both surface AI insights, requiring careful attention to the polling flow. Descriptions are detailed and help differentiate, but the boundaries are not always crisp.
Most tools follow a verb_noun pattern, but object naming is inconsistent: 'testruns' vs 'test_run' vs 'run', and 'testcase' vs 'test_case' (e.g., list_testruns vs list_run_test_cases, list_testcase vs list_manual_test_cases). The tool 'health' breaks the pattern, and pluralization is inconsistent (list_testcase vs list_manual_test_cases). Still, the style is readable and mostly uniform.
35 tools is a very large surface for an MCP server, even for a comprehensive QA platform. While the scope is broad (automated runs, manual cases, releases, sessions, integrations, AI, audit), many tools are CRUD variants that could potentially be combined or streamlined. The count exceeds the threshold for 'too many' and would likely increase selection difficulty.
The tool surface covers CRUD for manual test cases, releases, manual runs, and sessions, but lacks delete operations for these entities (e.g., delete_manual_test_case, delete_release, delete_session). Automated test runs are read-only (list/get) with no way to trigger or modify them. The audit flow and integrations are well covered, but the missing lifecycle operations are notable.
Maintenance
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Manage test suites, run tests, view results, and automate QA workflows via AI with testRigor.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute โ analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation โ pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI โ explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility โ audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 โ no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Connect AI agents to Process Street workflows, tasks, runs, data sets, and operational records.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.1812 npmMIT
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to interact with Testmo test management platform for creating, reading, updating, and deleting test cases, managing folders, and organizing test runs through natural language.4MIT
- AlicenseAqualityBmaintenanceConnects AI coding assistants to TestCollab for managing test cases, plans, and suites directly through natural language. It enables users to create, update, and query testing resources within integrated development environments and AI chat clients.1717 npm4MIT

Tesults MCPofficial
AlicenseNot gradedqualityDmaintenanceConnect AI agents to your test results, insights, and targets. Query test runs, failures, flaky tests, and regressions across frameworks including Playwright, Jest, Pytest, Cypress and more.23 npmMIT