Zebrunner MCP Server
The Zebrunner MCP Server integrates with Zebrunner Test Case Management to enable QA teams, developers, and managers to comprehensively manage, analyze, and improve test cases through natural language commands via AI assistants.
Core Capabilities:
Test Case Management: Retrieve detailed test case information by key, title, or advanced filters (automation state, priority, dates), with support for batch operations and comprehensive pagination
Test Suite Organization: Navigate hierarchical test suite structures, get tree views with configurable depth, and identify root suites and subsuites
Quality Validation & Improvement: Validate test cases against 100+ checkpoints using a 3-tier intelligent rules system, receive AI-powered improvement suggestions, and apply automated fixes
Test Coverage & Code Generation: Analyze test case coverage against actual implementations, generate draft test code for multiple frameworks (Java/Carina, JavaScript/Jest, Python/Pytest), and perform enhanced rules-based coverage analysis
Duplicate Analysis: Identify similar test cases using both step-based similarity and advanced LLM-powered semantic analysis with detailed similarity matrices
Launch & Execution Management: Access comprehensive launch details, summaries, and test execution results with filtering by milestone, build number, or launch name
Reporting & Analytics: Platform-specific test results by time period, top bug analysis with issue links, project milestone tracking, and completion status
Configuration & Customization: Customizable rules system via Markdown files, multiple output formats (JSON, markdown, string, DTO), and clickable links to Zebrunner web UI
The server supports framework detection, intelligent validation, semantic analysis with LLM-powered clustering, and provides comprehensive test run management capabilities.
Supports Android platform test execution and results analysis through Zebrunner's test management system
Supports iOS platform test execution and results analysis through Zebrunner's test management system
Generates automated test code in JavaScript with frameworks like Jest based on test cases stored in Zebrunner
Generates Jest test code from Zebrunner test cases with intelligent framework detection and coverage analysis
Generates rich formatted reports and documentation from Zebrunner test data in Markdown format
Creates visual diagrams showing test suite hierarchies and test case relationships from Zebrunner data
Generates Python pytest code from Zebrunner test cases with automated framework detection
Generates automated test code in Python based on test cases and test execution data from Zebrunner
Generates Selenium WebDriver test automation code from Zebrunner test cases with coverage analysis
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Zebrunner MCP Servershow me test cases for the login feature that need automation"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Advanced Zebrunner MCP Server
A Model Context Protocol (MCP) server that brings advanced analytics, reporting, failure forensics, and safe TCM mutations to Zebrunner. Designed to coexist with the official Zebrunner MCP (beta) โ see the "Dual-MCP setup" section below.
Tool naming: every tool on this server is registered under the canonical
adv_<name>form (e.g.adv_create_test_case,adv_list_test_runs) so it never collides with the official Zebrunner MCP. The legacy names are kept as deprecated aliases so prompts/scripts that called the old names continue to work for now; aliases will be removed in the next major release.
๐ v9.2.8 โ New
adv_analyze_test_impact+/test-impactprompt for PR/code-change test planning (regression + coverage gaps). See TEST_IMPACT_WORKFLOW.md and release notes.๐ v9.2.7 โ New
adv_scaffold_test_casewizard (hybrid form/conversational) to author test cases from best practices with an automatic warn-only similar-case check, plus configurableprojectAliasesinzebrunner-config.json. See release notes.๐ v9.2.5 โ
adv_get_test_authoring_trend(TAM template 7) completes 22/22 dashboard widget MCP coverage. Hub tools, pass-rate views, period modes. See TEST_PROMPTS.md ยง18 and change-logs.md.
v9.2.1 โ LLM-visible metrics (
include_call_metrics, session breakdown), compact expansion (+19 tools), format/truncation bugfixes. See Token efficiency guide and change-logs.md.
v9.2.0 โ Opt-in token/cost optimizations:
format:'compact',detail:'summary',adv_batch_get_test_cases, reportinline:false. Defaults unchanged. See change-logs.md.
v9.1.0 โ Launch mutations:
adv_rerun_launch_failures,adv_start_launch(Jenkins Build Now), plus/relaunch-regression-failuresand/feature-scoped-launchprompts. See GitHub Release v9.1.0.๐ Need help with installation? Check out our Step-by-Step Install Guide for detailed setup instructions.
๐ Installing via npm? See our MCP NPM Installation Guide for Claude Desktop, Cursor, IntelliJ IDEA, and ChatGPT Desktop configuration.
๐ Test vs Test Case vs Launch? See Terminology & Glossary โ definitions, counting rules, and how metrics are calculated.
๐ฅ Why This Server
This is the Advanced Zebrunner MCP Server โ built to go well beyond basic test case management and help QA teams work smarter and faster with AI. Compared to the official Zebrunner MCP (beta, ~70 tools spanning Public REST + Reporting/TAM/Launcher), this server provides 69 analytics-focused tools (adv_* prefix) and is safe to run side-by-side with the official server:
Reporting & Analytics โ dashboards, pass-rate trends, regression stability reports, runtime efficiency analysis, bug reviews, and weekly delta tracking
Test Coverage & Analysis โ coverage gaps, automation readiness scoring, and cross-suite analysis
Intelligent Rules System โ 3-tier quality rules with 100+ automated checkpoints for test case validation and improvement
Screenshot & Video Analysis โ OCR, image processing, and video analysis powered by sharp, ffmpeg, and tesseract
MCP Resources & Prompts โ
@context injection and/workflow commands for streamlined daily operationsMulti-Transport & Multi-Auth โ STDIO, Streamable HTTP, self-service OAuth, Okta SSO, and token exchange (Modes 1โ5)
Related MCP server: MCP Universal Test Suite Generator
๐ Table of Contents
๐ฏ What is this tool?
This tool allows you to:
Retrieve test cases and test suites from Zebrunner
Analyze test coverage and generate test code
Get test execution results and launch details
Validate test case quality with automated checks using intelligent rules
Generate reports and insights from your test data
Improve test cases with AI-powered suggestions and automated fixes
All through natural language commands in AI assistants!
๐ง Intelligent Rules System
What Makes This Tool Special
Our MCP server includes a sophisticated 3-tier rules system that transforms how you work with test cases:
๐ฏ Test Case Review Rules (test_case_review_rules.md)
Purpose: Core quality standards and writing guidelines
What it does: Defines fundamental principles for writing high-quality test cases
Key areas: Independence, single responsibility, comprehensive preconditions, complete step coverage
Used by:
adv_validate_test_caseandadv_improve_test_casetools
โ
Test Case Analysis Checkpoints (test_case_analysis_checkpoints.md)
Purpose: Detailed validation checklist with 100+ checkpoints
What it does: Provides granular validation criteria for thorough test case analysis
Key areas: Structure validation, automation readiness, platform considerations, quality assurance
Used by:
adv_validate_test_casefor comprehensive scoring and issue detection
โ๏ธ MCP Zebrunner Rules (mcp-zebrunner-rules.md)
Purpose: Technical configuration for test generation and coverage analysis
What it does: Defines framework detection patterns, code templates, and coverage thresholds
Key areas: Framework detection, test generation templates, coverage thresholds, quality standards
Used by:
adv_generate_draft_test_by_keyandadv_get_enhanced_test_coverage_with_rulestools
How the Rules Work Together
graph TD
A[Test Case] --> B[adv_validate_test_case]
B --> C[test_case_review_rules.md]
B --> D[test_case_analysis_checkpoints.md]
B --> E[Validation Result + Issues]
E --> F[adv_improve_test_case]
F --> G[AI-Powered Improvements]
A --> H[adv_generate_draft_test_by_key]
H --> I[mcp-zebrunner-rules.md]
H --> J[Generated Test Code]
A --> K[adv_get_enhanced_test_coverage_with_rules]
K --> I
K --> L[Coverage Analysis + Rules Validation]Why This Matters
Consistency: All team members follow the same quality standards
Automation: Reduce manual review time with automated validation
Learning: New team members learn best practices through AI feedback
Customization: Adapt rules to your project's specific needs
Continuous Improvement: AI suggests improvements based on proven patterns
Customizing Rules for Your Project
You can customize any of the three rules files:
# Copy default rules to customize
cp test_case_review_rules.md my-project-review-rules.md
cp test_case_analysis_checkpoints.md my-project-checkpoints.md
cp mcp-zebrunner-rules.md my-project-technical-rules.md
# Use custom rules in validation
"Validate test case PROJ-123 using custom rules from my-project-review-rules.md"Example customizations:
Mobile projects: Add mobile-specific validation rules
API projects: Focus on API testing patterns and data validation
Different frameworks: Customize code generation templates
Company standards: Align with your organization's testing guidelines
๐ Prerequisites
What you need to know
Basic command line usage (opening terminal, running commands)
Your Zebrunner credentials (login and API token)
Basic understanding of test management (test cases, test suites)
Software requirements
Node.js 18 or newer - Download here
npm (comes with Node.js)
Access to a Zebrunner instance with API credentials
How to check if you have Node.js
Open your terminal/command prompt and run:
node --version
npm --versionIf you see version numbers, you're ready to go!
๐ Quick Start Guide
๐ก Want more detailed instructions? Check out our More Detailed Step-by-step Install Guide with troubleshooting tips and platform-specific instructions.
Step 1: Get the code
Choose one of these methods:
Option A: Clone from repository (recommended)
git clone https://github.com/maksimsarychau/mcp-zebrunner.git
cd mcp-zebrunnerOption B: Download and extract
Download the project files and extract them to a folder.
Step 2: Install dependencies
npm installStep 3: Configure your Zebrunner connection
Create a .env file in the project folder with your Zebrunner details:
# Your Zebrunner instance URL (without trailing slash)
ZEBRUNNER_URL=https://your-company.zebrunner.com/api/public/v1
# Your Zebrunner login (usually your email)
ZEBRUNNER_LOGIN=your.email@company.com
# Your Zebrunner API token (get this from your Zebrunner profile)
ZEBRUNNER_TOKEN=your_api_token_here
# Optional: Enable debug logging (default: false)
DEBUG=false
# Optional: Enable intelligent rules system (auto-detected if rules file exists)
ENABLE_RULES_ENGINE=trueHow to get your Zebrunner API token:
Log into your Zebrunner instance
Go to your profile settings
Find the "API Access" section
Generate a new API token
Copy the token to your
.envfile
Step 4: Build the project
npm run buildStep 5: Test your connection
npm run test:healthIf you see "โ Health check completed", you're ready to go!
๐ Updating to New Version
Check current version
# Check your current version
npm run version
# or manually check package.json
cat package.json | grep '"version"'Update steps
# 1. Pull latest changes from master branch
git pull origin master
# 2. Install any new dependencies
npm install
# 3. Rebuild the project
npm run build
# 4. Test your connection (requires valid .env file)
npm run test:healthImportant Notes:
โ Your
.envfile must be properly configured for the health check to workโ Restart Claude Desktop/Code after updating to reload the MCP server
โ Check release notes for any breaking changes before updating
If the health check fails, verify your .env configuration and Zebrunner credentials.
๐ง Usage Methods
Method 1: Use with Claude Desktop/Code (Recommended)
Add this configuration to your Claude Desktop or Claude Code settings. Important: You must use the full absolute path to your project folder.
{
"mcpServers": {
"mcp-zebrunner": {
"command": "node",
"args": ["/full/absolute/path/to/mcp-zebrunner/dist/server.js"],
"env": {
"ZEBRUNNER_URL": "https://your-company.zebrunner.com/api/public/v1",
"ZEBRUNNER_LOGIN": "your.email@company.com",
"ZEBRUNNER_TOKEN": "your_api_token_here",
"DEBUG": "false",
"ENABLE_RULES_ENGINE": "true",
"DEFAULT_PAGE_SIZE": "100",
"MAX_PAGE_SIZE": "100"
}
}
}
}Example paths:
Windows:
C:\\Users\\YourName\\Projects\\mcp-zebrunner\\dist\\server.jsmacOS/Linux:
/Users/YourName/Projects/mcp-zebrunner/dist/server.js
Alternative: Command Line Integration (Claude Code)
You can also add the server using the command line:
claude mcp add mcp-zebrunner \
--env ZEBRUNNER_URL="https://your-company.zebrunner.com/api/public/v1" \
--env ZEBRUNNER_LOGIN="your.email@company.com" \
--env ZEBRUNNER_TOKEN="your_api_token_here" \
--env DEBUG="false" \
--env ENABLE_RULES_ENGINE="true" \
-- node /full/absolute/path/to/mcp-zebrunner/dist/server.jsImportant: Replace /full/absolute/path/to/mcp-zebrunner/ with the actual full path to your project folder.
Method 2: Run as standalone server
Development mode (with auto-reload)
npm run devProduction mode
npm startMethod 3: Smart URL-Based Analysis ๐
NEW in v5.4.1+: Claude can automatically detect Zebrunner URLs and analyze them with optimal settings!
Just paste a Zebrunner URL in your conversation, and Claude will automatically:
Parse the URL to extract project, launch, and test IDs
Call the appropriate analysis tool
Use recommended settings (videos, screenshots, AI analysis enabled)
๐ Supported URL Patterns
1. Test Analysis URLs
https://your-workspace.zebrunner.com/projects/PROJECT/automation-launches/LAUNCH_ID/tests/TEST_IDWhat happens:
Claude automatically calls
adv_analyze_test_failureExtracts:
projectKey,testRunId(launch ID),testIdEnables:
includeVideo: true,analyzeScreenshotsWithAI: true, all diagnostics
Example:
User: "Analyze https://your-workspace.zebrunner.com/projects/MCP/automation-launches/120911/tests/5455386"
Claude automatically calls:
{
projectKey: "MCP",
testRunId: 120911,
testId: 5455386,
includeVideo: true,
analyzeScreenshotsWithAI: true,
includeLogs: true,
includeScreenshots: true,
analyzeSimilarFailures: true,
screenshotAnalysisType: "detailed",
format: "detailed"
}2. Launch Analysis URLs
https://your-workspace.zebrunner.com/projects/PROJECT/automation-launches/LAUNCH_IDWhat happens:
Claude automatically calls
adv_detailed_analyze_launch_failuresExtracts:
projectKey,testRunId(launch ID)Enables:
includeScreenshotAnalysis: true, comprehensive analysis
Example:
User: "Analyze https://your-workspace.zebrunner.com/projects/MCP/automation-launches/120911"
Claude automatically calls:
{
projectKey: "MCP",
testRunId: 120911,
filterType: "without_issues",
includeScreenshotAnalysis: true,
screenshotAnalysisType: "detailed",
format: "summary",
executionMode: "sequential"
}โจ Advanced Usage
Override Default Settings
Claude understands natural language overrides:
User: "Analyze https://...url... but without screenshots"
โ Claude sets: analyzeScreenshotsWithAI: false
User: "Analyze https://...url... in jira format"
โ Claude sets: format: "jira"
User: "Quick analysis of https://...url..."
โ Claude sets: format: "summary", screenshotAnalysisType: "basic"Multiple URLs
Analyze multiple tests/launches in one request:
User: "Compare these failures:
https://your-workspace.zebrunner.com/projects/MCP/automation-launches/120911/tests/5455386
https://your-workspace.zebrunner.com/projects/MCP/automation-launches/120911/tests/5455390"
โ Claude analyzes both sequentially and compares resultsCross-Workspace Support
โ ๏ธ URLs from different workspaces will show a warning but still attempt analysis:
User: "Analyze https://other-workspace.zebrunner.com/..."
โ Claude warns: "URL is from 'other-workspace.zebrunner.com' but configured workspace is 'your-workspace.zebrunner.com'"
โ Proceeds with analysis using available credentials๐ URL Pattern Reference
Component | Example | Extracted As | Used In Tool |
Workspace |
| Validation only | N/A |
Project Key |
|
| All tools |
Launch ID |
|
| All tools |
Test ID |
|
|
|
๐ฏ Why Use URL-Based Analysis?
โ
Faster: No need to manually specify IDs
โ
Convenient: Copy-paste URLs directly from Zebrunner UI
โ
Optimized: Automatic use of recommended settings
โ
Smart: Claude detects intent and adjusts parameters
โ
Flexible: Natural language overrides work seamlessly
๐ก Pro Tips
Direct from Zebrunner: Copy URL directly from your browser while viewing a test/launch
Batch Analysis: Paste multiple URLs separated by newlines
Custom Settings: Add natural language instructions to override defaults
Quick Checks: URLs work great for quick "what happened here?" questions
Reports: Combine with format requests: "Generate JIRA ticket for https://...url..."
๐ค Dual-MCP setup (official zebrunner + Advanced)
Zebrunner now offers an official hosted MCP (beta) at
https://{workspace}.zebrunner.com/api/mcp. It is safe to run both servers
in the same Cursor / Claude session: every tool on this server is exposed under
the adv_<name> prefix, so it never collides with the official tools.
Example .cursor/mcp.json:
{
"mcpServers": {
"zebrunner": {
"url": "https://your-workspace.zebrunner.com/api/mcp",
"headers": {
"X-Zebrunner-Username": "your.name@company.com",
"X-Zebrunner-Api-Token": "<your-zebrunner-api-token>"
}
},
"mcp-zebrunner": {
"command": "npx",
"args": ["-y", "mcp-zebrunner"],
"env": {
"ZEBRUNNER_URL": "https://your-workspace.zebrunner.com/api/public/v1",
"ZEBRUNNER_LOGIN": "your.name@company.com",
"ZEBRUNNER_TOKEN": "<your-zebrunner-api-token>"
}
}
}
}Rule of thumb when both are connected:
Use | Use |
| Failure forensics, runtime + flakiness analytics |
Server-side | Reporting, dashboards, regression analyzers |
Bulk launch/test comments, issue linking, launcher/preset CRUD, | Preview/confirm TCM mutations; Jenkins Build Now / rerun failures |
Inside chat, ask adv_about_mcp_tools with mode: "routing" or open the
zebrunner://mcp-routing resource for the full live-generated diff and routing table.
๐ ๏ธ Available Tools
Once connected, you can use these tools through natural language in your AI assistant. This section highlights all 69 tools organized by category. For the complete catalog with natural-language examples for every tool, see TOOLS_CATALOG.md.
Dashboard widgets: All 22 Zebrunner dashboard templates have MCP coverage (v9.2.5). See docs/TEST_PROMPTS.md ยง18 for example prompts and npm run test:api verification.
Token-efficient reads
Large TCM exports can consume significant context. Use these opt-in parameters (defaults stay backward-compatible). Full guide: docs/TOKEN_EFFICIENCY.md.
Technique | Example | When to use |
|
| Minified JSON (~22% smaller than pretty |
|
| After filtering โ returns id, key, title, priority, automationState, webUrl only |
|
| Fetch a shortlist without N round-trips |
| Writes HTML/PNG to disk, returns paths | Huge dashboards in chat clients |
| Any bulk TCM/suite read | Metrics without payload |
Env flags (off) |
| Server-wide default flips after eval passes |
Workflow: list/filter with detail:'summary' + format:'compact' โ adv_get_test_case_by_key for full body (steps, preconditions) before create/update.
Example prompts for your assistant:
"List test cases in project MCP with summary detail and compact JSON."
"Fetch MCP-1 and MCP-2 in one batch call with summary and compact format."
"Generate a quality dashboard for MCP with inline false โ save to disk."
Tool naming: All tools are exposed as
adv_<name>(e.g.adv_get_test_case_by_key). Use these names in prompts, scripts, and when both the official and Advanced MCP are connected. Legacy short names (get_test_case_by_key, etc.) are deprecated and only registered whenZEBRUNNER_REGISTER_LEGACY_ALIASES=true.
๐ Test Case Management
Core Test Case Tools
Tool | Description | Example Usage | Best For |
| Get detailed test case information |
| All roles |
| Fetch multiple cases by key (partial success) |
| All roles |
| Advanced filtering with automation states, dates |
| QA, SDETs |
| Filter by specific automation states |
| SDETs, Managers |
| Search test cases by title (partial match) |
| All roles |
| Advanced filtering by suite, dates, priority, automation state |
| QA, Managers |
| List available automation states |
| All roles |
| List available priorities with IDs |
| All roles |
Batch Test Case Operations
Tool | Description | Example Usage | Best For |
| Get ALL test cases (handles pagination) |
| Managers, Leads |
| All test cases with hierarchy info |
| Analysts |
| Smart suite test case retrieval with root/child auto-detection |
| QA, Analysts |
Test Case Change History
Most test case tools support optional change history enrichment โ fetching the audit log of modifications for each test case. This is useful for understanding when steps were changed, when automation state transitioned, or when a test case was deprecated.
Parameter | Type | Default | Description |
| boolean |
| Attach a |
|
|
| Filter: step/precondition diffs only, lifecycle events only, or all changes |
| number (1โ100) |
| Max history entries per test case |
Supported on: adv_get_test_case_by_key, adv_get_test_cases_advanced, adv_get_test_cases_by_automation_state, adv_get_test_case_by_title, adv_get_test_case_by_filter, adv_get_all_tcm_test_cases_by_project, adv_get_test_cases_by_suite_smart
Example prompts:
"Get test case MCP-29 with change history"โ setsinclude_history=true"Show me MCP-29 history, events only"โ setsinclude_history=true,history_filter='events_only'"Get all automated test cases with their last 5 changes"โ setsinclude_history=true,history_limit=5
Detected events: became_automated, became_manual, became_deprecated, became_undeprecated, steps_changed, preconditions_changed, postconditions_changed, and dynamically generated became_<state_name> for any project-specific automation state.
๐ณ Test Suite Hierarchy & Organization
Suite Management
Tool | Description | Example Usage | Best For |
| List suites with pagination |
| All roles |
| Hierarchical tree view |
| Managers, QA |
| Get top-level suites |
| Managers |
| Get all child suites |
| QA, Analysts |
Suite Analysis Tools
Tool | Description | Example Usage | Best For |
| Get suite by ID (simple: fast direct API, full: hierarchy-enriched) |
| All roles |
| Comprehensive suite listing |
| Managers |
| Get all TCM test case suites with pagination |
| Managers, Analysts |
| Find root suite for any suite |
| Analysts |
๐ง Mutation Tools (Beta)
Safety Model: Every mutation tool follows a two-call confirmation gate. The first call returns a preview; only after user approval should
confirm: truebe passed to execute the mutation. All mutations are audit-logged to~/.mcp-zebrunner-audit.jsonl. Usedry_run: truefor raw payload inspection.Next-step steering (v7.2.2): After every successful mutation, the server appends a
Tip:block guiding the LLM to the most useful next action (e.g., "validate quality", "publish the draft", "populate the test run"). Hints are conditional -- they are suppressed when redundant. For example, the quality-check hint is omitted ifreview: truewas already used. Created test cases are always forced todraft=true, and the hint always reminds the LLM to publish viaadv_update_test_case. This approach is inspired by the Strands Agents steering pattern and delivers just-in-time guidance without bloating system prompts.
Suite Mutations
Tool | Description | Example Usage | Best For |
| (Beta) Create a new Test Suite |
| QA, Managers |
| (Beta) Update an existing Test Suite (full PUT) |
| QA, Managers |
Test Case Mutations
Tool | Description | Example Usage | Best For |
| (Beta) Create a new Test Case with runtime validation of priorities, automation states, and custom fields. Accepts |
| QA, SDETs |
| (Beta) Partially update a Test Case by ID or key (PATCH). Accepts |
| QA, SDETs |
๐ Test Coverage & Analysis
Test Impact (PR / code changes) (v9.2.8)
Tool | Description | Example Usage | Best For |
| Rank Zebrunner cases affected by code changes from compact semantic context |
| Developers, SDETs |
Use the /test-impact MCP prompt (optional pr_url) for guided workflow. See TEST_IMPACT_WORKFLOW.md. Configure repositoryProjectMap via .env.example.
Coverage Analysis
Tool | Description | Example Usage | Best For |
| Analyze implementation coverage |
| Developers, SDETs |
| Rules-based coverage analysis |
| SDETs, Leads |
Duplicate Analysis
Tool | Description | Example Usage | Best For |
| Find and group similar test cases by step similarity |
| QA Managers, SDETs |
| Advanced semantic analysis with LLM-powered step clustering |
| Senior QA, Test Architects |
๐ Clickable Links Feature: Both duplicate analysis tools support clickable links to Zebrunner web UI:
Add
include_clickable_links: trueto make test case keys clickable in markdown outputJSON/DTO formats automatically include
webUrlfields when enabledLinks are generated from your
ZEBRUNNER_URLenvironment variableExample:
"Analyze suite 17585 for duplicates with clickable links enabled"
๐งช Test Code Generation & Validation
AI-Powered Tools
Tool | Description | Example Usage | Best For |
| Generate test code with framework detection |
| SDETs, Developers |
| Quality validation with improvement |
| QA, Managers |
| Dedicated improvement tool |
| QA, SDETs |
๐ Launch & Execution Management
Launch Operations โญ Essential for Managers
Tool | Description | Example Usage | Best For |
| Comprehensive launch information; optional |
| Managers, Leads, SDETs |
| ๐ v9.1.0 Rerun failed/aborted tests for one or many launches (preview/confirm) |
| Managers, SDETs |
| ๐ v9.1.0 Trigger Jenkins Build Now (not Launch Launchers); preview/confirm |
| Managers, SDETs |
| Quick launch overview |
| Managers |
| List individual launch executions with pagination |
| Managers, Leads |
| Search launches by milestone/build/name |
| Managers, Leads |
| Weekly regression stability report with WoW delta, linked issues, and strict Jira-ready output. Supports launch list or build-based auto-discovery (version-segment build lookup with |
| Managers, Leads |
| Regression Runtime Efficiency โ per-launch elapsed time, attempt/re-run breakdown, configurable duration classification (Short/Medium/Long), dual metrics for both Tests and Test Cases (Average Runtime, WRI), duration distribution with test case counts, and baseline comparison with delta tracking. |
| Managers, Leads, SDETs |
adv_analyze_regression_runtimeโ Usage Examples:
"Analyze regression runtime for the iOS project on the latest milestone. Show both average runtime per test and per test case, plus WRI and WRI per test case."
"Run regression runtime analysis for all three projects on their latest milestones. Compare how many test cases fall into Short vs Medium vs Long buckets per team."
"Analyze regression runtime for the Android project, latest vs previous milestone. Show WRI and WRI per test case for both. Has efficiency improved or degraded?"
"Analyze regression runtime with medium_threshold_seconds: 120 and long_threshold_seconds: 300. Show which tests fall into each bucket."
"What is the average time cost per test case in each duration bucket? Are long-running tests covering proportionally more test cases?"
๐ Reporting & Analytics
Test Failure Analysis ๐ Game Changer
Tool | Description | Example Usage | Best For |
| Deep forensic analysis of failed tests with logs, screenshots, error classification, and recommendations. NEW in v5.11.0: Compare with last passed execution! Shows what changed (logs, duration, environment). Also: |
| QA Engineers, SDETs, Managers |
| ๐ NEW in v5.11.0! Track test execution trends across launches. View pass/fail history, find last passed execution, calculate pass rate. Critical Detection: Highlights when test failed in all recent runs! |
| QA Engineers, SDETs, Managers |
| ๐ Enhanced v4.12.1 Analyze failures WITHOUT linked issues with Claude-level intelligence + Jira format support. Auto-deep-dive with executive summary, timeline, patterns, priorities. NEW: Generate Jira-ready tickets for entire launches! |
| QA Managers, SDETs, Team Leads |
โ FIXED in v5.2.4! Improved Reliability & Video Links
๐ฅ Video URLs fixed: Now uses test-sessions API (
/api/reporting/v1/launches/{id}/test-sessions) for reliable video artifact extraction๐ก๏ธ Comprehensive error handling: Gracefully handles missing screenshots/logs (returns empty arrays instead of throwing)
๐ซ No more "no result received" errors: All API calls have proper try-catch blocks with fallbacks
๐ Better debugging: Enhanced logging when
debug: trueis enabled in config๐ Schema updates: Supports both old and new API structures for backward compatibility
๐ซ NEW in v4.12.1! Jira-Ready Ticket Format
Use
format: 'jira'to generate ready-to-paste Jira ticketsAuto-calculated priority based on stability and impact
Smart labels:
test-automation,locator-issue,flaky-test, etc.Complete Jira markup: Tables, panels, code blocks, clickable links
๐ฅ Prominent video links: Beautiful panels + links section
Copy-paste ready: No manual formatting needed
Saves 5-10 minutes per ticket with consistent quality
๐ง Enhanced in v4.11.1!
adv_detailed_analyze_launch_failuresprovides automatic deep synthesis like Claude would manually provide:
๐ฏ Executive Summary: Key findings, patterns, and stability indicators
๐ Timeline Analysis: When failures first appeared, progression tracking
๐ฌ Pattern Analysis: Groups by root cause with affected tests and stability %
๐ฏ Priority-Based Recommendations: ๐ด HIGH / ๐ก MEDIUM / ๐ข LOW with impact analysis
๐ Enhanced Test Details: Full error messages, stack traces, timestamps
โ Smart Follow-up Questions: Guides next investigation steps
Smart filtering: Analyzes only tests WITHOUT linked issues by default
Optional AI screenshot analysis for all tests
No manual follow-up needed - get complete picture in one call!
๐ NEW in v5.11.0! Test Execution History & Comparison
๐ Track execution trends: View pass/fail history across launches with
adv_get_test_execution_history๐ Compare with last passed: New
compareWithLastPassedparameter inadv_analyze_test_failure
Compare logs (new errors detection)
Compare duration (performance regression)
Compare environment (device/platform changes)
Compare screenshots (visual differences)
โ ๏ธ Critical detection: Automatically highlights when test failed in all recent executions
๐ฏ Regression analysis: See exactly what changed between passed and failed runs
๐ Pass rate metrics: Calculate test stability over time
See TOOLS_CATALOG.md for example prompts!
Screenshot Analysis & Visual Forensics ๐ฏ Enhanced in v4.11.0
Tool | Description | Example Usage | Best For |
| Download protected screenshots from Zebrunner with authentication |
| QA Engineers, Automation Engineers |
| Visual analysis with OCR, UI detection, and Claude Vision |
| QA Engineers, SDETs, Developers |
๐ธ Enhanced! Screenshot analysis now integrated directly into
adv_analyze_test_failureandanalyze_launch_failures- no need to call separately! See Screenshot Analysis Guide for details.
Platform & Results Analysis โญ Critical for Management
Tool | Description | Example Usage | Best For |
| Aggregated test results, pass rate, and statistics for a project over a time period |
| Managers, Leads |
| Most frequent defects |
| Managers, Developers |
| Detailed bug review with failure analysis, priority breakdown, and automatic detail fetching |
| Managers, QA, Developers |
| Comprehensive failure info by hashcode (alternative to auto-fetch) |
| Developers, SDETs |
| Available milestones |
| Managers, PMs |
Period behavior for reporting widgets: period is passed to Zebrunner widgets as-is (for example Today, Month, Last 30 Days). If a selected widget does not support a specific period label, the API will return an error.
Project Discovery
Tool | Description | Example Usage | Best For |
| Discover all accessible projects |
| All roles |
| Test API connectivity |
| All roles |
| Summarize tools, prompts, resources, or show session metrics. Modes: summary, tool, prompts, resources, metrics |
| All roles |
๐ Test Run Management
Public API Test Runs โญ Powerful for Analysis
Tool | Description | Example Usage | Best For |
| Advanced test run filtering |
| Managers, SDETs |
| Detailed test run information |
| Managers, QA |
| Test cases in a specific run |
| QA, Analysts |
Configuration Management
Tool | Description | Example Usage | Best For |
| Available result statuses |
| QA, SDETs |
| Configuration options |
| SDETs, Leads |
๐ MCP Resources & Prompts (NEW)
Full guide: docs/RESOURCES_AND_PROMPTS.md โ detailed usage, examples, reference tables, and contributor guide.
In addition to 69 tools, the server now provides 14 resources and 17 prompts that improve discoverability and automate complex workflows.
Resources โ @ Context Injection
Resources are read-only reference data you attach to your conversation via the @ menu. They help the AI use exact parameter values instead of guessing.
Resource | What it provides |
| 6 report types with params, defaults, and examples |
| 12 valid time period values (case-sensitive) |
| Chart formats, types, and 17 supported tools |
| 5 output format families with valid values |
| All accessible projects with keys and IDs |
| Root test suites for a project |
| Full suite tree with parent-child relationships |
| Automation state names and IDs |
| Priority levels and IDs |
| Active and completed milestones |
| Test run result statuses |
| Test run config groups and options |
| System and custom field definitions |
Quick example:
@ zebrunner://reports/types
@ zebrunner://projects
Generate an executive dashboard for all starred projects.Prompts โ / Workflow Commands
Prompts are pre-built workflow instructions triggered via the / command menu. Each prompt guides the AI through a tested multi-tool orchestration.
Prompt | Parameters | What it does |
|
| Cross-platform pass rate with target comparison |
|
| Runtime metrics with milestone delta |
|
| 7-metric coverage + automation intake rate |
|
| 5-section standup-ready dashboard |
|
| Go/No-Go assessment with evidence |
|
| Per-suite coverage tables (TOTAL + TOTAL REGRESSION) |
|
| Regression results overview, new bugs, top bugs, slowest tests |
|
| Validate + improve a test case |
|
| Post-regression failure analysis |
|
| ๐ v9.1.0 Discover failed launches and batch-rerun failures |
|
| ๐ v9.1.0 Feature keyword โ test_run_rules โ Build Now per root suite |
|
| Flaky test detection + stabilization plan |
|
| Structural + semantic duplicate analysis |
|
| Daily standup summary with action items |
|
| Automation backlog prioritization |
|
| Comprehensive project health card |
| (none) | Show tool usage stats for the current session |
Quick example:
/executive-dashboard
projects: android,ios,web๐ฏ Management-Focused Quick Commands
๐ Daily Standup Reports
# Get yesterday's results
"Get platform results for last 7 days for project MCP"
# Check recent failures
"Show me top 5 bugs from last week"
# Review recent launches
"Get all launches for project MCP from last 3 days"๐ Test Suite Optimization
# Basic duplicate analysis
"Analyze suite 12345 for duplicates with 80% similarity threshold"
# Advanced semantic analysis with step clustering
"Semantic analysis of suite 12345 with 85% step clustering and medoid selection"
# Analyze specific test cases for duplicates
"Analyze test cases MCP-123, MCP-124, MCP-125 for duplicates"
# Project-wide duplicate analysis (use with caution - large datasets)
"Analyze project MCP for test case duplicates with 85% similarity"
# Get detailed similarity matrix with pattern types
"Analyze suite 12345 for duplicates with similarity matrix included"
# Two-phase clustering with semantic insights
"Semantic duplicate analysis with step clustering threshold 90% and insights enabled"
# Enable clickable links for easy navigation
"Analyze suite 17585 for duplicates with clickable links enabled"๐ Weekly Management Reports
# Comprehensive project health
"Get all launches for project MCP with milestone filter"
# Platform performance analysis
"Get iOS and Android test results for the last month"
# Quality metrics
"Get all test cases by automation state for project MCP"๐ฏ Milestone & Release Planning
# Milestone tracking
"Get project milestones for MCP with completion status"
# Build-specific results
"Get launches for build 'mcp-app-2.1.0-release' and milestone '2.1.0'"
# Release readiness
"Get automation readiness for all test cases in project MCP"๐ Issue Analysis & Troubleshooting
# Bug analysis
"Show me top 10 most frequent bugs with issue links"
# Failure investigation
"Get test run 12345 details with all test cases"
# Platform-specific issues
"Get Android test results for last 7 days with failure analysis"๐ญ Role-Specific Prompts & Workflows
๐ฉโ๐ป Manual QA Engineers
Daily Test Case Review
"Get test case MCP-45 details and validate its quality"
"Show me all test cases in suite 18708 that need improvement"
"Validate test case MCP-67 and suggest specific improvements"
"Find test cases with title containing 'login' to review authentication tests"
"Get test cases from suite 491 with high priority for today's testing"Test Case Creation & Improvement
"I'm writing a test case for login functionality. What should I include based on our quality standards?"
"Improve test case MCP-89 - it's missing some preconditions"
"Check if test case MCP-12 is ready for manual execution"Test Suite Organization
"Show me the hierarchy of test suites for project MYAPP to understand the structure"
"Get all subsuites from Authentication suite to review test coverage"
"List test cases in suite 18708 and identify which ones need validation"
"Find test cases with title containing 'payment' to organize payment testing"
"Get all high priority test cases from suite 491 for release testing"Coverage Analysis
"I executed test case MCP-34 manually. Here's what I did: [paste your execution notes]. Analyze coverage against the documented steps."
"Compare test case MCP-56 with this manual testing session: [paste session details]"๐ค Test Automation Engineers & SDETs
Automation Readiness Assessment
"Validate test case MCP-78 for automation readiness"
"Get all test cases in suite 18708 and identify which ones are ready for automation"
"Check test case MCP-23 - does it have clear, unambiguous steps for automation?"
"Find test cases with title containing 'API' to prioritize API automation"
"Get automation priorities to understand which test cases to automate first"
"Get test cases from suite 491 with 'Not Automated' state for automation planning"Test Code Generation
"Generate Java/Carina test code for MCP-45 based on this existing framework: [paste framework code]"
"Create JavaScript/Jest test for MCP-67 using this test structure: [paste test example]"
"Generate Python/Pytest code for MCP-89 with these page objects: [paste page object code]"Coverage Analysis & Validation
"Analyze test coverage for MCP-34 against this automated test: [paste test code]"
"Enhanced coverage analysis for MCP-56 with rules validation - here's my implementation: [paste code]"
"Compare test case MCP-78 steps with this Selenium test: [paste selenium code]"Framework Integration
"Generate test code for MCP-45 using our Carina framework with these page objects: [paste existing code]"
"Create test automation for MCP-67 that integrates with this CI/CD pipeline: [paste pipeline config]"
"Generate API test for MCP-89 using this RestAssured setup: [paste API test framework]"Batch Automation Analysis
"Validate all test cases in Authentication suite for automation readiness"
"Generate coverage report for all test cases in project MYAPP"
"Identify test cases in suite 18708 that have automation blockers"
"Find test cases with title containing 'regression' for automation sprint planning"
"Get test cases from suite 491 created after 2025-01-01 with high priority for next automation cycle"
"Get automation priorities and states to create automation roadmap"๐จโ๐ป Developers
Test Case Understanding
"Get test case MCP-45 details to understand what I need to implement"
"Show me test cases related to login functionality in project MYAPP"
"Explain test case MCP-67 requirements in developer-friendly format"
"Find test cases with title containing 'authentication' for my feature development"
"Get high priority test cases from suite 491 that I need to implement"Implementation Validation
"I implemented this feature: [paste code]. Analyze coverage against test case MCP-34"
"Here's my API implementation: [paste code]. Check coverage against test case MCP-56"
"Validate my UI implementation against test case MCP-78: [paste component code]"Code Generation for Testing
"Generate unit tests for test case MCP-45 using Jest framework"
"Create integration tests for MCP-67 based on this API: [paste API code]"
"Generate test data setup for MCP-89 using this database schema: [paste schema]"Bug Analysis
"Get test execution results for launch 118685 to understand recent failures"
"Show me top bugs from last week related to my feature area"
"Get detailed bug review for Android project from last 14 days"
"Show me comprehensive failure information for hashcode 1051677506"
"What are the top 50 bugs affecting our project this month?"
"Give me a summary of bug failures with reproduction dates from last 7 days"
"Analyze test case MCP-34 - why might it be failing in automation?"๐ Test Managers & Team Leads
Team Quality Metrics
"Get quality metrics for all test cases in project MYAPP"
"Show me test cases that need improvement in suite 18708"
"Generate quality report for test cases created this month"
"Find test cases with title containing 'critical' to assess critical path quality"
"Get automation priorities to align team efforts with business priorities"
"Get test cases from suite 491 with high priority that need quality improvements"Test Suite Analysis
"Show me the complete test suite hierarchy for project MYAPP"
"Analyze test coverage across all suites in project MYAPP"
"Get automation readiness status for all test cases in Authentication suite"Team Performance & Planning
"Get test execution results by platform for the last 30 days"
"Show me top 10 most frequent bugs to prioritize fixes"
"Analyze test case quality trends in project MYAPP"
"Get all launches for project MYAPP from last 30 days with milestone tracking"
"Show me platform results for last 7 days to track team performance"
"Get test runs with status 'FAILED' from last week for team retrospective"Process Improvement
"Validate all test cases in suite 18708 to identify common quality issues"
"Generate improvement recommendations for test cases created by junior team members"
"Analyze which test cases are consistently failing automation"
"Get top 10 bugs from last month to identify process improvements"
"Show me test runs with detailed failure analysis for process optimization"
"Get automation readiness metrics across all test cases"
"Find test cases with title containing 'flaky' to address test stability"
"Get test cases from suite 491 with medium priority that could be automated"
"Get automation priorities to optimize team resource allocation"Reporting & Stakeholder Communication
"Generate comprehensive test coverage report for project MYAPP in markdown format"
"Get test execution summary for launch 118685 for stakeholder presentation"
"Show me test quality metrics and improvement suggestions for quarterly review"
"Get platform results by period for executive dashboard"
"Create milestone progress report with test execution data"
"Generate weekly team performance report with launch and bug metrics"Daily Management Tasks
"Get all launches for project MYAPP from yesterday"
"Show me top 5 bugs from last 7 days with issue links"
"Get platform results for iOS and Android from last week"
"Check automation readiness for upcoming release milestone"
"Get test run details for failed runs from last 24 hours"
"Show me project milestones and their completion status"๐ข Project Owners & Product Managers
Project Health Overview
"Get overall test coverage status for project MYAPP"
"Show me test execution results by platform for the last quarter"
"Generate project testing health report in markdown format"
"Get all launches for project MYAPP with milestone and build tracking"
"Show me platform results summary for executive review"
"Get project milestones with completion status and testing metrics"Feature Testing Status
"Get test cases related to [feature name] in project MYAPP"
"Show me test execution results for [feature name] functionality"
"Analyze test coverage for [epic/story] requirements"
"Get launches filtered by milestone for feature release tracking"
"Show me test runs for specific build versions"Quality Assurance Metrics
"Get quality metrics for all test cases in project MYAPP"
"Show me test case validation results and improvement areas"
"Generate testing quality report for stakeholder presentation"
"Get top bugs analysis for quality trend assessment"
"Show me automation vs manual testing ratio across the project"
"Find test cases with title containing 'smoke' to assess smoke test coverage"
"Get automation priorities to communicate testing strategy to stakeholders"
"Get test cases from suite 491 with critical priority for risk assessment"Risk Assessment
"Show me top 10 most frequent bugs in project MYAPP"
"Get test cases that are not ready for automation and assess risk"
"Analyze test execution trends to identify potential quality risks"
"Get platform-specific failure rates for the last month"
"Show me test runs with high failure rates for risk mitigation"
"Get milestone-based testing progress for release risk assessment"Planning & Resource Allocation
"Get automation readiness assessment for all test cases in project MYAPP"
"Show me test cases that need quality improvement and estimate effort"
"Analyze test suite structure to identify optimization opportunities"
"Get testing resource utilization by platform and time period"
"Show me milestone testing progress for sprint planning"
"Get comprehensive launch analysis for capacity planning"
"Find test cases with title containing 'performance' to plan performance testing"
"Get automation priorities to allocate automation resources effectively"
"Get test cases from suite 491 created in last month to plan review sessions"Executive Reporting
"Generate executive dashboard with platform results and bug trends"
"Get quarterly testing metrics with milestone progress"
"Show me ROI analysis of automation vs manual testing efforts"
"Create board-ready testing status report with key metrics"
"Get testing velocity trends for project timeline assessment"๐ Output Formats
All tools support multiple output formats:
json- Structured data (default)markdown- Rich formatted output with sections and tablesstring- Human-readable text summariesdto- Raw data objects
Example:
"Get test case PROJ-123 in markdown format"
"Show me test suites as JSON"โ๏ธ Configuration Options
Environment Variables
# Required (STDIO mode) / Optional (HTTP selfauth mode โ see below)
ZEBRUNNER_URL=https://your-instance.zebrunner.com/api/public/v1
ZEBRUNNER_LOGIN=your.email@company.com
ZEBRUNNER_TOKEN=your_api_token
# Optional - HTTP Authentication Mode
MCP_AUTH_MODE=selfauth # Enable per-user credential form (Mode 3)
MCP_TRANSPORT=http # Use HTTP transport
PORT=3000 # HTTP server port
TOKEN_STORE_KEY=your-secret # Encryption key for stored credentials
TOKEN_STORE_PATH=./tokens.enc # Path for encrypted credential store
# Optional - Basic Settings
DEBUG=false # Enable detailed logging (default: false)
DEFAULT_PAGE_SIZE=100 # Default items per page (optional)
MAX_PAGE_SIZE=100 # Maximum items per page (optional)
# Optional - Intelligent Rules System
ENABLE_RULES_ENGINE=true # Enable intelligent rules (auto-detected if rules file exists)
MCP_RULES_FILE=custom-rules.md # Custom technical rules file (optional)
MIN_COVERAGE_THRESHOLD=70 # Minimum coverage percentage (optional)
REQUIRE_UI_VALIDATION=true # Require UI validation in tests (optional)
REQUIRE_API_VALIDATION=true # Require API validation in tests (optional)Instance Configuration File (zebrunner-config.json)
The MCP server ships with a zebrunner-config.json in the project root that contains instance-specific settings. Customize this file to adapt the server to your Zebrunner workspace. If the file is missing or contains invalid values, built-in defaults are used automatically.
{
"projectAliases": {
"web": "WEB",
"android": "AND",
"ios": "IOS",
"api": "WEB"
},
"testConnectionProjectKey": "MCP",
"widgetTemplates": {
"RESULTS_BY_PLATFORM": 8,
"TOP_BUGS": 4,
"BUG_REVIEW": 9,
"FAILURE_INFO": 6,
"FAILURE_DETAILS": 10
},
"dashboardNames": {
"weeklyResults": "Weekly results",
"bugsReproRate": "Bugs repro rate"
},
"platformMap": {
"web": [],
"api": ["api"],
"android": [],
"ios": ["ios"]
},
"featureAreaKeywords": {
"quicklog": "Search & Quick Log",
"search": "Search & Quick Log",
"notification": "Notifications"
},
"localeTestRunRules": {
"enabled": true,
"projectKeys": ["AND", "IOS", "WEB"],
"enUsOnlyFeatureSuites": ["Plans", "Workout Routines", "Recipe Discovery"],
"suiteNameMatch": "includes"
},
"relaunchFailures": {
"excludeLaunchNamePatterns": ["Performance"],
"maxLaunchesPerPlatform": 50
}
}Key | Description |
| Maps short names ( |
| Project key used by the |
| Numeric IDs for SQL widget templates used by reporting tools. These IDs are tenant-specific โ check your Zebrunner instance if reports return empty data. |
| Dashboard display names used by widget SQL queries. Must match dashboard names in your Zebrunner workspace. |
| Maps platform aliases to widget SQL |
| Keyword-to-label mapping used by regression stability reports to bucket test names into feature areas. Customize for your application's feature structure. |
| Project-scoped Build Now rules โ see below. Used by |
| Project-scoped rerun rules โ see below. Used by |
Individual keys can be omitted โ only the keys you include will override the defaults.
Project-specific automation rules (localeTestRunRules & relaunchFailures)
These optional blocks configure launch mutation workflows per project. They do not affect read-only tools, TCM tools, or projects outside the configured scope.
localeTestRunRules โ adv_start_launch (Jenkins Build Now) when the effective locale is not en_US:
Sub-key | Description |
| Master switch. Set |
| Zebrunner project keys where rules apply (e.g. |
| TCM feature suite names that are English-only; auto-excluded via |
|
|
relaunchFailures โ /relaunch-regression-failures prompt and batch rerun guidance:
Sub-key | Description |
| Launch names to skip when discovering failures (case-insensitive substring match). Default: |
| Cap per platform in the prompt workflow (default |
/feature-scoped-launch does not use zebrunner-config.json for Jenkins suite_path โ the prompt resolves it from args, recent launches, or by asking the user.
Examples
CUSTOMER-style deployment (shipped defaults in repo zebrunner-config.json):
"localeTestRunRules": {
"enabled": true,
"projectKeys": ["MCP", "DEF", "WEB"],
"enUsOnlyFeatureSuites": ["Plans", "Workout Routines", "Recipe Discovery"],
"suiteNameMatch": "includes"
},
"relaunchFailures": {
"excludeLaunchNamePatterns": ["Performance"],
"maxLaunchesPerPlatform": 50
}Generic / non-CUSTOMER deployment (e.g. only project MCP for demos, no locale exclusions):
"projectAliases": { "demo": "MCP" },
"testConnectionProjectKey": "MCP",
"localeTestRunRules": { "enabled": false },
"relaunchFailures": {
"excludeLaunchNamePatterns": [],
"maxLaunchesPerPlatform": 50
}Minimal override via env (Docker/K8s without mounting the file):
ZEBRUNNER_CONFIG_JSON='{"localeTestRunRules":{"enabled":false},"relaunchFailures":{"excludeLaunchNamePatterns":["Benchmark"]}}'See also: docs/RESOURCES_AND_PROMPTS.md for how prompts and tools consume these settings.
Per-User Zebrunner URL (v8.1.0+)
When running in HTTP mode with MCP_AUTH_MODE=selfauth and without setting ZEBRUNNER_URL, each user provides their own Zebrunner instance URL on the login form. This enables multi-tenant hosting where a single MCP server serves users across different Zebrunner organizations.
Login form shows a Zebrunner URL field (e.g.,
https://mcp.zebrunner.com)Users can update their URL, credentials, or disconnect at
/settingsPer-user URLs are stored encrypted alongside credentials in the token store
When
ZEBRUNNER_URLIS set as an environment variable, the URL field is hidden and the env value is used globally (existing behavior, no change)
Rules System Configuration
The rules system automatically detects and uses rules files in your project root:
Automatic Detection
If you have a mcp-zebrunner-rules.md file in your project root, the rules engine will automatically enable itself.
Custom Rules Files
You can customize the three types of rules:
Test Case Review Rules (
test_case_review_rules.md)
# Custom Test Case Review Rules
## Rule 1: Title Quality
- Titles must be descriptive and specific
- Minimum length: 10 characters
- Should not contain vague terms like "test", "check"
## Rule 2: Test Steps
- Each step must have clear action and expected result
- Steps should be numbered and sequential
- Avoid combining multiple actions in one stepAnalysis Checkpoints (
test_case_analysis_checkpoints.md)
# Custom Analysis Checkpoints
## Independence Assessment
- [ ] Can this test case run independently?
- [ ] Are all preconditions explicitly stated?
- [ ] No dependencies on other test cases?
## Automation Readiness
- [ ] All steps are unambiguous?
- [ ] Technical feasibility confirmed?
- [ ] Stable selectors available?Technical Rules (
mcp-zebrunner-rules.md)
# Technical Configuration
## Coverage Thresholds
- Overall Coverage: 80%
- Critical Steps: 95%
- UI Validation Steps: 85%
## Framework Detection
**Java/TestNG**:
- Keywords: @Test, TestNG, WebDriver
- File patterns: *Test.java, *Tests.java๐งช Testing Your Setup
Run health checks
npm run test:healthTest API connection
npm run smokeRun full test suite
npm testRun specific test types
npm run test:unit # Fast unit tests
npm run test:integration # API integration tests
npm run test:e2e # End-to-end tests๐ Troubleshooting
Common Issues
"Authentication failed" or 401 errors
โ Check your
ZEBRUNNER_LOGINandZEBRUNNER_TOKENโ Verify your API token is still valid
โ Ensure your user has proper permissions in Zebrunner
"Project not found" or 404 errors
โ Check the project key spelling (e.g., "MYAPP", not "myapp")
โ Verify you have access to the project in Zebrunner
โ Some endpoints may not be available on all Zebrunner instances
"Connection timeout" errors
โ Check your
ZEBRUNNER_URLis correctโ Ensure your network can reach the Zebrunner instance
โ Try increasing timeout in configuration
MCP integration not working
โ Verify the path to
dist/server.jsis correctโ Check that the project built successfully (
npm run build)โ Ensure environment variables are set in MCP configuration
โ Look at Claude Desktop/Code logs for error messages
Rules engine not working
โ Check that
ENABLE_RULES_ENGINE=truein your.envfileโ Verify rules files exist and have meaningful content
โ Restart the MCP server after changing rules files
โ Check debug logs for rules parsing errors
Debug Mode
Enable detailed logging to troubleshoot issues:
DEBUG=trueThis will show:
API requests and responses
Error details and stack traces
Performance metrics
Feature availability
Rules parsing and validation details
Getting Help
Check the logs - Enable debug mode and look for error messages
Test your connection - Run
npm run test:healthVerify your configuration - Double-check your
.envfileCheck Zebrunner permissions - Ensure your user has proper access
Validate rules files - Ensure rules files have meaningful content
Report issues - If you find a bug, please open an issue at https://github.com/maksimsarychau/mcp-zebrunner/issues
๐ฏ Example Workflows
Workflow 1: Test Case Review (Manual QA)
1. "Get test case PROJ-123 details"
2. "Validate test case PROJ-123"
3. "Improve test case PROJ-123 with specific suggestions"
4. "Check if test case PROJ-123 is ready for manual execution"Workflow 2: Test Automation (SDET)
1. "Validate test case PROJ-456 for automation readiness"
2. "Generate Java/Carina test code for PROJ-456"
3. "Analyze coverage between test case and my implementation"
4. "Get automation readiness assessment"Workflow 3: Implementation Validation (Developer)
1. "Get test case PROJ-789 details to understand requirements"
2. "Analyze coverage for PROJ-789 against my implementation"
3. "Generate unit tests based on test case requirements"
4. "Validate implementation completeness"Workflow 4: Quality Management (Team Lead)
1. "Get quality metrics for all test cases in project MYAPP"
2. "Show me test cases that need improvement"
3. "Generate team quality report"
4. "Identify automation readiness across the project"Workflow 5: Project Health (Product Manager)
1. "Get overall test coverage status for project MYAPP"
2. "Show me test execution results by platform"
3. "Generate project testing health report"
4. "Identify quality risks and improvement opportunities"๐ง Advanced Features
Batch Operations
Process multiple test cases at once:
"Validate all test cases in suite 18708"
"Generate coverage report for all test cases in project MYAPP"
"Improve all test cases that have quality issues"Custom Output Formats
Get data in the format you need:
"Get test cases as JSON for API integration"
"Show test suite hierarchy in markdown for documentation"
"Generate quality report in markdown for stakeholder presentation"Filtering and Search
Find exactly what you need:
"Get test cases created after 2025-01-01"
"Find test cases with automation state 'Manual'"
"Show me test cases that are not ready for automation"Rules-Based Analysis
Leverage intelligent validation:
"Validate test case PROJ-123 using custom rules from my-project-rules.md"
"Enhanced coverage analysis with framework-specific rules"
"Generate improvement suggestions based on team quality standards"๐ Additional Documentation
๐ Tool References
TOOLS_CATALOG.md - Complete catalog of all 69 tools with natural language examples
docs/TEST_PROMPTS.md - Test prompts per tool; ยง18 covers all 22 dashboard widgets (MCP + API verify)
docs/archive/TCM_TAM_WIDGET_BACKLOG.md - Archived widget template โ MCP matrix (live: TEST_PROMPTS ยง18)
docs/RESOURCES_AND_PROMPTS.md - MCP Resources & Prompts โ full usage guide, reference tables, and contributor guide
INSTALL-GUIDE.md - ๐ฅ Step-by-step installation and setup guide
๐ง Intelligent Rules System
docs/INTELLIGENT_RULES_SYSTEM.md - ๐ง Complete guide to the 3-tier intelligent rules system
docs/archive/RULES_QUICK_REFERENCE.md - โก Quick reference for rules system commands and configuration
๐ Rules Files (Customizable)
test_case_review_rules.md - ๐ฏ Core quality standards and writing guidelines
test_case_analysis_checkpoints.md - โ 100+ detailed validation checkpoints
mcp-zebrunner-rules.md - โ๏ธ Technical configuration for test generation and coverage analysis
๐ Specialized Guides
docs/archive/SCREENSHOT_ANALYSIS.md - ๐ธ Screenshot download and visual analysis guide
change-logs.md - ๐ Version history and feature updates
GitHub Release v9.1.0 - ๐ v9.1.0 release notes โ launch mutations, prompts, and config
๐ ๏ธ Feature Documentation
docs/TERMINOLOGY.md - ๐ Test vs Test Case vs Test Run vs Launch โ glossary and counting rules
docs/archive/NEW_LAUNCHER_TOOL.md - Detailed information about launch and reporting tools
docs/archive/SUITE_HIERARCHY.md - Complete guide to suite hierarchy features
docs/archive/TEST_CASE_VALIDATION_IMPLEMENTATION.md - Test case validation system details
docs/archive/ENHANCED_VALIDATION_FEATURES.md - Advanced validation and improvement features
๐ค Contributing
Fork the repository
Create a feature branch
Make your changes with appropriate tests
Ensure all tests pass:
npm testSubmit a pull request
๐ License
License
This project is licensed under AGPL-3.0 to ensure that all improvements to the MCP server remain open-source, especially when the software is deployed as a network service.
If you modify and run the server in a way that users interact with it over a network (e.g., Claude Desktop / Cursor clients), you must make the full modified source code available to those users.
Commercial usage is allowed. For closed-source modifications or enterprise licensing, please contact the author. See LICENSE.md file for details.
๐ You're Ready!
Once you've completed the setup:
Test your connection with
npm run test:healthConfigure your AI assistant with the MCP server
Start asking questions about your test cases!
Example first commands to try:
"List test suites for project [YOUR_PROJECT_KEY]"
"Get test case [YOUR_TEST_CASE_KEY] details"
"Validate test case [YOUR_TEST_CASE_KEY]"
"Show me the test suite hierarchy"
The intelligent rules system will help ensure your test cases meet quality standards and are ready for both manual execution and automation. Happy testing! ๐
Available Tools
49 toolsaggregate_test_cases_by_featureA
๐ Find ALL test cases related to a specific feature across the project. Searches in title, description, preconditions, and test steps (case-insensitive, partial match). Groups results by Root Suite and Feature Suite, avoiding duplicates. Output formats: detailed (full hierarchy), short (summary), dto (JSON), test_run_rules (for automation tags)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'MCPAND', 'MCP') | |
| feature_keyword | Yes | Feature keyword to search for (case-insensitive, partial match) | |
| output_format | No | Output format: detailed, short, dto, or test_run_rules | short |
| tags_format | No | TAGS output format: by_root_suite (separate TAGS line per root suite, default) or single_line (all combined on one line) | by_root_suite |
| max_results | No | Maximum test cases to process |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it searches across multiple text fields, uses case-insensitive partial matching, groups results by Root Suite and Feature Suite while avoiding duplicates, and offers multiple output formats. This covers search scope, matching logic, result organization, and output options, though it doesn't mention rate limits, permissions, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with four sentences that each add value: the core purpose, search scope and matching logic, result grouping method, and output formats. It uses emojis and formatting for readability without wasting space, and every sentence contributes essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (search across multiple fields with hierarchical grouping), no annotations, and no output schema, the description does well to cover purpose, behavior, and output options. It could be more complete by mentioning what the different output formats contain or any limitations, but it provides sufficient context for an agent to understand when and how to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds some context by mentioning 'case-insensitive, partial match' for the feature_keyword parameter and listing output formats, but doesn't provide additional semantic meaning beyond what's in the schema descriptions. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Find ALL test cases related to a specific feature'), the resource ('test cases'), and scope ('across the project'). It distinguishes from siblings by specifying it searches across multiple fields (title, description, preconditions, test steps) and groups results hierarchically, unlike tools like 'get_test_case_by_title' or 'get_test_cases_by_suite_smart' which have narrower scopes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: when searching for test cases related to a feature across the entire project with case-insensitive partial matching. It doesn't explicitly state when not to use it or name specific alternatives, but the context is sufficient to differentiate from siblings that filter by suite, key, or other criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyze_screenshotB
๐ Analyze test screenshot with OCR and visual analysis - returns image to Claude Vision for detailed analysis
| Name | Required | Description | Default |
|---|---|---|---|
| screenshotUrl | No | Screenshot URL to download and analyze | |
| screenshotPath | No | Local path to screenshot file | |
| testId | No | Test ID for context | |
| enableOCR | No | Enable OCR text extraction (slower) | |
| analysisType | No | basic=metadata+OCR only, detailed=includes image for Claude Vision | detailed |
| expectedState | No | Expected UI state for comparison |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions OCR and visual analysis, and that it 'returns image to Claude Vision for detailed analysis,' which gives some insight into processing behavior. However, it fails to disclose critical traits: whether this is a read-only or mutating operation, performance implications (e.g., 'slower' is noted in schema but not description), authentication needs, rate limits, or error handling. For a tool with 6 parameters and no annotations, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded: a single sentence efficiently conveys the core functionality (analyze screenshot with OCR/visual analysis) and key behavioral detail (returns to Claude Vision). Every word earns its place with no redundancy or fluff, making it easy to scan and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations), the description is moderately complete. It covers the purpose and high-level behavior but lacks details on when to use it vs. siblings, full behavioral traits, and output expectations. Without annotations or output schema, the description should do more to compensate, but it provides a basic foundation that's adequate for simple use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it implies the tool uses OCR and visual analysis, which relates to 'enableOCR' and 'analysisType' parameters, but doesn't provide additional syntax, format details, or usage examples. With high schema coverage, the baseline is 3, and the description doesn't significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Analyze test screenshot with OCR and visual analysis' specifies the verb (analyze) and resource (test screenshot) with the methods used (OCR and visual analysis). It distinguishes from sibling tools like 'download_test_screenshot' (which only downloads) and 'analyze_test_execution_video' (which analyzes videos). However, it doesn't explicitly mention what makes it unique from other analysis tools in the list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides minimal guidance: it mentions the tool returns 'image to Claude Vision for detailed analysis,' which implies when detailed visual analysis is needed. However, it lacks explicit when-to-use criteria, doesn't specify alternatives among siblings (e.g., when to use this vs. 'analyze_test_failure'), and offers no exclusions or prerequisites. This leaves usage context largely implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyze_test_cases_duplicatesB
๐ Analyze test cases for duplicates and group similar ones by step similarity (80-90%)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'ANDROID', 'IOS') | |
| suite_id | No | Optional: Analyze specific test suite ID | |
| test_case_keys | No | Optional: Analyze specific test case keys instead of suite | |
| similarity_threshold | No | Similarity threshold percentage (50-100, default: 80) | |
| format | No | Output format | markdown |
| include_similarity_matrix | No | Include detailed similarity matrix in output | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI (markdown format only) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but provides minimal behavioral context. It mentions grouping by step similarity with a threshold range (80-90%), but doesn't disclose what the analysis returns, whether it's read-only or modifies data, performance characteristics, or authentication requirements. The description adds some value but leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes the key constraint (similarity threshold range) in parentheses. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter analysis tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the analysis returns, how results are structured, whether it's a read operation, or any behavioral constraints. The description should do more to compensate for the lack of structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description mentions 'step similarity (80-90%)' which aligns with the 'similarity_threshold' parameter's default range, but doesn't add meaningful semantics beyond what the schema already provides. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('analyze'), resource ('test cases'), and specific purpose ('for duplicates and group similar ones by step similarity (80-90%)'). It distinguishes from siblings like 'analyze_test_cases_duplicates_semantic' by specifying step-based similarity rather than semantic analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for duplicate detection with step similarity, but doesn't explicitly state when to use this tool versus alternatives like 'analyze_test_cases_duplicates_semantic' or 'aggregate_test_cases_by_feature'. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyze_test_cases_duplicates_semanticC
๐ง Advanced semantic duplicate analysis using LLM-powered step clustering and two-phase analysis
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'ANDROID', 'IOS') | |
| suite_id | No | Optional: Analyze specific test suite ID | |
| test_case_keys | No | Optional: Analyze specific test case keys instead of suite | |
| similarity_threshold | No | Test case similarity threshold percentage (50-100, default: 80) | |
| step_clustering_threshold | No | Step clustering threshold percentage (50-100, default: 85) | |
| analysis_mode | No | Analysis mode: basic (fast), semantic (LLM-powered), hybrid (both) | hybrid |
| use_step_clustering | No | Enable two-phase clustering (step clusters first, then test case clusters) | |
| use_medoid_selection | No | Use medoid-based representative selection instead of heuristic | |
| include_semantic_insights | No | Generate semantic insights about workflows and patterns | |
| format | No | Output format | markdown |
| include_similarity_matrix | No | Include detailed similarity matrix in output | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI (markdown format only) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions 'advanced semantic duplicate analysis' and 'two-phase analysis,' it fails to describe critical behaviors such as computational intensity, potential rate limits, authentication requirements, or what the output looks like (e.g., clusters, insights). For a complex tool with 12 parameters, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads key information ('Advanced semantic duplicate analysis') and uses emojis and technical terms appropriately. Every word earns its place, making it highly concise and well-structured for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (12 parameters, no annotations, no output schema), the description is incomplete. It lacks details on behavioral traits, output format expectations, and usage context. While the schema covers parameters well, the description doesn't address the broader operational context needed for effective tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 12 parameters thoroughly. The description adds no additional meaning about parameters beyond what's in the schema (e.g., it doesn't explain how 'step_clustering_threshold' interacts with 'similarity_threshold' or clarify the 'hybrid' mode). With high schema coverage, the baseline is 3, and the description doesn't compensate with extra insights.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs 'semantic duplicate analysis' using 'LLM-powered step clustering and two-phase analysis,' which is a specific verb+resource combination. However, it doesn't explicitly differentiate from its sibling 'analyze_test_cases_duplicates' (which likely performs basic duplicate analysis), leaving some ambiguity about when to choose one over the other.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, compare with sibling tools like 'analyze_test_cases_duplicates,' or specify scenarios where this advanced analysis is preferred over basic methods. This leaves the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyze_test_execution_videoA
๐ฌ Download and analyze test execution video with Claude Vision - extracts frames, compares with test case, and predicts if failure is bug or test issue. NEW: Analysis depth modes (quick/standard/detailed), parallel frame extraction, similar failures search, and historical trends analysis!
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | Test ID from Zebrunner | |
| testRunId | Yes | Launch ID / Test Run ID | |
| projectKey | No | Project key (MCP, etc.) | |
| projectId | No | Project ID (alternative to projectKey) | |
| extractionMode | No | Frame extraction mode: failure_focused (10 frames), smart (20 frames), full_test (30 frames) | smart |
| frameInterval | No | Seconds between frames for full_test mode | |
| failureWindowSeconds | No | Time window around failure (seconds) | |
| compareWithTestCase | No | Compare with test case steps | |
| testCaseKey | No | Override test case key | |
| analysisDepth | No | Analysis depth: quick_text_only (no frames, ~10-20s), standard (8-12 frames for failure+coverage, ~30-60s), detailed (20-30 frames with OCR, ~60-120s) | standard |
| includeOCR | No | Extract text from frames using OCR (slow, adds 2-3s per frame) | |
| analyzeSimilarFailures | No | Find similar failures in project (last 30 days, top 10) | |
| includeHistoricalTrends | No | Analyze test stability and flakiness (last 30 runs) | |
| includeLogCorrelation | No | Correlate frames with log timestamps | |
| format | No | Output format | detailed |
| generateVideoReport | No | Generate timestamped report |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it mentions performance characteristics ('quick_text_only (no frames, ~10-20s)'), processing details ('parallel frame extraction'), and additional capabilities ('similar failures search, historical trends analysis'). However, it doesn't mention authentication requirements, rate limits, or error handling scenarios.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded with the core purpose in the first sentence. The feature list in the second sentence is somewhat dense but relevant. Every sentence earns its place by conveying important capabilities, though the exclamation point and 'NEW' tag could be considered slightly promotional rather than purely informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (16 parameters, no output schema, no annotations), the description does a good job covering the tool's scope and capabilities. It explains what the tool does, mentions analysis modes, and highlights key features. However, for such a complex tool, it could benefit from more guidance on output format or result interpretation since there's no output schema provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description doesn't add meaningful parameter semantics beyond what's already in the schema - it mentions analysis depth modes and parallel frame extraction but doesn't explain parameter interactions or provide usage examples. The schema already thoroughly documents all 16 parameters with descriptions and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Download and analyze test execution video with Claude Vision - extracts frames, compares with test case, and predicts if failure is bug or test issue.' It uses specific verbs (download, analyze, extracts, compares, predicts) and distinguishes from sibling tools like analyze_screenshot or analyze_test_failure by focusing specifically on video analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through its feature list ('NEW: Analysis depth modes...') but doesn't explicitly state when to use this tool versus alternatives like analyze_screenshot or analyze_test_failure. It suggests video analysis is appropriate but doesn't provide guidance on prerequisites or when other tools might be better suited.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyze_test_failureA
๐ Deep forensic analysis of failed test including logs, screenshots, error classification, and similar failures. ๐ก NEW: Compare with last passed execution to see what changed! ๐ก TIP: Can be auto-invoked from Zebrunner test URLs like: https://workspace.zebrunner.com/projects/PROJECT/automation-launches/LAUNCH_ID/tests/TEST_ID
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | Test ID (e.g., 5451420) | |
| testRunId | Yes | Test Run ID / Launch ID (e.g., 120806) | |
| projectKey | No | Project key (e.g., 'MCP') - alternative to projectId | |
| projectId | No | Project ID - alternative to projectKey | |
| includeScreenshots | No | Include screenshot links | |
| includeLogs | No | Include log analysis | |
| includeArtifacts | No | Include all test artifacts | |
| includePageSource | No | Include page source analysis | |
| includeVideo | No | Include video URL | |
| analyzeSimilarFailures | No | Find similar failures in the launch | |
| analyzeScreenshotsWithAI | No | Download and analyze screenshots with AI (Claude Vision) | |
| screenshotAnalysisType | No | Screenshot analysis type: basic (metadata+OCR) or detailed (includes Claude Vision) | detailed |
| format | No | Output format: detailed, summary, or jira (ready for Jira ticket creation) | detailed |
| compareWithLastPassed | No | Compare current failure with last passed execution to identify what changed |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the tool's capabilities (logs, screenshots, error classification, similar failures, comparison) and auto-invocation from URLs, which adds useful behavioral context. However, it doesn't disclose operational traits like rate limits, authentication needs, or potential side effects (e.g., whether it triggers downloads or external AI calls). The description is informative but lacks full transparency on such behavioral aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by new features and a usage tip. Each sentence adds value (e.g., highlighting new comparison capability and auto-invocation). It's efficient with no wasted words, though the emojis and formatting slightly reduce professionalism without harming clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (14 parameters, nested objects, no output schema) and no annotations, the description is moderately complete. It covers the purpose, key features, and a usage hint, but lacks details on output format, error handling, or prerequisites. For a forensic analysis tool with many parameters, more context on expected results or limitations would enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 14 parameters thoroughly. The description adds minimal parameter semantics beyond the schemaโit implies analysis includes logs, screenshots, and comparisons, which aligns with parameters like 'includeLogs' and 'compareWithLastPassed', but doesn't provide additional syntax or format details. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Deep forensic analysis of failed test including logs, screenshots, error classification, and similar failures.' It specifies the verb ('analyze') and resource ('failed test'), plus the scope of analysis. It distinguishes from siblings like 'analyze_screenshot' or 'detailed_analyze_launch_failures' by focusing on a single test's forensic details and comparison features.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use it: analyzing failed tests with forensic details and comparisons. It explicitly mentions auto-invocation from Zebrunner URLs, which is helpful. However, it doesn't specify when NOT to use it or name alternatives among siblings (e.g., 'detailed_analyze_launch_failures' for broader analysis), so it's not fully explicit about alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
detailed_analyze_launch_failuresA
๐ Analyze failed tests WITHOUT linked issues in a launch with grouping, statistics, and recommendations. Automatically analyzes all tests if โค10, otherwise first 10 (use offset/limit for more). Use filterType: 'all' to include tests with issues. Supports pagination and screenshot analysis. NEW: Jira format with smart grouping - creates combined tickets for similar errors! ๐ก TIP: Can be auto-invoked from Zebrunner launch URLs like: https://workspace.zebrunner.com/projects/PROJECT/automation-launches/LAUNCH_ID
| Name | Required | Description | Default |
|---|---|---|---|
| testRunId | Yes | Launch ID / Test Run ID (e.g., 120806) | |
| projectKey | No | Project key (e.g., 'MCP') - alternative to projectId | |
| projectId | No | Project ID - alternative to projectKey | |
| filterType | No | Filter: 'without_issues' = only tests without linked Jira tickets (DEFAULT), 'all' = all failed tests | without_issues |
| includeScreenshotAnalysis | No | Download and analyze screenshots with AI for each test (increases analysis time) | |
| screenshotAnalysisType | No | Screenshot analysis type if enabled | detailed |
| format | No | Output format: 'detailed' = full analysis, 'summary' = condensed, 'jira' = ready for Jira tickets with smart grouping | summary |
| jiraDetailLevel | No | Jira detail level: 'basic' = fast (no deep analysis), 'full' = comprehensive with deep analysis (DEFAULT, slower but thorough) | full |
| executionMode | No | Execution mode: sequential (safe), parallel (fast), or batches (balanced) | sequential |
| batchSize | No | Batch size if executionMode is 'batches' (default: 5) | |
| offset | No | Pagination offset - start from test N (e.g., 0 for first 20, 20 for next 20) | |
| limit | No | Number of tests to analyze (default: 20, max recommended: 30) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: automatic analysis limits (โค10 tests analyzed fully, otherwise first 10), pagination support, screenshot analysis impact on time, Jira smart grouping, and execution mode options affecting speed/safety. It could improve by clarifying output format details or error handling, but covers most critical aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, but contains some redundancy (e.g., mentioning grouping/statistics/recommendations multiple times) and includes promotional elements like emojis and 'NEW' tags that don't add functional clarity. The tip about auto-invocation is useful but could be more concise. Overall, it's adequately structured but could be tighter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 12 parameters and no output schema, the description provides substantial context about behavior, limitations, and practical usage. It covers analysis scope, pagination, format options, and performance considerations. The main gap is the lack of information about return values or output structure, which would be helpful given the absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds some value by explaining the practical implications of parameters (e.g., filterType usage, automatic analysis limits, Jira grouping), but doesn't provide significant additional semantics beyond what's already well-documented in the schema. It reinforces parameter purposes without contradicting them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: analyzing failed tests in a launch with grouping, statistics, and recommendations. It specifies the scope (tests without linked issues by default, with filter options) and distinguishes from siblings by mentioning unique capabilities like Jira format with smart grouping and screenshot analysis, which aren't present in other tools like 'analyze_test_failure' or 'get_launch_summary'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool: for analyzing failed tests in a launch, with explicit guidance on filterType options (e.g., 'without_issues' vs 'all'). It mentions auto-invocation from Zebrunner URLs as a tip. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the many sibling tools, though the unique features imply differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
download_test_screenshotC
๐ธ Download test screenshot with authentication from Zebrunner
| Name | Required | Description | Default |
|---|---|---|---|
| screenshotUrl | Yes | Screenshot URL (e.g., 'https://your-workspace.zebrunner.com/files/abc123' or '/files/abc123') | |
| testId | No | Test ID for context | |
| projectKey | No | Project key for context | |
| outputPath | No | Custom output path (default: temp directory) | |
| returnBase64 | No | Return base64 encoded image |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It mentions authentication but doesn't specify required permissions, rate limits, or what happens if the download fails. The behavioral context is minimal for a tool that likely involves network operations and file handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to understand at a glance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns (e.g., file path, base64 data) or error handling, leaving gaps in understanding its full behavior and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond implying authentication is needed, which doesn't clarify parameter usage. Baseline 3 is appropriate as the schema handles parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Download') and resource ('test screenshot') with the platform ('Zebrunner'), and mentions authentication. However, it doesn't explicitly differentiate from sibling tools like 'analyze_screenshot', which might be for analysis rather than downloading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'analyze_screenshot' or other test-related tools. The description lacks context about prerequisites or scenarios for downloading screenshots.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generate_draft_test_by_keyB
๐งช Generate draft test code from Zebrunner test case with intelligent framework detection
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | No | Project key (auto-detected from case_key if not provided) | |
| case_key | Yes | Test case key (e.g., 'ANDROID-6') | |
| implementation_context | Yes | Implementation context (existing code, file paths, or framework hints) | |
| target_framework | No | Target test framework (auto-detected if 'auto') | auto |
| output_format | No | Output format for generated test | code |
| include_setup_teardown | No | Include setup and teardown code | |
| include_assertions_templates | No | Include assertion templates | |
| generate_page_objects | No | Generate page object classes | |
| include_data_providers | No | Include data provider templates | |
| include_suite_hierarchy | No | Include featureSuiteId and rootSuiteId information | |
| file_path | No | File path for saving generated code (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only mentions 'intelligent framework detection' as a behavioral trait. It doesn't disclose whether this is a read-only operation, if it modifies data, what permissions are needed, rate limits, or what the output looks like. For a code generation tool with 11 parameters, this is insufficient behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. The emoji adds character but doesn't detract from clarity. Every word earns its place, though it could potentially be more specific about the generation scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex code generation tool with 11 parameters and no output schema, the description is inadequate. It doesn't explain what the generated output contains, how the framework detection works, what happens when file_path is provided, or the relationship between parameters. With no annotations and rich parameter schema, the description should provide more contextual guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema, but doesn't need to compensate for gaps. The baseline 3 is appropriate when the schema does all the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Generate draft test code') and resource ('from Zebrunner test case'), including the intelligent framework detection feature. It distinguishes itself from sibling tools like 'get_test_case_by_key' or 'improve_test_case' by focusing on code generation rather than retrieval or enhancement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through 'intelligent framework detection' and the tool name suggests it's for creating test drafts from existing cases. However, it lacks explicit guidance on when to use this vs. alternatives like 'improve_test_case' or 'get_test_case_by_key', and doesn't mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_launches_for_projectB
๐ Get all launches for a project with pagination (uses new reporting API)
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias (web/android/ios/api), project key, or project ID | |
| page | No | Page number (starts from 1) | |
| pageSize | No | Number of launches per page (max 100) | |
| format | No | Output format - 'raw' for full API response, 'formatted' for user-friendly display | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses behavioral traits like pagination support and API version ('new reporting API'), which is helpful. However, it doesn't mention potential rate limits, authentication requirements, error conditions, or what the output looks like (though there's no output schema). For a tool with 4 parameters and no annotations, this is a moderate but incomplete disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. The emoji (๐) adds visual cue but minimal informational value. No wasted words, though it could be slightly more structured (e.g., separating pagination note).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, 100% schema coverage, no annotations, and no output schema, the description provides basic context but lacks depth. It mentions pagination and API version, which helps, but doesn't cover output format details, error handling, or sibling tool differentiation. For a list/retrieval tool, this is minimally adequate but leaves gaps in full agent guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with all parameters well-documented in the schema itself. The description adds no additional parameter semantics beyond what's in the schema (e.g., it doesn't explain 'project' aliases or 'format' implications further). Baseline 3 is appropriate when the schema does the heavy lifting, though the description could have enhanced understanding of parameter interactions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get all launches') and resource ('for a project'), making the purpose unambiguous. It specifies 'with pagination' and 'uses new reporting API', which adds useful context. However, it doesn't explicitly differentiate from sibling tools like 'get_all_launches_with_filter' or 'get_launch_details', missing full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through 'with pagination' and 'uses new reporting API', suggesting this is for retrieving multiple launches in a paginated manner. However, it doesn't provide explicit guidance on when to use this tool versus alternatives like 'get_all_launches_with_filter' or 'get_launch_details', leaving the agent to infer based on tool names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_launches_with_filterC
๐ Get launches with filtering by milestone, build number, or launch name (uses new reporting API)
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias (web/android/ios/api), project key, or project ID | |
| milestone | No | Filter by milestone name (e.g., '25.39.0') | |
| query | No | Search query for build number or launch name (e.g., 'your-app-25.39.0-45915' or 'Performance') | |
| page | No | Page number (starts from 1) | |
| pageSize | No | Number of launches per page (max 100) | |
| format | No | Output format - 'raw' for full API response, 'formatted' for user-friendly display | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'uses new reporting API' which hints at implementation details but doesn't describe critical behaviors like pagination handling (implied by page/pageSize parameters but not explained), rate limits, authentication requirements, or what happens when no filters are applied. The description is insufficient for a tool with 6 parameters and no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the main purpose with an emoji for visual emphasis. Every element serves a purpose: the emoji indicates search/retrieval, the verb 'Get' is clear, and the parenthetical adds implementation context. However, the 'new reporting API' mention could be more specific about benefits.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain the relationship between parameters (e.g., how 'query' interacts with 'milestone'), doesn't describe the return format or structure, and provides no error handling or behavioral context. The mention of 'new reporting API' is vague and doesn't compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by mentioning filtering capabilities ('by milestone, build number, or launch name') which aligns with the 'milestone' and 'query' parameters, but doesn't provide additional context beyond what's in the schema descriptions. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get launches') and specifies filtering capabilities ('with filtering by milestone, build number, or launch name'), which distinguishes it from simple listing tools. However, it doesn't explicitly differentiate from sibling tools like 'get_all_launches_for_project' or 'get_launch_details' beyond mentioning the 'new reporting API'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_all_launches_for_project' or 'get_launch_details'. It mentions the 'new reporting API' but doesn't explain why this matters or when to prefer this tool over others. No exclusions or prerequisites are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_subsuitesB
๐ Get all subsuites from a root suite as flat list with pagination
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| root_suite_id | Yes | Root suite ID to get all subsuites from | |
| include_root | No | Include the root suite in results | |
| format | No | Output format | json |
| page | No | Page number (0-based) | |
| size | No | Page size (configurable via MAX_PAGE_SIZE env var) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions pagination but doesn't disclose rate limits, authentication needs, whether it's read-only or destructive, or what happens with invalid inputs. For a tool with 6 parameters and no annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. Every word earns its place: the emoji adds visual cue, 'Get all subsuites' states the action, 'from a root suite' specifies scope, and 'as flat list with pagination' clarifies output format and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read operation with 6 parameters and no output schema, the description is adequate but incomplete. It covers the basic purpose and output structure but lacks behavioral details (rate limits, errors) and sibling differentiation. With no annotations and no output schema, more context would be helpful for safe invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema itself. The description adds minimal value beyond implying the tool operates on a 'root suite' and produces a 'flat list', which aligns with the schema but doesn't provide additional semantic context beyond what's already in parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('all subsuites from a root suite'), and specifies the output structure ('as flat list with pagination'). However, it doesn't differentiate from sibling tools like 'get_suite_hierarchy' or 'get_root_suites', which likely provide hierarchical or root-only views respectively.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like 'get_suite_hierarchy' (likely hierarchical) and 'get_root_suites' (likely root-only), the description lacks explicit comparisons or context for choosing this flat-list approach over other suite retrieval methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_tcm_test_cases_by_projectB
๐ Get ALL TCM test cases by project using comprehensive pagination
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| format | No | Output format | json |
| include_clickable_links | No | Include clickable links to Zebrunner web UI | |
| max_results | No | Maximum number of results (configurable limit for performance) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only mentions 'comprehensive pagination', leaving gaps: it doesn't disclose rate limits, authentication needs, error handling, or whether this is a read-only operation. For a tool with 4 parameters and no annotations, this is insufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key information ('Get ALL TCM test cases by project'), and uses an emoji for visual clarity. Every word earns its place with zero waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, 100% schema coverage, no output schema, and no annotations, the description is minimally adequate. It covers the core purpose but lacks details on behavioral traits, usage context, and output expectations, leaving gaps for an AI agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description adds no additional parameter semantics beyond implying pagination (via 'comprehensive pagination'), which aligns with the 'max_results' parameter but doesn't provide extra syntax or format details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get ALL TCM test cases') and resource ('by project'), with 'comprehensive pagination' hinting at scope. It distinguishes from siblings like 'get_test_case_by_key' or 'get_test_cases_by_suite_smart' by emphasizing 'ALL' and project-level retrieval, though it doesn't explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'get_all_tcm_test_cases_with_root_suite_id' or 'get_test_cases_by_suite_smart'. The description implies it's for bulk retrieval by project, but lacks context on prerequisites, performance trade-offs, or specific use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_tcm_test_case_suites_by_projectC
๐ Get ALL TCM test case suites by project using comprehensive pagination
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| include_hierarchy | No | Include hierarchy information (rootSuiteId, parentSuiteName, etc.) | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'comprehensive pagination', which hints at handling large datasets, but fails to describe critical traits like whether this is a read-only operation, potential rate limits, authentication requirements, or what the output looks like (e.g., structure, size). This leaves significant gaps for a tool that likely returns extensive data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the key action ('Get ALL TCM test case suites by project') and adds a useful detail ('using comprehensive pagination'). There's no wasted verbiage, though it could be slightly more structured (e.g., separating purpose from behavioral notes).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity implied by 'ALL' and 'comprehensive pagination', and the absence of both annotations and an output schema, the description is incomplete. It doesn't address how results are returned (e.g., pagination details, response format), performance considerations, or error handling, which are crucial for an agent to use this tool effectively in a TCM context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all parameters thoroughly (project_key, include_hierarchy, format). The description adds no additional meaning beyond what the schema provides, such as explaining the impact of 'include_hierarchy' or when to use different 'format' options. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('ALL TCM test case suites by project'), making the purpose understandable. It distinguishes itself from sibling tools like 'get_tcm_test_suites_by_project' by emphasizing 'ALL' and 'comprehensive pagination', though it doesn't explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_tcm_test_suites_by_project' or 'get_all_subsuites'. It mentions 'comprehensive pagination' but doesn't explain when this is necessary or preferable, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_all_tcm_test_cases_with_root_suite_idC
๐ณ Get ALL TCM test cases enriched with root suite ID information
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions retrieving 'ALL' test cases but doesn't clarify scope limitations, pagination, performance implications, authentication requirements, or what 'enriched with root suite ID information' entails in practice. This leaves significant gaps for a tool that presumably handles large datasets.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It uses an emoji for visual distinction but maintains clarity. Every element ('Get ALL', 'TCM test cases', 'enriched with root suite ID information') serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that retrieves 'ALL' test cases with no output schema and no annotations, the description is insufficient. It doesn't address critical context like return format details (beyond the 'format' parameter), handling of large result sets, error conditions, or how the enrichment differs from basic test case data. This leaves too many unknowns for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema (e.g., it doesn't explain why 'project_key' is required or how 'format' affects the 'enriched' output). Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get ALL TCM test cases') and specifies the enrichment ('with root suite ID information'), which is a specific verb+resource combination. However, it doesn't explicitly distinguish this tool from sibling tools like 'get_all_tcm_test_cases_by_project' or 'get_test_cases_advanced', which likely have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple sibling tools for retrieving test cases (e.g., 'get_all_tcm_test_cases_by_project', 'get_test_cases_advanced'), there's no indication of what makes this tool unique or when it should be preferred over others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_automation_prioritiesB
๐ฏ Get available priorities for a project (names and IDs)
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias (web/android/ios/api), project key, or project ID | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the output includes 'names and IDs' but doesn't describe the return format beyond that, nor does it address authentication requirements, rate limits, error conditions, or whether this is a read-only operation. The description is minimal and leaves critical behavioral aspects unspecified for a tool with parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just one sentence with an emoji, front-loading the core purpose immediately. Every word earns its place, with no redundant information or fluff. The structure is optimal for quick comprehension while being complete enough for the stated purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally complete. It states what the tool does but lacks context about when to use it, what the output looks like beyond 'names and IDs', or how it fits into broader workflows. For a read operation with good schema coverage, this is adequate but leaves gaps in usage guidance and behavioral transparency.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with both parameters well-documented in the schema itself. The description adds no additional parameter semantics beyond what's already in the schema (e.g., it doesn't explain the relationship between project alias/key/ID or when to use each format). This meets the baseline of 3 for high schema coverage without adding value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'available priorities for a project', including the specific output 'names and IDs'. It distinguishes this tool from siblings by focusing on priorities rather than test cases, launches, or other project data. However, it doesn't explicitly differentiate from similar tools like 'get_automation_states' or 'get_project_milestones' that might also retrieve project metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing considerations, or relationships to other tools like 'get_available_projects' (which might need to be called first) or 'get_automation_states' (which might retrieve related metadata). The agent must infer usage from the tool name and description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_automation_statesB
๐ง Get available automation states for a project (names and IDs)
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias (web/android/ios/api), project key, or project ID | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the tool retrieves data but doesn't disclose behavioral traits like whether it's read-only, requires authentication, has rate limits, or what the output looks like (e.g., list format, pagination). This leaves significant gaps for a tool that likely interacts with project data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It front-loads the purpose with an emoji for visual clarity, making it easy to scan and understand quickly without unnecessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks context on usage, behavioral traits, or output details. With no annotations or output schema, more completeness would be beneficial, but it meets a basic threshold.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with clear descriptions for both parameters (project and format). The description adds no additional parameter semantics beyond what the schema provides, such as explaining the significance of automation states or output format implications. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('automation states for a project'), specifying that it retrieves names and IDs. It distinguishes from siblings like 'get_test_cases_by_automation_state' by focusing on available states rather than test cases, though it doesn't explicitly mention this distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing a valid project identifier, or compare it to siblings like 'get_available_projects' or 'get_test_cases_by_automation_state' for context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_available_projectsC
๐๏ธ Discover available projects with their keys and IDs for dynamic project selection
| Name | Required | Description | Default |
|---|---|---|---|
| starred | No | Filter by starred projects (true=only starred, false=only non-starred, undefined=all) | |
| publiclyAccessible | No | Filter by public accessibility (true=only public, false=only private, undefined=all) | |
| format | No | Output format: raw API response or formatted data | formatted |
| includePaginationInfo | No | Include pagination metadata from projects-limit endpoint |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'discovers' projects, implying a read-only operation, but doesn't mention critical behaviors like authentication requirements, rate limits, pagination handling (despite a parameter for pagination info), or error conditions. This is inadequate for a tool with multiple parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the core purpose and uses an emoji for visual clarity. Every part of the sentence contributes to understanding the tool's role, making it appropriately concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, no annotations, no output schema), the description is insufficient. It lacks details on behavioral traits, output format expectations (beyond 'keys and IDs'), and usage context. While the schema covers parameters well, the description doesn't compensate for missing annotations or output schema, leaving gaps in overall understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter-specific information beyond what the schema provides. Since schema description coverage is 100%, the baseline score is 3. The description's mention of 'dynamic project selection' loosely relates to filtering but doesn't elaborate on parameter usage or interactions, so it doesn't enhance the schema's documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Discover available projects with their keys and IDs for dynamic project selection.' It uses specific verbs ('discover') and identifies the resource ('projects') along with key output attributes ('keys and IDs'). However, it doesn't explicitly differentiate from sibling tools like 'get_project_milestones' or 'get_all_launches_for_project,' which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions 'dynamic project selection' but doesn't specify scenarios, prerequisites, or exclusions. With many sibling tools available (e.g., 'get_all_launches_for_project'), the lack of comparative context leaves the agent without clear usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_bug_failure_infoC
๐ฌ Get comprehensive failure information including failure summary and detailed test runs (SQL widgets, templateId: 6 & 10)
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api'), project key, or numeric projectId | web |
| dashboardId | Yes | Dashboard ID from bug review (e.g., 99) | |
| hashcode | Yes | Hashcode from bug review failure link (e.g., '1051677506') | |
| period | No | Time period for failure analysis | Last 14 Days |
| format | No | Output format: detailed (full info), summary (concise), or json (raw data) | detailed |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. While it mentions what information is retrieved (failure summary, detailed test runs), it doesn't disclose behavioral aspects like whether this is a read-only operation, potential performance characteristics, authentication requirements, error conditions, or what happens with invalid inputs. The mention of 'SQL widgets' and templateIds provides some implementation context but not enough operational transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. The emoji adds visual distinction but doesn't detract from clarity. While concise, it could potentially benefit from slightly more structure given the tool's complexity and lack of annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description provides adequate basic purpose but leaves significant gaps. It doesn't explain what the output looks like (crucial without an output schema), doesn't cover error handling, and provides no guidance on tool selection among many similar siblings. The mention of specific templateIds (6 & 10) adds some domain context but doesn't compensate for the broader completeness gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents all 5 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema - it doesn't explain relationships between parameters (e.g., how dashboardId and hashcode work together) or provide usage examples. The baseline of 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('comprehensive failure information') with specific components mentioned (failure summary, detailed test runs, SQL widgets, templateId references). It distinguishes from some siblings like 'analyze_test_failure' by emphasizing retrieval rather than analysis, but doesn't explicitly differentiate from all similar tools like 'get_launch_details' or 'get_test_execution_history'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools dealing with test failures, launches, and bug reviews (e.g., 'analyze_test_failure', 'get_bug_review', 'get_launch_details'), there's no indication of when this specific failure information retrieval tool is preferred over other options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_bug_reviewC
๐ Get detailed bug review with failures, defects, reproduction dates, and optional automatic failure detail fetching (SQL widget, templateId: 9)
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api'), project key, or numeric projectId | web |
| period | No | Time period for bug review | Last 7 Days |
| limit | No | Maximum number of bugs to return (default: 100, max: 500) | |
| include_failure_details | No | When true, automatically fetches detailed failure info for each bug (affected test runs, error details). Enables comprehensive single-call analysis. | |
| failure_detail_level | No | Level of failure details: none (just bug list), summary (error + count), full (all affected test runs) | summary |
| max_details_limit | No | Maximum bugs to fetch detailed failure info for (default: 30, max: 50). Prevents excessive API calls. | |
| templateId | No | Override templateId if needed (default: 9 for Bug Review) | |
| format | No | Output format: detailed (full info with markdown links), summary (concise), or json (raw data) | detailed |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions 'automatic failure detail fetching' and includes a templateId reference, it doesn't address critical behavioral aspects: whether this is a read-only operation, what permissions are required, rate limits, error handling, or what the output looks like. The SQL widget mention adds some context but doesn't fully compensate for the missing behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. The emoji adds visual distinction but doesn't detract from clarity. The parenthetical note about templateId is somewhat technical but relevant. There's minimal waste, though it could be slightly more structured for optimal readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no annotations, no output schema), the description is moderately complete. It covers the basic purpose and hints at some functionality but leaves significant gaps: no output description, no behavioral context, and no guidance on usage relative to siblings. The description alone is insufficient for an agent to fully understand how to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'failures, defects, reproduction dates' which loosely maps to output content rather than input parameters, and references 'automatic failure detail fetching' which corresponds to the 'include_failure_details' parameter. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get detailed bug review with failures, defects, reproduction dates, and optional automatic failure detail fetching'. It specifies the verb ('Get') and resource ('bug review') with specific content details. However, it doesn't explicitly differentiate from sibling tools like 'get_bug_failure_info' or 'get_top_bugs', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions 'optional automatic failure detail fetching' but doesn't explain when this should be enabled or how this tool compares to other bug-related tools in the sibling list. There's no mention of prerequisites, use cases, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_enhanced_test_coverage_with_rulesC
๐ Enhanced test coverage analysis with configurable rules validation and quality scoring
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | No | Project key (auto-detected from case_key if not provided) | |
| case_key | Yes | Test case key (e.g., 'ANDROID-6') | |
| implementation_context | Yes | Actual implementation details (code snippets, file paths, or implementation description) | |
| analysis_scope | No | Scope of analysis: steps, assertions, data coverage, or full analysis | full |
| output_format | No | Output format: chat response, markdown file, detailed analysis, or all formats | detailed |
| include_recommendations | No | Include improvement recommendations | |
| validate_against_rules | No | Validate coverage against configured rules | |
| show_framework_detection | No | Show detected framework and patterns | |
| include_suite_hierarchy | No | Include featureSuiteId and rootSuiteId in analysis | |
| file_path | No | File path for adding code comments or saving markdown (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. While it mentions 'analysis', 'validation', and 'scoring', it doesn't clarify whether this is a read-only operation, if it modifies data, what permissions are required, or what the output looks like (beyond format options). For a complex 10-parameter tool with no annotation coverage, this leaves significant behavioral gaps unaddressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. The emoji adds visual distinction without being distracting. Every word contributes to understanding the tool's enhanced nature. It could potentially benefit from a second sentence about output characteristics, but as-is it's appropriately concise for the complexity level.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 10-parameter analysis tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'enhanced' means relative to basic coverage tools, what 'quality scoring' entails, what rules are validated against, or what the analysis output contains. The user must infer these critical details from parameter names alone, which is inadequate for proper tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly with descriptions, enums, defaults, and requirements. The description adds no specific parameter information beyond what's in the schema - it doesn't explain relationships between parameters (e.g., how 'analysis_scope' affects 'validate_against_rules') or provide usage examples. The baseline 3 is appropriate when the schema does all the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs 'enhanced test coverage analysis with configurable rules validation and quality scoring', which is a specific verb+resource combination. It distinguishes itself from siblings like 'get_test_coverage_by_test_case_steps_by_key' by emphasizing 'enhanced' analysis with rules validation. However, it doesn't explicitly differentiate from other analysis tools like 'analyze_test_failure' or 'detailed_analyze_launch_failures' beyond the coverage focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools focused on test analysis (e.g., 'analyze_test_failure', 'get_test_coverage_by_test_case_steps_by_key'), there's no indication of when this 'enhanced' analysis is preferred, what prerequisites exist, or when other tools might be more appropriate. The description assumes the user already knows the context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_launch_detailsB
๐ Get comprehensive launch details including test sessions (uses new reporting API with enhanced authentication)
| Name | Required | Description | Default |
|---|---|---|---|
| projectKey | No | Project key (e.g., 'android' or 'ANDROID') - alternative to projectId | |
| projectId | No | Project ID (e.g., 7) - alternative to projectKey | |
| launchId | Yes | Launch ID (e.g., 118685) | |
| includeLaunchDetails | No | Include detailed launch information | |
| includeTestSessions | No | Include test sessions data | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'enhanced authentication', which hints at potential auth requirements, but doesn't specify what those are (e.g., API keys, permissions). It also doesn't describe rate limits, error handling, pagination, or what 'comprehensive' entails beyond test sessions. For a read operation with 6 parameters and no annotations, this leaves significant gaps in understanding how the tool behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get comprehensive launch details including test sessions') and adds contextual notes ('uses new reporting API with enhanced authentication'). There's no wasted verbiage, and every part serves a purpose. It loses a point because the emoji (๐) is decorative and doesn't add functional clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and hints at authentication needs, but doesn't explain output structure, error cases, or how 'comprehensive' differs from sibling tools. For a tool with moderate complexity and no structured behavioral hints, it should do more to guide effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, meaning all parameters are documented in the schema itself. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain relationships between projectKey and projectId, or clarify what 'dto' format means). Since the schema does the heavy lifting, the baseline score of 3 is appropriateโthe description neither compensates for gaps nor adds extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get comprehensive launch details including test sessions'. It specifies the verb ('Get'), resource ('launch details'), and scope ('including test sessions'), which distinguishes it from simpler summary tools. However, it doesn't explicitly differentiate from sibling tools like 'get_launch_summary' or 'detailed_analyze_launch_failures', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage context by mentioning 'uses new reporting API with enhanced authentication', which implies this is a preferred or updated method. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_launch_summary' or 'get_all_launches_for_project', nor does it specify prerequisites or exclusions. The guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_launch_summaryB
๐ Get quick launch summary without detailed test sessions (uses new reporting API)
| Name | Required | Description | Default |
|---|---|---|---|
| projectKey | No | Project key (e.g., 'android' or 'ANDROID') - alternative to projectId | |
| projectId | No | Project ID (e.g., 7) - alternative to projectKey | |
| launchId | Yes | Launch ID (e.g., 118685) | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'uses new reporting API' which adds some context about implementation, but doesn't describe what 'quick launch summary' contains, whether it's read-only, what permissions are needed, rate limits, or response format. For a tool with no annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise - one sentence with an emoji that adds visual context. Every word earns its place: 'Get quick launch summary' states the core purpose, 'without detailed test sessions' provides important differentiation, and 'uses new reporting API' adds implementation context. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description provides adequate basic purpose but lacks important context about what the 'quick launch summary' actually contains, response format details, or behavioral constraints. For a tool with rich sibling context and no structured safety/behavior annotations, it should do more to compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description adds no additional parameter information beyond what's in the schema. According to guidelines, when schema coverage is high (>80%), the baseline is 3 even with no param info in description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get quick launch summary without detailed test sessions' with the specific verb 'Get' and resource 'launch summary'. It distinguishes from siblings like 'get_launch_details' and 'get_launch_test_summary' by emphasizing 'quick' and 'without detailed test sessions'. However, it doesn't explicitly name these alternatives for full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool ('quick launch summary without detailed test sessions') versus more detailed alternatives, but doesn't explicitly state when-not-to-use or name specific sibling tools. It mentions 'uses new reporting API' which provides some context, but lacks explicit guidance on prerequisites or comparisons.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_launch_test_summaryA
๐ Get lightweight launch test summary with statistics (auto-paginated, token-optimized)
| Name | Required | Description | Default |
|---|---|---|---|
| projectKey | No | Project key (e.g., 'MCP') - alternative to projectId | |
| projectId | No | Project ID (e.g., 7) - alternative to projectKey | |
| launchId | Yes | Launch ID (e.g., 119783) | |
| statusFilter | No | Filter by status (e.g., ['FAILED', 'SKIPPED']) | |
| minStability | No | Minimum stability percentage (0-100) | |
| maxStability | No | Maximum stability percentage (0-100) | |
| sortBy | No | Sort order (stability=most unstable first) | stability |
| limit | No | Limit number of tests returned (e.g., 10 for first 10 tests) | |
| summaryOnly | No | Return only statistics without full test list (most lightweight) | |
| includeLabels | No | Include labels array (increases token usage) | |
| includeTestCases | No | Include testCases array (increases token usage) | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and adds valuable behavioral context: it discloses 'auto-paginated' (handling pagination automatically) and 'token-optimized' (efficient for token usage), which aren't obvious from the schema. However, it doesn't mention potential rate limits, authentication needs, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, using emojis and parentheses effectively to convey key points ('lightweight', 'statistics', 'auto-paginated', 'token-optimized'). It's appropriately sized and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (12 parameters, no output schema, no annotations), the description is somewhat complete by adding behavioral context, but it lacks details on output format, error cases, or how it differs from siblings like 'get_launch_summary'. It's adequate but has clear gaps for a tool with many parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 12 parameters thoroughly. The description doesn't add any parameter-specific semantics beyond what's in the schema (e.g., it doesn't explain how 'lightweight' relates to parameters like 'summaryOnly'), meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('lightweight launch test summary with statistics'), making the purpose specific and understandable. It distinguishes from some siblings like 'get_launch_details' or 'get_launch_summary' by emphasizing 'lightweight' and 'statistics', though it doesn't explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through terms like 'lightweight' and 'token-optimized', suggesting it's for quick summaries rather than detailed analysis, but it doesn't provide explicit when-to-use guidance or name specific alternatives from the sibling list (e.g., 'get_launch_details' for more detailed info).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_platform_results_by_periodC
๐ Get test results by platform for a given period (SQL widget, templateId: 8)
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api'), project key, or numeric projectId | web |
| period | No | Time period | Last 7 Days |
| platform | No | Platform alias or explicit array for paramsConfig.PLATFORM | |
| browser | No | Optional BROWSER filter, e.g., ['chrome'] for web | |
| milestone | No | Optional MILESTONE filter, e.g., ['25.39.0'] for milestone filtering | |
| templateId | No | Override templateId if needed | |
| dashboardName | No | Override dashboard title | |
| format | No | Output format: raw widget response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'SQL widget' and 'templateId: 8' which gives some implementation context, but doesn't describe what type of data is returned, whether this is a read-only operation, performance characteristics, or any limitations. For a tool with 8 parameters and no annotations, this is inadequate behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise - a single sentence with an emoji for visual emphasis. Every element earns its place: the emoji signals reporting/analytics, the core function is stated clearly, and the SQL widget context provides implementation specificity without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what 'test results' actually means in terms of data structure, doesn't mention return format or typical response content, and provides minimal guidance on parameter usage despite the complex parameter set. The SQL widget reference helps but doesn't compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'platform' and 'period' in the main description and references 'templateId' in parentheses, but doesn't provide additional semantic context about how parameters interact or typical usage patterns.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get test results') and resource ('by platform for a given period'), with the emoji and SQL widget context adding specificity. However, it doesn't explicitly differentiate from sibling tools like 'get_all_launches_for_project' or 'get_launch_summary' that might also retrieve test data, leaving some ambiguity about its unique scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives among the many sibling tools. It mentions 'SQL widget, templateId: 8' which hints at a specific context, but doesn't explain when this is preferred over other test result retrieval tools or what makes it distinct in usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_project_milestonesC
๐ฏ Get available milestones for a project with pagination and filtering
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api'), project key, or numeric projectId | web |
| page | No | Page number for pagination (1-based) | |
| pageSize | No | Number of milestones per page (max 100) | |
| status | No | Filter by completion status: incomplete (default, excludes overdue), completed, overdue (incomplete but past due date), or all | incomplete |
| format | No | Output format: raw API response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, the description doesn't address authentication requirements, rate limits, error conditions, or what the response structure looks like. The mention of 'pagination and filtering' is helpful but insufficient for a tool with 5 parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get available milestones for a project') and adds two key features ('with pagination and filtering'). Every word earns its place with zero redundancy or unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description is insufficiently complete. While the schema documents parameters well, the description doesn't address what the tool returns, error handling, authentication needs, or how it differs from similar data retrieval tools in the sibling list. The agent would need to guess about the response format and appropriate usage contexts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with each parameter well-documented in the schema itself. The description adds minimal value beyond the schema - it mentions 'pagination and filtering' which aligns with the 'page', 'pageSize', and 'status' parameters, but doesn't provide additional context or usage examples. This meets the baseline expectation when schema coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('available milestones for a project'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'get_all_launches_for_project' or 'get_test_cases_by_filter', which follow similar patterns for different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'pagination and filtering' which implies some usage context, but provides no explicit guidance on when to use this tool versus alternatives. With many sibling tools for retrieving project-related data (e.g., 'get_all_launches_for_project', 'get_test_cases_by_filter'), there's no indication of when milestones specifically are needed versus other project data types.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_root_id_by_suite_idB
๐ Get root suite ID for a specific suite ID
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| suite_id | Yes | Suite ID to find root for | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the tool's function without disclosing behavioral traits like error handling, rate limits, authentication needs, or output format details beyond the 'format' parameter. This is inadequate for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with an emoji for emphasis. It's front-loaded with the core purpose and wastes no words, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is minimal but covers the basic purpose. However, it lacks details on behavior, error cases, or output structure, which are important for a tool that likely returns hierarchical data. It's adequate but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond what's in the schema, such as explaining relationships between 'suite_id' and the returned root ID. Baseline 3 is appropriate when schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get root suite ID') and target resource ('for a specific suite ID'), using an emoji for visual emphasis. It specifies the verb and resource but doesn't differentiate from sibling tools like 'get_root_suites' or 'get_suite_hierarchy', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or compare to sibling tools like 'get_root_suites' or 'get_suite_hierarchy', leaving the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_root_suitesB
๐ณ Get root suites (suites with no parent) from project
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation ('Get') but doesn't specify permissions, rate limits, pagination, or error handling. The description adds minimal context beyond the basic action, leaving gaps in understanding how the tool behaves in practice.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the core purpose ('Get root suites') and includes a relevant emoji for visual clarity. Every element earns its place, making it easy to scan and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only tool with 2 parameters and 100% schema coverage but no output schema, the description is minimally adequate. It states what the tool does but lacks details on return values, error cases, or integration with sibling tools. Without annotations or output schema, more behavioral context would improve completeness, but it meets the basic threshold.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents both parameters (project_key and format). The description doesn't add any meaning beyond what the schema provides, such as explaining the significance of 'root suites' in relation to parameters. Baseline 3 is appropriate as the schema handles parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('root suites from project'), specifying that root suites are those with no parent. It distinguishes from siblings like 'get_all_subsuites' or 'get_suite_hierarchy' by focusing on root-level suites only. However, it doesn't explicitly contrast with these siblings in the text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_all_subsuites', 'get_suite_hierarchy', or 'list_test_suites'. It lacks context about prerequisites, such as needing a valid project key, and doesn't mention any exclusions or specific scenarios where this tool is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suite_hierarchyB
๐ณ Get hierarchical test suite tree with configurable depth
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| root_suite_id | No | Start from specific root suite | |
| max_depth | No | Maximum tree depth | |
| format | No | Output format | json |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'configurable depth' which hints at a behavioral trait, but doesn't disclose important aspects like whether this is a read-only operation, potential performance impacts with deep trees, authentication needs, rate limits, or what the hierarchical output looks like. The description is minimal and leaves key behavioral questions unanswered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise - a single sentence with an emoji that reinforces the tree concept. Every word earns its place: 'Get' (action), 'hierarchical test suite tree' (resource and structure), 'with configurable depth' (key capability). No wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the hierarchical output looks like, how to interpret the tree structure, performance considerations with depth, or relationships with sibling tools. The 100% schema coverage helps, but the description should provide more context about the tool's behavior and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain relationships between parameters (e.g., how root_suite_id interacts with project_key) or provide usage examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get hierarchical test suite tree') and the resource ('test suite'), with the emoji reinforcing the tree concept. It distinguishes from siblings by specifying 'hierarchical' and 'tree', unlike other tools like 'get_all_subsuites' or 'list_test_suites' that might return flat lists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving hierarchical structures with depth control, but doesn't explicitly state when to use this versus alternatives like 'get_all_subsuites' or 'get_root_suites'. It provides some context through 'configurable depth' but lacks explicit when/when-not guidance or named alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_tcm_suite_by_idC
๐ Find TCM suite by ID with comprehensive search
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| suite_id | Yes | Suite ID to find | |
| only_root_suites | No | Search only in root suites | |
| format | No | Output format | json |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'comprehensive search' but doesn't disclose behavioral traits like whether this is a read-only operation, if it requires authentication, rate limits, error handling, or what 'comprehensive' entails (e.g., depth of data returned). This leaves significant gaps for a tool with 5 parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and uses an emoji for visual emphasis without adding fluff. Every word earns its place, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format, what 'comprehensive search' includes, or behavioral aspects like safety or performance. For a tool with this complexity, more context is needed to guide effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds no additional meaning beyond implying ID-based search, which is already clear from the schema. Baseline 3 is appropriate as the schema does the heavy lifting, but the description doesn't compensate with extra context like parameter interactions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear purpose: 'Find TCM suite by ID with comprehensive search' specifies the verb ('Find'), resource ('TCM suite'), and method ('by ID'). It distinguishes from siblings like 'get_all_tcm_test_case_suites_by_project' (which lists all suites) by focusing on ID-based retrieval, but doesn't explicitly name alternatives or contrast scope beyond the 'comprehensive search' hint.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives is provided. The description mentions 'comprehensive search' but doesn't clarify what that entails or differentiate from siblings like 'get_suite_hierarchy' or 'get_root_suites'. Usage context is implied (ID-based lookup) but lacks exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_tcm_test_suites_by_projectC
๐ Get TCM test suites by project with pagination (Java methodology)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| max_page_size | No | Maximum page size for pagination | |
| page_token | No | Page token for pagination | |
| format | No | Output format | json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only mentions pagination and Java methodology. It doesn't disclose whether this is a read-only operation, potential rate limits, authentication requirements, error conditions, or what the output looks like (e.g., structure of returned test suites).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise single sentence with zero waste. Every element (verb, resource, pagination note, methodology) serves a purpose. The emoji adds visual distinction without compromising clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what 'TCM test suites' are, what the Java methodology entails, what the output contains, or how pagination works in practice. The agent would struggle to use this effectively without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema, maintaining the baseline score of 3 for adequate coverage through structured data alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'TCM test suites by project', specifying pagination and Java methodology. It distinguishes from some siblings like 'get_all_tcm_test_cases_by_project' by focusing on test suites rather than test cases, though it doesn't explicitly contrast with 'get_root_suites' or 'list_test_suites'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_root_suites', 'list_test_suites', or 'get_all_tcm_test_case_suites_by_project'. The description mentions pagination but doesn't explain when this is necessary or preferable over non-paginated options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_case_by_filterC
๐ Get test cases using advanced filtering options with exact matching
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| test_suite_id | No | Filter by exact test suite ID | |
| created_after | No | Filter test cases created after this date (ISO format: '2025-01-01T00:00:00Z') | |
| created_before | No | Filter test cases created before this date (ISO format: '2025-12-31T23:59:59Z') | |
| last_modified_after | No | Filter test cases last modified after this date (ISO format: '2025-01-01T00:00:00Z') | |
| last_modified_before | No | Filter test cases last modified before this date (ISO format: '2025-12-31T23:59:59Z') | |
| priority_id | No | Filter by priority ID (use get_automation_priorities to see available priorities) | |
| automation_state_id | No | Filter by automation state ID (use get_automation_states to see available states) | |
| max_page_size | No | Maximum number of results per page | |
| page_token | No | Token for pagination (from previous response _meta.nextPageToken) | |
| get_all | No | Get all matching test cases across all pages | |
| format | No | Output format | json |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'advanced filtering options with exact matching', which hints at query behavior, but fails to cover critical aspects like pagination handling (implied by parameters but not described), rate limits, authentication needs, or whether it's a read-only operation. This leaves significant gaps for an agent to understand how to use it effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with an emoji for visual emphasis. It's front-loaded with the core action and includes no redundant information, making it highly concise and well-structured for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (13 parameters, no annotations, no output schema), the description is insufficient. It doesn't explain the tool's behavior, output format (beyond the 'format' parameter), error handling, or how results are structured (e.g., pagination details). For a tool with many parameters and no structured guidance, more descriptive context is needed to ensure proper usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, providing detailed parameter documentation. The description adds minimal value beyond the schema by emphasizing 'advanced filtering options with exact matching', which loosely aligns with parameters like test_suite_id and date filters. However, it doesn't explain parameter interactions or provide additional context, so it meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('test cases') with the method ('using advanced filtering options with exact matching'), making the purpose evident. However, it doesn't explicitly distinguish this tool from similar siblings like 'get_test_cases_advanced' or 'get_test_cases_by_suite_smart', which likely offer overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as 'get_test_cases_advanced' or 'get_test_case_by_key'. It mentions 'advanced filtering options' but doesn't specify what makes it advanced or when it's preferred over simpler tools, leaving the agent without clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_case_by_keyB
๐ Get detailed test case by key (โ Verified Working)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | No | Project key (e.g., 'android' or 'ANDROID') - auto-detected from case_key if not provided | |
| case_key | Yes | Test case key (e.g., 'ANDROID-29', 'IOS-2') | |
| format | No | Output format | json |
| include_debug | No | Include debug information in markdown | |
| include_suite_hierarchy | No | Include featureSuiteId and rootSuiteId with suite hierarchy path | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool is 'Verified Working', which hints at reliability, but fails to describe critical behaviors: whether it's a read-only operation, what happens if the key doesn't exist, authentication requirements, rate limits, or response format details. The description adds minimal value beyond the basic purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise and front-loaded: the core purpose is stated in the first few words. The verification note is brief and doesn't distract. Every element earns its place with no wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read operation with 6 parameters and no output schema, the description is minimally adequate. It states what the tool does but lacks details about return values, error handling, or behavioral constraints. With no annotations and no output schema, more context would be helpful, but the high schema coverage partially compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds no additional parameter semantics beyond what's in the schema (e.g., no examples of case_key formats beyond what's implied, no explanation of when to use different formats). Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get detailed test case') and resource ('by key'), making the purpose immediately understandable. It distinguishes from siblings like 'get_test_case_by_title' by specifying the key-based lookup. The emoji and verification note add flavor but don't detract from clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'get_test_case_by_title', 'get_test_case_by_filter', or 'get_test_cases_advanced'. The description implies it's for retrieving a single test case by its identifier, but lacks context about prerequisites, error conditions, or comparison with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_case_by_titleC
๐ Get test cases by title using partial match search with pagination support
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| title | Yes | Title to search for (partial match) | |
| max_page_size | No | Maximum number of results per page | |
| page_token | No | Token for pagination (from previous response _meta.nextPageToken) | |
| get_all | No | Get all matching test cases across all pages | |
| format | No | Output format | json |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions 'partial match search' and 'pagination support', it doesn't describe important behavioral aspects like authentication requirements, rate limits, error conditions, what happens when no matches are found, or the structure of returned data. For a tool with 7 parameters and no annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise - a single sentence that efficiently communicates the core functionality. Every word earns its place: 'Get test cases by title' establishes the purpose, 'partial match search' clarifies the search behavior, and 'pagination support' indicates scalability features. No wasted words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns (test case objects, their structure, or error formats), doesn't mention authentication or rate limiting, and provides no context about the Zebrunner system it operates on. The description should do more to compensate for the lack of structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so all parameters are well-documented in the schema itself. The description adds minimal value beyond the schema - it mentions 'partial match search' (implied by the 'title' parameter description) and 'pagination support' (implied by 'page_token' and 'max_page_size'). This meets the baseline of 3 when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get test cases by title using partial match search with pagination support.' It specifies the verb ('Get'), resource ('test cases'), and key functionality ('partial match search', 'pagination support'). However, it doesn't explicitly differentiate from sibling tools like 'get_test_case_by_key' or 'get_test_case_by_filter', which would be needed for a score of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools for retrieving test cases (e.g., 'get_test_case_by_key', 'get_test_case_by_filter', 'get_test_cases_advanced'), there's no indication of when this partial-title-search approach is preferred or what distinguishes it from other search methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_cases_advancedA
๐ Advanced test case retrieval with filtering and pagination (โจ Enhanced with automation state and date filtering) โ ๏ธ IMPORTANT: Use 'suite_id' for direct parent suites, 'root_suite_id' for root suites that contain sub-suites. ๐ก TIP: Use 'get_test_cases_by_suite_smart' for automatic suite type detection!
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| suite_id | No | Filter by direct parent suite ID (for child suites) | |
| root_suite_id | No | Filter by root suite ID (includes all sub-suites) | |
| include_steps | No | Include detailed test steps | |
| automation_states | No | Filter by automation state(s). Can be: single name ('Not Automated'), single ID (10), array of names (['Not Automated', 'To Be Automated']), array of IDs ([10, 12]), or mixed array (['Not Automated', 12]) | |
| created_after | No | Filter test cases created after this date (ISO format: '2025-01-01' or '2025-01-01T10:00:00Z') | |
| created_before | No | Filter test cases created before this date (ISO format: '2025-12-31' or '2025-12-31T23:59:59Z') | |
| modified_after | No | Filter test cases modified after this date (ISO format: '2025-01-01' or '2025-01-01T10:00:00Z') | |
| modified_before | No | Filter test cases modified before this date (ISO format: '2025-12-31' or '2025-12-31T23:59:59Z') | |
| format | No | Output format | json |
| page | No | Page number (0-based) | |
| size | No | Page size (configurable via MAX_PAGE_SIZE env var) | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'advanced retrieval with filtering and pagination' which implies read-only behavior, but doesn't explicitly state whether this is a read operation, what permissions might be required, rate limits, or what the return format looks like. The description adds some context about suite filtering but lacks comprehensive behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with emoji icons and clear sections (purpose, important notes, tip). While slightly verbose with the emoji decorations, every sentence earns its place by providing distinct value. The information is front-loaded with the core purpose stated first.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 13 parameters and no output schema, the description provides adequate context about filtering capabilities and sibling alternatives. However, it doesn't describe the return format, pagination behavior, or error conditions. With no annotations and no output schema, more comprehensive behavioral context would be helpful for this advanced retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 13 parameters thoroughly. The description adds minimal parameter semantics beyond what's in the schema - it only clarifies the suite_id vs root_suite_id distinction. This meets the baseline expectation when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as 'Advanced test case retrieval with filtering and pagination' with specific enhancements mentioned (automation state and date filtering). It distinguishes itself from basic retrieval tools but doesn't explicitly differentiate from all sibling tools beyond the one mentioned alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool vs alternatives: it specifies to use 'suite_id' for direct parent suites and 'root_suite_id' for root suites, and explicitly recommends using 'get_test_cases_by_suite_smart' for automatic suite type detection. This gives clear when/when-not/alternative instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_cases_by_automation_stateA
๐ค Get test cases filtered by automation state (๐ก Use get_automation_states to see available states)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key | |
| automation_states | Yes | Automation state(s) to filter by. Examples: 'Not Automated', ['Not Automated', 'To Be Automated'], [10, 12], or 'Automated' | |
| suite_id | No | Optional: Filter by specific suite ID | |
| created_after | No | Optional: Filter test cases created after this date (ISO format: '2025-01-01') | |
| format | No | Output format | json |
| page | No | Page number (0-based) | |
| size | No | Page size | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only mentions filtering by automation state and references another tool for state options. It lacks critical behavioral details such as pagination behavior (implied by 'page' and 'size' params but not explained), rate limits, authentication needs, or what happens with invalid states. For a read operation with 8 parameters, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the core purpose and includes a helpful tip in parentheses. The emojis ('๐ค', '๐ก') add visual cues without compromising brevity. Every element serves a clear purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read operation with 8 parameters and no output schema, the description is minimally adequate. It covers the primary filtering logic and references a related tool, but lacks details on pagination, output structure, error handling, or performance considerations. With no annotations to fill gaps, it leaves the agent with incomplete operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 8 parameters. The description adds no parameter-specific information beyond implying filtering by automation state, which is already covered in the schema. It references 'get_automation_states' for state options, but this doesn't enhance parameter understanding beyond the schema's examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('test cases') with the specific filtering criterion ('by automation state'). It distinguishes itself from siblings like 'get_test_cases_advanced' or 'get_test_case_by_filter' by specifying the automation state filter, though it doesn't explicitly compare to them. The emojis add visual emphasis but don't detract from clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool by mentioning 'get_automation_states' to see available states, which helps the agent understand prerequisites. However, it doesn't specify when NOT to use it or compare it to alternative filtering tools like 'get_test_cases_advanced', leaving some ambiguity in tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_cases_by_suite_smartA
๐ง Smart test case retrieval by suite ID - automatically detects if suite is root suite and uses appropriate filtering with enhanced pagination
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'MCP') | |
| suite_id | Yes | Suite ID to get test cases from | |
| include_steps | No | Include detailed test steps for first few cases | |
| format | No | Output format | json |
| get_all | No | Get all test cases (true) or paginated results (false) | |
| include_sub_suites | No | Include test cases from sub-suites (if any) | |
| page | No | Page number (0-based, only used if get_all=false) | |
| size | No | Page size (only used if get_all=false) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some context: 'smart' behavior with automatic root suite detection and 'enhanced pagination' hints at adaptive filtering and pagination features. However, it lacks details on permissions, rate limits, error handling, or what 'enhanced pagination' entails (e.g., performance implications). The description doesn't contradict annotations, but it's insufficient for a mutation-like tool (implied retrieval).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads key information: smart retrieval, suite ID focus, automatic detection, and enhanced pagination. Every element earns its place with no redundant or vague phrasing, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, smart logic) and lack of annotations or output schema, the description is moderately complete. It covers the core purpose and hints at behavioral traits but lacks details on return values, error cases, or how the 'smart' detection works in practice. For a retrieval tool with many parameters, more context would be helpful, but it's minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds no parameter-specific semantics beyond implying that 'suite_id' triggers smart detection logic. It doesn't explain interactions between parameters (e.g., how 'get_all' affects pagination) or provide usage examples. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Smart test case retrieval by suite ID' specifies the verb (retrieval) and resource (test cases). It distinguishes itself by mentioning 'smart' behavior with automatic detection of root suites and enhanced pagination. However, it doesn't explicitly differentiate from sibling tools like 'get_test_cases_advanced' or 'get_all_tcm_test_cases_by_project' beyond the 'smart' aspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through 'automatically detects if suite is root suite and uses appropriate filtering,' suggesting this tool handles root vs. non-root suite scenarios intelligently. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_test_cases_advanced' or 'get_all_tcm_test_cases_with_root_suite_id,' nor does it provide exclusion criteria or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_coverage_by_test_case_steps_by_keyC
๐ Analyze test case coverage against actual implementation with recommendations
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | No | Project key (auto-detected from case_key if not provided) | |
| case_key | Yes | Test case key (e.g., 'ANDROID-6') | |
| implementation_context | Yes | Actual implementation details (code snippets, file paths, or implementation description) | |
| analysis_scope | No | Scope of analysis: steps, assertions, data coverage, or full analysis | full |
| output_format | No | Output format: chat response, markdown file, code comments, or all formats | chat |
| include_recommendations | No | Include improvement recommendations | |
| include_suite_hierarchy | No | Include featureSuiteId and rootSuiteId in analysis | |
| file_path | No | File path for adding code comments or saving markdown (optional) | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'recommendations' but lacks critical details: whether this is a read-only analysis or modifies data, what the output looks like (format/content), error handling, or performance characteristics. For a 9-parameter analysis tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. It uses an emoji for visual emphasis but maintains focus. No wasted words, though it could potentially benefit from slightly more structure for a complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex analysis tool with 9 parameters and no output schema, the description is inadequate. It doesn't explain what the analysis produces, how recommendations are structured, or the tool's behavioral characteristics. With no annotations and rich parameter schema, the description should provide more context about outputs and usage patterns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 9 parameters. The description doesn't add any parameter-specific context beyond what's in the schema (e.g., explaining relationships between parameters or usage patterns). Baseline 3 is appropriate when the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: analyzing test case coverage against implementation with recommendations. It uses specific verbs ('analyze') and identifies the resource ('test case coverage'), but doesn't explicitly differentiate from sibling tools like 'get_enhanced_test_coverage_with_rules' or 'get_test_case_by_key', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, appropriate contexts, or exclusions, leaving the agent to infer usage from the tool name and parameters alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_execution_historyB
๐ Get execution history for a test across multiple launches - shows pass/fail history, last passed execution, and pass rate
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | Test ID | |
| testRunId | Yes | Test Run ID / Launch ID containing the test | |
| projectKey | No | Project key (e.g., 'MCP') - alternative to projectId | |
| projectId | No | Project ID - alternative to projectKey | |
| limit | No | Number of history items to return (default: 10, max: 50) | |
| format | No | Output format: dto (structured), json, or string (markdown table) | string |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states what data is returned (pass/fail history, last passed execution, pass rate). It lacks critical behavioral details: whether this is a read-only operation, authentication requirements, rate limits, pagination behavior, or error conditions. The description is insufficient for a tool with 6 parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose and key return data. Every element earns its place with no wasted words, though it could benefit from additional behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address authentication, error handling, rate limits, or the structure of returned data. While it mentions what data is included, it doesn't provide enough context for reliable agent usage given the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, providing complete parameter documentation. The description adds no specific parameter semantics beyond implying historical data retrieval across launches. It doesn't explain relationships between parameters (e.g., projectKey vs projectId alternatives) or clarify the 'format' parameter's practical implications.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('execution history for a test across multiple launches'), specifying the scope of data retrieval. It distinguishes from siblings by focusing on historical pass/fail data rather than current status or analysis, though it doesn't explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when historical test execution data is needed, but provides no explicit guidance on when to choose this tool over similar siblings like 'get_launch_test_summary' or 'get_test_run_result_statuses'. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_run_by_idC
๐ Get detailed Test Run information by ID from Public API
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Test Run ID | |
| project | No | Project alias ('web', 'android', 'ios', 'api') or project key | web |
| format | No | Output format: raw API response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It states the tool retrieves 'detailed Test Run information' from a 'Public API', implying a read-only operation, but lacks critical details such as authentication requirements, rate limits, error handling, or the nature of the returned data (e.g., structured vs. raw). This leaves significant gaps in understanding the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the core purpose in a single sentence, using an emoji for visual emphasis. There's no wasted text, but it could be slightly more structured by explicitly separating key details (e.g., 'Retrieves detailed Test Run data by ID from the Public API. Supports project aliases and output formatting.'), keeping it efficient but not perfectly optimized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what 'detailed information' includes, how the output varies by 'format', or potential side effects like API rate limits. For a tool interacting with an external API and returning unspecified data, more context is needed to ensure reliable use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds minimal parameter semantics beyond the input schema, which has 100% coverage. It implies the 'id' parameter is for a Test Run, but doesn't clarify the meaning of 'project' or 'format' choices. Since the schema already documents all parameters well, the baseline score of 3 is appropriate, as the description doesn't significantly enhance understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get detailed Test Run information') and resource ('by ID from Public API'), making the purpose specific and understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'get_launch_details' or 'get_test_case_by_key', which also retrieve specific entities by identifier, so it doesn't reach the highest score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid Test Run ID), exclusions, or comparisons to sibling tools like 'list_test_runs' for browsing or other 'get_*' tools for different entity types, leaving usage context unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_run_configuration_groupsC
Get list of Configuration Groups and their Options for a project. These are used to configure Test Runs.
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias ('web', 'android', 'ios', 'api') or project key | |
| format | No | Output format | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves a list but does not describe key behavioral traits such as pagination, rate limits, authentication requirements, error handling, or the structure of the returned data. The mention of 'format' parameter hints at output variations, but without details, this is insufficient for a mutation-free read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, consisting of two sentences that directly state the tool's purpose and context. There is no unnecessary verbiage, and each sentence contributes meaning. However, it could be slightly more structured by explicitly separating purpose from usage context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a read operation with 2 parameters) and the absence of annotations and output schema, the description is minimally adequate. It covers the basic purpose and hints at usage context but lacks details on behavior, output format, or error handling. The high schema coverage helps, but the description does not fully compensate for the missing behavioral and output information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear documentation for both parameters ('project' and 'format'). The description adds no additional parameter semantics beyond what the schema provides, such as explaining the significance of 'project' values or the differences between 'raw' and 'formatted' outputs. Since schema coverage is high, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get list of Configuration Groups and their Options for a project.' It specifies the verb ('Get'), resource ('Configuration Groups and their Options'), and scope ('for a project'), which is specific and actionable. However, it does not explicitly differentiate from sibling tools, as none appear to be direct alternatives for retrieving configuration groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides minimal usage guidance: it mentions that configuration groups 'are used to configure Test Runs,' implying a context for test configuration. However, it lacks explicit instructions on when to use this tool versus alternatives, prerequisites, or exclusions. No sibling tools are named as alternatives, and there is no clear differentiation from other tools in the list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_run_result_statusesB
Get list of Result Statuses configured for a project. These statuses are used when assigning results to Test Cases.
| Name | Required | Description | Default |
|---|---|---|---|
| project | Yes | Project alias ('web', 'android', 'ios', 'api') or project key | |
| format | No | Output format | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes a read operation ('Get list'), which implies it's non-destructive, but doesn't mention any behavioral traits such as authentication requirements, rate limits, error handling, or response format details. For a tool with no annotations, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, consisting of two sentences that directly state the tool's purpose and usage context without unnecessary details. It's front-loaded with the core action and resource, making it easy to understand quickly. Every sentence earns its place by adding value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and context but lacks details on behavioral aspects, output expectations, and usage guidelines. Without annotations or an output schema, the description should do more to compensate, but it falls short of being fully complete for effective tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear documentation for both parameters ('project' and 'format'). The description doesn't add any parameter-specific semantics beyond what the schema provides, such as explaining the difference between 'raw' and 'formatted' outputs or detailing project key formats. Given the high schema coverage, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get list of Result Statuses configured for a project.' It specifies the resource (Result Statuses) and the action (Get list), and mentions their usage context ('used when assigning results to Test Cases'). However, it doesn't explicitly differentiate from sibling tools that might also retrieve project-related data, such as 'get_project_milestones' or 'get_automation_states', which slightly reduces clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions that the statuses are 'used when assigning results to Test Cases,' which implies a context but doesn't specify prerequisites, exclusions, or direct comparisons to sibling tools like 'get_automation_states' or 'get_test_run_by_id.' This lack of explicit usage instructions limits its effectiveness for an AI agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_top_bugsC
๐ Top N most frequent defects with optional issue links (SQL widget, templateId: 4)
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api'), project key, or numeric projectId | web |
| period | No | Time period | Last 7 Days |
| limit | No | How many bugs to return | |
| templateId | No | Override templateId if needed | |
| issueUrlPattern | No | e.g., 'https://yourcompany.atlassian.net/browse/{key}' | |
| platform | No | Optional platform filter; defaults to [] for this widget | |
| milestone | No | Optional MILESTONE filter, e.g., ['25.39.0'] for milestone filtering | |
| format | No | Output format: raw widget response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'SQL widget' and 'templateId: 4' which gives some implementation context, but doesn't describe what the tool returns (format, structure), whether it's read-only or has side effects, authentication requirements, rate limits, or error behavior. The description is insufficient for a tool with 8 parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point. It uses emoji and parenthetical notes effectively without unnecessary verbosity. However, the 'SQL widget, templateId: 4' note feels somewhat cryptic and could be better integrated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns (critical since there's no output schema), doesn't provide usage context beyond a cryptic SQL widget reference, and offers minimal behavioral transparency. The description should do much more to compensate for the lack of structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema - it mentions 'optional issue links' which relates to the 'issueUrlPattern' parameter, and 'templateId: 4' which matches the default value in schema. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: retrieving 'Top N most frequent defects' with 'optional issue links'. It specifies the resource (defects/bugs) and action (get top N), but doesn't explicitly differentiate from sibling tools like 'get_bug_failure_info' or 'get_bug_review' that might handle bugs differently.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions 'SQL widget, templateId: 4' which hints at a specific implementation context, but doesn't explain when this tool is appropriate compared to other bug-related tools in the sibling list or what prerequisites might exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
improve_test_caseC
๐ง Analyze and improve a test case with detailed suggestions and optional automatic fixes
| Name | Required | Description | Default |
|---|---|---|---|
| projectKey | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| caseKey | Yes | Test case key (e.g., 'ANDROID-29') | |
| rulesFilePath | No | Path to custom rules markdown file | |
| checkpointsFilePath | No | Path to custom checkpoints markdown file | |
| format | No | Output format | markdown |
| applyHighConfidenceChanges | No | Automatically apply high-confidence improvements | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only vaguely mentions 'optional automatic fixes' without detailing what changes are made, permissions required, or side effects. It doesn't specify if improvements are saved automatically, require review, or affect other test cases, leaving key behavioral traits unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. It uses an emoji for visual emphasis but avoids unnecessary elaboration, though it could be slightly more structured by separating analysis from fixes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters, no annotations, and no output schema, the description is minimally adequate. It covers the basic purpose but lacks details on behavior, output format implications, or integration with sibling tools, leaving gaps in understanding how to effectively use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds no additional meaning beyond what's in the schema, such as explaining how 'rulesFilePath' or 'checkpointsFilePath' influence analysis. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('analyze and improve') and resource ('test case'), with additional detail about providing 'detailed suggestions and optional automatic fixes'. It distinguishes from siblings like 'validate_test_case' or 'get_test_case_by_key' by focusing on enhancement rather than retrieval or validation, though it doesn't explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'validate_test_case' or 'get_enhanced_test_coverage_with_rules'. The description implies usage for test case improvement but lacks context on prerequisites, constraints, or specific scenarios where it's most appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_runsB
๐ List Test Runs from Public API with advanced filtering
| Name | Required | Description | Default |
|---|---|---|---|
| project | No | Project alias ('web', 'android', 'ios', 'api') or project key | web |
| pageToken | No | Token for pagination (from previous response) | |
| maxPageSize | No | Number of test runs per page (max 100) | |
| nameFilter | No | Filter by test run name (partial match) | |
| milestoneFilter | No | Filter by milestone ID (use get_project_milestones to find ID) or milestone name (will be converted to ID) | |
| buildNumberFilter | No | Filter by build number (searches in configurations, title, and description) | |
| closedFilter | No | Filter by closed status (true=closed, false=open) | |
| sortBy | No | Sort order: -createdAt (newest first), createdAt (oldest first), -title (Z-A), title (A-Z) | -createdAt |
| format | No | Output format: raw API response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only mentions 'advanced filtering' without detailing behavioral aspects. It doesn't disclose pagination behavior (implied by pageToken), rate limits, authentication requirements, response format expectations, or whether this is a read-only operation. The description adds minimal value beyond what's obvious from the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with an emoji, which is reasonably concise but front-loads style over substance. While it efficiently conveys the core purpose, it could be more structured by separating key capabilities. The emoji adds character but doesn't enhance functional understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what constitutes a 'test run', what data is returned, how pagination works, or any error conditions. The lack of behavioral context and output information leaves significant gaps for an agent to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the input schema thoroughly documents all 9 parameters. The description adds no specific parameter information beyond 'advanced filtering', which is already evident from the parameter names. The baseline score of 3 reflects adequate coverage through schema alone, with no additional value from the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('Test Runs'), and specifies the source ('from Public API') and capability ('with advanced filtering'). It distinguishes from siblings like 'get_test_run_by_id' by indicating it returns multiple items with filtering, but doesn't explicitly contrast with other list-like tools like 'get_all_launches_for_project'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing filtered test runs from the public API, but doesn't explicitly state when to choose this tool over alternatives like 'get_all_launches_for_project' or 'list_test_run_test_cases'. The parameter descriptions provide some context (e.g., referencing 'get_project_milestones' for milestone IDs), but no explicit guidance on tool selection is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_run_test_casesC
๐ List all Test Cases in a Test Run from Public API
| Name | Required | Description | Default |
|---|---|---|---|
| testRunId | Yes | Test Run ID | |
| project | No | Project alias ('web', 'android', 'ios', 'api') or project key | web |
| format | No | Output format: raw API response or formatted data | formatted |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'Public API' which hints at external access, but doesn't disclose critical behavioral traits like authentication requirements, rate limits, pagination behavior, error handling, or whether this is a read-only operation. For a tool with no annotation coverage, this leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point. The emoji adds visual distinction but doesn't detract from clarity. It's appropriately sized for a straightforward list operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read operation with good schema coverage but no annotations or output schema, the description is adequate but incomplete. It specifies the scope ('all Test Cases in a Test Run') but lacks context about authentication, pagination, error scenarios, or relationship to sibling tools. The absence of output schema means the description should ideally hint at return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema itself. The description adds no additional parameter semantics beyond what's already in the schema descriptions. This meets the baseline expectation when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List all Test Cases') and resource ('in a Test Run from Public API'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get_test_case_by_key' or 'get_test_cases_by_suite_smart', but the scope is well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools for retrieving test cases (e.g., 'get_test_case_by_key', 'get_test_cases_by_suite_smart'), there's no indication of when this list-all approach is preferred or what prerequisites might exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_suitesC
๐ List test suites for a project (โ Verified Working)
| Name | Required | Description | Default |
|---|---|---|---|
| project_key | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| project_id | No | Project ID (alternative to project_key) | |
| format | No | Output format | json |
| include_hierarchy | No | Include hierarchy information | |
| page | No | Page number (0-based) | |
| size | No | Page size (configurable via MAX_PAGE_SIZE env var) | |
| page_token | No | Page token for pagination | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only mentions that the tool is 'Verified Working' but fails to describe pagination behavior (implied by page/size parameters), authentication requirements, rate limits, or what happens when no test suites exist. For a listing tool with 8 parameters, this leaves significant gaps in understanding how it behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately brief with a single sentence that states the core purpose. The emoji and verification status add minimal clutter. However, the verification note ('โ Verified Working') doesn't provide actionable information for tool selection and could be considered slightly extraneous.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a listing tool with 8 parameters, pagination capabilities, and no output schema, the description is insufficient. It doesn't explain what format the listing returns, how pagination works with page_token, or what hierarchy information includes. With no annotations and no output schema, the description should provide more context about the tool's behavior and results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'project' which aligns with the required 'project_key' parameter, but adds no additional semantic context beyond what the schema already provides. With 100% schema description coverage, the baseline is 3 - the description doesn't compensate for any gaps because there are none in the schema, but it also doesn't add meaningful parameter insights.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('test suites for a project'), making the purpose immediately understandable. It distinguishes itself from siblings like 'get_all_tcm_test_suites_by_project' by being a general listing tool rather than TCM-specific. However, it doesn't explicitly differentiate from 'get_root_suites' or 'get_all_subsuites', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools that also retrieve test suites or related data (e.g., 'get_all_tcm_test_suites_by_project', 'get_root_suites', 'get_suite_hierarchy'), there's no indication of when this listing tool is preferred over those more specific options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
test_reporting_connectionA
๐ Test connection to Zebrunner Reporting API with new authentication
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral disclosure burden. It indicates this is a test/verification operation (implied read-only) and mentions authentication, but doesn't specify what 'test connection' entails (e.g., whether it validates credentials, endpoint availability, or both), what happens on success/failure, or any rate limits. It provides basic context but lacks operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that immediately conveys the tool's purpose with no wasted words. It's front-loaded with the core action and includes only essential contextual information about authentication.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no annotations and no output schema, the description provides adequate basic purpose and context. However, it doesn't explain what constitutes a successful test, what format/values the response might contain, or potential error conditions, leaving some operational ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on the tool's purpose and authentication context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Test connection') and target resource ('Zebrunner Reporting API'), with the additional context of 'with new authentication' that distinguishes it from generic connectivity checks. It uses a precise verb and identifies the exact system being tested.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context ('with new authentication') suggesting this should be used when authentication credentials have changed or need verification. However, it doesn't explicitly state when NOT to use it or name alternative tools for similar purposes among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_test_caseC
๐ Validate a test case against quality standards and best practices (Dynamic Rules Support + Improvement)
| Name | Required | Description | Default |
|---|---|---|---|
| projectKey | Yes | Project key (e.g., 'android' or 'ANDROID') | |
| caseKey | Yes | Test case key (e.g., 'ANDROID-29') | |
| rulesFilePath | No | Path to custom rules markdown file | |
| checkpointsFilePath | No | Path to custom checkpoints markdown file | |
| format | No | Output format | markdown |
| improveIfPossible | No | Attempt to automatically improve the test case | |
| include_clickable_links | No | Include clickable links to Zebrunner web UI |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions 'Dynamic Rules Support + Improvement' which hints at configurable rules and potential auto-improvement, but doesn't disclose critical details like whether this is a read-only analysis or makes changes, what permissions are required, error handling, or rate limits. The 'improveIfPossible' parameter suggests mutation capability, but this isn't explicitly stated in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that conveys the core purpose. The emoji adds visual distinction without being distracting. However, the parenthetical '(Dynamic Rules Support + Improvement)' could be integrated more smoothly, and the description lacks any structural separation of key concepts.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter tool with no annotations and no output schema, the description is inadequate. It doesn't explain what the validation output looks like, what 'quality standards and best practices' entail, how 'improvement' manifests, or the consequences of validation. The agent must rely entirely on parameter names and schema descriptions to understand this tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. It mentions 'Dynamic Rules Support' which loosely relates to 'rulesFilePath' and 'checkpointsFilePath', but doesn't explain their purpose or format. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('validate') and target ('test case') with additional context about quality standards and best practices. It distinguishes from siblings by mentioning 'Dynamic Rules Support + Improvement', which suggests a specific validation approach not present in tools like 'improve_test_case' or 'get_test_case_by_key'. However, it doesn't explicitly differentiate from all sibling tools that might involve test case analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when validation is needed, what triggers it, or how it differs from similar tools like 'improve_test_case' or 'get_enhanced_test_coverage_with_rules'. The agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
TDQS
Multiple tools have overlapping purposes, causing significant ambiguity. For example, 'get_all_tcm_test_cases_by_project', 'get_all_tcm_test_cases_with_root_suite_id', 'get_test_case_by_filter', 'get_test_cases_advanced', and 'get_test_cases_by_suite_smart' all retrieve test cases with subtle distinctions that an agent would struggle to differentiate. Similarly, 'analyze_test_cases_duplicates' and 'analyze_test_cases_duplicates_semantic' overlap in purpose, and 'get_launch_details' vs. 'get_launch_summary' vs. 'get_launch_test_summary' are confusingly similar. The descriptions help but don't fully resolve the redundancy.
The naming follows a mixed convention with some consistency but notable deviations. Most tools use snake_case (e.g., 'aggregate_test_cases_by_feature', 'get_launch_details'), which is good, but there are inconsistencies like 'list_test_runs' (verb-first) vs. 'get_all_launches_for_project' (get-first), and some tools lack clear verb patterns (e.g., 'test_reporting_connection'). While readable, the mix of 'get', 'list', 'analyze', and other verbs without a strict pattern reduces predictability.
With 49 tools, the count is excessive for a test management server, leading to bloat and confusion. Many tools could be consolidated (e.g., multiple test case retrieval methods, duplicate analysis variants), and the high number suggests poor scoping. This overwhelms agents and makes navigation difficult, as typical MCP servers for similar domains (like GitHub with 10-15 tools) are more manageable. The server's purpose is clear, but the tool surface is unnecessarily fragmented.
The tool set is highly comprehensive for test management, covering analysis, retrieval, reporting, and automation tasks with few obvious gaps. It includes CRUD-like operations for test cases, launches, and suites, along with advanced features like failure analysis and duplicate detection. Minor gaps might include direct update or delete tools for test cases, but the server focuses on read and analyze operations, which aligns with its reporting and analytics domain. Overall, the coverage is thorough and supports complex workflows.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Manage test suites, run tests, view results, and automate QA workflows via AI with testRigor.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Katalon True Platform MCP: design tests, run with AI, upload reports, assess release readiness.
Agentic testing: HyperExecute jobs, test failure triage, SmartUI visual diffs, a11y audits
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables intelligent analysis of regression test failures and automatic discovery of solutions in JIRA. Analyzes test logs using AI-driven algorithms and matches errors with relevant JIRA issues through natural language interactions.
- FlicenseNot gradedqualityDmaintenanceEnables AI assistants to analyze Java and Web projects, automatically generate unit tests (JUnit) and end-to-end tests (Selenium), and execute them directly through Maven or NPM commands.
- AlicenseAqualityBmaintenanceConnects AI coding assistants to TestCollab for managing test cases, plans, and suites directly through natural language. It enables users to create, update, and query testing resources within integrated development environments and AI chat clients.17174MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI models (Claude, ChatGPT, GitHub Copilot) to run and analyze local tests, rerun failures, and orchestrate QA workflows using existing UI and API test frameworks.19MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/maksimsarychau/mcp-zebrunner'
If you have feedback or need assistance with the MCP directory API, please join our Discord server