Skip to main content
Glama

@gaffer-sh/mcp

MCP (Model Context Protocol) server for Gaffer - give your AI assistant memory of your tests.

What is this?

This MCP server connects AI coding assistants like Claude Code and Cursor to your Gaffer test history and coverage data. It runs in code mode: three MCP tools over a namespace of 17 functions — 16 read-only analytics functions plus upload_test_results. It allows AI to:

  • Check your project's test health (pass rate, flaky tests, trends)

  • Look up the history of specific tests to understand stability

  • Get context about test failures when debugging

  • Analyze code coverage and identify untested areas

  • Browse all your projects (with user API Keys)

  • Access test report files (HTML reports, coverage, etc.)

Related MCP server: Tesults MCP

Prerequisites

  1. A Gaffer account with test results uploaded

  2. An API Key from Account Settings > API Keys

Setup

Claude Code (CLI)

The easiest way to add the Gaffer MCP server is via the Claude Code CLI:

claude mcp add gaffer -e GAFFER_API_KEY=gaf_your_api_key_here -- npx -y @gaffer-sh/mcp

Claude Code (Manual)

Alternatively, add to your Claude Code settings (~/.claude.json or project .claude/settings.json):

{
  "mcpServers": {
    "gaffer": {
      "command": "npx",
      "args": ["-y", "@gaffer-sh/mcp"],
      "env": {
        "GAFFER_API_KEY": "gaf_your_api_key_here"
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project:

{
  "mcpServers": {
    "gaffer": {
      "command": "npx",
      "args": ["-y", "@gaffer-sh/mcp"],
      "env": {
        "GAFFER_API_KEY": "gaf_your_api_key_here"
      }
    }
  }
}

How this server works

This server uses code mode. Instead of exposing one MCP tool per API call, it exposes three tools plus a codemode namespace you call from JavaScript. Fewer tool definitions occupy the context window, and a single execution can chain several calls.

MCP tool

What it does

execute_code

Run JavaScript against codemode.<function>(). Max 20 API calls, 30s timeout.

search_tools

Find available functions by keyword. An empty query lists all of them.

list_projects

List projects. Registered only when the token is a user API Key (gaf_).

const health = await codemode.get_project_health({ projectId: "proj_abc" });
if (health.flakyTestCount > 0) {
  const flaky = await codemode.get_flaky_tests({ projectId: "proj_abc" });
  return { health, flaky };
}
return { health };

Functions available via execute_code

Function

Category

Description

get_project_health

health

Health score, pass rate, flaky count, trend

get_test_history

testing

Pass/fail history for a specific test

get_flaky_tests

testing

Tests with high flip rates (pass↔fail)

list_test_runs

testing

Recent test runs, filterable by commit/branch/status

get_test_run_details

testing

Parsed individual results for one run

get_failure_clusters

testing

Failed tests grouped by root cause

get_slowest_tests

testing

Slowest tests by P95 duration

compare_test_metrics

testing

Compare test performance between commits or runs

search_failures

testing

Search failures by error or test-name pattern, or list all recent failures

get_coverage_summary

coverage

Overall coverage metrics and trend

get_coverage_for_file

coverage

Coverage for specific files or paths

get_untested_files

coverage

Files below a coverage threshold

find_uncovered_failure_areas

coverage

Files with low coverage AND test failures

get_report

reports

Report file URLs for a test run

get_report_browser_url

reports

Signed browser-navigable report URL (30 min)

get_upload_status

uploads

Whether CI results are uploaded and processed

upload_test_results

uploads

Upload test results (write) — rate-limited and audit-logged

Every function except upload_test_results is read-only.

Function Reference

list_projects

List all projects you have access to.

  • Input: organizationId (optional), limit (optional, default: 50)

  • Returns: List of projects with IDs, names, and organization info

  • Example: "What projects do I have in Gaffer?"

get_project_health

Get the health metrics for a project.

  • Input: projectId (required), days (optional, default: 30)

  • Returns: Health score (0-100), pass rate, test run count, flaky test count, trend

  • Example: "What's the health of my test suite?"

get_test_history

Get the pass/fail history for a specific test.

  • Input: projectId (required), testName or filePath (one required), limit (optional)

  • Returns: History of runs with status, duration, branch, commit, errors

  • Example: "Is the login test flaky? Check its history"

get_flaky_tests

Get the list of flaky tests in a project.

  • Input: projectId (required), threshold (optional, default: 0.1), days (optional), limit (optional)

  • Returns: List of flaky tests with flip rates, transition counts, run counts

  • Example: "Which tests are flaky in my project?"

list_test_runs

List recent test runs with optional filtering.

  • Input: projectId (required), commitSha (optional), branch (optional), status (optional), limit (optional)

  • Returns: List of test runs with pass/fail/skip counts, commit and branch info

  • Example: "What tests failed in the last commit?"

get_test_run_details

Get parsed test results for a specific test run.

  • Input: testRunId (required), projectId (required), status (optional filter), limit (optional)

  • Returns: Individual test results with name, status, duration, file path, errors

  • Example: "Show me all failed tests from this test run"

get_report

Get URLs for report files uploaded with a test run.

  • Input: testRunId (required)

  • Returns: List of files with filename, size, content type, download URL

  • Example: "Get the Playwright report for the latest test run"

get_report_browser_url

Get a browser-navigable URL for viewing a test report.

  • Input: projectId (required), testRunId (required), filename (optional)

  • Returns: Signed URL valid for 30 minutes

  • Example: "Give me a link to view the test report"

get_slowest_tests

Get the slowest tests in a project, sorted by P95 duration.

  • Input: projectId (required), days (optional), limit (optional), framework (optional), branch (optional)

  • Returns: List of tests with average and P95 duration, run count

  • Example: "Which tests are slowing down my CI pipeline?"

compare_test_metrics

Compare test metrics between two commits or test runs.

  • Input: projectId (required), testName (required), beforeCommit/afterCommit OR beforeRunId/afterRunId

  • Returns: Before/after metrics with duration change and percentage

  • Example: "Did my fix make this test faster?"

get_coverage_summary

Get the coverage metrics summary for a project.

  • Input: projectId (required), days (optional, default: 30)

  • Returns: Line/branch/function coverage percentages, trend, report count, lowest coverage files

  • Example: "What's our test coverage?"

get_coverage_for_file

Get coverage metrics for specific files or paths.

  • Input: projectId (required), filePath (required - exact or partial match)

  • Returns: List of matching files with line/branch/function coverage

  • Example: "What's the coverage for our API routes?"

get_untested_files

Get files with little or no test coverage.

  • Input: projectId (required), maxCoverage (optional, default: 10%), limit (optional)

  • Returns: List of files below threshold sorted by coverage (lowest first)

  • Example: "Which files have no tests?"

find_uncovered_failure_areas

Find code areas with both low coverage AND test failures (high risk).

  • Input: projectId (required), days (optional), coverageThreshold (optional, default: 80%)

  • Returns: Risk areas ranked by score, with file path, coverage %, failure count

  • Example: "Where should we focus our testing efforts?"

get_failure_clusters

Group failed tests by root cause using error message similarity.

  • Input: projectId (required), testRunId (required)

  • Returns: Clusters of failed tests grouped by similar error messages, with representative error and test count

  • Example: "Are these 15 failures from the same bug?"

search_failures

Search past failures by error message, stack trace, or test name — or list every failure in the window.

  • Input: query (optional — omit to return all failures), projectId (required for gaf_ keys), searchIn (optional: errors/names/all, default all), days (optional, default: 30), branch (optional), limit (optional, default: 20)

  • Returns: Matching failures with test name, error message, run and commit context, plus truncated when scan caps cut the list short

  • Example: "Have we seen this connection-refused error before?" / "What failed in the last 7 days?"

get_upload_status

Check if CI results have been uploaded and processed.

  • Input: projectId (required), sessionId (optional), commitSha (optional), branch (optional)

  • Returns: Upload session(s) with processing status, linked test runs and coverage reports

  • Example: "Are my test results ready for commit abc123?"

upload_test_results

Upload structured test results. This is the only function that writes.

Use it when you have results in hand — parsed from CI output or a runner's JSON report — and no Gaffer CLI is available to upload them.

  • Input: projectId (required for gaf_ keys), framework (required), tests (required), branch, commitSha, ciProvider, startedAt, finishedAt, coverage

  • Returns: uploadSessionId, the generated runId, and the derived pass/fail/skip summary

  • Example: "Upload these 42 parsed pytest results so we can track them"

runId, the run timestamps and the summary are derived from tests — pass startedAt/finishedAt only if you know the real wall-clock window.

Two constraints worth knowing:

  • Not idempotent. Each call creates a new run, so a retry after an uncertain failure produces a duplicate. Check get_upload_status instead of retrying.

  • Rate-limited per project, and every call is written to the project's audit log with the id of the credential that made it.

Processing is asynchronous: results take a few seconds to become visible to the read functions.

Agentic CI Workflows

These workflows show how an AI agent diagnoses CI failures, waits for results, and finds coverage gaps. Each step is a codemode function, so a whole chain runs inside one execute_code call rather than one round-trip per step.

Workflow: Diagnose CI Failures

list_test_runs(projectId, status="failed")
  → get_test_run_details(projectId, testRunId, status="failed")
  → get_failure_clusters(projectId, testRunId)
  → get_test_history(projectId, testName="...")
  → compare_test_metrics(projectId, testName, beforeCommit, afterCommit)
  1. Find the failed test run

  2. Get individual failure details with stack traces

  3. Group failures by root cause — often 15 failures are 2-3 bugs

  4. Check if each failure is new (regression) or recurring

  5. Verify fixes by comparing before/after

Workflow: Wait for Results

get_upload_status(projectId, commitSha="abc123")
  → poll until processingStatus="completed"
  → get_test_run_details(projectId, testRunId)
  1. Check if results for a commit have been uploaded

  2. Wait for processing to complete

  3. Use linked test run IDs to get results

Workflow: Find Coverage Gaps

find_uncovered_failure_areas(projectId)
  → get_untested_files(projectId)
  → get_coverage_for_file(projectId, filePath="src/critical/")
  1. Find files with both low coverage and test failures (highest risk)

  2. Find files with no coverage at all

  3. Drill into specific directories for targeted analysis

Function Quick Reference

Agent Question

Function

"What failed?"

get_test_run_details

"Same root cause?"

get_failure_clusters

"Seen this error before?"

search_failures

"Is it flaky?"

get_flaky_tests

"Is this new?"

get_test_history

"Did my fix work?"

compare_test_metrics

"Are results ready?"

get_upload_status

"What's untested?"

find_uncovered_failure_areas

"What's slow?"

get_slowest_tests

Prioritizing Coverage Improvements

When using coverage tools to improve your test suite, combine coverage data with codebase exploration for best results:

1. Understand Code Utilization

Before targeting files purely by coverage percentage, explore which code is actually critical:

  • Find entry points: Look for route definitions, event handlers, exported functions - these reveal what code actually executes in production

  • Find heavily-imported files: Files imported by many others are high-value targets

  • Identify critical business logic: Look for files handling auth, payments, data mutations, or core domain logic

2. Prioritize by Impact

Low coverage alone doesn't indicate priority. Consider:

  • High utilization + low coverage = highest priority - Code that runs frequently but lacks tests

  • Large files with 0% coverage - More uncovered lines means bigger impact on overall coverage

  • Files with both failures and low coverage - Use find_uncovered_failure_areas for this

3. Use Path-Based Queries

The get_untested_files tool may return many frontend components. For backend or specific areas:

# Query specific paths with get_coverage_for_file
get_coverage_for_file(filePath="server/services")
get_coverage_for_file(filePath="src/api")
get_coverage_for_file(filePath="lib/core")

4. Iterative Improvement

  1. Get baseline with get_coverage_summary

  2. Identify targets with get_coverage_for_file on critical paths

  3. Write tests for highest-impact files

  4. Re-check coverage after CI uploads new results

  5. Repeat

Authentication

User API Keys (gaf_ prefix) provide read-only access to all projects across your organizations. Get your API Key from: Account Settings > API Keys

Project Tokens

Project Tokens (gfr_ prefix) are designed for uploading test results and only provide access to a single project. When you use one, omit projectId — it resolves automatically. User API Keys are preferred for the MCP server because they enable list_projects and read across projects.

Environment Variables

Variable

Required

Description

GAFFER_API_KEY

Yes

Your Gaffer API Key (starts with gaf_)

GAFFER_API_URL

No

API base URL (default: https://app.gaffer.sh)

Local Development

pnpm install
pnpm build

Test locally with Claude Code (use absolute path to built file):

{
  "mcpServers": {
    "gaffer": {
      "command": "node",
      "args": ["/absolute/path/to/dist/index.js"],
      "env": {
        "GAFFER_API_KEY": "gaf_..."
      }
    }
  }
}

License

MIT

Available Tools

3 tools
execute_codeExecute CodeA

Execute JavaScript code that calls Gaffer API functions via the codemode namespace.

Write async JavaScript — all functions are available as codemode.<function_name>(input). Use return to send results back. Use console.log() for debug output.

Available Functions

/** Get the health metrics for a project.

Returns:
- Health score (0-100): Overall project health based on pass rate and trend
- Pass rate: Percentage of tests passing
- Test run count: Number of test runs in the period
- Flaky test count: Number of tests with inconsistent results
- Trend: Whether test health is improving (up), declining (down), or stable

Use this to understand the current state of your test suite. */
get_project_health(input: { projectId?: string; days?: number }): Promise<any>

/** Get the pass/fail history for a specific test.

Search by either:
- testName: The exact name of the test (e.g., "should handle user login")
- filePath: The file path containing the test (e.g., "tests/auth.test.ts")

Returns:
- History of test runs showing pass/fail status over time
- Duration of each run
- Branch and commit information
- Error messages for failed runs
- Summary statistics (pass rate, total runs)

Use this to investigate flaky tests or understand test stability. */
get_test_history(input: { projectId?: string; testName?: string; filePath?: string; limit?: number }): Promise<any>

/** Get the list of flaky tests in a project.

A test is considered flaky if it frequently switches between pass and fail states.
Tests are ranked by a composite flakinessScore that factors in flip behavior,
failure rate, and duration variability.

Returns:
- List of flaky tests sorted by flakinessScore (most flaky first), with:
  - name: Test name
  - flipRate: How often the test flips between pass/fail (0-1)
  - flipCount: Number of status transitions
  - totalRuns: Total test executions analyzed
  - lastSeen: When the test last ran
  - flakinessScore: Composite score (0-1) combining flip proximity, failure rate, and duration variability
- Summary with threshold used and total count

Use this after get_project_health shows flaky tests exist, to identify which
specific tests are flaky and need investigation. */
get_flaky_tests(input: { projectId?: string; threshold?: number; limit?: number; days?: number }): Promise<any>

/** List recent test runs for a project with optional filtering.

Filter by:
- commitSha: Filter by commit SHA (supports prefix matching)
- branch: Filter by branch name
- status: Filter by "passed" (no failures) or "failed" (has failures)

Returns:
- List of test runs with:
  - id: Test run ID (can be used with get_test_run for details)
  - commitSha: Git commit SHA
  - branch: Git branch name
  - passedCount/failedCount/skippedCount: Test counts
  - createdAt: When the test run was created
- Pagination info (total count, hasMore flag)

Use cases:
- "What tests failed in commit abc123?"
- "Show me recent test runs on main branch"
- "What's the status of tests on my feature branch?" */
list_test_runs(input: { projectId?: string; commitSha?: string; branch?: string; status?: 'passed' | 'failed'; limit?: number }): Promise<any>

/** Get URLs for report files uploaded with a test run.

IMPORTANT: This tool returns download URLs, not file content. You must fetch the URLs separately.

Returns for each file:
- filename: The file name (e.g., "report.html", "results.json", "junit.xml")
- size: File size in bytes
- contentType: MIME type (e.g., "text/html", "application/json", "application/xml")
- downloadUrl: Presigned URL to download the file (valid for ~5 minutes)

How to use the returned URLs:

1. **JSON files** (results.json, coverage.json):
   Use WebFetch with the downloadUrl to retrieve and parse the JSON content.
   Example: WebFetch(url=downloadUrl, prompt="Extract test results from this JSON")

2. **XML files** (junit.xml, xunit.xml):
   Use WebFetch with the downloadUrl to retrieve and parse the XML content.
   Example: WebFetch(url=downloadUrl, prompt="Parse the test results from this JUnit XML")

3. **HTML reports** (Playwright, pytest-html, Vitest):
   These are typically bundled React/JavaScript applications that require a browser.
   They cannot be meaningfully parsed by WebFetch.
   For programmatic analysis, use get_test_run_details instead.

Recommendations:
- For analyzing test results programmatically: Use get_test_run_details (returns parsed test data)
- For JSON/XML files: Use this tool + WebFetch on the downloadUrl
- For HTML reports: Direct users to view in browser, or use get_test_run_details

Use cases:
- "What files are in this test run?" (list available reports)
- "Get the coverage data from this run" (then WebFetch the JSON URL)
- "Parse the JUnit XML results" (then WebFetch the XML URL) */
get_report(input: { testRunId: string }): Promise<any>

/** Get the slowest tests in a project, sorted by P95 duration.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- days (optional): Analysis period in days (default: 30, max: 365)
- limit (optional): Max tests to return (default: 20, max: 100)
- framework (optional): Filter by framework (e.g., "playwright", "vitest")
- branch (optional): Filter by git branch (e.g., "main", "develop")

Returns:
- List of slowest tests with:
  - name: Short test name
  - fullName: Full test name including describe blocks
  - filePath: Test file path (if available)
  - framework: Test framework used
  - avgDurationMs: Average test duration in milliseconds
  - p95DurationMs: 95th percentile duration (used for sorting)
  - runCount: Number of times the test ran in the period
- Summary with project info and period

Use cases:
- "Which tests are slowing down my CI pipeline?"
- "Find the slowest Playwright tests to optimize"
- "Show me e2e tests taking over 30 seconds"
- "What are the slowest tests on the main branch?" */
get_slowest_tests(input: { projectId?: string; days?: number; limit?: number; framework?: string; branch?: string }): Promise<any>

/** Get parsed test results for a specific test run.

Parameters:
- testRunId (required): The test run ID to get details for
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- status (optional): Filter by test status: "passed", "failed", or "skipped"
- limit (optional): Max tests to return (default: 100, max: 500)
- offset (optional): Pagination offset (default: 0)

Returns:
- testRunId: The test run ID
- commitSha: Git commit SHA (null if not recorded)
- branch: Git branch name (null if not recorded)
- framework: Test framework (e.g., "playwright", "vitest")
- createdAt: When the test run was created (ISO 8601)
- summary: Overall counts (passed, failed, skipped, total)
- tests: Array of individual test results with:
  - name: Short test name
  - fullName: Full test name including describe blocks
  - status: Test status (passed, failed, skipped)
  - durationMs: Test duration in milliseconds (null if not recorded)
  - filePath: Test file path (null if not recorded)
  - error: Error message for failed tests (null otherwise)
  - errorStack: Full stack trace for failed tests (null otherwise)
- pagination: Pagination info (total, limit, offset, hasMore)

Use cases:
- "Show me all failed tests from this test run"
- "Get the test results from commit abc123"
- "List tests that took the longest in this run"
- "Find tests with errors in the auth module"

Note: For aggregate analytics like flaky test detection or duration trends,
use get_test_history, get_flaky_tests, or get_slowest_tests instead. */
get_test_run_details(input: { testRunId: string; projectId?: string; status?: 'passed' | 'failed' | 'skipped'; limit?: number; offset?: number }): Promise<any>

/** Group failed tests by root cause using error message similarity.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- testRunId (required): The test run ID to analyze

Returns:
- clusters: Array of failure clusters, each containing:
  - representativeError: The error message representing this cluster
  - count: Number of tests with this same root cause
  - tests: Array of individual failed tests in this cluster
    - name: Short test name
    - fullName: Full test name including describe blocks
    - errorMessage: The specific error message
    - filePath: Test file path (null if not recorded)
  - similarity: Similarity threshold used for clustering (0-1)
- totalFailures: Total number of failed tests across all clusters

Use cases:
- "Group these 15 failures by root cause" — often reveals 2-3 distinct bugs
- "Which error affects the most tests?" — fix the largest cluster first
- "Are these failures related?" — check if they land in the same cluster

Tip: Use get_test_run_details with status='failed' first to see raw failures,
then use this tool to understand which failures share the same root cause. */
get_failure_clusters(input: { projectId?: string; testRunId: string }): Promise<any>

/** Compare test metrics between two commits or test runs.

Useful for measuring the impact of code changes on test performance or reliability.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- testName (required): The test name to compare (short name or full name)
- Option 1 - Compare by commit:
  - beforeCommit: Commit SHA for "before" measurement
  - afterCommit: Commit SHA for "after" measurement
- Option 2 - Compare by test run:
  - beforeRunId: Test run ID for "before" measurement
  - afterRunId: Test run ID for "after" measurement

Returns:
- testName: The test that was compared
- before: Metrics from the before commit/run
  - testRunId, commit, branch, status, durationMs, createdAt
- after: Metrics from the after commit/run
  - testRunId, commit, branch, status, durationMs, createdAt
- change: Calculated changes
  - durationMs: Duration difference (negative = faster)
  - percentChange: Percentage change (negative = improvement)
  - statusChanged: Whether pass/fail status changed

Use cases:
- "Did my fix make this test faster?"
- "Compare test performance between these two commits"
- "Did this test start failing after my changes?"
- "Show me the before/after for the slow test I optimized"

Tip: Use get_test_history first to find the commit SHAs or test run IDs you want to compare. */
compare_test_metrics(input: { projectId?: string; testName: string; beforeCommit?: string; afterCommit?: string; beforeRunId?: string; afterRunId?: string }): Promise<any>

/** Get the coverage metrics summary for a project.

Returns:
- Current coverage percentages (lines, branches, functions)
- Trend direction (up, down, stable) and change amount
- Total number of coverage reports
- Latest report date
- Top 5 files with lowest coverage

Use this to understand your project's overall test coverage health.

After getting the summary, use get_coverage_for_file with path prefixes to drill into
specific areas (e.g., "server/services", "src/api", "lib/core"). This helps identify
high-value targets in critical code paths rather than just the files with lowest coverage. */
get_coverage_summary(input: { projectId?: string; days?: number }): Promise<any>

/** Get coverage metrics for a specific file or files matching a path pattern.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- filePath: File path to search for (exact or partial match)

Returns:
- Line coverage (covered/total/percentage)
- Branch coverage (covered/total/percentage)
- Function coverage (covered/total/percentage)

This is the preferred tool for targeted coverage analysis. Use path prefixes to focus on
specific areas of the codebase:
- "server/services" - Backend service layer
- "server/utils" - Backend utilities
- "src/api" - API routes
- "lib/core" - Core business logic

Before querying, explore the codebase to identify critical paths - entry points,
heavily-imported files, and code handling auth/payments/data mutations.
Prioritize: high utilization + low coverage = highest impact. */
get_coverage_for_file(input: { projectId?: string; filePath: string }): Promise<any>

/** Find areas of code that have both low coverage AND test failures.

This cross-references test failures with coverage data to identify high-risk
areas in your codebase that need attention. Files are ranked by a "risk score"
calculated as: (100 - coverage%) × failureCount.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- days: Analysis period for test failures (default: 30)
- coverageThreshold: Include files below this coverage % (default: 80)

Returns:
- List of risk areas sorted by risk score (highest risk first)
- Each area includes: file path, coverage %, failure count, risk score, test names

Use this to prioritize which parts of your codebase need better test coverage. */
find_uncovered_failure_areas(input: { projectId?: string; days?: number; coverageThreshold?: number }): Promise<any>

/** Get files with little or no test coverage.

Returns files sorted by coverage percentage (lowest first), filtered
to only include files below a coverage threshold.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- maxCoverage: Include files with coverage at or below this % (default: 10)
- limit: Maximum number of files to return (default: 20, max: 100)

Returns:
- List of files sorted by coverage (lowest first)
- Each file includes line/branch/function coverage metrics
- Total count of files matching the criteria

IMPORTANT: Results may be dominated by certain file types (e.g., UI components) that are
numerous but not necessarily the highest priority. For targeted analysis of specific code
areas (backend, services, utilities), use get_coverage_for_file with path prefixes instead.

To prioritize effectively, explore the codebase to understand which code is heavily utilized
(entry points, frequently-imported files, critical business logic) and then query coverage
for those specific paths. */
get_untested_files(input: { projectId?: string; maxCoverage?: number; limit?: number }): Promise<any>

/** Get a browser-navigable URL for viewing a test report (Playwright, Vitest, etc.).

Returns a signed URL that can be opened directly in a browser without requiring
the user to log in. The URL expires after 30 minutes for security.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- testRunId: The test run to view (required)
- filename: Specific file to open (optional, defaults to index.html)

Returns:
- url: Browser-navigable URL with signed token
- filename: The file being accessed
- expiresAt: ISO timestamp when the URL expires
- expiresInSeconds: Time until expiration

The returned URL can be shared with users who need to view the report.
Note: URLs expire after 30 minutes for security. */
get_report_browser_url(input: { projectId?: string; testRunId: string; filename?: string }): Promise<any>

/** Check if CI results have been uploaded and processed.

Use this tool to answer "are my test results ready?" after pushing code.

Parameters:
- projectId (optional): Project ID — required for user API keys, auto-resolved for project tokens
- sessionId (optional): Specific upload session ID for detailed status
- commitSha (optional): Filter by commit SHA to find uploads for a specific commit
- branch (optional): Filter by branch name

Behavior:
- If sessionId is provided: returns detailed status with linked test runs and coverage reports
- Otherwise: returns a list of recent upload sessions (filtered by commitSha/branch if provided)

Processing statuses:
- "pending" — upload received, processing not started
- "processing" — files are being parsed
- "completed" — all files processed successfully, results are ready
- "error" — some files failed to process

Workflow:
1. After pushing code, call with commitSha to find the upload session
2. Check processingStatus — if "completed", results are ready
3. If "processing" or "pending", wait and check again
4. Once completed, use the linked testRunIds with get_test_run_details

Returns (list mode):
- sessions: Array of upload sessions with processing status
- pagination: Pagination info

Returns (detail mode):
- session: Upload session details
- testRuns: Linked test run summaries (id, framework, pass/fail counts)
- coverageReports: Linked coverage report summaries (id, format) */
get_upload_status(input: { projectId?: string; sessionId?: string; commitSha?: string; branch?: string }): Promise<any>

/** Search across test failures by error message, stack trace, or test name.

Use this to find specific failures across test runs — like grep for your test history.

Examples:
- "TypeError: Cannot read properties of undefined" → find all occurrences of this error
- "timeout" → find timeout-related failures
- "auth" with searchIn="names" → find failing auth tests

Returns matching failures with test run context (branch, commit, timestamp) for investigation. */
search_failures(input: { projectId?: string; query: string; searchIn?: 'errors' | 'names' | 'all'; days?: number; branch?: string; limit?: number }): Promise<any>

Examples

// Single call
const health = await codemode.get_project_health({ projectId: "proj_abc" });
return health;
// Multi-step: get flaky tests and check history for each
const flaky = await codemode.get_flaky_tests({ projectId: "proj_abc", limit: 5 });
const histories = [];
for (const test of flaky.flakyTests) {
  const history = await codemode.get_test_history({ projectId: "proj_abc", testName: test.name, limit: 5 });
  histories.push({ test: test.name, score: test.flakinessScore, history: history.summary });
}
return { flaky: flaky.summary, details: histories };
// Coverage analysis
const summary = await codemode.get_coverage_summary({ projectId: "proj_abc" });
const lowFiles = await codemode.get_coverage_for_file({ projectId: "proj_abc", maxCoverage: 50, limit: 10 });
return { summary, lowCoverageFiles: lowFiles };

Constraints

  • Max 20 API calls per execution

  • 30s timeout

  • No access to Node.js globals (process, require, etc.)

ParametersJSON Schema
NameRequiredDescriptionDefault
codeYesJavaScript code to execute. Use `codemode.<function>()` to call API functions. Use `return` for results.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and excels. It discloses all behavioral traits: async execution, codemode namespace, return/console.log usage, timeout, call limit, and no access to Node.js globals. It also details each function's behavior, inputs, and outputs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is lengthy but well-structured with clear sections, function signatures, and examples. It earns its length by documenting many functions. Minor deduction for verbosity in some function descriptions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is extremely complete, covering all available functions, their parameters, return values, use cases, and limitations. It compensates for the lack of an output schema by detailing return structures.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the single parameter 'code' is well-described in the schema (100% coverage), the description adds immense value by explaining the execution context, provided API functions, and usage examples. This far exceeds the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Execute JavaScript code that calls Gaffer API functions via the `codemode` namespace.' It specifies the execution environment, available functions, and how to return results. This distinguishes it from siblings like list_projects and search_tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides extensive usage guidance, including when to use each specific function (e.g., 'Use this to understand the current state of your test suite'), examples of multi-step operations, and explicit constraints (max 20 calls, 30s timeout, no Node.js globals). It implicitly tells when not to use this tool by listing alternatives within the function descriptions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_projectsList ProjectsA

List all projects you have access to.

Returns a list of projects with their IDs, names, and organization info. Use this to find project IDs for other tools like get_project_health.

Requires a user API Key (gaf_). Get one from Account Settings in the Gaffer dashboard.

ParametersJSON Schema
NameRequiredDescriptionDefault
organizationIdNoFilter by organization ID (optional)
limitNoMaximum number of projects to return (default: 50)

Output Schema

ParametersJSON Schema
NameRequiredDescription
projectsYes
totalYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but the description discloses the API key requirement and where to obtain it. It also describes the return content. For a read-only list operation, this is adequate behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the main action, no wasted words. Concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (not shown), the description sufficiently covers the tool's behavior. It mentions the API key requirement and return fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and includes descriptions for both parameters. The description adds no additional meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists all accessible projects and returns IDs, names, and org info. This is distinct from sibling tools execute_code and search_tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a concrete use case: find project IDs for get_project_health. However, it does not mention when not to use or alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_toolsSearch ToolsA

Search for available Gaffer API functions by keyword.

Returns matching functions with their TypeScript declarations so you can use them with execute_code.

Examples:

  • "coverage" → coverage-related functions

  • "flaky" → flaky test detection

  • "" (empty) → list all available functions

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoSearch query to find relevant functions. Leave empty to list all available functions.

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description does not mention safety, permissions, or side effects. It indicates read-only operation (search) but does not explicitly state behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences plus examples, front-loaded purpose, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple search tool with one optional param and no output schema, the description fully explains functionality and return type (TypeScript declarations). No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and baseline is 3. Description adds value with examples and context about using with execute_code, improving beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it searches for Gaffer API functions by keyword, provides examples, and distinguishes from siblings (execute_code, list_projects). Verb 'search' and resource 'Gaffer API functions' are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description gives clear usage context (search by keyword, list all with empty query) but lacks explicit when-not-to-use or alternatives beyond implied sibling differentiation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 17 tool updatesv0.4.3
    • Removedcompare_test_metrics
    • Addedexecute_code
    • Removedfind_uncovered_failure_areas
    • Removedget_coverage_for_file
    • Removedget_coverage_summary
    • Removedget_failure_clusters
    • Removedget_flaky_tests
    • Removedget_project_health
    • Removedget_report
    • Removedget_report_browser_url
    • Removedget_slowest_tests
    • Removedget_test_history
    • Removedget_test_run_details
    • Removedget_untested_files
    • Removedget_upload_status
    • Removedlist_test_runs
    • Addedsearch_tools
  2. 16 tool updatesv0.4.2
    • First observedcompare_test_metrics
    • First observedfind_uncovered_failure_areas
    • First observedget_coverage_for_file
    • First observedget_coverage_summary
    • First observedget_failure_clusters
    • First observedget_flaky_tests
    • First observedget_project_health
    • First observedget_report
    • First observedget_report_browser_url
    • First observedget_slowest_tests
    • First observedget_test_history
    • First observedget_test_run_details
    • First observedget_untested_files
    • First observedget_upload_status
    • First observedlist_projects
    • First observedlist_test_runs

TDQS

A4.5/5.0

Scored across 3 tools

Disambiguation5/5

The three tools have clearly distinct purposes: executing code, listing projects, and searching tools. There is no overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with snake_case: execute_code, list_projects, search_tools.

Tool Count5/5

Three tools is well-scoped for this server's purpose. The primary tool execute_code encapsulates a wide range of functions, and the auxiliary tools support it effectively.

Completeness5/5

The server covers the full test analysis lifecycle: project health, test history, flaky tests, test runs, reports, coverage, failure clustering, comparison, and more. No obvious gaps.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Connect engineering metrics, DORA performance, deploy risk scoring, and PR health to any AI assistant. Score PRs for deployment risk using a 36-signal model, query team health, incidents, coverage, and more.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Connect AI agents to your test results, insights, and targets. Query test runs, failures, flaky tests, and regressions across frameworks including Playwright, Jest, Pytest, Cypress and more.
    26 npm
    MIT