How do I use blacksmith-mcp?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@blacksmith-mcp Show me the failed tests from the latest workflow run" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

blacksmith-mcp

by grahamnotgrant

Overview Schema Source Related Servers Discussions Admin

TypeScript

Remote

Blacksmith MCP

An MCP server that connects Claude to your Blacksmith CI data. Query workflow runs, analyze test failures, detect flaky tests, and monitor usage—all through natural conversation.

Why?

Debugging CI failures usually means clicking through dashboards, copying run IDs, and piecing together information across multiple pages. With this MCP, you can just ask:

"Why did the last CI run fail?"
"Which tests are flaky this week?"
"Compare test failures between main and my PR"
"What's using the most cache storage?"

Claude handles the API calls and gives you actionable insights.

Related MCP server: Langfuse MCP Server

Quick Start

Zero-config if you're logged into Blacksmith in Chrome:

# Add to Claude Code
claude mcp add blacksmith -- npx blacksmith-mcp

# Set your org (run once)
export BLACKSMITH_ORG="your-org-name"

The MCP automatically extracts your session from Chrome cookies. No manual token copying needed.

Installation

Option 1: Claude Code CLI

claude mcp add blacksmith -- npx blacksmith-mcp

Option 2: Project Configuration

Add to your .mcp.json:

{
  "mcpServers": {
    "blacksmith": {
      "type": "stdio",
      "command": "npx",
      "args": ["blacksmith-mcp"],
      "env": {
        "BLACKSMITH_ORG": "your-org-name"
      }
    }
  }
}

Option 3: Global Install

npm install -g blacksmith-mcp

Configuration

Authentication

Automatic (recommended): Log into app.blacksmith.sh in Chrome. The MCP extracts your session cookie automatically.

Manual: Set BLACKSMITH_SESSION_COOKIE environment variable with your session cookie value.

Environment Variables

Variable	Required	Description
`BLACKSMITH_ORG`	Yes	Your Blacksmith organization name
`BLACKSMITH_SESSION_COOKIE`	No	Session cookie (auto-extracted from Chrome if not set)

Available Tools

Workflow Runs

Tool	Description
`list_runs`	List workflow runs with filters (status, branch, workflow, actor, PR)
`get_run`	Get run details including all jobs
`list_jobs`	List jobs for a workflow run
`get_job`	Get job details (steps, timing, runner info)
`get_job_logs`	Get raw log output for a job

Test Analytics

Tool	Description
`get_job_tests`	Get all test results for a job
`get_failed_tests`	Get failed tests with full error messages
`get_failures_by_pattern`	Group failures by error pattern (e.g., "Cannot read properties")
`compare_test_runs`	Compare failures between two runs (find regressions)
`get_flaky_tests`	Detect tests that fail intermittently
`get_slow_tests`	Find tests exceeding duration threshold
`get_test_history`	Track a specific test's pass/fail history
`get_trends`	Analyze trends: duration, failure rate, test count

Usage & Billing

Tool	Description
`get_current_usage`	Current core usage (active vs max)
`get_invoice_amount`	Current billing period amount
`get_usage_summary`	Billable minutes vs free tier
`get_cache_stats`	Cache size, entries by repository
`get_cache_entries`	Detailed cache entries for a repo

Organization

Tool	Description
`list_orgs`	List accessible organizations
`get_org_status`	Org status (personal, onboarded, region)
`search_logs`	Search logs across all jobs

Example Conversations

Debug a Failed Run

You: Why did the last CI run fail?
Claude: Uses list_runs(status="failure", limit=1) then get_failed_tests
The last run failed with 34 test failures. The main issues are:
Connection timeout (12 tests) - Tests in api/ are timing out connecting to the test database
Undefined property (8 tests) - user.settings.theme is undefined in the new dark mode code
Assertion failures (14 tests) - Snapshot mismatches in UI components

Find Flaky Tests

You: Which tests are flaky?
Claude: Uses get_flaky_tests(days=7, min_runs=5)
Found 3 flaky tests:
Test
Flake Rate
Pattern
should handle concurrent requests
40% (4/10 runs)
Fails on CI, passes locally
renders loading state
20% (2/10 runs)
Race condition with setTimeout
validates auth token
30% (3/10 runs)
Intermittent network timeout

Compare Branches

You: Did my PR introduce new test failures?
Claude: Uses compare_test_runs with PR run vs base branch
Your PR introduced 2 new failures:
UserProfile.test.tsx: "Cannot read property 'avatar' of undefined"
api/users.test.ts: Expected 200, got 404
5 previously failing tests now pass (nice!).

Development

# Install dependencies
pnpm install

# Build
pnpm build

# Development mode (watch)
pnpm dev

# Test with MCP Inspector
npx @modelcontextprotocol/inspector node dist/index.js

Troubleshooting

Session Expired

If you see SESSION_EXPIRED, your Blacksmith session has expired. Simply log back into app.blacksmith.sh in Chrome and retry.

The automatic cookie extraction requires:

macOS with Chrome installed
Being logged into Blacksmith in Chrome
Chrome not running with a locked profile

If it fails, set BLACKSMITH_SESSION_COOKIE manually.

No Organization Set

Run list_orgs to see available organizations, then set BLACKSMITH_ORG to your org name.

API Notes

This MCP uses Blacksmith's internal web API, which is undocumented. The API was reverse-engineered from the Blacksmith web app and may change without notice.

License

MIT

Contributing

Contributions welcome! Please open an issue first to discuss proposed changes.

Available Tools

37 tools

compare_test_runsA

Compare test failures between two runs to identify regressions. Shows new failures, fixed tests, and persistent failures. If base_run_id not provided, compares against most recent prior run.

Inputs: run_id (required), job_name (required), base_run_id.

get_active_branchesB

Get branches with CI activity. Optionally filter by repository.

Inputs: repository (required), end_date, start_date.

get_active_repositoriesA

Get repositories with CI activity in a period. Shows which repos are actively running workflows.

Inputs: end_date, start_date.

get_actorsB

Get users who have triggered workflow runs. Useful for filtering runs by actor.

Inputs: no named parameters.

get_cache_entriesA

Get detailed cache entries for a repository. Shows cache keys, sizes, scopes (branches), and last hit times. Useful for debugging cache issues.

Inputs: repository (required), limit.

get_cache_statsA

Get Blacksmith cache statistics: total size, hit rate, entries by repository. Shows how effectively caching is being used.

Inputs: include_history.

get_core_usage_timeseriesA

Get core usage over time. Shows how many cores are being used at each interval. Defaults to last 24 hours with 15-minute intervals.

Inputs: hours, window_size.

get_current_usageA

Get current core usage snapshot (active cores vs max cores).

Inputs: no named parameters.

get_failed_testsA

Get failed tests for a job with full error details. Use error_lines param to control stack trace length. Returns all failures by default (no limit).

Inputs: job_id (required), run_id (required), limit, suite, +1 more.

get_failures_by_patternA

Group failed tests by error pattern (e.g., "is not a function", "Cannot read properties"). Shows count, affected suites/files, and sample error for each pattern. Best for quickly identifying root causes.

Inputs: job_id (required), run_id (required), top_n.

get_flaky_testsA

Detect flaky tests by analyzing pass/fail patterns across recent runs. Returns tests that fail intermittently (e.g., "failed 3 of 10 runs"). Killer feature for CI stability.

Inputs: job_name (required), days, min_runs, threshold.

get_invoice_amountA

Get the current billing period invoice amount.

Inputs: no named parameters.

get_jobC

Get details of a specific job including steps, runner info, and timing.

Inputs: job_id (required), run_id (required).

get_job_duration_distributionA

Get job duration distribution with percentile breakdowns (P50/P90/P95/P99/Max). Shows histogram of how long jobs take and identifies outliers.

Inputs: end_date, start_date, bucket_count.

get_job_logsA

Get the logs for a specific job. Returns raw log output.

Inputs: job_id (required), limit, vm_id.

get_jobs_by_dimensionA

Break down job metrics by dimension: repository, workflow, branch, or runner_type. Shows which repos/workflows consume the most CI time. Example: get_jobs_by_dimension(dimension="repository", metric="duration")

Inputs: top_n, metric, end_date, dimension, +1 more.

get_jobs_dailyA

Get daily job metrics: counts, durations, and failure rates per day. Great for spotting trends. Defaults to last 7 days.

Inputs: end_date, start_date.

get_jobs_summaryA

Get aggregate job statistics for a period: total jobs, total duration, average duration, failure rate. The big picture view.

Inputs: end_date, start_date.

get_job_testsB

Get test results for a job. Optionally filter by status (pass/fail/skip).

Inputs: job_id (required), run_id (required), limit, status, +1 more.

get_log_filter_optionsA

Get available filter values for log search (e.g., job names, log levels). Use before search_logs to discover valid filter values.

Inputs: property (required), hours.

get_log_histogramA

Get log volume over time as a histogram. Useful for spotting error spikes or unusual activity patterns.

Inputs: hours, query.

get_monitorsC

Get alerting/monitoring rules configured for the organization. Shows active monitors and their timeline history.

Inputs: days, limit.

get_org_settingsA

Get all organization settings: email, alerts, timeouts, feature flags (Docker caching, SSH, branch protection, log ingestion), and PR comment config.

Inputs: no named parameters.

get_org_statusA

Get the status of the current organization (personal org, onboarding, runner region).

Inputs: no named parameters.

get_runA

Get details of a specific workflow run by ID. Includes list of jobs.

Inputs: run_id (required).

get_run_filter_optionsA

Get available filter values for workflow runs: statuses, repositories, branches, workflows, and users. Use this to discover valid filter values before calling list_runs.

Inputs: no named parameters.

get_run_histogramB

Get workflow run duration distribution as a histogram. Shows how long runs typically take.

Inputs: end_date, start_date, bucket_count.

get_runner_typesA

Get runner types used for CI jobs in a period. Shows what machine types are being used.

Inputs: end_date, start_date.

get_run_testsA

Get test results for an entire run (all jobs). Simpler than calling get_job_tests per job. Truncates at 500 tests.

Inputs: run_id (required).

get_slow_testsB

Find tests exceeding a duration threshold. Shows slowest tests, their percentage of total test time, and average duration stats.

Inputs: job_id (required), run_id (required), limit, threshold_ms.

get_test_historyA

Get the failure history for a specific test across recent runs. Shows when it passed/failed, on which branches, and error messages for failures.

Inputs: job_name (required), test_name (required), limit, suite.

get_trendsA

Track metrics over time: duration (are tests getting slower?), failure_rate (are tests getting flakier?), test_count (are we adding tests?). Returns trend analysis with data points.

Inputs: metric (required), job_name (required), days, granularity.

get_usage_summaryA

Get usage summary showing billable minutes vs free tier allowance. Shows remaining free minutes and overage.

Inputs: no named parameters.

list_jobsA

List all jobs for a specific workflow run. Use this to get job IDs for get_job, get_job_logs, and get_job_tests.

Inputs: run_id (required).

list_orgsA

List all Blacksmith organizations accessible to your account. Use this first to see available orgs.

Inputs: no named parameters.

list_runsA

List workflow runs with filtering. Filter by status (success/failure/cancelled/skipped/in_progress), branch, workflow name, actor, or PR number. Example: list_runs(status="failure") to find failed runs.

Inputs: actor, limit, branch, status, +4 more.

search_logsA

Search logs across all jobs. Filter by query (e.g., "error", "timeout"), log level (INFO/WARN/ERROR/DEBUG), and time range. Great for finding issues across runs.

Inputs: hours, level, limit, query.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

B3.3/5.0

Scored 2026-07-13 03:15 across 37 tools

Disambiguation4/5

Most tools have distinct purposes (runs, jobs, tests, usage, logs, etc.), but there are several test-related tools (get_failed_tests, get_failures_by_pattern, get_flaky_tests, get_slow_tests, get_test_history) that could cause confusion despite detailed descriptions.

Naming Consistency3/5

The predominant pattern is 'get_*', but there are exceptions like 'list_jobs', 'list_orgs', 'list_runs', 'compare_test_runs', and 'search_logs', mixing verbs and styles.

Tool Count2/5

With 37 tools, the server is overloaded for a CI analytics domain. While each tool has a clear purpose, the sheer number can overwhelm agents and suggests insufficient consolidation.

Completeness3/5

The tool surface covers monitoring, debugging, and analytics (runs, jobs, tests, logs, cache, usage) well, but lacks management actions like triggering runs or updating settings, leaving the domain partially incomplete.

Maintenance

ActivityInactive

ResponsivenessNo recent community issues

Resources

Related MCP Connectors

Rudow
io.github.prenzio
Query your app's logs, metrics and incidents from your AI assistant.
11
https://mcp.rudow.ai/mcp
Cypress Cloud MCPB
io.cypress.mcp
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Testing & QA Tools CI/CD & DevOps
24
https://mcp.cypress.io/mcp
mcpulseA
com.getmcpulse
Analytics for MCP servers. Query your tool calls, first-call success, retries and schema cost.
Data Visualization Monitoring Observability
1
https://api.getmcpulse.com/mcp
convalyticsA
io.github.Dan-Cleary
Read-only analytics for Convex apps, queryable via MCP from Claude, Cursor, and other clients.
Developer Tools Monitoring Databases
1526
https://api.convalytics.dev/mcp

Related MCP Servers

PyTorch HUD MCP Server
CI/CD & DevOps Monitoring
izaitsevfb
F
license
Not graded
quality
D
maintenance
Provides access to PyTorch CI/CD analytics data including workflows, jobs, test runs, and log analysis through an MCP interface.
Updated 3 months ago (2026-07-29 20:48 UTC)
4
-
Langfuse MCP Server
Monitoring Observability RAG Systems
therealsachin
A
license
B
quality
D
maintenance
Enables querying Langfuse analytics, cost metrics, and usage data across multiple projects. Provides tools for trace analysis, model/service cost breakdowns, and daily usage trends through natural language queries.
Updated a year ago (2025-11-09 15:23 UTC)
24
104 npm
MIT
Claude Stats MCP
Monitoring Developer Tools
upJiang
F
license
Not graded
quality
D
maintenance
Provides statistical analysis and usage tracking for Claude Code, enabling users to query daily/monthly stats, compare users, analyze trends, detect anomalies, and generate usage reports through natural language.
Updated a year ago (2025-12-22 13:31 UTC)
1
-
rybbit-mcp
Monitoring Search
nks-hub
A
license
B
quality
B
maintenance
Enables querying Rybbit Analytics data directly through MCP-compatible clients like Claude Code. It provides tools for monitoring website statistics, user sessions, error logs, funnels, and performance metrics via natural language.
Updated a month ago (2026-10-10 09:09 UTC)
43
14 npm
4
MIT

Test	Flake Rate	Pattern
`should handle concurrent requests`	40% (4/10 runs)	Fails on CI, passes locally
`renders loading state`	20% (2/10 runs)	Race condition with setTimeout
`validates auth token`	30% (3/10 runs)	Intermittent network timeout

Blacksmith MCP

Why?

Quick Start

Installation

Option 1: Claude Code CLI

Option 2: Project Configuration

Option 3: Global Install

Configuration

Authentication

Environment Variables

Available Tools

Workflow Runs

Test Analytics

Usage & Billing

Organization

Example Conversations

Debug a Failed Run

Find Flaky Tests

Compare Branches

Development

Troubleshooting

Session Expired

Cookie Extraction Failed

No Organization Set

API Notes

License

Contributing

Available Tools

Tool Schema Changelog

TDQS

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

PyTorch HUD MCP Server

Langfuse MCP Server

Claude Stats MCP

rybbit-mcp