Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@blacksmith-mcpShow me the failed tests from the latest workflow run"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Blacksmith MCP
An MCP server that connects Claude to your Blacksmith CI data. Query workflow runs, analyze test failures, detect flaky tests, and monitor usage—all through natural conversation.
Why?
Debugging CI failures usually means clicking through dashboards, copying run IDs, and piecing together information across multiple pages. With this MCP, you can just ask:
"Why did the last CI run fail?"
"Which tests are flaky this week?"
"Compare test failures between main and my PR"
"What's using the most cache storage?"
Claude handles the API calls and gives you actionable insights.
Quick Start
Zero-config if you're logged into Blacksmith in Chrome:
The MCP automatically extracts your session from Chrome cookies. No manual token copying needed.
Installation
Option 1: Claude Code CLI
Option 2: Project Configuration
Add to your .mcp.json:
Option 3: Global Install
Configuration
Authentication
Automatic (recommended): Log into app.blacksmith.sh in Chrome. The MCP extracts your session cookie automatically.
Manual: Set BLACKSMITH_SESSION_COOKIE environment variable with your session cookie value.
Environment Variables
Variable | Required | Description |
| Yes | Your Blacksmith organization name |
| No | Session cookie (auto-extracted from Chrome if not set) |
Available Tools
Workflow Runs
Tool | Description |
| List workflow runs with filters (status, branch, workflow, actor, PR) |
| Get run details including all jobs |
| List jobs for a workflow run |
| Get job details (steps, timing, runner info) |
| Get raw log output for a job |
Test Analytics
Tool | Description |
| Get all test results for a job |
| Get failed tests with full error messages |
| Group failures by error pattern (e.g., "Cannot read properties") |
| Compare failures between two runs (find regressions) |
| Detect tests that fail intermittently |
| Find tests exceeding duration threshold |
| Track a specific test's pass/fail history |
| Analyze trends: duration, failure rate, test count |
Usage & Billing
Tool | Description |
| Current core usage (active vs max) |
| Current billing period amount |
| Billable minutes vs free tier |
| Cache size, entries by repository |
| Detailed cache entries for a repo |
Organization
Tool | Description |
| List accessible organizations |
| Org status (personal, onboarded, region) |
| Search logs across all jobs |
Example Conversations
Debug a Failed Run
You: Why did the last CI run fail?
Claude: Uses
The last run failed with 34 test failures. The main issues are:
Connection timeout (12 tests) - Tests in
api/are timing out connecting to the test databaseUndefined property (8 tests) -
user.settings.themeis undefined in the new dark mode codeAssertion failures (14 tests) - Snapshot mismatches in UI components
Find Flaky Tests
You: Which tests are flaky?
Claude: Uses
Found 3 flaky tests:
Test
Flake Rate
Pattern
should handle concurrent requests40% (4/10 runs)
Fails on CI, passes locally
renders loading state20% (2/10 runs)
Race condition with setTimeout
validates auth token30% (3/10 runs)
Intermittent network timeout
Compare Branches
You: Did my PR introduce new test failures?
Claude: Uses
Your PR introduced 2 new failures:
UserProfile.test.tsx: "Cannot read property 'avatar' of undefined"
api/users.test.ts: Expected 200, got 4045 previously failing tests now pass (nice!).
Development
Troubleshooting
Session Expired
If you see SESSION_EXPIRED, your Blacksmith session has expired. Simply log back into app.blacksmith.sh in Chrome and retry.
Cookie Extraction Failed
The automatic cookie extraction requires:
macOS with Chrome installed
Being logged into Blacksmith in Chrome
Chrome not running with a locked profile
If it fails, set BLACKSMITH_SESSION_COOKIE manually.
No Organization Set
Run list_orgs to see available organizations, then set BLACKSMITH_ORG to your org name.
API Notes
This MCP uses Blacksmith's internal web API, which is undocumented. The API was reverse-engineered from the Blacksmith web app and may change without notice.
License
MIT
Contributing
Contributions welcome! Please open an issue first to discuss proposed changes.