How do I use Scorecard MCP Server?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Scorecard MCP Server Analyze the results of my latest model evaluation experiment" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

Scorecard MCP Server on Cloudflare

This repository allows you to deploy a remote MCP server on Cloudflare Workers that enables Claude and other MCP clients to access Scorecard's evaluation tools.

Get started:

Deploy to Workers

This will deploy your Scorecard MCP server to a URL like: scorecard-mcp.<your-account>.workers.dev/sse

Alternatively, you can clone this repository and deploy it using Wrangler:

git clone https://github.com/scorecard-ai/scorecard-mcp.git cd scorecard-mcp npm install npm run deploy

About This MCP Server

This MCP server provides access to Scorecard's evaluation tools directly from Claude and other MCP-compatible clients. It uses Clerk for authentication and is built on Cloudflare Workers for reliable, global deployment.

The server implements the MCP specification (2025-03-26) and provides secure access to Scorecard's API for running experiments, generating synthetic data, configuring metrics, and analyzing model performance.

Connect to MCP Clients

This MCP server works with various MCP-compatible clients:

Connect to claude.ai, Cursor, and Windsurf

Once deployed, you can connect to your MCP server from Claude and other MCP-compatible clients by providing your server URL:

https://scorecard-mcp.<your-account>.workers.dev/sse

Connect via Cloudflare AI Playground

You can also connect through the Cloudflare AI Playground:

Go to https://playground.ai.cloudflare.com/
Enter your deployed MCP server URL (scorecard-mcp.<your-account>.workers.dev/sse)
You can now use Scorecard's evaluation tools directly from the playground!

Connect via Claude Desktop

For local testing, you can connect to your MCP server from Claude Desktop by using the mcp-remote proxy.

Follow Anthropic's Quickstart and within Claude Desktop go to Settings > Developer > Edit Config.

Update with this configuration:

{ "mcpServers": { "scorecard": { "command": "npx", "args": [ "mcp-remote", "https://scorecard-mcp.<your-account>.workers.dev/sse" // or http://localhost:8787/sse for local testing ] } } }

Restart Claude and you should see the tools become available.

Local Development

For local development, create a ".dev.vars" file with your Clerk credentials:

cp .dev.vars.example .dev.vars

Configure the following variables in your .dev.vars file:

Variable	Source	Notes
CLERK_CLIENT_ID	Clerk Dashboard -> Configure -> OAuth Applications
CLERK_CLIENT_SECRET	Clerk Dashboard -> Configure -> OAuth Applications	Cannot be viewed after initial generation
CLERK_DOMAIN	Clerk Dashboard -> Configure -> API Keys -> Frontend API URL	Override this with the Clerk development URL if using with local Scorecard server
CLERK_PUBLISHABLE_KEY	Clerk Dashboard -> Configure -> API Keys -> Publishable Key	Override this with the pk_test_* one if using with local Scorecard server