mcp-testbed
Allows running Jest test suites and parsing results to provide pass/fail verdicts and failure details.
Allows running Mocha test suites and parsing results to provide pass/fail verdicts and failure details.
Allows running Vitest test suites and parsing results to provide pass/fail verdicts and failure details.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-testbed Run the tests for this candidate solution and give me a pass/fail verdict."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-testbed
An MCP server that gives an agent an isolated workspace, runs its tests, and returns a deterministic pass/fail verdict.
The use case is grading. If you are evaluating whether a model can solve a software engineering problem, you need to put its candidate solution somewhere safe, run a suite against it, and get back an answer a program can act on. Doing that naively produces results that are almost right, which is the worst kind: output littered with absolute paths, wall clock durations and colour codes, so two runs of identical code never match, and any comparison against a reference solution fails for reasons that have nothing to do with the code.
This server exists to make that last part go away.
create_workspace -> write_files -> run_tests -> read_workspace -> destroy_workspaceQuick start
npm install
npm test # 50 tests
npm run buildRegister it with any MCP host. For Claude Desktop or Claude Code:
{
"mcpServers": {
"testbed": {
"command": "node",
"args": ["/absolute/path/to/mcp-testbed/dist/index.js"]
}
}
}Related MCP server: Soma MCP Server
What a run looks like
const { workspaceId } = (await client.callTool({
name: 'create_workspace',
arguments: {
files: {
'package.json': JSON.stringify({ name: 'candidate', type: 'module' }),
'add.js': 'export const add = (a, b) => a * b;', // wrong on purpose
'add.test.js': `
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { add } from './add.js';
test('adds two numbers', () => assert.equal(add(2, 3), 5));
`,
},
},
})).structuredContent;
const result = (await client.callTool({
name: 'run_tests',
arguments: { workspaceId, command: 'node', args: ['--test'] },
})).structuredContent;{
"passed": false,
"reporter": "node-test",
"total": 1,
"passedCount": 0,
"failedCount": 1,
"failures": [{ "name": "adds two numbers" }],
"exitCode": 1,
"timedOut": false,
"truncated": false,
"stdout": "...normalized...",
"rawStdout": "...untouched..."
}Determinism
This is the part worth reading. Four sources of run to run variation are removed from stdout and stderr, while rawStdout and rawStderr keep the untouched output for a human to debug with.
Source of noise | Handling |
Absolute paths | Workspace root becomes |
Durations |
|
Timestamps | ISO 8601 becomes |
Colour codes | ANSI sequences stripped before anything else, since an escape can sit mid-token and stop other patterns matching. |
The environment is pinned rather than inherited. TZ=UTC, LC_ALL=C, a fixed PATH, HOME pointed at the workspace, and package manager banners suppressed. Inheriting the parent environment is the most common reason a suite passes on one machine and fails on another, so the parent environment is dropped entirely.
Directory listings are sorted, because readdir order is filesystem dependent.
The end to end test asserts the actual property: the same failing test in two different workspaces produces byte identical output.
Normalization is deliberately narrow. Over-aggressive scrubbing is worse than none, because it can erase the difference between a passing run and a failing one. There is a test asserting that a pass and a fail do not normalize to the same string.
Verdicts
run_tests recognises node --test, TAP, Jest, Vitest and Mocha, and reports which parser matched. When none matches it returns reporter: "exit-code" with null counts rather than guessing. A wrong count is worse than an absent one, since a grader would treat it as ground truth.
Two rules override a clean-looking summary:
A non-zero exit code is a failure even when every test reported passing. A suite can print all green and still exit non-zero because teardown crashed.
A run killed on timeout is never a pass, whatever the partial output said.
Isolation
Workspaces are addressed by opaque UUID. A caller never sees or supplies an absolute path.
Path containment resolves the target and checks it against a separator terminated prefix. A plain startsWith(root) would accept /tmp/ws-evil for a root of /tmp/ws, and there is a test for exactly that case. Absolute paths, .. traversal and null bytes are rejected, as are files that would change how a run behaves (.npmrc, .git). Writes are validated as a batch before any of them touch disk, so a bad path halfway through cannot leave a partial write behind.
Runs are bounded on both axes: a wall clock timeout that kills the whole process group (killing only the direct child leaves orphaned test workers alive), and a per stream output cap so a runaway console.log cannot exhaust memory.
Threat model. This is workspace isolation, not a security sandbox. run_tests executes a command you hand it, with your user's permissions and network access. It protects a grading run from the mess an agent makes, not a host from hostile code. If you are running genuinely untrusted code, put a container or VM boundary around this process.
Tools
Tool | Purpose |
| Creates an isolated temp directory, optionally seeded with files. Returns its id. |
| Writes or overwrites files. All or nothing. |
| Runs a command under a fixed environment and timeout. Returns a structured verdict plus normalized and raw output. |
| Lists files, or reads one. Listings are sorted and exclude |
| Deletes a workspace. Idempotent. |
Tool descriptions are written for the model that will call them. A model decides whether to reach for a tool almost entirely from its description, so each states what it does, what it returns, and when not to use it.
Layout
src/normalize.ts output normalization
src/workspace.ts workspace lifecycle and path containment
src/runner.ts bounded process execution
src/parsers.ts test reporter parsing
src/server.ts MCP tool surface
src/index.ts stdio entry pointTests drive the server through a linked in-memory transport rather than calling handlers directly, so schema validation, serialization and tool registration are all exercised the way a real host exercises them. A unit test that called the handler would pass even if the tool were never registered.
Requirements
Node 20 or newer. Runtime dependencies are @modelcontextprotocol/server and zod, nothing else.
License
MIT
Available Tools
5 toolscreate_workspaceCreate workspaceA
Creates an isolated temporary directory and returns its id. Optionally seeds it with files. Every other tool addresses a workspace by this id; paths are always relative to it. Call destroy_workspace when finished.
Inputs: files. Returns: workspaceId, filesWritten
destroy_workspaceDestroy workspaceADestructiveIdempotent
Deletes a workspace and everything in it. Safe to call on an id that no longer exists.
Inputs: workspaceId (required). Returns: destroyed
read_workspaceRead workspaceARead-onlyIdempotent
Lists the files in a workspace, or returns the contents of one file when path is given. node_modules and .git are excluded from listings. Listings are sorted, so repeated calls match.
Inputs: workspaceId (required), path. Returns: path, files, contents
run_testsRun testsAIdempotent
Runs a command in a workspace under a fixed environment and a wall clock timeout, then parses the output into a pass/fail summary with failing test names. Output is normalized (paths, durations, timestamps and colour codes removed) so two identical runs produce identical text. Returns the verdict plus both normalized and raw output.
Inputs: workspaceId (required), env, args, command, +2 more. Returns: total, passed, stderr, stdout, +10 more
write_filesWrite filesADestructiveIdempotent
Writes or overwrites files in an existing workspace, creating parent directories as needed. Paths must be relative and stay inside the workspace. Either every file is written or none is.
Inputs: files (required), workspaceId (required). Returns: filesWritten
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Each tool has a clearly distinct role: create workspace, read/list files, write files, run tests, and destroy workspace. There is no overlap in purpose, so an agent can easily select the right tool.
All tool names follow a consistent verb_noun pattern (create_workspace, read_workspace, write_files, run_tests, destroy_workspace), using snake_case throughout. The naming is predictable and intuitive.
Five tools is appropriate for the scope of managing isolated test workspaces and running tests. Each tool serves a necessary function without redundancy or bloat.
The set covers the full lifecycle of a test workspace: creation, reading, writing, test execution, and destruction. No essential operations are missing; write_files handles overwrites, and read_workspace supports both listing and file content retrieval.
Maintenance
Related MCP Connectors
Execute code in 8 languages (Python, JS, TS, Go, Java, C++, C, Bash) in gVisor sandboxes.
Hosted Python sandbox for package runtime verification: pip install and usable-import evidence.
Run Python, Java, C, C++, C#, Go, Rust and 39 more languages, each run in a fresh, offline sandbox.
Execution platform for AI agents on Firecracker microVMs in the EU. Interactive Linux sandboxes with root, templates for Python, Node.js, Go, Rust and a headless browser, public HTTPS URLs. Background jobs that outlive the connection, batches of up to 1,000 jobs with dependencies, hard cost caps reserved up front, execution receipts with SHA-256, log digests that fit the context. IPv4, dual-stack or IPv6-only with its own public address, allowlists or no network. WASI runtime for small tasks. Idle sandboxes sleep and keep their files. Also REST API, Python SDK and LangChain.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides secure access to containerized build environments for software projects, enabling AI assistants to execute builds, run tests, manage git operations, and inspect build artifacts without requiring local installation of dependencies.6MIT
- AlicenseAqualityDmaintenanceEnables running code against tests in an isolated sandbox to obtain PASS/FAIL verdicts with signed, offline-checkable certificates, and generating verified code with attached certificates after execution against derived tests.238 npmMIT
- AlicenseAqualityDmaintenanceProvides structured, sandboxed test and lint feedback for coding agents, returning compact typed verdicts with failure fingerprints instead of raw runner output. It enables impact-selected test execution in isolated containers, distinguishing pre-existing failures from regressions.417 PyPIApache 2.0
- AlicenseAqualityCmaintenanceEnables coding agents to compile supplied files, run optional focused tests, and get static findings across ten programming languages, providing a deterministic feedback loop on the code they are about to return.1MIT