codabench-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@codabench-mcp search for competitions about natural language processing"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
codabench-mcp
MCP server for the Codabench REST API. Lets an AI agent drive a full participant ML-benchmark workflow: discover competitions, read rules, download data, submit, poll, read leaderboards.
Features
Curated tools for the participant path:
search_competitions,get_competition,list_competition_phases,get_phase,get_competition_rules,list_competition_tasks,get_taskdownload_dataset(streaming + SHA-256)list_my_submissions,get_submission,get_submission_logssubmit_to_phase(handles Codabench's 3-step upload flow)poll_submission(backoff + timeout + non-error "still_running")get_leaderboard,get_my_profilecodabench_request— generic REST escape hatch (GET-only by default)
Related MCP server: kaggle-mcp
Authentication
You need a Codabench API token. Two ways to get one:
1. Swagger UI (recommended)
Find the
api-token-authendpoint → Try it outPOST your username/password
Copy the
tokenvalue from the response
2. curl
curl -X POST https://www.codabench.org/api/api-token-auth/ \
-H "Content-Type: application/json" \
-d '{"username":"YOUR_USERNAME","password":"YOUR_PASSWORD"}'The response is {"token": "..."}.
Install
uvx codabench-mcpTo run the bleeding-edge main branch instead of the last release:
uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcpOr for development:
git clone https://github.com/harshitAgr/codabench-mcp.git
cd codabench-mcp
uv sync
uv run codabench-mcp # requires CODABENCH_API_TOKENConfigure an MCP client
Claude Desktop (claude_desktop_config.json)
{
"mcpServers": {
"codabench": {
"command": "uvx",
"args": ["codabench-mcp"],
"env": {
"CODABENCH_API_TOKEN": "paste-your-token-here"
}
}
}
}Claude Code (plugin marketplace)
/plugin marketplace add harshitAgr/codabench-mcp
/plugin install codabench-mcp@codabench-mcpThen export your token in the shell where you launch Claude Code:
export CODABENCH_API_TOKEN=paste-your-token-hereClaude Code (manual)
If you'd rather skip the plugin layer:
claude mcp add codabench \
--env CODABENCH_API_TOKEN=paste-your-token-here \
-- uvx codabench-mcpEnvironment variables
Variable | Required | Default | Purpose |
| yes | — | DRF token, sent as |
| no |
| Override for tests |
| no |
| Set to |
| no |
| Cap for |
Development
uv sync
uv run pytest # unit tests, no network
uv run ruff check . # lint
uv run ruff format --check . # format checkLicense
MIT — see LICENSE.
mcp-name: io.github.harshitAgr/codabench-mcp
Available Tools
16 toolscodabench_requestA
Generic Codabench REST escape hatch. Use when no curated tool fits. Path must start with '/api/'. By default only GET/HEAD/OPTIONS are allowed; start the server with CODABENCH_ALLOW_WRITE_RAW=1 to permit POST/PUT/PATCH/DELETE.
Inputs: method (required), path (required), params, body. Returns: object
download_datasetA
Download a Codabench dataset/bundle to a local absolute path. Streams directly to disk and returns the SHA-256. Refuses to overwrite unless overwrite=True.
Inputs: key (required), dest_path (required), overwrite. Returns: object
get_competitionB
Get full details of a single Codabench competition.
Inputs: competition_id (required). Returns: object
get_competition_rulesA
Get a competition's rules, terms, evaluation criteria, and any other descriptive pages concatenated into a single markdown string.
Inputs: competition_id (required). Returns: object
get_leaderboardC
Get the leaderboard for a phase.
Inputs: phase_id (required). Returns: object
get_my_profileA
Get the authenticated user's Codabench profile.
Inputs: no named parameters. Returns: object
get_phaseC
Get full details of a single phase.
Inputs: phase_id (required). Returns: object
get_submissionC
Get full details of a submission including scores.
Inputs: submission_id (required). Returns: object
get_submission_logsC
Get stdout/stderr/ingestion logs for a submission.
Inputs: submission_id (required). Returns: object
get_taskB
Get full details of a single task, including dataset keys.
Inputs: task_id (required). Returns: object
list_competition_phasesC
List all phases of a Codabench competition.
Inputs: competition_id (required), max_pages. Returns: object
list_competition_tasksC
List all tasks attached to a Codabench competition.
Inputs: competition_id (required), max_pages. Returns: object
list_my_submissionsB
List the authenticated user's submissions, newest first.
Inputs: phase, page_size, max_pages. Returns: object
poll_submissionA
Poll a submission until it reaches a terminal state (finished/failed/cancelled) or timeout_seconds elapses. Returns the final submission record, or {status: 'still_running'} on timeout — the agent can choose to poll again.
Inputs: submission_id (required), timeout_seconds, interval_seconds. Returns: object
search_competitionsC
Search public Codabench competitions. Auto-paginates up to max_pages.
Inputs: q, public, page_size, max_pages. Returns: object
submit_to_phaseB
Submit a zipped model bundle to a competition phase. Performs Codabench's 3-step upload: POST /api/data/, PUT to signed URL, PUT /api/data/upload_completed/, POST /api/submissions/.
Inputs: phase_id (required), bundle_path (required), description, task_ids. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 16 tools
Each tool targets a distinct resource or action (competitions, phases, tasks, submissions, datasets, profile), and the generic escape hatch is clearly marked for edge cases. No two tools overlap in purpose.
Most tools follow a consistent verb_noun pattern (e.g., 'get_competition', 'list_phase', 'submit_to_phase'). The only outlier is 'codabench_request', which uses a different convention, but it's a single exception.
16 tools are well-scoped for a Codabench API wrapper, covering all common participant operations without being excessive. Each tool addresses a specific need.
The tool set covers the full participant workflow (browsing competitions, submitting, polling, downloading datasets, checking leaderboards) and includes a generic escape hatch for missing operations. Minor administrative actions (e.g., managing teams) are absent but not core to the domain.
Related MCP Connectors
MCP server exposing the Backtest360 engine API as tools for AI agents.
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP Server for an Agent Task Marketplace
MCP server that lets AI assistants use all OneSchema features exposed via the public API.
Related MCP Servers
AlicenseAqualityBmaintenanceMCP server that gives AI coding agents direct access to evaluation tools.23Apache 2.0- AlicenseBqualityDmaintenanceA full-featured MCP server for the Kaggle API — competitions, datasets, kernels, models, benchmarks, and discussions.5188 PyPI4MIT

AgentMark MCP Serverofficial
AlicenseNot gradedqualityAmaintenanceMCP server for AgentMark. Exposes the full AgentMark API to AI editors: list traces and spans, manage datasets, write scores, run experiments, manage apps, deployments, environments, and annotation queues.352AGPL 3.0- AlicenseBqualityDmaintenanceA local MCP server that exposes the SAIR Competition Public API as tools for MCP clients, enabling account introspection, competition browsing, submissions, playground operations, cheatsheet management, and Contributor Network interactions.48MIT