Skip to main content
Glama

codabench-mcp

MCP server for the Codabench REST API. Lets an AI agent drive a full participant ML-benchmark workflow: discover competitions, read rules, download data, submit, poll, read leaderboards.

Features

Curated tools for the participant path:

  • search_competitions, get_competition, list_competition_phases, get_phase, get_competition_rules, list_competition_tasks, get_task

  • download_dataset (streaming + SHA-256)

  • list_my_submissions, get_submission, get_submission_logs

  • submit_to_phase (handles Codabench's 3-step upload flow)

  • poll_submission (backoff + timeout + non-error "still_running")

  • get_leaderboard, get_my_profile

  • codabench_request — generic REST escape hatch (GET-only by default)

Related MCP server: kaggle-mcp

Authentication

You need a Codabench API token. Two ways to get one:

  1. Visit https://www.codabench.org/api/docs/

  2. Find the api-token-auth endpoint → Try it out

  3. POST your username/password

  4. Copy the token value from the response

2. curl

curl -X POST https://www.codabench.org/api/api-token-auth/ \
  -H "Content-Type: application/json" \
  -d '{"username":"YOUR_USERNAME","password":"YOUR_PASSWORD"}'

The response is {"token": "..."}.

Install

uvx codabench-mcp

To run the bleeding-edge main branch instead of the last release:

uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcp

Or for development:

git clone https://github.com/harshitAgr/codabench-mcp.git
cd codabench-mcp
uv sync
uv run codabench-mcp  # requires CODABENCH_API_TOKEN

Configure an MCP client

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "codabench": {
      "command": "uvx",
      "args": ["codabench-mcp"],
      "env": {
        "CODABENCH_API_TOKEN": "paste-your-token-here"
      }
    }
  }
}

Claude Code (plugin marketplace)

/plugin marketplace add harshitAgr/codabench-mcp
/plugin install codabench-mcp@codabench-mcp

Then export your token in the shell where you launch Claude Code:

export CODABENCH_API_TOKEN=paste-your-token-here

Claude Code (manual)

If you'd rather skip the plugin layer:

claude mcp add codabench \
  --env CODABENCH_API_TOKEN=paste-your-token-here \
  -- uvx codabench-mcp

Environment variables

Variable

Required

Default

Purpose

CODABENCH_API_TOKEN

yes

—

DRF token, sent as Authorization: Token <token>

CODABENCH_BASE_URL

no

https://www.codabench.org

Override for tests

CODABENCH_ALLOW_WRITE_RAW

no

0

Set to 1 to allow non-GET methods through codabench_request

CODABENCH_MAX_DOWNLOAD_BYTES

no

5368709120 (5 GB)

Cap for download_dataset

Development

uv sync
uv run pytest                              # unit tests, no network
uv run ruff check .                        # lint
uv run ruff format --check .               # format check

License

MIT — see LICENSE.

mcp-name: io.github.harshitAgr/codabench-mcp

Available Tools

16 tools
codabench_requestA

Generic Codabench REST escape hatch. Use when no curated tool fits. Path must start with '/api/'. By default only GET/HEAD/OPTIONS are allowed; start the server with CODABENCH_ALLOW_WRITE_RAW=1 to permit POST/PUT/PATCH/DELETE.

Inputs: method (required), path (required), params, body. Returns: object

download_datasetA

Download a Codabench dataset/bundle to a local absolute path. Streams directly to disk and returns the SHA-256. Refuses to overwrite unless overwrite=True.

Inputs: key (required), dest_path (required), overwrite. Returns: object

get_competitionB

Get full details of a single Codabench competition.

Inputs: competition_id (required). Returns: object

get_competition_rulesA

Get a competition's rules, terms, evaluation criteria, and any other descriptive pages concatenated into a single markdown string.

Inputs: competition_id (required). Returns: object

get_leaderboardC

Get the leaderboard for a phase.

Inputs: phase_id (required). Returns: object

get_my_profileA

Get the authenticated user's Codabench profile.

Inputs: no named parameters. Returns: object

get_phaseC

Get full details of a single phase.

Inputs: phase_id (required). Returns: object

get_submissionC

Get full details of a submission including scores.

Inputs: submission_id (required). Returns: object

get_submission_logsC

Get stdout/stderr/ingestion logs for a submission.

Inputs: submission_id (required). Returns: object

get_taskB

Get full details of a single task, including dataset keys.

Inputs: task_id (required). Returns: object

list_competition_phasesC

List all phases of a Codabench competition.

Inputs: competition_id (required), max_pages. Returns: object

list_competition_tasksC

List all tasks attached to a Codabench competition.

Inputs: competition_id (required), max_pages. Returns: object

list_my_submissionsB

List the authenticated user's submissions, newest first.

Inputs: phase, page_size, max_pages. Returns: object

poll_submissionA

Poll a submission until it reaches a terminal state (finished/failed/cancelled) or timeout_seconds elapses. Returns the final submission record, or {status: 'still_running'} on timeout — the agent can choose to poll again.

Inputs: submission_id (required), timeout_seconds, interval_seconds. Returns: object

search_competitionsC

Search public Codabench competitions. Auto-paginates up to max_pages.

Inputs: q, public, page_size, max_pages. Returns: object

submit_to_phaseB

Submit a zipped model bundle to a competition phase. Performs Codabench's 3-step upload: POST /api/data/, PUT to signed URL, PUT /api/data/upload_completed/, POST /api/submissions/.

Inputs: phase_id (required), bundle_path (required), description, task_ids. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 16 tools

    Disambiguation5/5

    Each tool targets a distinct resource or action (competitions, phases, tasks, submissions, datasets, profile), and the generic escape hatch is clearly marked for edge cases. No two tools overlap in purpose.

    Naming Consistency4/5

    Most tools follow a consistent verb_noun pattern (e.g., 'get_competition', 'list_phase', 'submit_to_phase'). The only outlier is 'codabench_request', which uses a different convention, but it's a single exception.

    Tool Count5/5

    16 tools are well-scoped for a Codabench API wrapper, covering all common participant operations without being excessive. Each tool addresses a specific need.

    Completeness4/5

    The tool set covers the full participant workflow (browsing competitions, submitting, polling, downloading datasets, checking leaderboards) and includes a generic escape hatch for missing operations. Minor administrative actions (e.g., managing teams) are absent but not core to the domain.

    Related MCP Connectors

    Related MCP Servers