Skip to main content
Glama

Datapoint MCP

Get real human opinions from inside any MCP client. Run surveys, A/B preference comparisons, ratings, and rankings on text, images, audio, and video β€” without leaving your editor.

MCP License: MIT Powered by Datapoint AI

Datapoint MCP is an MCP server that gives Claude, GPT, Gemini, and any other MCP-capable agent the ability to recruit real humans for evaluation tasks, then return aggregated results back into the conversation. Built on top of Datapoint AI.

Why

LLMs are great at generating options and bad at telling you which one a real person will prefer. Datapoint MCP closes that loop β€” your agent can hand off to a panel of real humans and pick up the results a few minutes later.

Related MCP server: GetABrain🧠 | Live Human-in-the-Loop MCP for AI Agents

Use cases

  • Design & UX β€” A/B test logos, landing pages, screens, ad creative, copy

  • AI evaluation β€” human ratings of model outputs, side-by-side comparisons, hallucination checks

  • Preference data β€” collect RLHF / DPO pairs at scale

  • Dataset labeling β€” classification, ranking, captioning, content moderation

  • Product research β€” quick concept tests, naming, pricing reads

  • Human-in-the-loop checks β€” gate an agent before it ships something irreversible

Tools

Tool

Description

setup

Authenticate with your Datapoint AI account (opens browser)

upload_media

Upload local images, audio, or video so they can be used in a survey

plan_survey

Design a survey from a natural language description

create_survey

Launch a survey from a plan

check_survey

Check status, progress, and aggregated results

get_survey_responses

Get raw per-annotator responses (paginated)

list_surveys

List all your surveys

pause_survey

Pause task serving for an active survey (in-flight responses keep arriving)

resume_survey

Resume task serving for a paused survey

cancel_survey

Permanently cancel a survey and refund unused reserved credits (irreversible)

check_balance

Check your account balance

add_credits

Open a checkout link to top up your account

Video uploads accept browser-compatible H.264 MP4/MOV or VP8/VP9 WebM files. The API validates the actual media bytes before storage; upload_media reports unsupported, unreadable, or temporarily unprobeable videos in its failed-files section.

Install

Requires uv on your PATH.

Claude Code

As a plugin (recommended):

/plugin marketplace add impel-intelligence/datapoint-mcp
/plugin install datapoint@datapoint

To pick up new versions: /plugin marketplace update datapoint then /plugin update datapoint@datapoint.

As a raw MCP server (in ~/.claude/settings.json):

{
  "mcpServers": {
    "datapoint": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json (~/Library/Application Support/Claude/ on macOS, %APPDATA%\Claude\ on Windows):

{
  "mcpServers": {
    "datapoint": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
    }
  }
}

Restart Claude Desktop, then ask it to run setup.

Cursor

Add to ~/.cursor/mcp.json (or via Settings β†’ MCP):

{
  "mcpServers": {
    "datapoint": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "datapoint": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
    }
  }
}

VS Code (GitHub Copilot Chat / agent mode)

Add to your workspace .vscode/mcp.json:

{
  "servers": {
    "datapoint": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
    }
  }
}

Any other MCP client

Run the binary over stdio:

uvx --from git+https://github.com/impel-intelligence/datapoint-mcp.git datapoint-mcp

Usage

Once installed, just ask:

"Survey 20 people: which logo do they prefer, A or B?"

"Get human ratings on these three model outputs β€” which sounds most natural?"

"Run a quick A/B test on these two landing-page headlines."

The agent calls plan_survey to design it, shows you the plan and cost, then calls create_survey to launch. Use check_survey to monitor progress and read aggregated results.

Run setup first to authenticate if you haven't already.

Chain surveys (multi-step flow)

Some surveys have dependent questions β€” the second only makes sense given a specific answer to the first. Describe it that way and Claude will plan a chain:

"Ask 20 listeners if they could understand the speaker in this clip. If yes, rate the audio quality 1–5. If not, skip the rating."

A chain ties 2–5 steps together into a single unit of annotator work: every step is served to the same annotator, in order, and a per-step skip_if rule can end the response early. Claude will show you the full chain structure (steps, any skip conditions, cost) and wait for your confirmation before calling create_survey.

The cost shown in plan_survey is the upper bound (every response answers every step); when skip_if rules fire, responses cost proportionally less.

Configuration

Environment variable

Description

DATAPOINT_API_KEY

API key (overrides saved config)

DATAPOINT_BASE_URL

API base URL (default: https://api.trydatapoint.com/data-labelling/v1)

How it compares

Datapoint MCP

Mechanical Turk

Prolific

UserTesting

Run from inside an AI agent / IDE

βœ…

❌

❌

❌

Designed for AI/LLM evaluation

βœ…

⚠️

⚠️

❌

Pay-as-you-go via API

βœ…

βœ…

βœ…

❌

Supports media (image / audio / video)

βœ…

βœ…

βœ…

βœ…

Minutes to first response

βœ…

⚠️

⚠️

❌

License

MIT β€” see LICENSE.

Available Tools

13 tools
add_creditsA

Open a checkout link to purchase Datapoint AI credits.

Returns a hosted checkout URL. The user completes payment in their browser; credits land on their account once payment confirms.

Args: product_id: Optional product identifier. Omit to use the default credit bundle configured on the server.

Inputs: product_id. Returns: result

cancel_surveyA

Permanently cancel a survey and settle its reserved credits.

⚠ Cancellation is irreversible. Once cancelled, the survey stops serving new tasks, any in-flight responses still complete, and all unused reserved credits are returned to the balance. Already-consumed credit (for responses already collected) is NOT refunded β€” the response is shown in the result so the user knows the final cost.

Before calling, show the user the survey id and confirm they want to cancel. Prefer pause_survey when the user just wants to stop temporarily.

Backend rejects with 400 if the survey is already in a terminal state (completed, failed, or cancelled) β€” the message will say which.

Args: job_id: The job ID returned by create_survey.

Inputs: job_id (required). Returns: result

check_balanceA

Check your Datapoint AI account balance.

Inputs: no named parameters. Returns: result

check_surveyA

Check the status, progress, and results of a survey.

Args: job_id: The job ID returned by create_survey.

Inputs: job_id (required). Returns: result

create_surveyA

Create a survey from a plan generated by plan_survey.

⚠ Only call this after the user has explicitly confirmed the plan summary and cost from plan_survey. This reserves credits immediately and dispatches paid human annotation work β€” there is no draft or staging state, and it cannot be undone.

Pass the plan dict as returned by plan_survey. You may edit it first if anything is off β€” this is a regular Python dict.

Supports both standalone plans and chain plans (with a top-level steps array alongside datapoints). For chain plans, the backend dispatches each step of each datapoint as a linked task; the full sequence is served together, in order, to one annotator per response.

MEDIA VALIDATION: every media entry must use either:

  • a dp://media/… reference returned by upload_media, or

  • a public https:// URL the annotator's browser can reach.

Media lives at datapoints[*].media (shared across steps) OR datapoints[*].media_per_step["0"], ["1"], ... (one entry per step when steps need different media shapes). The two are mutually exclusive per datapoint.

Local paths, file:// URLs, and private/auth-gated URLs will be rejected or served broken to annotators. If the plan came back with wrong refs (e.g. the planner invented a URL because the description didn't supply one), fix the refs before calling this:

# shared media
plan["datapoints"][0]["media"]["candidates"][0]["url"] = "dp://media/real.png"
# per-step media (chain)
plan["datapoints"][0]["media_per_step"]["1"]["subject"][0]["url"] = "dp://media/real.mp3"

Args: plan: The survey plan dict (typically from plan_survey). Contains name, summary, datapoints, and either task_type + instruction (standalone) or a steps array (chain).

Inputs: plan (required). Returns: result

get_survey_responsesA

Get the raw per-annotator responses for a survey.

check_survey returns aggregated results (consensus, votes, mean/median, ranks). Use this tool when you want to see individual responses from each annotator β€” useful for spotting outliers, seeing the spread of opinion, or understanding disagreement.

The default view excludes answered rows from chains the annotator abandoned mid-flow or that are still in progress, matching what counts toward your survey's response total. Set the include flags to surface them.

Args: job_id: The job ID returned by create_survey. page: Page number (default 1). per_page: Responses per page (default 100, max 200). include_abandoned: Include answered rows from abandoned chains. include_in_progress: Include answered rows from in-flight chains.

Inputs: job_id (required), page, per_page, include_abandoned, +1 more. Returns: result

list_surveysB

List all your surveys (active and recent).

Inputs: no named parameters. Returns: result

pause_surveyA

Pause task serving for an active survey.

In-flight responses keep arriving; new tasks stop being served. The backend rejects with 400 if the survey is completed, failed, or already paused β€” the message will say which.

Args: job_id: The job ID returned by create_survey.

Inputs: job_id (required). Returns: result

plan_surveyA

Plan a survey from a natural language description.

Describe what you want to learn and from whom. The Datapoint AI service will design an effective survey structure for you.

MEDIA: If the survey compares/rates media (images, audio, video), first call upload_media on any local files to get dp:// refs, then mention those refs (or public https:// URLs) directly in this description so the planner can wire them into the datapoints. Example:

description = (
    "Compare two logo designs for memorability: "
    "A = dp://media/abc123.png, B = dp://media/def456.png. "
    "Target: general software developers."
)

Without explicit refs in the description, the planner will produce placeholder or invented URLs that will fail at create_survey.

After this returns, present the summary and cost to the user and wait for explicit confirmation before calling create_survey. Never chain these two calls β€” create_survey spends money and dispatches real work.

Args: description: What you want to survey, in plain language. Include the target audience, what you're comparing/rating, any screening criteria β€” including who should answer (e.g. respondents in specific countries, excluding VPN/bot traffic, balanced regional mix) β€” and, for media surveys, the dp:// or https:// URLs to use. max_responses: Number of human responses per datapoint (default 10). More = higher confidence but higher cost.

Inputs: description (required), max_responses. Returns: result

resume_surveyA

Resume task serving for a paused survey.

Backend rejects with 400 if the survey is not paused.

Args: job_id: The job ID returned by create_survey.

Inputs: job_id (required). Returns: result

retry_failed_datapointsA

Re-queue failed datapoints on a survey.

Each retried datapoint reserves credit again, the same way the original submission did β€” only call this when the failures are worth recovering.

Args: job_id: The job ID returned by create_survey. datapoint_indices: Specific failed indices to retry. Omit to retry every failed datapoint in the survey.

Inputs: job_id (required), datapoint_indices. Returns: result

setupA

Authenticate with Datapoint AI.

Opens your browser to sign in with Google and set up your account. Your API key is saved locally and never shared.

Inputs: no named parameters. Returns: result

upload_mediaA

Upload local media files (images, audio, video) and return dp:// references.

Use this BEFORE plan_survey / create_survey whenever the user wants a survey over local files β€” annotators cannot reach file:// paths or local disk, and the server rejects non-dp:// / non-https:// URLs.

After uploading, pass the returned dp:// refs verbatim in the plan_survey description so the generated plan references the uploaded media. Example:

refs = upload_media(["/tmp/a.png", "/tmp/b.png"])
# β†’ dp://media/abc123.png and dp://media/def456.png
plan_survey(
    description="Compare design A (dp://media/abc123.png) against "
                "design B (dp://media/def456.png). Target: UX designers.",
    max_responses=10,
)

Already-hosted public https:// URLs do NOT need uploading β€” you can reference them directly in the description.

Args: file_paths: List of absolute local paths to media files.

Inputs: file_paths (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.3/5.0

    Scored across 13 tools

    Disambiguation5/5

    Each tool has a distinct purpose: authentication, credit management, media upload, survey planning, creation, status checking, control (pause/resume/cancel), response retrieval, and failure recovery. No two tools overlap in functionality.

    Naming Consistency5/5

    All tool names follow the snake_case convention with a verb_noun or verb_preposition pattern (e.g., cancel_survey, upload_media, check_balance). The style is uniform and predictable.

    Tool Count5/5

    13 tools cover the full survey lifecycle without excess. Each tool addresses a specific step (plan, create, monitor, control, retrieve), and the count is appropriate for the server's scope.

    Completeness5/5

    The tool set covers authentication, credit management, media handling, survey design, execution, monitoring, control (pause/resume/cancel), result retrieval, and failure recovery. No obvious gaps hinder the typical workflow.

    Maintenance

    ActivityNo data
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      C
      maintenance
      Real human judgment as agent tools -- an AI agent can ask a question and get back a structured, schema-validated JSON answer from a real quality-scored human. 16 response types (yes/no, ratings, rankings, A/B tests, sentiment, image/video/audio review, voice/video/photo capture). Fully programmatic signup with a $5 free trial credit, no card required.
      7
      64 npm
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to hire real human operators for tasks requiring physical presence, human perception, or judgment, such as verification, testing, data collection, and physical-world tasks.
      4
      46 npm
      1
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to hire verified human operators for tasks requiring physical presence, human perception, or judgment, such as real-world verification, product testing, and data collection.
      6
      MIT