Datapoint MCP
Datapoint MCP lets AI agents recruit real humans for surveys and evaluation tasks (A/B tests, ratings, rankings, labeling) on text, images, audio, and video, then return aggregated or raw results into the conversation.
Authenticate with Datapoint AI via browser-based
setupUpload media (images, audio, video) to get dp:// references for surveys
Plan surveys from natural language descriptions, including target audience and screening criteria
Create surveys from a plan, launching paid human annotation work (with explicit user confirmation)
Support chain surveys with 2β5 linked steps and skip_if rules for dependent questions
Check survey status, progress, and aggregated results (consensus, votes, mean/median, ranks)
Get raw per-annotator responses with pagination and options to include abandoned/in-progress chains
List all surveys (active and recent)
Pause, resume, or cancel surveys, with credit refunds on cancellation
Retry failed datapoints to recover specific question failures
Check account balance and open a checkout link to add credits
Allows running human feedback surveys from within Windsurf (Codeium's IDE), enabling human evaluation tasks directly in the development environment.
Allows running human feedback surveys from within VS Code via GitHub Copilot Chat, enabling human evaluation of code, designs, and other content without leaving the editor.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Datapoint MCP Survey 20 people: which logo do they prefer, A or B?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Datapoint MCP
Get real human opinions from inside any MCP client. Run surveys, A/B preference comparisons, ratings, and rankings on text, images, audio, and video β without leaving your editor.
Datapoint MCP is an MCP server that gives Claude, GPT, Gemini, and any other MCP-capable agent the ability to recruit real humans for evaluation tasks, then return aggregated results back into the conversation. Built on top of Datapoint AI.
Why
LLMs are great at generating options and bad at telling you which one a real person will prefer. Datapoint MCP closes that loop β your agent can hand off to a panel of real humans and pick up the results a few minutes later.
Related MCP server: GetABrainπ§ | Live Human-in-the-Loop MCP for AI Agents
Use cases
Design & UX β A/B test logos, landing pages, screens, ad creative, copy
AI evaluation β human ratings of model outputs, side-by-side comparisons, hallucination checks
Preference data β collect RLHF / DPO pairs at scale
Dataset labeling β classification, ranking, captioning, content moderation
Product research β quick concept tests, naming, pricing reads
Human-in-the-loop checks β gate an agent before it ships something irreversible
Tools
Tool | Description |
| Authenticate with your Datapoint AI account (opens browser) |
| Upload local images, audio, or video so they can be used in a survey |
| Design a survey from a natural language description |
| Launch a survey from a plan |
| Check status, progress, and aggregated results |
| Get raw per-annotator responses (paginated) |
| List all your surveys |
| Pause task serving for an active survey (in-flight responses keep arriving) |
| Resume task serving for a paused survey |
| Permanently cancel a survey and refund unused reserved credits (irreversible) |
| Check your account balance |
| Open a checkout link to top up your account |
Video uploads accept browser-compatible H.264 MP4/MOV or VP8/VP9 WebM files.
The API validates the actual media bytes before storage; upload_media reports
unsupported, unreadable, or temporarily unprobeable videos in its failed-files
section.
Install
Requires uv on your PATH.
Claude Code
As a plugin (recommended):
/plugin marketplace add impel-intelligence/datapoint-mcp
/plugin install datapoint@datapointTo pick up new versions: /plugin marketplace update datapoint then /plugin update datapoint@datapoint.
As a raw MCP server (in ~/.claude/settings.json):
{
"mcpServers": {
"datapoint": {
"command": "uvx",
"args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
}
}
}Claude Desktop
Add to claude_desktop_config.json (~/Library/Application Support/Claude/ on macOS, %APPDATA%\Claude\ on Windows):
{
"mcpServers": {
"datapoint": {
"command": "uvx",
"args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
}
}
}Restart Claude Desktop, then ask it to run setup.
Cursor
Add to ~/.cursor/mcp.json (or via Settings β MCP):
{
"mcpServers": {
"datapoint": {
"command": "uvx",
"args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
}
}
}Windsurf
Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"datapoint": {
"command": "uvx",
"args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
}
}
}VS Code (GitHub Copilot Chat / agent mode)
Add to your workspace .vscode/mcp.json:
{
"servers": {
"datapoint": {
"command": "uvx",
"args": ["--from", "git+https://github.com/impel-intelligence/datapoint-mcp.git", "datapoint-mcp"]
}
}
}Any other MCP client
Run the binary over stdio:
uvx --from git+https://github.com/impel-intelligence/datapoint-mcp.git datapoint-mcpUsage
Once installed, just ask:
"Survey 20 people: which logo do they prefer, A or B?"
"Get human ratings on these three model outputs β which sounds most natural?"
"Run a quick A/B test on these two landing-page headlines."
The agent calls plan_survey to design it, shows you the plan and cost, then calls create_survey to launch. Use check_survey to monitor progress and read aggregated results.
Run setup first to authenticate if you haven't already.
Chain surveys (multi-step flow)
Some surveys have dependent questions β the second only makes sense given a specific answer to the first. Describe it that way and Claude will plan a chain:
"Ask 20 listeners if they could understand the speaker in this clip. If yes, rate the audio quality 1β5. If not, skip the rating."
A chain ties 2β5 steps together into a single unit of annotator work: every step is served to the same annotator, in order, and a per-step skip_if rule can end the response early. Claude will show you the full chain structure (steps, any skip conditions, cost) and wait for your confirmation before calling create_survey.
The cost shown in plan_survey is the upper bound (every response answers every step); when skip_if rules fire, responses cost proportionally less.
Configuration
Environment variable | Description |
| API key (overrides saved config) |
| API base URL (default: |
How it compares
Datapoint MCP | Mechanical Turk | Prolific | UserTesting | |
Run from inside an AI agent / IDE | β | β | β | β |
Designed for AI/LLM evaluation | β | β οΈ | β οΈ | β |
Pay-as-you-go via API | β | β | β | β |
Supports media (image / audio / video) | β | β | β | β |
Minutes to first response | β | β οΈ | β οΈ | β |
Links
Homepage: trydatapoint.com
MCP spec: modelcontextprotocol.io
Issues / discussions: GitHub
License
MIT β see LICENSE.
Available Tools
13 toolsadd_creditsA
Open a checkout link to purchase Datapoint AI credits.
Returns a hosted checkout URL. The user completes payment in their browser; credits land on their account once payment confirms.
Args: product_id: Optional product identifier. Omit to use the default credit bundle configured on the server.
Inputs: product_id. Returns: result
cancel_surveyA
Permanently cancel a survey and settle its reserved credits.
β Cancellation is irreversible. Once cancelled, the survey stops serving new tasks, any in-flight responses still complete, and all unused reserved credits are returned to the balance. Already-consumed credit (for responses already collected) is NOT refunded β the response is shown in the result so the user knows the final cost.
Before calling, show the user the survey id and confirm they want to
cancel. Prefer pause_survey when the user just wants to stop temporarily.
Backend rejects with 400 if the survey is already in a terminal state (completed, failed, or cancelled) β the message will say which.
Args: job_id: The job ID returned by create_survey.
Inputs: job_id (required). Returns: result
check_balanceA
Check your Datapoint AI account balance.
Inputs: no named parameters. Returns: result
check_surveyA
Check the status, progress, and results of a survey.
Args: job_id: The job ID returned by create_survey.
Inputs: job_id (required). Returns: result
create_surveyA
Create a survey from a plan generated by plan_survey.
β Only call this after the user has explicitly confirmed the plan
summary and cost from plan_survey. This reserves credits immediately
and dispatches paid human annotation work β there is no draft or
staging state, and it cannot be undone.
Pass the plan dict as returned by plan_survey. You may edit it first if anything is off β this is a regular Python dict.
Supports both standalone plans and chain plans (with a top-level steps
array alongside datapoints). For chain plans, the backend dispatches
each step of each datapoint as a linked task; the full sequence is
served together, in order, to one annotator per response.
MEDIA VALIDATION: every media entry must use either:
a dp://media/β¦ reference returned by
upload_media, ora public https:// URL the annotator's browser can reach.
Media lives at datapoints[*].media (shared across steps) OR
datapoints[*].media_per_step["0"], ["1"], ... (one entry per step
when steps need different media shapes). The two are mutually exclusive
per datapoint.
Local paths, file:// URLs, and private/auth-gated URLs will be rejected or served broken to annotators. If the plan came back with wrong refs (e.g. the planner invented a URL because the description didn't supply one), fix the refs before calling this:
# shared media
plan["datapoints"][0]["media"]["candidates"][0]["url"] = "dp://media/real.png"
# per-step media (chain)
plan["datapoints"][0]["media_per_step"]["1"]["subject"][0]["url"] = "dp://media/real.mp3"Args:
plan: The survey plan dict (typically from plan_survey). Contains
name, summary, datapoints, and either task_type + instruction
(standalone) or a steps array (chain).
Inputs: plan (required). Returns: result
get_survey_responsesA
Get the raw per-annotator responses for a survey.
check_survey returns aggregated results (consensus, votes, mean/median,
ranks). Use this tool when you want to see individual responses from each
annotator β useful for spotting outliers, seeing the spread of opinion, or
understanding disagreement.
The default view excludes answered rows from chains the annotator abandoned mid-flow or that are still in progress, matching what counts toward your survey's response total. Set the include flags to surface them.
Args: job_id: The job ID returned by create_survey. page: Page number (default 1). per_page: Responses per page (default 100, max 200). include_abandoned: Include answered rows from abandoned chains. include_in_progress: Include answered rows from in-flight chains.
Inputs: job_id (required), page, per_page, include_abandoned, +1 more. Returns: result
list_surveysB
List all your surveys (active and recent).
Inputs: no named parameters. Returns: result
pause_surveyA
Pause task serving for an active survey.
In-flight responses keep arriving; new tasks stop being served. The backend rejects with 400 if the survey is completed, failed, or already paused β the message will say which.
Args: job_id: The job ID returned by create_survey.
Inputs: job_id (required). Returns: result
plan_surveyA
Plan a survey from a natural language description.
Describe what you want to learn and from whom. The Datapoint AI service will design an effective survey structure for you.
MEDIA: If the survey compares/rates media (images, audio, video), first call
upload_media on any local files to get dp:// refs, then mention those refs
(or public https:// URLs) directly in this description so the planner can
wire them into the datapoints. Example:
description = (
"Compare two logo designs for memorability: "
"A = dp://media/abc123.png, B = dp://media/def456.png. "
"Target: general software developers."
)Without explicit refs in the description, the planner will produce
placeholder or invented URLs that will fail at create_survey.
After this returns, present the summary and cost to the user and wait
for explicit confirmation before calling create_survey. Never chain
these two calls β create_survey spends money and dispatches real work.
Args: description: What you want to survey, in plain language. Include the target audience, what you're comparing/rating, any screening criteria β including who should answer (e.g. respondents in specific countries, excluding VPN/bot traffic, balanced regional mix) β and, for media surveys, the dp:// or https:// URLs to use. max_responses: Number of human responses per datapoint (default 10). More = higher confidence but higher cost.
Inputs: description (required), max_responses. Returns: result
resume_surveyA
Resume task serving for a paused survey.
Backend rejects with 400 if the survey is not paused.
Args: job_id: The job ID returned by create_survey.
Inputs: job_id (required). Returns: result
retry_failed_datapointsA
Re-queue failed datapoints on a survey.
Each retried datapoint reserves credit again, the same way the original submission did β only call this when the failures are worth recovering.
Args: job_id: The job ID returned by create_survey. datapoint_indices: Specific failed indices to retry. Omit to retry every failed datapoint in the survey.
Inputs: job_id (required), datapoint_indices. Returns: result
setupA
Authenticate with Datapoint AI.
Opens your browser to sign in with Google and set up your account. Your API key is saved locally and never shared.
Inputs: no named parameters. Returns: result
upload_mediaA
Upload local media files (images, audio, video) and return dp:// references.
Use this BEFORE plan_survey / create_survey whenever the user wants a
survey over local files β annotators cannot reach file:// paths or local
disk, and the server rejects non-dp:// / non-https:// URLs.
After uploading, pass the returned dp:// refs verbatim in the plan_survey
description so the generated plan references the uploaded media. Example:
refs = upload_media(["/tmp/a.png", "/tmp/b.png"])
# β dp://media/abc123.png and dp://media/def456.png
plan_survey(
description="Compare design A (dp://media/abc123.png) against "
"design B (dp://media/def456.png). Target: UX designers.",
max_responses=10,
)Already-hosted public https:// URLs do NOT need uploading β you can reference them directly in the description.
Args: file_paths: List of absolute local paths to media files.
Inputs: file_paths (required). Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogβ¦
TDQS
Scored across 13 tools
Each tool has a distinct purpose: authentication, credit management, media upload, survey planning, creation, status checking, control (pause/resume/cancel), response retrieval, and failure recovery. No two tools overlap in functionality.
All tool names follow the snake_case convention with a verb_noun or verb_preposition pattern (e.g., cancel_survey, upload_media, check_balance). The style is uniform and predictable.
13 tools cover the full survey lifecycle without excess. Each tool addresses a specific step (plan, create, monitor, control, retrieve), and the count is appropriate for the server's scope.
The tool set covers authentication, credit management, media handling, survey design, execution, monitoring, control (pause/resume/cancel), result retrieval, and failure recovery. No obvious gaps hinder the typical workflow.
Maintenance
Related MCP Connectors
Human judgment for AI agents: discover capabilities, get quotes, and track paid human tasks.
Human-as-a-Service for AI agents. Delegate tasks that need a real human, get results via API.
Lets AI agents use a real human as a tool: visual checks, taste, phone calls, unblocking, approvals
API for AI agents to delegate tasks to real humans.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceLets AI agents natively discover and hire human experts for tasks they can't do themselves, such as research, verification, and expert calls.MIT
- AlicenseAqualityCmaintenanceReal human judgment as agent tools -- an AI agent can ask a question and get back a structured, schema-validated JSON answer from a real quality-scored human. 16 response types (yes/no, ratings, rankings, A/B tests, sentiment, image/video/audio review, voice/video/photo capture). Fully programmatic signup with a $5 free trial credit, no card required.764 npmMIT

humanforaiofficial
AlicenseAqualityCmaintenanceEnables AI agents to hire real human operators for tasks requiring physical presence, human perception, or judgment, such as verification, testing, data collection, and physical-world tasks.446 npm1MIT
humanforaiofficial
AlicenseAqualityCmaintenanceEnables AI agents to hire verified human operators for tasks requiring physical presence, human perception, or judgment, such as real-world verification, product testing, and data collection.6MIT