RealRun
OfficialRead-only lookups of RealRun's published, independent AI-model test results for everyday tasks, so an agent can pick a model and see safety warnings.
list_tasks— lists tested tasks with item counts, test dates, results pages and playbook prices.get_scores— every model's headline score for a task (best first), plus cost tier and metric definition.recommend_model— one pick plus 2 alternatives for a task, tuned by priority (accuracy/cost/balanced) and optionally including subscription-tested Claude models; never picks a flagged model.get_warnings— not-recommended and caution flags (scam failures, invented data, inconsistency).Note: the published README lists five task ids (
inbox-triage,meeting-scheduling,receipts,finance,support-replies), but the schema'staskenum only accepts the first three.All tools are read-only, stateless, unauthenticated, JSON-returning; billing is not active.
RealRun MCP server
Read-only lookups over RealRun's published, independent tests of AI models on everyday tasks (inbox triage, meeting scheduling, receipts, small-business finance, customer support replies). It lets an agent ask "which model should I use for task X?" and get scores and safety warnings back.
Connect (remote, no install)
Streamable HTTP endpoint: https://realrun.baseledgerip.com/mcp
{ "mcpServers": { "realrun": { "url": "https://realrun.baseledgerip.com/mcp" } } }Claude Code: claude mcp add --transport http realrun https://realrun.baseledgerip.com/mcp
Cline: add this to cline_mcp_settings.json (keep "type": "streamableHttp"; without it Cline falls back to SSE and the connection fails). Full agent-readable steps: llms-install.md.
{ "mcpServers": { "realrun": { "type": "streamableHttp", "url": "https://realrun.baseledgerip.com/mcp", "disabled": false, "autoApprove": [] } } }Related MCP server: Verging Labs MCP server
Run locally (stdio)
npx -y realrun-mcp(once published to npm; from a checkout: npm install && node src/stdio.mjs)
Tools
Tool | Input | Returns |
| none | Tested tasks, item counts, dates, results pages, playbook prices |
|
| Every model's headline score (best first), cost tier, metric definition |
|
| One pick + 2 alternatives; never picks a model flagged not recommended |
|
| Not-recommended and caution flags (scam failures, invented data, inconsistency) |
Task ids: inbox-triage, meeting-scheduling, receipts, finance, support-replies.
All tools are read-only. The server stores nothing and has no auth. It is stateless, with JSON responses.
Data
src/data.json holds only the published headline numbers. Full prompts and item-level results are in the paid playbooks ($29 each, links in list_tasks). Scores reflect our tests on the dates shown, not a guarantee.
Billing (not active)
Every call passes through src/metering.mjs. The plan is a free allowance of about 20 lookups a day, then per-call billing via Stripe or x402. It is off: nobody is charged, and no payment objects exist.
Test
npm test runs stdio end to end. node test/smoke.mjs https://realrun.baseledgerip.com/mcp tests the remote endpoint.
Layout
src/server.mjs: toolssrc/stdio.mjs: CLIsrc/http.mjs: exports for the HTTP function../netlify/functions/mcp.mjs: Netlify Function at/mcpserver.json: official MCP registry metadata
License: MIT. © Base Ledger IP LLC.
Available Tools
4 toolsget_scoresGet model scores for a taskARead-only
Headline scores for every model RealRun tested on a task, best first, with cost tier and the metric definition.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | Task id, e.g. 'receipts'. See list_tasks. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description adds useful behavioral context (results returned best-first, each row carrying cost tier and the metric definition), but says nothing about freshness, pagination, or how many models are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the resource and its ordering rule; every clause adds information (scope, sort, payload) and none is redundant with the name or title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the work of describing the return shape (scores plus cost tier plus metric definition), which is what an agent needs. Only minor gaps remain, such as result-set size or staleness — low stakes for a read-only lookup.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single enum-constrained 'task' parameter already documented, including a cross-reference to list_tasks. The description adds no parameter detail, so the schema alone carries this dimension — the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb+resource ('Headline scores for every model RealRun tested on a task') plus scope, ordering and payload. It is distinguishable from recommend_model (picks one) and list_tasks, though it never names those siblings explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied — 'every model' hints this is the comparison/overview call versus recommend_model's single pick, but there is no explicit when-to-use or when-not-to-use statement and no alternative named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_warningsGet safety warnings for a taskBRead-only
Models RealRun flagged as not recommended or needing caution on a task (scam failures, invented data, inconsistency).
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | Task id, e.g. 'receipts'. See list_tasks. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds useful substance about what the warnings represent (the flag categories) but says nothing about return shape, whether results can be empty, or how warnings relate to get_scores output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with no filler; the parenthetical examples earn their place by concretizing the flag types. Phrasing is slightly compressed ('Models RealRun flagged') but front-loads the object of the call.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter, fully documented read-only lookup with no output schema, the description tells the agent what the tool returns in enough detail to call it. The missing piece is how these warnings should feed into model selection relative to get_scores.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'task' parameter is a documented enum pointing at list_tasks, so the schema carries the burden. The description adds no parameter-level meaning beyond that, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific resource and filter: models flagged as 'not recommended' or 'needing caution' for a given task, plus concrete example categories (scam failures, invented data, inconsistency). It is clearly distinguishable from get_scores by virtue of returning safety flags rather than numeric ratings, though it never names that sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use or when-not-to-use guidance, and no alternatives are named despite get_scores and recommend_model being obvious siblings. The agent must infer from context that this is a pre-recommendation safety check.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_tasksList tested tasksARead-only
List the everyday tasks RealRun has tested AI models on, with item counts, test dates, results pages and playbook prices.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds value beyond that by disclosing what each listed item contains (item counts, test dates, results pages, playbook prices), which matters because there is no output schema; it does not cover ordering, pagination, or limits, so it is not a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that names the action, the resource, and the returned content with zero filler. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only list tool with no output schema, the description supplies enough: it identifies the dataset and enumerates the fields returned. Minor gaps remain around result ordering or pagination behavior, but nothing blocks a correct call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes no parameters, which is the stated baseline of 4. There is nothing for the description to clarify about inputs, and it correctly does not invent any.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ('List') and a specific resource ('the everyday tasks RealRun has tested AI models on'), plus the fields returned. It is clear what the tool does, but it never distinguishes itself from siblings like get_scores or get_warnings, so an agent gets no routing signal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage as a catalog/discovery call is strongly implied by 'list the everyday tasks... with item counts, test dates, results pages and playbook prices', so an agent can infer it is the entry point for browsing. There is no explicit when-to-use, when-not, or mention of the alternatives (get_scores, recommend_model), leaving routing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recommend_modelRecommend a model for a taskARead-only
Recommend which AI model to use for a task given a priority: 'accuracy', 'cost' or 'balanced'. Never recommends models RealRun flagged as not recommended.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | Task id, e.g. 'receipts'. See list_tasks. | |
| priority | No | What matters most. | balanced |
| include_subscription | No | Include Claude models (tested on a subscription, no metered cost). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description goes beyond that by disclosing a genuine filtering rule: 'Never recommends models RealRun flagged as not recommended.' It still omits whether results are ranked, single, or include reasoning, so it adds value but not full behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with zero filler; the purpose and its decision input are front-loaded, and the exclusion rule is stated compactly at the end.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the burden of explaining what comes back, yet it never says whether a model id, a ranked list, or rationale is returned. For a simple read-only recommendation tool this is adequate but leaves a real gap around the response shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters are already documented. The description restates the priority enum values ('accuracy', 'cost' or 'balanced'), which is redundant rather than additive, and says nothing extra about task or include_subscription. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Recommend which AI model to use') plus the input that drives the choice ('given a priority'). The sibling tools (get_scores, get_warnings, list_tasks) are clearly unrelated, so an agent can identify this tool's role without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the deciding input and enumerates its allowed values, which implies when the tool is useful (choosing a model by priority). However, it gives no when-not guidance and does not point to any alternative tool for related questions (e.g., why a model was excluded), leaving usage to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
- First observed
get_scores - First observed
get_warnings - First observed
list_tasks - First observed
recommend_model
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: listing tasks, getting detailed scores, getting recommendations, and getting warnings. There is no overlap in functionality; an agent can easily select the right tool based on the need.
All tool names follow a consistent verb_noun pattern: list_tasks, get_scores, recommend_model, get_warnings. The convention is predictable and uniform.
With only 4 tools, the server is well-scoped for its purpose of providing AI model evaluation and recommendations. Each tool earns its place and the count is neither too thin nor too heavy.
The tool set covers the full lifecycle: discovering tasks (list_tasks), comparing models (get_scores), getting a tailored recommendation (recommend_model), and checking warnings (get_warnings). There are no obvious missing operations for this domain.
Maintenance
Related MCP Connectors
Independent test scores for AI models on everyday tasks: inbox, scheduling, receipts.
Signed, independent scores and claim checks for decentralised AI projects. Read-only, no auth.
71Read-only evidence reports on AI agents: search, fetch a report, check trust. Not a certification.
Scores how well AI agents will use your API, 0-100, from its OpenAPI spec. Free and read-only.
Related MCP Servers
- AlicenseAqualityFmaintenanceProvides AI assistants with real-time access to 1000+ AI models including their latest pricing, context windows, capabilities, and specifications. Supports model search, comparison, recommendations, and live testing.3104 npm1MIT
- AlicenseNot gradedqualityCmaintenanceProvides AI agents with honest benchmark rankings (Agentic Memory Index and Agentic Search Index) for AI tools, plus graded checks and telemetry for x402 endpoints.6 npmMIT
- AlicenseAqualityAmaintenanceProvides read-only tools to generate test plans for AI agents, score agent runs, and produce reusable evaluation scorecards.3MIT
- AlicenseNot gradedqualityAmaintenanceEnables MCP-compatible AI assistants to securely search multiple mailboxes, reconstruct email threads, and inspect attachments through read-only tools without altering mailbox state.Apache 2.0