Skip to main content
Glama

validate_study

Check a completed study against real data, as the web app's Validate panel does: pass the split a real survey or poll found on the same question, and get back how far the simulated split was from it. Free. Works on a study with a sentiment split (run_focus_group); a debate or a ranking has none and is refused. The check is saved with the study.

real_distribution takes percentages that sum to 100, or raw respondent counts, for positive, neutral and negative (agree, undecided and disagree when the study had a stance_statement).

Returns: { sim_id, grade (A-F), parity_pct (0-100), mae (mean absolute error, percentage points), rmse, pearson_r, simulated: { positive, neutral, negative }, real: {...}, buckets: [ { bucket, simulated_pct, real_pct, delta_pp } ], real_source, real_sample_size }.

Input Schema

NameRequiredDescriptionDefault
sim_idYesThe id of a completed study you ran.
real_sourceNoWhere the real figures come from, e.g. 'Pew, March 2026'.
real_sample_sizeNoHow many people the real survey asked.
real_distributionYesThe real split: percentages summing to 100, or raw counts. Keys: positive, neutral, negative.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and discloses key behavior: the check is free, it saves the check with the study, and it refuses studies without a sentiment split. It does not cover permissions, re-run/overwrite behavior, or failure modes beyond refusal, but the main traits are clearly stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and appropriately sized: purpose and restrictions come first, parameter clarification follows, and the return shape comes last. Because there is no output schema, the explicit return summary earns its space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the nested real_distribution parameter, no output schema, and the need to understand study-type restrictions, the description is complete enough to call the tool correctly. It explains accepted input forms, refusal cases, persistence behavior, and the full returned metric shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds useful meaning beyond the schema: real_distribution may be percentages summing to 100 or raw respondent counts, and maps positive/neutral/negative to agree/undecided/disagree when the study had a stance_statement.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: check a completed study against real data and return how far the simulated split was from it. It also distinguishes compatible studies (sentiment split from run_focus_group) from incompatible ones (debate or ranking).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly says when the tool works and when it does not: only for a study with a sentiment split, while a debate or ranking is refused. It also names run_focus_group as the relevant sibling for the supported study type.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources