Skip to main content
Glama

Stat Effect Size

stat_effect_size

Computes effect sizes with confidence intervals for two-group comparisons, helping determine whether a statistically significant t-test or pairwise difference is practically meaningful.

Instructions

Effect size with a confidence interval for a two-group comparison.

WHEN TO USE: alongside every t-test and every pairwise comparison. A p-value says whether an effect is distinguishable from zero; the effect size says whether it matters.

WHEN NOT TO USE: for 3+ groups in a single call — use the effect sizes reported by ANOVA or by stat_pairwise_tests.

Note on paired designs: the standardiser is the SD of the difference scores, not the pooled SD. Cohen's d and dz are different quantities — state which you report.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "dv": "<dv>",
      "group": "<group>"
    },
    "name": "stat_effect_size"
  }
}

Input Schema

NameRequiredDescriptionDefault
dvYes
nameYes
groupYes
eftypeNocohen (d) | hedges (small-sample correction) | glass | r | eta-squared.cohen
pairedNo
subjectNo

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full behavioral burden, and it does disclose a genuinely non-obvious trait: that paired designs standardise by the SD of difference scores rather than pooled SD, making d and dz distinct quantities. It does not discuss distributional assumptions or what happens with unbalanced/missing data, which keeps it short of a 5 given the absent annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core statement, then cleanly sectioned into WHEN TO USE / WHEN NOT TO USE, then a targeted design note. The p-value-versus-effect-size sentence is motivational rather than operational, so a small amount of wording is expendable, but overall there is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be explained, and usage/routing is fully covered. Against a six-parameter specification tool with 17% schema coverage and no annotations, though, the missing guidance on what `dv`, `group`, and especially `subject` should contain is a real completeness gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 17% (only `eftype` is documented in-schema), so the description needs to compensate, and it partially does by clarifying the paired-vs-unpaired distinction that governs `paired` and by warning the user to state which of d/dz is reported. However, `dv`, `group`, `subject`, and `name` receive no explanatory treatment, so four of six parameters remain undocumented in both places.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific computation (effect size with confidence interval) and a specific scope (two-group comparison), which immediately separates it from ANOVA and multi-group tools. It also names `stat_pairwise_tests` as the route for 3+ groups, so an agent can tell the two apart without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description has explicit WHEN TO USE (alongside every t-test and pairwise comparison) and WHEN NOT TO USE (3+ groups in a single call) sections, and names the alternative to use in the excluded case. Nothing about routing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.