Skip to main content
Glama

Stat Ttest

stat_ttest

Run independent, paired, or one-sample t-tests to compare means, with options for Welch correction, effect size, and confidence intervals.

Instructions

t-test: independent samples, paired, or one-sample.

WHEN TO USE

  • Two independent groups, continuous outcome, roughly normal, variances similar.

  • Same participants in two conditions → paired=True with subject.

  • One sample against a reference value → pass x (and y as the constant, or omit).

WHEN NOT TO USE

  • Three or more groups → stat_anova (a t-test on all pairs inflates Type I error).

  • Ordinal outcome, or badly non-normal with small n → stat_nonparametric_2.

  • Variances unequal → still a t-test, but set correction=True to get Welch's.

  • To claim two groups are equivalent → stat_equivalence_test; a non-significant t-test is not evidence of equivalence.

Always report the effect size (Cohen's d, given in the output) and the CI.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "dv": "<dv>"
    },
    "name": "stat_ttest"
  }
}

Input Schema

NameRequiredDescriptionDefault
xNoFor one-sample or two-vector mode.
yNoSecond variable for two-vector mode.
dvYesDependent variable.
nameYesSession dataset name.
groupNoColumn with exactly two levels (omit when using x/y).
pairedNoTrue for within-subject / matched designs.
subjectNoParticipant identifier — required when paired=True.
confidenceNoConfidence level for the CI.
correctionNo'auto' (Welch when sizes/variances differ) | True | False.auto
alternativeNotwo-sided | less | greater.two-sided

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full burden and does so substantially: it discloses that Welch's correction is applied via correction=True, that `subject` is mandatory in paired mode, that Cohen's d and a CI appear in the output, and it warns that a non-significant result is not evidence of equivalence. It does not mention what happens with missing data or whether any state is written to the session dataset, which keeps it below a 5 for a 10-parameter tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is front-loaded with the operation, then partitioned into WHEN TO USE / WHEN NOT TO USE blocks that an agent can scan quickly, and closes with a reporting requirement. The length is justified by the number of design modes; there is no filler or restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter statistical test with an output schema, the description covers mode selection, assumptions, the correction switch, and reporting obligations, which is close to complete. The one gap is that it never explains how the `name` (session dataset) parameter is used or how to load data first, which an agent encounters before any of the described modes apply.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds interpretive meaning the schema does not: how to encode a one-sample test (pass `x`, supply `y` as the constant or omit it), the constraint that `group` must have exactly two levels and is omitted in x/y mode, and that correction=True maps to Welch's rather than the schema's 'auto' default. The relationship between the `x`/`y` mode and the `dv`/`group` mode is still only implied.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line names a specific statistical procedure and immediately enumerates its three variants (independent samples, paired, one-sample). The WHEN NOT TO USE block names concrete siblings (stat_anova, stat_nonparametric_2, stat_equivalence_test), so an agent can distinguish this tool from its nearest neighbours without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Guidance is fully explicit in both directions: WHEN TO USE lists the design conditions (two groups, continuous outcome, normality, similar variances; paired with subject; one-sample via x), and WHEN NOT TO USE lists the disallowed conditions with the correct alternative for each. The variance case, which stays with this tool but flips a parameter (correction=True for Welch's), is a particularly useful nuance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.