Skip to main content
Glama

BRAINIAC A/B Testing & Experimentation Statistics

Estimate a ratio metric with the delta method

experimentation_delta-method-ratio
Read-onlyIdempotent

Confidence interval for a ratio metric whose analysis unit differs from the randomisation unit, plus the treatment-vs-control difference and relative lift. Accepts unit rows or aggregates. Price $0.03 USDC. POST /api/v1/experimentation/delta-method-ratio. An unpaid call returns the x402 payment terms and does not run.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "arms": "<arms>"
    },
    "name": "experimentation_delta-method-ratio"
  }
}

Input Schema

NameRequiredDescriptionDefault
armsYesarms, array. Required.
alphaNoalpha, number. Required.

Output Schema

NameRequiredDescriptionDefault
armsNoarms. Array.
sourcesNosources. Array.
differenceNodifference. Object.
relativeCiNorelativeCi. Null.
limitationsNolimitations. Array.
relativeLiftNorelativeLift. Null.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so safety is covered; the description instead adds operational context the annotations cannot convey: a $0.03 USDC price, the POST endpoint, and the fact that an unpaid call returns x402 payment terms without executing. That payment-gating behavior is genuinely important for an agent deciding whether to retry. It stops short of describing failure modes for malformed arms or the required control designation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with what is returned and when it applies, followed by input modes, price/endpoint, and payment behavior. Nothing is redundant with the annotations or schema except the restated endpoint, which is still useful for direct calls. No filler or hedging.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and the description still summarizes the returns (CI, difference, relative lift), so return values are adequately covered. For a two-parameter tool with a nested arms array, the main residual gap is that nothing explains how to mark the control arm or what the second-moment aggregates are for. Pricing and payment semantics, which are the highest-risk unknowns, are handled well.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The sentence 'Accepts unit rows or aggregates' adds real meaning by signaling that `units` and the sum* fields are alternative input modes rather than cumulative requirements. However, the several aggregate fields (sumNum2, sumDen2, sumNumDen) and the `control` flag inside each arm remain semantically opaque, and the description does not explain them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: computes a confidence interval for a ratio metric with the delta method, plus the treatment-vs-control difference and relative lift. The qualifier 'whose analysis unit differs from the randomisation unit' accurately pins down the statistical use case. It does not name or contrast any sibling (e.g. ab-test-frequentist or sample-ratio-mismatch), so an agent must infer the distinction from the qualifier alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the 'analysis unit differs from the randomisation unit' clause effectively describes the situation that warrants this tool, and 'Accepts unit rows or aggregates' indicates the two input modes. There is no explicit when-not guidance, no mention of when to prefer a sibling test, and no stated prerequisites about needing a control arm.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources