Skip to main content
Glama

BRAINIAC A/B Testing & Experimentation Statistics

Compare A/B arms with a Bayesian Beta-Binomial model

experimentation_bayesian-ab
Read-onlyIdempotent

Posterior, probability to beat control, probability of being best, expected loss and equal-tailed credible intervals for 2-10 conversion arms. Deterministic: exact Miller sum or quadrature, no Monte Carlo. Price $0.03 USDC. POST /api/v1/experimentation/bayesian-ab. An unpaid call returns the x402 payment terms and does not run.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "arms": "<arms>"
    },
    "name": "experimentation_bayesian-ab"
  }
}

Input Schema

NameRequiredDescriptionDefault
armsYesarms, array. Required.
priorNoprior, object. Required.
credibleLevelNocredibleLevel, number. Required.

Output Schema

NameRequiredDescriptionDefault
armsNoarms. Array.
methodNomethod. Object.
controlNocontrol. String.
sourcesNosources. Array.
limitationsNolimitations. Array.
credibleLevelNocredibleLevel. Number.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already declaring readOnly/idempotent/non-destructive, the description still adds substantial context: the computation is deterministic (exact Miller sum or quadrature, no Monte Carlo), the call costs $0.03 USDC, and crucially an unpaid call returns x402 payment terms rather than results. That last fact is exactly the failure mode an agent would otherwise stumble into.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short clauses, front-loaded with the returned quantities, then the method, then price, endpoint and payment behavior. No filler and no repetition of annotation content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists so return shapes needn't be re-explained, parameters are fully documented, and cost plus payment-gate behavior are disclosed. For a stateless compute endpoint, an agent has everything needed to call it and to interpret a payment-terms response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds a real constraint absent from the schema: the arms array must hold 2-10 entries (the schema specifies no minItems/maxItems). It says nothing about prior or credibleLevel beyond their defaults in the schema, so it stops short of 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the concrete outputs (posterior, probability to beat control, probability of being best, expected loss, equal-tailed credible intervals) and the input domain (2-10 conversion arms), and the model name 'Bayesian Beta-Binomial' pre-emptively separates it from the frequentist sibling. An agent can tell exactly what it produces without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the 2-10 arm bound and the Bayesian framing signal when it applies, but there is no explicit 'use this instead of experimentation_ab-test-frequentist when...' statement or exclusion. The reader must infer the decision rule from the model name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources