Skip to main content
Glama

Persuasion Taxonomy

Plan an A/B test

plan_ab_test
Read-onlyIdempotent

Plan an A/B test for marketing copy. It takes the current conversion rate, the smallest lift worth detecting and the daily traffic, and returns how many visitors each version needs and how many days to run. It also frames the test as two different answers to the same reader question, so the result teaches you something you can reuse, and not just that "B won". It answers questions like how long to run a test, how much traffic it needs, whether a test is worth running, or what to test next.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "baseline_conversion_rate_percent": "<baseline_conversion_rate_percent>",
      "minimum_detectable_lift_percent": "<minimum_detectable_lift_percent>",
      "daily_visitors": "<daily_visitors>"
    },
    "name": "plan_ab_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
powerNo
questionNoThe reader question both versions answer, each in its own way.
versionsNoHow many versions, counting the original.
version_aNoHow version A answers it.
version_bNoHow version B answers it.
confidenceNo
daily_visitorsYesHow many visitors a day enter the test, across all versions.
minimum_detectable_lift_percentYesThe smallest relative lift worth detecting, in percent. 20 means a 20% lift, like going from 3% to 3.6%.
baseline_conversion_rate_percentYesThe current conversion rate, in percent, so 3 means 3% and 0.8 means 0.8%.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and closed-world, so safety is covered. The description adds real behavioral value beyond that: it discloses the shape of the result (per-version sample size and run duration) and that output includes reusable framing of the test as two answers to one reader question. It says nothing about the statistical assumptions behind the sample-size math, how confidence/power defaults affect the answer, or what happens when traffic is too low to ever reach significance.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded: what it does, what it takes, what it returns, then when to use it. The middle sentence ('so the result teaches you something you can reuse, and not just that "B won"') is the least efficient, but it carries real design intent about the output rather than pure filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nine-parameter, no-output-schema tool, the description covers purpose, inputs, return shape and use cases adequately. The remaining gap is the statistical knobs (power, confidence, multi-version behavior) which an agent may need to reason about but which the description never mentions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 78%, so the schema largely documents the parameters, and the description only restates the three required ones. Its one genuine addition is the phrase 'two different answers to the same reader question', which implicitly explains the question/version_a/version_b trio that the schema describes only thinly. Optional power, confidence and versions are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Plan an A/B test for marketing copy') and then names both the inputs (conversion rate, minimum detectable lift, daily traffic) and the outputs (visitors per version, days to run). An agent knows exactly what comes back. It does not, however, distinguish itself from the sibling read_ab_test_result, so a reader must infer that this one plans rather than reads.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The closing sentence enumerates concrete situations ('how long to run a test, how much traffic it needs, whether a test is worth running, or what to test next'), which is clear when-to-use guidance. It stops short of exclusions or naming an alternative tool for adjacent needs, so it lands at 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.