create_ab_test

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses a key behavioral trait not captured by annotations: a sample percentage is sent first and the winner is sent to the remainder only after selection. This is useful context for an agent. However, it leaves ambiguous whether winner selection is automatic or requires a separate tool call, which is a minor transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary purpose and no redundant filler. Every word contributes to understanding the tool's behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core process but does not mention what the tool returns (e.g., an A/B test ID) or explicitly direct the caller to use select_ab_test_winner for winner selection. Given there is no output schema, this missing information leaves some ambiguity about the full workflow and how to proceed after creation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While parameter schema coverage is 100%, the description adds meaning by explaining the role of sample_percentage and the staged send behavior. It also confirms the variants requirement ('two or more variants'). This goes beyond the basic schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Create an A/B test on a campaign with two or more variants.' It uses a specific verb and resource, and distinguishes it from sibling tools like select_ab_test_winner by focusing on creation rather than winner selection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage through the name and lifecycle mention ('the winner goes to the remainder once selected'), but it does not explicitly state when to use this tool versus alternatives. There is no direct reference to select_ab_test_winner for the subsequent step, leaving the workflow partially implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.