Skip to main content
Glama

Create a draft experiment

create_experiment

Create an A/B test as a DRAFT: nobody sees it until start_experiment. 2 to 4 arms, each pointing at a paywall or onboarding flow of this app on the experiment's surface (ids from list_surfaces); one arm may be a holdout with no target (it keeps the live config). Exactly one control; the first arm when none is marked. Defaults: mode notify (report the winner, don't ship it), new users only, the default placement, trial_started as the metric. Only works when the app allows MCP writes.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>",
      "surface": "<surface>",
      "variants": "<variants>"
    },
    "name": "create_experiment"
  }
}

Input Schema

NameRequiredDescriptionDefault
modeNonotify (default) reports a winner; auto ships it
nameYes
is_qaNoA QA run on test builds. Default false
surfaceYes
priorityNoEnrollment order, lowest first. Default 100
variantsYes
placementNoPlacement id (letters, digits, _ or -). Default `default`
targetingNoWho is eligible; every rule must pass, {} = everyone. Keys: countries (ISO-2), platforms (ios, android, web), app_version_min/max, days_since_install_min/max, billing_states, custom
enrollmentNoDefault new: users first seen after the start
hypothesisNoWhat you expect to happen and why
audience_pctNo% of eligible users enrolled. Default 100
min_confidenceNoChance to be best a winner needs. Default 0.95
primary_metricNoDefault trial_started
experiment_typeNoA label. Default other

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses the draft/live boundary, the write-permission gate, and non-obvious structural behavior (holdout arm keeps the live config, exactly one control, first arm is control by default). Annotations already mark it non-destructive and non-readonly, so the remaining gaps are minor.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The draft-state constraint is front-loaded, then arm rules, then defaults, then the permission gate — a logical build-up with no filler sentences. Every clause carries either a constraint or a default an agent would otherwise get wrong.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 14-parameter tool with nested variants and no output schema, the description covers the decision logic and structural invariants well, and the schema documents the remaining fields. The main omission is what the call returns (draft id, created object), which matters for chaining into start_experiment, though no output schema exists to cover it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 79%, so the baseline is 3, and the description earns an upgrade by stating cross-field rules the schema cannot express: 2 to 4 arms, each targeting a paywall or onboarding flow on the experiment's surface, one control, first arm defaults to control, holdout carries no target. It also restates key defaults (mode notify, new users, default placement, trial_started) though those already appear in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Create an A/B test as a DRAFT" gives a specific verb, resource, and lifecycle state, and the follow-on clause "nobody sees it until start_experiment" immediately separates it from the start_experiment sibling. An agent can distinguish create vs. launch without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names the downstream alternative (start_experiment) and states the condition for it, points at list_surfaces for target ids, and states the precondition "Only works when the app allows MCP writes." That covers when to use it, what it does not do, and a blocking prerequisite.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources