Skip to main content
Glama
splithub-io

SplitHub

Official

splithub_ship

Finalize an A/B experiment by keeping one variant, archiving the test in the cloud, and rewriting code to remove experiment branches and unused imports.

Instructions

Keep one variant and take the experiment out: archives it in the cloud (numbers stay readable), removes it from splithub.config.ts and rewrites the code ( → the kept , useValue → the kept literal, useVariant branches folded, unused imports dropped). Ask the user which variant to keep first (recommend verdict.winner; never a variant listed in verdict.guardrailBreached). Call with dryRun: true, show the diff, then without. Fix leftovers[] by hand, then follow next[] (push, doctor, tests).

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
keepYesvariant key to keep
dryRunNoonly return the diff
experimentYes

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
diffYesunified diff of the config and the source files
keptYes
kindYes
liveYes
nextYes
filesYes
dryRunYes
leftoversYes
configFileYes
experimentYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden and does well: it states that the experiment is archived in the cloud with its numbers preserved, that the config file is modified, and that source code is rewritten with specific transformations. It does not state whether the operation is reversible or what permissions are required, which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences that front-load the purpose, then the pre-call decision, then the dry-run/apply/post-call sequence. Parentheticals carry real detail (winner selection, rewrite targets) rather than filler; nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive multi-file operation this is complete: side effects, the variant-selection rule, the dry-run gate, and the follow-up steps are all present. Because an output schema exists, the references to leftovers[] and next[] are pointers rather than omissions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 67%, but the description compensates: it defines keep semantically ('which variant to keep', constrained to verdict.winner and never a guardrail-breached variant) and shows dryRun's role in a diff-then-apply sequence. The experiment parameter is left to the schema, but the important semantics are supplied.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a precise verb and resource and enumerates the concrete effects: cloud archive with numbers kept readable, removal from splithub.config.ts, and code rewrites (<Experiment> → <Variant>, useValue → literal, branch folding, import cleanup). This is far more specific than the title-less name. It never explicitly contrasts itself with the nearby splithub_remove, which is the one sibling an agent could confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It prescribes the whole workflow: ask the user which variant to keep, prefer verdict.winner and never a variant in verdict.guardrailBreached, run with dryRun: true and show the diff before the real call. It also tells the agent what to do after the call (fix leftovers[], then follow next[] for push/doctor/tests).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.