Skip to main content
Glama
SplitHub

SplitHub

Official

One install gives you both: A/B experiments with a plain verdict, and analytics for every page, screen and event, with funnels, paths and retention. It works on websites (React, Next.js, a script tag) and in apps (React Native and Expo, iOS, Android). Declare experiments in splithub.config.ts or the dashboard, render variants, call track() for anything that matters, and read the numbers from the CLI, the MCP server or the dashboard. Your coding agent can do all of it for you.

npx splithub init

Why SplitHub

  • Config in your code. Experiments live in splithub.config.ts, reviewed in pull requests like everything else.

  • No flicker, no network on render. Who sees which variant is a local hash, decided before the first paint. The same hash runs in the browser, on the server and in the iOS and Android SDKs, so a visitor gets the same variant everywhere.

  • Websites and apps. React Native, Expo, Swift and Kotlin apps send screens and events to the same reports, and keep them on the device while offline.

  • One track() for everything. Any event you send feeds the analytics reports and converts every experiment that measures it. No second tracking setup.

  • A verdict, not a chart to decode. SplitHub Cloud says when a variant wins, and peeking early never fools it.

  • Made for agents. The CLI answers in JSON, and the MCP server gives Claude Code, Cursor and Codex the same tools you have.

  • Your data, if you want it. Send events to your own database instead (SQLite, D1, Postgres, MySQL) and read the raw numbers.

Related MCP server: Kumiki A/B

Experiments in three steps

1. Describe the experiment.

// splithub.config.ts
import { defineConfig } from '@splithub/core';

export default defineConfig({
  project: 'my-app',
  storage: { mode: 'cloud', key: 'pk_...' }, // or { mode: 'self', endpoint: '/api/splithub' }
  goals: { signup: { type: 'event' } },
  experiments: {
    'hero-title': {
      kind: 'value',
      status: 'running',
      variants: {
        control: { weight: 50, value: 'Ship faster' },
        direct: { weight: 50, value: 'Know which version wins' },
      },
      goal: 'signup',
    },
  },
});

2. Show the variant and count the goal.

import { SplitHubProvider, useValue, useTrack } from '@splithub/react';
import config from './splithub.config';

function Hero() {
  const title = useValue<string>('hero-title');
  const track = useTrack();
  return (
    <>
      <h1>{title}</h1>
      <button onClick={() => track('signup')}>Sign up</button>
    </>
  );
}

export const App = () => (
  <SplitHubProvider config={config} property="sh-x1h78w">
    <Hero />
  </SplitHubProvider>
);

3. Read the result.

npx splithub status              # every experiment with its verdict
npx splithub results hero-title  # lift, chance to beat control, confidence interval

What experiments can do

  • Four kinds: value (data per variant), component (JSX per variant), redirect (a page per variant) and mvt (every combination of a few factors, with the effect of each one).

  • Targeting: show an experiment only to some visitors, by country, device, browser, UTM tags or cookies; in apps also by platform and app version.

  • Layers and a holdout: run experiments side by side without overlap, and keep a share of visitors out of all of them to measure the long-term effect.

  • Guardrails and secondary metrics: a variant that hurts a metric you protect is never called the winner.

  • Sample size up front: npx splithub plan says how many visitors an experiment needs, no account required.

  • Slices and breakdowns: results by country, device, platform, app version, source, landing page or day; every event per variant.

  • Ship the winner: npx splithub ship <id> --keep <variant> rewrites your code to the kept variant and removes the experiment.

  • In your pull requests: npx splithub ci init posts the results as a PR comment.

Analytics

Name an analytics property on the provider (or on the script tag) and every page and track() call is counted. npx splithub analytics connect sets it up.

<SplitHubProvider config={config} property="sh-x1h78w">
const track = useTrack();
track('Video Played', { id: 3 });       // any event name, with props
track('checkout', { revenue: 49 });     // revenue counts too

const identify = useIdentify();
identify('user_42', { plan: 'pro' });   // tie visits to your user and their traits

What you get, in the dashboard, the CLI and the MCP server:

  • Traffic: visitors, visits, pageviews, bounce rate and time on site, against any earlier period.

  • Sources: channels, referrers and UTM tags; pages, landing and exit pages, with page groups like /blog/*.

  • Audience: country, region, city, device, browser, OS, language, user traits, and the variant each visitor saw.

  • Events and goals: every event with its props; mark any event or page as a goal to count conversions and revenue.

  • Funnels: visitors through up to 8 steps in one visit or within 7 or 30 days, split by any dimension. Save them for the whole team.

  • Paths and retention: where visitors go next, and who comes back week after week.

  • Realtime, segments and export: who is on the site now, saved filters for any report, CSV of any table or the raw events.

npx splithub analytics sh-x1h78w --by source --compare
npx splithub analytics funnel sh-x1h78w --steps "page:/,event:Signup,goal:paid"
npx splithub analytics funnels sh-x1h78w --save Signup --steps "page:/,goal:signup"

No build step? One script tag counts pages and events and runs redirect experiments:

<script src="https://go.splithub.io/sh.js" data-key="pk_..." data-property="sh-x1h78w"></script>

Apps

React Native and Expo:

npx expo install @splithub/react-native @react-native-async-storage/async-storage expo-application expo-device
import { SplitHubProvider, ExpoRouterScreens, Experiment, Variant, useTrack } from '@splithub/react-native';

<SplitHubProvider publicKey="pk_..." property="sh-x1h78w">
  <ExpoRouterScreens />   {/* every screen counted as a page */}
  <Stack />
</SplitHubProvider>

npx splithub init does this for you in a React Native or Expo app.

iOS (Swift package, iOS 14+):

.package(url: "https://github.com/splithub-io/splithub-sdk", from: "0.5.1")

SplitHub.start(publicKey: "pk_...", property: "sh-x1h78w")
let button = SplitHub.variant("checkout-button")
SplitHub.track("Purchase", props: ["revenue": 29])

Android (Kotlin, min SDK 21):

implementation("io.splithub:splithub:0.5.1")

SplitHub.start(this, "pk_...", property = "sh-x1h78w")
val cta = SplitHub.variant("hero-cta")
SplitHub.track("signup")

An app can't redeploy in minutes, so its variants ship in the build and everything else comes from SplitHub Cloud: start, pause, weights, values and new value experiments reach installed apps on the next launch. Retention by day or week shows who comes back after the first open.

Also in SplitHub Cloud

  • Notes: mark launches, outages and campaigns; they show on every chart.

  • Short links: links with click counts, and split links that send traffic to several pages.

  • Alerts: a winner, a broken split, a traffic drop or spike, a goal gone silent, by email, Slack, Discord or webhook.

  • Allowed domains, API keys with scopes, and team roles per area or per experiment.

  • AI reviews of an experiment at start, on a verdict change and at the end (Pro).

With your coding agent

claude mcp add splithub -- npx -y @splithub/mcp

Or let the CLI write .mcp.json for Claude Code and .cursor/mcp.json for Cursor:

npx splithub skills --mcp

Then ask in plain words: "test a shorter headline on the pricing page", "is the new checkout winning?" or "where do visitors drop off between pricing and signup?".

Packages

Package

npm

What it does

splithub

npm

The CLI: experiments, verdicts, analytics reports, funnels, notes, links and alerts.

@splithub/mcp

npm

The CLI as MCP tools for Claude Code, Cursor and Codex.

@splithub/core

npm

The config, who sees which variant, and the event queue. No dependencies.

@splithub/react

npm

<Experiment>, useValue, useTrack, useIdentify. No flicker, nothing fetched on render.

@splithub/react-native

npm

The same components and hooks for React Native and Expo, with screens and offline events.

@splithub/next

npm

Middleware, server components and redirect experiments for Next.js.

@splithub/browser

npm

One script tag: analytics and redirect experiments for sites without a build step.

@splithub/node

npm

One handler for events and raw results on your own server.

@splithub/adapter-sqlite

npm

SQLite store, on the SQLite built into Node.

@splithub/adapter-d1

npm

Cloudflare D1 store.

Native SDKs:

SDK

Install

What it does

SplitHub for iOS and macOS

Swift Package Manager, this repository's URL

Variants, screens and events in Swift, with SwiftUI helpers. No dependencies.

io.splithub:splithub for Android

Gradle

Variants, screens and events in Kotlin; io.splithub:splithub-compose adds Jetpack Compose helpers.

Three complete apps live in examples/: Vite + React, Next.js and an Expo app (examples/expo-app, installed on its own).

Cloud or your own database

SplitHub Cloud

Self-hosted

Where events go

SplitHub Cloud

Your database: SQLite, D1, Postgres, MySQL

Experiments

Verdict, lift, data quality, sample size, slices

Visitors, conversions, rate and revenue per variant

Analytics

Every report above

Raw counts of pages and events

Setup

npx splithub login

npx splithub init --mode self

Develop

pnpm install
pnpm build
pnpm test

Node 22.13 or newer, pnpm 10. The native SDKs build with their own tools: swift test in sdks/ios, Gradle in sdks/android.

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md. Found a security problem? See SECURITY.md.

License

MIT

Available Tools

36 tools
splithub_alertsC

Alert rules (SplitHub Cloud; they fire on Pro, a rule added on another plan is saved switched off). action list (default): rules with condition, params (params.progress { total, fired, month? } on usage and event_count is read-only), enabled; channels; history (recent alerts). add: condition + params. Experiment conditions (experiment = one key, omit for every running one): srm (broken split), winner (minVisitors?), loser (lift %, minVisitors?), exposures (visitors), conversions (conversions), duration (days), no_events (hours). Account: quota (percent of the monthly allowance), usage (every: alert each time this month's billed events cross another multiple, e.g. 1000). Analytics (property required): traffic_drop / traffic_spike (percent vs the same 24 hours a week before), goal_silent (goal, hours), event_count (event, any event when omitted; count; repeat = every count instead of once). Experiment rules may also pause, finish or dropLoser (winner / loser). enable, disable, delete, test (dry run: what would fire now, nothing sent): rule = id, the start of it, or the name. Where alerts go (email, Slack, Discord, webhook) is set in the dashboard.

NameRequiredDescriptionDefault
daysNo
goalNo
liftNo
nameNo
ruleNorule id, id prefix or name (enable, disable, delete, test)
countNo
eventNo
everyNo
hoursNo
pauseNo
actionNo
finishNo
repeatNo
enabledNoadd: false saves it switched off
percentNo
propertyNo
visitorsNo
conditionNo
dropLoserNo
experimentNo
conversionsNo
minVisitorsNo
cooldownHoursNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses some traits (rules fire on Pro, rules added on other plans are saved off, test is a dry run, where alerts go is set in the dashboard), but it omits critical details like required permissions, side effects of delete, rate limits, and what the response looks like. This is a significant gap for a 23-parameter mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense paragraph with semicolons and parentheticals, lacking front-loading or logical grouping. Information is packed but not organized, making it hard to parse quickly. It is not concise; many clauses could be clarified or split into structured sections.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (23 parameters, no annotations, no output schema), the description should be more complete. It covers many aspects but misses key details like return values, pagination for list, and the full parameter space. The lack of an output schema means the description should explain what list returns, which it doesn't.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 9%, so the description must compensate. It attempts to explain many parameters (e.g., condition types, params.progress behavior, action-specific fields like rule), but the explanations are terse, cryptic, and incomplete—cooldownHours and other parameters are not mentioned. It adds some value but falls short of full compensation for the low schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it manages alert rules and enumerates the actions (list, add, enable, disable, delete, test), making the tool's purpose specific. It doesn't explicitly differentiate from siblings, but no other alert-related tools exist among the siblings, so the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage through the action list and some context (e.g., rules fire on Pro, test is dry run), but it never explicitly states when to use this tool versus alternatives or what the alternatives would be. Since there are no overlapping siblings, the guidance is adequate but not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analyticsA

Analytics properties (pageviews or app screens, and custom events, no variants). A property is kind site or app; in an app a screen is a page (url /ScreenName) and platform / app_version are dimensions for by and filters. Without property: every property with visitors, visits, pageviews and bounce for the last 30 days. With property: totals (visitors, newVisitors, sessions = visits, pageviews, events, bounces, duration in ms; bounce rate = bounces / sessions), goals (conversions = unique visitors who reached it in the range, completions = how many times, revenue), topPages and topSources (value "(none)" = direct). With by: rows grouped by a visit dimension, page, event or goal; goal adds that goal's conversions per row; variant (needs experiment) splits visits by the variant each visitor saw, trait (needs key) by an identify() trait. Range: days (last N days) or from / to (YYYY-MM-DD, project time zone). filters narrow the visits, e.g. { country: "DE,FR", source: "(none)" }. Self-hosted projects get raw counts only (analysis.available false): report them as they are and say full reports come with SplitHub Cloud. compare adds previous (the compared range, with its dates in previous.range) for change against it: 'previous' (the same-length range right before), 'year' (the same days a year earlier) or 'YYYY-MM-DD..YYYY-MM-DD'. segment applies a saved segment (splithub_segments) by name or id. These are counts, not experiments: no verdicts, never call a winner from them.

NameRequiredDescriptionDefault
byNo
toNo
keyNoby trait: the trait name, e.g. plan
daysNo
fromNo
goalNowith by: also count this goal per row
limitNo
compareNoalso read another period: 'previous' (right before), 'year' (a year earlier) or 'YYYY-MM-DD..YYYY-MM-DD'
filtersNodimension → comma-separated values
segmentNosaved segment name or id
propertyNoproperty id; omit to list every property
experimentNoby variant: the experiment key

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that self-hosted projects return raw counts only with analysis.available false, that counts must not be treated as experiment verdicts, that '(none)' means direct traffic, and that compare returns previous.range. It does not mention auth requirements, rate limits, or how large result sets are truncated, which leaves some behavioral gaps for a 12-parameter reporting tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Nearly every clause carries information, and the without-property/with-property/with-by branching is reasonably front-loaded. However, the text is a single dense run-on with nested parentheticals that makes it hard to parse and re-read while invoking the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter tool with no annotations and no output schema, the description covers most of what an agent needs: result shape for the no-property and with-property modes, dimension semantics for app screens, filters, segments, comparison ranges, and self-hosted caveats. Missing are the limit parameter's effect and any note on result caps or ordering, which are the main remaining gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 58%, so the description must compensate, and it largely does: it explains by, key (for trait), goal (per-row goal counts), compare formats including the date-range pattern, filters value syntax with an example, segment lookup, and experiment for variant. The limit parameter is never explained, and the interplay of days with from/to is only loosely implied ('Range: days (last N days) or from / to').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly establishes a read-only analytics report over properties, sites, or apps, returning totals, goals, topPages, and topSources. It distinguishes itself from experiments and implicitly from funnels/paths/retention by describing the shape of the returned visit metrics. It does not explicitly name the sibling analytics tools, so differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives useful branch conditions: omit property to list every property with a 30-day summary, supply property for full reports, use by to group rows, compare for period deltas, segment to apply a saved segment. However, it never says when to prefer splithub_analytics_funnel, splithub_analytics_paths, splithub_analytics_retention, or splithub_analytics_realtime over this tool. The warning 'These are counts, not experiments: no verdicts' is a valuable usage constraint but is the only explicit exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_connectA

Set up analytics in this app: use a property (id), create one (name, or nothing when the project has none), then name it on the one / in the code (property="sh-…"). Answers property, created, wired { status: added | already | other | none | many, file, line }, snippets { provider, script (sh.js tag with data-property), track, identify } and next. status none or many: add the provider snippet yourself (or the sh.js tag on a site without React). other: the provider names another property; ask before calling again with force. Pageviews are sent for you; events are track(name, props). In an app, script is empty and snippets.screen shows useScreen for screens the navigation tracker does not see. In cloud mode a new property is created in the cloud at once (live: applied), so no push: deploy.

NameRequiredDescriptionDefault
nameNocreate a new property with this name
forceNoreplace another property already named on the provider
propertyNoproperty id to use (added to the config when new)

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well by disclosing return fields, status behaviors, cloud-mode automatic creation, and that pageviews are sent automatically. It does not mention permissions or error behavior, but it adds substantial context beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and all sentences appear relevant, but the dense, fragment-heavy paragraph is harder to parse than ideal. It earns its length given the tool's complexity, though structure could be improved.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex setup tool with no annotations or output schema, the description covers return values, statuses, snippets, and next steps sufficiently. Minor gaps remain around error handling and full 'next' semantics, but an agent has enough to invoke and interpret the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters. The description reinforces parameter roles and adds a caution about 'force', but does not provide significant additional semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: 'Set up analytics in this app', and outlines the setup steps involving a property and provider. It is distinguishable from sibling analytics tools, though it does not explicitly contrast itself with splithub_property_create or splithub_create_property.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides operational guidance for property selection, creation, and wiring, including how to handle statuses like 'none', 'many', and 'other', and when to ask before using 'force'. It lacks explicit when-not-to-use guidance or direct alternative tool references.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_eventsB

The latest raw pageviews and custom events of an analytics property (cloud: the raw tail, 7 days free / 30 Pro; self-hosted: the user's database). Use to check that tracking works after wiring track() or sh.js.

NameRequiredDescriptionDefault
limitNo
propertyYes

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description still discloses meaningful behavior: the data source differs by deployment (cloud raw tail vs self-hosted user database) and retention is 7 days free / 30 Pro, which directly limits what results can exist. It omits read-only confirmation, ordering, and whether the result is capped by the limit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, action and scope front-loaded, with the deployment caveat parenthesized where it belongs. The 'raw tail' phrasing is slightly cryptic but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations, so the description should say more about the return shape (event list vs counts, fields, ordering) and the property identifier format. It covers the deployment/retention model well but leaves the agent guessing on what it will actually receive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry parameter meaning. It identifies 'property' only as an analytics property (no format: ID, slug, or name) and never addresses 'limit' at all, despite the schema capping it at 200. 'Latest' hints at a tail but does not explain the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific resource and scope: 'the latest raw pageviews and custom events of an analytics property', with 'raw' contrasting against the aggregate siblings (splithub_analytics, _funnel, _paths, _retention). It does not explicitly name those siblings, but an agent can tell this is the event-level tail rather than an aggregated report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives one concrete scenario: 'Use to check that tracking works after wiring track() or sh.js.' That is clear positive guidance, but there is no indication of when to prefer splithub_analytics_realtime or splithub_experiment_events instead, and no exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_funnelA

Funnel of an analytics property (SplitHub Cloud): visitors who reached each step in order. Steps: page:/path (patterns like page:/blog/* work), event:, goal:; 2 to 8. Or funnel: a saved funnel's name or id (splithub_analytics_funnels), which brings its steps and within. within: visit (same visit, default), 7d or 30d (from the first step). Answers steps[] with visitors, conversion (share of the first step) and dropoff (share lost since the step before); by splits it by a visit dimension. rolledUp: days older than 90 count nothing. Counts, not an experiment: never call a winner from it.

NameRequiredDescriptionDefault
byNo
toNo
daysNo
fromNo
stepsNoe.g. ["page:/", "event:Signup", "goal:paid"]; omit with funnel
funnelNoa saved funnel name or id
withinNo
filtersNodimension → comma-separated values
segmentNosaved segment name or id
propertyYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the load and does well: it discloses the return shape (steps[] with visitors, conversion, dropoff), what 'by' does (splits by a visit dimension), and the non-obvious retention caveat that days older than 90 count nothing. Missing only operational details like auth or limits, and no explicit statement that this is read-only.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core definition, then progressively tighter syntax notes and caveats; no filler sentences. It is telegraphic and dense, which costs a little readability, but each clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter, nested-object tool with no output schema, the description covers the tricky parameters (steps/funnel/within/by) and even describes the response structure. Date-range parameters, filters, and segment are left to the thin schema, so it is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 40%, so the description must compensate and largely does: it explains step syntax and the 2–8 bound, the page:/blog/* pattern, the funnel alternative, the within semantics and what 'from the first step' means, plus 'by' and rolledUp. It says nothing about from/to/days interplay or filters/segment semantics, leaving real gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific analytical object (ordered-step funnel over visitors) for a named resource (an analytics property), and implicitly distinguishes it from sibling path/retention tools by describing 'reached each step in order' rather than free navigation. An agent can tell it apart from splithub_analytics_paths and splithub_analytics_funnels from the text alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing: supply inline 'steps' OR a saved 'funnel' (pointing at splithub_analytics_funnels), and states the default for 'within' (visit). It also warns off the wrong use case ('never call a winner from it'). It doesn't contrast against splithub_analytics_paths/retention, so not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_funnelsA

Saved funnels of an analytics property (SplitHub Cloud), shown on its Funnels tab for the whole team. Without save or delete: the list (id, name, steps, within). save + steps: save one under that name (a name already saved updates it; steps and within as in splithub_analytics_funnel). delete: remove one by id or name. Run one with splithub_analytics_funnel { funnel }.

NameRequiredDescriptionDefault
saveNoname of the funnel to save or update
stepsNoe.g. ["page:/", "event:Signup", "goal:paid"]
deleteNoid or name to delete
withinNo
propertyYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It usefully discloses the list fields, that saving under an existing name updates it, and that delete removes by id or name, but it omits permission/auth requirements, whether deletion is reversible, and return behavior for save/delete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the resource, then immediately moves through list, save, delete, and run behavior. Every sentence earns its place with no redundant explanation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a multi-operation mutation tool with no annotations and no output schema, the description gives adequate call routing and discloses list return fields. It still leaves important context implicit, especially safety around deletion, permissions, and the meaning of the required property parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 60%, with schema descriptions for save, steps, and delete but not for within or property. The description adds valuable operation mapping and the update-on-existing-name behavior, plus points to splithub_analytics_funnel for steps/within semantics, but it does not explain the within enum values or the required property parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the specific resource (saved funnels of an analytics property) and covers the list/save/delete modes, including what each mode does. It also distinguishes this from splithub_analytics_funnel, which runs a saved funnel rather than managing the saved list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly maps parameter combinations to operations: without save or delete it lists, save plus steps saves or updates, and delete removes. It also names the sibling tool to use for running a funnel, leaving no ambiguity about tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_pathsC

Where visits went right after a page (or, with direction prev, where they came from), within the visit (SplitHub Cloud). rows[] { value, visits }; (exit) = left the site, (entrance) = landed on the page.

NameRequiredDescriptionDefault
toNo
daysNo
fromNo
pageYesa path like /pricing or /blog/*
filtersNodimension → comma-separated values
segmentNosaved segment name or id
propertyYes
directionNo

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It usefully discloses the return shape (rows[] { value, visits }) and the special (exit)/(entrance) markers, but says nothing about permissions, pagination, result limits, or default time windows for an 8-parameter analytics query. Useful but far short of what a no-annotation tool needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the core question, but the return-shape fragment and nested parentheticals make it dense and slightly awkward to parse. Every element is relevant, but the packing of three ideas into one sentence hurts readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no output schema, and no annotations, this is a fairly complex tool. The description covers the return shape and one enum, which is valuable, but omits date-range semantics, limits (days max 400), and filter/segment behavior — leaving real gaps for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 38%, so the description must compensate. It explains the direction enum ('with direction prev, where they came from') and implies page path formats, but says nothing about property, from/to/days date-scoping behavior, filters, or segment — leaving most parameters undocumented in both schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific analytical question — where visits went (or came from) relative to a page, scoped to the visit — which is a clear verb-like operation on a defined resource. It doesn't explicitly name sibling tools like splithub_analytics_funnel or splithub_analytics_retention to differentiate, but the path/next-page semantics are distinct enough to identify the operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical 'with direction prev' hints at a usage variant, but there is no guidance on when to choose this tool over funnel, retention, or events siblings, nor any prerequisites (auth, property requirements). The agent is left to infer when this analysis is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_realtimeB

Who is on an analytics property right now (SplitHub Cloud): activeVisitors5m, pageviews and events per minute over the last 30 minutes, top pages and sources. Use to confirm tracking works after a deploy.

NameRequiredDescriptionDefault
propertyYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It does disclose the meaningful trait an agent needs — that data is scoped to a fixed 30-minute rolling window with a 5-minute active-visitor metric — but says nothing about auth requirements, rate limits, or whether an empty result means 'not tracking' versus 'no traffic'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The return payload is front-loaded in the first clause, followed by the single usage tip — compact and waste-free. The metric list is dense but each item earns its place by telling the agent what it will get back.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description does well to enumerate returned metrics, so the agent knows the response shape. It remains incomplete on the one input parameter's format and on the semantics of an empty/zero-activity response, both of which matter for a tracking-verification tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and there is one required parameter, 'property', whose expected form (numeric property ID vs. domain vs. display name) is never clarified. The description mentions 'analytics property' but adds no lookup or formatting guidance, leaving the agent to guess what string to pass.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and scope — 'who is on an analytics property right now' — and enumerates the returned metrics (activeVisitors5m, pageviews/events per minute over 30 minutes, top pages and sources). The 'right now' framing implicitly separates it from the historical siblings like splithub_analytics and splithub_analytics_funnel, but no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

One concrete use case is given ('confirm tracking works after a deploy'), which implies when to reach for it. However, there is no guidance on when NOT to use it, nor any pointer to the non-realtime analytics siblings, so the agent must infer the boundary from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_analytics_retentionA

Retention cohorts of an analytics property (SplitHub Cloud), the key report for apps: users grouped by the day or week they started (start: visit, event: or goal:), and how many came back (return: visit or event:) in period 0, 1, 2… cohorts[] { start, users, retained[] }; retained[0] is the cohort period itself, retained[i] / users is the share. Ranges longer than 90 days / 26 weeks are cut (clamped: true). Default: start visit, return visit, period week. filters take platform (ios, android, web…) and app_version too. Counts, not an experiment.

NameRequiredDescriptionDefault
toNo
daysNo
fromNo
startNovisit | event:<name> | goal:<key>
periodNo
returnNovisit | event:<name>
filtersNodimension → comma-separated values
segmentNosaved segment name or id
propertyYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does substantial work: it discloses the clamping behavior (over-90-day / over-26-week ranges are truncated and flagged clamped: true) and the meaning of retained[0] and the retained[i]/users ratio. It omits permission/authentication requirements and cost/latency expectations for a potentially heavy cohort query.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense but well-ordered paragraph: resource, then start/return semantics, then output shape, then clamping caveat, then defaults and filters. No filler sentences, and the most decision-relevant facts (what it is, what it returns, what gets truncated) come first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter cohort tool with no output schema, the description supplies the response structure, defaults, and truncation rules, which is most of what an agent needs. Remaining gaps are the undocumented segment parameter and the ambiguity between days and from/to that the description does not resolve.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 44% across 9 parameters, but the description explains the two most confusing ones (start, return value grammars), the period default, and what filter dimensions are valid. It says nothing about segment, days, or how from/to interact with days (and days allows 400 while the description says ranges are clamped at 90 days), leaving real gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (retention cohorts of an analytics property) with concrete semantics: users grouped by start (visit, event:<name>, goal:<key>) and how many returned in period 0, 1, 2… It even sketches the response shape (cohorts[] { start, users, retained[] }) and explicitly excludes experiment reporting ('Counts, not an experiment'), which distinguishes it from splithub_results/experiment_* siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete defaults (start visit, return visit, period week) and states what filters accept (platform, app_version), plus an explicit exclusion ('not an experiment'). It does not directly name when to prefer this over splithub_analytics_funnel or splithub_analytics_paths, so the sibling routing is left partly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_create_experimentA

Old name of splithub_experiment_create, kept for one release: call splithub_experiment_create instead.

NameRequiredDescriptionDefault
idYesexperiment key, lowercase
toNoredirect experiments: variant → destination path
endNoISO date the experiment ends
goalNothe goal: any event name the app sends with track() (signup, Video Played) or a goal key from the config; an event not in goals becomes a goal, nothing to declare first
kindYes
nameNohuman name shown instead of the key; can be renamed later, the key stays
layerNomutual exclusion: experiments in one layer never share a visitor; bucket ranges [start, end) must not overlap
matchNoredirect experiments: the path it runs on, e.g. /
startNoISO date the experiment starts
statusNo
valuesNovalue experiments: variant → value
factorsNomvt only: factor → level → value, e.g. { headline: { old: "Old", new: "New" }, button: { blue: "#2563eb", green: "#16a34a" } }; first level = control; at most 4 factors × 4 levels
minDaysNoopt-in fewest days before a verdict (default 0 = none); verdicts otherwise depend on sample size
trafficNopercent of visitors in the experiment (default 100)
weightsNorelative weights, same order as variants (not for mvt)
variantsNovariant keys, first = control (not for mvt)
secondaryNogoals (event names or goal keys) to watch next to the primary goal
goodSampleNototal visitors at which verdict.sample.quality turns good (default 5000); informational, never changes the verdict
guardrailsNogoals (event names or goal keys) that must not get worse; { goal, worse: "higher" } when a rise is bad (refunds, errors). A breach makes the verdict guardrail_breached: do not ship
descriptionNo
maxVisitorsNostop counting after this many visitors

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations and 21 parameters, the description carries the whole disclosure burden. It usefully discloses the deprecation window ('kept for one release'), but doesn't say whether calls still succeed, whether they behave identically to the replacement, or whether a deprecation warning is returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, zero waste, front-loads the identity ('old name of...') before the redirect. Ideal shape for a deprecation notice.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deprecated alias with a richly documented schema and no output schema, the description is sufficient: it identifies the tool and redirects. A note on whether the alias is byte-for-byte equivalent to the replacement would make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 86%, so the schema already explains nearly all 21 parameters in detail. The description adds nothing parameter-specific, which is acceptable for a pure deprecation stub where the caller should be using the replacement anyway.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states exactly what this tool is (a deprecated alias of splithub_experiment_create) and names the replacement sibling, so an agent can route away from it immediately. It stops short of restating the underlying operation (creating an experiment), but the sibling name conveys it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit and unambiguous: 'call splithub_experiment_create instead' tells the agent precisely when not to use this tool and which alternative to pick. No inference required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_create_propertyA

Old name of splithub_property_create, kept for one release: call splithub_property_create instead.

NameRequiredDescriptionDefault
idNoomit to get a generated id
kindNoapp: a mobile or desktop app (screens, versions, retention); default: app in an app project, else site
nameNo
goalsNo
statusNo
descriptionNo

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description says nothing about whether invocations still succeed, whether it is a mutation with side effects, or how it behaves differently from the target. Only the deprecation itself is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; the deprecation warning and the redirect come first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deprecation shim the redirect is sufficient to prevent misuse, but with 6 low-coverage parameters, no annotations, and no statement of whether the call still works or errors, it leaves gaps for an agent that does invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 33% (name, goals, status, and description are undocumented), and the description adds no parameter meaning at all. Since it defers to the sibling, an agent gets no help from this text on how the six parameters behave.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies this as the legacy alias of splithub_property_create, so an agent can infer it creates a property. It doesn't state a verb-and-resource directly, but the reference to the canonical tool and the sibling list make it unambiguous which capability it wraps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the replacement tool and directs the agent to call splithub_property_create instead, with a stated lifetime ('kept for one release'). This is exactly the when-not-to-use / use-alternative routing an agent needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_doctorA

Check the installation: config, provider, middleware, routes, code references, endpoint; in a React Native / Expo app also the package, its storage module and redirect experiments; in a native iOS / Android project the SDK in the build files. Run after wiring code; ok: false lists what to fix.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden. It does disclose a behavioral contract ('ok: false lists what to fix'), which tells the agent this is a read/diagnostic operation and how failure is surfaced. However, it omits whether it mutates anything, what a passing response looks like, or any auth/runtime preconditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action ('Check the installation') followed by scoped enumerations, ending with the actionable 'ok: false' hint. Dense but each clause (framework-specific checks, timing cue, output contract) earns its place; only the long run-on structure is a minor weakness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description does reasonable work by hinting at the ok:false contract, but for a multi-target diagnostic it remains thin on what a successful result contains and how findings are structured. Adequate to attempt the call, incomplete for interpreting results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters (schema coverage 100%), so the baseline is 4. Nothing about parameters needs explaining, and the description correctly avoids inventing parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Check the installation') and enumerates the concrete targets (config, provider, middleware, routes, code references, endpoint) plus framework-specific variations. This is far more specific than a tautology, though it never names or distinguishes itself from likely-adjacent siblings like splithub_status or splithub_whoami.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides one lifecycle cue ('Run after wiring code'), which is a genuine usage signal, but offers no exclusions or alternatives among the 35 siblings, and does not clarify how it differs from related diagnostic tools such as splithub_status or splithub_whoami. Usage is implied rather than fully guided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_domainsA

Allowed domains, per experiment and per analytics property (SplitHub Cloud). Without arguments: every allowed domain with where it applies, which experiments / properties accept any origin, and the origins seen sending events. With kind + id + hosts: replace that experiment's or property's list (subdomains included; an empty list accepts any origin again). Events from an origin outside the list are dropped (403 when nothing in the batch is allowed).

NameRequiredDescriptionDefault
idNo
kindNoexperiment or property (test is an old name for experiment)
hostsNoe.g. ["example.com"]; [] clears the list

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that replacement is destructive, that subdomains are included, that an empty list resets to accept any origin, and that disallowed origins are dropped with 403 behavior. It does not mention auth or rate-limit considerations, but the core mutation and enforcement behavior is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but efficient: it front-loads the resource, then states the no-argument behavior and the argument-based replacement behavior. Every sentence adds useful information and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description explains what the no-argument form returns: all allowed domains, where they apply, which experiments/properties accept any origin, and origins seen sending events. It does not describe the response for the replacement mode, but the operational behavior is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, and the description meaningfully supplements it by explaining that kind + id + hosts together trigger replacement, that hosts includes subdomains, and that an empty hosts list clears the restriction. The id parameter is only implicitly defined as the target experiment/property, which keeps this from being a perfect score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies the resource (allowed domains per experiment/property) and clearly distinguishes its read mode from its replace mode. It is specific enough that an agent can separate it from sibling tools such as splithub_status or splithub_analytics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states the condition for listing ('Without arguments') and the condition for replacing ('With kind + id + hosts'). This gives the agent clear routing rules without needing to infer from the schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_experiment_createA

Add an experiment to splithub.config.ts (checked by reloading the file). It is written as running: it goes live when the code is deployed (and pushed in cloud mode). Tell the user that, and that splithub_experiment_status paused holds it. Pass status draft only when the user asks to hold it. kind: value (text/number/URL read with useValue), component (/ markup), redirect (match path + per-variant destination), mvt (factors instead of variants, every combination compared, read with useValue as { factor: value }). First variant / first level is the control. In an app (React Native, Expo) redirect does nothing, and the config ships inside the build: the variants' code needs a release, then starting, pausing and weights apply live from the cloud; target new builds with an app_version gte condition in the flow. goodSample: total visitors at which the sample counts as good (default 5000). minDays: opt-in fewest days before a verdict (default 0 = none). Then wire the returned snippet, call splithub_doctor and splithub_refs. The loop: splithub_plan → splithub_experiment_create → wire the code → splithub_doctor → splithub_push (cloud) → splithub_results (re-check after daysToVerdict days) → splithub_experiment_status finished → splithub_ship.

NameRequiredDescriptionDefault
idYesexperiment key, lowercase
toNoredirect experiments: variant → destination path
endNoISO date the experiment ends
goalNothe goal: any event name the app sends with track() (signup, Video Played) or a goal key from the config; an event not in goals becomes a goal, nothing to declare first
kindYes
nameNohuman name shown instead of the key; can be renamed later, the key stays
layerNomutual exclusion: experiments in one layer never share a visitor; bucket ranges [start, end) must not overlap
matchNoredirect experiments: the path it runs on, e.g. /
startNoISO date the experiment starts
statusNo
valuesNovalue experiments: variant → value
factorsNomvt only: factor → level → value, e.g. { headline: { old: "Old", new: "New" }, button: { blue: "#2563eb", green: "#16a34a" } }; first level = control; at most 4 factors × 4 levels
minDaysNoopt-in fewest days before a verdict (default 0 = none); verdicts otherwise depend on sample size
trafficNopercent of visitors in the experiment (default 100)
weightsNorelative weights, same order as variants (not for mvt)
variantsNovariant keys, first = control (not for mvt)
secondaryNogoals (event names or goal keys) to watch next to the primary goal
goodSampleNototal visitors at which verdict.sample.quality turns good (default 5000); informational, never changes the verdict
guardrailsNogoals (event names or goal keys) that must not get worse; { goal, worse: "higher" } when a rise is bad (refunds, errors). A breach makes the verdict guardrail_breached: do not ship
descriptionNo
maxVisitorsNostop counting after this many visitors

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses the file write and reload check, that the experiment is written as running and only goes live on deploy/push, that status paused holds it, and that in an app build redirect is a no-op and variant code requires a release while start/pause/weights apply from the cloud. That is exactly the non-obvious behavior an agent would otherwise get wrong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded and information-dense, but it is a single run-on block mixing purpose, parameter semantics, app caveats and a workflow chain, with no structural grouping. The goodSample and minDays sentences largely restate schema text (and drop the schema's more useful 'informational, never changes the verdict'), so they do not fully earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 21-parameter mutation with no annotations and no output schema, the description covers the essential: what is written, when it takes effect, app-mode limitations, and that a snippet is returned and must be wired. It could go further on failure modes (duplicate key, invalid layer ranges) and on how the snippet is consumed, but the schema carries the parameter detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 86% so the baseline is 3, and the description earns above that by explaining the otherwise opaque kind enum ('value ... read with useValue', 'component markup', 'redirect = match path + per-variant destination', 'mvt = factors instead of variants, every combination compared') and by defining control selection ('First variant / first level is the control'). It leaves unaddressed parameters (layer, guardrails, traffic, maxVisitors) entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+location: 'Add an experiment to splithub.config.ts (checked by reloading the file)'. This tells the agent exactly what artifact is mutated, but it never distinguishes itself from the sibling splithub_create_experiment, whose name suggests the same action — an ambiguity the description leaves unresolved.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Strong conditional guidance: 'Pass status draft only when the user asks to hold it', the app-mode caveat, and the explicit lifecycle 'splithub_plan → splithub_experiment_create → wire the code → splithub_doctor → splithub_push' with follow-up calls named. It stops short of the sibling-differentiation case (splithub_create_experiment) and does not say what to do if the experiment key already exists.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_experiment_eventsA

The latest raw events of one experiment (SplitHub Cloud, newest first): exposures (type exposure, the variant a visitor saw) and goals (type goal) with time, visitor, url, country. Same from / to / filters as splithub_results; type keeps one kind; nextBefore pages to older ones (pass it as before). Raw events stay 7 days (30 on Pro); the key needs events:read. Use to check that exposures and goals arrive after wiring code.

NameRequiredDescriptionDefault
toNo
fromNo
typeNo
limitNo
beforeNonextBefore of the previous page (unix ms)
filtersNovisitor column → comma-separated values
experimentYesexperiment id (key)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses retention ('raw events stay 7 days, 30 on Pro'), required scope ('key needs events:read'), ordering (newest first), and pagination mechanics (nextBefore pages older, pass as before). It does not mention rate limits or error behavior, keeping it below a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense but front-loaded paragraph: what it returns first, then filters/paging, then retention/auth, then the use case. Every clause is informative, though the telegraphic phrasing ('Same from / to / filters as splithub_results') is slightly compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with a nested filters object, no annotations and no output schema, the description covers purpose, filter/paging semantics, retention window, required scope and event shape. An agent has enough to call it correctly and interpret what comes back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 43%, so the description must compensate, and it does: it explains that type keeps one kind, that before is fed by nextBefore from the previous page, and that from/to/filters behave identically to splithub_results. Only limit is left undocumented in both places, a minor gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (raw events of one experiment), names the two event kinds returned (exposure, goal) and the fields carried (time, visitor, url, country). The 'raw events' framing plus the pointer to splithub_results implicitly separates it from the aggregated sibling, so an agent can pick it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use to check that exposures and goals arrive after wiring code,' giving a concrete invocation context, and references splithub_results as the place to get the same from/to/filters for aggregated queries. It stops short of stating when NOT to use it (e.g., for conversion numbers), so it is clear context without full exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_experiment_statusA

Start, pause or finish an experiment. The config file is updated (and checked); in cloud mode it applies live within a minute. After finishing, the variants are still in the code: ask the user which variant to keep, then call splithub_ship.

NameRequiredDescriptionDefault
statusYes
experimentYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the full burden and does add real behavioral detail: the config file is updated and validated, and cloud mode propagates changes live within about a minute. It also discloses the important side effect that variants remain in the code after finishing. It does not mention permissions/auth, whether transitions are reversible, or what happens on invalid transitions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action, then behavioral notes, then the downstream instruction. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation tool with no annotations and no output schema, the description covers the main behavioral traits an agent needs: config mutation, propagation latency, and the required follow-up tool. The missing piece is disambiguating the experiment identifier and its relationship to the near-identical set_status sibling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description should compensate. It implies the status values through the verbs 'start, pause or finish', which loosely map to the enum running/paused/finished, but the mapping is not stated explicitly and the 'experiment' parameter (name vs. key vs. id) is never explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb set (start, pause, finish) and resource (experiment), which is much clearer than a tautology. However, the sibling splithub_set_status sounds like it does the same thing, and the description never distinguishes the two, leaving an agent to guess which tool to call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit workflow guidance for the post-finish path: ask the user which variant to keep, then call splithub_ship. That is a genuine routing instruction. It lacks any when-not guidance or comparison to splithub_set_status / splithub_experiment_create.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_initA

Set up SplitHub in this project: config file, framework files, agent skills; React Native / Expo apps also get @splithub/react-native and their entry (App.tsx, app/_layout.tsx) wrapped in with screen tracking (app.entry.status added | already | manual: wire nextStep by hand); native iOS / Android get install lines and a quick start (app.install, app.quickStart) and no config file; in cloud mode see splithub_login to connect. Use once per project. githubAction: also write a GitHub workflow (checks on PRs, results comment on a schedule).

NameRequiredDescriptionDefault
keyNopublic key pk_… when known
modeNocloud
githubActionNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full load and does well: it discloses concrete side effects (writing config/framework files, wrapping App.tsx and app/_layout.tsx in <SplitHubProvider>, adding screen tracking, writing a GitHub workflow). It does not cover auth requirements or exact permissions, leaving a minor gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, but the body is one sprawling run-on sentence stuffed with parentheticals and pipe-delimited status values (added | already | manual: wire nextStep by hand), which hurts scannability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and only 33% schema coverage, the description is behaviorally rich but leaves real gaps: what changes under mode='self', whether key is required in self mode, and what the tool returns. Adequate for a scaffolding tool but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Coverage is only 33%. The description explains githubAction well ('also write a GitHub workflow (checks on PRs, results comment on a schedule)') and touches cloud mode, but never explains the 'self' enum value or when 'key' is needed beyond the schema's own 'public key pk_… when known'. Useful but incomplete compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Set up SplitHub in this project') and then enumerates exactly what it creates: config file, framework files, agent skills, React Native wiring, native install lines. It also differentiates from siblings by directing cloud-mode users to splithub_login.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use once per project' and routes cloud-mode connection steps to splithub_login. It gives clear context for invocation but no explicit when-not-to-use it or what to do if already initialized (though 'app.entry.status added | already' implies re-run safety).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_loginA

Connect this folder to SplitHub Cloud. Returns a sign-in link and a code: show them to the user, who approves in the browser (and can create a project there). The CLI then stores a project API key for this folder. Call splithub_whoami afterwards.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that the tool returns a link plus a code, that a human must approve in a browser, that a project may be created there, and that an API key gets persisted for the folder. This is meaningful side-effect disclosure well beyond the name. It omits failure modes such as code expiry or behavior when already logged in.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the action and its output before the follow-up instruction. Every sentence conveys information the agent needs — nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Since no output schema exists, the description usefully covers what is returned and what the caller must do with it, plus the mandatory next call to splithub_whoami. A brief note on failure/expiry behavior would make it fully complete for an auth handshake tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so there is nothing for the description to disambiguate; baseline for a no-param tool is 4. The description adds no misleading parameter language.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Connect this folder to SplitHub Cloud') and explicitly enumerates the return artifacts (sign-in link and code). It is clearly distinguishable from siblings like splithub_whoami, splithub_status, or splithub_init, which the description implicitly positions as follow-ups rather than substitutes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear triggering condition ('Connect this folder') and an explicit follow-up step ('Call splithub_whoami afterwards'), which is genuinely useful sequencing guidance. It stops short of stating when NOT to use it (e.g., already authenticated, or already have an API key), so it falls just short of the top mark.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_notesA

Notes: what happened on a day (a launch, an outage, a campaign), shown on experiment timelines and analytics charts (SplitHub Cloud). action list (default): newest first; experiment or property narrows it (project-wide notes come too), kind filters, days / from / to set the range. add: text (day defaults to today in the project time zone; kind note | issue | win; experiment or property ties it to one, none = the whole project). edit: id + text / kind / day. delete: id. Add one when you ship something that may move the numbers.

NameRequiredDescriptionDefault
idNonote id (edit, delete)
toNo
dayNo
daysNo
fromNo
kindNo
textNo
actionNo
propertyNoanalytics property id
experimentNoexperiment key

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the burden. It discloses that add defaults day to today in the project time zone and that omitting experiment/property makes the note project-wide — useful behavioral context. However, it never states that delete is irreversible, whether add/edit/delete require confirmation, or what a successful mutation returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Definition is front-loaded, then actions are presented in a compact per-action order with zero filler sentences. It is dense to the point of being terse, but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter, multi-mode tool with no annotations and no output schema, the description covers action selection and parameter semantics adequately but omits the return shape (what a list yields, what a mutation returns) and any failure/permission behavior. The agent knows how to call it but not entirely what to expect back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 30%, so the description compensates well: it maps every parameter to an action (id+text/kind/day for edit, id for delete, experiment/property/kind/days/from/to for list) and adds defaults (day = today in project time zone, kind enum values). The date-range and project-wide semantics go beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete definition of the resource (day-level notes shown on experiment timelines and analytics charts) and then enumerates the four actions (list/add/edit/delete) with their individual behavior. It is specific about verb+resource, though it doesn't explicitly name a sibling tool it differs from.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the default action (list), explains how to narrow with experiment/property/kind/days/from/to, and gives an explicit trigger for the write path: 'Add one when you ship something that may move the numbers.' It stops short of naming alternatives or when-not-to-use conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_planA

Before creating an experiment: visitors per variant (and in all) to detect a relative lift mde on a baseline conversion rate, sized by SplitHub Cloud for its sequential verdict (no account needed; fails with UNREACHABLE offline). With an experiment id, also the days left at its current traffic when its numbers are readable. If it takes months, suggest a bigger change or a busier page.

NameRequiredDescriptionDefault
mdeYessmallest relative lift worth finding, e.g. 0.1 for +10%
baselineYesconversion rate today, e.g. 0.05
variantsNonumber of variants including the control (default 2)
experimentNoexperiment id: adds days at its traffic

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does so: it discloses that sizing is performed by SplitHub Cloud, that no account is needed, and that it fails with UNREACHABLE when offline. It also frames the sequential verdict context. Missing only rate-limit/timeout behavior for the remote call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the precondition 'Before creating an experiment', which is good. However the middle sentence is a run-on with stacked parentheticals ('(and in all)', '(no account needed; fails with UNREACHABLE offline)'), making it harder to parse than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter tool with no annotations and no output schema, the description tells the agent what comes back (visitors per variant and total, optional days-left) and what failure looks like. Adequate, though it doesn't state the return shape explicitly enough to rely on without an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with unit examples already in the schema ('0.1 for +10%', '0.05'), so parameter meaning is largely carried by structured data. The description adds the aggregate-vs-per-variant distinction and the 'days at current traffic' effect of `experiment`, but not much beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific function: sizing visitors per variant (and total) to detect a relative lift `mde` against a `baseline` rate, plus optional days-left for an experiment id. That verb+resource is distinguishable from creation/status siblings, though the phrasing 'visitors per variant (and in all) to detect a relative lift' is dense enough to require a re-read.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Before creating an experiment' gives an explicit temporal condition for use, and 'if it takes months, suggest a bigger change or a busier page' guides interpretation. It does not name a sibling alternative (e.g. splithub_experiment_create) by name, so routing is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_projectsA

Every folder on this machine connected to a SplitHub Cloud project, the current one marked (current).

NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden. It does add genuine behavioral context by disclosing a return-value trait — the current project folder is annotated '(current)' — and listing implies a non-destructive read. However, it says nothing about ordering, response format, or what happens when no projects are connected.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with no padding; the '(current)' marker is the useful payload and it lands early. It loses a point for being a fragment without a leading verb, which slightly weakens front-loading of the action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-param, no-annotation, no-output-schema tool, the description does state what the result set contains, which is the main thing an agent needs. It leaves open the shape of the entries (paths vs. names) and the empty-state behavior, so it is adequate rather than complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and the schema is an empty object at 100% coverage, so there is nothing for the description to compensate for. Baseline 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies a specific resource — folders on this machine linked to SplitHub Cloud projects — which is clearly distinct from siblings like splithub_status, splithub_init, or splithub_whoami. It is phrased as a noun fragment rather than a verb+resource ('lists...'), so the action is inferred rather than stated, but an agent can still tell what it returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the '(current)' marker hints this is for discovering or confirming which project folder you are working in, but the description never says when to call it versus splithub_status or splithub_init. No exclusions or alternatives are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_property_createA

Add an analytics property to splithub.config.ts, checked by reloading the file. Its id is generated (sh-x1h78w) unless given; it comes back as property. Events go only to a property named explicitly: wire (pageviews on load and every route change, and every track() call) or data-property="" on the sh.js script tag. Every goal of the project counts as a conversion unless goals narrows them. In cloud mode it is created in the cloud at once (live: applied), so no push is needed: tell the user to deploy. If live names a missing API key, log in and call again (or push). In an app (React Native, Expo) screens are page events: the screen tracker or useScreen(name) sends them.

NameRequiredDescriptionDefault
idNoomit to get a generated id
kindNoapp: a mobile or desktop app (screens, versions, retention); default: app in an app project, else site
nameNo
goalsNo
statusNo
descriptionNo

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the load and does so well: it discloses that the config file is mutated and reloaded for verification, that ids are generated (sh-x1h78w) unless supplied, what cloud mode does immediately without a push, and how to recover from a missing API key. It never states what happens if the id already exists or whether a name is required, which are the main remaining behavioral unknowns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is front-loaded in the first sentence, but the rest is a dense run-on that drifts into SDK wiring instructions (the SplitHubProvider snippet, data-property attribute) that are documentation for the integrating app rather than help in invoking this tool. Several parentheticals could be trimmed without losing selection signal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter, no-annotation, no-output-schema mutation, the description covers the write behavior and post-call steps well, but omits three parameters' semantics (name, status, description) and gives only a hint at the return shape ('comes back as property'). Adequate but with identifiable gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33%, so the description must compensate, and it does partially: it explains id generation ('unless given'), that it 'comes back as property', and that goals default to the whole project ('Every goal of the project counts as a conversion unless goals narrows them'). It says nothing about name, status, or description, and only implies kind via the React Native/Expo sentence, leaving half the parameters undocumented in both places.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

A specific verb and resource: 'Add an analytics property to splithub.config.ts, checked by reloading the file' names both the object and the persistence target. It does not, however, distinguish itself from the sibling splithub_create_property, which an agent will have to disambiguate on its own.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Operational guidance is present ('In cloud mode it is created in the cloud at once (live: applied), so no push is needed: tell the user to deploy'; 'If live names a missing API key, log in and call again (or push)'), which tells the agent what happens after the call. But there is no explicit statement of when to choose this over splithub_create_property or splithub_set_property_status – the closest sibling is never mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_property_statusA

Pause or resume an analytics property (paused: its pageviews and events are dropped and not billed). Updates splithub.config.ts; in cloud mode it applies live within a minute. archived (SplitHub Cloud) stops collecting and hides it like the dashboard Archive, numbers kept; restored brings it back paused. removed takes it out of the config (a dashboard-made property is deleted in the cloud with its numbers).

NameRequiredDescriptionDefault
statusYes
propertyYes

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it explains billing impact, config file updates, cloud propagation delay, archived behavior keeping numbers, restored behavior, and the destructive deletion caused by removed. These are exactly the side effects an agent needs before invoking a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded: it opens with the core action and then explains each status. Every clause carries useful behavioral meaning, though the long parenthetical structure could be slightly easier to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations or output schema, the description is nearly complete on behavior and status semantics. The main remaining gap is the meaning and expected format of the property parameter, plus any permission or auth prerequisites.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It thoroughly explains the status enum values, including running by implication, but never explains what the property parameter expects (ID, slug, path, etc.), leaving one of two required parameters underspecified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource — pausing/resuming an analytics property — and names the statuses it controls. It is clear what the tool does, though it does not explicitly differentiate itself from sibling tools like splithub_set_status or splithub_set_property_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The status semantics imply when to use each mode and the config/cloud behavior gives useful context. However, it never explicitly says when to choose this tool over similarly named siblings, so guidance remains inferred rather than direct.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_pullB

Fetch the cloud config and write dashboard-created experiments into splithub.config.ts.

NameRequiredDescriptionDefault

No parameters

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It says it writes into splithub.config.ts but not whether local edits are overwritten, merged, or preserved, nor auth requirements or failure behavior — a serious gap for a tool that mutates a local file.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tight sentence with the action and destination front-loaded and no filler. It is terse to the point of omitting the overwrite/merge behavior, but nothing present is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description covers the what but not the consequences: whether the local config is replaced or merged, and what the caller gets back. Adequate to identify the tool, insufficient to predict its effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the schema imposes no documentation burden and there is nothing for the description to disambiguate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific pair of verbs (fetch, write) and both resources: cloud config and dashboard-created experiments, landed into splithub.config.ts. The direction (cloud → local) is clear enough to separate it from the sibling splithub_push, though the description never names push explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The cloud-to-local direction is implied, so an agent can infer this is the sync-down step after dashboard edits, but there is no explicit 'use when' condition, no prerequisite (e.g. login/init), and no named alternative such as splithub_push.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_pushA

Upload splithub.config.ts to SplitHub Cloud (cloud mode). Also returns kept: experiments the cloud has from an earlier push that this file lacks, left as they are. prune removes those: only when the user asked for it. Returns experiments created in the dashboard that were written into the file. dryRun: only check the config and show what would change. A key that already belonged to another experiment fails with KEY_CONFLICT: ask the user, then push with replace (new experiment, old numbers cleared) or same (same experiment, numbers stay).

NameRequiredDescriptionDefault
sameNokeys that are the same experiment as before: the numbers stay
pruneNoalso remove experiments the cloud has that the file lacks (ask the user first)
dryRunNo
replaceNokeys that are new experiments: the old numbers under them are cleared

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: it discloses return contents (kept experiments, dashboard-created experiments written back into the file), the destructive effect of prune, the non-mutating dryRun mode, and the KEY_CONFLICT failure mode with its two resolutions and their differing effects on numbers. It does not mention authentication, permissions, or what happens on partial failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The primary action and return values are front-loaded, and every subsequent sentence conveys a distinct behavioral fact (kept set, prune, dashboard-created experiments, dryRun, KEY_CONFLICT). Some sentences are long and clause-stacked, which slightly reduces scanability, but there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with zero annotations and no output schema, the description supplies what an agent needs: what gets uploaded, what is preserved, what gets removed, how to preview safely, what comes back, and how to recover from the one named error. Missing only auth/permission prerequisites and idempotency behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All four parameters carry meaning from the description: prune, dryRun, replace, and same are each explained in terms of consequence ('numbers stay' vs 'old numbers cleared'). This is especially valuable for dryRun, which has no schema description. Coverage is 75% and the description closes the remaining gap rather than restating the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete verb and resource: upload splithub.config.ts to SplitHub Cloud, plus the 'cloud mode' qualifier. This is clearly distinct from read-side siblings like splithub_status or splithub_results. It does not explicitly compare itself to close neighbors such as splithub_plan, splithub_ship, or splithub_pull, so an agent must infer the division of labor.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit conditions for the optional flags: prune 'only when the user asked for it', dryRun 'only check the config and show what would change'. It also prescribes the escalation path on KEY_CONFLICT (ask the user, then choose replace or same). It never states when to choose this tool over splithub_plan or splithub_ship, which is the main gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_refsA

Files and lines that use an experiment id or its goal. Use after wiring code (is it referenced?) and after splithub_ship (count should be 0 for the experiment).

NameRequiredDescriptionDefault
experimentYes

Output Schema

NameRequiredDescription
goalYes
refsYes
countYes
inConfigYes
experimentYes
goalSharedYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden. It usefully discloses the return content (files and lines) and implies a count, but says nothing about scope (which directory/repo is searched), read-only safety, or permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with the core purpose front-loaded and the workflow guidance second. No filler, though the parenthetical phrasing is slightly informal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be explained, and the workflow context is present. However, an agent still cannot tell the search scope, whether uncommitted files count, or exactly what string to pass for the experiment-goal variant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the single parameter is undocumented in the schema, but the description compensates partially by indicating the input is an experiment id or its goal. It does not specify format or whether the goal is a free-text string.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource — 'Files and lines that use an experiment id or its goal' — which is clearly a code-reference lookup and distinguishable from siblings like splithub_ship or splithub_review. It is a noun fragment rather than a verb+resource statement, which slightly weakens it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives two concrete usage points: after wiring code to verify references, and after splithub_ship where the count should be zero, tying it explicitly to a sibling tool's workflow. It does not say when not to use it or name an alternative lookup tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_removeA

Remove an experiment from splithub.config.ts (and archive it in the cloud when connected) without touching the code. Prefer splithub_ship, which also cleans the code. Returns refs still in the code.

NameRequiredDescriptionDefault
experimentYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations the description carries the full burden, and it discloses meaningful traits: config-file removal, optional cloud archival when connected, that source code is left untouched, and the return payload ('refs still in the code'). It does not say whether removal is reversible or what happens if the experiment does not exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and file affected, then the preferred alternative, then the return value. Every sentence adds information with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter mutation with no annotations and no output schema, the description covers behavior and return value well, but leaves the identifier format and failure/irreversibility semantics unexplained. Adequate, with clear remaining gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the description never clarifies whether 'experiment' is a slug, display name, or ID, nor any format constraints. Only the generic implication that the argument identifies an experiment is conveyed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource ('Remove an experiment from splithub.config.ts'), plus the cloud-archival side effect, and explicitly contrasts with splithub_ship which also cleans code. An agent can distinguish it from all sibling tools without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Prefer splithub_ship, which also cleans the code' gives an explicit alternative and the condition that selects it. It lacks an explicit 'when not to use' or prerequisite statement, but the routing guidance is clear enough to act on.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_resultsA

Results of one experiment. SplitHub Cloud gives the full verdict: sample per variant, lift, P(beat), SRM, winner; mvt adds factors[]. verdict.status guardrail_breached means do not ship: a variant made a guardrail metric worse (verdict.guardrailBreached[]); say what broke. verdict.metrics[] has secondary metrics and guardrails. The experiment __holdout is the project holdout (long-term impact of all experiments). Verdicts are sequential (always-valid, safe to check any time) and depend on sample size, not days. verdict.sample.quality is small (below the minimums), ok, or good (every variant has its share of verdict.sample.goodSample, default 5000 visitors): say it next to the verdict; it never changes the decision. verdict.sample.stars (0..5) is a data-quality score from verdict.sample.checks[] (minimum, split, goodSample, power, week; each { met, label }): report the stars and the unmet labels, never as a reason to override the verdict. Quote verdict.message verbatim, use comparisons[].sequential.pValue / sequential.ci (the plain pValue is fixed-horizon, display only: p = 0.01 there can be not significant) and relay verdict.notes[]. daysToVerdict says when to check again. Facts are in verdict.message; voice.line is the friendly line for humans (voice.advice move_on = suggest finishing the experiment and starting the next one); display { label, lines, next } is ready to show. If analysis.available is false the project is self-hosted: report the raw numbers as they are, say verdicts come with SplitHub Cloud (analysis.url), and never call a winner from raw rates. If finished, ask which variant to keep, then splithub_ship. SLICE (SplitHub Cloud): from / to (YYYY-MM-DD, project time zone) keep the visitors first exposed on those days, filters keep visitors by country, device, platform (ios, android, web…), app_version, source, channel, landing page, UTM…; the answer carries scope and the verdict is computed on that slice only: say so and quote scope.note, the decision stays with the unfiltered verdict. A slice carries no voice and no advice (numbers only): never suggest shipping or finishing from one. VIEWS (one at a time, same from / to / filters; the answer has view set): by splits visitors by a dimension, segments[] each with cells per variant (order[] = variants, control first) and comparisons[] (lift vs the control inside the segment, pValue, significant, better; small = too few visitors); timeline gives exposures[] and conversions[] per day and variant (a day = visitors first seen that day, conversions whenever they happened); engagement gives bounce rate, pages per visit and avgVisit (ms) per variant from the analytics property on the same visitors (property null = no analytics there); events gives rows[] for every goal (measured goal / metric / guardrail first) and event of these visitors, each with perVariant visitors and rate and comparisons[] against the control (property null = goals only, other events need analytics on the same site). Segments, engagement and events are for exploring: never call a winner from one segment or one event.

NameRequiredDescriptionDefault
byNoview: lift per segment of this dimension
toNofirst-exposure day, to (inclusive)
fromNofirst-exposure day, from (inclusive)
eventsNoview: every goal and event per variant, with lift against the control
filtersNovisitor column → comma-separated values, e.g. { country: "DE,FR", channel: "Paid Search" }; "(none)" matches empty
timelineNoview: visitors and conversions per day and variant
engagementNoview: bounce rate, pages per visit, visit length per variant
experimentYesexperiment id (key)

Output Schema

NameRequiredDescription
goalNo
kindNo
nextNo
rowsNoview events: every goal and event of these visitors; perVariant { visitors, count, rate, revenue } and comparisons[] against the control (lift, pValue, significant); small = too few visitors
viewNoset when by, timeline, engagement or events was asked: the shape is that view, not a verdict
scopeNopresent on a slice (from / to / filters): the verdict is computed on that slice only; quote note
voiceNofriendly line for humans; quote verdict.message for facts. advice move_on = suggest finishing and starting the next experiment
statusNo
displayNoready-to-print label (WINNER, TOO EARLY, ...), lines and next step, for showing the result
verdictNoSplitHub Cloud only
analysisNoself-hosted: raw numbers only. Tell the user verdicts come with SplitHub Cloud; never invent one
variantsNoself-hosted: raw counts per variant (conversions, rate), control first; view engagement: sessions, pagesPerVisit, bounceRate, avgVisit (ms) per variant; view events: the variant keys, control first
experimentYes
daysToVerdictNodays until a verdict at the current traffic; schedule the next check then

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully carries the behavioral burden. It explains that verdicts are sequential (always-valid, safe to check any time), that guardrail_breached means do not ship, that sample quality never changes the decision, that stars are not a reason to override the verdict, that plain pValue is display-only while sequential pValue should be used, and that slices and views carry no voice or advice. This is unusually thorough for a results tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is dense and almost entirely front-loaded into a single unbroken block with no paragraph breaks or bullet structure. While many sentences carry non-trivial information, the sheer length and run-on style make it hard to parse quickly, and the sheer volume reduces readability for an agent scanning for the most relevant usage rule.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity—an experiment-results tool with slices, views, self-hosted fallback, voice/advice distinctions, and a rich output schema—the description covers all critical decision points: how to handle missing analysis, how to treat guardrail breaches, when to invoke splithub_ship, and how to interpret sequential vs plain p-values. Despite the dense formatting, no essential behavioral or interpretative guidance appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents parameters, making 3 the baseline. The description goes further: it clarifies that from/to refer to first-exposure days in the project time zone, that filters keep visitors by country/device/platform/app_version/source/etc., that timeline means 'a day = visitors first seen that day, conversions whenever they happened', and that engagement returns property null when no analytics exist. These additions meaningfully exceed the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence 'Results of one experiment' plus the detailed breakdown of verdict fields makes the tool's function unmistakable: it returns the full statistical verdict for a single experiment. However, it never explicitly names sibling tools like splithub_experiment_status or splithub_experiment_events to clarify why an agent would choose this over those, so it falls short of the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete usage guidance: when analysis.available is false, report raw numbers and never call a winner; when finished, ask which variant to keep and then use splithub_ship; for slices, views, and segments, never call a winner from them. It lacks explicit 'use this instead of X' comparisons with sibling result-related tools, but the when-to-use and when-not-to-use conditions are otherwise richly specified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_reviewB

AI reviews of an experiment stored in SplitHub Cloud (Pro): written at the start, when the verdict changes while it runs, when it finishes, or on demand in the dashboard. Newest first. Quote them as an opinion; the verdict stays the source of truth.

NameRequiredDescriptionDefault
experimentNoexperiment id; omit for every experiment of the project

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses that this is a Pro feature, returns newest first, and should be treated as opinion, but it omits permissions, read-only nature, rate limits, and response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and mostly efficient, with no wasted sentences. The first sentence is dense but still readable and purposeful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity retrieval tool, the description covers the nature and ordering of the reviews. However, with no output schema and no annotations, it stops short of describing the return structure or format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single optional experiment parameter is fully documented in the schema, including the omit-for-all behavior. The description adds no parameter-specific meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource as AI reviews for an experiment and explains the contexts in which those reviews are generated. It distinguishes the reviews from the verdict/status by stating the verdict remains the source of truth, though it does not use an explicit retrieval verb.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives important interpretive guidance: quote reviews as opinion and treat the verdict as authoritative. However, it does not explicitly say when to choose this tool over siblings like splithub_status or splithub_results.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_segmentsA

Saved segments: named report filters every report can apply (segment on splithub_analytics, funnel, paths, retention). Without arguments: the list. save + filters: save one (filters like { country: "DE,FR", channel: "Paid Search" }; property limits it to one property). delete: remove one by id or name. SplitHub Cloud.

NameRequiredDescriptionDefault
saveNoname of the segment to save
deleteNoid or name to delete
filtersNodimension → comma-separated values
propertyNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full burden: it does disclose that delete removes a segment (a destructive action) and that property-scoping limits the save to one property. However, it says nothing about permissions, whether deletion is reversible, name-vs-id resolution behavior, or what listing returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the concept, then mode-routing, then an example — dense and free of filler. Slightly telegraphic fragments and the closing "SplitHub Cloud." tagline are the only non-earning elements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter tool with no annotations and no output schema, the description covers invocation well but never describes what the list mode returns or what a delete/save responds with. Auth requirements and error conditions are also unaddressed, leaving a moderate but real gap for an unannotated mutation-capable tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75%, so the schema does some work, but the description adds real value by giving a concrete filters example ({ country: "DE,FR", channel: "Paid Search" }) that explains the dimension→comma-separated-values convention, and by clarifying that property constrains the save to a single property rather than being a generic string.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the resource precisely — "named report filters every report can apply" — and enumerates where they apply (analytics, funnel, paths, retention), which is more than a name restatement. It doesn't need sibling differentiation because no other segment-oriented tool exists, though it could have clarified it versus the report tools it feeds.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit mode dispatch: no args returns the list, save+filters saves one, delete removes one by id or name — so the agent can pick the right invocation without opening the schema. It stops short of exclusions or preconditions (e.g., what happens if save is given without filters, or how delete resolves id vs name collisions).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_set_property_statusA

Old name of splithub_property_status, kept for one release: call splithub_property_status instead.

NameRequiredDescriptionDefault
statusYes
propertyYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose the deprecation lifecycle ('kept for one release'), which is valuable context beyond the schema, but it does not say what actually happens if called — whether the alias forwards, errors, or silently no-ops — nor the effect of the status enum values on the target property.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with the deprecation notice front-loaded ahead of the redirect. Every clause earns its place; nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-release deprecation shim, the essential information an agent needs — that it is superseded and by what — is present. The remaining gap (whether it still functions, exact removal timing) is minor, though the absence of annotations and output schema makes it slightly more consequential.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% on two required parameters, so the description must compensate and adds nothing about 'property' or the meaning/lifecycle of the 'running'/'paused'/'archived'/'restored'/'removed' enum. The enum listing itself lives in the schema, not the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies this as the deprecated alias of splithub_property_status, which is enough for an agent to know what the underlying operation is (setting a property's status) and to distinguish it from the live sibling. It never states the verb+resource in its own words, relying on the name and the named successor, but the routing intent is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to call splithub_property_status instead, which is the only correct action for this tool. The when-not-use condition and the alternative are both named outright, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_set_statusA

Old name of splithub_experiment_status, kept for one release: call splithub_experiment_status instead.

NameRequiredDescriptionDefault
statusYes
experimentYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden; it does disclose the deprecation window and that this is an alias, which is useful behavioral context. It does not say whether the alias still performs the mutation identically or whether it warns/errors on invocation, leaving the actual runtime behavior unclear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence, front-loaded with the rename and the call-to-action. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deprecated shim, the agent mainly needs to know not to use it and where to go instead, and that is covered thoroughly. The missing piece is whether invocation still succeeds or fails during the one-release window.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description was expected to compensate, and it adds nothing about the experiment identifier or the running/paused/finished status enum. Only the schema itself conveys the accepted values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the tool as the former name of splithub_experiment_status, which lets an agent infer the operation from that sibling. It never directly states the verb+resource (setting an experiment's status), so the purpose is only reachable through the pointer rather than stated outright.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit directive: call splithub_experiment_status instead, and frames the lifespan as 'kept for one release.' There is no ambiguity about which tool to prefer among the siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_shipA

Keep one variant and take the experiment out: archives it in the cloud (numbers stay readable), removes it from splithub.config.ts and rewrites the code ( → the kept , useValue → the kept literal, useVariant branches folded, unused imports dropped). Ask the user which variant to keep first (recommend verdict.winner; never a variant listed in verdict.guardrailBreached). Call with dryRun: true, show the diff, then without. Fix leftovers[] by hand, then follow next[] (push, doctor, tests).

NameRequiredDescriptionDefault
keepYesvariant key to keep
dryRunNoonly return the diff
experimentYes

Output Schema

NameRequiredDescription
diffYesunified diff of the config and the source files
keptYes
kindYes
liveYes
nextYes
filesYes
dryRunYes
leftoversYes
configFileYes
experimentYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden and does well: it states that the experiment is archived in the cloud with its numbers preserved, that the config file is modified, and that source code is rewritten with specific transformations. It does not state whether the operation is reversible or what permissions are required, which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences that front-load the purpose, then the pre-call decision, then the dry-run/apply/post-call sequence. Parentheticals carry real detail (winner selection, rewrite targets) rather than filler; nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive multi-file operation this is complete: side effects, the variant-selection rule, the dry-run gate, and the follow-up steps are all present. Because an output schema exists, the references to leftovers[] and next[] are pointers rather than omissions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 67%, but the description compensates: it defines keep semantically ('which variant to keep', constrained to verdict.winner and never a guardrail-breached variant) and shows dryRun's role in a diff-then-apply sequence. The experiment parameter is left to the schema, but the important semantics are supplied.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a precise verb and resource and enumerates the concrete effects: cloud archive with numbers kept readable, removal from splithub.config.ts, and code rewrites (<Experiment> → <Variant>, useValue → literal, branch folding, import cleanup). This is far more specific than the title-less name. It never explicitly contrasts itself with the nearby splithub_remove, which is the one sibling an agent could confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It prescribes the whole workflow: ask the user which variant to keep, prefer verdict.winner and never a variant in verdict.guardrailBreached, run with dryRun: true and show the diff before the real call. It also tells the agent what to do after the call (fix leftovers[], then follow next[] for push/doctor/tests).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_statusA

Every experiment of this project with status, visitors, days, the verdict headline, sample (verdict.sample.quality: small | ok | good), stars (verdict.sample.stars, data quality 0..5) and daysToVerdict (days until a verdict at the current traffic: schedule the next check then, do not poll). verdict guardrail_breached = do not ship that variant. The __holdout row is the project holdout (long-term impact). properties[] (SplitHub Cloud) is each analytics property's last 7 days against the 7 before (visitors, change, goal conversions): mention a big move; splithub_analytics has the details. Start here. A finished experiment is over but its variants are still in the code: report its verdict, ask the user which variant to keep and whether to remove the experiment, then call splithub_ship. display { label, lines, next } is ready to show; voice.line is the friendly line for humans; voice.advice move_on means suggest finishing the experiment and starting the next one. If analysis.available is false the project is self-hosted: report the raw numbers as they are, say verdicts come with SplitHub Cloud (analysis.url), and never call a winner from raw rates. The loop: splithub_plan → splithub_experiment_create → wire the code → splithub_doctor → splithub_push (cloud) → splithub_results (re-check after daysToVerdict days) → splithub_experiment_status finished → splithub_ship.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
nextNo
projectYes
analysisNoself-hosted: raw numbers only. Tell the user verdicts come with SplitHub Cloud; never invent one
propertiesNoSplitHub Cloud: analytics properties, last 7 days (propertiesRange) against the 7 before; change is relative, null = no visitors before
experimentsYes
propertiesRangeNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses guardrail_breached semantics (do not ship), the __holdout long-term-impact row, the self-hosted fallback when analysis.available is false (never call a winner from raw rates), and the post-verdict workflow. That is substantive behavioral context an agent could not infer from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with the resource and scope, but the body is an overstuffed block that explains display{label,lines,next}, voice.line, and voice.advice — return-value semantics already carried by the output schema. Several clauses could be trimmed without losing call correctness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a no-parameter status tool: it covers the verdict lifecycle, holdout handling, the self-hosted caveat, and the surrounding workflow. With an output schema present, it need not exhaustively document return fields, and it doesn't have to.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero input parameters, so the baseline is 4; there is nothing for the description to disambiguate. Its field explanations concern return values rather than inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause states a specific resource (every experiment of this project) with the payload it returns (status, visitors, days, verdict headline), so an agent knows this is a project-wide experiment status read. It distinguishes itself from splithub_analytics (details) and splithub_results (re-check). It is clear but the purpose is buried under a dense wall of field semantics rather than stated crisply up front.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Start here,' tells the agent not to poll but to schedule the next check at daysToVerdict, names splithub_analytics for detail, splithub_ship for finishing, and enumerates the full splithub_plan → ... → splithub_ship loop. When-to-use, when-not-to-use and alternatives are all present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

splithub_whoamiC

Which cloud project this folder is connected to, and its public key.

NameRequiredDescriptionDefault

No parameters

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It never states that this is a read-only, side-effect-free operation, nor what happens when the folder is unconnected to a project (error vs. empty result), nor whether authentication is required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is short and front-loads the main output, but it is a sentence fragment rather than a proper description, so it reads as terse rather than efficiently structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must explain the return values; it names two (connected project and public key) but gives no format or shape for either. Combined with the absent usage guidance, an agent has just enough to attempt a call but not to choose it confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; baseline 4 applies. It correctly avoids inventing arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the resource (the cloud project this folder is connected to) and an extra field (its public key), which lets an agent infer a whoami/identity lookup. But it is a noun phrase with no verb and makes no attempt to distinguish itself from close siblings like splithub_projects or splithub_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to call this versus splithub_projects, splithub_status, or splithub_login, all of which could plausibly reveal connection/project information. The agent must guess based on the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 36 tool updatesv0.1.0
    • First observedsplithub_alerts
    • First observedsplithub_analytics
    • First observedsplithub_analytics_connect
    • First observedsplithub_analytics_events
    • First observedsplithub_analytics_funnel
    • First observedsplithub_analytics_funnels
    • First observedsplithub_analytics_paths
    • First observedsplithub_analytics_realtime
    • First observedsplithub_analytics_retention
    • First observedsplithub_create_experiment
    • First observedsplithub_create_property
    • First observedsplithub_doctor
    • First observedsplithub_domains
    • First observedsplithub_experiment_create
    • First observedsplithub_experiment_events
    • First observedsplithub_experiment_status
    • First observedsplithub_init
    • First observedsplithub_links
    • First observedsplithub_login
    • First observedsplithub_notes
    • First observedsplithub_plan
    • First observedsplithub_projects
    • First observedsplithub_property_create
    • First observedsplithub_property_status
    • First observedsplithub_pull
    • First observedsplithub_push
    • First observedsplithub_refs
    • First observedsplithub_remove
    • First observedsplithub_results
    • First observedsplithub_review
    • First observedsplithub_segments
    • First observedsplithub_set_property_status
    • First observedsplithub_set_status
    • First observedsplithub_ship
    • First observedsplithub_status
    • First observedsplithub_whoami

TDQS

B3.4/5.0

Scored across 36 tools

Disambiguation3/5

Most tools target distinct resources and the descriptions carefully differentiate them (e.g. splithub_status vs splithub_experiment_status vs splithub_results), but there are several overlap traps: four deprecated alias tools (splithub_create_experiment, splithub_set_status, splithub_create_property, splithub_set_property_status) duplicate existing tools with reversed-word names, splithub_remove overlaps splithub_ship, and splithub_property_create overlaps splithub_analytics_connect. An agent can disambiguate with careful reading, but the aliases and near-duplicates are real misselection risk.

Naming Consistency3/5

There is a consistent splithub_ prefix and mostly snake_case, but the convention is mixed: bare-noun tools (status, results, plan, ship, refs, domains, links, notes, alerts) sit alongside noun_noun tools (experiment_create, analytics_funnel), and verb placement is inconsistent — splithub_property_create and splithub_create_property both exist, one deprecated. Multi-action tools (links, notes, alerts, analytics) pack CRUD into an action parameter, further diverging from the verb_noun pattern.

Tool Count2/5

36 tools is above the 25+ threshold and feels heavy, especially since 4 of them are deprecated one-release aliases that should not inflate the surface. The domain is genuinely broad (experiments plus a full analytics suite), but the set could be consolidated (e.g. links/notes/alerts/se gems action multiplexing is already used elsewhere).

Completeness5/5

Coverage is exceptional: full experiment lifecycle (plan → create → doctor/refs → push → results → status → ship/remove → review), full analytics suite (properties, funnel, paths, retention, realtime, events, segments), plus links, alerts, notes, domains, and cloud connection/init tooling. Obvious dead ends are rare — remaining gaps (alert channel config, link targeting) are explicitly dashboard-only and documented.

Maintenance

ActivityNo data
ResponsivenessSyncing

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    MCP-first A/B testing server that enables agents to manage experiments (create, update traffic splits, read results, apply winning variants) from tools like Claude Code, with self-hosted Cloudflare backend.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    MCP server and Claude skills for AI agents to build landing pages, run A/B experiments, and track first-party conversions. Lets agents onboard clients, create and publish pages, manage variants, and pull performance reports via the UXON API.
    13
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables coding agents to run Playwright e2e tests, validate Supabase RLS policies with role-based SQL, typecheck TypeScript, execute npm scripts with parsed test output, and validate Next.js builds, all through a structured MCP interface.
    5
    MIT