Skip to main content
Glama
littlebigbrains

@littlebigbrain/mcp

lbb_model_choice_manage

Model choice, writes. start tests a candidate model on the uses in jobs (without jobs, every use it can do that has checked calls): the server answers those checked calls again with the candidate and scores it and the model in use against…

Instructions

Model choice, writes. start tests a candidate model on the uses in jobs (without jobs, every use it can do that has checked calls): the server answers those checked calls again with the candidate and scores it and the model in use against the same ground truth. It changes no model and spends at most the trial budget ($3.00 by default) of model and judge cost; an open trial of the same use and candidate comes back as it is. Pick candidates from lbb_model_choice action=options (provider anthropic with an effort, or typesafe jev-latest). A trial runs by itself: poll lbb_model_choice action=trial until its status is no longer running. stop ends a trial; it keeps what it compared. switch makes the trial's use run its candidate on this graph from the next call on; only a trial that qualifies and compared with the model in use switches (ask to a Claude model, route to a Jev model). Show the user the trial's report (score, difference and interval, cost per call, time) and switch only what the user confirmed. revert puts the use back on LBB's model.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
jobNorevert: the use to put back on LBB's model.
daysNostart: the days the trial takes new checks (default 14).
jobsNostart: the uses to test it on.
graphNoGraph to target; defaults to the connection's graph
modelNostart: the candidate model id, e.g. claude-haiku-5-5 or jev-latest.
actionYes
detailNoResponse detail level. Defaults to compact.
effortNostart: the effort of a Claude candidate.
targetNostart: the calls to compare (default 40).
providerNostart: the candidate's provider.
trial_idNostop / switch: the trial id.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv0.14.0

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (which only say non-read-only, non-destructive) by disclosing the trial budget cap ($3.00 default), that start mutates no model, idempotency of an open trial, that stop preserves comparisons, and that switch only applies to a qualifying trial already compared against the incumbent. This is exactly the mutation/state semantics an agent needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Information-dense and largely front-loaded, but it is delivered as a single run-on paragraph with nested parentheticals and colons that make per-action scanning hard. The content earns its place; the structure does not.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 11 params and no output schema, it covers the important gaps: async polling lifecycle, budget ceiling, user-confirmation gate, and even the report contents (score, difference and interval, cost per call, time). It omits any auth/permission note, but is otherwise sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is already 91% with action-scoped prefixes, so the baseline is 3. The description adds genuine meaning on top: candidates are provider anthropic with an effort setting, or typesafe jev-latest, clarifying how provider/effort/model interact beyond the raw enum lists.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a terse 'Model choice, writes' but then defines each of the four actions (start/stop/switch/revert) with concrete behavior, so an agent can grasp the resource and operations. It implicitly distinguishes itself from the sibling lbb_model_choice (which serves options/trial reads), though it never states that contrast in so many words.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing: pick candidates from lbb_model_choice action=options, poll lbb_model_choice action=trial until status leaves running, and switch only what the user confirmed. It names the alternative tool call and the condition that selects it, plus the confirmation prerequisite.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.