Skip to main content
Glama
tabular-ds-mcp
by YusufErem

cv_baseline

Run a leakage-safe, cross-validated gradient-boosting baseline against a no-skill score, with time- or group-aware folds, to check if random validation overstates performance.

Instructions

Cross-validated gradient-boosting baseline reported next to a no-skill baseline. Encoders are fitted inside each training fold, so the scores are free of preprocessing leakage. With time_col the folds follow time; with group_col no group is split across folds. In both cases the same model is also scored with random folds, and the tool says whether random validation would overstate the score.

Args: train_path: training table. target: target column. id_col: ID column, excluded from the features. task: 'auto', 'binary', 'multiclass' or 'regression'. metric: the competition metric. Binary: roc_auc, pr_auc, log_loss, accuracy, f1, mcc. Multiclass: log_loss, accuracy, macro_f1, roc_auc_ovr, qwk, map@3 (any k). Regression: rmse, rmsle, mae, mape, r2, qwk. Default: roc_auc / log_loss / rmse. time_col: date or time-index column; folds train on the past and score the next time block. group_col: key column (customer, store, patient...); a group never sits in both parts of a fold. n_splits: number of folds. max_rows: train on a random sample of this many rows. oof_path: optional CSV path to save out-of-fold predictions.

Input Schema

NameRequiredDescriptionDefault
taskNoauto
id_colNo
metricNo
targetYes
max_rowsNo
n_splitsNo
oof_pathNo
time_colNo
group_colNo
train_pathYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that encoders are fitted inside each fold (leakage-free), that time_col produces forward-chaining folds and group_col produces group-disjoint folds, and that a random-fold comparison is always run and judged for optimism. It stops short of saying whether anything is persisted besides the optional oof_path or how long the run takes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Behavioral guarantees are front-loaded in the opening paragraph and the argument list follows, so structure is strong and each entry adds information. It is somewhat long, and the hard-wrapped line breaks add noise, but no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex ten-parameter tool with no output schema, the description gives enough to call it correctly: input semantics, fold behavior, metric choices, and a sketch of the result (skill vs no-skill comparison, overstatement verdict). It does not describe the returned object's structure, which keeps it from being fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the description compensates fully, documenting all ten parameters. It enumerates the metric vocabulary per task type with defaults (roc_auc / log_loss / rmse), explains that id_col is dropped from features, and gives real semantics for time_col ('folds train on the past and score the next time block') and group_col ('a group never sits in both parts of a fold').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence gives a specific verb and resource ('cross-validated gradient-boosting baseline reported next to a no-skill baseline') and frames it as an evaluation tool, which separates it from siblings like fit_predict and explain_model. It never names an alternative directly, so the differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: you run this to get a trustworthy CV estimate before committing to a model. There is no explicit when-to-use versus fit_predict, no stated prerequisites (e.g., data must already be loaded, target must exist), and no when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.