AI-ML-Lab-MCP

Overview Schema Source Related Servers Discussions Admin

Compare Models

compare_models

Train and rank multiple candidate models on the same train/test split, then return a leaderboard with stored models and metrics to select a winner for tuning.

Instructions

Train several candidate models on the same split, score them all on the same train and test data, and return a ranked leaderboard. The trained models are stored (model_id) so you can plot or inspect them. The split must have only numerical, complete features (run encode_categorical / impute_missing first). Each model uses the default settings of its own training tool; tune the winner afterwards. Args: split_id (str): ID of a split made by train_test_split. models (list[str], optional): Training tool names to try, e.g. ["ridge_regression","random_forest_regressor"]. Default: linear, ridge, lasso, decision tree, random forest, gradient boosting and xgboost (regression); logistic, knn, decision tree, random forest, svm, gradient boosting and xgboost (classification). elastic_net is available but not tried by default. Models whose package is missing are skipped and listed. task (str): "auto" (inferred from the target), "regression" or "classification". Default is "auto". sort_by (str, optional): Metric that ranks the models, measured on the test data. Regression: r2 (default), mae, mse, rmse, mape_percent. Classification: f1 (default), accuracy, balanced_accuracy, precision, recall, roc_auc, f1_macro (multiclass only). include_slow (bool): Also run SVM on large data (over 10,000 training rows). Default is False. keep (str): "all" keeps every trained model; "best" deletes all but the winner (less clutter). Default is "all". Returns: dict: leaderboard (rank, model_id, algorithm, test score on sort_by, full train/test metrics, generalisation gap, training seconds), best_model_id, failed/skipped models, and notes (ties, overfitting, imbalance).

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "split_id": "<split_id>"
    },
    "name": "compare_models"
  }
}

Input Schema

Name	Required	Default
`keep`	No	all
`task`	No	auto
`models`	No
`sort_by`	No
`split_id`	Yes
`include_slow`	No

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.1.0Nov 14, 2026

TDQS

A4.5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does so: it discloses that trained models are persisted and addressable by model_id, that keep='best' deletes all but the winner (an explicit destructive consequence), that include_slow gates SVM on data over 10,000 rows, that missing-package models are skipped and listed, and that the response includes ties, overfitting and imbalance notes. These are non-obvious side effects and failure modes that go well beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and prerequisite are front-loaded before the Args/Returns blocks, and nearly every sentence carries actionable information. The default model enumeration is long but functional, since the schema cannot express it. Minor redundancy exists between the prose prerequisite and the per-parameter notes, but nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by describing the returned dict field-by-field: leaderboard columns, best_model_id, failed/skipped models, and notes. Combined with prerequisites, defaults and destructive keep behavior, an agent has everything needed to invoke this correctly on the first try.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must supply all parameter meaning, and it does for every one of the six parameters: split_id's origin, the models list with a full default enumeration and the elastic_net caveat, task's three accepted values with default, sort_by's allowed metric sets split by regression vs classification with defaults, include_slow's threshold, and keep's two modes with their effect. The enum-like value lists are especially valuable because the schema declares no enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: train several candidate models on one split, score them on the same train/test data, and return a ranked leaderboard. It is clearly distinguishable from single-model training tools by the word 'several' and the leaderboard output. It does not, however, explicitly differentiate itself from similarly named siblings such as benchmark_models or rank_models, leaving the agent to infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states a concrete precondition ('The split must have only numerical, complete features — run encode_categorical / impute_missing first') and a concrete follow-up ('tune the winner afterwards'), which is exactly the workflow context an agent needs. It also notes that elastic_net exists but is not tried by default, and that models with missing packages are skipped. What is missing is explicit routing versus alternative comparison/benchmark tools, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server