P2predict-mcp

Overview Schema Source Related Servers Discussions Admin

get_model_quality

Obtain a structured verdict on model trustworthiness, bias, and price-band reliability to decide which estimates to quote versus benchmark.

Instructions

Structured, agent-readable model-quality report — the JSON form of the PDF.

Use this (not just generate_report, which only writes a PDF) when you need
to *reason about or relay* model quality. Every judgment is computed so you
don't eyeball thresholds:

  - `assessment.verdict` — LEAD WITH THIS. One of: 'trustworthy' | 'usable'
    | 'unreliable' | 'unknown' | 'insufficient_data'. It folds bias and
    sample size into a plain `headline` you can quote verbatim — e.g. a
    modest model that is even-handed reads 'usable', not just 'Needs
    Improvement'. `assessment.confidence` is 'high' | 'limited' |
    'insufficient'.
  - `assessment.typical_bias_pct` — how far the model runs high or low on a
    typical part, in percent. `assessment.bias_resolution_pct` is the
    smallest offset this holdout could have detected; when the verdict is
    'unknown', quote it as the bound you CAN rule out.
  - `calibration_by_price_band[].reliability` — 'trust' | 'caution' |
    'quote' per price range (with `low_confidence` when a band is thin).
    Each band carries a `say_to_user` sentence in plain words — quote it to
    tell the user which prices to benchmark vs. get a quote on.
  - `feature_importance[].signal` — 'strong' | 'moderate' | 'weak', each
    with its own `say_to_user` sentence. Only quote findings resting on
    'strong' drivers to a stakeholder.

The default payload is deliberately business-only — every string is safe to
read to a category manager. NEVER say 'SHAP', 'R²', 'p-value', 'log-target',
'residual' to the user. The raw statistics (R², p-value, algorithm,
log-target) are NOT in the default response; pass include_metrics=true to
add them under `metrics`/`provenance` for your own developer-level reasoning.

Set include_holdout=true to also get the raw actual/predicted arrays, so an
agent with a code/plotting tool can draw its own charts (predicted-vs-actual,
residuals, error-by-band).

Requires a model trained via the MCP train tool (which stores holdout data).

Input Schema

TableJSON Schema

Name	Required	Description	Default
`model_id`	Yes
`include_holdout`	No
`include_metrics`	No

Output Schema

TableJSON Schema

Name	Required	Description	Default
`result`	Yes

Schema Changelog

Changes observed during successful MCP inspections.

First observedv1.1.1Sep 16, 2026

TDQS

A5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does so thoroughly. It discloses the safe business-only default, warns against exposing technical jargon, explains that raw statistics are excluded unless include_metrics is set, and describes what include_holdout adds. It also reveals the computation semantics, such as how verdicts fold bias and sample size, which is far beyond a minimal summary.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but appropriately structured with front-loaded purpose, usage guidance, bullets, and flags. Every sentence contributes meaningful information; there is no filler or repetition, and the formatting makes the key decision points easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and the presence of an output schema, the description is remarkably complete. It covers tool selection, invocation parameters, output semantics, user-facing safety constraints, and prerequisites. There is no apparent gap an agent would face when deciding to call or invoking this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains include_metrics and include_holdout in meaningful detail, including what payloads they add and for what purpose. model_id's meaning is clear from context, and the description adds the important prerequisite that the model must have holdout data stored by the train tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it returns a structured, agent-readable model-quality report, positioned as the JSON form of the PDF. It explicitly distinguishes itself from the sibling generate_report by saying generate_report only writes a PDF, so an agent can immediately tell this tool apart.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit guidance: use this instead of generate_report when you need to reason about or relay model quality. It also explains when to pass include_metrics and include_holdout, and notes the prerequisite that the model must be trained via the MCP train tool, leaving no ambiguity about selection and invocation context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server