inoreader-mcp

Analyze classifier calibration

analyze_classifier_calibration

Read-onlyIdempotent

Build a reliability diagram and calibration diagnostics for the Inoreader LLM classifier, binned by confidence with posterior intervals, ECE, monotonicity checks, and threshold/FNR bounds.

Instructions

Build a reliability diagram for the LLM classifier on the verified-recommended slice. Bins articles by CONFIDENCE score, computes per-bin Beta-Binomial posteriors with an empirical-Bayes prior from the marginal worth-it rate, and reports posterior mean + 95% credible interval per bin. Pool-adjacent-violators (isotonic regression, weighted by bin n) gives a monotone-smoothed curve for visualization; per-bin intervals are the source of truth for inference. Reports ECE, monotonicity violations, a threshold diagnostic (lowest-score recommend-read bin gives an upper bound on what's just below threshold by monotonicity), Manski-style FNR bounds (always identifiable), and an audit-conditional recall posterior via Monte Carlo when audit data exists. Optional per-feed and per-folder slices for multicalibration-lite. Calls extract_classifier_data internally; cost is the same as that call. Base costs exclude safe-read retries (up to 4 attempts, each counted); honor cached status/cooldown. Never blindly replay uncertain writes. See inoreader://usage for limits, URI examples and prompts.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "analyze_classifier_calibration"
  }
}

Input Schema

Name	Required	Description
`bins`	No	Number of equal-width score bins on 0-100 (default 10).
`refresh`	No	Forwarded to extract_classifier_data.
`min_bin_n`	No	Bins below this n are reported as low-data (default 3).
`prompt_name`	No	Forwarded to extract_classifier_data.
`breakdown_by`	No	Additional slices to compute calibration on. Each slice with total n < min_bin_n is reported but flagged.
`prior_strength`	No	Beta prior pseudo-observation count, allocated empirically from the marginal worth-it rate (default 4). Higher values shrink small-bin posteriors more aggressively toward the global mean.
`population_pages`	No	Forwarded to extract_classifier_data.
`population_folder`	No	Forwarded to extract_classifier_data.
`population_months`	No	Forwarded to extract_classifier_data.
`decision_threshold`	No	Score threshold for the diagnostic (default 50). Actual decisions are read from parsed RECOMMENDATION; this is only used to locate the diagnostic bin.
`include_population`	No	Forwarded to extract_classifier_data.

Schema Changelog

Changes observed during successful MCP inspections.

Addedv1.6.1Nov 3, 2026

TDQS

A4.3/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the read-only, idempotent annotations, the description discloses internal data extraction, cost accounting for safe-read retries, cached status/cooldown handling, and the statistical outputs generated. It also clarifies that per-bin credible intervals, not the smoothed curve, are the source of truth for inference.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence, followed by dense but relevant methodology. Some boilerplate about uncertain writes and safe-read retries is less central for a read-only analysis tool, but overall the description earns most of its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex statistical tool with no output schema, the description is substantially complete: it lists the key outputs (ECE, monotonicity violations, threshold diagnostic, FNR bounds, recall posterior) and explains internal dependencies and cost. Exact return structure is not specified, which keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameter meanings are already well documented. The description adds conceptual context for the threshold diagnostic and breakdown slices, but it does not provide parameter-by-parameter guidance beyond what the schema supplies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource: build a reliability diagram for the LLM classifier on the verified-recommended slice. It distinguishes the tool from data-extraction siblings by naming the diagnostic outputs and explaining that it calls extract_classifier_data internally.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly frames when to use the tool for calibration analysis, notes optional per-feed/per-folder slices, and states the cost relationship to extract_classifier_data. It does not explicitly rule out alternatives or describe when not to use it, keeping it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.