Skip to main content
Glama

SCRIPTMASTERLABS — Calibration Court

Server Details

Calibration leaderboard + paid recalibration via x402. Settle-proven 2026-11-02 (SELF-PROOF).

Ownership verified
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

A3.6/5.0

Scored across 5 tools

Disambiguation3/5

court_calibrate_labels and court_recalibrate both perform paid recalibration of probabilities, and the names read like a version pair rather than distinct operations. The descriptions do carve out the boundary (your own labelled pairs, stateless, vs. a ledger model's resolved outcomes), but an agent must read carefully to avoid misselection. The remaining three tools (leaderboard, model_report, verify_ledger) are clearly distinct.

Naming Consistency3/5

All tools share a consistent 'court_' prefix, which aids grouping. However, only two follow a strict verb_noun pattern (court_calibrate_labels, court_verify_ledger); the others are noun-phrase (court_leaderboard, court_model_report) or bare verb (court_recalibrate), so the naming convention is mixed.

Tool Count4/5

Five tools is a tight, well-scoped set for a calibration-and-ledger service, with no redundant entries. It sits at the lean end of the scale but each tool maps to a distinct capability.

Completeness3/5

The surface covers scoring, reporting, leaderboard, recalibration and audit verification, but nothing records predictions or resolves questions — the ledger that three tools depend on is read-only here. There is also no tool to list models or fetch a single prediction, leaving agents with dead ends if the write path lives elsewhere.

Available Tools

5 tools
court_calibrate_labelsCalibrate Probabilities on Your Labelled OutcomesA
Read-only
Inspect

Paid (x402: USDC on Base, USDC on Solana, XRP on XRPL, USDC on Arc, or XAH on Xahau). Fit isotonic calibration on YOUR labelled outcomes ({p,y} pairs) and recalibrate the given probabilities. Stateless; nothing is stored.

Inputs: probs (required), examples (required).

court_leaderboardModel Calibration LeaderboardA
Read-only
Inspect

Live ranking of decision models by Brier score on resolved real-world questions. Free. Returns empty lists when nothing has resolved.

Inputs: task_type.

court_model_reportModel Calibration ReportA
Read-only
Inspect

Calibration report for one model, computed from its resolved predictions recorded on the ledger: Brier score, expected calibration error (ECE), reliability bins with Wilson score intervals, overconfidence measures and drift over time. Free. Returns not_found when the model has no resolved predictions.

Inputs: model (required), task_type.

court_recalibrateRecalibrate a Model ProbabilityA
Read-only
Inspect

Paid (x402: USDC on Base, USDC on Solana, XRP on XRPL, USDC on Arc, or XAH on Xahau). Map a model's raw probability to an empirically calibrated one using resolved outcomes on the ledger. Refuses with insufficient_data below 30 samples.

Inputs: prob (required), model (required), task_type.

court_verify_ledgerVerify Ledger Hash ChainA
Read-only
Inspect

Independently recompute the ledger's SHA-256 hash chain from genesis and cross-check the derived tables against it, returning the verification result together with the current head hash. Free. Use it to confirm that recorded predictions and outcomes have not been altered after the fact.

Inputs: no named parameters.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources