Skip to main content
Glama

Server Details

Independent statistical feature evaluation for machine learning and forecasting. ML Referee tests up to 1,000 candidate features in one run against a declared baseline, helping researchers and AI agents decide which inputs to pursue, which bounded feature repairs merit further testing, and where more evidence is needed—without a separate ablation run for every feature.

Use the MCP tools to discover input schemas, check access and credits, preflight a dataset, obtain a quote, submit an authorized evaluation, and retrieve structured results and reports. Results are scoped to the submitted evidence and evaluation design; they do not guarantee future model performance.

Invite-only beta with manual approval. Public tool discovery requires no authentication; account and evaluation tools require OAuth through approved client connections and available evaluation credits. Connect directly to https://api.mlreferee.com/mcp.

Website: https://mlreferee.com Request beta access: https://mlreferee.com/beta Agent setup and documentation: https://mlreferee.com/agents

Ownership verified
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

B3.3/5.0

Scored across 26 tools

Disambiguation4/5

Most tools target a clearly distinct resource+action, and descriptions distinguish the three parallel families (baseline audit, repair, campaign) and their upload/preflight/quote steps. The main risk is misselecting between the three similarly-named families (e.g. quote_audit vs quote_repair vs quote_campaign, or the three upload_* tools), though the payload descriptions help separate them.

Naming Consistency5/5

Every name follows a strict verb_noun snake_case pattern (upload_private_dataset, preflight_repair, quote_campaign, get_audit_result, delete_audit). Prefixes are used predictably across get_, delete_, upload_, preflight_, quote_ and verify_ operations.

Tool Count3/5

26 tools is on the heavy side and sits at the threshold where the surface starts to feel sprawling. The three full lifecycle families with their own upload/preflight/quote variants justify many of them, but consolidation (e.g. unified upload/preflight/quote with a mode parameter) would reduce load.

Completeness4/5

Coverage is strong: upload, preflight, quote, submit, status, result, receipt, verification, deletion, plus credits, beta access, schemas, verticals, usage, readiness and followup. Minor gaps like no explicit cancel/abort for a submitted audit or update operation are workable around.

Available Tools

26 tools
delete_auditA
DestructiveIdempotent
Inspect

Delete retained inputs and results for an audit owned by this tenant.

Inputs: audit_id (required).

delete_uploadA
DestructiveIdempotent
Inspect

Delete a tenant-private upload that is no longer needed.

Inputs: upload_id (required).

describe_refereeB
Read-onlyIdempotent
Inspect

Discover capabilities, privacy, and the approved public catalog.

Inputs: no named parameters.

get_audit_receiptA
Read-onlyIdempotent
Inspect

Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.

Inputs: audit_id (required).

get_audit_resultB
Read-onlyIdempotent
Inspect

Read a completed tenant-private result and signed output manifest.

Inputs: audit_id (required).

get_audit_statusB
Read-onlyIdempotent
Inspect

Read the state of an audit owned by the authenticated tenant.

Inputs: audit_id (required).

get_beta_request_statusA
Read-onlyIdempotent
Inspect

Read the authenticated tenant's beta-access request status.

Inputs: request_id (required).

get_creditsB
Idempotent
Inspect

Read beta or paid audit-credit availability and usage.

Inputs: no named parameters.

get_input_schemaC
Read-onlyIdempotent
Inspect

Get neutral column roles and validation defaults for a vertical.

Inputs: vertical (required).

get_referee_readinessA
Read-onlyIdempotent
Inspect

Report connection, credential, audit-credit, and submission readiness with the next action.

Inputs: no named parameters.

get_referee_usageB
Read-onlyIdempotent
Inspect

Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.

Inputs: window_days.

get_repair_schemaC
Read-onlyIdempotent
Inspect

Get the feature-only contract for diagnosing degradation and testing bounded repairs.

Inputs: no named parameters.

list_verticalsB
Read-onlyIdempotent
Inspect

List the three supported evaluation profiles and their discovery tags.

Inputs: no named parameters.

preflight_datasetBInspect

Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.

Inputs: contract (required), vertical (required), upload_id (required).

preflight_repairB
Idempotent
Inspect

Validate a tenant-private feature-repair matrix without spending a credit.

Inputs: contract (required), upload_id (required).

quote_auditB
Idempotent
Inspect

Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.

Inputs: contract (required), vertical (required), upload_id (required), idempotency_key (required).

quote_campaignC
Idempotent
Inspect

Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.

Inputs: shards (required), vertical (required), idempotency_key (required).

quote_repairB
Idempotent
Inspect

Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.

Inputs: contract (required), upload_id (required), idempotency_key (required).

record_audit_followupC
Idempotent
Inspect

Record whether a completed audit changed a decision or was confirmed prospectively.

Inputs: kind (required), value (required), audit_id (required).

redeem_beta_inviteA
Idempotent
Inspect

Redeem one operator-issued invite for free beta audit credits before January 1, 2027.

Inputs: invite_code (required).

request_beta_accessC
Idempotent
Inspect

Request operator-reviewed beta access without purchasing or granting credits.

Inputs: lane (required), use_case (required), idempotency_key (required).

submit_auditC
Idempotent
Inspect

Accept a quote and atomically reserve its disclosed audit credits.

Inputs: quote_id (required), idempotency_key (required).

upload_campaign_shardC
Idempotent
Inspect

Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.

Inputs: data (required), format, retention_seconds.

upload_private_datasetB
Idempotent
Inspect

Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.

Inputs: data (required), format, retention_seconds.

upload_repair_matrixC
Idempotent
Inspect

Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.

Inputs: data (required), format, retention_seconds.

verify_audit_reportA
Read-onlyIdempotent
Inspect

Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.

Inputs: sha256 (required), audit_id (required), representation.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      A
      quality
      D
      maintenance
      High-value scientific tools for AI agents — literature search (PubMed, arXiv, Semantic Scholar), chemical compound lookup (PubChem, ChEMBL), patent prior art search (USPTO, EPO), GPU spot prices across 4 providers, and real-time earth science data (USGS, NASA, OpenAQ). Per-call billing via API key. Keys issued instantly at https://mcp-site.com/keys/request
      6
      -
    • A
      license
      A
      quality
      D
      maintenance
      Deterministic time-series statistics for AI agents. This MCP server gives any LLM agent unit-tested statistical tools — anomaly detection, changepoint detection, seasonal decomposition, stationarity/trend tests, data-quality audits, baseline forecasts — with schema-validated structured output and no arbitrary code execution.
      17
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      MCP server exposing statistical regression testing for LLM agents as a "run" tool: p-value, effect size, and confidence interval on whether agent behavior actually changed.
      1
      113 PyPI
      Apache 2.0
    • A
      license
      Not graded
      quality
      C
      maintenance
      Human-evaluation infrastructure for AI quality. 25,000+ blind human reviews by 200+ verified reviewers across 58 AI models — query the data via five MCP tools (get_model_scores, compare_models, get_flags, check_content, get_latest).
      16 PyPI
      2
      MIT
    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources