ML Referee
Server Details
Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 26 tools
Most tools are distinguishable by workflow stage and resource type (upload, preflight, quote, submit, result, receipt, repair, campaign). The main ambiguity is among related families such as quote_audit/quote_repair/quote_campaign and the three upload/preflight variants, though the descriptions clarify their boundaries.
All tool names use a consistent snake_case verb_noun pattern, such as get_audit_status, upload_private_dataset, quote_repair, and verify_audit_report. The vocabulary is predictable across the entire surface.
The server covers a broad multi-stage ML audit workflow, so many tools are justified. However, 26 tools is heavy and includes several narrow lifecycle/admin operations, making the surface borderline over-scoped.
The surface covers discovery, access, credits, schemas, uploads, preflight validation, quoting, submission, status, results, receipts, verification, follow-up, usage, and deletion. Minor gaps remain, such as no obvious list-audits or cancellation flow.
Available Tools
26 toolsdelete_auditADestructiveIdempotentInspect
Delete retained inputs and results for an audit owned by this tenant.
Inputs: audit_id (required).
delete_uploadADestructiveIdempotentInspect
Delete a tenant-private upload that is no longer needed.
Inputs: upload_id (required).
describe_refereeBRead-onlyIdempotentInspect
Discover capabilities, privacy, and the approved public catalog.
Inputs: no named parameters.
get_audit_receiptARead-onlyIdempotentInspect
Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.
Inputs: audit_id (required).
get_audit_resultBRead-onlyIdempotentInspect
Read a completed tenant-private result and signed output manifest.
Inputs: audit_id (required).
get_audit_statusBRead-onlyIdempotentInspect
Read the state of an audit owned by the authenticated tenant.
Inputs: audit_id (required).
get_beta_request_statusARead-onlyIdempotentInspect
Read the authenticated tenant's beta-access request status.
Inputs: request_id (required).
get_creditsBIdempotentInspect
Read beta or paid audit-credit availability and usage.
Inputs: no named parameters.
get_input_schemaCRead-onlyIdempotentInspect
Get neutral column roles and validation defaults for a vertical.
Inputs: vertical (required).
get_referee_readinessARead-onlyIdempotentInspect
Report connection, credential, audit-credit, and submission readiness with the next action.
Inputs: no named parameters.
get_referee_usageBRead-onlyIdempotentInspect
Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.
Inputs: window_days.
get_repair_schemaCRead-onlyIdempotentInspect
Get the feature-only contract for diagnosing degradation and testing bounded repairs.
Inputs: no named parameters.
list_verticalsBRead-onlyIdempotentInspect
List the three supported evaluation profiles and their discovery tags.
Inputs: no named parameters.
preflight_datasetBInspect
Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.
Inputs: contract (required), vertical (required), upload_id (required).
preflight_repairBIdempotentInspect
Validate a tenant-private feature-repair matrix without spending a credit.
Inputs: contract (required), upload_id (required).
quote_auditBIdempotentInspect
Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.
Inputs: contract (required), vertical (required), upload_id (required), idempotency_key (required).
quote_campaignCIdempotentInspect
Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.
Inputs: shards (required), vertical (required), idempotency_key (required).
quote_repairBIdempotentInspect
Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.
Inputs: contract (required), upload_id (required), idempotency_key (required).
record_audit_followupCIdempotentInspect
Record whether a completed audit changed a decision or was confirmed prospectively.
Inputs: kind (required), value (required), audit_id (required).
redeem_beta_inviteAIdempotentInspect
Redeem one operator-issued invite for free beta audit credits before January 1, 2027.
Inputs: invite_code (required).
request_beta_accessCIdempotentInspect
Request operator-reviewed beta access without purchasing or granting credits.
Inputs: lane (required), use_case (required), idempotency_key (required).
submit_auditCIdempotentInspect
Accept a quote and atomically reserve its disclosed audit credits.
Inputs: quote_id (required), idempotency_key (required).
upload_campaign_shardCIdempotentInspect
Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.
Inputs: data (required), format, retention_seconds.
upload_private_datasetBIdempotentInspect
Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.
Inputs: data (required), format, retention_seconds.
upload_repair_matrixCIdempotentInspect
Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.
Inputs: data (required), format, retention_seconds.
verify_audit_reportARead-onlyIdempotentInspect
Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.
Inputs: sha256 (required), audit_id (required), representation.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Synthetic UX reviews, findings, evidence images, and repair prompts. OAuth required.
Preflight QA for AI-agent deliverables with structured verdicts and repair guidance.
Applied ML validation, delivered per API call as an atom service on Conduit Atoms.
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables AI coding agents to submit code, files, or diffs for deterministic checks and optional re-verified LLM repair, returning cleaned code, structured findings, and a pass/fixed/reject verdict.MIT
- AlicenseNot gradedqualityBmaintenanceEnables evaluating AI applications, inspecting reliability evidence, and gating releases from development and CI workflows.33Apache 2.0
- AlicenseNot gradedqualityBmaintenanceEnables agents to diagnose JSON incompatibilities against target schemas and explicit rules, then receive bounded deterministic repairs independently verified before delivery.MIT

Trustwise MCP Serverofficial
AlicenseNot gradedqualityDmaintenanceProvides advanced evaluation tools for assessing AI safety, alignment, and performance of LLM outputs. Enables programmatic evaluation of quality, safety metrics like toxicity and PII detection, and operational metrics including carbon footprint and cost estimation.4Apache 2.0
Glama MCP Gateway
Add one secure layer between your agents and this server.