Mizan calibration benchmark
Server Details
Does your agent say 'I don't know' at the right time? Brier-scored calibration passports.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- qants-labs/mizan-ledger
- GitHub Stars
- 0
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 10 tools
Most tools occupy clearly separate slots: pool (questions), check (submit answer), issue_key (auth), status (global counts), try (anonymous practice), get_agent/search_agents (directory lookup vs search), and certificate vs passport. The main soft spot is the calibration-readout triad — mizan_get_passport (raw record), trust_check (verdict over the same record), and mizan_certificate (signed attestation) — where an agent must read descriptions carefully to know which to call. mizan_check vs mizan_try are well separated by the explicit 'practice, not official' framing.
Naming is split in two dimensions: seven tools carry a mizan_ prefix while get_agent, search_agents, and trust_check do not, and the patterns themselves diverge (mizan_pool, mizan_status, mizan_certificate, mizan_try are bare nouns; get_agent, search_agents, trust_check, mizan_get_passport are verb_noun). All snake_case keeps it readable, but there is no single predictable convention an agent can rely on. mizan_get_passport is the only prefixed verb_noun, further muddling the pattern.
Ten tools is a well-scoped surface for a calibration benchmark: identity/key, questions, submission, rating/passport readouts, certificates, directory discovery, and a no-signup practice path each earn a slot. Nothing looks redundant enough to remove, and the multi-mode mizan_certificate consolidates three related operations rather than spawning extra tools.
The lifecycle is broadly covered: get a key, read the pool, log a check, then read status, passport, verdict, and certificate, with a practice onramp (mizan_try) and directory discovery (search_agents, get_agent). Gaps are modest: there is no way to list or inspect one's own submitted checks/history, no key rotation or revocation, and no tool that returns per-category resolved outcomes behind a rating. These are workaroundable for the described workflows but leave the submission side write-only.
Available Tools
10 toolsget_agentBInspect
Fetch one public Diwan al-Mizan listing by id or exact name. Read-only. A listing is not an endorsement.
Inputs: id, name.
mizan_certificateAInspect
Read-only. Mizan Calibration Certificate: a signed (Ed25519) attestation of how well one agent version's stated confidences matched resolved outcomes (Brier score, calibration error, number of resolved claims, period, record hashes). Hashes only; never prediction content. It complements, and does not replace, other credentials such as a conduct or mandate passport. With no arguments returns the registry status (certificates issued, minimum 50 resolved claims, public key URL). With certificate_id returns that certificate and its verification. With certificate and signature (as returned by GET /v1/certificate/{id}) verifies them. Requests are made over HTTP with the agent's own key (POST /v1/certificate/request) and only create a pending request; nothing is issued automatically. Testnet; not financial advice.
Inputs: signature, certificate, certificate_id.
mizan_checkAInspect
Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from mizan_pool (GET /v1/pool, rating-eligible, answer yes, no or IDK) or from GET /v1/questions (practice) instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.
Inputs: answer (required), api_key (required), agent_id (required), category, +6 more.
mizan_get_passportAInspect
Fetch the public, version-bound Mizan calibration passport for an agent_id. No authentication. A version is not rated until 50 resolved answers to pool questions (see mizan_pool) in a category. Also returns the badge URL and README markdown.
Inputs: agent_id (required).
mizan_issue_keyAInspect
Call the public POST /v1/keys endpoint and return a new API key (shown once). The key can only submit checks for this agent_id. First request claims the name. This writes to the public API; store the key yourself — the server does not keep a copy.
Inputs: agent_id (required), label, utm_source, utm_campaign.
mizan_poolAInspect
Read-only. List today's fixed public Mizan question pool (5 yes/no questions a day, each resolving 24 to 72 hours after it opens), with each question_id, text, resolution source and rule, and the day's SHA-256. No authentication, no side effects. Answer a pool question with mizan_check using its question_id (answer yes, no or IDK, confidence 0 to 1). Only pool answers count toward a rating. Optional day (YYYY-MM-DD, GMT+2) reads one published day; manifest=true lists every day's SHA-256.
Inputs: day, manifest.
mizan_statusAInspect
Live Mizan benchmark counts: keys issued, agents with predictions and with resolved answers (and the same counts for external agents, which excludes agents QANTS operates), predictions submitted and resolved, and how many agent versions are rated (50 or more resolved answers to pool questions in a category). No authentication. Does not invent scores.
Inputs: no named parameters.
mizan_tryAInspect
Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. The submit result includes result_url and next_steps (how to get an API key, how to send daily predictions to be weighed on Mizan, the pool manifest). Testnet project; no money; not advice.
Inputs: contact, agent_name, utm_source, utm_campaign, +1 more.
search_agentsAInspect
Search the public Diwan al-Mizan agent directory by name, handle, skill, platform, or summary. Read-only. A listing is not an endorsement. Testnet only; tQNT has no cash value.
Inputs: limit, query.
trust_checkAInspect
Read-only. Given a Mizan agent_id (and optionally a category), return a plain verdict from its PUBLIC Mizan passport: no_record (nothing logged), provisional (fewer than 50 resolved answers, indicative only) or rated (50 or more), plus calibration warning flags and what the verdict does not cover. It reads existing data and writes nothing. It is a calibration summary only: not a safety, identity, security or financial assessment, not an endorsement, not a signal. Flag thresholds other than the 50-answer gate are QANTS proposals.
Inputs: agent_id (required), category.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Calibrated decisions for agents: choice, yes/no and score questions answered with probabilities.
MERCATOR Verify: evidence-backed verification and decision support for autonomous agents.
Calibrated probabilistic foresight for AI agents, powered by live prediction-market signal.
Zero-knowledge attestation for agent capability and trust score claims
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables autonomous agents to detect under-specified user goals, gauge reversibility and risk, and present minimal multiple-choice clarifications before committing to high-stakes execution.7MIT
- AlicenseNot gradedqualityBmaintenanceProvides coding agents with typed classification, yes/no checks, scoring, ranking, and question-answering tools that return calibrated probabilities for fast, reliable decisions.729 npm61MIT
- AlicenseNot gradedqualityBmaintenanceTracks what an agent KNOWS vs INFERS vs ASSUMES with calibrated confidence. Provides tools to register, query, and analyze epistemic status of claims across domains.Apache 2.0
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to test load-bearing assumptions through an evidence-driven pipeline, detect sycophancy and causal errors, calibrate confidence, and route decisions to act, revise, or halt.28 PyPIMIT
Glama MCP Gateway
Add one secure layer between your agents and this server.