Skip to main content
Glama

Mizan calibration benchmark

Server Details

Does your agent say 'I don't know' at the right time? Brier-scored calibration passports.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
Repository
qants-labs/mizan-ledger
GitHub Stars
0
ChatGPT
–
Claude
–

TDQS

A3.8/5.0

Scored across 10 tools

Disambiguation4/5

Most tools occupy clearly separate slots: pool (questions), check (submit answer), issue_key (auth), status (global counts), try (anonymous practice), get_agent/search_agents (directory lookup vs search), and certificate vs passport. The main soft spot is the calibration-readout triad — mizan_get_passport (raw record), trust_check (verdict over the same record), and mizan_certificate (signed attestation) — where an agent must read descriptions carefully to know which to call. mizan_check vs mizan_try are well separated by the explicit 'practice, not official' framing.

Naming Consistency2/5

Naming is split in two dimensions: seven tools carry a mizan_ prefix while get_agent, search_agents, and trust_check do not, and the patterns themselves diverge (mizan_pool, mizan_status, mizan_certificate, mizan_try are bare nouns; get_agent, search_agents, trust_check, mizan_get_passport are verb_noun). All snake_case keeps it readable, but there is no single predictable convention an agent can rely on. mizan_get_passport is the only prefixed verb_noun, further muddling the pattern.

Tool Count5/5

Ten tools is a well-scoped surface for a calibration benchmark: identity/key, questions, submission, rating/passport readouts, certificates, directory discovery, and a no-signup practice path each earn a slot. Nothing looks redundant enough to remove, and the multi-mode mizan_certificate consolidates three related operations rather than spawning extra tools.

Completeness4/5

The lifecycle is broadly covered: get a key, read the pool, log a check, then read status, passport, verdict, and certificate, with a practice onramp (mizan_try) and directory discovery (search_agents, get_agent). Gaps are modest: there is no way to list or inspect one's own submitted checks/history, no key rotation or revocation, and no tool that returns per-category resolved outcomes behind a rating. These are workaroundable for the described workflows but leave the submission side write-only.

Available Tools

10 tools
get_agentBInspect

Fetch one public Diwan al-Mizan listing by id or exact name. Read-only. A listing is not an endorsement.

Inputs: id, name.

mizan_certificateAInspect

Read-only. Mizan Calibration Certificate: a signed (Ed25519) attestation of how well one agent version's stated confidences matched resolved outcomes (Brier score, calibration error, number of resolved claims, period, record hashes). Hashes only; never prediction content. It complements, and does not replace, other credentials such as a conduct or mandate passport. With no arguments returns the registry status (certificates issued, minimum 50 resolved claims, public key URL). With certificate_id returns that certificate and its verification. With certificate and signature (as returned by GET /v1/certificate/{id}) verifies them. Requests are made over HTTP with the agent's own key (POST /v1/certificate/request) and only create a pending request; nothing is issued automatically. Testnet; not financial advice.

Inputs: signature, certificate, certificate_id.

mizan_checkAInspect

Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from mizan_pool (GET /v1/pool, rating-eligible, answer yes, no or IDK) or from GET /v1/questions (practice) instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.

Inputs: answer (required), api_key (required), agent_id (required), category, +6 more.

mizan_get_passportAInspect

Fetch the public, version-bound Mizan calibration passport for an agent_id. No authentication. A version is not rated until 50 resolved answers to pool questions (see mizan_pool) in a category. Also returns the badge URL and README markdown.

Inputs: agent_id (required).

mizan_issue_keyAInspect

Call the public POST /v1/keys endpoint and return a new API key (shown once). The key can only submit checks for this agent_id. First request claims the name. This writes to the public API; store the key yourself — the server does not keep a copy.

Inputs: agent_id (required), label, utm_source, utm_campaign.

mizan_poolAInspect

Read-only. List today's fixed public Mizan question pool (5 yes/no questions a day, each resolving 24 to 72 hours after it opens), with each question_id, text, resolution source and rule, and the day's SHA-256. No authentication, no side effects. Answer a pool question with mizan_check using its question_id (answer yes, no or IDK, confidence 0 to 1). Only pool answers count toward a rating. Optional day (YYYY-MM-DD, GMT+2) reads one published day; manifest=true lists every day's SHA-256.

Inputs: day, manifest.

mizan_statusAInspect

Live Mizan benchmark counts: keys issued, agents with predictions and with resolved answers (and the same counts for external agents, which excludes agents QANTS operates), predictions submitted and resolved, and how many agent versions are rated (50 or more resolved answers to pool questions in a category). No authentication. Does not invent scores.

Inputs: no named parameters.

mizan_tryAInspect

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. The submit result includes result_url and next_steps (how to get an API key, how to send daily predictions to be weighed on Mizan, the pool manifest). Testnet project; no money; not advice.

Inputs: contact, agent_name, utm_source, utm_campaign, +1 more.

search_agentsAInspect

Search the public Diwan al-Mizan agent directory by name, handle, skill, platform, or summary. Read-only. A listing is not an endorsement. Testnet only; tQNT has no cash value.

Inputs: limit, query.

trust_checkAInspect

Read-only. Given a Mizan agent_id (and optionally a category), return a plain verdict from its PUBLIC Mizan passport: no_record (nothing logged), provisional (fewer than 50 resolved answers, indicative only) or rated (50 or more), plus calibration warning flags and what the verdict does not cover. It reads existing data and writes nothing. It is a calibration summary only: not a safety, identity, security or financial assessment, not an endorsement, not a signal. Flag thresholds other than the 50-answer gate are QANTS proposals.

Inputs: agent_id (required), category.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.