Skip to main content
Glama

Mizan calibration benchmark

mizan_try

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. The submit result includes result_url and next_steps (how to get an API key, how to send daily predictions to be weighed on Mizan, the pool manifest). Testnet project; no money; not advice.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "mizan_try"
  }
}

Input Schema

NameRequiredDescriptionDefault
contactNoOptional. Email, GitHub URL or agent URL. Optional — we use it only to send you your calibration result when the questions resolve. Never shown publicly.
agent_nameNoOptional. Your agent's name (1-64 characters). Optional — we use it only to send you your calibration result when the questions resolve.
utm_sourceNoOptional. Where you found Mizan: x, linkedin, email, discord, directory, facebook, instagram or whop (anything else is counted as other).
utm_campaignNoOptional. Campaign tag from the link you followed.
probabilitiesNoOptional. Map of question id (q01..q10) to the probability 0..1 that the statement is TRUE. Omit to fetch the questions instead.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose the key behaviors: no signup or key, one anonymised practice run written on submit, comparison limited to stored practice runs, returned Brier score/calibration summary/run_id/badge/result_url, and testnet/no-money status. This is strong behavioral disclosure for a tool that can mutate stored practice data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the purpose and the no-argument discovery step, then packs submission, returns, and caveats into a dense paragraph. Every sentence carries useful information, but the single-paragraph density makes it slightly harder to scan than it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter nested-object tool with no output schema and no annotations, the description is complete: it explains both calling modes, submission requirements, return values, data-writing side effect, public result page, comparison caveat, and scope limitations. No critical behavior is left solely to the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines contact, agent_name, utm fields, and probabilities. The description adds meaningful constraints not encoded in the schema: all 10 q01–q10 ids are required, probabilities mean P(statement is TRUE), and omitting probabilities fetches questions. It does not add much about the optional contact and attribution parameters, which is acceptable because the schema covers them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action (a two-step calibration self-test), the resource (10 fixed practice questions and a probability submission), and its scope (practice, not an official sealed round). An agent can distinguish it from official-round or passport siblings without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit two-step usage flow: no arguments first to fetch q01–q10, then resubmit with a complete probabilities map. It also states the practice-only exclusion and 'indicative only' limitation, but it does not name a sibling for official rounds, so routing is clear yet not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.