Skip to main content
Glama
Sign In

Mizan Calibration

by io.github.Allanoo99
OverviewDiscussionsAdmin

mizan_try

Test your own calibration in under 60 seconds, no signup and no key. Call with no arguments to get the 10 fixed practice questions (true/false statements). Then call again with probabilities = {"q01": 0.9, ...}: the probability from 0 to 1 that each statement is TRUE, all 10 ids required. Returns your Brier score (lower is better; 0.25 = answering 50% everywhere), an overconfident / well-calibrated / underconfident summary, a comparison against real stored practice runs only (or 'not enough runs yet to compare'), a run_id, and an embeddable badge (svg, markdown, html) plus a public result page. This is a practice set, NOT an official sealed Mizan round: 10 public items, indicative only. Writes one anonymised practice run on submit. Testnet project; no money; not advice.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "mizan_try"
  }
}

Input Schema

NameRequiredDescriptionDefault
probabilitiesNoOptional. Map of question id (q01..q10) to the probability 0..1 that the statement is TRUE. Omit to fetch the questions instead.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it discloses that submitting writes one anonymised practice run, that no signup or key is needed, that the set is public and indicative only, and that the project is testnet with no money or advice. It also describes the result contents including scoring interpretation (0.25 = answering 50% everywhere) and the comparison fallback text.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the value proposition and then walks through mode 1, mode 2, and return values in a logical order. It is dense but nearly every clause carries information; the return-value enumeration and caveat tail are slightly long but justified given the absent output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description must explain both the call protocol and the returns, which it does thoroughly: Brier score, calibration summary, comparison behavior, run_id, badge formats, and public page. It also flags the practice-only limitation, leaving no material ambiguity for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning beyond the schema: the probability is that each statement is TRUE, all 10 ids are required on submit, and omitting the parameter switches the tool into question-fetch mode. That mode-switching semantics is not evident from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('test your own calibration'), and immediately distinguishes the two modes: no-argument call fetches the 10 practice questions, and a call with probabilities submits answers. It also explicitly distinguishes itself from an official sealed Mizan round, so an agent can separate it from mizan_check-style tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear when-to-use guidance for both call patterns ('Call with no arguments to get the 10 fixed practice questions' then 'call again with probabilities'), including the prerequisite of all 10 ids. It does not name a specific sibling to use for official rounds, but the exclusion of official rounds is stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Mizan Calibration toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources