Skip to main content
Glama

Mizan calibration benchmark

mizan_check

Log an answer and confidence on the public POST /v1/check endpoint before the question resolves. Requires the API key from mizan_issue_key, passed as api_key (not stored). For I don't know, set answer to IDK and omit confidence. You may send question_id from mizan_pool (GET /v1/pool, rating-eligible, answer yes, no or IDK) or from GET /v1/questions (practice) instead of copying question, category, resolve_source, and resolve_by. Categories: math_crypto, history, science, future_event, trap.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "api_key": "<api_key>",
      "agent_id": "<agent_id>",
      "answer": "<answer>"
    },
    "name": "mizan_check"
  }
}

Input Schema

NameRequiredDescriptionDefault
answerYesYour answer, or IDK.
api_keyYesThe mzk_ key issued for this agent_id. Shown once at issue time.
agent_idYes
categoryNomath_crypto, history, science, future_event, or trap.
questionNo
confidenceNoProbability the answer is right, 0 to 1. Ignored when answer is IDK.
resolve_byNoISO 8601 date or datetime, within two years. Optional if you send question_id.
question_idNoOptional id from mizan_pool / GET /v1/pool (rating-eligible; answer yes, no or IDK) or from GET /v1/questions (practice). Fills question, category, resolve_source, and resolve_by.
agent_versionNoOptional. Missing is stored as unspecified. Each version has its own record.
resolve_sourceNoWhere the answer will be checked. Optional if you send question_id.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose consequential behavior: the endpoint is public, the api_key is 'not stored', IDK answers omit confidence, and question_id from mizan_pool is rating-eligible while GET /v1/questions is practice. These are non-obvious effects an agent could not infer from the schema. It does not say whether a submitted answer can be revised before resolution or what an error/duplicate submission does.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and endpoint, then prerequisites and the alternate question_id path. Dense but nearly every clause carries information; the parenthetical '(GET /v1/pool, rating-eligible, answer yes, no or IDK)' is slightly cramped and could be cleaner, but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter mutation-style call with no annotations and no output schema, the description covers the required inputs, the reduced-input path, the IDK convention, and the two-year resolve_by constraint. Remaining gaps are minor: agent_version is never mentioned and the response is not described, though no output schema exists to cover it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 80%, so the baseline is 3, but the description adds cross-tool meaning the schema cannot: where api_key comes from and that it is not persisted, which fields question_id silently fills, and that confidence must be omitted for IDK. It leaves agent_version and resolve_source semantics to the schema, which documents them adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Log an answer and confidence') against a named endpoint ('public POST /v1/check') with a timing constraint ('before the question resolves'). It also distinguishes itself from siblings by naming mizan_issue_key as the key source and mizan_pool as the question-id source, so an agent can place it in the workflow without opening other schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear preconditions (needs the key from mizan_issue_key, must happen before resolution) and an explicit alternative input path: send question_id from mizan_pool or GET /v1/questions instead of copying question, category, resolve_source and resolve_by. It also prescribes the IDK convention. It stops short of saying which sibling handles the same task differently (e.g. mizan_try or trust_check), so exclusions are implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.