Skip to main content
Glama
Sign In

MCP for Claude & ChatGPT

by com.tuqui
OverviewDiscussionsAdmin

Evaluate content

judge
Read-only

Ask a judgment model for decisions your code or your answer can act on: it returns a typed answer with a probability per option and a confidence, never prose. Three question types. 'choice' picks one option from a set you define (up to 255). 'score' rates against 2-10 ordered levels you describe. 'yes_no' answers a yes/no as a probability. Ask as many as you want in ONE call — they run in parallel and each extra question costs only its own tokens. Use it instead of reasoning when the judgment repeats over many items (classify these rows, rank these candidates, does this passage answer the question) or when you want a confidence you can branch on: act when it is high, ask the user when it is low. Do NOT use it for anything whose answer is text — it cannot write, explain or summarise. Send the relevant extract as the state, not everything you have: accuracy drops as unrelated content grows. The current judgment model takes a state of up to ~32,000 tokens. It also looks at images you pass with the state.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "state": "<state>",
      "questions": "<questions>"
    },
    "name": "judge"
  }
}

Input Schema

NameRequiredDescriptionDefault
stateYesWhat the questions are about: a string, or an object/array when the material has structure. Send the relevant extract — a sample plus the shape beats the whole payload.
questionsYesOne entry per question, keyed by a name you choose (it comes back on the answer). Each is {"type": "choice"|"score"|"yes_no", "instructions": "...", "criteria": ...}. criteria is an object of option -> meaning for a choice, an ordered list of level descriptions for a score, and is omitted for a yes_no.
image_attachment_idsNoOdoo ir.attachment ids of images the questions are also about, e.g. the photo on a delivery or a ticket. PNG, JPEG, WebP or GIF.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses the approximate 32,000-token state limit, image support, parallel question execution, and the fact that extra questions cost only their own tokens. It also warns that accuracy drops with unrelated content, which is valuable behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and return behavior, then follows with question types, usage guidance, and limits. It is somewhat long, and the 'never prose' point is repeated later, but nearly every sentence carries useful operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex judgment tool with nested question definitions and no output schema, the description explains the return concept, three question modes, input composition, token limit, and image support. Together with the detailed schema and annotations, an agent has enough information to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all three parameters at 100% coverage, but the description adds useful constraints and semantics such as choice questions allowing up to 255 options, score questions using 2-10 ordered levels, and yes_no returning a probability. It also reinforces that state should be a relevant extract rather than the full payload.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: ask a judgment model for decisions and receive a typed answer with probabilities and confidence. It clearly distinguishes the tool from prose-producing siblings by emphasizing 'never prose' and 'cannot write, explain or summarise'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit when-to-use guidance, including repeated judgments over many items and cases where confidence should drive branching. It also names when not to use it: anything whose answer is text.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All MCP for Claude & ChatGPT toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources