Skip to main content
Glama

Evaluate

evaluate

Score text, files, images, URLs, JSON, or YouTube content against a stored or inline rubric to get total scores, grades, pass/fail results, criterion rationales, and pending human tasks.

Instructions

Score a subject against a rubric. rubric is a stored rubric id or an inline rubric. Inputs can be text, json, file (path or base64), url (set screenshot: true to let vision judges such as Clef see the page), image (PNG/JPEG/WebP, path or base64), or youtube. Returns the total score, grade, pass/fail, per-criterion scores with rationales, and any pending human tasks.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "rubric": "<rubric>",
      "inputs": "<inputs>"
    },
    "name": "evaluate"
  }
}

Input Schema

NameRequiredDescriptionDefault
nameNoSubject name for reports
inputsYes
rubricYes
metadataNo
idempotencyKeyNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose real behavior: the return payload (score, grade, pass/fail, per-criterion rationales) and the vision-judge screenshot path. It omits whether an evaluation record is persisted for later retrieval (implied by get_evaluation), any permission or quota constraints, and the meaning of idempotencyKey, so the mutation/state profile stays opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then the rubric disambiguation, then the input taxonomy, then the return shape. The input enumeration is a single long sentence but every clause carries distinct information; no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with nested oneOf input objects and no output schema, the description covers the two required parameters and previews the return values, which is the critical missing piece. Minor incompleteness remains around idempotency, metadata, and the persistence side effect of running an evaluation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20%, but the description compensates well: it explains that `rubric` accepts a stored id or an inline object, and it maps each input variant to its required form (file path or base64, image formats, url with screenshot, youtube). It adds no meaning for `metadata`, `idempotencyKey`, `label`, or `tags`, leaving a residual gap on auxiliary parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Score a subject against a rubric') and immediately qualifies the core parameter (stored id vs inline rubric). An agent can tell this apart from siblings like compare, create_rubric, or check_against_humans without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the verb and the input-type enumeration, so an agent knows this is the tool for grading content against a rubric. However, there is no explicit when-not guidance and no routing to alternatives such as compare (rubric-less comparison) or check_against_humans (human calibration), even though those siblings plausibly overlap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.