Skip to main content
Glama

WAI Quantum job service

Grade a program against a task's cases

grade_quantum_program
Read-onlyIdempotent

Grade a program against a task (wai.quantum.grade/1, quantum-providers.md §8.10): each case run with its intent and graded passed, failed or unchecked, with the verdict that decided it and that verdict's layer and limits; each constraint measured; what went wrong with where to look; and whether the program passes. The grade is a job sealed with the service's key, the task and the program in its request, so anyone can run it again and get the same answer: verify_quantum_result works it out again. Quote each case's grade and layer; a case decided by evidence is not proved. Each case's intent is held to the service's bounds before any case runs, and a grade past them is refused with the local command (wai-quantum grade).

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "grade": "<grade>"
    },
    "name": "grade_quantum_program"
  }
}

Input Schema

NameRequiredDescriptionDefault
gradeYes

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and closed-world, and the description reinforces rather than contradicts them by explaining why the result is deterministic ('sealed with the service's key... anyone can run it again and get the same answer'). It adds genuine behavioral context beyond the annotations: intents are checked against service bounds before any case runs, and over-bound grades are refused with the local `wai-quantum grade` command. It loses a point for not saying what the sealing implies for storage or visibility.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, but the body is two long, comma-spliced sentences packed with spec references and repeated restatements of return contents. Several clauses (verdict layer and limits, quoting rules) earn their place, while the return-value enumeration duplicates the existing output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not spell out return values, yet it spends substantial text doing so while leaving the input structure under-explained for a required nested object with 0% coverage. The bounds/refusal behavior and determinism story are strong, but the reliance on external documents (wai-quantum://docs/verdicts, §8.10) means an agent without them may not fully grasp the contract.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is reported as 0% at the top level, so the description should carry more weight for the nested `grade` object; instead it only gestures at concepts (cases, intents, constraints, program) without mapping them to fields or clarifying the required {task, program} wrapper. The rich nested property descriptions in the schema do much of the work, so a middle score is fair, but the description adds little field-level meaning of its own.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause names a specific verb and resource ('Grade a program against a task') and immediately unpacks what the grade contains (per-case passed/failed/unchecked, deciding verdict, layer and limits, constraints measured, overall pass). It also distinguishes itself from verify_quantum_result by noting that sibling 'works it out again'. It stops short of 5 because the dense spec-jargon (wai.quantum.grade/1, §8.10) makes the core action harder to grasp than the sibling distinction warrants.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the purpose: you call it when you want a sealed grade of a program against a task. The one comparative statement points at verify_quantum_result for re-derivation, but the more obvious sibling, check_quantum_program, is never mentioned, and there is no explicit when-to-use/when-not framing or prerequisite statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources