Skip to main content
Glama

Quiz me on real AI results

quiz_me
Read-only

A round from AI PM Lab's challenges, built from real recorded runs: Call It (predict what a model did), Spot the Failure (ship or flag a real output) or Which Wins? (pick the better of two setups). Returns the question without its answer; call again with the round id and the user's choice to mark it, see how players on the site called it, and get the next round.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "quiz_me"
  }
}

Input Schema

NameRequiredDescriptionDefault
gameNocall-it, spot-the-failure or which-wins; any if left out
roundNoThe id of the round the user is answering, from the last quiz_me result
answerNoThe user's choice for that round: its letter

Output Schema

NameRequiredDescriptionDefault
textYesThe answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text
linksYesThe AI PM Lab pages the answer links to, where each idea can be tried hands-on

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "$schema": "https://json-schema.org/draft/2020-12/schema",
      +  "additionalProperties": false,
      +  "properties": {
      +    "links": {
      +      "description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on",
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "text": {
      +      "description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text",
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "text",
      +    "links"
      +  ],
      +  "type": "object"
      +}
  2. Added

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by disclosing the key behavior: the first response deliberately withholds the answer, and only the second call reveals scoring plus how other players answered. This is consistent with readOnlyHint=true and idempotentHint=false (each fresh call yields a new round). The phrase 'mark it' implies a write, which sits in mild tension with readOnlyHint, but the read-only nature of the round-trip is recoverable from the described flow.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what the tool is and immediately followed by the return/round-trip contract. The parenthetical glosses are compact and each earns its place, though the first sentence is dense enough that an agent must parse a list before reaching the actionable flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, yet the description still covers the essential return shape (question without answer, then scoring, peer results, and next round), which is exactly what an agent needs to sequence calls correctly. Nothing required to invoke this three-optional-parameter tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, and the description earns an extra point by explaining what the three enum values actually mean in play (Call It = predict what a model did, Spot the Failure = ship or flag a real output, Which Wins? = pick the better of two setups). It also reinforces that the round id comes from the previous result, which clarifies the sequencing dependency.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource ('a round from AI PM Lab's challenges') and enumerates the three game modes with a one-line gloss of each, so an agent knows exactly what comes back. It does not explicitly distinguish itself from the sibling list_challenges, which is the one nearby tool an agent could confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The two-phase flow is spelled out: call with no arguments to get a question, then call again with the round id and the user's choice to mark it and get the next round. That is clear operational context, but there is no explicit statement of when to prefer this over list_challenges or the other learning tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources