AI PM Lab

Quiz me on real AI results

quiz_me

Read-only

A round from AI PM Lab's challenges, built from real recorded runs: Call It (predict what a model did), Spot the Failure (ship or flag a real output) or Which Wins? (pick the better of two setups). Returns the question without its answer; call again with the round id and the user's choice to mark it, see how players on the site called it, and get the next round.

Input Schema

Name	Required	Description
`game`	No	call-it, spot-the-failure or which-wins; any if left out
`round`	No	The id of the round the user is answering, from the last quiz_me result
`answer`	No	The user's choice for that round: its letter

Output Schema

Name	Required	Description	Default
`text`	Yes	The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text
`links`	Yes	The AI PM Lab pages the answer links to, where each idea can be tried hands-on

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 12, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "additionalProperties": false,
+  "properties": {
+    "links": {
+      "description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on",
+      "items": {
+        "type": "string"
+      },
+      "type": "array"
+    },
+    "text": {
+      "description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text",
+      "type": "string"
+    }
+  },
+  "required": [
+    "text",
+    "links"
+  ],
+  "type": "object"
+}

AddedOct 11, 2026

TDQS

A4.2/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the safety profile (readOnlyHint, destructiveHint=false, openWorldHint=false, idempotentHint=false). The description adds genuinely useful behavior not in the structured data: the answer is deliberately withheld on the first call, a second call with round+answer performs the marking and returns cohort stats plus the next round. It could say more about whether the marking call persists state, which leaves a mild gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence front-loads the game modes before the second sentence explains the two-phase return behavior, and no sentence is wasted. It is a touch dense because the mode list and the call flow are packed together, but the structure is sound.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not detail return values, and it still covers the key non-obvious behavior: the answer is withheld, a second call marks it and returns the next round plus community results. That is complete enough for an agent to drive the tool across calls.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents game, round, and answer. The description explains the friendly names of the enum game modes and ties 'round id' and 'the user's choice' to their parameters, but adds only marginal meaning beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource (a quiz round from AI PM Lab's challenges) and enumerates the three modes (Call It, Spot the Failure, Which Wins?), clarifying exactly what this tool produces. It is distinguishable from siblings like list_challenges and get_lesson, which merely enumerate or teach rather than deliver an interactive round.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives strong procedural guidance for both phases: the first call returns a question without its answer, and a second call with the round id and choice marks the answer, shows community results, and fetches the next round. It does not name when-not-to-use it or point to alternative tools, so it stops short of the explicit alternatives a 5 would require.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All AI PM Lab tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?