Skip to main content
Glama

ToolQuest

The escape room for tool-using agents.

ToolQuest exposes deterministic puzzle rooms as a local Model Context Protocol (MCP) server. Connect an agent, let it explore with tools, and inspect a reproducible event trace and score at the end.

简体中文

Why ToolQuest?

Most agent demos show only the final answer. ToolQuest makes the path testable:

  • deterministic room state and scoring;

  • strict, machine-readable tool results;

  • explicit run isolation and optimistic state versions;

  • idempotency keys for safe action retries;

  • atomic local run persistence and restart recovery;

  • deterministic replay verification and Markdown reports;

  • JSONL traces with redacted final answers;

  • no LLM judge and no external service required.

Related MCP server: pypddlengine

Quick start

Requirements: Node.js 20 or newer.

npm install
npm run check
npm start

For the visual interface, run:

npm run web

Then open http://127.0.0.1:4310. The interface is designed for non-technical players and keeps all run data on the local machine.

The server communicates over stdio. A common MCP client configuration looks like this; replace the path with an absolute path on your machine:

{
  "mcpServers": {
    "toolquest": {
      "command": "node",
      "args": ["/absolute/path/to/toolquest/dist/server.js"]
    }
  }
}

Agent loop

  1. Call list_rooms and choose a challenge.

  2. Call start_run with the selected roomId.

  3. Call look with the returned runId.

  4. Inspect visible target IDs to discover clues and interaction IDs.

  5. Use move or use with a unique actionId and the latest stateVersion.

  6. Call submit when the final mechanism is ready and you know the answer.

  7. Call replay_run to verify the trace and export_report for a Markdown result.

After a client or server restart, call list_runs to rediscover recent run IDs, then call get_run and continue from the returned stateVersion and public snapshot.

Visual interface

Version 0.4 adds a local-first browser experience powered by the same RunService as the MCP server. Players can choose a room, inspect visible objects, move, use inventory items, submit answers, resume earlier runs, inspect the public event timeline, verify deterministic replay, and download a redacted report.

The browser never receives hidden room definitions or plaintext answers. The Web server listens only on 127.0.0.1, applies restrictive browser security headers, limits request bodies, and requires a per-process page token for every state-changing request.

MCP tools

Tool

Purpose

Changes world state

list_rooms

Discover challenges, difficulty, and par actions

No

list_runs

Discover recent persisted runs; filter by status and limit

No

start_run

Create an isolated deterministic run

Creates a run

get_run

Resume a persisted run with a public snapshot

No

replay_run

Rebuild and verify a run from its event log

No

export_report

Return a redacted Markdown benchmark report

No

look

Read location, objects, exits, and inventory

No

inspect

Read an object's clue and interactions

No

move

Move to a destination returned by look

Yes

use

Execute an interaction returned by inspect

Sometimes

submit

Submit the final room answer

Sometimes

Mutating calls require:

  • actionId: a unique retry key;

  • expectedStateVersion: the latest version returned by ToolQuest.

An exact retry with the same actionId returns the cached first result. Reusing an actionId with different arguments is rejected.

Result shape

Every successful call returns text for broad client compatibility and structuredContent for deterministic automation:

{
  "ok": true,
  "runId": "run_...",
  "eventSeq": 8,
  "stateVersion": 3,
  "stateHash": "ed39a61c",
  "status": "active",
  "message": "The brass key turns.",
  "data": {},
  "events": []
}

Normal game failures, such as using the wrong item or missing an interaction prerequisite, remain successful MCP calls with a world_failure event. Invalid IDs, stale versions, and missing runs are recoverable MCP tool errors with a stable code and recoveryHint.

Persistent runs and traces

By default, the stdio server atomically persists authoritative run state and appends a separate public event trace:

.toolquest/state/<runId>.json
.toolquest/runs/<runId>.jsonl

Use TOOLQUEST_STATE_DIR to change the state directory. Set TOOLQUEST_DISABLE_STATE=1 for ephemeral in-memory runs, or TOOLQUEST_DISABLE_TRACES=1 to disable public traces.

State files are private server data. Action arguments are stored only as SHA-256 idempotency digests, and the submitted answer is never written in plaintext. Public JSONL events contain only answer length and outcome. Run discovery returns only public summaries. Structurally malformed state files fail closed instead of returning partial records.

Architecture

MCP transport
      |
MCP schema and presenters
      |
RunService
      |
deterministic domain engine
      |
repository, clock, IDs, event sink

The domain and application layers do not import the MCP SDK. See docs/architecture.md for boundaries and invariants.

Development

npm run typecheck
npm run lint
npm test
npm run build
npm run check

The test suite includes domain and application tests, Web API security and flow, restart discovery and recovery, malformed-state rejection, tamper-detecting replay, report redaction, an in-memory MCP contract test, and isolated real stdio subprocess tests.

Built-in rooms

Room ID

Difficulty

What it tests

the-vault

Starter

Exploration, clue combination, item use

signal-station

Intermediate

Multi-location planning, consumed items, chained prerequisites

Each room publishes a par action count so efficiency scores remain comparable as scenarios become more complex.

Current scope

Version 0.4 includes a responsive local Web interface, two built-in rooms, eleven MCP tools, atomic local run persistence, restart discovery and recovery, deterministic event replay, redacted Markdown reports, JSONL traces, and room-aware scoring. The file repository supports one server process per state directory. Remote hosting, authentication, community room loading, multi-process transactions, and a public model leaderboard remain out of scope.

Contributing and security

Read CONTRIBUTING.md before opening a pull request. Report security issues according to SECURITY.md. Release notes are in CHANGELOG.md.

License

MIT

Available Tools

10 tools
export_reportExport ToolQuest Run ReportA
Read-onlyIdempotent

Generate a Markdown benchmark report containing run metadata, score, replay verification, and a redacted event timeline. Returns content without writing a file.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal read-only, idempotent, non-destructive behavior. The description adds meaningful context beyond those annotations by disclosing the return behavior ('Returns content without writing a file') and the redaction of the event timeline, which informs the agent about side-effect-free and privacy-conscious execution.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence. It front-loads the primary action and resource, then lists the report contents and closes with the key non-writing behavior. Every clause earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with a rich output schema and comprehensive annotations, the description is sufficiently complete. It covers what the report contains, the output format, the redaction behavior, and the fact that no file is written, leaving no critical gap for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%; the runId parameter is fully described as an 'Opaque run ID returned by start_run.' The description itself does not add extra parameter details, but because the schema already carries the full parameter meaning, a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('Generate'), a concrete deliverable ('Markdown benchmark report'), and the exact contents (run metadata, score, replay verification, redacted event timeline). It also distinguishes itself from file-writing by stating it 'Returns content without writing a file,' which separates it from any sibling that might persist output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The implied use case is clear: call this tool when you need a Markdown benchmark report, and the 'without writing a file' note clarifies output handling. However, there is no explicit guidance about when to choose this over siblings like get_run or replay_run, nor any stated exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runGet ToolQuest RunA
Read-onlyIdempotent

Resume a persisted run by reading its public current snapshot, status, stateVersion, stateHash, event count, and score when terminal. This does not append an event.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive semantics, so the description only needs to add useful behavioral detail. It adds that no event is appended and that score is only present when terminal, which is meaningful beyond the annotations. No contradiction found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The action and resource are front-loaded, followed by a compact field list and a one-sentence behavioral caveat. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one fully documented parameter, an output schema, and annotations covering read-only/idempotent/non-destructive behavior, the description supplies enough to select and invoke the tool correctly. The only additional context, non-appending behavior, is included.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with runId described as an opaque ID from start_run. The description adds the 'persisted run' context but does not need to explain runId further; baseline 3 is appropriate because the schema carries the parameter burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('Resume a persisted run by reading'), a concrete resource (current snapshot), and the exact fields returned (status, stateVersion, stateHash, event count, score). The closing 'This does not append an event' separates it from event-producing siblings like start_run or replay_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly states the context: use this to resume/read a persisted run's current state. It also states a non-behavior that distinguishes it from event producers, though it does not explicitly name alternative sibling tools or give a when-not-to-use condition beyond not appending an event.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inspectInspect ObjectA
Read-onlyIdempotent

Inspect one visible object by targetId. Returns clues and valid interaction IDs without changing virtual world state.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.
targetIdYesID of an object returned by the latest look result.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds that the call doesn't change virtual world state and that it returns clues plus valid interaction IDs, which gives useful behavioral context beyond the structured hints. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler. The core purpose is front-loaded, and the side-effect guarantee and return content are stated in the second sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, read-only inspection tool with an output schema and clear annotations, the description covers what the tool does, what it needs, what it returns, and that it is side-effect free. Nothing necessary for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters are fully documented in the schema, including the origin and constraints of targetId and the pattern for runId. The description adds only visibility wording ('visible object') but does not need to paraphrase what the schema already explains.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Inspect'), a specific resource ('one visible object by targetId'), and states what the call yields (clues and valid interaction IDs). This distinguishes it from the sibling tools such as look, use, and move, since it is a targeted, non-mutating inspection action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this should be used after look has provided a targetId and when the agent wants clues or interaction IDs for one object. It does not explicitly say when not to use it or name an alternative, leaving some inference to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_roomsList ToolQuest RoomsA
Read-onlyIdempotent

Discover all built-in rooms with their IDs, versions, difficulty, introductions, and par action counts. This does not create or change a run.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description reinforces this by stating it does not create or change a run. It also adds behavioral context about the returned room attributes, which goes beyond the annotation metadata. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The first sentence front-loads the core purpose and return details, and the second sentence usefully clarifies side-effect behavior. Both sentences earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero params, a rich annotation set, an output schema, and no nested objects, the description covers everything needed to invoke the tool correctly. It states what the tool does, what it returns, and that it has no side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description correctly focuses on return value content rather than parameter details, since there are no inputs to document. The schema's 100% coverage further removes any ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb ('Discover') and a specific resource ('all built-in rooms'), and enumerates the attributes returned (IDs, versions, difficulty, introductions, par action counts). It is distinct from run-oriented siblings like start_run or get_run, though it does not explicitly differentiate itself from exploration tools like look or inspect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrasing 'Discover all built-in rooms' implies a list/exploration use case, and the note 'This does not create or change a run' implies safe use without side effects. However, it does not state when to prefer this over siblings such as look or inspect, nor does it provide explicit conditions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lookLook AroundA
Read-onlyIdempotent

Read the current room, visible object IDs, destination IDs, inventory, and attempts remaining. This does not change virtual world state.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, and the description reinforces this with 'This does not change virtual world state.' It also discloses the specific behavioral scope (room, visible objects, destinations, inventory, attempts). While the state-change statement aligns with annotations, it adds little beyond them, but the explicit behavioral detail is useful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, front-loaded with the action and resource. The first sentence lists exact outputs, and the second sentence clarifies side-effect-free behavior. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return value details are already covered. The description covers the operation's scope, safety, and game-specific context (attempts remaining), and the single parameter is fully documented in the schema. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with runId fully documented as an opaque run ID returned by start_run. The description does not add parameter-level detail, so the baseline score of 3 applies because the schema carries the semantic burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Read') and a specific resource ('the current room'), then enumerates exactly what is returned: visible object IDs, destination IDs, inventory, and attempts remaining. This clearly differentiates 'look' from siblings like 'move' or 'submit' and even from 'list_rooms' by focusing on the current room context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for reading the current room state but does not explicitly state when to use it over alternatives like 'inspect' or 'list_rooms'. There is no when/when-not guidance or mention of alternatives, leaving the agent to infer usage from the stated purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

moveMoveA

Move to a destinationId returned by look. This changes virtual room state. Provide a unique actionId and the latest expectedStateVersion; exact retries with the same actionId are safe.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.
actionIdYesUnique idempotency key for this action. Reuse only when retrying the exact same action.
destinationIdYesdestinationId returned by the latest look result.
expectedStateVersionYesLatest stateVersion observed from a previous ToolQuest result.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A3.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description claims 'exact retries with the same actionId are safe,' which is an idempotency guarantee, but annotations set idempotentHint=false. This is a direct contradiction, so per the rubric the score must be 1 despite useful context about state mutation and version checking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences front-load the core purpose, then state the side effect and the key invocation constraints. No filler or redundant framing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the essential operational details: where destinationId comes from, that the action mutates state, and how to handle retries/versioning; output schema covers return values. The idempotency contradiction prevents the definition from being fully coherent, so it is not a perfect 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description reinforces the roles of actionId, expectedStateVersion, and destinationId, but mostly restates the schema's own parameter descriptions rather than adding substantial new meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise action ('Move') and the target source ('a destinationId returned by look'), and explicitly notes it changes virtual room state. This distinguishes it from read-only siblings like look and inspect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly signals it should follow a successful look result by requiring a look-returned destinationId and the latest expectedStateVersion. It does not enumerate exclusions or alternatives beyond look, so it stops short of a full when-not-to-use guide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

replay_runReplay ToolQuest RunA
Read-onlyIdempotent

Deterministically rebuild a run from its redacted event log and verify every stateVersion, stateHash, outcome, and final state. This does not change the run.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive; the description adds non-obvious behavioral context by stating determinism and that verification covers every stateVersion, stateHash, outcome, and final state. It does not contradict the annotations and adds meaningful operational detail beyond the structured hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences convey what the tool does, how it behaves, and that it is non-mutating with no filler. The key action and verification scope are front-loaded before the no-change guarantee.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one required parameter fully described in the schema, an output schema present to define return data, and annotations covering the safety and idempotence profile, the description leaves no critical gap for an agent deciding whether or how to call replay_run.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter runId is fully documented in the schema (opaque ID returned by start_run, pattern and length constraints), so the description does not need to repeat parameter details. Per the high-schema-coverage baseline, this is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('rebuild') and resource ('a run from its redacted event log') and enumerates the verification targets: stateVersion, stateHash, outcome, and final state. This distinguishes replay_run from get_run and export_report even without naming alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The purpose implies when to use it (when a deterministic reconstruction/verification of a run is needed) and the sentence 'This does not change the run' signals safe read-side usage. However, it does not explicitly state when to prefer this over get_run or other siblings, leaving routing largely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_runStart ToolQuest RunA

Create an isolated deterministic escape-room run. Returns an opaque runId, initial stateVersion, room introduction, and next-step guidance.

ParametersJSON Schema
NameRequiredDescriptionDefault
seedNoOptional deterministic seed for reproducing this run.
roomIdNoRoom to start. Call list_rooms to discover the available room IDs.the-vault

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Because the annotations are all false and provide little safety or mutability information, the description carries the burden of explaining behavior. It adds meaningful detail: the run is isolated, deterministic, and returns an opaque runId, initial stateVersion, room introduction, and guidance. This gives the agent a solid mental model beyond what the annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the purpose and lists the return items without excess wording. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only two optional parameters and an output schema present, the description provides enough context for correct invocation. The return values are disclosed concisely; mentioning how replay_run relates could slightly improve context but is not essential for starting a run.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents seed and roomId clearly. The description adds only indirect semantic hints like 'deterministic' linking to the seed and 'isolated' for run behavior, but it does not substantially improve parameter understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create an isolated deterministic escape-room run') with a clear resource, and the listed return values make the tool's function unambiguous. It is easily distinguishable from sibling tools like list_rooms and replay_run because it is the creation entry point.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Create' and the term 'run' imply this is the tool for starting new runs, giving reasonable context. However, the description does not explicitly mention alternatives such as replay_run or state when this tool should be used versus resumed/replayed runs, so usage guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

submitSubmit Room AnswerA

Submit a candidate answer to the final room challenge. Incorrect answers may consume an attempt; a correct answer ends the run and returns a deterministic score.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.
answerYesCandidate answer for the room's final challenge.
actionIdYesUnique idempotency key for this action. Reuse only when retrying the exact same action.
expectedStateVersionYesLatest stateVersion observed from a previous ToolQuest result.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses important behavioral traits that annotations do not cover: incorrect answers may consume an attempt, a correct answer ends the run, and the score is deterministic. This goes beyond the minimal mutation hints and helps the agent predict side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tight sentence that front-loads the action and resource, then packs the key consequences with no filler. Every clause adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a 4-parameter action that involves idempotency and state versioning, the description covers the main behavioral outcomes. The output schema exists, so return values need not be described. It is complete enough for an agent to invoke the tool correctly, though it leaves retry/stale-version behavior to the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already explains all four parameters. The description adds no extra parameter-level meaning, but that is acceptable given the schema's thoroughness.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Submit'), a specific resource ('a candidate answer to the final room challenge'), and the outcome ('a correct answer ends the run and returns a deterministic score'). This clearly distinguishes it from siblings like start_run, get_run, and use, which handle different phases or actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: this is the tool for submitting the final challenge answer. It also implies when not to use it (e.g., before reaching the final room) but does not explicitly name alternatives or exclusions relative to siblings like use or inspect.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

useUse InteractionA

Perform an interactionId returned by inspect, optionally with an inventory itemId. This changes only virtual room state and is protected by actionId and stateVersion.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesOpaque run ID returned by start_run.
itemIdNoOptional inventory item ID to use for the interaction.
actionIdYesUnique idempotency key for this action. Reuse only when retrying the exact same action.
interactionIdYesInteraction ID returned by inspect.
expectedStateVersionYesLatest stateVersion observed from a previous ToolQuest result.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYes
codeNo
dataYes
runIdNo
scoreNo
eventsYes
statusNo
messageYes
eventSeqNo
retryableNo
stateHashNo
recoveryHintNo
stateVersionNo
correlationIdNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is not read-only, not idempotent, and not destructive. The description adds useful context beyond those annotations by stating that only virtual room state changes and that the action is protected by actionId and stateVersion. This gives agents a clearer model of side effects and concurrency expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler. It front-loads the primary operation, then adds the optional parameter and the key behavioral constraints. Every clause contributes necessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema, the parameters are fully documented and the description explains behavior, scope, and protection. It is complete enough to invoke correctly. A small gap is that it gives no explicit 'when not to use' guidance, but this is not necessary given the clear interactionId-from-inspect trigger.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters in detail. The description repeats interactionId's origin and itemId's optionality but adds no new parameter semantics beyond the schema. This is the baseline 3: the schema carries the burden, and the description does not materially supplement it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation: perform an interactionId returned by inspect, optionally with an inventory itemId. It identifies the resource and distinguishes this tool from ancestors like inspect and look. The phrase 'changes only virtual room state' further clarifies the tool's specific effect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the correct context: use this tool after inspect has returned an interactionId, and optionally provide an inventory itemId. It does not explicitly name exclusions or alternatives, but the 'returned by inspect' phrasing gives clear contextual guidance. Sibling tools like move and inspect are implicitly distinguished by this source requirement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A4.1/5.0
Disambiguation5/5

Each tool has a clearly distinct role: run lifecycle tools (start/get/replay/export) are separate from in-room inspection tools (look/inspect) and state-changing actions (move/use/submit). There is no meaningful overlap between any pair of tools.

Naming Consistency4/5

Most tools follow a clear verb_noun pattern such as list_rooms, start_run, get_run, and export_report. The in-room actions use single verbs like look, inspect, move, use, and submit, which is a minor deviation but still predictable and readable.

Tool Count5/5

10 tools is well-scoped for an escape-room benchmark server. Each tool covers a necessary part of discovery, run lifecycle, in-room interaction, and reporting without redundancy.

Completeness4/5

The toolset covers the main workflow: discover rooms, start a run, observe state, act in rooms, submit answers, resume runs, replay, and export reports. Minor gaps like no run-listing or explicit run-abandon tool are not required for the core benchmark use case.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/sysu19351015/toolquest'

If you have feedback or need assistance with the MCP directory API, please join our Discord server