Mockbird

verdict

One call = the whole eval grade. Composes the state check (snapshot diff vs live data) with trajectory constraints on the request log into a single {pass, checks[]} verdict. Pass snapshot:"expected" to require live data to match that snapshot (author it as the answer key first via snapshots action:"save" with data), and/or trajectory constraints like [{method:"DELETE", count:0}, {method:"POST", path:"/orders", count:1}, {status_gte:400, count:0}] — each needs an expectation: count (exact), min and/or max. Typical harness: fork_project per run → agent works the fork → verdict {snapshot:"expected", trajectory:[...]} → assert .pass → delete_project. Trajectory counts see the retained request window (last 50) — fork per run so the log is exactly one episode's trace. Or pass name to run a SAVED spec (authored via PUT /api/projects/:id/verdicts/:name; forks copy them) — graders without the admin key can run saved specs keylessly via the share link: GET /api/share/:token/verdict/:name.

Input Schema

TableJSON Schema

Name	Required	Description
`name`	No	Optional: run a saved verdict spec by name instead of an inline spec (mutually exclusive with snapshot/ignore/trajectory).
`ignore`	No	Optional (with snapshot): comma-separated field names excluded from the state comparison, e.g. "updatedAt,createdAt".
`project`	Yes	Project id.
`adminKey`	Yes	The project's adminKey.
`snapshot`	No	Optional: snapshot name to diff live data against — pass means identical.
`trajectory`	No	Optional: constraint objects — filters (method comma-list, path segment-prefix, status, status_gte, status_lte, since) + expectation (count exact, min, max). Example: [{"method":"DELETE","count":0}].

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedSep 10, 2026

addedInput schema / properties / name

Added value: +{
+  "description": "Optional: run a saved verdict spec by name instead of an inline spec (mutually exclusive with snapshot/ignore/trajectory).",
+  "type": "string"
+}

AddedSep 10, 2026

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the full behavioral burden. It discloses the last-50 request window, the mutual exclusivity of inline spec vs saved spec, the need to author the snapshot first, and the admin-key-free share-link path. It does not explicitly state whether the call mutates state or is read-only, but the workflow strongly implies evaluation-only behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Information-dense and for the most part well-sequenced (definition → usage → saved-spec mode), but the first sentence packs several concepts together and the block is somewhat dense for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the full operational context: authoring snapshots, trajectory examples with count semantics, the 50-request log window, the fork-per-run pattern, saved specs, and the share-link auth model. Practically all behaviors an agent needs to invoke this correctly are in the description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes each parameter, and the description adds critical usage detail: how snapshot interacts with the snapshots tool, how trajectory constraints are shaped, and how name resolves to a saved spec. The ignore parameter is named but not expanded in the prose; however, the schema carries that weight.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's purpose: produce a single pass/checks verdict from a state diff and trajectory constraints. It differentiates the tool from sibling operations (snapshots, projects) by framing it as the verdict step in a fork → act → verify → delete harness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides concrete workflow guidance: fork a project, run the verdict with snapshot and/or trajectory, assert pass, then delete. It also explains when to pass a snapshot name vs. raw inline data deme, and how saved specs plus share links work for graders without an admin key.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Mockbird tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?