OpenRouter

spawn-ori-eval

Read-onlyIdempotent

Get the instructions for running a model eval with Ori, then follow them. Ori runs the user's own agent on their own prompts, on a pinned harness and model, and grades what it did — so a score change means the model changed, not the environment. Call this tool FIRST, before writing any eval code: it returns a step-by-step recipe (install and auth checks, how to spawn ori code -p, how to relay Ori's scoping questions to the user, how to report results) that you carry out yourself. Do not hand-roll an eval instead.

Use it when the user asks which model they should use, wants to compare or bake off models, wants to measure whether their agent or prompt does the right thing, wants to catch regressions in agent behavior, or asks how good their current model is. Works for any codebase in any language.

Do not use it for plain unit tests that involve no model, and do not use it to re-run an eval that already exists (run ori eval <file> directly instead). Takes no arguments; the same document is published at https://openrouter.ai/skills/spawn-ori-eval.

Input Schema

TableJSON Schema

Name	Required	Description	Default
No arguments

TDQS

A4.9/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, idempotent, openWorld, and non-destructive behavior. The description adds useful context: the tool returns a recipe the agent must follow, includes installation and auth checks, requires relaying Ori's scoping questions to the user, and has a documented public URL. This clearly sets expectations beyond the annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is moderately detailed but every section earns its place: first-sentence purpose, followed by when-to-use, when-not-to-use, and a parameter note. Key phrases like 'Call this tool FIRST' are front-loaded, and the no-arguments statement prevents extra lookups.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-input, no-output-schema tool, this description fully explains what will happen, how to invoke it, what the agent must do after calling it, and where the upstream document lives. An agent can select and use this tool correctly without needing additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no parameter burden. The description explicitly says 'Takes no arguments', which satisfies the baseline for a no-parameter tool and removes any doubt.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: get and then follow instructions for running an Ori eval, rather than executing an eval directly. It also distinguishes itself from hand-rolling or re-running an existing eval, making its role unique among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly enumerates applicable use cases: model comparison, baking off models, measuring agent/prompt behavior, catching regressions, and assessing current model quality. It also gives exclusions — plain unit tests without a model and already-existing evals — and names the alternative command `ori eval <file>`.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All OpenRouter tools Inspect

Try in Browser

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.9/5.0

Disambiguation4/5

Most tools are clearly differentiated by resource and action: the eight list-* tools each target a distinct view (models, endpoints, rankings, apps, providers, presets, tasks, benchmarks), and cross-references between them reduce mis-selection. A few mild boundaries exist—list-models and list-benchmarks both include benchmark data, and install-ori-harness vs spawn-ori-eval are both Ori recipe tools—but their detailed descriptions mostly resolve these.

Naming Consistency4/5

The naming is overwhelmingly consistent with a verb_noun pattern using the same prefix set: generate-, get-, list-, send-, along with install-, spawn-, search-, and transcribe-. The only deviation is ping, which is a standard bare health-check tool and does not follow the verb_noun convention.

Tool Count3/5

At 22 tools, the set feels heavier than the ideal 3-15 range, though each tool is arguably purposeful given the broad surface: model catalog, rankings, benchmarks, presets, generation, audio, image, docs, uptime, credits, and Ori workflows. The variety justifies the size to some extent, but the sheer number puts it in borderline territory.

Completeness4/5

Core workflows are well covered: model discovery (get-model, list-models, list-model-endpoints), generation (send-message, generate-image, generate-speech, transcribe-audio), observability (get-credits, get-generation, get-endpoint-uptime-history), and docs. Notable gaps include no create/update/delete for presets and no persistent provider configuration methods, but these are workable since presets are dashboard-managed and providers can be pinned per request.

Resources

Need Help?