Skip to main content
Glama
Hardik-Singh

Invariance MCP

Official
by Hardik-Singh

invariance_monitor_preview_evaluator

Read-only

Dry-run a monitor evaluator against historical runs to preview matched nodes, match reasons, and observed values—no data written.

Instructions

Dry-run a monitor evaluator against history to see which nodes it would match and why. Writes nothing. Returns {sampled, matched, matches:[{run_id,node_id,matched,reason,observed_value?}]}.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
bodyYesJSON object string with {evaluator} (required, same shape as create) plus optional target and sample_limit. Example: {"evaluator":{"type":"keyword","field":"output.text","keywords":["refund"]},"target":{"kind":"current_run"}}

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.0

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint and openWorldHint, and the description reinforces this with 'Writes nothing.' It also discloses that it returns a sample/matches summary with per-node reasons, which is behavioral detail not in the annotations. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences put the main purpose first, then safety and output contract. Every clause earns its place; there is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter dry-run tool with rich schema coverage and read-only annotations, the description is nearly sufficient: it specifies the return contract ({sampled, matched, matches...}) and side-effect profile. It falls just short of naming the sibling tools/conditions that differentiate it from monitor_evaluate or preview_target, but this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the single parameter, including the required evaluator shape, optional target/sample_limit, and an example. The description's return-shape note does not explain the body parameters, so it adds no semantic detail beyond the schema; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with the specific verb 'Dry-run' and names the exact resource ('monitor evaluator') and goal ('see which nodes it would match and why'). It also states the output shape, making the tool's function unmistakable and distinct from monitor execution or preview-target tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description communicates a clear use case—validating an evaluator against history before real execution—but never explicitly contrasts it with siblings like invariance_monitor_evaluate or invariance_monitor_preview_target. No when-not conditions or alternatives are named, so an agent must infer when to choose this tool from the word 'Dry-run' and 'Writes nothing'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools