Skip to main content
Glama

noise_estimate

Measure run-to-run noise from a repeated selection arm and record the estimate required by gate_check. Return per-task flip rates and a noise ID for gating.

Instructions

Measure the current base's run-to-run noise from one repeated arm and record the estimate that gate_check requires. Use it once per base, on a complete, non-smoke selection arm of the current version with at least two repeats; use compare_paired for candidate effects. Appends a noise record to the ledger. Returns noise_id, per-task flip rates, net spread, threshold and method; pass noise_id to gate_check.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "config": "<config>",
      "run_id": "<run_id>"
    },
    "name": "noise_estimate"
  }
}

Input Schema

NameRequiredDescriptionDefault
configYesPath of the scenario YAML file on the server host, absolute or relative to the server's working directory, for example `scenario.yaml`. It selects the workspace; a server started with --config rejects any other path.
run_idYesRun id of a complete selection arm of the current base.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv0.1.9
    • addedInput schema / properties / config / description
      Added value: +"Path of the scenario YAML file on the server host, absolute or relative to the server's working directory, for example `scenario.yaml`. It selects the workspace; a server started with --config rejects any other path."
    • addedInput schema / properties / run_id / description
      Added value: +"Run id of a complete selection arm of the current base."
  2. First observedv0.1.7

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare a non-read-only, non-idempotent mutation, and the description supplies the missing 'what happens': it appends a noise record to the ledger, which explains the write and reinforces the 'once per base' non-idempotency. It does not cover failure modes (e.g., what happens if the arm has fewer than two repeats or noise was already recorded), so it falls short of fully disclosing behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four front-loaded sentences: purpose, usage constraints plus alternative, side effect, then return values and handoff. No sentence is redundant with the structured fields and none is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by enumerating the return payload (noise_id, per-task flip rates, net spread, threshold, method) and explaining the downstream handoff to gate_check. An agent has everything needed to call it and consume the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real constraints beyond the schema: the run_id must reference a 'non-smoke' arm with 'at least two repeats', which the schema text does not say. That materially narrows what values are valid.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Measure the current base's run-to-run noise from one repeated arm') and pairs it with a second action ('record the estimate that gate_check requires'). It explicitly distinguishes itself from compare_paired's role, so an agent can tell the two apart without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states when to use it ('once per base, on a complete, non-smoke selection arm of the current version with at least two repeats') and names the alternative for a different need ('use compare_paired for candidate effects'). Both the trigger conditions and the routing exclusion are explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.