Skip to main content
Glama

evaluate

Measure one factor against forward returns and record it as a trial, returning RankIC, Newey-West t, p-value, trial index, and family-adjusted verdict.

Instructions

Measure one factor and record it as a trial. Returns cross-sectional RankIC, Newey-West t, single-test p, the trial's index within its family, and the family-adjusted verdict. This is the only supported way to produce evidence.

Input Schema

NameRequiredDescriptionDefault
exprNo
alphaNo
labelNo
factorYesA (dates x names) panel. Prefer {'path': 'f.npz', 'key': 'factor'}; inline {'values': [[...]]} is allowed only up to 200x200.
forwardYesForward returns, already shifted: forward[t] follows factor[t].
horizonNo
with_gateNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so the description carries full behavioral burden. It usefully discloses that a trial is recorded (a state-mutating side effect) and enumerates the statistical outputs, but says nothing about permissions, idempotency, or what 'family-adjusted verdict' means operationally.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with purpose front-loaded, then return values, then positioning. No filler, though the return-value sentence is dense and could be enumerated more legibly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates the returned statistics, which is its main strength. But for a 7-parameter, nested-object tool it leaves the trial/family concept and five parameters undefined, so an agent cannot fully reason about the call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 29%, and the description adds no parameter-level meaning at all. Five parameters (expr, alpha, label, horizon, with_gate) are undocumented in both schema and description, including the horizon and the gate behavior that materially change the result.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Measure one factor and record it as a trial') and names what it produces. It gestures at differentiation via 'the only supported way to produce evidence,' implying fingerprint/gate are not evidence producers, but never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'This is the only supported way to produce evidence' gives a positive usage cue, and 'record it as a trial' implies the ledger/family context. However there is no when-not guidance, no named alternative, and no prerequisites (e.g., what must exist before evaluating).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools