Skip to main content
Glama
agent-eval-policy

agent-eval-policy

by romanfjose

route_agent_seat

Routes a worker role to a cost tier and validates planned spend against a budget snapshot, failing closed for unknown roles, modes, or malformed accounting values.

Instructions

Route a worker role to a cost tier and admit its planned spend against the budget snapshot. Fails closed on unknown roles, modes, or malformed accounting values.

Input Schema

NameRequiredDescriptionDefault
modeNoread-only (default) or workspace-write.
roleYesWorker role. auditor is always read-only.
budgetYesAccounting snapshot for the admission decision. All values are supplied by the caller and must be finite.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the full burden, and it does disclose one genuinely useful trait: it fails closed on unknown roles, modes, or malformed accounting values. However, it never says whether the call is purely a check or whether it persists/consumes budget, nor whether it needs any authorization, nor how the circuit-breaker and judgment-allowance inputs affect the decision. For an unannotated tool with a nested accounting object, that leaves meaningful behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, the primary action front-loaded and the error contract second. There is no filler, no restatement of the tool name, and nothing that could be cut without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool takes a nested seven-field accounting object plus two enums and has no output schema, so the description should clarify what an admission decision yields (tier granted, allowance, or refusal) and how the breaker fields interact. It covers the failure contract but not the success contract, leaving the agent unable to predict the response shape. Adequate but with a clear gap given the complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including enum explanations for role and mode and per-field constraints on the budget object, so the schema already does the heavy lifting and baseline 3 applies. The description adds only loose framing ('budget snapshot', 'unknown roles, modes, malformed accounting values') and no interpretation of the judgment/circuit-breaker fields beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action on specific resources: routing a worker role to a cost tier and admitting planned spend against a budget snapshot. The domain is abstract (seats, tiers, CU), but a reader can tell this is an admissibility/pricing decision rather than a lint or evaluation call, which distinguishes it from the two siblings. It stops short of five because 'route' and 'admit' are internal jargon that a fresh agent may not map to a concrete outcome.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no statement of prerequisites (for example that a budget snapshot must be assembled by the caller), and no mention of when not to call it. The only usage-adjacent signal is the failure behavior, which tells the agent what happens on bad input, not when this tool is the right choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.