Skip to main content
Glama

get_agent_reliability

Read-only

Read verified pass/fail reliability for agents across all projects by role and task type; low_sample flags show when history is insufficient to trust scores.

Instructions

Read Beta-Bernoulli reliability across every project on this home, grouped by agent/model/role/task_type — only ever from verified PASSED/FAILED evaluations. low_sample is true when there isn't yet enough history to trust the number. Read-only and not tied to one project; all three filters are optional and combine. For one task's own scores use get_task_evaluations.

Input Schema

NameRequiredDescriptionDefault
roleNoOnly this role, for example REVIEW.
agentNoOnly this agent/provider, for example codex or claude.
task_typeNoOnly this task_type label, as given to evaluate_task (default 'general').

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changedv0.1.3
    • addedInput schema / properties / agent / description
      Added value: +"Only this agent/provider, for example codex or claude."
    • addedInput schema / properties / role / description
      Added value: +"Only this role, for example REVIEW."
    • addedInput schema / properties / task_type / description
      Added value: +"Only this task_type label, as given to evaluate_task (default 'general')."
  2. First observedv0.1.0

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and closed-world scope, so the safety profile is covered. The description adds real context beyond that: data comes only from verified PASSED/FAILED evaluations, and the low_sample flag signals when history is insufficient to trust the number. It does not describe grouping/pagination mechanics, but an output schema exists to carry return structure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with what is read and at what scope, then caveats, then the alternative tool. Every sentence carries information; nothing restates the tool name or schema redundantly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a read-only, zero-required-parameter query tool with full schema coverage, an output schema, and complete annotations, the description covers scope, data provenance, the low_sample caveat, and sibling routing. Nothing needed to select or invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so each filter is already documented with examples in the schema. The description adds combining semantics ('all three filters are optional and combine') that the schema does not express, which is genuine value over the structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and metric ('Read Beta-Bernoulli reliability') over an explicit scope ('across every project on this home') with named grouping dimensions (agent/model/role/task_type). It also names the sibling get_task_evaluations and the boundary between them, so an agent can distinguish the two without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing: use this for home-wide reliability, use get_task_evaluations for one task's own scores. It also clarifies that the tool is not tied to one project and that all three filters are optional and combinable, which covers the main usage decision points.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.