Skip to main content
Glama

policy_evaluate

Read-onlyIdempotent

Evaluate team thresholds, explicit coverage requirements and expiring waivers without changing reports.

Instructions

Evaluate team thresholds, explicit coverage requirements and expiring waivers without changing reports.

Input Schema

NameRequiredDescriptionDefault
report_idNolatest
baseline_idNo
policy_pathYes
baseline_pathNo
baseline_sha256No

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv1.5.1
    • addedInput schema / properties / baseline_path
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Baseline Path"
      +}
    • addedInput schema / properties / baseline_sha256
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Baseline Sha256"
      +}
  2. First observedv1.0.3

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the 'without changing reports' clause largely restates structured data. What the description does add is the scope of evaluation (thresholds, coverage requirements, waivers), which is genuine behavioral context the annotations do not carry. It omits what happens on an invalid policy file or what a failed evaluation looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One compact sentence with the subject matter front-loaded and no filler. Minor deduction because the trailing 'without changing reports' clause repeats what the annotations already assert, spending words on redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be documented, but for a five-parameter tool with 0% schema coverage and no routing guidance the description is materially incomplete. An agent cannot determine how baseline selection works or when this tool should be chosen over its many validation-adjacent siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across five parameters, and the description addresses none of them. The baseline_id / baseline_path / baseline_sha256 trio is genuinely ambiguous (identifier vs. file vs. integrity hash) and the required policy_path is never explained, so the description fails to compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (evaluate) plus the concrete subject matter being evaluated: team thresholds, explicit coverage requirements, and expiring waivers. That is far more informative than a name restatement. It does not, however, distinguish this from plausible siblings such as specs_validate, verify_finding, or audit_scan, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to reach for policy_evaluate versus specs_validate, verify_finding, or audit_scan, and no preconditions or exclusions are given. The only usage-flavored clause ('without changing reports') is a safety property, not routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.