Skip to main content
Glama

Audit Feature Manifest

warden.audit_features
Read-onlyIdempotent

Audit feature manifests for lookahead leakage and provenance violations, flagging banned sources, pre-tape features, and unknown entries; optionally checks coverage vs snapshot universe.

Instructions

Audit a feature manifest for lookahead leakage and provenance violations.

manifest: [{name, source, value_ts, as_of}]. Flags banned actuals sources (open-meteo.archive etc.), features predating tape start, and unknown features fail closed to tier C. Optional coverage check vs MIN_COVERAGE=0.90 for challenger-selected symbols against the archived snapshot universe.

Input Schema

NameRequiredDescriptionDefault
manifestYesFeature manifest: [{name, source, value_ts, as_of}] to audit.
tape_startedNoOptional tape start date (YYYY-MM-DD); features predating it are flagged.
selected_symbolsNoOptional challenger-selected symbols for the coverage check.
snapshot_universeNoOptional archived snapshot universe to measure coverage against.

Output Schema

NameRequiredDescriptionDefault
howNo
toolNo
errorNo
accessNo
verdictNo
coverageNo
featuresNo
provenanceNo
violationsNo
request_access_atNo
banned_actuals_sourcesNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.4.1

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, so the safety profile is covered. The description adds real behavioral substance beyond that: banned actuals sources (open-meteo.archive), fail-closed treatment of unknown features to tier C, and the MIN_COVERAGE=0.90 threshold used for the optional coverage check.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then the manifest shape, then the checks, then the optional mode. Dense and largely waste-free, though the second paragraph is a run-on sentence that could be split for faster scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and annotations cover safety. The description covers the checks, fail-closed behavior, and the coverage threshold. It could be more explicit about severity/format of the flags, but nothing an agent must know to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, giving a baseline of 3, but the description goes further by naming the manifest item shape {name, source, value_ts, as_of}, which the schema does not expose (items are additionalProperties:true with no declared properties). It also explains the semantics of the coverage parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (audit) plus resource (feature manifest) and the two things it checks: lookahead leakage and provenance violations. It is clearly distinct from the sibling family (validate_order, plan_rebalance, promote/validate_genome, etc.), so an agent can select it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The domain implies when to use it (pre-flight review of a feature manifest), and it hints at a conditional mode ('optional coverage check for challenger-selected symbols'), but it never says when-not or names an alternative tool. Usage context is inferable rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.