Skip to main content
Glama

Quant Swarm Audit — Preflight & Scope Matrix

audit.preflight
Read-onlyIdempotent

Validates strategy returns, trades, benchmark, cost assumptions, and feature manifests locally before promotion, returning a coverage and scope matrix that separates tested, untested, and outbound data previews.

Instructions

Validates strategy returns, trades, benchmark, cost assumptions, and feature manifests locally at 0 credits. Returns a Coverage & Scope Matrix separating available checks, NOT_TESTED checks, and an outbound data preview.

Input Schema

NameRequiredDescriptionDefault
trades_pathNoOptional path to CSV/Parquet/JSON trade records (ts, symbol, pnl_bps, position_pct).
trades_rowsNoOptional inline trade records [{ts, symbol, pnl_bps, position_pct}].
returns_pathNoPath to CSV/Parquet/JSON strategy returns (columns: ts/date, strategy_return/return).
returns_rowsNoInline list of strategy return rows [{ts, strategy_return}].
strategy_nameNoHuman-readable strategy or experiment identifier.candidate_strategy
benchmark_pathNoOptional path to CSV/Parquet/JSON benchmark return series.
benchmark_rowsNoOptional inline benchmark return rows [{ts, return}].
cost_assumptionsNoOptional inline execution cost assumptions dict.
features_manifestNoOptional inline feature provenance manifest dict.
cost_assumptions_pathNoOptional path to JSON execution cost assumptions {spread_bps, slippage_bps, adverse_selection_bps}.
features_manifest_pathNoOptional path to JSON feature provenance manifest {as_of, features: [...]}.

Output Schema

NameRequiredDescriptionDefault
howNo
modeNo
toolNo
errorNo
accessNo
job_idNo
reasonNo
statusNo
metricsNo
eligibleNo
findingsNo
gbp_costNo
job_id_aNo
job_id_bNo
next_stepNo
strategy_aNo
strategy_bNo
html_reportNo
max_creditsNo
credits_costNo
generated_atNo
input_sha256No
metrics_diffNo
new_warningsNo
preflight_idNo
random_seedsNo
has_benchmarkNo
strategy_nameNo
written_filesNo
approval_tokenNo
execution_planNo
input_sha256_aNo
input_sha256_bNo
trades_summaryNo
artifact_hashesNo
coverage_matrixNo
credits_chargedNo
idempotency_keyNo
markdown_reportNo
package_versionNo
returns_summaryNo
assessment_afterNo
cost_assumptionsNo
headline_metricsNo
not_tested_countNo
required_creditsNo
assessment_beforeNo
idempotent_replayNo
request_access_atNo
resolved_findingsNo
validation_errorsNo
counts_by_categoryNo
overall_assessmentNo
finding_transitionsNo
remediation_verdictNo
has_features_manifestNo
not_investment_adviceNo
outbound_data_previewNo
reproducibility_bundleNo
estimated_runtime_secondsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.0.0

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/non-openWorld, so the description carries a lighter load — and it adds genuinely new behavior: execution is local, costs zero credits, and it emits an 'outbound data preview' which is important disclosure for a tool that inspects data files. It stops short of saying which checks constitute the NOT_TESTED set or what conditions produce them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and the cost differentiator, then the return shape. Nothing is redundant, though the second sentence is somewhat compressed and could more clearly flag the NOT_TESTED concept.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter, zero-required tool with full schema coverage, an output schema, and strong annotations, this is close to sufficient: purpose, cost model, and output nature are all stated. The remaining gap is precedence/validation behavior when no inputs are supplied, since every parameter defaults to null.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% across all 11 parameters, including accepted file formats, row shapes, and expected columns, so the schema already does the semantic work. The description only echoes the parameter categories (returns, trades, benchmark, cost assumptions, feature manifests) without adding format or precedence guidance, such as what happens if both a path and inline rows are supplied.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb (validates) and enumerates the exact artifact classes it consumes: returns, trades, benchmark, cost assumptions, feature manifests. 'Locally at 0 credits' implicitly separates it from the paid audit.run / audit.compare siblings, though no sibling is named outright.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied via the 'preflight' naming and the 0-credit local execution note — an agent can infer this is a cheap sanity pass before committing to audit.run. There is no explicit when-to-use, no exclusions, and no statement about what to do after a preflight (e.g., proceed to audit.run).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.