AlphaAssay is a signal validation and overfitting forensics platform that statistically audits trading signals, backtests, and strategies — returning deterministic pass/fail verdicts with signed, replayable audit records.
Free tools
assay_demo — Run a free demo to see the full output schema and known-verdict specimens.
assay_graveyard — Look up anonymised falsification statistics for a signal family before investing time in an idea.
assay_calibration — View AlphaAssay's own signed public calibration and verdict hit-rate.
assay_certificate_verify — Offline Ed25519 signature verification of a signed verdict certificate.
assay_provider_protocol — Get a machine-readable 7-test falsification checklist to audit any signal provider.
assay_preflight — Lint a payload (DSL schema, OHLCV data, trade rows) before spending a paid check.
Signal & backtest validation
assay_signal — Validate a trading strategy export with deflated Sharpe, multiple-testing correction, overfitting detection, look-ahead checks, and cost stress.
assay_backtest — Run a deterministic DSL backtest with honest family-level trial accounting.
assay_gauntlet — Full reality-check battery: validator, family deflation, placebo, capacity ceiling, and graveyard prior in one call.
assay_falsify — Actively stress-test a signal with adversarial attacks (execution lag, cost stress, time jackknife, regime split, parameter perturbation).
Overfitting & forensics
assay_forensics — Diagnose look-ahead leakage, front-loading, random timing comparison, and event independence.
assay_pbo — Compute Probability of Backtest Overfitting (CSCV) over a full parameter payoff matrix.
assay_survivors — Apply Romano-Wolf stepwise FWER control to disclose which variants survive family-wise error budgeting.
assay_batch — Submit up to 25 parameter sweep variants in one call with cumulative deflation applied.
Reproducibility & pre-registration
assay_reproduce — Independently recompute a claimed track record from trades + candles, grading metrics against disclosed tolerances.
assay_register — Pre-register a hypothesis with a tamper-proof Merkle-anchored timestamp, making the later verdict immune to overfitting.
assay_verdict — Fetch a post-cutoff verdict for a pre-registered signal, enforcing sealed hypothesis terms.
Open-source client tooling for the AlphaAssay validation service: an offline certificate verifier (Python) and a stdio-to-remote MCP proxy (Node), plus runnable examples and the registry manifest for the hosted server.
Component | Language | Path | What it does |
| Python ≥3.10 | Verifies certificates offline: Ed25519 signatures, canonical serialisation, signed key/revocation history. No network access. | |
MCP proxy | Node ≥18 | Bridges a local stdio MCP client to the remote streamable-HTTP endpoint. | |
Examples | — | Known-answer request/response pairs you can run against the free demo endpoint. | |
| — | Secondary artifact: the manifest the MCP registry reads to describe the hosted server. Not needed to use the code above. |
The validation engine itself is a hosted service and is not part of this repository.
Python verifier, from source:
pip install .
# or, for development
pip install -e ".[dev]"Node proxy:
npm installRelated MCP server: Decker
Full platform trust needs three local inputs: the certificate with its signature, a signed version-2 revocation/key-history bundle, and at least one externally obtained SHA-256 root fingerprint. The verifier performs no network or database access.
python -m alphaassay_verify verify \
--certificate cert.json \
--trust-bundle trust-bundle.json \
--root-fingerprint <64-hex-digest>Signature-only evidence, when you deliberately do not want to establish full platform trust:
python -m alphaassay_verify signature-only \
--certificate cert.json \
--public-key key.pemExit code 0 means verified; a non-zero exit code and a machine-readable
reason mean it is not. See SPEC.md for the canonical byte format
and the trust rules the implementation follows.
node server/index.mjsThe proxy speaks stdio to your MCP client and forwards to the remote endpoint.
The tool list it exposes is in server/tools.json.
Each example is a request payload, the expected response, and a shell script that posts it to the free demo endpoint:
cd examples && ./golden_lookahead.shThe expected responses are checked in, so they double as fixtures for integration tests in your own code.
python -m pytest tests/ # verifier: unit tests and canonical vectors
npm test # proxy: contract and behaviour testsTest vectors live in tests/canonical_vectors.json
and tests/fixtures/; they are the same vectors the
implementation is developed against.
Issues and pull requests are welcome. Please:
keep changes focused and include tests for behaviour changes;
run python -m pytest tests/ and npm test before opening a PR;
for the verifier, do not add network or database access — offline operation is a design constraint, not an implementation detail;
report security issues privately as described in SECURITY.md
rather than in a public issue.
Apache-2.0. See LICENSE.
Deterministic DSL backtest with honest family-level trial accounting.
Define the strategy as an executable JSON DSL (indicators: sma, ema, rsi,
atr, roc, zscore, price; ops: cross_above, cross_below, gt, lt, and, or,
not), supply your own candles, and get net-of-cost per-bar returns plus a
deflated-Sharpe family verdict. Every call is recorded in your family's
trial ledger, so repeated searching deflates future verdicts -- that is
the feature, not a bug: the verdict stays meaningful. The ledger is
keyed to your authenticated account, never to a caller-supplied name.
Price: $0.05 per check (api_key required -- free account at
https://api.alphaassay.com/account includes 3 free checks).| Name | Required | Description | Default |
|---|---|---|---|
| spec | Yes | ||
| api_key | No | ||
| ohlcv_by_symbol | Yes | ||
| fee_bps_per_side | No | ||
| variants_in_call | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the transparency burden. It clearly discloses that every call is recorded in a family trial ledger, repeated calls deflate future verdicts, the ledger keys to account (not caller-supplied name), cost per check ($0.05), and api_key requirement. This is highly transparent about side effects and pricing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single focused paragraph that front-loads the core purpose and then adds details. All sentences are informative (DSL, inputs, outputs, ledger, cost, api_key). Slightly verbose with the repeated emphasis on the ledger being intentional, but otherwise efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters including nested objects, no output schema, and a large sibling family, the description covers the tool's core mechanics, inputs, outputs (returns and verdict), behavioral constraints, and pricing. It lacks error/validation info but is otherwise complete for typical usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning by listing DSL indicators/ops for `spec` and implying `ohlcv_by_symbol` is candles, `fee_bps_per_side` (default 10), `variants_in_call` (default 1). However, it does not explain the structure of `spec` or `ohlcv_by_symbol` beyond examples, and contradicts the schema by claiming `api_key` is required while schema lists it as optional with a default. This reduces clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool executes a deterministic DSL backtest with family-level trial accounting, providing net-of-cost returns and a deflated-Sharpe verdict. However, it does not explicitly differentiate from sibling tools like assay_demo or assay_falsify, missing the chance to clarify when to use this specific tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for backtesting strategies via DSL and candles, but provides no guidance on when to avoid this tool or suggest alternatives among the many assay_ siblings. Usage context is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
AlphaAssay's own public track record -- verify us, not our marketing.
Self-calibration of the validator: how many pre-registered signals were
evaluated, how many survived vs. were deflated out, and (as forward
windows mature) the hit-rate of our own verdicts. Signed with the
platform key when configured, honest about insufficient history. This is
the document that makes our 'no' worth something. Price: free.| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the document is signed with the platform key when configured and is honest about insufficient history. It does not mention read-only nature or other behavioral traits, but no annotations are provided to contradict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat verbose and metaphorical (e.g., 'makes our 'no' worth something'). While informative, it could be more direct and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains what the tool returns (statistics on signals, hit-rates, signed). It covers the key aspects but could be more specific about format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so the description cannot add parameter information. According to guidelines, baseline is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool provides the platform's own public track record, including statistics on pre-registered signals and hit-rates. It distinguishes from sibling tools by focusing on self-calibration rather than other assays.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for verifying the platform claims ('verify us, not our marketing') but does not explicitly state when to use or when not to use. No alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify a signed AlphaAssay verdict certificate offline -- without us.
Third-party verification of an Ed25519-signed verdict certificate over
canonical JSON, including hash-chained revocation status. Needs only the
certificate, the signature and the public key -- no platform access, no
network. Use it to check a certificate someone attached to a signal they
are selling. Price: free.| Name | Required | Description | Default |
|---|---|---|---|
| certificate | Yes | ||
| signature_b64 | Yes | ||
| public_key_pem | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool works offline, requires only the three inputs, and uses Ed25519 and canonical JSON. It does not describe the return value (e.g., boolean or details) or behavior on failure. Since no annotations are provided, the description carries the burden but leaves some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded: the first sentence states the primary purpose, followed by technical details and a usage example. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a verification tool with three parameters and no output schema, the description covers the purpose, inputs, and usage context well. It lacks specification of the return format or error handling, but the core functionality is clear given the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining the three required inputs: certificate, signature (base64 encoded), public key (PEM format). It adds context about the certificate being a signed AlphaAssay verdict certificate over canonical JSON.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it verifies a signed AlphaAssay verdict certificate offline. It specifies the cryptographic algorithm (Ed25519), format (canonical JSON), and revocation checking, distinguishing it from sibling tools like assay_verdict which likely creates certificates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a concrete use case: checking a certificate attached to a signal being sold. It emphasizes offline, third-party verification, implying when to use. However, it does not explicitly exclude other uses or mention alternatives, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Free demo verdict -- see AlphaAssay's full output shape in one call.
Runs the real fail-closed validator over a built-in 40-trade example and
returns the complete verdict envelope (verdict, qualitative findings,
leakage taxonomy, provenance hashes) plus the machine-readable
remediation explanation -- and a free synthetic-null preview: three
seeded no-edge worlds (Heston stochastic volatility, Merton jump
diffusion, symmetric drift bursts) showing the process-placebo
machinery the paid falsify battery attacks your strategy with. No
auth, no payment, no input needed. Use this first to understand the
schema before submitting your own strategy with assay_signal.
Price: free.| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: it's a demo, runs a real fail-closed validator, uses a built-in example, returns specific output components, and requires no input/auth/payment.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense and front-loaded with purpose, but could be slightly shorter. Each sentence adds value, and the structure is logical.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no annotations, and no output schema, the description compensates fully by detailing the output envelope and synthetic-null preview. It is complete for a demo tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema coverage is 100%. The description does not need to explain parameters; it appropriately focuses on behavior. Baseline 4 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a demo that runs a validator over a built-in example and returns a verdict envelope plus synthetic-null preview. It distinguishes itself from sibling tool 'assay_signal' by advising to use this first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'No auth, no payment, no input needed' and instructs to 'Use this first to understand the schema before submitting your own strategy with assay_signal', providing clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The full reality-check battery in ONE call -- validator, family deflation, matched-random placebo, capacity ceiling and graveyard prior chained into a single consolidated dossier. Instead of pass/fail you get WHICH gate killed the signal first (machine-readable failure_codes), the placebo percentile, the tradable capacity ceiling, how often the family was already buried, and the family's remaining search budget. Built for iterating agents: it is training feedback with budget economics, not a bare verdict. Supply the strategy as an executable DSL spec plus your own candles; optional per-symbol ADV in USD unlocks the capacity gate, and optional per-symbol daily volatility (sigma_by_symbol_pct, percent) sharpens its impact model for volatile assets. Demote-only, deterministic. Price: $0.05 per check (api_key required -- free account at https://api.alphaassay.com/account includes 3 free checks).
| Name | Required | Description | Default |
|---|---|---|---|
| spec | Yes | ||
| api_key | No | ||
| ohlcv_by_symbol | Yes | ||
| fee_bps_per_side | No | ||
| variants_in_call | No | ||
| adv_by_symbol_usd | No | ||
| sigma_by_symbol_pct | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the tool is 'Demote-only, deterministic', explains the process (chaining gates, machine-readable failure codes), and notes budget economics. It does not detail side effects but provides sufficient behavioral context for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed but not verbose, front-loading the purpose and key outcomes. It uses technical jargon efficiently. Minor redundancy (e.g., 'training feedback with budget economics') but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, nested objects, no output schema), the description covers inputs, behavior, pricing, and output nature (failure codes, percentiles). Lacks explicit output schema details but provides enough for an agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description explains key parameters: spec (DSL), ohlcv_by_symbol (candles), optional adv_by_symbol_usd and sigma_by_symbol_pct. It misses fee_bps_per_side and variants_in_call, but the primary parameters are well-described, adding significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a 'reality-check battery' combining multiple gates, yielding specific outputs like failure codes and placebo percentile. It distinguishes from sibling tools (e.g., assay_verdict, assay_falsify) by being a consolidated assessment rather than a single test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use: supply DSL spec and candles, with optional ADV and volatility for enhanced results. It also mentions pricing and required API key. However, it does not explicitly contrast with alternative sibling tools or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Free lookup: how often has this signal family already died?
Anonymised falsification statistics per structural signal family --
tested / killed / survived counts, top kill reasons, and the crowd prior
your submission would be deflated by. Check BEFORE you spend weeks on an
idea whether the crowd already buried it. k-anonymous: families with
fewer than 5 distinct submitters fall back to coarser taxonomy stats.
Price: free.| Name | Required | Description | Default |
|---|---|---|---|
| variant | No | ||
| strategy | No | ||
| timeframe | No | ||
| fingerprint | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses key behaviors: it returns anonymised counts, top kill reasons, and a crowd prior. It also explains k-anonymity fallback and states it's free. Missing explicit mention of side effects or read-only nature, but sufficient for most contexts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: front-loaded with the core purpose, followed by details and usage directive. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks details on return format and parameter semantics, making it incomplete for correct invocation despite good purpose and usage guidance. Output schema is absent, and parameters are undocumented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description provides no explanation of the four parameters (variant, strategy, timeframe, fingerprint). An agent has no guidance on what these inputs mean or how to use them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines the tool as a free lookup for historical signal family falsification statistics, using verbs like 'lookup' and specifying the resource 'signal family'. It distinguishes itself from sibling assay tools by being a pre-check tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use it before investing time in an idea, providing a clear when-to-use scenario. Mentions fallback behavior for small families, aiding in tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Free payload lint -- fix your submission BEFORE spending a check.
Validates the SHAPE of what you are about to submit, with the same
machine-readable failure vocabulary the paid tools use: DSL schema
validity, OHLCV sanity (finite positive prices, aligned series,
strictly increasing timestamps), per-symbol data presence, trade-row
types, and an honest size warning when the sample is below the paid
gates' evidential floor. Send the same spec/ohlcv_by_symbol/trades you
would send to assay_gauntlet or assay_signal; get back ok plus named
findings (dsl_invalid, timestamps_not_monotonic,
ohlcv_non_finite_or_non_positive, ohlcv_series_misaligned,
symbol_data_missing, trades_rows_invalid, sample_below_engine_floor)
with plain-language details. Shape lint only -- a clean preflight is
NOT evidence of an edge and never blesses a signal; it just means your
paid check will not bounce on format. Deterministic, no ledger write,
no account needed. NOT financial advice. Price: free.| Name | Required | Description | Default |
|---|---|---|---|
| spec | No | ||
| trades | No | ||
| ohlcv_by_symbol | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavior: deterministic, no ledger write, no account needed, price free. It lists the exact validation checks and the failure vocabulary returned. Since no annotations are provided, the description carries the entire burden and handles it thoroughly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet comprehensive, with a clear opening summary followed by details in a logical order. Every sentence adds unique value, and there is no repetition or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 optional parameters, no output schema, and no annotations, the description covers purpose, usage, behavior, parameter semantics, return format, and limitations. It is complete enough for an agent to correctly select and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must explain parameters. It does so by referencing the paid tools ('Send the same spec/ohlcv_by_symbol/trades you would send to assay_gauntlet or assay_signal') and describing the validation categories (DSL schema, OHLCV sanity, etc.), which map to the three parameters. While not exhaustive, this provides sufficient context for an agent familiar with the domain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Free payload lint -- fix your submission BEFORE spending a check.' It uses specific verbs like 'lint', 'validate', and 'check', and distinguishes itself from sibling tools like assay_gauntlet and assay_signal, which are paid alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: use this tool before submitting to paid tools to avoid format bounces. It warns that a clean preflight does not guarantee a signal, and mentions the paid alternatives (assay_gauntlet, assay_signal) as the appropriate next steps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Test ANY signal provider in an afternoon -- the falsification protocol as machine-readable rules.
Seven falsifiable tests that separate edge from selection, runnable
against any signal seller (paid channel, platform, bot) without their
cooperation: provenance (tamper-proof timestamps or it did not happen),
survivorship (the full issue history incl. losers), pre-registration
(ten sealed forward calls), placebo (does random timing do as well?),
costs (profitable AFTER fees/spread/slippage?), trial accounting (one
winner out of how many quiet attempts?), and the examiner test (apply
it all to whoever grades the signals -- including AlphaAssay: golden
specimens, offline signature check, public daily calibration). Each
test carries a machine-checkable failure_condition, and tests 3-6 name
the AlphaAssay endpoint that automates them. Static and deterministic:
this tool hands you the checklist, it does not rate, score or rank any
provider and it never fetches external data. NOT financial advice.
Price: free.| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral transparency. It states the tool is 'Static and deterministic,' does not fetch external data, and only returns a checklist. This fully discloses its read-only, non-destructive behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat lengthy but front-loads the purpose and uses structured formatting for the seven tests. Every sentence adds value, though a slightly more concise version could be possible without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, no output schema, and no annotations, the description fully covers what the tool does, what it returns, and how it differs from siblings. It provides all necessary context for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% coverage. The description adds extensive context about what the tool outputs (the machine-readable checklist of seven tests), exceeding the baseline expectation for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Test ANY signal provider in an afternoon -- the falsification protocol as machine-readable rules.' It lists the seven falsifiable tests and explicitly distinguishes from siblings by stating it does not rate, score, or rank providers and never fetches external data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'this tool hands you the checklist, it does not rate, score or rank any provider and it never fetches external data.' This tells users when to use it (to get the protocol) and when not to (for actual scoring or external data). It also includes a disclaimer 'NOT financial advice.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Pre-register a signal NOW; earn a verdict that cannot be overfit.
Signal anchoring: your strategy spec (executable JSON DSL) is
canonically hashed today and anchored into a public Merkle tree; the
later verdict (assay_verdict) uses exclusively data from AFTER
registration -- a true post-cutoff out-of-sample that is mathematically
immune to overfitting, because the data did not exist when you
committed. v2: optionally seal a machine-writable HYPOTHESIS tuple
alongside the spec -- declared_pass_sharpe_annualized (your
pre-committed success bar), max_evaluation_bars (ex-ante window cap:
waiting longer than promised cannot improve the verdict),
fee_bps_per_side and trading_calendar (the sealed evaluation terms),
plus an optional rationale. The tuple is canonically hashed and bound
into the same Merkle anchor as the spec, and assay_verdict ENFORCES
it: deviating terms are recomputed under the seal and flagged
goalpost_moved -- success criteria chosen after seeing the data are
not criteria. Registrations are idempotent, withdrawals still count
toward your family's trial budget (anti-gaming). Registrations are
keyed to your authenticated account. Price: $0.05 per check (api_key
with the register scope required -- free account at
https://api.alphaassay.com/account includes 3 free checks).| Name | Required | Description | Default |
|---|---|---|---|
| spec | Yes | ||
| api_key | No | ||
| hypothesis | No | ||
| embargo_bars | No | ||
| maturity_bars | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully covers behavioral traits: idempotency, cost ($0.05 per check, free account with 3 checks), account keying, withdrawal counting toward trial budget, and the enforcement of hypothesis terms with goalpost_moved flag. This is comprehensive for a registration tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed and well-structured with clear sections, but it is somewhat verbose. Every sentence adds value, but it could be trimmed slightly without losing meaning. Front-loading the core purpose helps an agent quickly grasp the tool's function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, no annotations), the description adequately covers purpose, behavior, and constraints. However, it lacks return-value details and error handling, and some parameters are under-documented. Overall, it is sufficient for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description only elaborates on 'spec' and 'hypothesis' parameters, neglecting 'api_key', 'embargo_bars', and 'maturity_bars'. These parameters remain unclear, failing to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines the tool's purpose: pre-register a signal to obtain a verdict that is immune to overfitting via Merkle anchoring. It specifies the action (register) and the resource (signal), and distinguishes from siblings like assay_verdict by focusing on the registration step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (before evaluating a signal to prevent overfitting) and the benefit (mathematical immunity). It does not explicitly state when not to use or compare to alternatives, but the context is clear enough for an agent to infer appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
WHICH variants of your sweep survive family-wise error control -- an error-budget disclosure, never a ranking.
Send the same T x N trial matrix assay_pbo grades (rows =
time-ordered periods, columns = every configuration you tried) and
get Romano-Wolf stepwise multiple testing over it: studentized
per-config statistics, circular block bootstrap over the time rows
(serial dependence respected), stepdown max-statistic critical values
(Romano/Wolf 2005, Econometrica; Hansen SPA 2005). Answer, per
variant IN INPUT ORDER: could this family's evidence kill it at
family-wise error rate alpha, and in which stepdown round? Survival-
map framing by construction: the verdict vocabulary is fail (nothing
survives: NO_SURVIVORS_AT_FWER), conditional (survivors disclosed --
explicitly NOT a pass) or insufficient_evidence (blocked with a named
reason); this tool structurally cannot bless, rank or recommend.
Natural partner of assay_pbo -- one matrix, two questions: is the
SELECTION overfit (pbo), and which variants does the error budget
leave standing (survivors). Deterministic (bootstrap seeded from the
input digest); fail-closed on size, budget and numerics; no ledger
write. NOT financial advice. Price: $0.05 per check (api_key required
-- free account at https://api.alphaassay.com/account, 3 free checks
included).| Name | Required | Description | Default |
|---|---|---|---|
| alpha | No | ||
| api_key | No | ||
| n_bootstrap | No | ||
| block_length | No | ||
| returns_matrix | No | ||
| periods_per_year | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the transparency burden. It discloses deterministic behavior, fail-closed on size/budget/numerics, no ledger write, output vocabulary, and pricing. It also states it cannot bless or rank.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is verbose but well-structured, with clear sections and no redundancy. It could be more concise, but every sentence adds unique value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complex statistical method, 6 parameters, and no output schema, the description is remarkably complete. It explains the methodology, output types, constraints, and pricing, leaving little ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While the description explains alpha and the trial matrix (returns_matrix) and mentions bootstrap parameters implicitly, it does not describe api_key or periods_per_year. Schema coverage is 0%, so description compensates partially but not fully for all six parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool identifies which variants survive family-wise error control, distinguishing it from ranking tools. It explicitly contrasts with siblings like assay_pbo by framing it as a partner for a different question.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidelines: it is a partner of assay_pbo, never a ranking tool, and requires an api_key with a free account. It also mentions deterministic behavior and fail-closed conditions, guiding when to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Post-cutoff verdict for a pre-registered signal (the tamper-proof test).
Evaluates a registration strictly on bars AFTER the registration cutoff,
with maturity floor and fail-closed data-gap handling (use
trading_calendar='weekdays' for equity daily bars so weekends do not
count as gaps; omit fee/calendar to use the registration's sealed
terms). The verdict embeds the family-deflated Sharpe: even an
anchored signal is deflated by how much its family was searched. For
v2 registrations with a sealed hypothesis, the verdict additionally
reports oos_sharpe_annualized against the pre-committed bar
(threshold_met -- a descriptive fact, never an endorsement) and flags
goalpost_moved when the requested terms deviate from the sealed ones
(the seal always wins). Price: $0.05 per check (api_key required --
free account at https://api.alphaassay.com/account includes 3 free
checks).| Name | Required | Description | Default |
|---|---|---|---|
| api_key | No | ||
| prereg_id | Yes | ||
| ohlcv_by_symbol | Yes | ||
| fee_bps_per_side | No | ||
| trading_calendar | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses key behaviors: strict post-cutoff evaluation, maturity floor, fail-closed data-gap handling, family-deflated Sharpe, goalpost_moved flag, pricing, and API key requirement. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph, front-loaded with purpose, each sentence adds value. Could be more structured but overall efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, description explains return values (embedded Sharpe, oos_sharpe_annualized, goalpost_moved). Covers behavior, pricing, and parameter usage. Complete for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% but description adds meaning to parameters: explains trading_calendar and fee parameters, and that omitting them uses sealed terms. Does not explain other parameters but they are self-explanatory from context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it as 'Post-cutoff verdict for a pre-registered signal (the tamper-proof test)', specifying the verb 'verdict' and resource 'pre-registered signal'. It distinguishes from siblings by focusing on post-cutoff evaluation, contrasting with tools like assay_register or assay_backtest.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides specific guidance: use 'trading_calendar="weekdays"' for equity daily bars, omit fee/calendar to use sealed terms, and explains behavior for v2 registrations. Does not explicitly state when not to use, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.6.1assay_batchassay_falsifyassay_forensicsassay_pboassay_reproduceassay_signalv0.4.0assay_backtestassay_batchassay_calibrationassay_certificate_verifyassay_demoassay_falsifyassay_forensicsassay_gauntletassay_graveyardassay_pboassay_preflightassay_provider_protocolassay_registerassay_reproduceassay_signalassay_survivorsassay_verdictScored across 11 tools
Each tool serves a distinct purpose within the signal validation workflow, and descriptions are detailed. The slight overlap between assay_backtest (basic DSL backtest) and assay_gauntlet (comprehensive battery) is clarified by their descriptions, making them distinguishable.
All tools share the 'assay_' prefix and mostly use noun phrases (e.g., assay_graveyard, assay_gauntlet). A few (assay_register, assay_certificate_verify) use verbs, which is a minor inconsistency but still readable and predictable.
With 11 tools, the server covers the essential functions for signal validation—registration, preflight checks, backtesting, full validation, certificate verification, and more. The count feels well-scoped without being excessive or insufficient.
The tool set addresses the core lifecycle of signal validation: registration, pre-submission validation, multiple validation gates, and certificate verification. Minor gaps exist (e.g., no tool for listing past registrations or verdicts), but the core workflow is well-covered.
Honest A-F grades for trading strategies, backtested on real out-of-sample data. No hype.
Validated trading edges across futures, equities, crypto. Live signals, full audit trail.
Fail-closed backtest/signal validation: overfitting, leakage, deflated Sharpe. NOT financial advice.
Praxis: published trading strategies re-run and audited — verdicts, claimed vs measured stats.
