AlphaAssay is a signal validation and overfitting forensics platform that statistically audits trading signals, backtests, and strategies — returning deterministic pass/fail verdicts with signed, replayable audit records.
Free tools
assay_demo — Run a free demo to see the full output schema and known-verdict specimens.
assay_graveyard — Look up anonymised falsification statistics for a signal family before investing time in an idea.
assay_calibration — View AlphaAssay's own signed public calibration and verdict hit-rate.
assay_certificate_verify — Offline Ed25519 signature verification of a signed verdict certificate.
assay_provider_protocol — Get a machine-readable 7-test falsification checklist to audit any signal provider.
assay_preflight — Lint a payload (DSL schema, OHLCV data, trade rows) before spending a paid check.
Signal & backtest validation
assay_signal — Validate a trading strategy export with deflated Sharpe, multiple-testing correction, overfitting detection, look-ahead checks, and cost stress.
assay_backtest — Run a deterministic DSL backtest with honest family-level trial accounting.
assay_gauntlet — Full reality-check battery: validator, family deflation, placebo, capacity ceiling, and graveyard prior in one call.
assay_falsify — Actively stress-test a signal with adversarial attacks (execution lag, cost stress, time jackknife, regime split, parameter perturbation).
Overfitting & forensics
assay_forensics — Diagnose look-ahead leakage, front-loading, random timing comparison, and event independence.
assay_pbo — Compute Probability of Backtest Overfitting (CSCV) over a full parameter payoff matrix.
assay_survivors — Apply Romano-Wolf stepwise FWER control to disclose which variants survive family-wise error budgeting.
assay_batch — Submit up to 25 parameter sweep variants in one call with cumulative deflation applied.
Reproducibility & pre-registration
assay_reproduce — Independently recompute a claimed track record from trades + candles, grading metrics against disclosed tolerances.
assay_register — Pre-register a hypothesis with a tamper-proof Merkle-anchored timestamp, making the later verdict immune to overfitting.
assay_verdict — Fetch a post-cutoff verdict for a pre-registered signal, enforcing sealed hypothesis terms.
Open-source client tooling for the AlphaAssay validation service: an offline certificate verifier (Python) and a stdio-to-remote MCP proxy (Node), plus runnable examples and the registry manifest for the hosted server.
Component | Language | Path | What it does |
| Python ≥3.10 | Verifies certificates offline: Ed25519 signatures, canonical serialisation, signed key/revocation history. No network access. | |
MCP proxy | Node ≥18 | Bridges a local stdio MCP client to the remote streamable-HTTP endpoint. | |
Examples | — | Known-answer request/response pairs you can run against the free demo endpoint. | |
| — | Secondary artifact: the manifest the MCP registry reads to describe the hosted server. Not needed to use the code above. |
The validation engine itself is a hosted service and is not part of this repository.
Python verifier, from source:
pip install .
# or, for development
pip install -e ".[dev]"Node proxy:
npm installRelated MCP server: Decker
Full platform trust needs three local inputs: the certificate with its signature, a signed version-2 revocation/key-history bundle, and at least one externally obtained SHA-256 root fingerprint. The verifier performs no network or database access.
python -m alphaassay_verify verify \
--certificate cert.json \
--trust-bundle trust-bundle.json \
--root-fingerprint <64-hex-digest>Signature-only evidence, when you deliberately do not want to establish full platform trust:
python -m alphaassay_verify signature-only \
--certificate cert.json \
--public-key key.pemExit code 0 means verified; a non-zero exit code and a machine-readable
reason mean it is not. See SPEC.md for the canonical byte format
and the trust rules the implementation follows.
node server/index.mjsThe proxy speaks stdio to your MCP client and forwards to the remote endpoint.
The tool list it exposes is in server/tools.json.
Each example is a request payload, the expected response, and a shell script that posts it to the free demo endpoint:
cd examples && ./golden_lookahead.shThe expected responses are checked in, so they double as fixtures for integration tests in your own code.
python -m pytest tests/ # verifier: unit tests and canonical vectors
npm test # proxy: contract and behaviour testsTest vectors live in tests/canonical_vectors.json
and tests/fixtures/; they are the same vectors the
implementation is developed against.
Issues and pull requests are welcome. Please:
keep changes focused and include tests for behaviour changes;
run python -m pytest tests/ and npm test before opening a PR;
for the verifier, do not add network or database access — offline operation is a design constraint, not an implementation detail;
report security issues privately as described in SECURITY.md
rather than in a public issue.
Apache-2.0. See LICENSE.
Deterministic DSL backtest with honest family-level trial accounting.
Define the strategy as an executable JSON DSL (indicators: sma, ema, rsi,
atr, roc, zscore, price; ops: cross_above, cross_below, gt, lt, and, or,
not), supply your own candles, and get net-of-cost per-bar returns plus a
deflated-Sharpe family verdict. Every call is recorded in your family's
trial ledger, so repeated searching deflates future verdicts -- that is
the feature, not a bug: the verdict stays meaningful. The ledger is
keyed to your authenticated account, never to a caller-supplied name.
Price: $0.05 per check (api_key required -- free account at
https://api.alphaassay.com/account includes 3 free checks).Inputs: spec (required), ohlcv_by_symbol (required), api_key, fee_bps_per_side, +1 more.
AlphaAssay's own public track record -- verify us, not our marketing.
Self-calibration of the validator: how many pre-registered signals were
evaluated, how many survived vs. were deflated out, and (as forward
windows mature) the hit-rate of our own verdicts. Signed with the
platform key when configured, honest about insufficient history. This is
the document that makes our 'no' worth something. Price: free.Inputs: no named parameters.
Verify a signed AlphaAssay verdict certificate offline -- without us.
Third-party verification of an Ed25519-signed verdict certificate over
canonical JSON, including hash-chained revocation status. Needs only the
certificate, the signature and the public key -- no platform access, no
network. Use it to check a certificate someone attached to a signal they
are selling. Price: free.Inputs: certificate (required), signature_b64 (required), public_key_pem (required).
Free demo verdict -- see AlphaAssay's full output shape in one call.
Runs the real fail-closed validator over a built-in 40-trade example and
returns the complete verdict envelope (verdict, qualitative findings,
leakage taxonomy, provenance hashes) plus the machine-readable
remediation explanation -- and a free synthetic-null preview: three
seeded no-edge worlds (Heston stochastic volatility, Merton jump
diffusion, symmetric drift bursts) showing the process-placebo
machinery the paid falsify battery attacks your strategy with. No
auth, no payment, no input needed. Use this first to understand the
schema before submitting your own strategy with assay_signal.
Price: free.Inputs: no named parameters.
The full reality-check battery in ONE call -- validator, family deflation, matched-random placebo, capacity ceiling and graveyard prior chained into a single consolidated dossier. Instead of pass/fail you get WHICH gate killed the signal first (machine-readable failure_codes), the placebo percentile, the tradable capacity ceiling, how often the family was already buried, and the family's remaining search budget. Built for iterating agents: it is training feedback with budget economics, not a bare verdict. Supply the strategy as an executable DSL spec plus your own candles; optional per-symbol ADV in USD unlocks the capacity gate, and optional per-symbol daily volatility (sigma_by_symbol_pct, percent) sharpens its impact model for volatile assets. Demote-only, deterministic. Price: $0.05 per check (api_key required -- free account at https://api.alphaassay.com/account includes 3 free checks).
Inputs: spec (required), ohlcv_by_symbol (required), api_key, fee_bps_per_side, +3 more.
Free lookup: how often has this signal family already died?
Anonymised falsification statistics per structural signal family --
tested / killed / survived counts, top kill reasons, and the crowd prior
your submission would be deflated by. Check BEFORE you spend weeks on an
idea whether the crowd already buried it. k-anonymous: families with
fewer than 5 distinct submitters fall back to coarser taxonomy stats.
Price: free.Inputs: variant, strategy, timeframe, fingerprint.
Free payload lint -- fix your submission BEFORE spending a check.
Validates the SHAPE of what you are about to submit, with the same
machine-readable failure vocabulary the paid tools use: DSL schema
validity, OHLCV sanity (finite positive prices, aligned series,
strictly increasing timestamps), per-symbol data presence, trade-row
types, and an honest size warning when the sample is below the paid
gates' evidential floor. Send the same spec/ohlcv_by_symbol/trades you
would send to assay_gauntlet or assay_signal; get back ok plus named
findings (dsl_invalid, timestamps_not_monotonic,
ohlcv_non_finite_or_non_positive, ohlcv_series_misaligned,
symbol_data_missing, trades_rows_invalid, sample_below_engine_floor)
with plain-language details. Shape lint only -- a clean preflight is
NOT evidence of an edge and never blesses a signal; it just means your
paid check will not bounce on format. Deterministic, no ledger write,
no account needed. NOT financial advice. Price: free.Inputs: spec, trades, ohlcv_by_symbol.
Test ANY signal provider in an afternoon -- the falsification protocol as machine-readable rules.
Seven falsifiable tests that separate edge from selection, runnable
against any signal seller (paid channel, platform, bot) without their
cooperation: provenance (tamper-proof timestamps or it did not happen),
survivorship (the full issue history incl. losers), pre-registration
(ten sealed forward calls), placebo (does random timing do as well?),
costs (profitable AFTER fees/spread/slippage?), trial accounting (one
winner out of how many quiet attempts?), and the examiner test (apply
it all to whoever grades the signals -- including AlphaAssay: golden
specimens, offline signature check, public daily calibration). Each
test carries a machine-checkable failure_condition, and tests 3-6 name
the AlphaAssay endpoint that automates them. Static and deterministic:
this tool hands you the checklist, it does not rate, score or rank any
provider and it never fetches external data. NOT financial advice.
Price: free.Inputs: no named parameters.
Pre-register a signal NOW; earn a verdict that cannot be overfit.
Signal anchoring: your strategy spec (executable JSON DSL) is
canonically hashed today and anchored into a public Merkle tree; the
later verdict (assay_verdict) uses exclusively data from AFTER
registration -- a true post-cutoff out-of-sample that is mathematically
immune to overfitting, because the data did not exist when you
committed. v2: optionally seal a machine-writable HYPOTHESIS tuple
alongside the spec -- declared_pass_sharpe_annualized (your
pre-committed success bar), max_evaluation_bars (ex-ante window cap:
waiting longer than promised cannot improve the verdict),
fee_bps_per_side and trading_calendar (the sealed evaluation terms),
plus an optional rationale. The tuple is canonically hashed and bound
into the same Merkle anchor as the spec, and assay_verdict ENFORCES
it: deviating terms are recomputed under the seal and flagged
goalpost_moved -- success criteria chosen after seeing the data are
not criteria. Registrations are idempotent, withdrawals still count
toward your family's trial budget (anti-gaming). Registrations are
keyed to your authenticated account. Price: $0.05 per check (api_key
with the register scope required -- free account at
https://api.alphaassay.com/account includes 3 free checks).Inputs: spec (required), api_key, hypothesis, embargo_bars, +1 more.
WHICH variants of your sweep survive family-wise error control -- an error-budget disclosure, never a ranking.
Send the same T x N trial matrix assay_pbo grades (rows =
time-ordered periods, columns = every configuration you tried) and
get Romano-Wolf stepwise multiple testing over it: studentized
per-config statistics, circular block bootstrap over the time rows
(serial dependence respected), stepdown max-statistic critical values
(Romano/Wolf 2005, Econometrica; Hansen SPA 2005). Answer, per
variant IN INPUT ORDER: could this family's evidence kill it at
family-wise error rate alpha, and in which stepdown round? Survival-
map framing by construction: the verdict vocabulary is fail (nothing
survives: NO_SURVIVORS_AT_FWER), conditional (survivors disclosed --
explicitly NOT a pass) or insufficient_evidence (blocked with a named
reason); this tool structurally cannot bless, rank or recommend.
Natural partner of assay_pbo -- one matrix, two questions: is the
SELECTION overfit (pbo), and which variants does the error budget
leave standing (survivors). Deterministic (bootstrap seeded from the
input digest); fail-closed on size, budget and numerics; no ledger
write. NOT financial advice. Price: $0.05 per check (api_key required
-- free account at https://api.alphaassay.com/account, 3 free checks
included).Inputs: alpha, api_key, n_bootstrap, block_length, +2 more.
Post-cutoff verdict for a pre-registered signal (the tamper-proof test).
Evaluates a registration strictly on bars AFTER the registration cutoff,
with maturity floor and fail-closed data-gap handling (use
trading_calendar='weekdays' for equity daily bars so weekends do not
count as gaps; omit fee/calendar to use the registration's sealed
terms). The verdict embeds the family-deflated Sharpe: even an
anchored signal is deflated by how much its family was searched. For
v2 registrations with a sealed hypothesis, the verdict additionally
reports oos_sharpe_annualized against the pre-committed bar
(threshold_met -- a descriptive fact, never an endorsement) and flags
goalpost_moved when the requested terms deviate from the sealed ones
(the seal always wins). Price: $0.05 per check (api_key required --
free account at https://api.alphaassay.com/account includes 3 free
checks).Inputs: prereg_id (required), ohlcv_by_symbol (required), api_key, fee_bps_per_side, +1 more.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 11 tools
Each tool serves a distinct purpose within the signal validation workflow, and descriptions are detailed. The slight overlap between assay_backtest (basic DSL backtest) and assay_gauntlet (comprehensive battery) is clarified by their descriptions, making them distinguishable.
All tools share the 'assay_' prefix and mostly use noun phrases (e.g., assay_graveyard, assay_gauntlet). A few (assay_register, assay_certificate_verify) use verbs, which is a minor inconsistency but still readable and predictable.
With 11 tools, the server covers the essential functions for signal validation—registration, preflight checks, backtesting, full validation, certificate verification, and more. The count feels well-scoped without being excessive or insufficient.
The tool set addresses the core lifecycle of signal validation: registration, pre-submission validation, multiple validation gates, and certificate verification. Minor gaps exist (e.g., no tool for listing past registrations or verdicts), but the core workflow is well-covered.
Honest A-F grades for trading strategies, backtested on real out-of-sample data. No hype.
Validated trading edges across futures, equities, crypto. Live signals, full audit trail.
Fail-closed backtest/signal validation: overfitting, leakage, deflated Sharpe. NOT financial advice.
Praxis: published trading strategies re-run and audited — verdicts, claimed vs measured stats.
