Skip to main content
Glama
doazvjettu

leakguard-mcp

by doazvjettu

leakguard-mcp

PyPI Python License: MIT release MCP registry glama

Squawk for backtests. A local-first MCP server that static-analyzes agent-generated Python code and flags lookahead bias & data leakage before the backtest runs.

Works for any time-series ML code — quant trading (crypto, equities, forex, futures), demand forecasting, energy, weather, IoT sensors — wherever a wrong .shift() or a global normalization silently poisons your results.

  • Pure source analysis (libcst) — your code never leaves the machine, no API calls.

  • Heuristic, not a proof: severity tiers (🔴 error / 🟡 warning), like an aviation squawk.

  • Framework-agnostic: pandas / numpy / polars, any stack.

  • MCP-native (stdio): works directly inside Claude Code, Cursor, and any MCP-compatible agent.

  • All 10 rules ship free — no license, no tiers, no phone-home.

leakguard flagging three lookahead leaks in a strategy file


Why this exists

AI agents (Claude Code, Cursor) write feature engineering and strategy code faster than humans can review it. But they introduce lookahead bias at scale — subtle time-boundary errors that backtest perfectly and fail catastrophically in live trading or production:

# Agent writes this — looks fine, is catastrophically wrong
df['momentum'] = df['close'].shift(-5)   # uses FUTURE prices as a feature
df['vol_norm'] = (df['close'] - df['close'].mean()) / df['close'].std()  # leaks future mean

leakguard catches these in the same agent loop — before the backtest runs:

LG001 🔴 line 2: Future shift used as feature — shift(-5) references 5 bars ahead.
  Fix: df['momentum'] = df['close'].shift(5)   (lag, not lead)

LG003 🔴 line 3: Global-fit normalization — mean/std computed over the full series
  before any train/test split, leaking future statistics into the past.
  Fix: df['vol_norm'] = (df['close'] - df['close'].expanding().mean()) / df['close'].expanding().std()

The agent reads the finding + fix snippet and self-corrects in one turn. No human review needed.


Related MCP server: Lanalyzer MCP Server

Install

Requirements

  • Python 3.11+

  • uv (recommended) or pip

From PyPI

pip install leakguard-mcp

From source

git clone https://github.com/doazvjettu/leakguard-mcp
cd leakguard-mcp
uv sync

Setup with Claude Code

Add to your MCP config (~/.claude/claude_desktop_config.json or .claude/settings.json in your project):

{
  "mcpServers": {
    "leakguard": {
      "command": "python",
      "args": ["-m", "leakguard.server"]
    }
  }
}

Or if installed via uv:

{
  "mcpServers": {
    "leakguard": {
      "command": "uv",
      "args": ["run", "python", "-m", "leakguard.server"]
    }
  }
}

Restart Claude Code. leakguard's tools are now available to the agent.

Setup with Cursor

Add the same block under mcpServers in your Cursor MCP settings file.


MCP Tools

Tool

Description

lint_code(code)

Analyze a code string, return findings

lint_file(path)

Analyze a file on disk

lint_paths(glob)

Analyze all matching files

list_rules()

List all rules with severities

explain_rule(rule_id)

Full rationale + fix patterns for a rule


CLI

The same scanner is available as a CLI — handy for a pre-commit hook or CI step (exits non-zero when leakage is found):

uv run leakguard path/to/strategy.py
# or, installed:  leakguard path/to/strategy.py

It prints each finding with its severity, line/col, and a concrete fix snippet — the same output shown in the demo above.


Rules

All 10 rules active, no tiers:

ID

Severity

Pattern

LG001

🔴

Future shift as feature: shift(-n) / diff(-n) / pct_change(-n)

LG002

🔴

Centered windows: rolling(center=True)

LG003

🔴

Global-fit scaling: StandardScaler().fit(full_df) / hand-rolled mean-std before split

LG004

🔴

Shuffled time-series split: train_test_split default, KFold, cross_val_score(cv=n)

LG005

🔴

Label leakage: future-derived target column reused in features

LG006

🟡

Whole-history aggregates as features: .max() / .mean() over full series

LG007

🔴

Backfill imputation: bfill() / fillna(method='bfill')

LG008

🔴

Forward asof-joins: merge_asof(direction='forward'/'nearest')

LG009

🟡

Resample label/closed mismatch on bar timestamps

LG010

🟡

groupby().transform()/agg() spanning train/test boundary

Each finding includes a concrete fix snippet so the calling agent can self-correct immediately.


Benchmark

Measured on two labeled corpora, 49 snippets total. Reproduce with uv run python -m benchmark.run.

Honesty note: the trading corpus was written by the tool's author — treat its numbers as regression fixtures, not independent validation. The general-ML corpus is one arm's length removed in domain (author-composed reproductions of widely documented leakage anti-patterns, not a downloaded public dataset). The corpus deliberately includes adversarial snippets the scanner is known to miss; they are counted against it.

Trading corpus — 39 snippets (23 leaky, 16 clean + hard negatives):

Rule

Precision

Recall

TP

FP

FN

LG001

75%

100%

6

2

0

LG002

100%

100%

5

0

0

LG003

75%

100%

3

1

0

LG004

100%

100%

4

0

0

LG005

100%

100%

5

0

0

LG006

100%

100%

5

0

0

LG007

100%

100%

5

0

0

LG008

100%

100%

2

0

0

LG009

75%

100%

3

1

0

LG010

100%

100%

2

0

0

Overall

91%

100%

40

4

0

General-ML corpus — 10 snippets (LG003/LG004/LG010): Precision 88%, Recall 100% (TP 7 / FP 1 / FN 0).

Combined: Precision 90.4%, Recall 100% (TP 47 / FP 5 / FN 0).

Recall is 100% on this corpus — every adversarial miss exposed has since been fixed (constant propagation, hand-rolled normalization, cv=<int>, drop-based selection). Leak shapes not yet in the corpus are still missed — see Known Limitations below.

Known false positives (clean code that gets flagged)

  • LG001: a forward-return label built with a negative shift and used only as y — pure AST cannot distinguish a target column from a feature.

  • LG003: fit inside a helper defined above the split call site — line-order heuristic confuses definition order with execution order.

  • LG004: shuffled splits on genuinely non-temporal data — no datetime-index inference yet.

  • LG009: resampling for reporting/plotting rather than features — intent is invisible to static analysis.

Known false negatives (leak shapes not yet covered)

  • LG004: cross_val_score(...) with cv omitted (defaults to KFold).

  • LG005: taint through df.loc[:, 'col'] = ... or df.assign(col=...).

  • All rules: values flowing through function calls, dicts, or non-constant variables — no cross-function dataflow.

These sets are pinned in tests/test_benchmark.py: any new miss or silent fix fails the suite until docs and corpus are updated to match.


Develop

uv sync --extra dev
uv run pytest                              # 96 tests
uv run python -m leakguard.server          # stdio MCP server
uv run leakguard demo/strategy_leaky.py    # CLI on the demo file
uv run python -m benchmark.run             # precision/recall tables + FP/FN lists

The scanner core lives in leakguard/core/ (pure, no MCP imports); server.py and cli.py are thin wrappers. Each rule has a fixture pair under tests/fixtures/.


Limitations (v1)

  • Heuristic static analysis — catches ~90% of common patterns, not 100%.

  • Single-file only — no cross-module taint tracking.

  • Python only (pandas / numpy / polars).

  • No runtime execution — cannot catch patterns that only emerge at runtime.

Contributions welcome: new corpus snippets (especially real bugs you've hit) strengthen the benchmark more than new rules do.

Available Tools

5 tools
explain_ruleA

Long-form rationale and canonical fix pattern for one rule id (e.g. 'LG001').

ParametersJSON Schema
NameRequiredDescriptionDefault
rule_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the tool returns 'long-form rationale' and 'canonical fix pattern', indicating a read-only information retrieval. However, it does not describe output format, latency, or any prerequisites (e.g., rule_id must exist). The description is adequate but lacks depth for a fully transparent behavioral profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the tool's purpose, scope (one rule id), and output type (long-form rationale and fix pattern). No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has only one parameter, no annotations, and an output schema (not shown but indicated), the description covers the core function without major gaps. It could be improved by clarifying that the rule_id must be valid and that the output is textual, but it is largely complete for its simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has one parameter 'rule_id' with no description. The tool description provides an example ('e.g. 'LG001''), which adds some meaning by hinting at the format (likely rule identifiers like those in sibling tools). However, with 0% schema description coverage, the description does not fully explain the parameter's validation or accepted values. The example is useful but not exhaustive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool provides 'long-form rationale and canonical fix pattern for one rule id', specifying the verb 'explain' (implied by the name) and the resource (a specific rule id). It distinguishes from sibling tools like 'list_rules' which lists all rules, by focusing on a single rule id.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for getting detailed rationale and fix pattern for a specific rule id, but it lacks explicit guidance on when to use this tool versus alternatives like 'lint_code' (which might apply rules) or 'list_rules' (which lists all rules). No when-not-to-use or alternative conditions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_codeC

Scan a source string for leakage patterns. Returns {ok, parse_error, findings}.

ParametersJSON Schema
NameRequiredDescriptionDefault
codeYes
filenameNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided. The description only mentions the return shape ({ok, parse_error, findings}) but omits any behavioral traits such as whether it is read-only, required permissions, side effects, or rate limits. The burden is on the description to disclose such behaviors, and it fails to do so.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short (one sentence plus return type), which is concise. However, it is too brief, missing important details. Every sentence does earn its place, but the description could be more informative without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With two parameters (one optional), no parameter descriptions, no annotations, and an output schema only hinted at in the description, the description is critically incomplete. It does not explain what 'leakage patterns' means, how to interpret the return, or any context about the tool's behavior. This is insufficient for proper tool selection and use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not elaborate on the parameters. 'code' is implied as the source string, but 'filename' is optional and has no explanation. The description provides no additional meaning beyond what the schema gives, which is minimal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool scans a source string for leakage patterns, which is a specific verb+resource combination. It implicitly distinguishes from siblings like lint_file and lint_paths that operate on files or paths. However, it does not explicitly contrast with these siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The description only implies it is for scanning a source string, but does not mention when to choose lint_code over lint_file or lint_paths, nor any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_fileA

Scan a single file on disk. Returns {ok, parse_error, findings} or an error dict.

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the return shape ({ok, parse_error, findings} or error dict) but omits details like error handling for missing files or performance characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. It could integrate sibling differentiation without added length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one param, output schema exists), the description covers the basic purpose and return format but lacks guidance on error states and prerequisite conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The sole parameter 'path' is a string with 0% schema coverage. The description adds only that it's a file on disk, failing to clarify path format (absolute/relative) or constraints beyond type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Scan a single file on disk' with a clear verb and resource, and distinguishes from siblings like lint_code and lint_paths through the 'single file' scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use for single files on disk but does not explicitly contrast with alternatives such as lint_code for code strings or lint_paths for multiple files.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_pathsA

Scan every file matching a glob. Returns {path: {ok, parse_error, findings}}.

ParametersJSON Schema
NameRequiredDescriptionDefault
patternYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes the core action (scan every file matching a glob) and return shape. While no annotations exist, the description implies a read-only operation without side effects. Could mention potential resource constraints or glob evaluation behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences front-loading the action and return type. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has one parameter and an output schema, the description adequately covers the purpose and return shape. Missing details like glob syntax or edge cases, but still functional for its simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description only says 'a glob' for the pattern parameter. Does not specify glob syntax, workspace location, or any constraints; relies on user understanding of glob patterns.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it scans files matching a glob pattern and returns a structured result with ok/parse_error/findings per path. Distinct from sibling tools that operate on single files or code snippets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage for batch scanning via glob, but no explicit guidance on when to use this tool vs. alternatives like lint_file or lint_code. No exclusion criteria or suggested use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_rulesA

Full rule catalog with id, name, severity, and one-line summary.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden but only states the output fields. No disclosure of side effects, auth needs, or rate limits, though for a read-only list tool this is minimal risk.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence that is direct and front-loaded, conveying exactly what the tool returns. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero parameters and an output schema, the description adequately covers the tool's behavior. The agent can infer the rest from the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so schema coverage is 100%. Baseline 4 applies; description adds no parameter info but none is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it provides a 'full rule catalog' with specific fields (id, name, severity, one-line summary). It distinguishes itself from siblings like explain_rule (explanation) and lint tools (analysis).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when/when-not guidance or alternatives to siblings. However, given the tool's simplicity and distinct sibling purposes, usage is implied but not explicitly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.1
    • First observedexplain_rule
    • First observedlint_code
    • First observedlint_file
    • First observedlint_paths
    • First observedlist_rules

TDQS

A3.6/5.0

Scored across 5 tools

Disambiguation5/5

Each tool targets a distinct action: listing rules, explaining a single rule, scanning code from a string, scanning a single file, and scanning files by glob. There is no ambiguity; an agent can easily distinguish between them.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with lowercase and underscores. The verbs vary but are descriptive, and the nouns specify the target clearly. This pattern is predictable and easy to understand.

Tool Count5/5

With 5 tools, the server is well-scoped for a linter. It provides rule introspection and scanning at three levels of granularity without unnecessary clutter, making the count appropriate for its purpose.

Completeness4/5

The server covers the core lifecycle: rule discovery (list_rules), rule understanding (explain_rule), and scanning (lint_code/file/paths). Minor gaps like rule configuration or suppression are absent, but the essential functionality is complete for standard usage.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Local-first backtesting engine with built-in overfitting detection (PBO, deflated Sharpe, bootstrap CI, walk-forward) and a native MCP server for AI agents to validate trading strategies.
    4
    Apache 2.0
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI models to perform static taint analysis on Python code, detecting security vulnerabilities by tracking data flows from sources to sinks.
    9
    AGPL 3.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    AI-powered security scanner for Python projects and GitHub repositories. Detects vulnerabilities, secrets, and provides AI risk assessment.
    11
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    Static analysis for vibe-coded apps. Flags security, reliability, performance, and AI quality issues in code generated by Cursor, v0, Bolt, and Copilot.
    575 npm
    15
    MIT