How do I use LeakageLens MCP?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@LeakageLens MCP Audit data/train.csv for leakage before training. Target=churned, entity=customer_id, split=split." That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

LeakageLens MCP

by ivky03

Overview Schema Source Related Servers Score Discussions Admin

Python

Hybrid

LeakageLens MCP

Dataset forensics for AI coding agents. LeakageLens catches target leakage, cross-split entities, time leakage, PII, identifier features, class imbalance, and preprocessing-before- split errors before an agent trains a misleading model.

It is not a “chat with CSV” server. The dataset remains local; the MCP client receives bounded, structured evidence and an explicit pass, review, or block verdict.

Why MCP?

Any MCP-compatible coding agent can discover the same audit tools and guidance without a custom integration. Resources provide experiment policy, prompts enforce a review workflow, and tools return typed evidence. The core engine works without an LLM; an optional Gemini check reasons about whether features exist at prediction time.

Related MCP server: Plugin Health Auditor MCP Server

Quick start

python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

leakagelens examples/leaky_churn.csv \
  --target churned --split split --entity customer_id --time-column signup_date

pytest -q
python evals/benchmark.py

Expected demo verdict: block, with direct-target, entity-overlap, PII, identifier, and semantic leakage findings.

Connect an MCP client

Example local configuration:

{
  "mcpServers": {
    "leakagelens": {
      "command": "/absolute/path/to/.venv/bin/leakagelens-mcp",
      "env": {
        "LEAKAGELENS_DATA_ROOT": "/absolute/path/to/leakagelens-mcp"
      }
    }
  }
}

Then ask your agent:

Review examples/leaky_churn.csv before training a churn model. The target is churned, the entity is customer_id, signup_date is the time column, and split defines train/test.

Remote transport

MCP_TRANSPORT=streamable-http LEAKAGELENS_DATA_ROOT="$PWD" leakagelens-mcp

Or run the container with a read-only data mount:

docker build -t leakagelens-mcp .
docker run --rm -p 8000:8000 -v "$PWD/examples:/data:ro" leakagelens-mcp

MCP surface

Primitive	Name	Purpose
Tool	`profile_dataset`	Bounded schema, missingness, uniqueness, and samples
Tool	`audit_dataset`	Leakage, PII, split, identifier, and metric checks
Tool	`audit_training_code`	AST audit of split/preprocessing order
Tool	`review_feature_availability`	Optional Gemini semantic review
Resource	`guidance://experiment-contract`	Minimum valid experiment contract
Prompt	`review_before_training`	Reusable pre-training workflow

All file tools are restricted to LEAKAGELENS_DATA_ROOT to avoid arbitrary host-file access.

Detection design

Direct leakage: equality, near-perfect numeric association, deterministic categorical maps
Semantic leakage: outcome-like and target-derived column names
Split leakage: entity overlap and invalid chronological boundaries
Privacy: PII column names and value-pattern scans
Pipeline leakage: Python AST detects fitting or resampling before the split
Evaluation risk: imbalance-aware metric recommendations

Every finding includes a stable code, severity, evidence, affected columns, and remediation. The risk score is deterministic: critical 30, high 15, medium 7, low 3, capped at 100.

Optional Gemini review

pip install -e '.[gemini]'
export GEMINI_API_KEY='...'
export GEMINI_MODEL='gemini-2.5-flash'

Only column names/types and the user-provided prediction moment are sent. Raw rows are not sent. Security decisions never depend solely on the model.

Benchmark

evals/benchmark.py generates 35 reproducible scenarios: 25 seeded leakage/privacy/evaluation cases and 10 clean negative controls. It reports scenario recall and false-block rate. Add cases before adding heuristics; this prevents a growing collection of unmeasured rules.

Current synthetic benchmark result: 35/35 scenarios passed, 100% scenario recall, 0% false-block rate. These figures validate the included seeded cases; they are not estimates of performance on arbitrary real-world datasets.

Current boundaries

Statistical association is a warning, not proof of leakage.
Semantic rules cannot know feature availability without a prediction-time contract.
The AST audit recognizes common scikit-learn patterns, not arbitrary dynamic Python.
This release audits supplied splits; it does not mutate the user’s dataset.

Architecture

MCP client → FastMCP tools → path boundary → audit engine → typed findings
                                      ├── dataframe checks
                                      ├── split validation
                                      ├── Python AST audit
                                      └── optional Gemini review

Development

ruff check .
pytest -q
python evals/benchmark.py

See HELPER_GUIDE.md for a concise code walkthrough.

Available Tools

4 tools

audit_datasetC

Detect target, entity and temporal leakage, PII, identifiers, and metric risks.

ParametersJSON Schema

Name	Required	Description	Default
`path`	Yes
`split`	No
`entity`	No
`target`	No
`time_column`	No

TDQS

C2.9/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not state whether the tool is read-only, what side effects or outputs are produced, or how it behaves with partial parameters. The term 'Detect' implies analysis, but this is not explicit and important behavioral details are missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with no fluff, but it is a flat list of risk categories rather than a structured explanation. It is brief and front-loaded, yet the brevity comes at the cost of needed detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no output schema and no annotations, the description is incomplete. It does not explain what the audit produces, how to provide required inputs, or what constraints exist. The agent lacks critical context needed to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there are 5 parameters, so the description must compensate. It mentions 'target', 'entity', and 'temporal', which vaguely map to the target, entity, and time_column parameters, but it does not explain their formats, relationships, or the required path parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Detect') and a clear resource (dataset risks), enumerating concrete risk categories: target, entity, and temporal leakage, PII, identifiers, and metric risks. This distinguishes it from siblings like profile_dataset and audit_training_code, which focus on profiling and code auditing respectively.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus profile_dataset, audit_training_code, or review_feature_availability. It does not state conditions, exclusions, or alternatives, leaving the agent to infer the appropriate selection context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

audit_training_codeA

Statically detect preprocessing or resampling before the train/test split.

ParametersJSON Schema

Name	Required	Description	Default
`path`	Yes

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A3.6/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the check is static ('statically detect') and that it targets preprocessing/resampling placement, but it does not describe side effects, failure modes, or what happens when no issues are found. The output schema covers some return expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single focused sentence with no filler. The verb and object are front-loaded and every word adds meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is adequate for a one-parameter tool with an output schema, but it omits path semantics and usage routing to dataset-focused siblings. It is not as complete as it could be for a first-time agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description never explains the 'path' parameter. It does not clarify whether path points to a file, directory, or specific language, so the agent must infer from the tool name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('statically detect') and a specific target ('preprocessing or resampling before the train/test split'). This clearly differentiates it from sibling dataset-focused tools like audit_dataset or profile_dataset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for auditing training code for leakage before splitting, but it does not explicitly state when to prefer it over siblings or mention exclusions. No alternative tools are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

profile_datasetA

Profile a local CSV, Parquet, or JSONL without sending the full dataset to the model.

ParametersJSON Schema

Name	Required	Description	Default
`path`	Yes
`sample_rows`	No

TDQS

A3.6/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It usefully reveals that the full dataset is not sent to the model, implying local processing, but it does not state whether the operation is read-only, what side effects occur, or what the profiling process entails beyond that privacy guarantee.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, well-structured sentence that front-loads the core purpose and adds the important privacy qualifier. Every word contributes value, with no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and no schema-level parameter descriptions, the one-line description leaves significant gaps: it does not describe the returned profile structure, clarify sample_rows semantics, or help the agent choose between this and audit_dataset. The simple input schema keeps it from being a 1, but an agent would still be guessing about important call behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds meaning to the path parameter by specifying it must be a local file in CSV, Parquet, or JSONL format. However, it does not explain sample_rows or how it interacts with profiling, and schema description coverage is 0%, so the description only partially compensates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Profile') and the resource ('a local CSV, Parquet, or JSONL'), and adds a meaningful qualifier ('without sending the full dataset to the model'). It does not explicitly differentiate from the sibling audit_dataset, which also operates on datasets, so it falls just short of a full 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: use this tool to profile local tabular/structured files while avoiding sending the full dataset to the model. However, it does not state exclusions or explicitly point to alternatives such as audit_dataset when a full audit is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

review_feature_availabilityC

Optionally ask Gemini which features may be unavailable at the prediction moment.

ParametersJSON Schema

Name	Required	Description	Default
`path`	Yes
`target`	Yes
`prediction_time`	Yes

TDQS

C2.4/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral disclosure, but it only says the tool asks Gemini about availability. It does not state whether the operation has side effects, requires external connectivity, returns a report, or behaves differently depending on inputs, leaving the agent to guess.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one sentence with no redundant clauses and the core action appears first. 'Optionally' is a minor filler, but overall the structure is appropriately compact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with three required parameters, no output schema, and no annotations, this description is far too thin: it leaves parameter meanings, expected input formats, return behavior, and relationship to siblings unspecified. An agent cannot invoke it correctly from this text alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description does not explain path, target, or prediction_time. With three required parameters and no semantics beyond their names, an agent cannot confidently construct valid arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action (asking Gemini) and subject (features that may be unavailable at prediction time), so an agent can roughly understand the tool's purpose. However, it does not define what 'features' refers to or differentiate it from the sibling dataset/audit tools, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It offers no guidance on when to invoke this tool, what conditions warrant it, or why an agent might choose it over profile_dataset, audit_dataset, or audit_training_code. The word 'Optionally' hints it is not mandatory but does not provide a decision rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

4 tool updatesv0.1.0Sep 5, 2026
- First observedaudit_dataset
- First observedaudit_training_code
- First observedprofile_dataset
- First observedreview_feature_availability

Install Server

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.3/5.0

Scored 2026-09-05 23:30 across 4 tools

Disambiguation4/5

Each tool targets a distinct stage of leakage detection: profiling, dataset auditing, code auditing, and feature availability review. However, profile_dataset and audit_dataset could be confused by an agent since both operate on datasets, though their descriptions help separate general profiling from leakage-specific auditing.

Naming Consistency5/5

All tool names consistently follow a verb_noun pattern using lowercase snake_case: profile_dataset, audit_dataset, audit_training_code, and review_feature_availability. This makes the tool set predictable and easy to navigate.

Tool Count5/5

Four tools is well-scoped for a specialized leakage-detection server. Each tool has a clear, non-redundant role, and the count feels appropriate rather than thin or bloated.

Completeness4/5

The tool surface covers the core leakage-detection workflow: profile data, audit data for leakage, audit training code, and verify feature availability. Minor gaps exist around remediation or actionable reporting after an audit, but the main detection loop is complete.

Maintenance

ActivityMaintained

ResponsivenessNo issues

Resources

GitHub Repository

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

The Ultimate Critic A
io.github.marksignals
Check AI work against requirements and return structured verdicts, findings, and repair steps.
Testing & QA Tools AI & Machine Learning
2
governance-platform A
ai.geodesiclabs
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
AI & Machine Learning Testing & QA Tools Monitoring
373
DJZS Trust MCP A
ai.djzs
Deterministic pre-execution audit for trading agents. PASS/WAIT/FAIL, reproducible verdict_hash.
Finance Blockchain
423
sigma-gate A
io.github.spektre-labs
Deterministic trust gate for AI output: leaked-secret, prompt-injection & PII in one call.
Security AI & Machine Learning
2

Related MCP Servers

Meridian
Legal & Compliance Code Analysis
KNambiarDJsc
A
license
Not graded
quality
B
maintenance
Local-first AI compliance scanner via Model Context Protocol, scanning codebases for violations of DPDPA 2023, RBI FREE-AI, SEBI AI/ML, and the EU AI Act.
Updated 3 months ago (2026-06-10 14:01 UTC)
1
Apache 2.0
Plugin Health Auditor MCP
Code Analysis Security Developer Tools
AMATSUGI-120
A
license
Not graded
quality
B
maintenance
Provides audit_plugin_health and prepare_semantic_review tools for deterministic inspection of Codex plugins and Agent Skills, generating evidence-backed reports without executing or transmitting target code.
Updated 2 months ago (2026-07-17 03:52 UTC)
1
MIT
tar-engine
Penetration Testing Code Analysis Security
qingxuantang
A
license
Not graded
quality
A
maintenance
Audits AI agent skills for safety using static, semantic, adversarial, and supply-chain analysis, providing scores and risk flags. Can be run via CLI, CI, or as an MCP tool from Claude Code, Cursor, and Codex.
Updated 2 months ago (2026-07-23 03:35 UTC)
2
Apache 2.0
shipcheck-mcp
Code Analysis Security
noureldeensalama
A
license
Not graded
quality
B
maintenance
Enables MCP-compatible AI agents to scan code for leaked secrets, copyleft licenses, unprotected routes, missing privacy policies, and risky card handling before committing or shipping.
Updated 17 days ago (2026-08-25 22:56 UTC)
40
MIT