de en es ja ko ru zh

vacuous-tests-mcp

Name: vacuous-tests-mcp
Author: zegroged

by zegroged

Overview Schema Source Related Servers Discussions Admin

Python

Local

vacuous-tests-mcp

MCP-сервер, который находит тесты, проходящие независимо от того, что делает код.

Пустой тест хуже отсутствующего. Отсутствующий тест заметно отсутствует. Пустой тест сидит в наборе, проходит зелёным, учитывается в общем количестве и цитируется как доказательство того, что поведение покрыто — хотя ничего не проверяет. Это покрытие, которое невидимо отсутствует, и оно переживает именно те ситуации, для выявления которых существуют тесты.

Их легко написать случайно и трудно заметить при чтении, потому что пустой тест и настоящий часто выглядят почти одинаково.

Случай, на основе которого это было создано

Rust-файл из 6 764 строк в проекте, чей набор тестов уже регулярно проверялся вручную. В нём два теста:

// flagged
const SRC: &str = include_str!("lib.rs");
assert!(SRC.contains("pub sas_verified: bool,"), "the field must be exposed");

// not flagged
const SRC: &str = include_str!("lib.rs");
let prod = SRC.split_once("\n#[cfg(test)]\nmod tests {").expect("test module").0;
assert!(prod.contains("InviterSecrets::create_for_group("), "production never calls it");

Оба встраивают исходный код самого файла. Первый утверждает непосредственно о SRC — а SRC содержит сам тест, включая искомую строку. Он истинен по построению: удалите производственный код, который он якобы защищает, и он останется зелёным. Второй сначала отрезает тестовый модуль и утверждает только о производственном срезе, поэтому он падает при изменении производственного кода. Это настоящий барьер.

Сканирование этого файла сообщает об одном результате, в правильной строке, и оставляет остальные четыре места с include_str! в покое. Сканер следует за привязкой, а не сопоставляет шаблон include_str!, что и разделяет эти два случая.

Related MCP server: openhonest-umbra

Правила

Правило	Серьёзность	Что оно выявляет
`self-referential-source`	high	Тест встраивает свой собственный исходный код и утверждает, что в нём встречается литерал. Литерал находится в утверждении, поэтому он никогда не может упасть.
`no-assertions`	high	Нет никаких утверждений. Только паника или исключение могут привести к падению теста, поэтому неверное, но тихое поведение проходит.
`tautological-assertion`	high	`assert!(true)`, `assert_eq!(x, x)`, `expect(true).toBe(true)` — выполняется независимо от кода.
`empty-body`	high	В теле нет ничего, что могло бы упасть.
`skipped-test`	info	`#[ignore]`, `it.skip`, `@pytest.mark.skip`. Проходит зелёным, потому что не выполняется.

Языки

Язык	Метод	Точность
Python	`ast` из стандартной библиотеки	Точный
Rust	сканер сопоставления скобок по исходному тексту	Эвристический
JavaScript / TypeScript	сканер сопоставления скобок по исходному тексту	Эвристический

Сканеры Rust и JS маскируют строковые литералы и комментарии перед сопоставлением, поэтому { внутри строки или закомментированное утверждение не может их обмануть. Они настроены скорее пропустить случай, чем выдумать его: ложное срабатывание здесь стоит дороже, чем пропуск, потому что первый неверный ответ учит людей игнорировать вывод.

Относитесь к каждому результату как к вопросу для проверки, а не как к приговору. Каждый указывает файл и строку, поэтому подтверждение занимает секунды.

Установка

Ещё нет на PyPI — установите из исходников:

git clone https://github.com/zegroged/vacuous-tests-mcp
cd vacuous-tests-mcp
pip install .

Это помещает команду vacuous-tests-mcp в ваш PATH.

Использование из MCP-клиента

Добавьте в конфигурацию MCP вашего клиента:

{
  "mcpServers": {
    "vacuous-tests": {
      "command": "vacuous-tests-mcp"
    }
  }
}

Для Claude Code:

claude mcp add vacuous-tests -- vacuous-tests-mcp

Затем попросите его просканировать что-нибудь:

Просканируйте ./src на предмет тестов, которые не могут упасть.

Инструменты

scan_tests(path, include_skipped=True, max_findings=100) Обходит файл или каталог и сообщает о тестах, которые не могут упасть. Каталоги сборки и зависимостей (target, node_modules, .venv, …) пропускаются. Результаты возвращаются сначала по наибольшей серьёзности, каждый с путём, строкой, именем теста, правилом и фрагментом.

list_rules() Каждое правило с описанием, чтобы модель могла решить, что запросить.

explain_rule(rule) Что обнаруживает одно правило и как обычно устраняется результат.

Сервер только читает. Он не записывает файлы, не выполняет сканируемый код и не выходит за пределы указанного пути.

Разработка

pip install -e ".[dev]"
pytest

Набор покрывает каждое правило на каждом языке и — что более важно — проверяет, что обычный тест, находящийся рядом с пустым, не сообщается. Также есть сквозной тест, который запускает сервер как подпроцесс и проводит его через реальное рукопожатие MCP, так что протокольный уровень покрыт, а не предполагается.

Ограничения

Обнаружение Rust и JS/TS является текстовым, а не синтаксическим. Макрогенерируемые тесты, необычное форматирование и сильно вложенные замыкания могут быть пропущены.
no-assertions не знает, что тест может существовать исключительно для доказательства того, что что-то не вызывает панику. Такой тест является истинно положительным по правилу и ложноположительным по намерению; укажите это явным утверждением, и он замолчит.
Сканируются только языки из таблицы выше. Остальные файлы игнорируются, а не угадываются.

Лицензия

MIT

Available Tools

3 tools

explain_ruleA

Explain one rule: what it detects and how the finding is usually resolved.

Args: rule: A rule id as returned by list_rules or by a finding, for example self-referential-source.

ParametersJSON Schema

Name	Required	Description	Default
`rule`	Yes

Output Schema

ParametersJSON Schema

Name	Required	Description
No output parameters

TDQS

A3.7/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose the core behavior (returns an explanation of detection and resolution), but says nothing about error handling for unknown rule ids, the structure of the returned explanation, or any side effects. Given an output schema exists, the return structure gap is partly covered, but the description alone is thin on behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence, and the argument documentation follows in a compact, readable block. Every sentence earns its place — the example and provenance note are exactly the information an agent needs. No filler, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool of low complexity with an output schema present, the description is largely sufficient: it states the tool's purpose, what the output covers, and where the argument value comes from. Minor gaps like behavior on invalid rule ids are acceptable given the simple scope and the output schema's presence.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description fully compensates for the 'rule' parameter by stating its provenance ('as returned by list_rules or by a finding') and providing a concrete example ('self-referential-source'). This adds real meaning beyond the bare string type in the schema, though it stops short of enumerating valid values or format constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource pair ('Explain one rule') and spells out what the explanation covers ('what it detects and how the finding is usually resolved'). The purpose is unambiguous and clearly distinct from list_rules (which lists rules) and scan_tests (which runs scans), though it never names those siblings explicitly, so differentiation relies on the reader inferring it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a workflow by telling the agent the rule id comes 'as returned by list_rules or by a finding', which subtly signals that explain_rule is the follow-up step to understanding a rule. However, it never explicitly states when to use this tool versus alternatives, nor any conditions where it should not be used. The guidance is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_rulesA

List every rule the scanner applies, with a one-paragraph description of each.

ParametersJSON Schema

Name	Required	Description	Default
No parameters

Output Schema

ParametersJSON Schema

Name	Required	Description
No output parameters

TDQS

A3.8/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden for behavioral disclosure. It states that the tool returns a list with one-paragraph descriptions, which is a behavioral outcome. However, it does not explicitly state that it is read-only, non-destructive, or what happens if there are no rules. For a simple list operation, this is adequate but not enriched—no mention of side effects, ordering, or potential errors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, front-loaded with the action and purpose. It conveys the exact scope ('every rule') and the output format ('one-paragraph description'). There is zero waste; every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with no parameters and an output schema, the description is sufficient to convey what the agent will get. It covers the full scope ('every rule') and the format of each entry. The main missing element is an explicit statement about its read-only nature, but given the simplicity and the existence of an output schema, the description is adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema already reflects that (coverage 100%). The description adds no parameter info because none are needed. Per the rubric, 0 params gives a baseline of 4, and the description does not need to compensate for any missing parameter documentation. There is nothing to clarify.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'List every rule' with a specific resource ('the scanner applies') and the nature of the content ('with a one-paragraph description of each'). It distinguishes itself from siblings: explain_rule presumably explains a single rule, and scan_tests likely performs scanning. The verb 'list' and scope 'every' make the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus the siblings (explain_rule, scan_tests). It does not mention any conditions, prerequisites, or exclusions that would help an agent decide. The agent must infer from the name and the minimal description that this is for getting an overview, but no explicit direction is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scan_testsA

Scan a file or directory for tests that cannot fail.

Reports tests that pass regardless of the behaviour of the code they claim to cover: assertions that are true by construction, tests with no assertion at all, empty bodies, and tests that are skipped while still appearing in the suite.

Python is analysed with the ast module and is exact. Rust and JavaScript/TypeScript use a brace-matching text scanner, which is a heuristic tuned to miss cases rather than invent them. Every finding carries a file and line so it can be checked directly.

Args: path: File or directory to scan. Build and dependency directories (target, node_modules, .venv, ...) are skipped automatically. include_skipped: Include tests marked skipped or ignored. These always run green because they do not run at all, but they are usually deliberate, so they are reported at info severity. max_findings: Cap on returned findings. The summary always counts every finding, including any beyond the cap.

ParametersJSON Schema

Name	Required	Description	Default
`path`	Yes
`max_findings`	No
`include_skipped`	No

Output Schema

ParametersJSON Schema

Name	Required	Description
No output parameters

TDQS

A4.6/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does so effectively: it explains what the tool detects, how detection differs across languages (exact vs. heuristic), and that every finding includes file and line. It also discloses that include_skipped findings are reported at 'info' severity. The one gap is that it does not explicitly state the tool is read-only (no file modifications), though 'scan' strongly implies it. Overall, it is far more transparent than typical tool descriptions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: the core purpose is stated in the first sentence, followed by detailed detection rules, then language-specific behavior, and finally parameter details. Each paragraph adds distinct value without repetition or fluff. The format is scannable and efficient for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, the description need not enumerate return fields, but it still provides rich context: what the tool detects, severity semantics, language-specific accuracy, and parameter behavior. An agent has all necessary information to invoke the tool correctly, choose appropriate inputs, and interpret results. It is complete for a scanning tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does. The 'Args' section fully explains each parameter: path (file/dir, auto-skipped dirs), include_skipped (why include, severity implications), and max_findings (cap behavior, summary counts all). This goes beyond the schema by providing behavioral context, making parameter semantics extremely clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Scan a file or directory for tests that cannot fail.' It then enumerates concrete categories of such tests (true-by-construction assertions, no assertions, empty bodies, skipped). The purpose is unambiguous and distinct from sibling tools list_rules and explain_rule, which operate on rules rather than tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clarifies when to use the tool: it is for scanning files/directories for non-failing tests, and it explicitly notes that build/dependency directories are auto-skipped. It also distinguishes Python analysis (exact via ast) from Rust/JS (heuristic), helping agents set expectations. While it does not name alternative tools for comparison (the siblings are clearly rule-focused, not test scanners), the use case is sufficiently well-scoped. No explicit when-not-to-use guidance, but this is minor given the clear domain.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

3 tool updatesv0.1.0Aug 27, 2026
- First observedexplain_rule
- First observedlist_rules
- First observedscan_tests

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.2/5.0

Scored 2026-08-27 10:50 across 3 tools

Disambiguation5/5

Each tool has a distinct purpose: listing available rules, explaining a specific rule, and scanning tests for vacuous assertions. There is no overlap between scan_tests and the two rule-related tools, and list_rules/explain_rule are clearly separated by verb (list vs. explain).

Naming Consistency5/5

All tools follow a consistent snake_case verb_noun pattern: list_rules, explain_rule, scan_tests. The naming is predictable and aligns perfectly with the tool's action.

Tool Count5/5

With 3 tools, the server is well-scoped for its purpose—a test scanner with rule documentation. Each tool earns its place, and the count falls squarely within the typical 3-15 range for a focused server.

Completeness5/5

The tool surface fully covers the intended workflow: discover rules, understand a rule, and run the scanner. There are no obvious gaps—the scanner's options are handled via arguments rather than requiring additional tools, and the rule lifecycle (list/explain) is complete for a read-only server.

Maintenance

ActivitySlowing

ResponsivenessNo issues

Resources

GitHub Repository

Need Help?

Related Servers

Related MCP Connectors

DevOps & Cloud Code Auditor
io.github.Ansarii
Detects database migration table locks, terraform cost leaks, and OWASP API flaws.
Cloud Platforms Code Analysis Security
1
Trunk Flaky TestsOAuth
io.trunk
Flaky test detection, root cause analysis, and fix suggestions for development teams.
CI/CD & DevOps Testing & QA Tools Autonomous Agents
2
HefestoAI A
io.github.artvepa80
Pre-commit code quality guardian. Detects semantic drift in AI-generated code.
Code Analysis Testing & QA Tools Developer Tools
421
spectraOAuth
io.github.jorgesandoval
Generates unit tests for Python code with coverage before/after reports and concrete edge cases.
Testing & QA Tools Code Analysis
2

Related MCP Servers

OrangePro MCPofficial
Code Analysis Testing & QA Tools Developer Tools
OrangeproAI
A
license
A
quality
A
maintenance
Analyzes code to map behaviors, identify untested gaps, and generate grounded integration tests that actually run.
Updated 3 days ago (2026-10-10 21:24 UTC)
15
1,364 npm
18
MIT
openhonest-umbraofficial
Developer Tools Code Analysis Testing & QA Tools
openhonest
A
license
A
quality
A
maintenance
Audits a module against its test suite and reports the cases the tests are structurally unable to see, then proves each gap with a real failing test rather than a warning. Covers Python, TypeScript, JavaScript, Java, Rust and Go.
Updated 2 months ago (2026-08-12 16:29 UTC)
4
3
Apache 2.0
faultlineofficial
Testing & QA Tools Code Analysis Developer Tools
Energixe
A
license
Not graded
quality
B
maintenance
Generates complete runnable adversarial test suites for your code, predicting the bugs they expose with exact code evidence, across multiple languages and test frameworks.
Updated a month ago (2026-09-17 05:35 UTC)
MIT
TestHeal
Testing & QA Tools Code Analysis Developer Tools
webscout9-png
A
license
A
quality
B
maintenance
Provides AI coding agents with reliable root-cause diagnosis and minimal, high-confidence fixes for failing tests.
Updated 24 days ago (2026-09-19 16:23 UTC)
3
1
MIT

vacuous-tests-mcp

Случай, на основе которого это было создано

Правила

Языки

Установка

Использование из MCP-клиента

Инструменты

Разработка

Ограничения

Лицензия

Available Tools

Output Schema

TDQS

Output Schema

TDQS

Output Schema

TDQS

Tool Schema Changelog

TDQS

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

OrangePro MCPofficial

openhonest-umbraofficial

faultlineofficial

TestHeal