Skip to main content
Glama

MCP Failure Lab

npm version CI

Набор инструментов для хаос-инжиниринга и тестирования отказоустойчивости серверов Model Context Protocol.

Документация · Страница проекта

MCP Failure Lab, демонстрирующий ограниченную задержку и ожидаемый тайм-аут

Быстрый старт

Запустите реальный детерминированный сценарий задержки без клонирования репозитория или глобальной установки пакета:

npx mcp-failure-lab demo

Пример вывода:

MCP Failure Lab — Demo
Running a real 500ms delay scenario...

Scenario: Deterministic delay demo
Outcome: success
Duration: ~500 ms
Assertions: passed

Точная длительность может незначительно отличаться между запусками. API-ключ или внешний MCP-сервер не требуются.

Покажите доступные команды:

npx mcp-failure-lab --help

Запустите встроенный MCP-сервер через stdio:

npx mcp-failure-lab serve

Related MCP server: mcp-chaos-rig

Назначение

MCP Failure Lab помогает авторам серверов воспроизводить задержки, зависающие инструменты, отмену и потерю транспорта детерминированным образом.

Он обеспечивает контролируемое поведение при сбоях для тестирования обработки тайм-аутов, очистки при отмене, восстановления после потери транспорта, утверждений и результатов CI.

Текущая область применения

MCP Failure Lab выполняет детерминированные JSON-сценарии против собственного встроенного MCP-сервера из командной строки.

Доступно сейчас:

  • инструменты ping, delay, hang и disconnect

  • MCP-взаимодействие через stdio

  • определения сценариев в виде кода и JSON

  • утверждения о результате и максимальной длительности

  • утверждения о результатах MCP

  • последовательные вызовы наблюдателя для проверки постусловий

  • вывод в консоль и JSON

  • машиночитаемые ошибки команд

  • коды выхода, удобные для CI

  • модульные, интеграционные и сквозные тесты

Не реализовано:

  • оркестрация внешних MCP-клиентов

  • адаптеры целевых клиентов

  • Streamable HTTP

  • отчёты JUnit

  • сбои из-за некорректных сообщений, дублирующихся ответов и потери сеанса

MCP Failure Lab в настоящее время не является универсальным прокси или оркестратором тестов внешних MCP-клиентов.

Как это работает

MCP Failure Lab выполняет детерминированные сценарии через встроенные MCP-клиент и сервер. Сценарий вызывает ping, delay, hang или disconnect, записывает наблюдаемый результат и длительность и оценивает объявленные ожидания.

Необязательные вызовы наблюдателя выполняются последовательно через то же соединение MCP-клиента для проверки постусловий через отдельный путь инструмента.

См. документацию по архитектуре для диаграмм, обязанностей и границ реализации.

Документация

Полные руководства и справочники доступны на mcplab.dev/docs.

Требования

  • Node.js 22.19.0 или новее

  • npm

Совместимость протоколов

MCP Failure Lab по умолчанию нацелен на MCP 2026-07-28. Его CLI-сервер использует точку входа stdio с учётом эпохи из SDK v2, а встроенный клиент сценариев закрепляет 2026-07-28, чтобы современное поведение проверялось явно.

Сервер также принимает поток инициализации 2025-11-25 для совместимости. Этот устаревший путь по-прежнему покрыт интеграционными тестами, но новая разработка нацелена на 2026-07-28. Существующие инструменты сбоев ping, delay, hang и disconnect имеют одинаковое пользовательское поведение в обеих эпохах; функции протокола, которые полагаются на запросы, инициируемые сервером, различаются между эпохами и выходят за рамки этих инструментов сбоев.

Установка

Запустите пакет напрямую с помощью npx:

npx mcp-failure-lab demo

Глобальная установка не требуется.

Чтобы установить команду глобально:

npm install -g mcp-failure-lab

CLI

# Run the built-in demonstration
npx mcp-failure-lab demo

# Display command help
npx mcp-failure-lab --help

# Display the installed version
npx mcp-failure-lab --version

# Start the MCP server over stdio
npx mcp-failure-lab serve

Процесс serve ожидает MCP-клиента. Нажмите Ctrl+C для корректного завершения.

Запуск сценария

Файлы сценариев используют JSON:

{
  "name": "bounded delay succeeds",
  "call": {
    "tool": "delay",
    "args": {
      "delayMs": 250
    }
  },
  "timeoutMs": 1000,
  "expect": {
    "outcome": "success",
    "maxDurationMs": 500
  }
}

Из клона репозитория запустите включённый сценарий:

npm run dev -- run examples/scenarios/delay-success.json

Сгенерируйте машиночитаемый вывод:

npm run dev -- run examples/scenarios/delay-success.json --report json

Команда завершается с кодом:

Код

Значение

0

Все ожидания выполнены

1

Сценарий не удалось загрузить или выполнить

2

Одно или несколько утверждений не выполнены

Для утверждений о результатах, вызовов наблюдателя, форматов отчётов и поведения тайм-аутов см. документацию по сценариям и отчётности.

Инструменты сбоев

Инструмент

Поведение

ping

Возвращает детерминированный ответ о состоянии

delay

Ожидает ограниченную длительность перед возвратом

hang

Остаётся в ожидании, пока клиент не отменит

disconnect

Прерывает активный транспорт, пока запрос находится в полёте

См. справочник по инструментам сбоев для аргументов и поведения.

Инспектирование сервера

Запустите MCP Inspector для опубликованного пакета:

npx @modelcontextprotocol/inspector npx mcp-failure-lab serve

Подключитесь через stdio, перечислите доступные инструменты и вызовите ping, delay, hang или disconnect.

Не делитесь и не фиксируйте временные токены аутентификации, включённые в URL-адреса Inspector.

Внешняя проверка интеграции

MCP Failure Lab был независимо проверен с помощью симуляции Future AGI с использованием независимого Python MCP-клиента. Эксперимент вызывал реальный инструмент hang через stdio и применял тайм-аут на стороне клиента перед оценкой смоделированных ответов агента.

Это пример внешней проверки, а не официальная интеграция или одобрение Future AGI.

См. пример Future AGI для результатов и шагов воспроизведения.

Разработка

Клонируйте репозиторий и установите зависимости:

git clone https://github.com/anilloutombam/mcp-failure-lab.git
cd mcp-failure-lab
npm install

Запустите CLI для разработки:

npm run dev -- --help

Перед открытием pull request выполните:

npm run format:check
npm run typecheck
npm test
npm run build

См. CONTRIBUTING.md для описания процесса внесения вклада.

Дорожная карта

Планируемые работы отслеживаются в GitHub Issues.

Элементы дорожной карты не являются частью текущей реализации, если явно не задокументированы как доступные.

Лицензия

MIT

Available Tools

10 tools
delayA

Delay a successful MCP response to test client timeout behavior.

NameRequiredDescriptionDefault
delayMsYesResponse delay in milliseconds.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does disclose the key behavioral trait: the response is delayed but still successful, which meaningfully separates it from 'hang' (no response) and 'disconnect' (dropped connection). It adds no information about blocking behavior, the 30s ceiling, or whether the call is cancellable mid-delay.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence that front-loads the action and ends with the purpose. No filler, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter test utility with no output schema, the description covers the essentials: what is delayed and why. It would be fully complete if it noted that the call blocks the connection for the given duration and that the schema-bound maximum is 30 seconds.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter 'delayMs' is fully documented with units, minimum, and maximum in the schema. The description adds no syntax or semantics beyond what the schema already provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Delay a successful MCP response') plus the intent ('test client timeout behavior'), so the agent knows exactly what the tool does. It does not distinguish itself from siblings like 'hang' or 'disconnect', which is close in function, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies its use case ('test client timeout behavior'), which is enough to infer that this is for fault-injection testing rather than production calls. However, it never says when to prefer this over 'hang', 'disconnect', or 'protocol_ping_liveness', leaving the sibling selection to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

disconnectA

Close the MCP transport before this request can receive a response.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose the key behavioral trait: the transport is torn down before a response can be delivered, so the caller will not get a normal reply. It stops short of saying whether the closure is permanent, whether reconnection is possible, or what error surfaces to the caller.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with the action verb first and only the timing detail that matters. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless tool with no annotations and no output schema, the description conveys the one thing an agent must know: no response will arrive. Minor gaps remain around session persistence and recoverability, but nothing essential to invoking it is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and the schema is empty, so there is nothing for the description to disambiguate. Baseline 4 applies for a 0-param tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (close) and resource (the MCP transport), which unambiguously separates it from siblings like ping, delay, hang, and malformed_message that manipulate the session in other ways. An agent can tell what this does without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied through the phrase 'before this request can receive a response', which signals it is a failure-injection/test tool, but it never says when to pick disconnect over hang, delay, or the other siblings. No exclusions or prerequisites are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

duplicate_responseB

Return the same JSON-RPC response twice for this request.

NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the core behavior (duplicating the response) but omits operational details such as whether duplication is one-time or persistent, and whether it affects only this request or future ones.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no wasted words. Appropriate for a zero-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a protocol-testing tool with no annotations and no output schema, the description is minimally viable but leaves important contextual gaps, such as when to choose this tool over siblings and whether the duplication has side effects on the connection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the schema is empty and there is nothing for the description to clarify. Baseline for zero-parameter tools is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and behavior: return the same JSON-RPC response twice for the request. It is clear what the tool does, but it does not differentiate itself from siblings like response_after_cancellation or malformed_message.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. The agent must infer its role in the testing suite.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hangB

Never respond unless the MCP request is cancelled.

NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the core behavior (no response until cancellation) but omits what happens after cancellation (e.g., error, silent stop) and any side effects or auth requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with the key behavior, and no wasted words. Appropriately sized for a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-param testing tool, the core behavior is stated, but missing usage context and post-cancellation behavior leaves the agent without full operational clarity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so baseline is 4. The empty schema requires no additional parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific behavior (never respond) and a condition (unless cancelled), which distinguishes it from siblings like delay or disconnect. However, it does not explicitly name alternatives or scope, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus siblings such as delay, disconnect, or malformed_message. The agent must infer its role in testing timeouts or cancellation scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

invalid_progressA

Send one bounded progress-notification fault for this request; requires a client progress token.

NameRequiredDescriptionDefault
variantYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose that the fault is 'one' and 'bounded' — i.e., a single, non-continuous emission — plus the token prerequisite. It does not say what the peer is expected to do in response, whether the fault is terminal for the request, or any side effects, so the behavioral picture is only partial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the action front-loaded and the prerequisite trailing; no filler, nothing redundant, and it is the right length for a one-parameter fault injector.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple fault-injection tool the description covers the core action and precondition, which is close to sufficient. But with no annotations, no output schema, and an unexplained variant enum, the agent still lacks the outcome semantics (what the fault produces, and whether it aborts the request) needed to use it confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one enum parameter at 0% schema description coverage, so the description should compensate; instead it explains the prerequisite ('client progress token') but never clarifies what 'unknown-token', 'regressing-progress', or 'after-completion' actually emit. The enum names are fairly self-describing, which keeps this at an adequate-but-incomplete 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Send one bounded progress-notification fault for this request.' An agent can tell it is a fault-injection primitive against the progress-notification channel, but the description never names or contrasts itself with siblings such as malformed_message or duplicate_response, which also emit protocol faults.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives one useful precondition — 'requires a client progress token' — implying the caller must already have an in-flight request with a token. However, it offers no when/when-not guidance and no routing among the many sibling fault injectors, so selection between them is left to inference from names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

malformed_messageC

Return one deterministic response that violates a selected JSON-RPC rule.

NameRequiredDescriptionDefault
variantYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. "Deterministic" is a useful signal, but it never states whether the malformed frame corrupts or terminates the session, whether subsequent calls are affected, or how the client is expected to observe the violation. For a fault-injection tool, those side-effect semantics are the key information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. It is efficient, though the brevity is partly under-specification rather than disciplined conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a fault-injection tool with no annotations, no output schema, and an undocumented enum, the definition is too thin: it omits what the response looks like, the side effects on session state, and how each variant maps to a violated rule. An agent can invoke it but cannot predict the consequences.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and there is one required enum parameter, so the description should have compensated. It does not list or explain the variant values, though "selected JSON-RPC rule" gestures at the enum and the values (missing-jsonrpc, invalid-jsonrpc-version, result-with-error) are largely self-describing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete verb and output ("Return one deterministic response") and a concrete domain constraint ("violates a selected JSON-RPC rule"), which is enough to distinguish it from fault-injection siblings like duplicate_response or disconnect. It does not explicitly name those siblings, but the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no statement of prerequisites, and no routing to alternatives. An agent must infer that this belongs in protocol-conformance testing and that siblings cover the other fault classes.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

oversized_responseA

Return a deterministic text payload of exactly sizeBytes UTF-8 bytes (maximum 256 KiB).

NameRequiredDescriptionDefault
sizeBytesYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and does disclose key traits: deterministic output, exact size in UTF-8 bytes, and a maximum of 256 KiB. It does not mention error handling for invalid input or any side-effect profile, but for a simple payload generator this is fairly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence with no wasted words. The core behavior is front-loaded and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no output schema or annotations, the description is nearly complete for invocation: it defines the output type, exact size, and cap. It lacks context for why or when an agent should choose this tool over its many siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single sizeBytes parameter, so the description must compensate. It does so by clarifying that sizeBytes is measured in UTF-8 bytes and capped at 256 KiB, adding meaningful semantic detail beyond the schema's integer type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Return') and resource ('deterministic text payload of exactly sizeBytes UTF-8 bytes') with clear scope. It is distinct from siblings like delay or hang, but does not explicitly differentiate itself from them or name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as duplicate_response or malformed_message. The description only states what the tool does, leaving context entirely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pingB

Check whether the MCP server is responsive.

NameRequiredDescriptionDefault

No parameters

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full burden, but it does convey that the operation is a non-destructive responsiveness probe. It does not say what a response looks like, whether it blocks/times out, or whether a failure is surfaced as an error versus a negative result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short, front-loaded sentence with no filler. It is appropriately sized for a trivial probe, though it leaves room to add a clause distinguishing it from the similarly named sibling.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-param, no-output-schema liveness probe, the description covers the essential purpose but omits the result semantics (what 'responsive' returns) and the sibling disambiguation needed in this crowded sibling set. Adequate but with clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate — the baseline of 4 applies. The description correctly implies a no-argument invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Check') and the target resource ('the MCP server') plus the intended signal (responsiveness), so the core purpose is unambiguous. However, it does not distinguish itself from the closely-named sibling 'protocol_ping_liveness', which an agent could easily confuse with this tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance is given, and no conditions for choosing this over 'protocol_ping_liveness' or any of the fault-injection siblings (delay, hang, disconnect) are provided. The agent must infer that this is the plain liveness check purely from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

protocol_ping_livenessC

Send an MCP protocol ping during this in-flight call and apply a bounded liveness policy.

NameRequiredDescriptionDefault
pingAfterMsYes
closeOnFailureNo
completionDelayMsNo
livenessTimeoutMsYes

TDQS

C2.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It mentions a 'bounded liveness policy' but never defines the bounds, the failure behavior, the effect of closeOnFailure, the role of completionDelayMs, or any side effects, leaving the tool's runtime behavior largely opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single front-loaded sentence with no filler, which is structurally clean. For a four-parameter tool with no annotations or output schema, however, this level of brevity reads as under-specification rather than appropriate conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the four parameters, zero schema description coverage, no annotations, and no output schema, the one-sentence description is far too thin. It omits parameter semantics, timeout behavior, failure handling, and sibling differentiation, so an agent cannot safely invoke the tool from the description alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for four parameters, and the description does not mention pingAfterMs, livenessTimeoutMs, closeOnFailure, or completionDelayMs at all. It therefore adds no meaning beyond the bare schema and fails to compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The sentence names a specific verb and resource ('Send an MCP protocol ping') and adds a scope ('during this in-flight call'), so it is not pure tautology. However, 'apply a bounded liveness policy' is vague, and the description never distinguishes this tool from the sibling 'ping' or explains what the liveness policy actually does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use, when-not-to-use, or alternative guidance is given. The phrase 'during this in-flight call' hints at a use context, but with siblings like ping, hang, and disconnect available, the agent receives no routing information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

response_after_cancellationA

Send a response after this call is cancelled (stdio only).

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the key behavior of sending a response after cancellation and restricts it to stdio, which are non-obvious traits, but it does not detail what response is sent or error behavior outside stdio.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no wasted words, covering action, timing, and transport constraint efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, zero-parameter test tool with no output schema or annotations, the description gives enough context to understand what it does and its transport limitation. It could optionally clarify the nature of the response or side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there are no parameter semantics to document. This meets the baseline of 4 for a no-param tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action (send a response) and timing condition (after cancellation), and adds a transport restriction (stdio only). It is clear, though it does not explicitly contrast itself with sibling tools like duplicate_response or disconnect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The stdio-only constraint implies usage context, but the description does not state when to use this tool versus alternatives or provide explicit preconditions. Usage must be inferred from the name and description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.13.0
    • Addedinvalid_progress
    • Addedoversized_response
  2. 8 tool updatesv0.12.0
    • First observeddelay
    • First observeddisconnect
    • First observedduplicate_response
    • First observedhang
    • First observedmalformed_message
    • First observedping
    • First observedprotocol_ping_liveness
    • First observedresponse_after_cancellation

TDQS

B3.3/5.0

Scored across 10 tools

Disambiguation4/5

Most tools target clearly distinct failure modes such as delay, disconnect, hang, malformed_message, and duplicate_response. However, ping and protocol_ping_liveness both concern liveness checking and could be confused, keeping this from a perfect score.

Naming Consistency4/5

All tool names use consistent snake_case and are descriptive. The main deviation is grammatical style: some are bare verbs (delay, disconnect, hang, ping) while others are noun phrases (oversized_response, malformed_message).

Tool Count5/5

Ten tools is well-scoped for a failure-injection lab. Each tool represents a distinct test scenario and none feels redundant or excessive.

Completeness4/5

The set covers core transport, protocol, timing, and cancellation failure modes, including oversized responses, malformed messages, duplicates, and post-cancellation responses. It lacks explicit injection of JSON-RPC error responses or invalid request IDs, which are common client-handling tests.

Maintenance

ActivityActive
ResponsivenessWithin a week

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    A comprehensive reference implementation demonstrating all features of the Model Context Protocol (MCP) specification, serving as documentation, learning resource, and testing tool for MCP implementations.
    1
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    A local MCP server that breaks on demand, allowing you to test your client against auth failures, disappearing tools, flaky responses, and token expiry from a web UI.
    7 npm
    10
    MIT