mcp-protocol-conformance
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-protocol-conformancerun full conformance suite for my MCP server"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Part of the StudioMeyer MCP Stack โ Built in Mallorca ๐ด ยท โญ if you use it
mcp-protocol-conformance
This is a Foundation build of the StudioMeyer MCP Factory: every other Factory build runs through this harness before promotion (npm publish, marketplace submit, upstream PR).
A note from us
We have been building tools and systems for ourselves for the past two years. The fact that this repo is small and has few stars is not because it is new. It is because we only just decided to share what we have built. It is not a fresh experiment, it is a long story with a recent commit.
We love building things and sharing them. We do not love social media tactics, growth hacks, or chasing stars and followers. So this repo is small. The code is real, it gets used, issues get answered. Judge for yourself.
If it helps you, sharing, testing, and feedback help us. If it could be better, an issue is more useful. If you build something with it, tell us at hello@studiomeyer.io. That genuinely makes our day.
From a small studio in Palma de Mallorca.
Related MCP server: MCP Test Server
Install
npm install --save-dev mcp-protocol-conformanceThe package ships both a CLI (mcp-conformance) and a TypeScript library entry (import ... from "mcp-protocol-conformance").
CLI usage
Run against a stdio server
mcp-conformance run \
--target stdio \
--cmd node \
--cmd-arg dist/server.js \
--spec 2025-06-18 \
--suite all \
--format terminalRun against an HTTP server (Streamable HTTP, 2025-03-26+)
mcp-conformance run \
--target http \
--url https://memory.studiomeyer.io/mcp \
--header "Authorization:Bearer ${TOKEN}" \
--spec 2025-06-18 \
--suite full \
--format json --out report.jsonRun only a subset of suites
mcp-conformance run --target stdio --cmd ./server --spec 2025-06-18 \
--suite jsonrpc,version,schemaCompare two manifests
mcp-conformance compare \
--expected manifests/v1.json \
--actual manifests/v2.jsonAssert no breaking changes between two reports
mcp-conformance assert-no-breaking \
--baseline reports/main.json \
--current reports/pr-42.jsonExit codes: 0 clean, 1 failures, 2 invocation error.
Library usage
import {
runFullSuite,
generateReport,
} from "mcp-protocol-conformance";
const report = await runFullSuite(
{ kind: "stdio", cmd: "node", args: ["dist/server.js"] },
"2025-06-18",
{ suite: "all" },
);
console.log(generateReport(report, "terminal"));
if (report.status === "fail") process.exit(1);MCP server usage
The harness is itself an MCP server. Start it over stdio and any MCP client can call its 12 tools:
node dist/server.jsTools (all read-only, all destructiveHint: false):
# | Tool | Purpose |
1 |
| JSON-RPC 2.0 error-code matrix + response-envelope (result/error mutual exclusivity) |
2 |
| Verify advertised protocolVersion |
3 |
| Transport-layer ping + session-id |
4 |
| OAuth 2.1 PKCE S256 (mock-AS or real-tenant) |
5 |
| inputSchema (+ 2025-11-25 outputSchema / title) is valid JSON-Schema |
6 |
| initialize.capabilities matches behaviour (+ 2025-11-25 tasks) |
7 |
| One tools/call per advertised tool |
8 |
| readOnlyHint / destructiveHint hygiene |
9 |
| All suites + summary |
10 |
| Render JUnit / JSON / terminal |
11 |
| Diff two tool manifests |
12 |
| Diff two FullReports |
Compatibility matrix
2024-11-05 | 2025-03-26 | 2025-06-18 | 2025-11-25 | |
jsonrpc | yes | yes | yes | yes |
version (handshake) | yes | yes | yes | yes |
transport (stdio) | yes | yes | yes | yes |
transport (http) | n/a | yes | yes | yes |
oauth (mock) | n/a | yes | yes | yes |
oauth (real) | n/a | yes | yes | yes |
schema | yes | yes | yes | yes |
capability | yes | yes | yes | yes |
smoke | yes | yes | yes | yes |
annotations | warn-only | warn-only | yes | yes |
tasks (capability) | n/a | n/a | n/a | yes |
tool title/output | n/a | n/a | n/a | warn-only |
yes = suite runs and produces actionable results.
n/a = capability not in spec; suite skips automatically.
warn-only = suite runs but the spec does not formally require the feature.
The 2026-07-28 RC (stateless core, SEP-2575/2567) is intentionally not here โ it ships final on 2026-07-28; conformance for it lands in a later release against the final SDK.
Integration in Factory builds
In every Factory build's package.json:
{
"scripts": {
"factory:conformance": "mcp-conformance run --target stdio --cmd 'node dist/server.js' --spec 2025-06-18 --suite full"
}
}In CI:
- run: npm run factory:conformanceSpec references
JSON-RPC 2.0: https://www.jsonrpc.org/specification
MCP 2024-11-05: https://modelcontextprotocol.io/specification/2024-11-05
MCP 2025-03-26: https://modelcontextprotocol.io/specification/2025-03-26
MCP 2025-06-18: https://modelcontextprotocol.io/specification/2025-06-18
MCP 2025-11-25: https://modelcontextprotocol.io/specification/2025-11-25
RFC 7636 (PKCE): https://datatracker.ietf.org/doc/html/rfc7636
About StudioMeyer
StudioMeyer is an AI and design studio based in Palma de Mallorca, working with clients worldwide. We build custom websites and AI infrastructure for small and medium businesses. Production stack on Claude Agent SDK, MCP and n8n, with Sentry, Langfuse and LangGraph for observability and an in-house guard layer.
License
MIT โ Copyright (c) 2026 Matthias Meyer (StudioMeyer)
Available Tools
12 toolsassertNoBreakingChangesARead-onlyIdempotent
Compare a baseline FullReport against a current FullReport. Returns a list of breaking changes and warnings.
| Name | Required | Description | Default |
|---|---|---|---|
| current | No | ||
| baseline | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive. Description adds that it returns a list of breaking changes and warnings, but no additional behavioral context like side effects or request patterns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single clear sentence conveying action, inputs, and output without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, describing return type helps. However, lacking references to FullReport origin (e.g., from generateReport) and not explaining what constitutes a breaking change leaves some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description names the two parameters as 'baseline FullReport' and 'current FullReport', providing basic semantic meaning that the schema lacks. However, no details on FullReport structure are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares two FullReport objects and returns breaking changes and warnings. It uses specific verbs and nouns, distinguishing it from siblings like compareManifests.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., compareManifests). No preconditions or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compareManifestsBRead-onlyIdempotent
Diff two tool manifests โ added/removed/changed tools.
| Name | Required | Description | Default |
|---|---|---|---|
| actual | Yes | ||
| expected | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint and idempotentHint are true, so the description doesn't need to repeat that. However, it adds no additional behavioral context (e.g., permissions, side effects, or what 'changed' means). Neutral value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise single sentence that frontloads the core purpose. Every word earns its place. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fails to specify the output format (diff result), how 'changed' is determined, or any details about the nested object structure. For a diff tool with no output schema, this is insufficient for an agent to understand the return value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the two parameters ('expected' and 'actual'). It only hints at two tool manifests but gives no detail about expected format, which is critical given both are complex nested objects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it diffs two tool manifests and identifies added/removed/changed tools. It uses a specific verb ('diff') and resource ('tool manifests'), and it distinguishes from sibling tools that focus on running compliance tests.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. No mention of prerequisites, context, or when not to use it. The description provides no usage direction beyond stating the basic function.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generateReportBRead-onlyIdempotent
Render a FullReport as JUnit XML, JSON, or human-readable terminal output.
| Name | Required | Description | Default |
|---|---|---|---|
| format | Yes | ||
| report | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, establishing safety. The description adds format output options (beyond annotations) but does not elaborate on behavior like handling of missing report fields or performance characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence long, direct, and contains no extraneous information. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 params, no output schema) and presence of annotations, the description minimally covers the tool's purpose and output formats, but lacks detail on input parameters and edge cases, making it adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It hints at the 'format' parameter values by listing output types, but it does not describe the 'report' parameter at all (type, structure, or constraints). This is insufficient for a 2-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Render'), identifies the resource ('FullReport'), and lists all output formats (JUnit XML, JSON, terminal), clearly distinguishing it from sibling tools that run tests or perform assertions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., other report tools or the test-running siblings). It only states what it does, leaving usage context implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runAnnotationsAuditBRead-onlyIdempotent
Apply the heuristic annotation rules (destructive name regex, readOnlyHint conflicts) to every advertised tool.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| specVersion | No | 2025-06-18 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive. The description adds specifics about applying heuristic rules (destructive name regex, readOnlyHint conflicts), providing behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loads the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description adequately conveys the core functionality but omits output details (no output schema) and does not explain how results are returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no explanation of the parameters. The complex 'target' parameter and optional 'specVersion' are entirely undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it applies heuristic annotation rules to advertised tools, using a specific verb and resource. It differentiates from siblings by focusing specifically on annotation rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no exclusions, and no context on prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runCapabilityIntrospectionARead-onlyIdempotent
Compare the capabilities the target advertises in initialize against what tools/list, resources/list, prompts/list actually return.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| specVersion | No | 2025-06-18 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive traits. The description adds the behavior of comparing advertised vs actual capabilities, which is consistent but not extensive beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no filler, effectively front-loading the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, no parameter descriptions, and 0% schema coverage, the description omits critical details about parameter usage and return values, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description does not explain any parameters (target, specVersion). The agent must infer from the schema alone, which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Compare') and resources ('capabilities... initialize... tools/list, resources/list, prompts/list'), and it distinguishes from sibling tools that perform other tests or assertions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the tool does but does not explicitly state when to use it versus alternatives like runJsonRpcCompliance or runFullSuite. Usage context is implied but not clearly delineated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runFullSuiteBRead-onlyIdempotent
Run all (or selected) suites and produce a FullReport with summary + per-suite + per-check status.
| Name | Required | Description | Default |
|---|---|---|---|
| oauth | No | ||
| suite | No | ||
| target | Yes | ||
| specVersion | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly, idempotent, and non-destructive behavior. The description adds that it produces a 'FullReport' with structure, but does not disclose details like error handling, timeout, or dependency on server. It is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and contains no unnecessary words. It efficiently conveys the main action and output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 parameters, no output schema, no param descriptions), the description is insufficient. It fails to explain key parameters like target or specVersion, and does not mention the tool's read-only nature or any prerequisites. The description leaves critical gaps for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, yet the description provides no explanation of any parameters (target, specVersion, oauth, suite). It only loosely references 'suites' but adds no meaning beyond the schema names and enums. This is a severe gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs 'all (or selected) suites' and produces a 'FullReport', which distinguishes it from sibling tools that run specific suites (e.g., runJsonRpcCompliance, runTransportSuite). The verb 'Run' and resource 'suites' are specific, and the scope is clarified.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus its siblings. It implies it is for comprehensive testing via 'all (or selected) suites', but lacks guidance on when to choose specific suite tools or when this is appropriate (e.g., for CI vs debugging).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runJsonRpcComplianceBRead-onlyIdempotent
Run JSON-RPC 2.0 compliance suite (parse-error, invalid-request, method-not-found, invalid-params, internal-error) against a target MCP server.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| specVersion | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, indicating safe operations. Description adds the list of error cases but does not disclose additional behavioral traits like execution duration, prerequisites (e.g., server must be running), or error handling beyond the listed cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the main purpose, but it omits critical parameter details. It is efficient but incomplete, which reduces its helpfulness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 2 parameters (one complex) and no output schema, the description provides no insight into return values, success/failure indicators, or side effects. It is insufficient for an agent to fully understand the tool's behavior without additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description provides no explanation of the complex 'target' parameter (stdio vs http connection) or the 'specVersion' enum. The agent must rely entirely on the JSON schema, which is detailed but lacks natural language guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly specifies the tool runs a JSON-RPC 2.0 compliance suite targeting an MCP server, listing specific error cases. It distinguishes itself from sibling tools (e.g., runTransportSuite, runToolSchemaValidation) which cover other aspects of testing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied by the tool name and description but no explicit guidance on when to use versus alternatives, nor any prerequisites or exclusions. The sibling tools are different test suites, so the purpose is clear but not comparative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runOauthPkceFlowBRead-onlyIdempotent
Exercise OAuth 2.1 PKCE S256 flow against the target. Mock-AS by default; real-tenant when --oauth-mode=real.
| Name | Required | Description | Default |
|---|---|---|---|
| scopes | No | ||
| target | Yes | ||
| clientId | Yes | ||
| redirectUri | Yes | ||
| specVersion | No | 2025-06-18 | |
| authorizationServerUrl | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds the behavioral nuance of default mock-AS mode and real-tenant mode, but does not elaborate on other traits like side effects or authentication needs beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences, with the core purpose in the first sentence and a mode distinction in the second. There is no redundancy or superfluous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite moderate complexity (6 parameters, nested objects, no output schema), the description covers only the high-level flow and a mode switch. It lacks information on return values, error handling, and parameter roles, making it insufficient for complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must explain parameter meanings. However, it mentions no parameters at all; terms like 'target', 'clientId', and 'redirectUri' are left undefined. This is a critical gap for an agent to invoke the tool correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as exercising an OAuth 2.1 PKCE S256 flow against a target, with distinct modes (mock vs. real). This verb+resource+mode combination is specific and differentiates it from sibling tools like runJsonRpcCompliance or runTransportSuite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description only mentions a flag to switch between mock and real tenant, but provides no explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, contexts, or exclusions relative to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runRoundtripSmokeBRead-onlyIdempotent
Perform one tools/call per advertised tool. Fails on JSON-RPC transport errors, warns on tool-level isError=true.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| sampleArgs | No | ||
| specVersion | No | 2025-06-18 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark it as read-only and idempotent. The description adds behavioral details: fails on transport errors, warns on tool-level errors. This goes beyond what annotations provide, giving the agent a clear picture of failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no extraneous words. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 params, nested objects, no output schema), the description omits important context: how sampleArgs affects tool calls, what the return value looks like, and that it requires a running server. Incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate for parameter explanation. It does not mention any parameters (target, sampleArgs, specVersion), leaving their meaning and usage entirely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs one tools/call per advertised tool, making it a smoke test. It distinguishes from sibling testing tools by specifying its lightweight roundtrip nature, though could be more explicit about its narrow scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like runFullSuite or runToolSchemaValidation. The description does not mention prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runSpecVersionAssertionARead-onlyIdempotent
Drive an initialize handshake and assert the target advertises the expected MCP spec version.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| expectedVersion | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, indicating a safe read operation. The description adds that the tool performs a handshake (implying network interaction) and assertion, which provides some behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundancy or unnecessary detail. Every word contributes to the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should mention return values or result format. It also lacks context about prerequisites (e.g., target must be a running MCP server). While adequate for a simple test tool, there are gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description partially compensates by stating that expectedVersion is the version to assert and target is the server to test. However, it does not explain the target structure (stdio vs http) beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool drives an initialize handshake and asserts the target advertises the expected MCP spec version. This specific verb+resource structure distinguishes it from sibling test tools like runJsonRpcCompliance or runTransportSuite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., runJsonRpcCompliance or runToolSchemaValidation) is provided. The description implies use for spec version testing but lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runToolSchemaValidationARead-onlyIdempotent
Validate that every advertised tool has a well-formed JSON-Schema inputSchema. Optionally diff against an expected manifest.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| specVersion | No | 2025-06-18 | |
| expectedManifest | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, indicating no side effects. The description ('validate', 'diff') is consistent but adds no new behavioral traits beyond what annotations provide. No contradiction, but no extra transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose, no redundant information. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's complexity (3 parameters, nested objects, no output schema, no schema descriptions), the description is minimal. It fails to explain the critical target parameter or how the tool connects to a server, making it incomplete for an AI agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the three parameters (target, specVersion, expectedManifest). It does not mention any of them, leaving the AI agent without guidance on what 'target' (a complex union), 'specVersion', or 'expectedManifest' mean or how to use them. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool's purpose: validating that advertised tools have well-formed JSON-Schema inputSchema, with an optional diff against an expected manifest. The verb 'validate' and resource 'tool inputSchemas' are explicit, and it distinguishes from sibling tools like runJsonRpcCompliance or runSpecVersionAssertion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the primary use case is schema validation and mentions the optional diff feature. However, it does not provide explicit guidance on when to use this tool versus alternative siblings, nor does it state when not to use it. The context is clear but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runTransportSuiteCRead-onlyIdempotent
Verify the target's transport layer (stdio line-delimited JSON or Streamable HTTP).
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | ||
| transport | No | both | |
| specVersion | No | 2025-06-18 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows it's a safe read operation. The description adds that it 'verifies' the transport layer, which is consistent. No additional behavioral details (e.g., side effects, authorization needs) are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence and front-loaded, but it omits essential details about parameters and usage. This is under-specification, not genuine conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters (including a complex required object target with conditional properties and enum options) and no output schema, the description is severely incomplete. It fails to explain how to specify the target or choose transport/specVersion, leaving the agent without necessary context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no explanation of any parameter (target, transport, specVersion). The agent must infer meaning solely from the schema, which is insufficient for proper invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool verifies the transport layer, specifying the two transports (stdio and HTTP). It is specific and actionable, but does not differentiate from sibling tools like runJsonRpcCompliance or runRoundtripSmoke.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No information is provided about when to use this tool versus alternatives (e.g., runJsonRpcCompliance for JSON-RPC compliance, runRoundtripSmoke for roundtrip tests). The agent receives no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
12 tool updates
v0.2.1- First observed
assertNoBreakingChanges - First observed
compareManifests - First observed
generateReport - First observed
runAnnotationsAudit - First observed
runCapabilityIntrospection - First observed
runFullSuite - First observed
runJsonRpcCompliance - First observed
runOauthPkceFlow - First observed
runRoundtripSmoke - First observed
runSpecVersionAssertion - First observed
runToolSchemaValidation - First observed
runTransportSuite
TDQS
Every tool has a clearly distinct purpose targeting a specific conformance aspect or utility. There is no overlap between tools like runJsonRpcCompliance, runTransportSuite, and runOauthPkceFlow, as they each test different parts of the protocol.
All tool names follow a consistent verbNoun pattern in camelCase, using descriptive verbs like run, generate, compare, and assert. This uniform convention makes the tool surface predictable and easy to navigate.
With 12 tools, the server covers a comprehensive set of conformance testing areas without being overwhelming. Each tool serves a necessary function, from individual compliance checks to full suite execution and reporting.
The tool set covers all major MCP protocol conformance areas: JSON-RPC errors, spec version, transport, OAuth, tool schema, capability introspection, roundtrip testing, annotations, and regression comparison. No obvious gaps remain for the stated purpose.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
MCP Spec Compliance MCP โ audits any MCP server.json against the official Model Context Protocol
MEOK MCP Test MCP โ golden-file + schema-drift + tool-failure tests for any MCP server. Drop-in
A Model Context Protocol server for Wix AI tools
3rd Generation Testing (3TG) โ generate deterministic test suites from Markdown spec tables via MCP.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceModel Context Protocol server that enables API test automation across various AI hosts (Claude, OpenAI, custom) using Postman Collections.MIT
- AlicenseNot gradedqualityDmaintenanceA TypeScript-based test implementation for the Model Context Protocol (MCP) with example configurations and Jest test cases. Provides a development foundation for building and testing MCP server functionality.10MIT
- AlicenseNot gradedqualityDmaintenanceA comprehensive reference implementation demonstrating all features of the Model Context Protocol (MCP) specification, serving as documentation, learning resource, and testing tool for MCP implementations.1MIT
- AlicenseNot gradedqualityDmaintenanceA comprehensive testing server for validating MCP scanner tools and implementations, providing diverse tools, resources, and prompts that exercise various aspects of the Model Context Protocol specification.19MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/studiomeyer-io/mcp-protocol-conformance'
If you have feedback or need assistance with the MCP directory API, please join our Discord server