Skip to main content
Glama
mcp-protocol-conformance

mcp-protocol-conformance

runRoundtripSmoke

Read-onlyIdempotent

Runs a tools/call against every advertised tool to verify round-trip JSON-RPC behavior, failing on transport errors and warning when a tool returns isError=true.

Instructions

Perform one tools/call per advertised tool. Fails on JSON-RPC transport errors, warns on tool-level isError=true.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "target": "<target>"
    },
    "name": "runRoundtripSmoke"
  }
}

Input Schema

NameRequiredDescriptionDefault
targetYes
sampleArgsNo
specVersionNo2025-06-18

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changed
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / sampleArgs / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • removedInput schema / properties / target / anyOf
      Removed value: -[
      -  {
      -    "additionalProperties": false,
      -    "properties": {
      -      "args": {
      -        "items": {
      -          "type": "string"
      -        },
      -        "type": "array"
      -      },
      -      "cmd": {
      -        "minLength": 1,
      -        "type": "string"
      -      },
      -      "cwd": {
      -        "type": "string"
      -      },
      -      "env": {
      -        "additionalProperties": {
      -          "type": "string"
      -        },
      -        "type": "object"
      -      },
      -      "kind": {
      -        "const": "stdio",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "kind",
      -      "cmd"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "additionalProperties": false,
      -    "properties": {
      -      "headers": {
      -        "additionalProperties": {
      -          "type": "string"
      -        },
      -        "type": "object"
      -      },
      -      "kind": {
      -        "const": "http",
      -        "type": "string"
      -      },
      -      "url": {
      -        "format": "uri",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "kind",
      -      "url"
      -    ],
      -    "type": "object"
      -  }
      -]
    • addedInput schema / properties / target / oneOf
      Added value: +[
      +  {
      +    "properties": {
      +      "args": {
      +        "items": {
      +          "type": "string"
      +        },
      +        "type": "array"
      +      },
      +      "cmd": {
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "cwd": {
      +        "type": "string"
      +      },
      +      "env": {
      +        "additionalProperties": {
      +          "type": "string"
      +        },
      +        "propertyNames": {
      +          "type": "string"
      +        },
      +        "type": "object"
      +      },
      +      "kind": {
      +        "const": "stdio",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "kind",
      +      "cmd"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "properties": {
      +      "headers": {
      +        "additionalProperties": {
      +          "type": "string"
      +        },
      +        "propertyNames": {
      +          "type": "string"
      +        },
      +        "type": "object"
      +      },
      +      "kind": {
      +        "const": "http",
      +        "type": "string"
      +      },
      +      "url": {
      +        "format": "uri",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "kind",
      +      "url"
      +    ],
      +    "type": "object"
      +  }
      +]
  2. First observedv0.2.1

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by defining the result semantics: transport-level JSON-RPC errors are hard failures, while tool-level isError=true results only warn. That distinction is the key behavioral fact an agent needs. It does not say whether the run spawns processes or opens network connections to the target, nor what the summary output looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action and then the pass/fail rule. No filler or repetition; every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a nested polymorphic target, an undocumented free-form sampleArgs parameter, and no output schema, the description is too thin. It never explains what 'advertised tool' means (the target server's tools/list output), how sampleArgs maps to those tools, or what the run returns on success versus warning.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across three parameters, including a nested oneOf target (stdio command vs HTTP URL) and a free-form sampleArgs map. The description says nothing about target, sampleArgs, or specVersion, so the agent must infer that sampleArgs supplies arguments for the smoke calls and that specVersion selects protocol behavior entirely from the schema shape.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: issue exactly one tools/call per advertised tool on a target server. That distinguishes it from runFullSuite and runTransportSuite by scope, though the description never names those siblings to make the contrast explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'One tools/call per advertised tool' implies a shallow, quick connectivity check, which is implied guidance for choosing it over the fuller runFullSuite. There is no explicit when-to-use, no prerequisite (e.g. server must already be reachable), and no stated exclusion versus runTransportSuite or runJsonRpcCompliance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.