Skip to main content
Glama

collection-mcp-check

Turn a Postman collection into an MCP server, then check whether an AI agent can actually use it.

cmcp lint   collection.json          # agent-readiness findings, CI exit code
cmcp tools  collection.json          # the MCP tools/list payload it would expose
cmcp serve  collection.json          # run it as an MCP server on stdio
cmcp check  collection.json --call-readonly   # spawn the server, speak MCP to it, call the safe tools

Python 3.10+, standard library only. No runtime dependencies.

Why this exists

Postman already turns collections into MCP servers (the MCP Generator on the API Network gives you one tool per request). The gap is the step after that: a collection written for humans clicking "Send" makes a shaky tool list for a model.

I ran it on echo-v2.postman_collection.json from postmanlabs/newman (the Postman Echo collection, 22 requests). What it found:

  • Set Cookies and Delete Cookies are GETs. A generator that marks every GET readOnlyHint: true tells the client these are safe to call without asking. cmcp reads the name and path, flags it (CM040), and sets readOnlyHint: false.

  • DigestAuth Success needs {{echo_digest_realm}} and {{echo_digest_nonce}}, which a test script in a different request writes with postman.setGlobalVariable. In Postman you run the folder in order. An agent calls one tool. CM023 catches that chain.

  • Digest and OAuth 1.0 auth aren't replayed by the generated server (CM060), so those calls go out unsigned.

  • 8 GET requests carry a formdata body (including a client_secret field) that the tool silently drops (CM022, CM030).

Then cmcp check --call-readonly called the 15 read-only tools against the live postman-echo.com. 12 returned 200. Of the 3 failures, 2 are exactly what lint predicted (missing chained variables, unsupported OAuth 1.0). The third is the first half of the digest flow, which gets a 401 on purpose and stores the nonce from it.

That's the point of shipping lint and live check together: the lint tells you which calls will fail before an agent finds out the hard way.

Related MCP server: Postman MCP Generator

Quickstart

git clone https://github.com/DeveshB-1/collection-mcp-check && cd collection-mcp-check
uv run cmcp lint examples/postman-echo-v2.postman_collection.json
uv run cmcp check examples/postman-echo-v2.postman_collection.json --call-readonly

# your own collection + environment export
uv run cmcp lint my.postman_collection.json --env my.postman_environment.json --strict

Use it from Claude Desktop (or any MCP client):

{
  "mcpServers": {
    "orders-api": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/collection-mcp-check",
               "cmcp", "serve", "/path/to/orders.postman_collection.json",
               "--env", "/path/to/prod.postman_environment.json"]
    }
  }
}

Variables resolve in this order: collection variables, then the --env file (a Postman environment export or a flat JSON object), then CMCP_VAR_<name> process env vars. Secrets stay out of the collection file.

What the lint checks

Rule

Severity

Catches

CM001

warn

two requests slug to the same tool name (renamed _2, and agents mix them up)

CM002

warn

name over 64 chars, truncated

CM003

error

name fails ^[a-zA-Z0-9_-]{1,64}$ (the rule the Anthropic and OpenAI tool APIs enforce)

CM010

warn

request has no description, so the model only sees GET {{baseUrl}}/...

CM011

info

description over 1,500 chars (sent on every tools/list)

CM012

warn

path or query parameter with no description

CM020

error

{{variable}} defined nowhere (collection or env)

CM021

error

body declared JSON that doesn't parse

CM022

warn

GET/HEAD with a body (dropped from the tool)

CM023

warn

depends on a variable another request's script sets

CM030

warn

literal credential in a header, query, form field or auth block

CM040

warn

GET whose name/path reads as a write; readOnlyHint set to false

CM050

error

request with no URL

CM060

warn

auth type the server won't replay (digest, oauth1, hawk, ...)

agent-readiness = share of tools with zero errors or warnings. lint exits 1 on errors (--strict: on warnings too), so it drops into CI next to a Newman run.

What the server does

  • One tool per request. Name from folder_request in snake_case, deduped, kept to 64 chars.

  • Input schema: path variables (required, with saved examples), query params (defaults from the collection), JSON body as an object with the saved example as default, form fields as an object.

  • Annotations: readOnlyHint for GET/HEAD/OPTIONS that don't read as writes, destructiveHint for DELETE/PUT/PATCH and cancel/remove/reset-style names, openWorldHint always.

  • Auth inherited collection → folder → request, as Postman does. Basic, bearer and API-key (header) are replayed.

  • Postman dynamic variables {{$guid}}, {{$timestamp}}, {{$isoTimestamp}}, {{$randomInt}}, {{$randomUUID}} are filled at call time.

  • HTTP 4xx/5xx come back as isError: true with the status, latency and the first 4,000 chars of the body.

  • MCP over stdio, protocol versions 2024-11-05 through 2025-06-18: initialize, ping, tools/list, tools/call.

It sets its own User-Agent, because postman-echo.com returns 403 to urllib's default Python-urllib/3.x (that's how I found out).

Sample run

Live run against postman-echo.com, 2026-10-25 (full output in demo/):

$ cmcp lint examples/postman-echo-v2.postman_collection.json
collection: Postman Echo (V2)
tools: 22  clean: 9  agent-readiness: 41%
errors: 0  warnings: 26  info: 0

warn  CM023 auth_digest_digestauth_success: depends on variables another request's script sets ({{echo_digest_nonce}}, {{echo_digest_realm}}); an agent calling this tool on its own won't have them.
warn  CM040 cookies_delete_cookies: GET whose name/path reads as a write ('Delete Cookies'); readOnlyHint set to false so clients ask before calling it.
warn  CM040 cookies_set_cookies: GET whose name/path reads as a write ('Set Cookies'); readOnlyHint set to false so clients ask before calling it.
warn  CM060 auth_digest_digestauth_success: auth type 'digest' is not replayed by the generated server; calls will go out unauthenticated.
warn  CM060 auth_others_oauth1_0_verify_signature: auth type 'oauth1' is not replayed by the generated server; calls will go out unauthenticated.
... (21 more in demo/echo-lint.txt)

$ cmcp check examples/postman-echo-v2.postman_collection.json --call-readonly
protocol: 2025-06-18  tools listed: 22  schema problems: 0
read-only calls:
  FAIL auth_digest_digestauth_request: HTTP 401 (328 ms) GET https://postman-echo.com/digest-auth
  FAIL auth_digest_digestauth_success: Variable '{{echo_digest_realm}}' is not set. Pass --env <postman_environment.json> or export CMCP_VAR_echo_digest_realm=<value>.
  ok   auth_others_basic_auth: HTTP 200 (271 ms) GET https://postman-echo.com/basic-auth
  FAIL auth_others_oauth1_0_verify_signature: HTTP 401 (274 ms) GET https://postman-echo.com/oauth1
  ok   cookies_get_cookies: HTTP 200 (291 ms) GET https://postman-echo.com/cookies
  ok   headers_request_headers: HTTP 200 (261 ms) GET https://postman-echo.com/headers
  ok   headers_response_headers: HTTP 200 (268 ms) GET https://postman-echo.com/response-headers?Content-Type=text%2Fhtml&Server=apibin
  ok   request_methods_get_request: HTTP 200 (244 ms) GET https://postman-echo.com/get?test=123
  ok   utilities_response_status_code: HTTP 200 (255 ms) GET https://postman-echo.com/status/200
  ok   utilities_streamed_response: HTTP 200 (296 ms) GET https://postman-echo.com/stream/10
  ok   utilities_delay_response: HTTP 200 (3275 ms) GET https://postman-echo.com/delay/3
  ok   utilities_get_utf8_encoded_response: HTTP 200 (280 ms) GET https://postman-echo.com/encoding/utf8
  ok   utilities_gzip_compressed_response: HTTP 200 (263 ms) GET https://postman-echo.com/gzip
  ok   utilities_deflate_compressed_response: HTTP 200 (264 ms) GET https://postman-echo.com/deflate
  ok   draft_auth_oauth2_0_oauth2_0_get_resource: HTTP 200 (275 ms) GET https://postman-echo.com/oauth2/user/info
result: FAIL

Write tools (POST/PUT/PATCH/DELETE, and the two cookie GETs) are never called by check.

Tests

$ uv run --with pytest --with mcp python -m pytest -q
................                                                         [100%]
16 passed in 3.68s

The suite runs offline against a local HTTP mock: parsing (v2.0 and v2.1, auth inheritance, string URLs), naming, schemas, every lint rule on examples/orders-api.postman_collection.json (a fixture built to trip them), the JSON-RPC loop, check spawning the server as a subprocess, and one interop test that drives the server with the official MCP Python SDK client (mcp 2.3.0; skipped when it isn't installed).

Limits

  • Pre-request and test scripts are read only to find which variables they set. They aren't executed.

  • No digest, OAuth 1.0, Hawk, AWS SigV4 or NTLM signing (lint warns instead).

  • formdata file fields are sent as plain text fields.

  • Mutating-GET detection is a word list (delete, set, cancel, ...). It will miss some.

Files

cmcp/collection.py  parse v2.0/v2.1 collections and environments
cmcp/tools.py       requests -> MCP tool definitions
cmcp/lint.py        agent-readiness rules
cmcp/runtime.py     variable resolution + HTTP call
cmcp/server.py      MCP stdio server (JSON-RPC 2.0)
cmcp/check.py       stdio client that validates a running server
examples/           Postman Echo collection (from postmanlabs/newman, Apache-2.0) + a fixture

examples/postman-echo-v2.postman_collection.json is copied unchanged from postmanlabs/newman (Apache-2.0). Everything else is MIT.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    An MCP server that generates AI agent tools from Postman collections and requests. This server integrates with the Postman API to convert API endpoints into type-safe code that can be used with various AI frameworks.
    1
    13
    MIT
  • F
    license
    Not graded
    quality
    Not graded
    maintenance
    Generates MCP servers from Postman API collections, automatically converting selected API requests into MCP-compatible tools that can be used with Claude Desktop and other MCP clients.
    -
  • A
    license
    Not graded
    quality
    D
    maintenance
    Generates MCP servers from Postman API requests, automatically creating JavaScript tools that enable LLMs to interact with any API documented in Postman collections. Supports authentication configuration and deployment via STDIO or SSE.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server that converts Postman API requests into executable tools for LLMs using the Postman Runtime. It supports complex authentication types and enables seamless integration between Postman collections and MCP clients like Claude Desktop.
    MIT