Conversational API Test MCP
Conversational API Test MCP lets an AI agent build, run, chain, and review API tests from YAML definitions while keeping credentials out of the conversation.
Define & organize APIs in YAML — list, summarize, read, create/append, or merge definitions under
.mcp/api/(list_api_definitions,summarize_api_definition,read_api_definition,upsert_canonical_api_definition,reorganize_mcp_api_definitions).Import existing work — convert Postman collections, OpenAPI/Swagger, Insomnia, or HAR files, paste cURL commands, and import Postman environments; credentials move to
.env.mcp.localand become{{VARIABLES}}(discover_legacy_api_sources,convert_legacy_to_canonical,import_curl,import_postman_environment).Send & chain requests — execute single requests or multi-step flows in one call, with retries, polling, captures, and inline asserts; preview requests without sending them (
execute_api_request,execute_api_flow,dry_run_request).Assert on responses — check status, latency, JSONPath existence, and rich value checks (contains, regex, equals, ranges, list membership, schema, snapshots) (
assert_response).Manage environments & variables — switch environments (e.g. STAGING), set/read session variables, and inspect available context (
set_environment,set_environment_variable,get_environment_variable,explain_request_context).Review endpoints with Vander — run a deterministic VANDER checklist (Verbs, Authorization, Negative, Data, Errors, Responsiveness) with automatic probes, including BOLA role checks and destructive-probe opt-in (
summon_vander,plan_vander_checks,run_vander_checks).Evaluate variable-answer endpoints — benchmark LLM/RAG/search endpoints over repeated cases with pass rates, latency percentiles, and optional agent-graded judgments (
run_eval).Bootstrap a workspace — scaffold
.mcp/api/, a runnable httpbin demo,.env.mcp.local, and.gitignoreentries without overwriting anything (init_workspace).
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Conversational API Test MCP Run Vander QA checks on the whoami endpoint from .mcp/api/httpbin.yaml"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Conversational API Test MCP
An API test bench your AI agent can drive — and a QA engineer named Vander to drive it.
Most API tools assume a human at the keyboard. This one assumes an agent: you describe your API once in YAML, and the agent runs requests, chains them, and reviews endpoints through a testing heuristic, while the server keeps three promises the agent cannot keep on its own — the request sent is exactly the one declared, the answer costs few tokens, and your credentials never enter the conversation.
It is an MCP server over stdio. It works with Cursor, VS Code + GitHub Copilot, Claude Code, and any other MCP stdio client.
What a session looks like
A real run, recorded on 2026-10-07 against httpbin.org with the definition in examples/httpbin.example.yaml. Only safe methods were sent.
You:
/vander.mcp/api/httpbin.yamlwhoamiVander: Vander here. Reviewing
GET /beareron httpbin.org. The plan has 13 automatic checks; 4 of them would send POST, PUT, PATCH or DELETE, so I left those out.8 passed, 1 failed, 4 skipped.
V ✅
OPTIONS→200. POST / PUT / PATCH / DELETE not sent.A ❌ No credentials →
401, as it should be. ButBearer vander-invalid-token→200with"authenticated": true.N ⚪ Nothing to negate: no body, no query parameters, no id in the path.
D ✅ Baseline
200,application/jsonwith a JSON body,$.authenticatedpresent.E ✅ No
5xx, and nothing leaked in the401.R ✅ 196 ms against a 1000 ms budget. One sample, not a load test.
One finding: the endpoint accepts any bearer token. It checks that a token is present, not that it is valid. For an echo service like httpbin that is to be expected; on a real API it is the difference between authentication and a header check.
Not tested: the four state-changing verbs, an expired token, another user's token, rate limiting.
endpoint:
id: whoami
method: GET
path: /bearer
summary:
pass: 8
fail: 1
skipped: 4
manual: 6
checks[13]{id,result,detail}:
V-options,pass,status 200 in 655ms
V-post,skipped,state-changing (POST)
V-put,skipped,state-changing (PUT)
V-patch,skipped,state-changing (PATCH)
V-delete,skipped,state-changing (DELETE)
A-none,pass,status 401 in 605ms
A-invalid,fail,status 200 in 605ms
D-baseline,pass,status 200 in 142ms
D-content-type,pass,"content-type \"application/json\", body is JSON"
D-captures,pass,""
E-no-5xx,pass,""
E-no-leak,pass,""
R-duration,pass,142ms
failures[1]{id,title,expected,actual,note}:
A-invalid,Request with an invalid token is rejected,"status in [401, 403]",status 200 in 605ms,"{\"authenticated\":true,\"token\":\"[REDACTED]\"}"
hint: "4 state-changing probes not sent; ask the user, then rerun with includeDestructive: true"How this was recorded: a script drove the built server over MCP stdio and saved the tool output shown above, unedited. Vander's reply was then written by Claude from that output, following the vander prompt. Every number in the reply comes from the output; timings will differ on your machine.
Related MCP server: API Debugger MCP
Three bets
1. The agent chooses; the server executes. The agent picks a requestId, and the server builds and sends the request from the YAML. There is no hand-written curl in the chat to get subtly wrong, and the same call gives the same request tomorrow.
2. Tokens are a budget. responseDetail, jsonPathSelect and maxBodyChars decide how much of a response comes back; execute_api_flow runs a whole scenario in one call; results are encoded as TOON rather than JSON.
3. A heuristic is worth more than a personality. Vander is a persona, but what he does is fixed: a checklist built by code and verdicts computed from status codes and timings. Research on personas in prompts found they do not, by themselves, make a model more accurate (sources), so the personality is there for the conversation and the rigour lives in run_vander_checks.
How it works
You test an API by talking to your agent. Three pieces make that possible:
Piece | Where | What it is | If you know Postman |
Definitions |
| Your requests, described once and committed with the code. | The collection |
Secrets and environments |
| Base URLs, tokens and passwords. Stays on your machine. | The environment |
The conversation | Your MCP client | You ask in plain language; the agent calls this server; the server sends the request. | Send, Runner and the Tests tab |
Getting started
Requires Node.js ≥ 20.
1. Build the server and register it in your MCP client.
npm ci
npm run build{
"servers": {
"conversational-api-test-mcp": {
"type": "stdio",
"command": "node",
"args": ["/absolute/path/to/conversational-api-test-mcp/dist/index.js"]
}
}
}That is the VS Code format. Cursor and Claude Code read the same entry under "mcpServers" instead of "servers", without the "type" line.
2. Open the project whose API you want to test and say hi Vander.
He explains the setup, checks which pieces your project already has, and offers to create the missing ones. Accepting runs init_workspace, which writes the three things below and never overwrites a file that exists. You can also create them by hand.
3. The .mcp/api/ folder and your first YAML. One file per service. Each request gets an id, which is how you and the agent refer to it.
# .mcp/api/weather.yaml
version: "1"
service: weather
base_url: "{{BASE_URL}}"
endpoints:
- id: get_token
method: POST
path: /oauth/token
form:
grant_type: client_credentials
client_id: "{{CLIENT_ID}}"
client_secret: "{{CLIENT_SECRET}}"
capture:
TOKEN: $.access_token
- id: forecast
method: GET
path: /forecast
params:
city: lisbon
auth: Bearer {{TOKEN}}
auth_dependency: get_token
flows:
smoke:
steps:
- requestId: forecast
assert:
status: 200
jsonPathExists: $.daysYou rarely have to write this by hand. Paste a cURL command, paste a whole Postman collection (the exported JSON), or point at an OpenAPI / Insomnia file, and the server writes the YAML for you. Tokens that come along are moved to .env.mcp.local and replaced by {{VARIABLE}}, so the YAML stays safe to commit.
4. The .env.mcp.local file. Create it at the project root, next to .mcp/. It holds what {{...}} stands for. You fill it in yourself, it must be in .gitignore, and its values never go into the chat.
# .env.mcp.local
BASE_URL=https://api.example.com/v1
CLIENT_ID=...
CLIENT_SECRET=...
STAGING_BASE_URL=https://staging.example.com/v1A prefix turns one YAML into several environments: after "use staging", {{BASE_URL}} reads STAGING_BASE_URL first and falls back to BASE_URL.
5. Ask. No file to write yet? init_workspace drops in a demo against httpbin.org that runs without any secret, and examples/ has three more you can copy into .mcp/api/.
Coming from Postman
You do not rebuild your requests. You hand over what you already have — a cURL command or an exported collection — and keep working from the chat. Everything below was run against the built server; the outputs are what it returned.
Where everything ends up. All of it sits in the project whose API you are testing, not in this repository:
your-project/
├── .mcp/
│ └── api/
│ ├── orders.yaml # one YAML per service: commit it
│ └── shop.yaml
├── .env.mcp.local # base URLs, tokens, passwords: never commit it
├── .gitignore # contains .env.mcp.local
└── shop.postman_collection.json # your export: read, never changedPaste a cURL
You: turn this into a request called
create_order, inorders.yamlcurl -X POST "https://httpbin.org/anything/orders?source=app" \ -H "Authorization: Bearer demo-token-123" \ -H "Content-Type: application/json" \ -d '{"item":"notebook","quantity":2}'
The agent calls import_curl, which answers:
definition: .mcp/api/orders.yaml
created: true
requestId: create_order
method: POST
url: "https://httpbin.org/anything/orders"
secretsMovedToEnvFile[1]: CREATE_ORDER_TOKENand writes this file. The query string became params, the JSON body became body, and the token is gone from it:
# .mcp/api/orders.yaml
version: "1"
base_url: https://httpbin.org
endpoints:
- id: create_order
method: POST
path: /anything/orders
headers:
Content-Type: application/json
params:
source: app
body:
item: notebook
quantity: 2
auth: Bearer {{CREATE_ORDER_TOKEN}}The token itself was appended to .env.mcp.local as CREATE_ORDER_TOKEN=..., and .env.mcp.local was added to .gitignore if it was not there. From here on it is "run create_order".
Hand over a collection
Export the collection from Postman (Export → Collection v2.1) and either drop the JSON in the project or paste its content in the chat.
You: here is my Postman collection,
shop.postman_collection.json. Convert it toshop.yaml.
The agent calls convert_legacy_to_canonical, which answers:
outputPath: .mcp/api/shop.yaml
source: postman
losses[1]{code,message}:
POSTMAN_PRE_REQUEST_SCRIPTS,"Some pre-request script lines could not be translated and will not run. Requests: Create order."
endpointCount: 2# .mcp/api/shop.yaml
version: "1"
service: Shop
base_url: "{{base_url}}"
variables:
base_url: https://httpbin.org
rid: "{{$uuid}}" # from pm.environment.set("rid", …{{$guid}}…)
endpoints:
- id: postman_0_List_orders
method: GET
path: /anything/orders?status=open
description: List orders
headers:
X-Request-Id: "{{rid}}"
assert: # from the pm.test blocks
status: 200
maxDurationMs: 1000
checks:
- path: $.args.status
equals: open
- id: postman_1_Create_order
method: POST
path: /anything/orders
description: Create order
body:
item: notebook
quantity: 2
headers:
Content-Type: application/json
assert:
status: 200
capture: # from pm.environment.set("orderItem", pm.response.json().json.item)
orderItem: $.json.itemEvery request in the collection, folders included, is now an endpoint. The losses list is the server telling you what it left behind, and the conversion needs a few minutes of finishing — ask the agent to do it, it previews each change before writing:
What you had in Postman | What the import does | How to finish |
The host in each URL | Becomes | Nothing, when it was found. Otherwise "set |
The Authorization tab (collection, folder or request) | Converted per request, following Postman's inheritance ( | Nothing for the converted ones. For the rest, |
Pre-request scripts | Generated values become variables: |
|
Tests ( | Translated to |
|
Dynamic variables |
| — |
Request names | Become ids such as | "Rename it to |
Query string | Stays inline in | Works as it is; move it to |
Bodies |
| Rewrite those as |
Disabled headers and params | Left out, as Postman leaves them out of the request. | — |
Postman environments | Imported with | Postman exports initial values only: fill in the empty ones it reports. |
The collection file is only read. Keep it, delete it, or leave it out of git — the YAML is the source of truth from now on.
When you ask the agent to look for something to import without naming a file, discover_legacy_api_sources searches the project for *postman*.json, collection*.json, openapi*.{yaml,yml,json}, swagger*.{yaml,yml}, insomnia*.{json,yaml} and *.har. A file with another name is converted when you give its path.
A day of work, in prompts
Moment | You say |
First time in the project | "hi Vander, set this project up" |
Bringing requests in | "turn this cURL into a request called |
Tidying an import | "set |
Running | "run |
Chaining | "create an order, then fetch it by the id that came back" · "save that as a flow called |
Checking | "check that it returns 200 and |
Reviewing | "Vander, is |
What you can ask
Each line is something you would otherwise do by hand in an API client.
You say | What happens | In Postman you would |
"Here is my Postman collection" + the JSON, or the path to it | Converts a collection, an OpenAPI spec, an Insomnia export or a HAR recording to YAML. | Import |
"Turn this cURL into a request" + the command | Creates the request in a YAML; any token in it goes to | Import → Raw text |
"Run | Sends that one request and summarises the response. | Click Send |
"Run the smoke flow" | Runs the requests of a flow in order, stopping at the first failure. | Collection Runner |
— (in CI) | Runs every flow and writes a JUnit report; fails the job on a failing step. | Newman / Postman CLI |
"Import | Copies its values to | Import → Environment |
"Check the response matches the | Validates the body against that schema. |
|
"Log in and list the orders" | Runs the login first, keeps the token, uses it, and logs in again on a | Pre-request script |
"Create a post, then fetch it by the id that came back" | Captures a value from one response and uses it in the next request. |
|
"Use staging" | Switches the active environment for the following requests. | Environment dropdown |
"Check that it returns 200 and has a | Asserts on status and on a JSONPath. |
|
"Start the export and wait until it is ready" | Polls a request until a field appears. |
|
"Show me what | Builds the request without sending it, credentials hidden. | Console |
"Add a request that creates an order with these fields" | Writes a new entry to the YAML, showing a preview first. | New request |
"Run the smoke eval of the assistant" | Sends each question several times and passes it on a pass rate; checks answers, citations and latency by code. | — |
"Vander, is | Reviews the endpoint on six axes and reports evidence. | — |
"Can Bob read Alice's order?" | Calls the endpoint with each other role's token, expecting 401/403/404 (BOLA). | — |
"Are the asserts of the smoke flow any good?" | Breaks each real response in small ways and reports which regressions no assert would notice. | — |
"Does the API still match | Sends the requests and validates status and body against the spec. | — |
"Tell me if the response shape changes" | Records a snapshot on the first run and reports drift on later ones. | Compare by eye |
"Create the CRUD flows for orders" | Writes create → read → update → delete → read-again flows from the endpoints you have. | — |
"Convert | Turns a browser recording into requests and a | Interceptor / Capture |
— (terminal) | Serves the definitions as a mock API. | Mock server |
"Export the flows as Arazzo" · "draft a Pact contract" | Writes OpenAPI 3.1, Arazzo 1.0 or Pact v3 under | Export |
Tools
Tool | Purpose |
| List YAMLs under |
| Cheap triage: endpoint ids, variable keys, flow names. |
| Full, validated definition. |
| Run one endpoint. |
| Run several endpoints in one call — inline |
| Show the request that would be sent, without sending it. |
| Assert on the last response: status, latency and value checks. |
| Evaluate an endpoint whose answer varies (LLM / RAG / search): cases × repeats, pass rate, optional samples for the agent to judge. |
| Select the active environment ( |
| Session variables. |
| Which variable keys are available from where. |
| Create |
| Create / append definitions (dry-run by default). |
| Merge many YAMLs into fewer (plan, then apply). |
| Hands the agent the Vander persona when you address him by name. |
| VANDER review of one endpoint: checklist, then automatic execution — one row per check, the id prefix being the axis. |
| Turn a pasted |
| Import Postman, OpenAPI, Insomnia or HAR — from a file or from pasted text. |
| Copy a Postman environment export into |
| Mutation testing for asserts: score a flow's asserts and get the ones that are missing. |
| Compare real responses with an OpenAPI / Swagger file. |
| Propose (or write) CRUD flows that end by checking the delete really deleted. |
| Write a definition as OpenAPI 3.1, Arazzo 1.0 or a Pact v3 draft. |
Every tool publishes MCP annotations: the read-only ones say so (clients can run them without asking), and the ones that send HTTP or replace files are marked as such.
Recommended order: list_api_definitions → summarize_api_definition → set_environment (if needed) → execute_api_request or execute_api_flow.
Vander
Vander is a persona shipped as an MCP prompt: a senior API QA engineer you talk to instead of issuing tool calls yourself. In clients that expose MCP prompts it shows up as a command (for example /vander), optionally with a definition path and an endpoint id. He reviews an endpoint through the VANDER heuristic:
Axis | Question | Checked automatically |
Verbs | What does the path do with methods it does not declare? | Undeclared methods are rejected (405/404/501); |
Authorization | Who can call it? | No credentials and an invalid token are rejected (401/403). With |
Negative | What happens with bad input? | Each body field removed, of the wrong type, |
Data | Does the response say what it should? | Baseline succeeds, |
Errors | Does it fail well? | Malformed JSON is a client error; no probe causes a 5xx or leaks a stack trace. |
Responsiveness | Is it fast enough? | Baseline answers within |
Each axis also carries manual ideas (expired tokens, idempotency, rate limiting…) that Vander explores with the regular tools.
Two declarations turn manual ideas into automatic checks:
roles: # other identities, as the Authorization each one sends
bob: Bearer {{BOB_TOKEN}}
support: Bearer {{SUPPORT_TOKEN}}
endpoints:
- id: get_order
method: GET
path: /orders/{{order_id}}
auth: Bearer {{ALICE_TOKEN}} # the owner
allow_roles: [support] # support may read it: not probed
- id: create_user
method: POST
path: /users
body: { name: Ana, age: 30 }
body_schema: openapi.yaml#/components/schemas/NewUser # or inlineA-role-bob calls get_order with Bob's token and fails if the order comes back: broken object level authorization, first in the OWASP API Security Top 10. Only endpoints that address one object (a {{placeholder}} or a numeric id in the path) are probed, since a list is expected to answer every user. body_schema feeds the boundary, enum and format probes; an OpenAPI import fills it in for you.
In clients without prompt support — or when you would rather just talk — say his name: "hi Vander, review whoami". The server's instructions tell the agent to call summon_vander, which hands it the same brief.
The check tools work without the persona too:
plan_vander_checksreturns the checklist for arequestIdand sends nothing.run_vander_checksexecutes the automatic checks and returns pass / fail / skipped per check (the id prefix is the axis), with the detail of each failure listed apart. Probes never capture variables. Anything that sendsPOST,PUT,PATCHorDELETEis skipped unlessincludeDestructive: true, so reviewing a write endpoint is an explicit decision.
VANDER builds on Stuart Ashman's VADER heuristic and adds an explicit Negative axis — see where the ideas come from.
Definitions
Variables and environments
{{KEY}} is resolved from, in increasing priority: YAML variables → .env.mcp.local → session variables (captures and set_environment_variable).
After
set_environment(e.g.STAGING),{{BASE_URL}}resolvesSTAGING_BASE_URLfirst and falls back toBASE_URL.Unresolved placeholders are left as literal
{{KEY}}so the problem is visible.Macros:
{{$uuid}},{{$timestamp}}(ms),{{$timestampSeconds}},{{$isoTimestamp}},{{$date}},{{$date:YYYY-MM-DD HH:mm:ss}}.{{env.NAME}}reads the server'sprocess.env, but only for names listed inMCP_API_ENV_PASSTHROUGH.
Requests
base_urlmay carry a path prefix (https://host/api/v1);pathis appended to it. An absolutepathoverridesbase_url.paramsis sent as the query string;headers,body(JSON) andform(URL-encoded) are interpolated.capturemaps a session variable name to a JSONPath on the response;assertchecks status, latency and values — see value checks.
Auth
auth: Bearer {{TOKEN}}sets theAuthorizationheader.auth_dependency: <endpoint id>runs that endpoint first when its captured variables are missing, and again after a401(disable withauth_retry_on_401: false).digest_auth: { username, password }performs HTTP Digest.roles(at the top of the file) andallow_roles(on an endpoint) drive the BOLA checks — see Vander.
Tracing
tracing: true at the top of a definition sends a W3C traceparent header with every request and reports its traceId (on each flow step, and under failures in CI reports), so a failing call can be looked up in New Relic, Datadog, Jaeger or any OpenTelemetry backend. tracing: { url: "https://your-apm/trace/{traceId}" } also returns a link. An endpoint that sets its own traceparent keeps it.
Flows
Each step is an endpoint id or an object:
flows:
create_and_wait:
steps:
- create_job
- requestId: get_job
poll: { untilJsonPath: "$.finishedAt", maxAttempts: 10, delayMs: 2000 }
retry: { max: 3, delayMs: 500 }
acceptStatus: [200]
assert: { status: 200, jsonPathExists: "$.result" }
optional: falseValue checks
assert (on a request, on a flow step, or through assert_response) takes status, jsonPathExists, maxDurationMs and a list of checks. Each check reads the value at a JSONPath path — or the whole body as text when path is omitted — and every operator in it must hold. All failing checks are reported, not just the first.
Operator | Holds when |
| The text has all of / one of / none of the given strings. |
| The text matches the regular expression. |
| Makes the four above case-insensitive. |
| The value is exactly this (any JSON value). |
| The number is within the bounds. |
| The string or list has that many characters / items. |
| The list at |
| Every value at |
An operator the server does not know is an error when the YAML is read, never a silent pass.
schema checks the whole JSON body against a JSON Schema — written inline, or taken from a file in the project with a JSON pointer, usually your OpenAPI spec:
assert:
status: 200
schema: openapi.yaml#/components/schemas/Order # or an inline { type: object, required: [id] }OpenAPI 3.0 and 3.1 both work (nullable, internal $refs, formats). A failure names the path and the rule, e.g. schema: /customer must have required property 'name'. The file is re-read when it changes. Converting an OpenAPI file (rather than pasting it) writes this schema on every endpoint for you.
snapshot compares the body with the one recorded the first time it ran, stored in .mcp/snapshots/<name>.json (commit it):
assert:
snapshot: order # shape: keys and value types
# snapshot: { name: order, mode: values, ignore: [$.requestId] }shape (the default) catches contract drift (a field that disappeared, changed type or appeared) without breaking on data that changes every day. values compares the values too, with UUIDs, dates, JWTs and long hex ids masked; ignore drops JSONPaths. A failure reads snapshot order: $.total: "<number>" → "<string>". To accept a change on purpose, run with --update-snapshots (or MCP_UPDATE_SNAPSHOTS=1).
Evals: endpoints whose answer varies
A request to an LLM, a RAG or a search endpoint can come back different every time, so one green run proves little. An eval sends one request over several cases, each repeated, and passes a case on its pass rate:
evals:
smoke:
requestId: ask
repeat: 3 # each case is sent 3 times
passRate: 0.66 # and passes when 2 of the 3 runs satisfy `expect`
expect: # applied to every case
status: 200
maxDurationMs: 8000
checks:
- path: $.citations
subsetOf: $.sources[*].id # no invented citation
judge: # optional: samples for the agent to grade
answerPath: $.answer
contextPath: $.sources[*].text
questionVariable: question
cases:
- name: refund_window
variables: { question: How many days do I have to ask for a refund? }
reference: 30 days from delivery.
expect:
checks:
- { path: $.answer, contains: "30" }
- { path: $.sources[*].id, includesAll: [policy-refunds, faq-returns], minRatio: 0.5 }
- name: out_of_scope
variables: { question: Who will win the next election? }
expect:
checks:
- { path: $.answer, matches: "(don't|do not|cannot) (know|answer)", ignoreCase: true }
- { path: $.citations, maxLength: 0 }Run it with "run the smoke eval" (run_eval + evalName), or pass requestId and cases inline. The result has one row per case (passed: 2/3, latency, how many different answers came back), the distinct failures with how often each happened, and p50 / p95 latency.
Two kinds of verdict come out of it, and they are kept apart:
Checked by code — everything under
expect. Same input, same verdict: status, latency, words that must or must not appear, a refusal for out-of-scope questions, expected documents among the retrieved ones, citations that point at retrieved documents.Judged by the agent — with
judge, the result carries one sample per case (question, answer, retrieved passages, reference) and a rubric:faithfulness,relevance,context_relevance,correctness, or criteria in your own words. The server calls no model; the agent you are talking to does the grading, so the verdict depends on that model and is reported as judgement.
Things to know:
Every run is a real request, usually a paid one.
run_evalrefuses to start when cases × repeat exceedsmaxRequests(default 30).Eval runs do not capture variables or replace the session's last response;
auth_dependencystill logs in when needed.A streamed answer (
text/event-stream) is folded into{ text, eventCount, events }, so$.textis the answer andeventskeeps what was not a text delta (sources, usage, stop reason). Setsse.textPathon the request when the delta sits somewhere unusual.examples/rag.example.yamlis a complete starting point.
Testing the tests
A green suite says the asserts passed, not that they would catch anything. mutate_assertions runs a flow (or one request) once, then breaks each real response in realistic ways and runs the asserts again on every copy: a field disappears, changes type or turns null, a list comes back empty, a value changes, the status becomes 500 or 404, the answer is ten times slower. Nothing extra is sent to the API.
For a request whose only assert is { status: 200, jsonPathExists: $.id }, against a mocked order (excerpt of the real output):
score: 0.17
killed: 5
total: 30
steps[1]{requestId,score,killed,total,note}:
get_order,0.17,5,30,""
survivors[10]{step,change,addToAssert}:
get_order,took 1200ms instead of 20ms,"{\"maxDurationMs\":200}"
get_order,$.status disappeared,"{\"jsonPathExists\":\"$.status\"}"
get_order,$.items changed type (array → object),"{\"schema\":{\"type\":\"object\",\"required\":[\"items\"],\"properties\":{\"items\":{\"type\":\"array\"}}}}"
…
moreSurvivors: 15Each surviving mutation is a regression the suite would let through, with an assert that would catch it. Keep the ones on stable fields; ids and timestamps change between runs. useLastResponse: true scores the last response of the session without sending anything. Snapshots are read, never rewritten, while mutants run.
Contracts: drift, mocks and exports
Drift. check_contract_drift sends the requests of a definition (read-only ones unless includeDestructive) and checks each answer against the operation the OpenAPI / Swagger file documents for it: an undocumented status, or a body that breaks the response schema, is drift. It also lists the operations of the spec that no request covers. Requests are matched by method and path, whether the path holds {{placeholders}} or real ids, with or without the server's base path.
Imports that keep the link. Converting an OpenAPI file turns {petId} into {{petId}}, copies request body examples, and points assert.schema (success response) and body_schema (request body) back into the spec, so every run checks the contract and Vander derives negative probes from it.
CRUD flows. infer_crud_flows looks for POST /things next to GET|PUT|PATCH|DELETE /things/{{id}} and proposes crud_things: create, read, update, read, delete, and a last read that must answer 404 or 410, so a delete that does not delete fails it. It captures the id from the create response ($.id unless you already capture it). apply: true writes the flow, keeping a .bak of the file.
Mock server. npx conversational-api-test-mcp mock [--port 4010] [--delay 200] serves the definitions so a frontend or a CI job can run without the real API. Each endpoint answers with its mock block, the body of a values snapshot, a sample built from its assert.schema, or {}, in that order:
- id: get_pet
method: GET
path: /pets/{{pet_id}}
mock: { status: 200, body: { id: "{{pet_id}}", name: Rex } } # path params fill the bodyLiteral paths win over templated ones (/pets/mine before /pets/{{id}}), CORS preflights are answered, and every request is logged with the endpoint and the source of the answer. The mock keeps no state: a POST does not create anything and a DELETE does not remove it, so the last step of a CRUD flow fails against it by design.
Exports. export_contract writes under .mcp/contracts/:
openapi: OpenAPI 3.1 from the requests. Placeholders become path parameters,assert.schemaandbody_schemabecome$refs,authbecomes a security scheme.arazzo: the flows as Arazzo 1.0 workflows next to that OpenAPI. Captures become step outputs that later steps read ($steps.create_pet.outputs.pet_id); placeholders nobody captured become workflow inputs.pact: a Pact v3 draft with one interaction per endpoint and type matching on response bodies (taken frommock, value snapshots or schema samples). Trim it to what the consumer uses before publishing it to a broker.
Running in CI
The same package runs your flows from a terminal, so the YAML that the agent uses also gates a pipeline:
npx conversational-api-test-mcp run --env STAGING --reporter junit --out reports/api.xmlWith no file, every definition under
.mcp/api/is read and each of itsflowsruns (a file with no flows is skipped and listed).--flow smokepicks flows,--request get_userruns single requests, and--all-endpointsruns every request one by one — POST and DELETE included.--envworks asset_environment; secrets come from.env.mcp.localor, in CI, from a step that writes it from your secret store.--reporterispretty(default),jsonorjunit(one testsuite per flow, one testcase per step, read by GitHub Actions, GitLab, Jenkins and Azure DevOps).--bailstops at the first failing flow.--update-snapshotsrewrites the snapshots that differ instead of failing. Withtracingon, a failing step shows its trace id.Exit code
0when everything passed,1when a step failed,2for a usage error or nothing to run.
# .github/workflows/api.yml (excerpt)
- run: printf '%s\n' "$ENV_MCP_LOCAL" > .env.mcp.local
env: { ENV_MCP_LOCAL: "${{ secrets.ENV_MCP_LOCAL }}" }
- run: npx conversational-api-test-mcp run --env STAGING --reporter junit --out reports/api.xmlKeeping output small
responseDetail: "minimal"returns status, duration and captures only;"summary"(default) adds the body, capped at 8,000 chars, and the headers that usually matter (content-type,location,retry-after, rate limits…);"full"returns every header and raises the cap to 50,000.The body comes back once:
bodyJsonfor JSON,bodyPreviewfor text. Both appear only when JSON exceeds the cap (truncated preview plustopLevelKeys).jsonPathSelectprojects the body before it is serialized. On anexecute_api_flowstep it returns that step's body, so reading it needs no second call.execute_api_flowreplaces N tool calls with one.Parsed definitions and
.env.mcp.localare cached in memory and invalidated by filemtime.
Security model
The server runs locally with your privileges and is driven by an LLM, so treat API responses and definitions as untrusted input.
Redaction. Auth headers, values under credential-looking JSON keys (
password,token,secret,api_key, …), JWT-looking strings, credential query parameters and captured credentials are returned as[REDACTED]. The real values stay in the session and are still used by later requests. Redaction is name- and shape-based, so it is a safety net, not a guarantee.No implicit environment access.
process.envis not part of the interpolation context; only names you list inMCP_API_ENV_PASSTHROUGHare reachable, via{{env.NAME}}.Responses are data. Values captured from a response are inserted literally and never re-expanded as templates.
Writes are confined. Definition tools write only under
.mcp/api/and default to dry-run.init_workspacealso creates.env.mcp.localand adds it to.gitignore, and never overwrites an existing file. The import tools append the credentials they lift to.env.mcp.local, never replacing a key that is already there.Optional host allowlist.
MCP_API_ALLOWED_HOSTSrestricts requests — including every redirect hop — to the listed hostnames. It is off by default and is not a complete SSRF defence (no private-range or DNS-rebinding checks).
Environment variable | Effect |
| Default |
| Comma-separated hostnames requests may target. Unset = any host. |
| Comma-separated env var names (trailing |
| Response size cap. Default 10 MB. |
|
|
Where the ideas come from
Nothing here was invented from scratch. This is what the project borrows, from whom, and what it changes.
Idea in this project | Source | What was taken, and what was changed |
The VANDER axes | Stuart Ashman, VADER – a REST API test heuristic (QA Matters, 2016) | The five original axes — Verbs, Authorization, Data, Errors, Responsiveness — are his. This project adds Negative as a sixth and turns part of each axis into executable checks. |
The Negative axis | Gwen Diagram & Ash Winter's BINMEN (Boundary, Invalid entries, NULL, Method, Empty, Negative), as listed in the Ministry of Testing Test Heuristics Cheat Sheet | The framing of negative input as its own concern. Missing field, wrong type, |
Splitting input from output concerns | Amber Race's POISED (Parameters, Output, Interop, Security, Errors, Data), same cheat sheet | Informs the manual ideas under Data and Negative. |
Which probes are "destructive" | RFC 9110, HTTP Semantics §9.2.1 (safe methods) |
|
Expected statuses | RFC 9110 §15.5.6 ( | An undeclared method should get |
What the Authorization axis looks for | OWASP API Security Top 10 – 2023, API2 Broken Authentication | Automated: no credentials, invalid token. |
Host allowlist, response size cap | OWASP API7 (Server Side Request Forgery) and API4 (Unrestricted Resource Consumption) | Applied to this server itself, since it makes requests on an agent's behalf. |
Vander as an MCP prompt | Prompts are "user-controlled": the user picks them explicitly, typically as a slash command. That is why the persona is opt-in rather than baked into every conversation. | |
Persona for tone, code for verdicts | Zheng et al., When "A Helpful Assistant" Is Not Really Helpful (Findings of EMNLP 2024) | They report that adding a persona to the system prompt did not improve accuracy over no persona. So nothing Vander asserts depends on the persona. |
Schema-driven negative probes, inferred CRUD flows | Schemathesis and its SBFT '26 paper Schemathesis 4: Multi-Phase Property-Based Testing | Boundary and type probes derived from the schema, and links between operations inferred instead of declared (the paper found 73 explicit OpenAPI links across 4,094 specs). Here the probes are a fixed, explainable list per field, not random generation, and the inferred chain is only create → read → update → delete → read again. |
Mutation testing of asserts | Mutation testing (PIT, Stryker) and SWE-Mutation (2026), which found LLM-written test suites often lack the power to tell a broken implementation from a correct one | The code under test is not mutated; the recorded response is. That keeps it offline and fast, and fits a tool whose tests are asserts on responses. |
BOLA checks with roles | OWASP API Security Top 10 – 2023, API1 Broken Object Level Authorization | Automated once the definition declares a second identity; still manual for function-level authorization (API5). |
Snapshots that ignore volatile data | Snapshot / approval testing (Jest snapshots, ApprovalTests "scrubbers") | Shape mode by default, so a snapshot of live API data does not break every day. |
Runtime contract drift | oasdiff compares two versions of a spec; this compares the spec with the live API | Matching by method and path, validation with the same Ajv engine as |
HAR import | Record-and-replay tools such as Keploy | Only the recording format is taken: no traffic capture, no replay of mocks, and credentials are never copied. |
Exports | Pact output is a draft to review, following the advice that agent-written consumer contracts need human review before they gate anything. | |
Trace ids on requests | Only the | |
TOON output | Used as-is for tool results. Its authors report the largest savings on uniform arrays and little or none on deeply nested data; this project has not benchmarked it independently. |
VANDER is this project's name for the extended heuristic and has no affiliation with the authors above.
Troubleshooting
Symptom | Check |
Empty definition list | Is |
|
|
| Key name, environment prefix ( |
A value shows as | Expected for credentials. It is still used in requests. |
Development
Command | Description |
| Compile TypeScript to |
| Run from source with |
| Run the test suite ( |
| ESLint. |
License
Available Tools
21 toolsassert_responseA
Assert on the last HTTP response: status, jsonPathExists, maxDurationMs and value checks (contains, regex, equality, ranges, list membership). Reports every check that failed. Run after execute_api_request.
Inputs: checks, status, maxDurationMs, workspaceRoot, +1 more.
convert_legacy_to_canonicalA
Turn a Postman collection, an OpenAPI/Swagger spec or an Insomnia export into a YAML definition under .mcp/api/. Give legacyRelativePath for a file in the workspace, or legacyContent with the JSON/YAML text the user pasted. Literal credentials found in headers or auth are moved to .env.mcp.local and replaced by {{VARIABLE}}. Refuses to replace an existing file unless overwrite is true. Apidog: export as OpenAPI or Postman first.
Inputs: overwrite, serviceName, legacyContent, workspaceRoot, +2 more.
discover_legacy_api_sourcesC
Find Postman collections, OpenAPI/Swagger, or Insomnia exports.
Inputs: maxDepth, maxFiles, workspaceRoot.
dry_run_requestB
Build the URL, method, headers and body preview for a requestId without sending the request. Credentials in headers, query string and body are redacted.
Inputs: requestId (required), definitionRelativePath (required), maxBodyChars, workspaceRoot.
execute_api_flowA
Run a sequence of requestIds in a single MCP round trip. Pass inline steps, or flowName to run a flow declared under flows: in the definition. Each step supports retry, poll (via jsonPath), an inline assert (same shape as assert_response), optional, and jsonPathSelect to get that step's body back ($ for all of it) without a follow-up call. stopOnError halts at the first failing required step.
Inputs: definitionRelativePath (required), steps, flowName, stopOnError, +2 more.
execute_api_requestA
Send one request from a YAML definition. Variables resolve YAML < .env.mcp.local < session (after set_environment, {ENV}_KEY wins over KEY); an unresolved placeholder is sent as the literal {{KEY}}. Runs auth_dependency first when needed, retries once on 401, applies capture and assert. The body comes back once: bodyJson for JSON that fits the cap, bodyPreview for text; JSON over the cap gives a truncated bodyPreview plus bodyJson.topLevelKeys — narrow it with jsonPathSelect. responseDetail: minimal = status only; summary = body (8k chars) and the headers that matter; full = every header, 50k chars.
Inputs: requestId (required), definitionRelativePath (required), timeoutMs, maxBodyChars, +3 more.
explain_request_contextC
Summarize YAML keys, .env.mcp.local keys, session keys, and optional endpoint snippet.
Inputs: definitionRelativePath (required), requestId, workspaceRoot.
get_environment_variableA
Read a session variable set by capture or set_environment_variable. Credential-looking values come back as [REDACTED] with masked: true.
Inputs: name (required), workspaceRoot.
import_curlA
Turn a curl command the user pasted into a request in a YAML definition under .mcp/api/ (creates the file, or appends to it). Method, URL, query string, headers and JSON / form body are carried over. Credentials (Authorization, cookies, -u, credential-looking fields) are moved to .env.mcp.local and replaced by {{VARIABLE}}, so the YAML stays safe to commit. Fails if requestId already exists.
Inputs: curl (required), requestId (required), description, workspaceRoot, +1 more.
init_workspaceA
Create the starting structure in a project: the .mcp/api/ folder, a runnable demo definition (demo.yaml, against httpbin.org) when there is no definition yet, an .env.mcp.local template with commented examples, and the .gitignore line that keeps that file out of git. Never overwrites an existing file. Ask the user before calling it.
Inputs: withDemo, workspaceRoot.
list_api_definitionsB
List YAML API definition files under .mcp/api/. Returns files (paths), or entries (path/service/summary) with includeCatalogMeta or query; plus totalMatched, truncated and, when capped by maxFiles, a nextCursor to pass back as cursor.
Inputs: query, cursor, fields, sortBy, +4 more.
plan_vander_checksA
Build the VANDER checklist (Verbs, Authorization, Negative, Data, Errors, Responsiveness) for one requestId, derived deterministically from its definition. Sends nothing. The id prefix is the axis (V-, A-, …). mode: auto (run_vander_checks executes it), destructive (auto, but state-changing) or manual (an idea to explore by hand).
Inputs: requestId (required), definitionRelativePath (required), axes, workspaceRoot.
read_api_definitionA
Read and validate a whole API YAML definition. Prefer summarize_api_definition for triage.
Inputs: definitionRelativePath (required), workspaceRoot.
reorganize_mcp_api_definitionsA
Plan or apply merging multiple YAML definitions into fewer files. mode plan (no writes) or apply (requires confirm true). groupBy manual_groups (requires manualGroups with targetRelativePath + sourceGlobs relative to .mcp/api/) or same_service_field (auto groups files sharing non-empty service into .mcp/api/_consolidated/*.yaml). Conflicts (duplicate endpoint id across files, mismatched base_url/service/variables) block apply. Optional deleteSourcesAfterMerge or moveSourcesToArchive (mutually exclusive).
Inputs: mode (required), groupBy (required), confirm, manualGroups, +3 more.
run_evalA
Evaluate an endpoint whose answer varies (LLM / RAG / search) — or any endpoint over many inputs — in one call. Runs each case repeat times and passes it when passRate of the runs satisfy expect; returns one row per case, the distinct failures, latency p50/p95 and how many different answers came back. With judge it also returns question / answer / retrieved passages for you to grade what code cannot (faithfulness, relevance). Runs do not capture variables or replace the session's last response. Every run is a real, possibly paid, request.
Inputs: definitionRelativePath (required), cases, judge, expect, +9 more.
run_vander_checksA
Execute the automatic VANDER checks for one requestId. Returns one row per check (id, whose prefix is the axis; result pass/fail/skipped; detail with the evidence) and failures with expected/actual/body snippet. Probes vary method, credentials and body; they never capture variables or replace the session's last response. State-changing probes (any POST/PUT/PATCH/DELETE) are skipped unless includeDestructive is true — ask the user before enabling it. maxDurationMs is the budget for the Responsiveness check.
Inputs: requestId (required), definitionRelativePath (required), axes, timeoutMs, +3 more.
set_environmentA
Set the active logical environment for this workspace session (session CURRENT_ENV, e.g. STAGING). Interpolation resolves {CURRENT_ENV}VAR from .env.mcp.local before generic VAR (prefix-first). Put all env secrets there (DEVELOP, STAGING_); YAML stays generic.
Inputs: environment (required), workspaceRoot.
set_environment_variableC
Set a session variable for subsequent requests.
Inputs: name (required), value (required), workspaceRoot.
summarize_api_definitionA
Summarize a canonical API YAML without returning the full definition. Use names_only for triage (optionally with fields to trim endpoint entries); use full only when needed.
Inputs: definitionRelativePath (required), fields, requestId, detailLevel, +1 more.
summon_vanderA
Call this whenever the user addresses Vander by name ("hi Vander", "oi Vander", "Vander, review X") or asks for a VANDER review. Pass workspaceRoot. Returns the Vander persona brief and, when no endpoint was named, an onboarding: what this server is, the three pieces it needs (.mcp/api/ YAML, .env.mcp.local, the conversation), which of them this workspace already has, and what the user can ask for. Adopt the persona for the rest of the conversation and follow its working method. Sends no HTTP request.
Inputs: requestId, workspaceRoot, definitionRelativePath.
upsert_canonical_api_definitionA
Create or update a canonical YAML under .mcp/api/ only. mergeMode replace_file (requires replaceDefinition) or append_endpoints (requires appendEndpoints). New file on append needs createIfMissing. Default dryRun true returns preview without writing; set dryRun false and confirm true to write (atomic replace + optional .bak backup). Endpoint ids must not collide on append.
Inputs: mergeMode (required), targetRelativePath (required), dryRun, confirm, +4 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 21 tools
Most tools target distinct actions (list vs read vs summarize, plan vs run VANDER checks, single request vs flow vs eval). A few pairs overlap in purpose—read_api_definition vs summarize_api_definition, convert_legacy_to_canonical vs import_curl—but descriptions provide clear guidance on when to use each.
Nearly all names follow a snake_case verb_noun pattern (execute_api_request, plan_vander_checks, set_environment_variable). Minor deviations such as set_environment vs set_environment_variable and the whimsical summon_vander keep it from being perfectly uniform, but the convention is still predictable.
21 tools is heavy for the domain; the server covers definition management, legacy conversion, execution, evals, VANDER checks, environment handling, and workspace init. Several pairs could arguably be merged (read/summarize, plan/run VANDER), making the count borderline rather than well-scoped.
The surface covers create/read/update via upsert, list, summarize, legacy conversion, request execution, assertions, flows, evals, VANDER checks, environment variables, and workspace init. A notable gap is an explicit delete tool for definitions or endpoints; upsert with replace_file can overwrite but does not obviously delete.
Maintenance
Related MCP Connectors
AI-callable tools for API mocking, testing, monitoring, security, and automation.
End-to-end API testing — generate and run tests from OpenAPI, curl, Postman, or real user traffic.
Free, reviewed pilot registry of reusable API knowledge and recipes for AI agents.
Give your AI hands. Identity, credential vault, and API gateway for autonomous agents.
Related MCP Servers
AlicenseAqualityBmaintenanceA deterministic eval engine for coding agents. Your agent claims it fixed the bug—this checks it. The same scenario runs twice against a sandbox of the services your code calls, with failures injected on purpose. It must fail on the old code and pass on the new. The verdict is an exit code, not a model's opinion. Every run leaves a receipt.19442 npm1MIT- AlicenseNot gradedqualityDmaintenanceEnables AI agents to inspect, test, validate, compare, and debug HTTP APIs through a structured toolkit, with features such as secret redaction, response contract validation, regression comparison, health checks, and incident investigation workflows.MIT
- AlicenseAqualityCmaintenanceEnables AI agents to call real API endpoints and verify responses against OpenAPI specs, supporting contract testing, auth presets, spec diffing, and health checks.854 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables agent-planned, human-reviewed HTTP API testing through MCP, turning OpenAPI/Swagger definitions and requirements into executable Hurl/k6 test plans. Supports plan review, execution, cleanup, and reports for coding agents such as Codex and Claude Code.MIT