Skip to main content
Glama
juliodelimas

jmeter-mcp-server

by juliodelimas

jmeter-mcp-server

npm version CI Node.js TypeScript License: MIT

Give an LLM real, deterministic control over Apache JMeter — build test plans, run real load tests, and read back real results, without ever hand-writing .jmx XML or opening the GUI.

"Load test my API: 20 users hitting POST /orders for 2 minutes,
 5% think time, fail anything over 800ms"

...turns into a running JMeter test and a real report, through typed tool calls an MCP client (Claude Code, Claude Desktop, etc.) makes directly.

Why not just ask an LLM to write the .jmx itself?

It can — a .jmx is just XML, and any capable model has seen plenty of JMeter test plans. The problem is how it fails: JMeter's format is a hashTree with dozens of fragile, easy-to-misremember details — exact guiclass/testclass pairs, property names that don't match their GUI label (ThreadGroup.num_threads is a stringProp, not an intProp), integer bitmasks for assertion match types, strict parent/child pairing with sibling <hashTree> tags. None of it is self-checking. A wrong value still produces valid, loadable XML that just quietly does the wrong thing.

That's not hypothetical — it happened building this project. An early version of the If Controller generated a property called useExpression set to true, which reads like "yes, evaluate my condition." The real JMeter source does the opposite: useExpression=true means don't evaluate it as an expression — just check if the string is literally "true". Every non-trivial condition silently, permanently failed. No error, no warning — the child sampler just never ran. It only surfaced by actually executing the generated plan against real JMeter and noticing a sample count of zero.

That's the whole case for this server in one story: an LLM regenerating XML from memory re-risks that exact mistake on every single request. This server encodes the correct shape once, in a serializer (and a matching parser for the reverse direction) checked against real JMeter source and bundled examples, exercised by 166 automated tests including real JMeter runs — and exposes it as typed tools instead. Concretely:

  • Correctness through one tested code path, not regenerated-from-memory XML every time.

  • Cheap incremental edits. Plans are a small JSON tree with stable node ids — adding, removing, renaming, moving, or disabling an element is one tool call by id, not rewriting a whole .jmx file. An existing .jmx (hand-written or exported from the GUI) can be imported and edited the same way.

  • Aggregated results, not raw samples. get_execution_report returns computed stats (error %, avg/median/p90/p95/p99, throughput) — not thousands of sample rows to average by hand.

  • Real async execution. execute_test_plan returns immediately with an executionId; long-running load tests never block anything.

The generated .jmx is standard JMeter output — open it in the real GUI any time.

Related MCP server: JMeter MCP Server

Example

You:    Build a load test: 10 users for 30s hitting GET https://api.example.com/health,
        fail anything that takes over 500ms, then run it and tell me the p95.

Claude: [create_test_plan, add_thread_group, add_http_sampler, add_duration_assertion,
         add_aggregate_report_listener, execute_test_plan, get_execution_status, get_execution_report]

        Ran 300 requests over 30s, 0 failures. p95 latency: 214ms, avg: 187ms, throughput: 10.1 req/s.

Every step above is a real typed MCP tool call — see Tools for the full set (34 element types across samplers, controllers, timers, extractors, assertions, and listeners, plus editing, inspection, and .jmx import/export tools) and Example workflow for the raw call sequence.

Quick start

claude mcp add jmeter \
  -e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
  -- npx -y jmeter-mcp-server

That's it — no cloning, no build. Adjust JMETER_HOME to your JMeter install (see Prerequisites). Full setup details, Claude Desktop config, and local-dev instructions are in Adding this server to Claude Code.

How a test plan is represented

Each plan is a JSON tree ({id, type, props, children[]}), not XML text. Authoring tools append a child under a given parentId, editing tools (remove_element, update_element, move_element, etc.) mutate that same tree in place, and the tree is only serialized to a real .jmx file on demand (get_test_plan_xml) or at execution time. import_test_plan runs the reverse direction, parsing an existing .jmx back into this same tree shape. This is what makes incremental edits cheap and keeps all the fiddly XML schema knowledge in two places (src/jmx/serializer.ts for tree → XML, src/jmx/parser.ts for XML → tree, sharing prop shapes from src/jmx/propTypes.ts) instead of spread across every tool.

Tools

Authoring (each returns the new node's id, used as parentId for whatever you attach under it next) — grouped the same way JMeter's own right-click Add menu groups them, so if you already know the GUI, you already know where to look:

Tool

Adds

create_test_plan

Root TestPlan node — returns planId and the root node id

Threads (Users):

Tool

Adds

add_thread_group

Thread Group (virtual users)

add_setup_thread_group

setUp Thread Group (runs once before all Thread Groups)

add_teardown_thread_group

tearDown Thread Group (runs once after all Thread Groups)

Sampler:

Tool

Adds

add_http_sampler

HTTP Request sampler

add_jdbc_request

JDBC Request sampler

add_jsr223_sampler

JSR223 Sampler (Groovy/BeanShell/JS/JEXL script as the sample)

add_ftp_request

FTP Request sampler

add_tcp_sampler

TCP Sampler

Logic Controller:

Tool

Adds

add_transaction_controller

Transaction Controller (groups child samplers into one named transaction)

add_loop_controller

Loop Controller (repeats child samplers)

add_if_controller

If Controller (conditionally runs child samplers)

add_while_controller

While Controller (repeats children while a condition holds)

add_random_controller

Random Controller (runs one random child per pass)

add_interleave_controller

Interleave Controller (alternates through children)

Config Element:

Tool

Adds

add_csv_data_set

CSV Data Set Config (parameterization from a file)

add_user_defined_variables

User Defined Variables

add_jdbc_connection_configuration

JDBC Connection Configuration (pooled datasource)

add_http_request_defaults

HTTP Request Defaults

add_cookie_manager

HTTP Cookie Manager

add_header_manager

HTTP Header Manager

Timer:

Tool

Adds

add_constant_timer

Constant Timer (pacing/think-time)

add_uniform_random_timer

Uniform Random Timer (randomized pacing)

add_constant_throughput_timer

Constant Throughput Timer (target rate pacing)

Pre Processors:

Tool

Adds

add_jsr223_preprocessor

JSR223 PreProcessor

add_user_parameters

User Parameters (per-thread variable value sets)

Post Processors:

Tool

Adds

add_json_extractor

JSON Extractor post-processor

add_regex_extractor

Regular Expression Extractor post-processor

add_xpath_extractor

XPath Extractor post-processor

add_jsr223_postprocessor

JSR223 PostProcessor

Assertions:

Tool

Adds

add_response_assertion

Response Assertion

add_json_assertion

JSON Assertion (JSONPath validation)

add_duration_assertion

Duration Assertion (response-time SLA)

add_size_assertion

Size Assertion (response byte-size check)

Listener:

Tool

Adds

add_aggregate_report_listener

Aggregate Report listener

add_summary_report_listener

Summary Report listener

add_view_results_tree_listener

View Results Tree listener (full request/response capture for debugging)

add_backend_listener

Backend Listener (streams live metrics to InfluxDB/Graphite/etc.)

Editing (mutate an already-built plan):

Tool

Purpose

remove_element

Remove an element (and its subtree); rejects removing the root TestPlan node

update_element

Shallow-merge (or replace) a node's props; a prop value of null deletes that key. Validated against the node's type when known

rename_element

Rename an element's testname

move_element

Move an element (and its subtree) to a new parent, optionally at a specific index; rejects moving a node into its own subtree

reorder_children

Reorder a node's direct children (must pass an exact permutation of the current children)

set_element_enabled

Enable/disable an element without removing it

Inspection:

Tool

Purpose

list_test_plans

List every plan in the workspace

get_test_plan

Full element tree of a plan, including every node's id

get_test_plan_xml

Serialize a plan to its JMeter .jmx XML, without running JMeter

import_test_plan

Import an externally authored .jmx (e.g. exported from the JMeter GUI) as a new plan. Element types this server doesn't model are kept as opaque UnknownElement nodes instead of being dropped

Execution & reporting (async — a run happens in the background):

Tool

Purpose

execute_test_plan

Serialize to .jmx and run JMeter in non-GUI mode; returns { executionId } immediately

get_execution_status

running / completed / failed, plus a tail of the JMeter log

stop_execution

Send SIGTERM to a running JMeter process

get_execution_report

Aggregated stats (per label + overall) parsed from the run's JTL output

Example workflow

create_test_plan            → { planId, rootNodeId }
add_thread_group             (parentId: rootNodeId)  → { nodeId: threadGroupId }
add_http_sampler              (parentId: threadGroupId) → { nodeId: samplerId }
add_response_assertion        (parentId: samplerId)
add_aggregate_report_listener (parentId: threadGroupId)
execute_test_plan             (planId) → { executionId }
get_execution_status           (executionId)   ← poll until "completed"
get_execution_report            (executionId) → aggregated latency/error stats

Testing

166 automated tests, no framework beyond Node's built-in test runner:

npm test               # 155 tests: tree-mutation and XML-shape unit tests, XML -> tree parsing,
                        # serialize -> parse round-trips, and every tool called over the real MCP
                        # protocol (stdio, the same way Claude Code/Desktop talk to it) - no
                        # JMeter install needed, fully hermetic
npm run test:integration  # 11 tests: real JMeter runs - the If Controller story above, While
                        # Controller loop counts, timer pacing, extractors, assertions, etc.
                        # (needs JMETER_HOME)
npm run test:all

npm test spawns the actual built server (dist/index.js) via StdioClientTransport and drives it exactly as a real client would — not just calling internal functions — so a broken tool schema or a malformed response shows up as a real protocol error, not a passing unit test.

Both suites run on every push and pull request via GitHub Actions — the integration job installs a real JMeter binary on the runner, so it's exercising the same code path as a local run, not a mock.

Prerequisites

  • Node.js 18+

  • JMeter installed locally, with the JMETER_HOME environment variable pointing at the installation directory (the one containing bin/jmeter). On macOS via Homebrew, brew install jmeter puts it at /opt/homebrew/opt/jmeter/libexec.

Adding this server to Claude Code

No cloning or building required; npx fetches and runs the published version on the fly:

claude mcp add jmeter \
  -e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
  -- npx -y jmeter-mcp-server

Adjust the JMETER_HOME path to wherever JMeter is installed on your machine. Optionally set JMETER_MCP_WORKSPACE too (see below) if you want plans and executions stored somewhere other than the default.

The default scope is local (this project directory only). To make it available across every project, add -s user:

claude mcp add jmeter -s user \
  -e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
  -- npx -y jmeter-mcp-server

Confirm it registered and is responding:

claude mcp list

From a local clone (development)

If you're working on this repository's code instead of using the published package, point at the built dist/index.js directly:

npm install
npm run build
claude mcp add jmeter \
  -e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
  -- node /absolute/path/to/jmeter-mcp-server/dist/index.js

Claude Desktop

Add this to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "jmeter": {
      "command": "npx",
      "args": ["-y", "jmeter-mcp-server"],
      "env": {
        "JMETER_HOME": "/opt/homebrew/opt/jmeter/libexec"
      }
    }
  }
}

Note: unlike a terminal-launched app, Claude Desktop does not inherit environment variables exported in your shell profile (.zshrc, etc.) — only true system-wide ones. Always set JMETER_HOME explicitly in the env block above rather than relying on it already being "set on your machine".

Environment variables

Variable

Required

Purpose

JMETER_HOME

Yes

JMeter installation directory (must contain bin/jmeter)

JMETER_MCP_WORKSPACE

No

Where plans and executions are stored. Defaults to ./jmeter-workspace relative to wherever the server process starts

Workspace layout

<workspace>/
  plans/<planId>/plan.json           # JSON tree — source of truth for a plan
  executions/<executionId>/
    generated.jmx                    # serialized at execute_test_plan time
    aggregate-report.jtl             # output of the Aggregate Report listener, if present
    summary-report.jtl               # output of the Summary Report listener, if present
    jmeter.log
    meta.json                        # execution status, pid, timestamps, exit code

Editing and importing plans

Beyond the add_* authoring tools, a plan can be mutated after the fact (remove_element, update_element, rename_element, move_element, reorder_children, set_element_enabled) and an externally authored .jmx (e.g. exported from the JMeter GUI) can be brought in with import_test_plan. import_test_plan understands the most common element types (thread groups, HTTP samplers, assertions, extractors, controllers, config elements, the three report listeners, etc.); anything it doesn't recognize is kept as an opaque UnknownElement node whose original XML is preserved and re-emitted as-is by get_test_plan_xml/execute_test_plan, instead of being dropped - import_test_plan's response reports unknownElementCount/ unknownElementTypes so you know what wasn't fully understood. Coverage can be extended incrementally in src/jmx/parser.ts.

v1 scope

Not yet supported (candidates for a future release): generating the HTML dashboard report (-e -o), parent-type validation on add_*/move_element/ import_test_plan (nothing stops attaching an element under a semantically wrong parent), distributed execution.

Note on add_csv_data_set: the filename must be an absolute path. execute_test_plan runs JMeter from a fresh per-execution directory, so a relative path (which JMeter's GUI would resolve against the .jmx file's own location) won't resolve there. An absolute path baked into a plan is also machine-specific — it won't travel if you share plan.json with someone on a different machine. This is only checked at creation time: add_csv_data_set rejects a relative or nonexistent path up front, but later changing a CSVDataSet's filename via update_element, or importing a .jmx that already has a relative one via import_test_plan, is not checked - it will only surface as a failure at execute_test_plan time.

Note on add_jdbc_request/add_jdbc_connection_configuration, add_ftp_request, and add_backend_listener: these generate correct, JMeter-loadable XML, but exercising them for real needs infrastructure this project doesn't provide (a database, an FTP server, an InfluxDB/Graphite instance) — they were verified structurally, not against a real backend.

Note on add_view_results_tree_listener's captureFullData option: it has no effect right now. execute_test_plan always runs JMeter with -Jjmeter.save.saveservice.output_format=csv, and JMeter's CSV writer never emits response body/header columns no matter what the SampleSaveConfiguration flags say — only its XML output format can carry full response bodies. The option is wired up correctly in the generated .jmx (verified: the flags really do flip in the XML) for the day this server supports XML-format runs, but until then it's a no-op — confirmed by running a real capture and checking the resulting JTL has no responseData/samplerData/ requestHeaders/responseHeaders columns regardless of the setting.

Note on add_tcp_sampler: server/port/request are live-verified. The numeric fields (connectTimeoutMs, timeoutMs) are rendered as stringProp following this project's general convention for sampler numeric fields, but that specific choice for TCPSampler wasn't confirmed against a real JMeter-GUI-saved example (none was available to check against) — flagging in case a real save turns out to expect intProp.

Roadmap

Ideas being explored for future releases — none of these are implemented yet:

Proposed tool

What it does

Why it's worth it

find_breaking_point

Automatic binary-search capacity finder: ramps thread count up/down on its own, run after run, until it finds the concurrency level that violates your SLA (p95 latency, error %)

JMeter has no native "find the limit" feature. Driving this search through raw LLM tool calls costs ~6-8 calls per round (adjust load, run, poll, read report, decide) across the several rounds a binary search needs

detect_bottleneck_class

Fits Little's Law / the Universal Scalability Law to collected concurrency vs. throughput vs. latency data, and classifies the bottleneck as contention, coherency, or saturation

JMeter only outputs raw numbers; re-deriving a queueing-theory curve fit through prose reasoning would mean reimplementing nonlinear regression by hand for every question

detect_soak_drift

Runs linear regression over the latency/error time series of a long-duration soak test to separate normal noise from a real trend (the classic memory-leak signal)

JMeter's graph shows the curve, but doesn't say whether it's a statistically real degradation or just noise

compare_execution_reports

Statistical diff across N executions (not just two), with a significance test for whether a p95 shift is real or noise

A ready-made performance regression gate for CI, instead of someone eyeballing two JSON reports and guessing

isolate_warmup_window

Automatically detects where warm-up (JIT, connection pools, cold caches) ends, and recomputes metrics only over the steady-state window

Today's overall average is polluted by the first few seconds of a run; JMeter doesn't separate this on its own

classify_error_flakiness

Re-runs failed samples with backoff and separates "real system error" from "one-off flake" (network timeout, etc.)

Produces a trustworthy error rate for a CI gate, instead of an errorPct that mixes both kinds together

orchestrate_distributed_run

Runs the same test plan across multiple JMeter injector nodes (master-slave, via -R or independent engines) and merges every node's .jtl into a single aggregated report

JMeter supports distributed mode, but wiring up remote engines, RMI ports, matching JMeter versions, and merging results across machines is entirely manual today — nobody sets this up for a quick test

How this compares

Checked against the other public JMeter MCP servers found on GitHub as of September 3, 2026 — open-source repositories with at least a README description (undocumented forks/clones excluded). Columns reflect what each project's own README documents, not independent verification of its internals.

Server

Approach

.jmx round-trip

Edit by id

Async run

Tests

Real JMeter

jmeter-mcp-server (this project)

JSON tree, 34 element types, authored and edited by node id

166, incl. real runs

QAInsights/jmeter-mcp-server

Runs an existing .jmx and analyzes results — doesn't author plans

aravindksk7/Jmeter-MCP

Generates a .jmx from parameters; no re-import of existing plans

chandanvars/jmeter-mcp-server

Generates a whole plan from one JSON payload, runs it via Docker

perfsage/perfsage-jmeter-mcp

Heals the local JMeter runtime, imports HAR/OpenAPI traffic, discovers capacity

shruthi-r18/ClaudeCode_MCP_QA_Automation_Performance

Demo project for a tutorial video, 11 fixed tools

KenLin-7/jmeter-mcp-server

Reimplements HTTP load testing in Node — no Apache JMeter underneath

MUYU0615/jmeter_mcp_server

Generates a .jmx with basic parameters (threads, ramp-up, duration)

vjgit-369/JmeterDemoUsingMCP

Demo script with parameters hardcoded in source, not a general-purpose server

Every capability in that table shows up somewhere across the other eight projects, individually: real JMeter execution, .jmx generation, traffic import. This is the only one that combines full .jmx round-trip (import and export), per-id incremental editing, non-blocking async execution, and a documented automated test suite in one place.

License

MIT

Available Tools

14 tools
add_aggregate_report_listenerA

Add an Aggregate Report listener under the given parent (Thread Group or TestPlan). Its output is what get_execution_report reads after a run.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoAggregate Report
planIdYes
filenameNoFilename kept in the .jmx for portability; ignored at execution time
parentIdYes

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must carry full behavioral transparency. It only indicates an 'add' operation without disclosing side effects (e.g., modifying the plan), requirements, reversibility, or safety considerations. The relationship to get_execution_report is mentioned, but no side effects are revealed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. The core action and the key dependency are stated directly and in a logical order.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description provides the essential role (feeds get_execution_report) and parent context, but lacks detail on required parameters (planId, parentId remain ambiguous) and does not disclose mutation side effects. For a tool with 4 parameters and no annotations, it is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, so the description must compensate. It clarifies parentId (Thread Group or TestPlan) and the tool's purpose, but does not explain planId or name beyond the schema's default, nor does it add much to filename. Compensation is partial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (add) and a resource (Aggregate Report listener), and specifies the parent types (Thread Group or TestPlan). It also links the listener's output to get_execution_report, which distinguishes it from other listener tools like add_summary_report_listener.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool (to produce output for get_execution_report), but it does not explicitly name alternatives or provide when-not-to-use guidance. The context is helpful but not as explicit as it could be.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_header_managerB

Add an HTTP Header Manager under an HTTP sampler (or a Thread Group, to apply to every sampler in it).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoHTTP Header Manager
planIdYes
headersYes
parentIdYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only states that it adds a header manager, but does not mention any side effects (e.g., modifying the test plan), required permissions, idempotency, or what happens if the parent doesn't exist. For a mutation tool, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no filler. It is front-loaded with the action and placement, making it easy to scan. However, the brevity contributes to the lack of parameter and behavior detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given four parameters with zero schema descriptions, no output schema, and no annotations, the description should provide substantial guidance. It only covers the core purpose and placement, leaving out parameter semantics, required IDs, header structure, and any behavioral caveats. The tool is underspecified for reliable agent invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain any of the four parameters (planId, parentId, headers, name). It fails to compensate for the schema's lack of documentation, leaving agents to guess the meaning and structure of the headers array and required IDs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (add an HTTP Header Manager) and specifies the target (under an HTTP sampler or Thread Group), which distinguishes it from sibling tools that add other elements. The placement detail adds precision beyond a generic 'add' tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides placement guidance (under sampler vs. Thread Group) which implicitly tells when to use it for scoping headers to all samplers. It does not explicitly mention alternatives or when not to use it, but there are no similar sibling tools that add headers, so this is adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_http_samplerC

Add an HTTP Request sampler under the given parent node (usually a Thread Group).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
pathYesRequest path, e.g. /v1/users
portNo
domainYesHost name, e.g. api.example.com
methodYes
planIdYes
bodyJsonNoRaw JSON request body, if any
parentIdYes
protocolNohttps

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavior. It only says 'Add', implying mutation, but doesn't mention side effects, requirements (like parent existence), or error behavior. This is a significant gap for a mutating operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that immediately conveys the primary action and target. There is no filler or redundancy; every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 9 parameters, 6 required, and no output schema or annotations, the description is grossly incomplete. It doesn't explain how to specify the HTTP request (method, domain, path, etc.) or what the parent node requirements are, leaving the agent without enough context to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the description adds nothing about any of the 9 parameters. It fails to compensate for the undocumented parameters (name, method, planId, parentId, protocol, etc.), leaving the agent to guess their meaning and usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Add an HTTP Request sampler' and specifies the location 'under the given parent node'. It clearly distinguishes from siblings like add_thread_group or add_json_extractor by naming the sampler type, though it doesn't explicitly contrast them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides minimal context by noting the parent is 'usually a Thread Group', but gives no guidance on when to use this tool versus alternatives like add_header_manager or add_response_assertion. No exclusions or conditions are mentioned, leaving the agent to infer when it applies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_json_extractorA

Add a JSON Extractor post-processor under an HTTP sampler, to save a value from the JSON response into a variable.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
planIdYes
parentIdYesId of the HTTP sampler node this extractor applies to
defaultValueNoValue to use if the JSONPath doesn't matchNOT_FOUND
jsonPathExprYesJSONPath expression, e.g. $.data.id
referenceNameYesJMeter variable name to store the extracted value in

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must fully disclose behavioral traits. It only says 'Add', implying a mutation, but does not explain side effects like whether existing extractors are replaced, what happens if the parent sampler does not exist, or whether the operation is reversible. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tightly worded sentence that front-loads the action and purpose. Every word earns its place, with no redundant information or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is adequate but not complete. It explains the main use case, but does not mention potential behavioral details (e.g., failure handling, idempotency) or prerequisites beyond the parent sampler. The missing parameter descriptions for name and planId are not addressed. A more comprehensive description would help for a mutation operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67% (4 of 6 parameters have descriptions). The description adds no additional parameter-specific details beyond what the schema already provides; it merely restates the purpose which maps to jsonPathExpr and referenceName. Since the schema covers most parameters, the baseline of 3 is appropriate, and the description does not compensate for the undocumented name and planId parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Add'), a clear resource ('JSON Extractor post-processor'), a location context ('under an HTTP sampler'), and the purpose ('save a value from the JSON response into a variable'). This clearly distinguishes it from sibling add_* tools like add_http_sampler or add_response_assertion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly indicates when to use this tool: when you need to add a JSON extraction post-processor. It provides context about the parent sampler. However, it does not explicitly state when not to use it or mention alternatives (e.g., if you need a different extractor type), but given the sibling list, the purpose is distinct enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_response_assertionC

Add a Response Assertion under an HTTP sampler, to fail the sample if the response doesn't match.

ParametersJSON Schema
NameRequiredDescriptionDefault
notNoNegate the match (assert the pattern does NOT match)
nameNoResponse Assertion
planIdYes
parentIdYes
patternsYes
matchTypeNocontains
testFieldNoresponse_data

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states that the tool fails the sample when the response doesn't match, which is the primary side effect. However, it does not mention constraints (e.g., parent must be an HTTP sampler), error conditions, or any side effects on existing assertions. The core behavior is clear but details are missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the action and purpose without unnecessary words. It is concise and efficient, scoring well for conciseness, though it could be expanded slightly with parameter guidance without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 7 parameters, no output schema, and low schema description coverage, the description is inadequate. It doesn't explain the required parameters (planId, parentId, patterns) or optional ones (matchType, testField, name, not), nor does it describe the return behavior. An agent would need to infer too much from the schema alone, making the description incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 14% (only 'not' has a description), so the description must compensate. It does not explain planId, parentId, patterns, matchType, testField, or name. The phrase 'if the response doesn't match' implies patterns are used for matching but offers no parameter semantics. The agent is left to guess from property names and enums, which is insufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: adding a Response Assertion under an HTTP sampler, with the purpose of failing the sample on mismatch. This is specific and distinguishes it from sibling add_* tools (e.g., add_http_sampler, add_json_extractor) by naming the resource and behavior. However, it doesn't explicitly contrast with alternatives, so a 4 is appropriate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus others, nor any conditions where it should not be used. It only states what the tool does without context on selection. No exclusions or alternatives are mentioned, leaving the agent to infer appropriateness from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_summary_report_listenerB

Add a Summary Report listener under the given parent (Thread Group or TestPlan). Its output is what get_execution_report reads after a run.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoSummary Report
planIdYes
filenameNoFilename kept in the .jmx for portability; ignored at execution time
parentIdYes

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden for behavioral disclosure. It does mention that the listener's output is what get_execution_report reads, which is useful context, but it does not disclose any side effects, permissions, or reversibility of adding the listener. As a mutation tool, this is a notable gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no filler. The primary action is front-loaded, and the follow-up sentence adds valuable context about the tool's relationship to get_execution_report. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 4 parameters and no annotations or output schema, the description is too thin. It clarifies the parent types but omits details on planId and the return value. The link to get_execution_report is helpful, but the description does not adequately equip an agent to invoke the tool correctly with all parameters understood.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, so the description must compensate for the undocumented parameters. It provides some context for parentId by naming allowed parent types, but planId and name are not explained. The description does not clarify the meaning or format of the parameters beyond what is already in the schema, which is minimal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: adds a Summary Report listener under a parent (Thread Group or TestPlan). It also connects it to get_execution_report, which helps place its role. It does not explicitly contrast with the sibling add_aggregate_report_listener, so a 4 rather than 5 is appropriate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Some usage guidance is implied by mentioning the valid parents (Thread Group or TestPlan), but the description does not state when to choose this listener over the aggregate report alternative, nor any prerequisites or exclusions. Sibling tools exist but no comparison is made, so guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_thread_groupC

Add a Thread Group (virtual users) under the given parent node (usually the TestPlan root).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
loopsNoNumber of loop iterations per thread, -1 for infinite
planIdYes
parentIdYesId of the node to attach this thread group under
numThreadsYesNumber of concurrent virtual users
durationSecondsNoIf set, run on a scheduler for this many seconds instead of a fixed loop count
rampTimeSecondsYesSeconds to reach full thread count

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only states that a thread group is added and implies mutation, but does not mention side effects, permissions, reversibility, or what happens on success/failure. The brief note about the parent node being usually the TestPlan root is the only extra context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words. It is front-loaded with the action and resource, and the parenthetical clarification adds useful context without bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 7 parameters and no output schema, the description is sparse. It does not mention key parameters like planId, nor the relationship between loops and durationSeconds (though the schema mentions it). It also lacks any indication of what the tool returns or how the agent should interpret the result. The description is insufficient for full correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes 5 of 7 parameters, providing explanations for parentId, numThreads, rampTimeSeconds, loops, and durationSeconds. The description adds minimal value by mentioning 'given parent node' which relates to parentId, but does not clarify name or planId. Given the high schema coverage, a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (Add) and the resource (Thread Group), and clarifies 'virtual users'. It also indicates the placement under a parent node. However, it does not explicitly differentiate from sibling add_* tools, though the resource type is distinct enough that an agent might infer the difference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives. It only mentions the typical parent node, but does not specify when to prefer this over other nodes or tools. There are no exclusions or alternative tool references.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_planA

Create a new JMeter test plan. Returns the planId and the id of its root TestPlan node, which you'll use as parentId for the first thread group.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesHuman-readable name for the test plan

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It discloses the return values (planId and root node id) and hints at their usage, but does not mention side effects, persistence, or any prerequisites. It is adequate but not rich enough for full transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero redundancy. It front-loads the core action, then concisely provides the critical return information and usage hint. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple create operation with a single parameter and no output schema, the description explains the return values and their practical use. It is complete for the immediate purpose, though it could mention potential side effects or idempotency, but that does not detract significantly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes the only parameter 'name' with an adequate human-readable description (100% coverage). The tool description adds no extra detail about the parameter, so it stays at the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a new JMeter test plan' with a specific verb and resource. It further differentiates from siblings by noting that it returns planId and the root TestPlan node id, which is unique to this tool among the listed siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on how to use the return value ('which you'll use as parentId for the first thread group'), implying it is the first step in building a plan. It does not explicitly contrast with alternatives like add_thread_group, but the usage context is evident and sufficient for a create-first tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

execute_test_planA

Start running a test plan with JMeter in non-GUI mode. Returns immediately with an executionId; the run continues in the background. Poll get_execution_status to know when it's done, then call get_execution_report to read the results.

ParametersJSON Schema
NameRequiredDescriptionDefault
planIdYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries full responsibility. It discloses the asynchronous execution model, the immediate return of an executionId, and the necessity to poll status and retrieve results. This is critical behavioral information that an agent needs to know to avoid blocking or assuming synchronous completion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action. The second sentence packs the async behavior and follow-up steps without waste. Every sentence adds value; there is no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter asynchronous tool with no output schema, the description covers the essential operational model: what it starts, how it returns, and the required polling/report steps. It leaves no critical ambiguity for an agent to execute correctly. The mention of non-GUI mode adds practical context without going overboard.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter, planId, with 0% schema description coverage. The description does not directly explain the parameter, but its name and the tool's reference to 'test plan' make it self-explanatory as the ID of the plan to run. While the description does not add extra meaning, the parameter's purpose is evident from context. Given the low coverage, a 3 is appropriate – it does not harm, but could be more explicit about where to get planId (e.g., from list_test_plans).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Start running a test plan'), the resource ('a test plan'), the mode (non-GUI), and the immediate return with an executionId. It distinguishes itself from siblings by focusing on the initiation of execution, while get_execution_status and get_execution_report are for monitoring/results, and create_test_plan is for creation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs the agent on the follow-up workflow: 'Poll get_execution_status to know when it's done, then call get_execution_report to read the results.' This tells when to use this tool (to start) and what to use next. It also states the async nature ('returns immediately... continues in the background'), providing clear context for correct usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_execution_reportA

Read and aggregate the results of a finished (or still-running) execution, computed from its Aggregate Report / Summary Report listener output: per-label and overall count, error rate, avg/min/max/median/p90/p95/p99 latency, throughput and KB/sec.

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the operation is read-only ('Read'), the data source (listener output), and that it can be called on running executions. It does not mention potential side effects, latency, or error handling for invalid IDs, but for a read operation this is acceptable. It adds context about the metrics but not deeper behavioral details like caching or consistency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence that front-loads the core purpose and immediately lists the output metrics. Every word contributes value; there is no fluff or repetition. It is concise without losing specificity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one parameter and no output schema, the description covers the essential elements: what it does, when it can be used (finished or running), and exactly which metrics it returns. The lack of an output schema is mitigated by the metrics list. It does not mention error conditions or return formatting, but that is a minor gap given the simplicity of the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has a single parameter, executionId, with 0% description coverage. The tool description does not explicitly explain what the parameter means beyond its name—it only says the tool retrieves results for 'an execution', but does not reiterate that executionId must be the ID of the execution whose report is desired. Since schema coverage is zero and the description fails to compensate, the parameter semantics are under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Read and aggregate') with a clear resource ('results of a finished (or still-running) execution') and enumerates the exact metrics returned (per-label and overall count, error rate, latency percentiles, throughput, KB/sec). It distinguishes itself from siblings like get_execution_status by focusing on detailed report data from listener output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states the tool works on finished or still-running executions and specifies the source (Aggregate Report / Summary Report listener output), giving context for when it applies. However, it does not explicitly contrast it with siblings like get_execution_status, nor does it state when NOT to use it (e.g., for just checking status). Usage is implied but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_execution_statusA

Check the status of a test run started with execute_test_plan (running/completed/failed), plus the tail of its log.

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that it returns status plus log tail, which is the core behavior, and implies read-only nature. However, it does not mention potential side effects (none expected), polling behavior, or log length limits, so it is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, well-structured sentence that front-loads the purpose and includes essential details (statuses, log tail). No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple status-check tool with one parameter and no output schema, the description covers what it does and what it returns. It does not explain how to interpret the status (e.g., polling until completed), but that could be considered usage guidance rather than essential for a correct call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, and the description does not explain the executionId parameter beyond its obvious relation to the run. It does not specify that executionId is the ID returned by execute_test_plan, which would have compensated for the coverage gap. The meaning is inferred but not stated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks the status of a test run, specifies it is for runs started with execute_test_plan, and enumerates the possible statuses (running/completed/failed) plus the log tail. This distinguishes it from siblings like get_execution_report (which likely provides full report) and stop_execution (which stops the run).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly ties usage to runs from execute_test_plan, which gives clear context on when to invoke it. However, it does not mention when not to use it or point to alternatives like get_execution_report for full results, so it lacks exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_planA

Get the full element tree of a test plan, including every node's id (needed as parentId for add_* tools) and type.

ParametersJSON Schema
NameRequiredDescriptionDefault
planIdYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It clearly indicates this is a retrieval operation ('get') and specifies what it returns, which is enough to infer it is read-only and non-destructive. It does not mention potential side effects or response size, but for a getter with a single parameter, this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clearly structured sentence that front-loads the primary action ('Get the full element tree') and then adds contextual detail (ids and their use). Every word serves a purpose with no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read tool with one parameter and no output schema, the description fully explains what is returned (the full element tree, including ids and types) and why it matters (for parentId). It gives an agent enough to decide when and how to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one parameter (planId) with 0% description coverage. The description does not explicitly explain what planId is, though it is easily inferred as the identifier of the test plan from the context. It adds no direct parameter detail, but the single, self-explanatory parameter name mitigates the weak coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Get the full element tree') and a specific resource ('a test plan'), and clarifies it returns every node's id and type. This distinguishes it from siblings like list_test_plans (which lists plans) and get_execution_report (which reports execution results), making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly mentions that node ids are needed as parentId for add_* tools, giving a clear reason to use this tool before making structural modifications. It does not explicitly say when not to use it, but the context of adding nodes is clear, and the sibling tools for execution vs. structure are implicitly separated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_test_plansA

List all test plans in the workspace.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that it lists test plans, without indicating whether the operation is read-only, if there are side effects, rate limits, or pagination behavior. For a potentially safe listing operation, the lack of any safety or side-effect context is a gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single efficient sentence that clearly states the action and scope. It is front-loaded with the verb and resource, avoiding any extraneous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (no parameters) and lack of an output schema, the description provides adequate context for most use cases. However, it does not detail the return format (e.g., list of test plan IDs, full objects, or metadata), which could be relevant for agents expecting a specific structure. Since no output schema exists, a brief note on return content would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. No parameter documentation is needed, and the description adds nothing beyond the schema, which is appropriate since there is nothing to clarify.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb ('List') and a specific resource ('all test plans in the workspace'). It naturally distinguishes itself from sibling tools like get_test_plan (which targets a single plan) and create_test_plan (which creates), so an agent can identify its purpose without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for enumerating all test plans, but it does not explicitly state when to use it versus alternatives like get_test_plan for a specific plan. No exclusions or conditions are mentioned, so the guidance remains implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_executionA

Stop a running test execution (sends SIGTERM to the JMeter process).

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of disclosing behavior. It does reveal the internal action (SIGTERM), but omits consequences such as whether the stop is graceful, what happens if the execution is already stopped, or whether any cleanup occurs. It also does not mention potential side effects or error states. The provided detail is useful but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tight sentence with no filler. It front-loads the primary action and includes the key behavioral detail (SIGTERM) without redundancy. Every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and no output schema. The description covers the core action and mechanism, but given the absence of annotations and output schema, it could state what the caller should expect (e.g., a success status or error if execution not found). It also omits edge cases like calling on an already stopped execution. These gaps are moderate for a low-complexity tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It mentions 'running test execution' which loosely implies the executionId identifies that execution, but it does not explicitly explain the parameter. For a single unambiguous identifier, this minimal inference is acceptable, but the description adds no explicit clarification beyond the schema's bare type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Stop a running test execution') and the specific mechanism ('sends SIGTERM to the JMeter process'). It distinguishes itself from siblings like execute_test_plan (which starts) and get_execution_status (which reports status) by focusing on termination. The verb and resource are specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies it should be used on a running execution but does not explicitly state when to use it versus alternatives. It lacks exclusions, such as 'only use if execution is in a running state' or 'for status checks, use get_execution_status instead'. No guidance is provided on prerequisite conditions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 14 tool updatesv0.1.2
    • First observedadd_aggregate_report_listener
    • First observedadd_header_manager
    • First observedadd_http_sampler
    • First observedadd_json_extractor
    • First observedadd_response_assertion
    • First observedadd_summary_report_listener
    • First observedadd_thread_group
    • First observedcreate_test_plan
    • First observedexecute_test_plan
    • First observedget_execution_report
    • First observedget_execution_status
    • First observedget_test_plan
    • First observedlist_test_plans
    • First observedstop_execution

TDQS

A3.5/5.0

Scored across 14 tools

Disambiguation5/5

Each tool targets a distinct action and resource: test plan lifecycle (create/list/get), adding different element types (thread group, sampler, extractor, etc.), and execution/reporting (execute/status/report/stop). Despite similar 'add_' prefixes, each add tool clearly specifies the element it creates, leaving no ambiguity about which tool to use.

Naming Consistency5/5

All tools follow a consistent verb_noun pattern in snake_case: create_, list_, get_, add_, execute_, stop_. The naming convention is uniform and predictable, making it easy for an agent to infer tool purpose from the name alone.

Tool Count5/5

With 14 tools, the server covers the full test plan workflow—creation, building, execution, and result retrieval—without being bloated. Each tool serves a clear purpose, and the count is well within the typical range for a focused domain.

Completeness3/5

The set supports creating and reading test plans, adding common elements, and running/stopping executions, but lacks update and delete operations for both plans and elements. This means agents cannot modify or remove existing configurations, forcing recreation of a plan for any change. While the core create-run-report cycle is covered, the absence of edit/removal capabilities is a notable gap.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    D
    maintenance
    Enables the execution and analysis of JMeter performance tests through MCP-compatible clients. It provides tools for running tests in non-GUI mode, identifying performance bottlenecks, and generating comprehensive insights and visualizations from result files.
    6
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    Integrates Apache JMeter with AI assistants to run and manage load tests through natural language. It enables users to execute test plans, parse results, inspect test structures, and compare performance metrics across different runs.
    -