jmeter-mcp-server
This server lets an LLM create JMeter test plans as editable JSON trees, run real JMeter load tests asynchronously, and read back aggregated results without hand-writing .jmx XML.
Create, list, and inspect test plans; every element gets a stable node id for incremental editing.
Add thread groups (standard, setUp, tearDown) and samplers: HTTP, JDBC, JSR223, FTP, and TCP.
Add logic controllers: Transaction, Loop, If, While, Random, Interleave.
Add config elements: CSV Data Set, User Defined Variables, JDBC Connection Configuration, HTTP Request Defaults, Cookie Manager, Header Manager.
Add timers: Constant, Uniform Random, Constant Throughput.
Add pre/post-processors: JSR223, User Parameters, JSON Extractor, Regex Extractor, XPath Extractor.
Add assertions: Response, JSON, Duration, Size.
Add listeners: Aggregate Report, Summary Report, View Results Tree, Backend Listener.
Serialize plans to .jmx and run them in JMeter non-GUI mode; execution is async and returns an executionId immediately.
Poll execution status, stop a running execution, and fetch an aggregated report with error rate, latency stats (avg/median/p90/p95/p99), throughput, and KB/sec.
Allows building, maintaining, running, and reading reports for Apache JMeter test plans without the GUI, including creating test plan elements, executing non-GUI load tests, and retrieving aggregated latency/error statistics.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@jmeter-mcp-serverCreate a test plan hitting https://api.example.com/login with 50 users and show me the report."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
jmeter-mcp-server
Give an LLM real, deterministic control over Apache JMeter — build test plans, run real load tests, and read back real results, without ever hand-writing .jmx XML or opening the GUI.
"Load test my API: 20 users hitting POST /orders for 2 minutes,
5% think time, fail anything over 800ms"...turns into a running JMeter test and a real report, through typed tool calls an MCP client (Claude Code, Claude Desktop, etc.) makes directly.
Why not just ask an LLM to write the .jmx itself?
It can — a .jmx is just XML, and any capable model has seen plenty of JMeter test plans. The problem is how it fails: JMeter's format is a hashTree with dozens of fragile, easy-to-misremember details — exact guiclass/testclass pairs, property names that don't match their GUI label (ThreadGroup.num_threads is a stringProp, not an intProp), integer bitmasks for assertion match types, strict parent/child pairing with sibling <hashTree> tags. None of it is self-checking. A wrong value still produces valid, loadable XML that just quietly does the wrong thing.
That's not hypothetical — it happened building this project. An early version of the If Controller generated a property called useExpression set to true, which reads like "yes, evaluate my condition." The real JMeter source does the opposite: useExpression=true means don't evaluate it as an expression — just check if the string is literally "true". Every non-trivial condition silently, permanently failed. No error, no warning — the child sampler just never ran. It only surfaced by actually executing the generated plan against real JMeter and noticing a sample count of zero.
That's the whole case for this server in one story: an LLM regenerating XML from memory re-risks that exact mistake on every single request. This server encodes the correct shape once, in a serializer (and a matching parser for the reverse direction) checked against real JMeter source and bundled examples, exercised by 166 automated tests including real JMeter runs — and exposes it as typed tools instead. Concretely:
Correctness through one tested code path, not regenerated-from-memory XML every time.
Cheap incremental edits. Plans are a small JSON tree with stable node ids — adding, removing, renaming, moving, or disabling an element is one tool call by
id, not rewriting a whole.jmxfile. An existing.jmx(hand-written or exported from the GUI) can be imported and edited the same way.Aggregated results, not raw samples.
get_execution_reportreturns computed stats (error %, avg/median/p90/p95/p99, throughput) — not thousands of sample rows to average by hand.Real async execution.
execute_test_planreturns immediately with anexecutionId; long-running load tests never block anything.
The generated .jmx is standard JMeter output — open it in the real GUI any time.
Related MCP server: JMeter MCP Server
Example
You: Build a load test: 10 users for 30s hitting GET https://api.example.com/health,
fail anything that takes over 500ms, then run it and tell me the p95.
Claude: [create_test_plan, add_thread_group, add_http_sampler, add_duration_assertion,
add_aggregate_report_listener, execute_test_plan, get_execution_status, get_execution_report]
Ran 300 requests over 30s, 0 failures. p95 latency: 214ms, avg: 187ms, throughput: 10.1 req/s.Every step above is a real typed MCP tool call — see Tools for the full set (34 element types across samplers, controllers, timers, extractors, assertions, and listeners, plus editing, inspection, and .jmx import/export tools) and Example workflow for the raw call sequence.
Quick start
claude mcp add jmeter \
-e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
-- npx -y jmeter-mcp-serverThat's it — no cloning, no build. Adjust JMETER_HOME to your JMeter install (see Prerequisites). Full setup details, Claude Desktop config, and local-dev instructions are in Adding this server to Claude Code.
How a test plan is represented
Each plan is a JSON tree ({id, type, props, children[]}), not XML text. Authoring tools append a child under a given parentId, editing tools (remove_element, update_element, move_element, etc.) mutate that same tree in place, and the tree is only serialized to a real .jmx file on demand (get_test_plan_xml) or at execution time. import_test_plan runs the reverse direction, parsing an existing .jmx back into this same tree shape. This is what makes incremental edits cheap and keeps all the fiddly XML schema knowledge in two places (src/jmx/serializer.ts for tree → XML, src/jmx/parser.ts for XML → tree, sharing prop shapes from src/jmx/propTypes.ts) instead of spread across every tool.
Tools
Authoring (each returns the new node's id, used as parentId for
whatever you attach under it next) — grouped the same way JMeter's own
right-click Add menu groups them, so if you already know the GUI, you
already know where to look:
Tool | Adds |
| Root |
Threads (Users):
Tool | Adds |
| Thread Group (virtual users) |
| setUp Thread Group (runs once before all Thread Groups) |
| tearDown Thread Group (runs once after all Thread Groups) |
Sampler:
Tool | Adds |
| HTTP Request sampler |
| JDBC Request sampler |
| JSR223 Sampler (Groovy/BeanShell/JS/JEXL script as the sample) |
| FTP Request sampler |
| TCP Sampler |
Logic Controller:
Tool | Adds |
| Transaction Controller (groups child samplers into one named transaction) |
| Loop Controller (repeats child samplers) |
| If Controller (conditionally runs child samplers) |
| While Controller (repeats children while a condition holds) |
| Random Controller (runs one random child per pass) |
| Interleave Controller (alternates through children) |
Config Element:
Tool | Adds |
| CSV Data Set Config (parameterization from a file) |
| User Defined Variables |
| JDBC Connection Configuration (pooled datasource) |
| HTTP Request Defaults |
| HTTP Cookie Manager |
| HTTP Header Manager |
Timer:
Tool | Adds |
| Constant Timer (pacing/think-time) |
| Uniform Random Timer (randomized pacing) |
| Constant Throughput Timer (target rate pacing) |
Pre Processors:
Tool | Adds |
| JSR223 PreProcessor |
| User Parameters (per-thread variable value sets) |
Post Processors:
Tool | Adds |
| JSON Extractor post-processor |
| Regular Expression Extractor post-processor |
| XPath Extractor post-processor |
| JSR223 PostProcessor |
Assertions:
Tool | Adds |
| Response Assertion |
| JSON Assertion (JSONPath validation) |
| Duration Assertion (response-time SLA) |
| Size Assertion (response byte-size check) |
Listener:
Tool | Adds |
| Aggregate Report listener |
| Summary Report listener |
| View Results Tree listener (full request/response capture for debugging) |
| Backend Listener (streams live metrics to InfluxDB/Graphite/etc.) |
Editing (mutate an already-built plan):
Tool | Purpose |
| Remove an element (and its subtree); rejects removing the root |
| Shallow-merge (or replace) a node's props; a prop value of |
| Rename an element's |
| Move an element (and its subtree) to a new parent, optionally at a specific index; rejects moving a node into its own subtree |
| Reorder a node's direct children (must pass an exact permutation of the current children) |
| Enable/disable an element without removing it |
Inspection:
Tool | Purpose |
| List every plan in the workspace |
| Full element tree of a plan, including every node's |
| Serialize a plan to its JMeter |
| Import an externally authored |
Execution & reporting (async — a run happens in the background):
Tool | Purpose |
| Serialize to |
|
|
| Send |
| Aggregated stats (per label + overall) parsed from the run's JTL output |
Example workflow
create_test_plan → { planId, rootNodeId }
add_thread_group (parentId: rootNodeId) → { nodeId: threadGroupId }
add_http_sampler (parentId: threadGroupId) → { nodeId: samplerId }
add_response_assertion (parentId: samplerId)
add_aggregate_report_listener (parentId: threadGroupId)
execute_test_plan (planId) → { executionId }
get_execution_status (executionId) ← poll until "completed"
get_execution_report (executionId) → aggregated latency/error statsTesting
166 automated tests, no framework beyond Node's built-in test runner:
npm test # 155 tests: tree-mutation and XML-shape unit tests, XML -> tree parsing,
# serialize -> parse round-trips, and every tool called over the real MCP
# protocol (stdio, the same way Claude Code/Desktop talk to it) - no
# JMeter install needed, fully hermetic
npm run test:integration # 11 tests: real JMeter runs - the If Controller story above, While
# Controller loop counts, timer pacing, extractors, assertions, etc.
# (needs JMETER_HOME)
npm run test:allnpm test spawns the actual built server (dist/index.js) via StdioClientTransport and drives it exactly as a real client would — not just calling internal functions — so a broken tool schema or a malformed response shows up as a real protocol error, not a passing unit test.
Both suites run on every push and pull request via GitHub Actions — the integration job installs a real JMeter binary on the runner, so it's exercising the same code path as a local run, not a mock.
Prerequisites
Node.js 18+
JMeter installed locally, with the
JMETER_HOMEenvironment variable pointing at the installation directory (the one containingbin/jmeter). On macOS via Homebrew,brew install jmeterputs it at/opt/homebrew/opt/jmeter/libexec.
Adding this server to Claude Code
Via npx (recommended — published on npm)
No cloning or building required; npx fetches and runs the published
version on the fly:
claude mcp add jmeter \
-e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
-- npx -y jmeter-mcp-serverAdjust the JMETER_HOME path to wherever JMeter is installed on your
machine. Optionally set JMETER_MCP_WORKSPACE too (see below) if you want
plans and executions stored somewhere other than the default.
The default scope is local (this project directory only). To make it
available across every project, add -s user:
claude mcp add jmeter -s user \
-e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
-- npx -y jmeter-mcp-serverConfirm it registered and is responding:
claude mcp listFrom a local clone (development)
If you're working on this repository's code instead of using the published
package, point at the built dist/index.js directly:
npm install
npm run build
claude mcp add jmeter \
-e JMETER_HOME=/opt/homebrew/opt/jmeter/libexec \
-- node /absolute/path/to/jmeter-mcp-server/dist/index.jsClaude Desktop
Add this to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"jmeter": {
"command": "npx",
"args": ["-y", "jmeter-mcp-server"],
"env": {
"JMETER_HOME": "/opt/homebrew/opt/jmeter/libexec"
}
}
}
}Note: unlike a terminal-launched app, Claude Desktop does not inherit
environment variables exported in your shell profile (.zshrc, etc.) — only
true system-wide ones. Always set JMETER_HOME explicitly in the env
block above rather than relying on it already being "set on your machine".
Environment variables
Variable | Required | Purpose |
| Yes | JMeter installation directory (must contain |
| No | Where plans and executions are stored. Defaults to |
Workspace layout
<workspace>/
plans/<planId>/plan.json # JSON tree — source of truth for a plan
executions/<executionId>/
generated.jmx # serialized at execute_test_plan time
aggregate-report.jtl # output of the Aggregate Report listener, if present
summary-report.jtl # output of the Summary Report listener, if present
jmeter.log
meta.json # execution status, pid, timestamps, exit codeEditing and importing plans
Beyond the add_* authoring tools, a plan can be mutated after the fact
(remove_element, update_element, rename_element, move_element,
reorder_children, set_element_enabled) and an externally authored .jmx
(e.g. exported from the JMeter GUI) can be brought in with import_test_plan.
import_test_plan understands the most common element types (thread groups,
HTTP samplers, assertions, extractors, controllers, config elements, the
three report listeners, etc.); anything it doesn't recognize is kept as an
opaque UnknownElement node whose original XML is preserved and re-emitted
as-is by get_test_plan_xml/execute_test_plan, instead of being dropped -
import_test_plan's response reports unknownElementCount/
unknownElementTypes so you know what wasn't fully understood. Coverage can
be extended incrementally in src/jmx/parser.ts.
v1 scope
Not yet supported (candidates for a future release): generating the HTML
dashboard report (-e -o), parent-type validation on add_*/move_element/
import_test_plan (nothing stops attaching an element under a semantically
wrong parent), distributed execution.
Note on add_csv_data_set: the filename must be an absolute path.
execute_test_plan runs JMeter from a fresh per-execution directory, so a
relative path (which JMeter's GUI would resolve against the .jmx file's
own location) won't resolve there. An absolute path baked into a plan is
also machine-specific — it won't travel if you share plan.json with
someone on a different machine. This is only checked at creation time:
add_csv_data_set rejects a relative or nonexistent path up front, but
later changing a CSVDataSet's filename via update_element, or
importing a .jmx that already has a relative one via import_test_plan,
is not checked - it will only surface as a failure at execute_test_plan
time.
Note on add_jdbc_request/add_jdbc_connection_configuration,
add_ftp_request, and add_backend_listener: these generate correct,
JMeter-loadable XML, but exercising them for real needs infrastructure this
project doesn't provide (a database, an FTP server, an InfluxDB/Graphite
instance) — they were verified structurally, not against a real backend.
Note on add_view_results_tree_listener's captureFullData option: it has
no effect right now. execute_test_plan always runs JMeter with
-Jjmeter.save.saveservice.output_format=csv, and JMeter's CSV writer never
emits response body/header columns no matter what the SampleSaveConfiguration
flags say — only its XML output format can carry full response bodies. The
option is wired up correctly in the generated .jmx (verified: the flags
really do flip in the XML) for the day this server supports XML-format runs,
but until then it's a no-op — confirmed by running a real capture and
checking the resulting JTL has no responseData/samplerData/
requestHeaders/responseHeaders columns regardless of the setting.
Note on add_tcp_sampler: server/port/request are live-verified. The
numeric fields (connectTimeoutMs, timeoutMs) are rendered as
stringProp following this project's general convention for sampler
numeric fields, but that specific choice for TCPSampler wasn't confirmed
against a real JMeter-GUI-saved example (none was available to check
against) — flagging in case a real save turns out to expect intProp.
Roadmap
Ideas being explored for future releases — none of these are implemented yet:
Proposed tool | What it does | Why it's worth it |
| Automatic binary-search capacity finder: ramps thread count up/down on its own, run after run, until it finds the concurrency level that violates your SLA (p95 latency, error %) | JMeter has no native "find the limit" feature. Driving this search through raw LLM tool calls costs ~6-8 calls per round (adjust load, run, poll, read report, decide) across the several rounds a binary search needs |
| Fits Little's Law / the Universal Scalability Law to collected concurrency vs. throughput vs. latency data, and classifies the bottleneck as contention, coherency, or saturation | JMeter only outputs raw numbers; re-deriving a queueing-theory curve fit through prose reasoning would mean reimplementing nonlinear regression by hand for every question |
| Runs linear regression over the latency/error time series of a long-duration soak test to separate normal noise from a real trend (the classic memory-leak signal) | JMeter's graph shows the curve, but doesn't say whether it's a statistically real degradation or just noise |
| Statistical diff across N executions (not just two), with a significance test for whether a p95 shift is real or noise | A ready-made performance regression gate for CI, instead of someone eyeballing two JSON reports and guessing |
| Automatically detects where warm-up (JIT, connection pools, cold caches) ends, and recomputes metrics only over the steady-state window | Today's overall average is polluted by the first few seconds of a run; JMeter doesn't separate this on its own |
| Re-runs failed samples with backoff and separates "real system error" from "one-off flake" (network timeout, etc.) | Produces a trustworthy error rate for a CI gate, instead of an |
| Runs the same test plan across multiple JMeter injector nodes (master-slave, via | JMeter supports distributed mode, but wiring up remote engines, RMI ports, matching JMeter versions, and merging results across machines is entirely manual today — nobody sets this up for a quick test |
How this compares
Checked against the other public JMeter MCP servers found on GitHub as of September 3, 2026 — open-source repositories with at least a README description (undocumented forks/clones excluded). Columns reflect what each project's own README documents, not independent verification of its internals.
Server | Approach |
| Edit by id | Async run | Tests | Real JMeter |
jmeter-mcp-server (this project) | JSON tree, 34 element types, authored and edited by node id | ✓ | ✓ | ✓ | 166, incl. real runs | ✓ |
Runs an existing | — | ✗ | ✗ | ✗ | ✓ | |
Generates a | ✗ | ✗ | ✗ | ✗ | ✓ | |
Generates a whole plan from one JSON payload, runs it via Docker | ✗ | ✗ | ✗ | ✗ | ✓ | |
Heals the local JMeter runtime, imports HAR/OpenAPI traffic, discovers capacity | — | ✗ | ✗ | ✗ | ✓ | |
Demo project for a tutorial video, 11 fixed tools | ✗ | ✗ | ✗ | ✗ | ✓ | |
Reimplements HTTP load testing in Node — no Apache JMeter underneath | ✗ | ✗ | ✗ | ✗ | ✗ | |
Generates a | ✗ | ✗ | ✗ | ✗ | ✓ | |
Demo script with parameters hardcoded in source, not a general-purpose server | ✗ | ✗ | ✗ | ✗ | ✓ |
Every capability in that table shows up somewhere across the other eight projects, individually:
real JMeter execution, .jmx generation, traffic import. This is the only one that combines full
.jmx round-trip (import and export), per-id incremental editing, non-blocking async execution,
and a documented automated test suite in one place.
License
MIT
Available Tools
14 toolsadd_aggregate_report_listenerA
Add an Aggregate Report listener under the given parent (Thread Group or TestPlan). Its output is what get_execution_report reads after a run.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Aggregate Report | |
| planId | Yes | ||
| filename | No | Filename kept in the .jmx for portability; ignored at execution time | |
| parentId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must carry full behavioral transparency. It only indicates an 'add' operation without disclosing side effects (e.g., modifying the plan), requirements, reversibility, or safety considerations. The relationship to get_execution_report is mentioned, but no side effects are revealed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The core action and the key dependency are stated directly and in a logical order.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description provides the essential role (feeds get_execution_report) and parent context, but lacks detail on required parameters (planId, parentId remain ambiguous) and does not disclose mutation side effects. For a tool with 4 parameters and no annotations, it is not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 25%, so the description must compensate. It clarifies parentId (Thread Group or TestPlan) and the tool's purpose, but does not explain planId or name beyond the schema's default, nor does it add much to filename. Compensation is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (add) and a resource (Aggregate Report listener), and specifies the parent types (Thread Group or TestPlan). It also links the listener's output to get_execution_report, which distinguishes it from other listener tools like add_summary_report_listener.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to produce output for get_execution_report), but it does not explicitly name alternatives or provide when-not-to-use guidance. The context is helpful but not as explicit as it could be.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_header_managerB
Add an HTTP Header Manager under an HTTP sampler (or a Thread Group, to apply to every sampler in it).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | HTTP Header Manager | |
| planId | Yes | ||
| headers | Yes | ||
| parentId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states that it adds a header manager, but does not mention any side effects (e.g., modifying the test plan), required permissions, idempotency, or what happens if the parent doesn't exist. For a mutation tool, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no filler. It is front-loaded with the action and placement, making it easy to scan. However, the brevity contributes to the lack of parameter and behavior detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given four parameters with zero schema descriptions, no output schema, and no annotations, the description should provide substantial guidance. It only covers the core purpose and placement, leaving out parameter semantics, required IDs, header structure, and any behavioral caveats. The tool is underspecified for reliable agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any of the four parameters (planId, parentId, headers, name). It fails to compensate for the schema's lack of documentation, leaving agents to guess the meaning and structure of the headers array and required IDs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (add an HTTP Header Manager) and specifies the target (under an HTTP sampler or Thread Group), which distinguishes it from sibling tools that add other elements. The placement detail adds precision beyond a generic 'add' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides placement guidance (under sampler vs. Thread Group) which implicitly tells when to use it for scoping headers to all samplers. It does not explicitly mention alternatives or when not to use it, but there are no similar sibling tools that add headers, so this is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_http_samplerC
Add an HTTP Request sampler under the given parent node (usually a Thread Group).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| path | Yes | Request path, e.g. /v1/users | |
| port | No | ||
| domain | Yes | Host name, e.g. api.example.com | |
| method | Yes | ||
| planId | Yes | ||
| bodyJson | No | Raw JSON request body, if any | |
| parentId | Yes | ||
| protocol | No | https |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It only says 'Add', implying mutation, but doesn't mention side effects, requirements (like parent existence), or error behavior. This is a significant gap for a mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that immediately conveys the primary action and target. There is no filler or redundancy; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters, 6 required, and no output schema or annotations, the description is grossly incomplete. It doesn't explain how to specify the HTTP request (method, domain, path, etc.) or what the parent node requirements are, leaving the agent without enough context to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 33%, and the description adds nothing about any of the 9 parameters. It fails to compensate for the undocumented parameters (name, method, planId, parentId, protocol, etc.), leaving the agent to guess their meaning and usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Add an HTTP Request sampler' and specifies the location 'under the given parent node'. It clearly distinguishes from siblings like add_thread_group or add_json_extractor by naming the sampler type, though it doesn't explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides minimal context by noting the parent is 'usually a Thread Group', but gives no guidance on when to use this tool versus alternatives like add_header_manager or add_response_assertion. No exclusions or conditions are mentioned, leaving the agent to infer when it applies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_json_extractorA
Add a JSON Extractor post-processor under an HTTP sampler, to save a value from the JSON response into a variable.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| planId | Yes | ||
| parentId | Yes | Id of the HTTP sampler node this extractor applies to | |
| defaultValue | No | Value to use if the JSONPath doesn't match | NOT_FOUND |
| jsonPathExpr | Yes | JSONPath expression, e.g. $.data.id | |
| referenceName | Yes | JMeter variable name to store the extracted value in |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully disclose behavioral traits. It only says 'Add', implying a mutation, but does not explain side effects like whether existing extractors are replaced, what happens if the parent sampler does not exist, or whether the operation is reversible. This is a significant gap for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly worded sentence that front-loads the action and purpose. Every word earns its place, with no redundant information or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is adequate but not complete. It explains the main use case, but does not mention potential behavioral details (e.g., failure handling, idempotency) or prerequisites beyond the parent sampler. The missing parameter descriptions for name and planId are not addressed. A more comprehensive description would help for a mutation operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67% (4 of 6 parameters have descriptions). The description adds no additional parameter-specific details beyond what the schema already provides; it merely restates the purpose which maps to jsonPathExpr and referenceName. Since the schema covers most parameters, the baseline of 3 is appropriate, and the description does not compensate for the undocumented name and planId parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Add'), a clear resource ('JSON Extractor post-processor'), a location context ('under an HTTP sampler'), and the purpose ('save a value from the JSON response into a variable'). This clearly distinguishes it from sibling add_* tools like add_http_sampler or add_response_assertion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use this tool: when you need to add a JSON extraction post-processor. It provides context about the parent sampler. However, it does not explicitly state when not to use it or mention alternatives (e.g., if you need a different extractor type), but given the sibling list, the purpose is distinct enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_response_assertionC
Add a Response Assertion under an HTTP sampler, to fail the sample if the response doesn't match.
| Name | Required | Description | Default |
|---|---|---|---|
| not | No | Negate the match (assert the pattern does NOT match) | |
| name | No | Response Assertion | |
| planId | Yes | ||
| parentId | Yes | ||
| patterns | Yes | ||
| matchType | No | contains | |
| testField | No | response_data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states that the tool fails the sample when the response doesn't match, which is the primary side effect. However, it does not mention constraints (e.g., parent must be an HTTP sampler), error conditions, or any side effects on existing assertions. The core behavior is clear but details are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that states the action and purpose without unnecessary words. It is concise and efficient, scoring well for conciseness, though it could be expanded slightly with parameter guidance without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters, no output schema, and low schema description coverage, the description is inadequate. It doesn't explain the required parameters (planId, parentId, patterns) or optional ones (matchType, testField, name, not), nor does it describe the return behavior. An agent would need to infer too much from the schema alone, making the description incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 14% (only 'not' has a description), so the description must compensate. It does not explain planId, parentId, patterns, matchType, testField, or name. The phrase 'if the response doesn't match' implies patterns are used for matching but offers no parameter semantics. The agent is left to guess from property names and enums, which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: adding a Response Assertion under an HTTP sampler, with the purpose of failing the sample on mismatch. This is specific and distinguishes it from sibling add_* tools (e.g., add_http_sampler, add_json_extractor) by naming the resource and behavior. However, it doesn't explicitly contrast with alternatives, so a 4 is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus others, nor any conditions where it should not be used. It only states what the tool does without context on selection. No exclusions or alternatives are mentioned, leaving the agent to infer appropriateness from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_summary_report_listenerB
Add a Summary Report listener under the given parent (Thread Group or TestPlan). Its output is what get_execution_report reads after a run.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Summary Report | |
| planId | Yes | ||
| filename | No | Filename kept in the .jmx for portability; ignored at execution time | |
| parentId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden for behavioral disclosure. It does mention that the listener's output is what get_execution_report reads, which is useful context, but it does not disclose any side effects, permissions, or reversibility of adding the listener. As a mutation tool, this is a notable gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. The primary action is front-loaded, and the follow-up sentence adds valuable context about the tool's relationship to get_execution_report. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 4 parameters and no annotations or output schema, the description is too thin. It clarifies the parent types but omits details on planId and the return value. The link to get_execution_report is helpful, but the description does not adequately equip an agent to invoke the tool correctly with all parameters understood.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 25%, so the description must compensate for the undocumented parameters. It provides some context for parentId by naming allowed parent types, but planId and name are not explained. The description does not clarify the meaning or format of the parameters beyond what is already in the schema, which is minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: adds a Summary Report listener under a parent (Thread Group or TestPlan). It also connects it to get_execution_report, which helps place its role. It does not explicitly contrast with the sibling add_aggregate_report_listener, so a 4 rather than 5 is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Some usage guidance is implied by mentioning the valid parents (Thread Group or TestPlan), but the description does not state when to choose this listener over the aggregate report alternative, nor any prerequisites or exclusions. Sibling tools exist but no comparison is made, so guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_thread_groupC
Add a Thread Group (virtual users) under the given parent node (usually the TestPlan root).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| loops | No | Number of loop iterations per thread, -1 for infinite | |
| planId | Yes | ||
| parentId | Yes | Id of the node to attach this thread group under | |
| numThreads | Yes | Number of concurrent virtual users | |
| durationSeconds | No | If set, run on a scheduler for this many seconds instead of a fixed loop count | |
| rampTimeSeconds | Yes | Seconds to reach full thread count |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states that a thread group is added and implies mutation, but does not mention side effects, permissions, reversibility, or what happens on success/failure. The brief note about the parent node being usually the TestPlan root is the only extra context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded with the action and resource, and the parenthetical clarification adds useful context without bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters and no output schema, the description is sparse. It does not mention key parameters like planId, nor the relationship between loops and durationSeconds (though the schema mentions it). It also lacks any indication of what the tool returns or how the agent should interpret the result. The description is insufficient for full correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes 5 of 7 parameters, providing explanations for parentId, numThreads, rampTimeSeconds, loops, and durationSeconds. The description adds minimal value by mentioning 'given parent node' which relates to parentId, but does not clarify name or planId. Given the high schema coverage, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Add) and the resource (Thread Group), and clarifies 'virtual users'. It also indicates the placement under a parent node. However, it does not explicitly differentiate from sibling add_* tools, though the resource type is distinct enough that an agent might infer the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives. It only mentions the typical parent node, but does not specify when to prefer this over other nodes or tools. There are no exclusions or alternative tool references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_test_planA
Create a new JMeter test plan. Returns the planId and the id of its root TestPlan node, which you'll use as parentId for the first thread group.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Human-readable name for the test plan |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It discloses the return values (planId and root node id) and hints at their usage, but does not mention side effects, persistence, or any prerequisites. It is adequate but not rich enough for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero redundancy. It front-loads the core action, then concisely provides the critical return information and usage hint. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple create operation with a single parameter and no output schema, the description explains the return values and their practical use. It is complete for the immediate purpose, though it could mention potential side effects or idempotency, but that does not detract significantly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema describes the only parameter 'name' with an adequate human-readable description (100% coverage). The tool description adds no extra detail about the parameter, so it stays at the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new JMeter test plan' with a specific verb and resource. It further differentiates from siblings by noting that it returns planId and the root TestPlan node id, which is unique to this tool among the listed siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on how to use the return value ('which you'll use as parentId for the first thread group'), implying it is the first step in building a plan. It does not explicitly contrast with alternatives like add_thread_group, but the usage context is evident and sufficient for a create-first tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
execute_test_planA
Start running a test plan with JMeter in non-GUI mode. Returns immediately with an executionId; the run continues in the background. Poll get_execution_status to know when it's done, then call get_execution_report to read the results.
| Name | Required | Description | Default |
|---|---|---|---|
| planId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full responsibility. It discloses the asynchronous execution model, the immediate return of an executionId, and the necessity to poll status and retrieve results. This is critical behavioral information that an agent needs to know to avoid blocking or assuming synchronous completion.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action. The second sentence packs the async behavior and follow-up steps without waste. Every sentence adds value; there is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter asynchronous tool with no output schema, the description covers the essential operational model: what it starts, how it returns, and the required polling/report steps. It leaves no critical ambiguity for an agent to execute correctly. The mention of non-GUI mode adds practical context without going overboard.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is only one parameter, planId, with 0% schema description coverage. The description does not directly explain the parameter, but its name and the tool's reference to 'test plan' make it self-explanatory as the ID of the plan to run. While the description does not add extra meaning, the parameter's purpose is evident from context. Given the low coverage, a 3 is appropriate – it does not harm, but could be more explicit about where to get planId (e.g., from list_test_plans).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start running a test plan'), the resource ('a test plan'), the mode (non-GUI), and the immediate return with an executionId. It distinguishes itself from siblings by focusing on the initiation of execution, while get_execution_status and get_execution_report are for monitoring/results, and create_test_plan is for creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs the agent on the follow-up workflow: 'Poll get_execution_status to know when it's done, then call get_execution_report to read the results.' This tells when to use this tool (to start) and what to use next. It also states the async nature ('returns immediately... continues in the background'), providing clear context for correct usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_execution_reportA
Read and aggregate the results of a finished (or still-running) execution, computed from its Aggregate Report / Summary Report listener output: per-label and overall count, error rate, avg/min/max/median/p90/p95/p99 latency, throughput and KB/sec.
| Name | Required | Description | Default |
|---|---|---|---|
| executionId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the operation is read-only ('Read'), the data source (listener output), and that it can be called on running executions. It does not mention potential side effects, latency, or error handling for invalid IDs, but for a read operation this is acceptable. It adds context about the metrics but not deeper behavioral details like caching or consistency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, information-dense sentence that front-loads the core purpose and immediately lists the output metrics. Every word contributes value; there is no fluff or repetition. It is concise without losing specificity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and no output schema, the description covers the essential elements: what it does, when it can be used (finished or running), and exactly which metrics it returns. The lack of an output schema is mitigated by the metrics list. It does not mention error conditions or return formatting, but that is a minor gap given the simplicity of the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has a single parameter, executionId, with 0% description coverage. The tool description does not explicitly explain what the parameter means beyond its name—it only says the tool retrieves results for 'an execution', but does not reiterate that executionId must be the ID of the execution whose report is desired. Since schema coverage is zero and the description fails to compensate, the parameter semantics are under-specified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Read and aggregate') with a clear resource ('results of a finished (or still-running) execution') and enumerates the exact metrics returned (per-label and overall count, error rate, latency percentiles, throughput, KB/sec). It distinguishes itself from siblings like get_execution_status by focusing on detailed report data from listener output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the tool works on finished or still-running executions and specifies the source (Aggregate Report / Summary Report listener output), giving context for when it applies. However, it does not explicitly contrast it with siblings like get_execution_status, nor does it state when NOT to use it (e.g., for just checking status). Usage is implied but not fully explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_execution_statusA
Check the status of a test run started with execute_test_plan (running/completed/failed), plus the tail of its log.
| Name | Required | Description | Default |
|---|---|---|---|
| executionId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that it returns status plus log tail, which is the core behavior, and implies read-only nature. However, it does not mention potential side effects (none expected), polling behavior, or log length limits, so it is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence that front-loads the purpose and includes essential details (statuses, log tail). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple status-check tool with one parameter and no output schema, the description covers what it does and what it returns. It does not explain how to interpret the status (e.g., polling until completed), but that could be considered usage guidance rather than essential for a correct call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description does not explain the executionId parameter beyond its obvious relation to the run. It does not specify that executionId is the ID returned by execute_test_plan, which would have compensated for the coverage gap. The meaning is inferred but not stated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks the status of a test run, specifies it is for runs started with execute_test_plan, and enumerates the possible statuses (running/completed/failed) plus the log tail. This distinguishes it from siblings like get_execution_report (which likely provides full report) and stop_execution (which stops the run).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly ties usage to runs from execute_test_plan, which gives clear context on when to invoke it. However, it does not mention when not to use it or point to alternatives like get_execution_report for full results, so it lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_planA
Get the full element tree of a test plan, including every node's id (needed as parentId for add_* tools) and type.
| Name | Required | Description | Default |
|---|---|---|---|
| planId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly indicates this is a retrieval operation ('get') and specifies what it returns, which is enough to infer it is read-only and non-destructive. It does not mention potential side effects or response size, but for a getter with a single parameter, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clearly structured sentence that front-loads the primary action ('Get the full element tree') and then adds contextual detail (ids and their use). Every word serves a purpose with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description fully explains what is returned (the full element tree, including ids and types) and why it matters (for parentId). It gives an agent enough to decide when and how to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one parameter (planId) with 0% description coverage. The description does not explicitly explain what planId is, though it is easily inferred as the identifier of the test plan from the context. It adds no direct parameter detail, but the single, self-explanatory parameter name mitigates the weak coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Get the full element tree') and a specific resource ('a test plan'), and clarifies it returns every node's id and type. This distinguishes it from siblings like list_test_plans (which lists plans) and get_execution_report (which reports execution results), making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly mentions that node ids are needed as parentId for add_* tools, giving a clear reason to use this tool before making structural modifications. It does not explicitly say when not to use it, but the context of adding nodes is clear, and the sibling tools for execution vs. structure are implicitly separated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_test_plansA
List all test plans in the workspace.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that it lists test plans, without indicating whether the operation is read-only, if there are side effects, rate limits, or pagination behavior. For a potentially safe listing operation, the lack of any safety or side-effect context is a gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that clearly states the action and scope. It is front-loaded with the verb and resource, avoiding any extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (no parameters) and lack of an output schema, the description provides adequate context for most use cases. However, it does not detail the return format (e.g., list of test plan IDs, full objects, or metadata), which could be relevant for agents expecting a specific structure. Since no output schema exists, a brief note on return content would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. No parameter documentation is needed, and the description adds nothing beyond the schema, which is appropriate since there is nothing to clarify.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a clear verb ('List') and a specific resource ('all test plans in the workspace'). It naturally distinguishes itself from sibling tools like get_test_plan (which targets a single plan) and create_test_plan (which creates), so an agent can identify its purpose without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for enumerating all test plans, but it does not explicitly state when to use it versus alternatives like get_test_plan for a specific plan. No exclusions or conditions are mentioned, so the guidance remains implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stop_executionA
Stop a running test execution (sends SIGTERM to the JMeter process).
| Name | Required | Description | Default |
|---|---|---|---|
| executionId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It does reveal the internal action (SIGTERM), but omits consequences such as whether the stop is graceful, what happens if the execution is already stopped, or whether any cleanup occurs. It also does not mention potential side effects or error states. The provided detail is useful but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tight sentence with no filler. It front-loads the primary action and includes the key behavioral detail (SIGTERM) without redundancy. Every word contributes meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and no output schema. The description covers the core action and mechanism, but given the absence of annotations and output schema, it could state what the caller should expect (e.g., a success status or error if execution not found). It also omits edge cases like calling on an already stopped execution. These gaps are moderate for a low-complexity tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'running test execution' which loosely implies the executionId identifies that execution, but it does not explicitly explain the parameter. For a single unambiguous identifier, this minimal inference is acceptable, but the description adds no explicit clarification beyond the schema's bare type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Stop a running test execution') and the specific mechanism ('sends SIGTERM to the JMeter process'). It distinguishes itself from siblings like execute_test_plan (which starts) and get_execution_status (which reports status) by focusing on termination. The verb and resource are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it should be used on a running execution but does not explicitly state when to use it versus alternatives. It lacks exclusions, such as 'only use if execution is in a running state' or 'for status checks, use get_execution_status instead'. No guidance is provided on prerequisite conditions or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
v0.1.2- First observed
add_aggregate_report_listener - First observed
add_header_manager - First observed
add_http_sampler - First observed
add_json_extractor - First observed
add_response_assertion - First observed
add_summary_report_listener - First observed
add_thread_group - First observed
create_test_plan - First observed
execute_test_plan - First observed
get_execution_report - First observed
get_execution_status - First observed
get_test_plan - First observed
list_test_plans - First observed
stop_execution
TDQS
Scored across 14 tools
Each tool targets a distinct action and resource: test plan lifecycle (create/list/get), adding different element types (thread group, sampler, extractor, etc.), and execution/reporting (execute/status/report/stop). Despite similar 'add_' prefixes, each add tool clearly specifies the element it creates, leaving no ambiguity about which tool to use.
All tools follow a consistent verb_noun pattern in snake_case: create_, list_, get_, add_, execute_, stop_. The naming convention is uniform and predictable, making it easy for an agent to infer tool purpose from the name alone.
With 14 tools, the server covers the full test plan workflow—creation, building, execution, and result retrieval—without being bloated. Each tool serves a clear purpose, and the count is well within the typical range for a focused domain.
The set supports creating and reading test plans, adding common elements, and running/stopping executions, but lacks update and delete operations for both plans and elements. This means agents cannot modify or remove existing configurations, forcing recreation of a plan for any change. While the core create-run-report cycle is covered, the absence of edit/removal capabilities is a notable gap.
Maintenance
Related MCP Connectors
- TimequipOAuthcom.timequip
Manage Timequip projects, tasks, comments, members, and dashboards through MCP.
An MCP server that provides access to Testiny projects, test cases and test runs
Read and write Mission Control state via MCP — projects, tasks, subtasks, templates, status updates.
Related MCP Servers
- FlicenseCqualityDmaintenanceA Model Context Protocol server that enables executing and interacting with JMeter tests through MCP-compatible clients like Claude Desktop, Cursor, and Windsurf.2-
- FlicenseCqualityDmaintenanceA Model Context Protocol server that enables execution of JMeter performance tests through AI assistants and MCP-compatible clients like Claude, Cursor, and Windsurf.2-
- FlicenseAqualityDmaintenanceEnables the execution and analysis of JMeter performance tests through MCP-compatible clients. It provides tools for running tests in non-GUI mode, identifying performance bottlenecks, and generating comprehensive insights and visualizations from result files.6-
- FlicenseNot gradedqualityCmaintenanceIntegrates Apache JMeter with AI assistants to run and manage load tests through natural language. It enables users to execute test plans, parse results, inspect test structures, and compare performance metrics across different runs.-