AI QA for coding agents. TestOptim explores your web app, generates end-to-end tests and reports bugs with severity. Over MCP, Claude Code, Cursor or Copilot can start a test run on just the pages they changed, poll the result, and read the issues it found. Authenticate with a TestOptim API key as a Bearer token.
Scored across 8 tools
The eight tools split cleanly along resource (project/exploration/test run/issue) and action (list/get/start/stop) lines, so misselection is unlikely. The only mild friction is the asymmetry between start/stop pairs — stop_test_run exists with no stop_exploration counterpart — which could momentarily confuse an agent looking to cancel an exploration.
Every tool follows a strict verb_noun snake_case pattern: list_projects, get_exploration, get_issue, get_test_run, list_issues, start_exploration, start_test_run, stop_test_run. The verbs (list/get/start/stop) map predictably onto read-detail, read-collection, and lifecycle operations.
Eight tools is well within the sweet spot and each one corresponds to a distinct step in the explore → test → triage workflow. No tool looks redundant or padded.
The core lifecycle (list projects, start/poll exploration, start/poll/stop run, list/get issues) is covered, but there are notable gaps: no way to list past test runs or explorations, no stop_exploration despite exploration consuming page allowance, and no update_issue to change issue status/severity even though triage is described. Agents will hit dead ends when they need history or issue mutation.
Check an exploration: its status, progress and, once it has ended, how many pages and flows it recorded. waiting_otp_ready or waiting_otp means the login needs a one-time code entered in the TestOptim app.
| Name | Required | Description | Default |
|---|---|---|---|
| explorationId | Yes | The exploration _id returned by start_exploration |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It usefully decodes the waiting_otp_ready/waiting_otp states and tells the agent the OTP must be entered in the TestOptim app, which is genuine behavioral context. However, it never states that the call is read-only/idempotent, and says nothing about rate limits or whether the operation is cheap to poll.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with zero waste; the primary purpose leads and the OTP caveat is appended as a secondary note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the work of explaining return content (status, progress, page/flow counts on completion) and even the meaning of special statuses. It is nearly complete for a simple read tool; only the guarantee of read-only behavior and polling guidance are missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single explorationId parameter is documented there as the _id returned by start_exploration. The description adds no further meaning about the identifier, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Check an exploration') with an enumeration of what is returned: status, progress, and page/flow counts once ended. It is clearly the read counterpart to start_exploration, though it never names that sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: 'check' plus the status semantics suggest polling an exploration after start_exploration, but there is no explicit when/when-not, no mention of polling cadence, and no reference to stop_test_run or other alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get one issue in full: description, steps to reproduce, expected vs actual behaviour, page URL and screenshot. Use after list_issues to dig into a specific issue.
| Name | Required | Description | Default |
|---|---|---|---|
| issueId | Yes | The issue _id returned by list_issues |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose the return payload field-by-field, which is the key behavioral fact for a fetch-one tool. It stops short of covering failure modes — whether a missing or inaccessible issueId errors versus returns empty — which matters for an agent handling a bad ID.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler, with the returned-fields summary front-loaded before the routing hint. Nothing could be cut without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with no output schema, the description effectively substitutes for one by enumerating the returned fields, and it supplies the sibling routing cue. Everything an agent needs to select and call it is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single issueId parameter is already documented in-schema as 'The issue _id returned by list_issues.' The description reinforces singularity ("one issue") but adds no format or sourcing detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Get one issue in full") and itemizes exactly what comes back: description, steps to reproduce, expected vs actual behaviour, page URL, screenshot. The singular scope plus the explicit contrast with list_issues makes it unmistakable among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Use after list_issues to dig into a specific issue" names the alternative and the sequencing that selects this tool over the list variant. There is no explicit when-not guidance (e.g., what to do if the issue ID is unknown), but the usage context is stated plainly rather than implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check a test run: whether it is still running, and how many issues it found.
| Name | Required | Description | Default |
|---|---|---|---|
| testRunId | Yes | The run _id returned by start_test_run |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. 'Check' signals a read-only operation and the description usefully discloses two return fields (running status and issue count), but it omits error behavior, authentication needs, and whether any other run details are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that front-loads the core purpose and immediately specifies the two pieces of information returned. No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with no output schema, the description is nearly complete: it explains what the tool checks and what it returns. The only notable gap is behavior on errors or missing run IDs, but the core agent need is covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter, and the schema itself explains that testRunId is the run _id returned by start_test_run. The description adds no additional parameter semantics beyond the schema, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Check') and resource ('a test run'), plus the exact information returned: running state and issue count. It does not explicitly differentiate itself from siblings like start_test_run or stop_test_run, though the read-oriented phrasing helps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: an agent can infer this is for checking an existing run's status rather than starting or stopping one. However, there is no explicit when-to-use guidance, no mention of alternatives such as list_issues for issue details, and no prerequisites like needing a testRunId from start_test_run.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List QA issues in a project as short summaries - id, title, severity, status. Pass testRunId for only what one run found, or filter by severity to triage. Call get_issue for the full detail of one issue.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | How many to return (default 20) | |
| severity | No | Only return issues of this severity | |
| projectId | Yes | The project to read issues from | |
| testRunId | No | Only return the issues this test run found |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full disclosure burden. It helpfully signals that results are truncated summaries rather than full records and routes depth to get_issue, but says nothing about read-only safety, ordering, or how paging beyond the limit works.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short clauses, front-loaded with the core action and return shape, then filtering guidance, then the sibling pointer. No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Since there is no output schema, the description compensates by naming the returned fields and noting they are short summaries. It lacks detail on result ordering and whether more results exist beyond the limit, which would matter for a listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so projectId, limit, severity, and testRunId are already documented. The description adds only the intent behind two of them ('only what one run found', 'to triage') and says nothing about limit or projectId beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (list QA issues in a project) plus the exact shape of what comes back (id, title, severity, status). It explicitly distinguishes itself from get_issue, which returns full detail for a single issue, so an agent can pick the right tool without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete guidance: pass testRunId to scope to one run, or filter by severity when triaging, and points to get_issue for detail. There is no explicit 'when not to use' (e.g., for cross-project listing or listing projects, which siblings cover), so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the QA projects this token can access. Use it to find the project id the other tools need.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral burden. It does disclose one real trait — results are scoped to the token's access rights — and hints the payload contains project ids, but it says nothing about ordering, pagination, or whether names/other fields are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no filler, with the scope constraint front-loaded before the usage hint. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read tool with no output schema, the description covers what it returns at a high level (projects, and specifically the id) and why an agent would call it. The exact shape of each returned record is the only material gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4; there is nothing the description could usefully add on that axis. The mention of the project id in the output is a small bonus beyond what an empty schema conveys.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("List the QA projects") plus a scope qualifier ("this token can access"). No sibling tool (get_exploration, list_issues, get_test_run, etc.) lists projects, so the tool is distinguishable without naming an alternative, though the description doesn't say so explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Use it to find the project id the other tools need" gives a concrete entry-point condition: call this first to obtain a prerequisite id. It doesn't state any when-not-to-use case (e.g., if the project id is already known), but the usage context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Explore the project's app so TestOptim learns new or changed pages and flows. Use it after adding pages, before start_test_run; tests for what it finds are generated and approved in the TestOptim app. Spends the plan's page allowance and replaces an exploration already running. Returns an exploration id - poll get_exploration, which takes minutes.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | The project whose app to explore |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does: it discloses a quota cost ('Spends the plan's page allowance'), a destructive side effect ('replaces an exploration already running'), an async duration ('takes minutes'), and the return value (an exploration id). This is exactly the context annotations would otherwise have to supply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four compact sentences, each carrying distinct information: purpose, sequencing, cost/side effect, and return/polling. The purpose leads and nothing is redundant with the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description supplies the return contract (exploration id) and the polling path (get_exploration, minutes). For a single-parameter async initiation tool, nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter exists, and the schema documents it at 100% coverage ('The project whose app to explore'); the description's scoping language ('the project's app') reinforces the same meaning. A baseline of 3 would apply if this were a multi-param tool with full coverage, but with a single, fully described parameter the schema-plus-description pair leaves nothing ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Explore the project's app') plus the outcome ('TestOptim learns new or changed pages and flows'), which distinguishes it from the read-only get_exploration sibling. An agent can tell immediately this is the initiation step, not a status check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit timing ('after adding pages, before start_test_run') and names the follow-up tool ('poll get_exploration'). The only mild gap is no explicit 'do not use when' clause, but the sequencing and the note that it replaces a running exploration effectively covers the exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start a QA test run against a project. Pass changedAreas to re-test only the pages or flows you just changed; omit it for the full suite. Returns a run id - poll get_test_run for the outcome. Runs take minutes, so do not poll in a tight loop.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | The project to test | |
| changedAreas | No | Run only the tests covering these pages or flows, e.g. ["/checkout", "/cart"]: the parts of the app you just changed. Omit to run the full suite. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden, and it does well: it discloses that the call is asynchronous (returns a run id, not a result), that runs take minutes, and that tight polling is discouraged. It stops short of stating permissions, cost, or what happens if a run is already in flight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, each doing distinct work: purpose, parameter branching, return contract with the follow-up tool, and timing/polling advice. The most important facts are front-loaded and nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter async tool with no output schema, the description covers what an agent needs: the return shape (run id), where to get the result, and how long to wait. Only sibling differentiation and failure-mode behavior are absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already explains both projectId and changedAreas including the omit-for-full-suite behavior. The description's parameter guidance largely restates that, so it adds emphasis rather than new semantics; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Start a QA test run against a project,' which is concrete enough to act on. It does not, however, distinguish this from the sibling start_exploration, which an agent would also plausibly consider when the goal is 'begin some testing.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit conditional guidance for the optional parameter: pass changedAreas to re-test only what changed, omit for the full suite. It also routes the agent to get_test_run for outcomes. The missing piece is when to choose this over start_exploration.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Stop a running test run. Use it when a run was started against the wrong project, or is no longer needed - every test case it would still execute is billed.
| Name | Required | Description | Default |
|---|---|---|---|
| testRunId | Yes | The run to stop |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does add real value by disclosing the billing consequence of leaving a run active. However, it omits whether the stop is immediate or graceful, whether already-completed test cases and their results are retained, and what permissions are required for a mutation of this kind.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the action first, then the trigger conditions and the cost rationale. Every clause earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter, non-destructive-to-data mutation with a fully documented schema and no output schema, the description covers purpose, trigger, and cost motive adequately. The remaining gap is lifecycle detail — what happens to partial results after stopping — which is a minor omission at this complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single required parameter already documented as 'The run to stop'. The description adds no syntax, format, or sourcing guidance beyond that, so the schema does all the work — baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource pair ('Stop a running test run') that an agent can immediately distinguish from the sibling start_test_run and get_test_run. No ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use conditions ('started against the wrong project, or is no longer needed'), which is more than most definitions offer. It stops short of naming when-not or pointing at an alternative recovery path, but the selection context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
get_explorationget_issueget_test_runlist_issueslist_projectsstart_explorationstart_test_runstop_test_run
Add one secure layer between your agents and this server.