Skip to main content
Glama

kiwi-tcms-mcp

license: MIT node: >=18 MCP

A Model Context Protocol server for Kiwi TCMS. It exposes your Kiwi test-management data to any MCP client — Claude Code, Claude Desktop, Cursor, Cline, Zed, Continue, or your own — so you can browse and create test cases, build test plans and runs, and record execution results from a chat prompt.

  • 49 tools over Kiwi's JSON-RPC API — products, versions, builds, categories, components, tags, test plans, cases, runs, executions — plus a raw kiwi_rpc escape hatch for anything without a dedicated tool.

  • Runs locally over stdio, authenticates with your own Kiwi username + password. Nothing is hosted, no shared credentials. You get exactly the permissions your Kiwi account has.

  • No build step — plain Node ESM, node src/index.js.

MCP client  ──MCP (stdio)──▶  kiwi-tcms-mcp  ──HTTPS JSON-RPC──▶  Kiwi TCMS

Coming from TestRail? See the tool map — the surface mirrors the @uarlouski/testrail-mcp-server MCP so a migration is mostly renaming fields.

Example prompts

  • "List the test plans for product 3 in Kiwi."

  • "Create a test case under category 12: title 'Login rejects expired token', priority P1, with steps and expected result."

  • "Mark cases 4001–4010 in run 87 as PASSED with the comment 'regression clean on build 2.3.1'."

  • "Show the execution history for test execution 55."

Setup

New here? Follow SETUP.md — step-by-step, cross-platform, plain language.

Quick version:

git clone https://github.com/ShahjahanAliTaimoor/kiwi-tcms-mcp.git
cd kiwi-tcms-mcp
npm install

Wire it into your MCP client

Every MCP client takes the same three things — a command, its args, and an env block. Point it at src/index.js with your own Kiwi login:

{
  "command": "node",
  "args": ["/absolute/path/to/kiwi-tcms-mcp/src/index.js"],
  "env": {
    "KIWI_URL": "https://kiwi.example.com",
    "KIWI_USERNAME": "<your kiwi login>",
    "KIWI_PASSWORD": "<your kiwi password>"
  }
}
  • Claude Code — put it under the top-level "mcpServers" in ~/.claude.json (C:\Users\<you>\.claude.json on Windows), or run claude mcp add.

  • Claude DesktopmcpServers in claude_desktop_config.json (%APPDATA%\Claude\ on Windows, ~/Library/Application Support/Claude/ on macOS); use an absolute path to node.

  • Cursor / Cline / Continue / Zed — their MCP settings use the same command / args / env shape.

Restart the client. Tools show up namespaced, e.g. kiwi_get_products. On Windows use double backslashes in JSON paths. Full walkthrough in SETUP.md.

Environment variables

Var

Required

Notes

KIWI_URL

yes

Base URL only, no path. e.g. https://kiwi.example.com

KIWI_USERNAME

yes

Kiwi login (email or username)

KIWI_PASSWORD

yes

Kiwi password

KIWI_INSECURE_TLS

no

Set to 1 only if the host's TLS cert is not trusted by Node (self-signed / internal CA). Disables cert verification for the whole process — prefer trusting the CA instead.

Credentials sit in ~/.claude.json in plain text (same posture as most MCP client configs). Keep that file private and out of version control.

Related MCP server: QTM4J MCP Server

Verify

1. Auth + connectivity smoke test (hits a real instance):

# macOS / Linux
KIWI_URL=https://kiwi.example.com KIWI_USERNAME=you KIWI_PASSWORD=secret npm run smoke
# Windows PowerShell
$env:KIWI_URL="https://kiwi.example.com"; $env:KIWI_USERNAME="you"; $env:KIWI_PASSWORD="secret"; npm run smoke

Expect Auth OK and a list of products. TLS error on an internal CA? Prefix KIWI_INSECURE_TLS=1 (then add the same to your config env).

2. Offline checks (no network):

npm run list-tools     # prints all 49 registered tools
npm test               # lint/registration sanity (alias of list-tools)

Remote / hosted (HTTP) — experimental

Status: built and smoke-tested, not battle-tested in production. Only host this if you understand the exposure: it puts an authenticated path to your Kiwi instance on whatever network the host is reachable from, protected by a URL/bearer token. Put it behind a real host with a dedicated Kiwi service account, not a personal tunnel with your own login.

The stdio server only works with a local Claude (CLI / desktop app). To use it from claude.ai in a browser, run the HTTP transport on a publicly reachable host and add it as a custom connector.

1. Generate an auth token (any client that knows it can call the server):

npm run gen-token

2. Run the HTTP server:

$env:KIWI_URL="https://kiwi.example.com"
$env:KIWI_USERNAME="you@example.com"
$env:KIWI_PASSWORD="..."
$env:MCP_AUTH_TOKEN="<token from step 1>"
npm run http

Listens on 127.0.0.1:8787 by default (MCP_HTTP_HOST / MCP_HTTP_PORT to change). GET /health is open; the MCP endpoint is POST /<token>/mcp (token in the path, so no custom headers are needed) or POST /mcp with an Authorization: Bearer <token> header.

3. Expose it with a tunnel from your PC:

cloudflared tunnel --url http://localhost:8787
# or:  ngrok http 8787

4. Add the connector in claude.ai → Settings → Connectors → Add custom connector, URL = https://<tunnel-host>/<token>/mcp.

Env vars specific to the HTTP server:

Var

Default

Notes

MCP_AUTH_TOKEN

Required (min 32 chars). Callers pass it in the URL path or as a bearer token.

MCP_HTTP_HOST

127.0.0.1

Set 0.0.0.0 in a container.

MCP_HTTP_PORT

8787

MCP_RATE_PER_MIN

120

Per-IP request cap.

MCP_ALLOW_NO_AUTH

1 disables the token check entirely. Don't.

A Dockerfile is included (node src/http.js, port 8787). npm run http-smoke starts the server on a random port and runs initialize / tools/list / tools/call / bad-token against live Kiwi.

Tool map

Grouped by area. Names and grouping mirror the @uarlouski/testrail-mcp-server MCP to make a TestRail migration mostly mechanical.

Area

Tools

Discovery

kiwi_get_products, kiwi_get_versions, kiwi_get_builds, kiwi_get_categories, kiwi_get_components, kiwi_get_tags, kiwi_get_priorities, kiwi_get_users, kiwi_get_execution_statuses, kiwi_get_case_statuses, kiwi_get_plan_types, kiwi_get_classifications

Test cases

kiwi_get_test_cases, kiwi_get_test_case, kiwi_create_test_case, kiwi_update_test_case, kiwi_update_test_cases, kiwi_delete_test_case, kiwi_get_test_case_history, kiwi_add_test_case_comment, kiwi_add_test_case_tag, kiwi_add_test_case_component, kiwi_add_test_case_attachment, kiwi_export_test_cases_for_rag

Test plans

kiwi_get_test_plans, kiwi_create_test_plan, kiwi_update_test_plan, kiwi_add_case_to_plan, kiwi_remove_case_from_plan, kiwi_add_plan_tag

Test runs

kiwi_get_test_runs, kiwi_create_test_run, kiwi_update_test_run, kiwi_delete_test_run, kiwi_add_case_to_run, kiwi_remove_case_from_run, kiwi_get_run_cases, kiwi_add_run_tag, kiwi_add_run_attachment

Executions (results)

kiwi_get_test_executions, kiwi_update_test_execution, kiwi_update_executions_bulk, kiwi_add_execution_comment, kiwi_get_execution_comments, kiwi_add_execution_link, kiwi_get_execution_links, kiwi_get_execution_history

Raw / generic

kiwi_rpc, kiwi_delete_entity

Not ported (no Kiwi equivalent)

  • Shared steps — Kiwi has no shared-step concept.

  • Milestones — use product versions / builds instead.

  • Templates, configurations — Kiwi uses Environments instead; reach them via kiwi_rpc if the Environments plugin is enabled.

  • get_case_fields / resolve_case_field — Kiwi has no per-product custom-field schema RPC. Custom key/value data lives on TestCase.properties (kiwi_get_test_case with include_properties: true).

Field-name gotchas (TestRail habits that will bite)

TestRail

Kiwi

title

summary

custom_steps / custom_expected

text (single rich field)

suite_id + section_id

category_id (+ link to a plan)

type_id

is_automated (bool) + category

result status_id 1..5 (TestRail scale)

Kiwi status ids — call kiwi_get_execution_statuses first

For the authoritative shape of any object, call kiwi_rpc with <Model>.filter and one example id, or read the RPC docs.

Requirements

  • Node.js 18+ (uses global fetch)

  • A Kiwi TCMS account with API access on the instance you point at

Every tool runs with your account's Kiwi permissions. Kiwi enforces per-model add/change/delete/view grants, so a login can easily be able to create a test case but not edit or delete one. When a grant is missing Kiwi returns error -32098 ("Authentication failed when calling <Method>") — that's a permissions problem, not a bug. Common ones: auth.view_user for kiwi_get_users; testcases.change_testcase / testcases.delete_testcase for kiwi_update_test_case / kiwi_delete_test_case; likewise for testplans.*, testruns.*, testexecutions.*. Grant the matching permission (Kiwi admin → user/group), or use an account that has it.

Verified against a live Kiwi 16.3 instance: all read tools, kiwi_create_test_case and kiwi_add_test_case_comment work end-to-end; update/delete were correctly refused on a create-only account with -32098.

Contributing

Issues and PRs welcome. Handy while developing:

npm run list-tools   # every registered tool, offline
npm run smoke        # live Auth.login + Product.filter (needs KIWI_* env)
npm run http-smoke   # spins up the HTTP server and runs initialize/list/call/health

License

MIT

Available Tools

49 tools
kiwi_add_case_to_planB

Link an existing test case to a test plan.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes
plan_idYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are absent, so the description carries the full burden. It does not disclose whether the operation is idempotent, what happens if the case is already linked, any permission requirements, or the response/return behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence conveys the core function with no filler or repetition. Every word contributes to the meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-integer operation, the description conveys the essential action. But without annotations or an output schema, important context about side effects, idempotency, and error handling is missing, leaving the description minimally adequate rather than complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The parameter names case_id and plan_id are self-explanatory and the description maps them to 'test case' and 'test plan'. However, no additional semantics or constraints are provided beyond the raw schema, and schema description coverage is 0%.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Link') and resource ('existing test case' to 'test plan'), clearly indicating the operation. It distinguishes itself from the sibling kiwi_add_case_to_run by specifying 'test plan' rather than 'test run'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives such as kiwi_add_case_to_run or kiwi_remove_case_from_plan. The usage context is only implied by the tool name and one-line description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_case_to_runA

Add a test case to a run. This creates the TestExecution rows for that case.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
case_idYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral burden. It does disclose a meaningful side effect—creating TestExecution rows—but does not mention duplicate behavior, required run/case existence, permissions, or what happens if the case already belongs to the run.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences convey the core action and the key behavioral effect with no filler. The information is front-loaded and each sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation, the description is adequate but not complete. It covers the primary effect but lacks edge-case context such as idempotency, error conditions, and whether the run must have a particular status. No output schema or annotations compensate for these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides only names and integer types, and the description does not explicitly define run_id and case_id. However, the phrase 'Add a test case to a run' makes the mapping to case_id and run_id reasonably clear. The description adds minimal semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Add') and resource ('test case to a run'), and explicitly states the underlying effect ('creates the TestExecution rows'). This distinguishes it from siblings like kiwi_remove_case_from_run and kiwi_add_case_to_plan by naming the actual artifact being created.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage context is implied by the purpose: use this when adding a test case to a run. However, there are no explicit exclusions or comparisons to alternatives, such as when to use kiwi_add_case_to_plan or kiwi_remove_case_from_run instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_execution_commentB

Add a comment to a test execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
commentYes
execution_idYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only states the action. It does not explain whether the comment is appended to existing comments, what response is returned, any permission requirements, or other side effects beyond the basic mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with no wasted words. The verb and resource are front-loaded, and there is no redundant restatement of the tool name or obvious filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, no-output-schema mutation tool, the description is nearly sufficient, but it lacks usage context and behavioral detail. It is adequate for the operation itself, but an agent would still be guessing about return values and how to choose this tool over similar siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the parameter names are self-explanatory and 'test execution' clarifies that execution_id identifies the execution to comment on. The description adds minimal semantic context beyond the schema, which is acceptable given the simplicity of the two parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Add') and a clear resource ('a comment to a test execution'), directly distinguishing it from the sibling kiwi_add_test_case_comment which targets test cases. The resource is clearly identified, so an agent can tell the tools apart without deep inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like kiwi_add_test_case_comment or kiwi_get_execution_comments. There is no mention of prerequisites, such as the execution needing to exist, or when another tool would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_plan_tagB

Attach a tag (by name) to a test plan.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagYes
plan_idYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only states that a tag is attached. It does not mention whether the operation is idempotent, whether duplicate tags are prevented, whether the tag must pre-exist, or what happens on failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The entire description is a single efficient sentence with no filler. The core action and target are front-loaded and immediately scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation, the core semantics are present, but important contextual details are missing: how to obtain valid tag names, whether tags are created on the fly, and what happens if the plan_id does not exist. The sibling kiwi_get_tags exists but is not referenced.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must compensate. 'Tag (by name)' meaningfully clarifies that tag is a tag name rather than an ID, and 'test plan' helps map plan_id to the correct entity, but it still leaves the meaning and valid values of both parameters under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Attach a tag') and a specific target ('to a test plan'), with the useful qualification that the tag is identified 'by name'. This distinguishes it from sibling tools like kiwi_add_run_tag and kiwi_add_test_case_tag.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance about when to use this tool versus alternatives, nor any prerequisites such as whether the tag must already exist or the plan must be retrievable via get_tags or get_test_plans. Usage is only implied by the action itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_run_attachmentA

Attach a file to a test run. Provide base64-encoded content.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
filenameYes
content_base64Yes

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states that the tool attaches a file and that content must be base64-encoded; it does not disclose side effects, permissions, file size limits, overwrite behavior, or return behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences deliver the core action and the key encoding requirement without filler. The description is front-loaded with the primary purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter tool, the description is minimally adequate, but with no annotations and no output schema, an agent still lacks information about what happens on success or failure, response format, or any constraints beyond base64 encoding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description adds some meaning: content_base64 must be base64-encoded and run_id refers to a test run. However, filename and run_id semantics remain mostly implicit, and no additional constraints or formats are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Attach' with the resource 'file' and target 'test run,' making the tool's purpose immediately clear. This also distinguishes it from sibling tools like kiwi_add_test_case_attachment, which target test cases instead of runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies the tool should be used when attaching a file to a test run. It provides context but does not explicitly name alternatives or state when not to use this tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_run_tagB

Attach a tag (by name) to a test run.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagYes
run_idYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It conveys only that a tag is attached to a run; it does not state whether duplicate tags are suppressed, whether the operation creates tags or requires existing ones, what side effects occur, or what the response will be. This is minimal for an operation that mutates state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, focused sentence with no filler. The key qualifier 'by name' is included, and the sentence is front-loaded with the action and target.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter operation this is nearly enough, and both parameters are inferable from the sentence. However, with no annotations or output schema, missing behavioral details such as tag existence requirements, idempotency, and return value leave gaps that an agent may need to discover by trial or external documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since schema description coverage is 0%, the description must explain the parameters. 'By name' explicitly defines the tag parameter as a tag name rather than an ID, and 'a test run' ties run_id to the run being tagged. This adds meaningful semantics beyond the raw schema, though it does not cover potential tag-existence requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Attach a tag') and identifies the target resource ('test run'), and clarifies that the tag is provided by name rather than ID. It is clear and not tautological, though it does not explicitly distinguish this from sibling tagging tools like kiwi_add_test_case_tag or kiwi_add_plan_tag.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to prefer this tool over alternatives. The description does not mention prerequisites, such as whether the tag must already exist, nor does it contrast with kiwi_add_test_case_tag or kiwi_add_plan_tag. Usage context is left entirely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_test_case_attachmentB

Attach a file to a test case. Provide base64-encoded content.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes
filenameYes
content_base64YesBase64-encoded file bytes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the mutation and the base64 requirement; it does not mention whether attachments are appended or replaced, file size limits, required permissions, or what happens on failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the primary purpose. The second sentence gives the key encoding requirement, though it is partly redundant with the schema property description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple mutation with three required parameters and no output schema, the description plus schema are largely sufficient for an agent to construct the call. Missing details like acceptable file types, maximum size, or response expectations are not covered, but the core invocation is clear.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the description adds little beyond the schema: 'Provide base64-encoded content' restates the existing property description. It does not explain the semantics of case_id or filename, though these are somewhat inferable from their names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action with a specific verb and resource: 'Attach a file to a test case.' This distinguishes it from sibling tools like kiwiadd_test_case_comment, kiwi_add_test_case_tag, and kiwi_add_run_attachment, which operate on different resources or attach different entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear object-level context ('to a test case'), so an agent can infer when to use this tool. However, it does not explicitly mention alternatives, exclusions, or when to prefer another sibling such as kiwi_add_run_attachment over this one.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_test_case_commentC

Add a comment to a test case.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes
commentYes

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose side effects itself. It only states the add action and does not explain whether the comment is appended, whether the test case must already exist, what permissions are needed, or what happens on success or failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with no filler or repetition. It is appropriately terse for a simple operation, though the brevity leaves out useful context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter tool with no output schema, the description is minimally callable, but it lacks surrounding context such as return behavior, prerequisites, or how the comment is attached. It is adequate but not fully complete for an agent-facing definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, but it adds no real parameter meaning. It does not explain what case_id refers to beyond the obvious name, nor does it describe comment length, format, or constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: 'Add a comment to a test case.' The test-case qualifier differentiates it from sibling tools like kiwi_add_execution_comment, though it does not explicitly name or contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus kiwi_add_execution_comment, kiwi_add_test_case_tag, or other add-* siblings. No prerequisites, exclusions, or decision criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_test_case_componentB

Attach a component (by name) to a test case.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes
componentYes

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the mutation intent but does not reveal whether the operation is idempotent, whether it replaces existing components, whether the component must already exist, or what the response/error behavior will be.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no filler. Every phrase adds meaning: 'Attach' is the action, 'component (by name)' covers one parameter, and 'to a test case' covers the other.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is too sparse to fully support an autonomous agent. It omits prerequisites (e.g., component must exist, test case must be valid), the relationship to existing components, and what a successful attach returns.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema is bare (types only, 0% description coverage), so the description's 'by name' clarifies that component is a component-name string rather than an ID, and 'to a test case' identifies case_id as the target test case. It adds meaningful semantics but still does not explain how to source valid component names or the uniqueness/ownership requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb 'Attach' with the object 'a component (by name)' and target 'to a test case' precisely identifies the action and resource. It is clearly distinguishable from sibling tools like add_test_case_comment, add_test_case_tag, and add_test_case_attachment without needing to open the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage is implied: use it when a component must be associated with a test case. However, it does not explicitly mention when not to use it, mention prerequisites, or point to kiwi_get_components for discovering valid component names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_add_test_case_tagA

Attach a tag (by name) to a test case. The tag is created if it does not exist.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagYes
case_idYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral disclosure burden. It does meaningfully disclose the key side effect: 'The tag is created if it does not exist,' which tells the agent that pre-creating tags is unnecessary and that the operation handles missing tags gracefully. It does not mention return behavior or permission requirements, but the most important behavioral trait is explicitly stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no filler. The main action is front-loaded, and the second sentence adds a relevant edge case. Every word contributes to the meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple mutation tool with two required scalar parameters and no nested structure, the description covers the essential context: what is attached, to what, and the automatic creation behavior. It omits return format and error cases, but given the tool's low complexity and the clarity of the core behavior, this is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema only provides types, with 0% schema description coverage. The description compensates by clarifying that 'tag' is a tag name rather than an ID ('Attach a tag (by name)'), and that the operation targets a test case, which maps to case_id. This is valuable semantic information not present in the schema, though additional constraints like tag format or case existence are not addressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Attach'), a clear object ('a tag'), a target resource ('to a test case'), and the important nuance that the tag is referenced 'by name'. This clearly distinguishes it from sibling tools like add_test_case_component, add_plan_tag, and add_run_tag. An agent can immediately understand what operation this performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'to a test case' provides clear context for when this tool should be used and implicitly routes an agent away from plan- or run-level tag tools. It does not explicitly name alternatives or list exclusions, but the target entity is explicit and unambiguous. This is strong practical guidance for a simple tagging operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_create_test_caseA

Create a test case. values must include at least: summary, category_id (or category), priority_id (or priority), case_status_id (or case_status). Optional: text, notes, is_automated, script, extra_link, requirement, default_tester_id. Returns the created case.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYesField dict passed straight to TestCase.create

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the burden. It clearly states the tool creates a case and returns the created case, which conveys persistence and return behavior. It does not mention validation failures, permissions, or duplicate handling, but for a create operation this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with the most important constraint at the beginning. The required-field list is organized clearly, and the optional fields are grouped without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single nested-object parameter with no annotations or output schema, the description provides the essential required fields, optional fields, and return behavior. It could also point to lookup tools like kiwi_get_categories or kiwi_get_priorities for resolving IDs, but the definition is usable as-is.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema only says values is a field dict passed to TestCase.create. The description adds real value by listing required keys with aliases and several optional keys, substantially clarifying what belongs inside the opaque values object.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Starts with 'Create a test case' — a specific verb and resource. The required/optional field list makes it clear this creates a new test case and not a plan, run, or update operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'create' implies this tool is for new test cases, and the sibling list includes update/delete variants. However, the description does not explicitly state when to prefer this over kiwi_update_test_case or kiwi_delete_test_case, leaving some routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_create_test_planA

Create a test plan. values needs: name, text, product_id (or product), product_version_id (or product_version), type_id (or type). Optional: parent_id, is_active, extra_link.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. 'Create' implies a persistent write, but the description discloses no side effects, permissions required, reversibility, or what a successful call returns. For a mutation tool with zero annotation coverage, this is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the purpose is front-loaded, and the field specification is compactly formatted into required and optional groups. Every sentence earns its place with zero filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description equips an agent to construct the `values` payload with required vs optional keys and aliases, which is the critical calling surface for this free-form nested object. However, it omits field value types (e.g., boolean for is_active, URL format for extra_link), any return-value description (no output schema exists), and pointers to sibling lookup tools for resolving product_id/type_id, leaving the request understandable but not fully grounded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 0% coverage — `values` is a free-form object with additionalProperties and no nested documentation — so the description fully compensates. It enumerates required fields (name, text, product_id/product, product_version_id/product_version, type_id/type), optional fields (parent_id, is_active, extra_link), and documents aliases for each ID. This is exactly the semantic value the schema fails to provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

'Create a test plan' is a specific verb+resource statement that immediately distinguishes this from sibling mutations like kiwi_update_test_plan, kiwi_create_test_case, and kiwi_create_test_run. The field detail on `values` further scopes it to plan-specific attributes (product_id, type_id, parent_id), making its purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives such as kiwi_update_test_plan (for modifying existing plans) or kiwi_create_test_case. There are no exclusions, prerequisites, or pointers to companion lookup tools like kiwi_get_products or kiwi_get_plan_types for sourcing required IDs — the required-field list only weakly implies preparation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_create_test_runA

Create a test run. values needs: summary, plan_id (or plan), build_id (or build), manager_id (or manager). Optional: notes, default_tester_id, planned_start, planned_stop.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations and no output schema, the description must disclose behavioral traits, but it only states that a test run is created and what fields are needed. It does not mention return value, error conditions, permissions, idempotency, or side effects beyond the creation itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one efficient sentence that front-loads the action before detailing field requirements. Every part adds value, and the backticked field references keep the structure readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The input-side context is well covered, but because there is no output schema and no annotations, the description omits behavioral details such as what the API returns after creation or how failures surface. This leaves an agent needing external context for a complete call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides no key-level descriptions and allows any additional properties, so schema coverage is effectively 0%. The description compensates strongly by enumerating required keys, acceptable aliases, and optional fields, giving the agent the exact information needed to construct the values object.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Create a test run.' This clearly distinguishes it from sibling tools like kiwi_create_test_plan and kiwi_create_test_case, so an agent can tell what resource is being acted on.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when creating a test run and clarifies required versus optional value keys, but it does not explicitly state when to prefer this tool over alternatives or mention any exclusions. There is no comparison with update/delete-run siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_delete_entityA

Generic delete: calls .remove with the given ORM filter dict. DESTRUCTIVE. model e.g. "TestCase", "TestRun", "TestExecution", "Tag". query e.g. {"id": 5} or {"id__in": [1,2,3]}.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYes
queryYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, and it does so explicitly: 'DESTRUCTIVE' is highlighted, and the ORM remove mechanism is described, making clear that matching records are permanently deleted. The 'id__in' example also hints that bulk deletion is possible, which is critical behavioral context. It does not mention permissions, rollback, or cascading effects, but the destructive warning is the most important safety disclosure an agent needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: it opens with the core purpose, immediately warns 'DESTRUCTIVE', and then gives necessary examples. Every sentence earns its place; there is no filler or repetition of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description covers the essential aspects: what the tool does, the parameters and their formats, and the destructive nature. It does not mention return values, auth requirements, or error behavior, but for a simple delete tool the examples and warning provide enough context for an agent to invoke it correctly. The only notable gap is the absence of an exhaustive model list, but the 'e.g.' phrasing appropriately signals extensibility.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does. It explains that 'model' receives an ORM class name and provides concrete examples ('TestCase', 'TestRun', 'TestExecution', 'Tag'). It defines 'query' as an ORM filter dict and gives two realistic examples: a single id lookup and an id__in list. This transforms two bare schema properties into actionable parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('delete'), the resource ('<Model>'), and the mechanism ('calls <Model>.remove with the given ORM filter dict'). It identifies itself as 'Generic delete', which distinguishes it from specialized sibling tools like kiwi_delete_test_case and kiwi_delete_test_run. The examples of valid model names further clarify the intended scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explain when to use this generic delete versus the specialized delete tools for test cases and test runs. While 'Generic' implies it covers multiple models, it gives no explicit guidance about when to prefer it over siblings or whether the specialized tools should be used for TestCase/TestRun. As a result, an agent must infer the correct choice from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_delete_test_caseA

Permanently delete test cases matching an ORM filter dict. DESTRUCTIVE. Pass e.g. {"id": 123} or {"id__in": [1,2,3]}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesORM filter identifying the case(s) to remove

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral burden and handles it well: it explicitly labels the operation DESTRUCTIVE and permanently deletes, and shows that it can match and remove multiple cases via id__in. It does not mention cascading effects or return behavior, but the core irreversibility is clearly disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The destructive nature is front-loaded, and the examples are placed exactly where they are needed. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive delete tool, the description covers the important operational details: what kind of filter to pass, that deletion is permanent, and that multiple matches can be removed. Return behavior is not described, but it is not necessary to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes query as an ORM filter, but the description adds concrete examples ({"id": 123} and {"id__in": [1,2,3]}), clarifies it is a dict, and implicitly warns that multiple records may be affected. This goes well beyond the schema's basic description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('delete'), a specific resource ('test cases'), and the method of selection ('ORM filter dict'). It also clarifies the operation is permanent, making it unmistakable and distinct from sibling tools such as kiwi_delete_test_run or kiwi_update_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: use this tool when test cases should be permanently removed, and the DESTRUCTIVE warning sets strong expectations. It does not explicitly name alternatives or exclusions, but the intended use case is unambiguous for a delete operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_delete_test_runA

Permanently delete test runs matching an ORM filter dict. DESTRUCTIVE. Pass e.g. {"id": 42}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of exposing risk; it does so by stating 'DESTRUCTIVE' and 'permanently delete,' clearly flagging irreversibility. It does not detail all consequences of a broad or empty filter, but the core danger is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short fragments cover action, resource, danger, and a parameter example with no filler. The warning is front-loaded, and every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is sufficient for a simple delete-by-id call, but it leaves the semantics of the filter open-ended; an agent could pass a broad dict and delete many runs without warning. There is also no return-value or verification note, which matters because the tool has no output schema and no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema only shows an unconstrained 'query' object, so the description adds essential meaning by identifying it as an ORM filter dict and giving the concrete example {"id": 42}. More filter-operator detail would be useful, but the example compensates for the 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action ('permanently delete'), a specific resource ('test runs'), and a precise selection mechanism ('ORM filter dict'), so an agent can tell it apart from sibling delete tools such as kiwi_delete_test_case or kiwi_delete_entity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance and no mention of alternatives like kiwi_get_test_runs for verifying a filter before deletion. The only usage hint is the example filter {"id": 42}, which demonstrates syntax but not choice criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_export_test_cases_for_ragA

Bulk-export full text of test cases matching query, flattened for RAG / review (id, summary, category, priority, status, text, notes). Mirrors TestRail export_cases_for_rag.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax cases (default 500)
queryYesORM filter, e.g. {"plan": 12} or {"category__product": 3}

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. It discloses the operation's nature ('Bulk-export'), the output shape ('flattened' with enumerated fields), and the full-text scope, and the export framing implies no side effects. It does not address scale or pagination behavior beyond the schema's limit default, but for a read-style export the key behavioral traits are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler; the first is front-loaded with action, scope, and output format, and the parenthetical field list earns its place given there is no output schema. The TestRail mirror clause is a single short anchor rather than noise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderate-complexity tool with a nested query object, no annotations, and no output schema, the description covers purpose, filter semantics, result fields, and bulk behavior — the field enumeration compensates for the missing output schema. The notable gap is explicit routing against kiwi_get_test_cases, which is close in name and function.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both limit ('Max cases (default 500)') and query ('ORM filter, e.g. {"plan": 12}') already documented inline, so the baseline of 3 applies. The description only references `query` in passing ('matching `query`') and adds no syntax or semantic detail beyond the schema examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Bulk-export full text of test cases matching `query`') and specifies the flattened return shape with exact fields (id, summary, category, priority, status, text, notes). The RAG/review framing and field list distinguish it from sibling lookup tools like kiwi_get_test_cases and kiwi_get_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context — bulk full-text retrieval for RAG/review — and the 'Mirrors TestRail export_cases_for_rag' clause anchors its behavior. However, it never explicitly contrasts this with the sibling kiwi_get_test_cases / kiwi_get_test_case tools, nor states when not to use it. An agent must infer the distinction from the flattened, full-text output shape.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_buildsA

List builds. Typical filter: {"version__product": } or {"version": }.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'List builds' implies a read-only operation and the typical filters convey the query behavior, but it does not disclose pagination, response shape, or any limits. This is acceptable for a minimal list tool but not richly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one concise sentence plus two practical examples. Every word earns its place, and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (one optional parameter, no output schema), the description is mostly complete: it names the operation and the important filter patterns. The main gaps are the absence of explicit return-value expectations and any pagination/discovery details, but the schema covers the list-all behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents the `query` parameter and its generic ORM-filter/list-everything behavior, so the baseline is 3. The description adds meaningful build-specific filter keys (`version__product`, `version`) beyond the schema's generic examples, which helps the agent construct correct calls.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List builds.' It also provides the domain-specific filters (version__product, version), which clearly differentiates this from sibling list tools like kiwi_get_products and kiwi_get_versions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The typical filters strongly imply when to use this tool: when builds need to be listed/filtered by product or version. It does not explicitly mention alternatives or exclusion cases, but for a simple list tool this is sufficiently clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_case_statusesA

List TestCase statuses (CONFIRMED / PROPOSED / ...) with their ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. 'List' implies read-only behavior and 'with their ids' hints at the return shape, but it does not explicitly disclose side effects, permissions, pagination, or ordering. It is adequate for a simple enumeration tool but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that immediately states the action, resource, and relevant examples. There is no filler or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a low-complexity list tool with one well-documented optional parameter and no output schema. The description combined with the schema is sufficient for an agent to invoke it correctly, though it could have been slightly more explicit about the full set of returned fields or alternative tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% parameter documentation with a detailed description of the 'query' ORM filter and examples. Since the schema already covers parameter semantics well, the description does not need to add much; it adds no additional parameter-related detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and identifies the exact resource ('TestCase statuses'), including example enum values and the fact that IDs are returned. This clearly separates it from sibling tools like kiwi_get_execution_statuses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The tool's intended use is implied by 'List TestCase statuses', so an agent can infer when to call it. However, there is no explicit guidance about when not to use it or how it compares to related status/type list tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_categoriesA

List test-case categories for a product (closest thing to TestRail sections). Typical filter: {"product": }.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the behavioral disclosure burden. 'List' implies a read-only operation, and the typical filter communicates scope, but the description does not mention return behavior, pagination, error conditions, or explicitly confirm no side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with the main action front-loaded. The TestRail analogy earns its place by orienting the agent, and the typical filter example is directly actionable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one optional query parameter and no output schema, the description plus schema cover what an agent needs to call it. The main small gap is that the description does not explicitly state that omitting query lists all categories, though the schema does.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds a realistic typical filter and domain context. The example uses 'product' while the schema example uses 'product_id', which adds a small semantic ambiguity, but the ORM filter dict behavior is already well documented in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('test-case categories for a product'), and the parenthetical 'closest thing to TestRail sections' gives helpful domain context. It does not explicitly name sibling tools or contrast with them, but the entity is distinct enough among the many kiwi_get_* siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Typical filter: {"product": <product_id>}' example implies the main use case: fetching categories filtered by product. There is no explicit statement of when to use this tool versus alternatives or when not to use it, so usage guidance stays at the implied level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_classificationsA

List product classifications with their ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. 'List' clearly implies a read-only operation and the mention of IDs gives some output expectation, but the description does not disclose filtering behavior, pagination, permissions, or other side effects. The input schema covers the query parameter, but the tool description itself adds only minimal behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tightly worded sentence with no filler. The verb and resource are front-loaded, and every word contributes to the core meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list operation with a well-documented optional query parameter, the basic call is understandable. However, the description lacks usage guidance relative to sibling tools and provides no output schema or pagination/limit details. It is minimally adequate but leaves contextual gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for the single optional 'query' parameter, including an example and note that omitting it lists everything. The description adds no parameter-level meaning beyond what the schema already provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') and resource ('product classifications'), and mentions that IDs are included in the output. This makes the tool's purpose clear and distinguishes it from the many sibling getter tools such as kiwi_get_categories or kiwi_get_components.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus the many sibling getter tools. There are no conditions, exclusions, or alternatives mentioned, so an agent must infer usage from the name and general pattern rather than from explicit instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_componentsA

List components. Typical filter: {"product": }.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral context. 'List components' clearly signals a read-only listing operation, and the filter example hints at scoping, but it does not disclose return shape, pagination, ordering, or edge cases. It is not misleading, so a mid score is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, no filler, and the core operation is front-loaded. The typical filter example earns its place by showing the most common usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-optional-parameter list operation, the description plus schema cover what the agent needs to invoke it correctly. The lack of an output schema and pagination notes is a minor gap, but the operation is simple enough that this is not a significant omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully documents the single 'query' parameter with examples, so schema coverage is high and the baseline is 3. The description's typical filter example reinforces 'product' as a common filter but adds little beyond the schema's existing examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the action ('List') and the resource ('components'), and the typical filter example shows the expected scope. It does not explicitly contrast itself with sibling get_* tools, but 'components' is a distinct resource, so the purpose is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when the agent needs to list components, optionally scoped by a product filter. It provides no explicit alternatives or when-not-to-use conditions, but the resource-specific phrasing offers enough guidance for the common case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_execution_commentsB

Get the comments on a test execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry behavioral disclosure. It gives no information about read-only guarantees, response format, pagination, permissions, or error cases. The verb 'Get' weakly implies a read operation, but the description is otherwise silent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundant words. It front-loads the core action and resource, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity—one required integer parameter and no output schema—the description is minimally sufficient for an agent to understand what to call and what it returns in a general sense. However, it lacks explicit details on the returned comment structure or any behavioral constraints, leaving some context undefined.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the undocumented execution_id parameter. The phrase 'on a test execution' does clarify that execution_id identifies the execution whose comments are retrieved. However, it adds no details about the parameter's origin, validity, or format beyond what the parameter name and schema type already suggest.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get the comments on a test execution' uses a specific verb and resource, clearly identifying the action. It implicitly distinguishes itself from siblings like kiwi_add_execution_comment and kiwi_get_execution_history, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives such as kiwi_add_execution_comment or kiwi_get_execution_links. The description only states what the tool does, not the conditions or context for selecting it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_execution_historyA

Return the change history for a test execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The verb 'Return' implies a read-only operation and there is no hint of destructive side effects. However, no annotations are provided, and the description does not disclose ordering, pagination, permission requirements, or what kinds of changes are included in the history.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler or redundant restatement. Every word contributes to the core meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter read tool, the description is minimally adequate: it names the target and the operation. But with no annotations, no output schema, and no detail about return shape, ordering, or error behavior, an agent still has to infer important details on first use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one integer execution_id with no property description, so the description must carry the semantic burden. 'For a test execution' does clarify that execution_id identifies the execution whose history is requested, but the description does not explain where to obtain the ID or any constraints beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb and resource: it returns the change history for a test execution. This distinguishes it from siblings like kiwi_get_test_executions, which returns execution data, and kiwi_get_test_case_history, which targets test cases rather than executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus the many related lookup tools. There is no mention of alternatives, prerequisites, or the intended scenario beyond the core description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_execution_statusesA

List TestExecution statuses (PASSED / FAILED / BLOCKED / ...) with their ids. Needed before updating an execution result.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral burden. The verb 'List' implies a read-only operation, and the status examples add useful context. Still, it does not explicitly state side-effect-free behavior, return format, or any quirks such as filtering limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only two short sentences with no filler. The action and resource are front-loaded, the status examples are immediately useful, and the workflow context is stated in the second sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple lookup with one optional parameter and no output schema, the description covers enough: what is returned (statuses and ids) and why it is needed. The exact response shape and enum values remain implicit, but the description is adequate for tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema documents the single optional query parameter with 100% coverage, including examples and how to list everything. The description adds no additional parameter-level meaning, but the schema already handles that responsibility, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: 'List TestExecution statuses ... with their ids.' The examples 'PASSED / FAILED / BLOCKED' plus 'ids' clarify what the tool returns. It does not explicitly name sibling tools like kiwi_get_case_statuses, but the 'TestExecution' qualifier disambiguates from related status-listing tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Needed before updating an execution result' gives explicit context for when to call this tool, tying it to the update workflow. However, it does not mention when not to use it or point to alternatives such as kiwi_get_case_statuses, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_plan_typesA

List test-plan types (Unit / Integration / Function / ...) with their ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. 'List' clearly signals a non-mutating read and 'with their ids' clarifies the return content, but it does not mention filtering behavior, pagination, or edge cases. This is adequate but minimal for a simple lookup.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One compact sentence front-loads the action and resource, adds illustrative examples in parentheses, and states the output expectation. There is no filler or redundant restatement of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a trivial lookup tool with one self-documenting optional parameter and no output schema, the description plus schema are sufficient for correct invocation. The only minor gap is explicit usage guidance relative to siblings, which is not critical for such a simple catalog operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single optional query parameter is already fully documented in the input schema, including an example filter dict and the 'Omit or pass {} to list everything' behavior. The description adds no parameter-specific details, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'List test-plan types' with concrete examples (Unit / Integration / Function / ...) and explicitly states the output includes ids. This clearly distinguishes it from sibling get_* tools that target other Kiwi entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or alternative guidance is provided. The intended use is implied by the name and resource, but the description does not tell an agent when to choose this over other catalog-like get_* tools or exclude any cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_prioritiesB

List test-case priorities and their ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are absent, so the description carries the full burden of behavioral disclosure. 'List' implies a read-only call, but it does not state whether all priorities are returned by default, what the response shape is, or any permissions or side effects. The schema covers the omit-query behavior, but the description adds little beyond the basic action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; every word contributes to identifying the resource and the return value. It is concise without being incomplete to the point of tautology.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity read-only list with one optional parameter and full schema coverage, the description is minimally viable. However, with no annotations and no output schema, the lack of explicit usage guidance and output structure beyond 'ids' leaves noticeable gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with a clear explanation of the Kiwi ORM filter dict and explicit guidance to omit or pass {} to list everything. The description adds no parameter detail, but with full schema coverage the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete action ('List') and a specific resource ('test-case priorities'), plus the return focus ('and their ids'). It is not a tautology and is clearly distinct from sibling get_* tools, none of which target priorities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives like kiwi_get_categories or kiwi_get_case_statuses. The optional query behavior is documented in the schema, but the description itself provides no selection criteria, exclusions, or context for choosing it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_productsB

List Kiwi products (the TestRail 'project' equivalent). Filter dict optional.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral burden. 'List' communicates a read-only operation and 'Filter dict optional' indicates the query behavior, but pagination, result cardinality, or output shape are not disclosed. This is sufficient for a simple list operation, though minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with a front-loaded verb and no redundant filler. The parenthetical clarifying TestRail terminology earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-optional-parameter list tool, the schema covers the parameter and the description states the operation clearly. There is no output schema, but 'List' implies a collection of products; however, no annotations or return-format hints mean the description is adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: the schema already documents the Kiwi ORM filter dict, gives examples, and states that omitting it or passing {} lists everything. The tool description merely repeats 'Filter dict optional' and adds no semantics beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and the resource ('Kiwi products'), and the TestRail 'project' equivalence adds useful domain context. It does not explicitly contrast with sibling getter tools, but the product resource is distinct enough that the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to invoke this tool instead of other 'get_*' tools, nor any mention of prerequisites or cases where a filter should be supplied. The description only restates the basic operation, leaving the agent to infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_run_casesA

List the test cases currently in a run.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. 'List' clearly indicates a read-only operation, and 'currently' signals that the returned set reflects the run's present membership. It does not mention pagination or ordering, but those are minor for such a simple retrieval.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single focused sentence with no filler. The action and target are front-loaded, and every word contributes to understanding the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only endpoint, the description sufficiently conveys what the tool does. Since there is no output schema, the return value is inferable as test cases, and missing details like ordering or filtering are minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage and only provides the name 'run_id' with integer type. The description mentions 'a run' but does not explain how run_id is obtained, what it references, or any constraints. The parameter's meaning is mostly recoverable from the name, not from the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') and a specific resource ('test cases currently in a run'). It clearly distinguishes this from siblings like kiwi_get_test_runs and kiwi_get_test_cases, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'currently in a run' implies this tool is for retrieving the case members of one specific run, but it does not explicitly say when to prefer this over alternatives or address exclusions. Usage context is recognizable but left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_tagsA

List tags (the TestRail 'labels' equivalent). Filter e.g. {"name__startswith": "reg"}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries full weight. It discloses that this is a read operation and that filtering is supported via ORM-style lookups, but it does not mention pagination, response shape, or whether the filter supports multiple conditions—gaps that are notable because no annotations exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that leads with the action and resource, then adds a clarifying parenthetical and a concrete filter example. There is no filler, repetition, or unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only list tool, the description covers the core purpose and filterability. However, with no output schema, it omits the return format, pagination behavior, and tag field details, leaving the agent to infer what a successful response looks like. The lack of annotations makes this gap more significant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already explains the 'query' parameter with examples and the contrast between omitting it and passing {}. The description adds one supplementary filter example but no deeper semantics about supported lookup operators or allowed keys, which is consistent with the high schema coverage baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'List tags', and adds useful domain context by calling them the TestRail 'labels' equivalent. It does not explicitly contrast with sibling list tools, but the resource name alone distinguishes it from all other kiwi_get_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The filter example ('name__startswith') implies how to use the tool, and the schema adds the omit/empty behavior. However, there is no explicit guidance on when to prefer this tool over alternatives or mention of practical use cases like retrieving tag IDs before attaching tags.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_caseA

Fetch one test case by id, optionally with its comments, custom properties and attachment list.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesTestCase id
include_commentsNo
include_propertiesNo
include_attachmentsNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must bear the behavioral disclosure burden. It conveys that this is a non-destructive fetch and that the three includes are optional, but it does not describe behavior for missing/invalid ids, default values of the boolean flags, or the shape of the returned test case.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single efficient sentence front-loads the action and resource, then adds the optionality. No filler or redundant wording.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core action and optional parameters, which is adequate for a simple getter. It is less complete about default include behavior and the response structure, and with no output schema or annotations the description leaves those details unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 25%; only case_id is described. The description compensates by explaining that the three boolean parameters correspond to optionally including comments, custom properties, and an attachment list, adding meaning not present in the bare boolean schemas.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Fetch') and a precise resource ('one test case by id'), and it enumerates the optional data (comments, custom properties, attachment list). This clearly distinguishes the tool from sibling kiwi_get_test_cases, which fetches a collection, and from mutation tools like kiwi_update_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied: retrieve a single test case by its id. However, the description does not explicitly name alternatives (e.g., kiwi_get_test_cases for multiple cases) or state when not to use this tool, leaving routing partly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_case_historyC

Return the change history for a test case.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Return the change history,' offering no detail about pagination, ordering, what changes are included, or whether any constraints apply to retrieving history.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single clear sentence with no filler or redundant content. It is concise and front-loaded, though it is also minimal enough that some depth is sacrificed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter getter, the description is minimally viable: the agent knows what resource is targeted and what action is performed. However, with no annotations, no output schema, and no detail about history content or ordering, the description leaves ambiguity about what the agent will actually receive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain the case_id parameter beyond implying it identifies a test case. It does not mention where the ID comes from, whether it must be an existing case, or any format expectations beyond the schema's integer type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Return') and the resource ('change history for a test case'). It does not explicitly distinguish itself from sibling tools like kiwi_get_test_case or kiwi_get_execution_history, but the resource is specific enough to be understandable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance about when to use this tool versus alternatives. There is no mention of when to prefer this over kiwi_get_test_case or kiwi_get_execution_history, and no context about typical use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_casesA

Search test cases. query is a Kiwi ORM filter dict. Common keys: {"plan": }, {"category__product": }, {"summary__icontains": "data usage"}, {"case_status__name": "CONFIRMED"}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict. Omit/{} returns ALL cases — usually narrow it.

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the disclosure burden, and it does communicate the core read-only search behavior and filter semantics through examples. However, it does not state whether the result is a list, mention pagination or limits, or note the all-cases behavior; the all-cases warning is only in the schema, so the description alone is only partially transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the purpose, and the example filter dicts are high signal. There is no filler; each sentence either states the action or provides the syntax needed to call the tool correctly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, optional-query search tool, the description plus schema gives enough information for an agent to construct valid calls. The only minor gaps are output shape and pagination, which are less critical here because the tool's purpose and simple schema are clear.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the query parameter is already explained, so the baseline is 3. The description goes further by giving concrete kiwi ORM keys with example values, which teaches the agent the exact filter format and common access patterns.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action, 'Search', and a clear resource, 'test cases', so an agent can recognize it as a filtered lookup tool. It does not explicitly distinguish it from singular kiwi_get_test_case or kiwi_get_run_cases, so it is not a top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description demonstrates practical filter patterns for plan, product, summary, and status, and the schema warns that omitting the query returns all cases. It never names alternatives or says when not to use this tool, so usage context is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_executionsA

Search test executions (results). query common keys: {"run": }, {"run": , "case": }, {"status__name": "FAILED"}, {"tested_by__username": "sqa1"}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the burden; 'Search' signals a read-only operation and the query examples expose some behavior. It doesn't mention pagination, default behavior when query is empty, match semantics, or response scope, but these are less critical for a simple search tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The entire definition is one front-loaded sentence with compact JSON examples and no filler. Every clause adds useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single flexible query parameter and no output schema, the examples give enough to start invoking, but the description omits response details, pagination, and query combination semantics. It is adequate but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema only defines an opaque query object, so the common-key examples such as run, case, status__name, and tested_by__username add essential meaning. They don't exhaust all keys or value formats, but they compensate for the 0% schema description coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly uses 'Search test executions (results)' with a specific verb and resource, and the parenthetical clarifies these are result records rather than test cases or runs. It doesn't explicitly name sibling alternatives, but the resource is distinct enough among the get_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for querying execution result records and gives common filter shapes, so an agent can infer the primary use case. It doesn't say when to prefer this over related tools like kiwi_get_execution_history or kiwi_get_run_cases, nor give exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_plansA

Search test plans. query common keys: {"product": }, {"name__icontains": "regression"}, {"is_active": true}.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. It adds useful filtering semantics beyond the schema by showing accepted query keys like product, name__icontains, and is_active. However, it does not disclose default behavior, return format, pagination, or whether it returns all plans when query is empty.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: a clear purpose statement followed by a tight set of query-key examples. Every sentence earns its place, with no filler or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with one opaque parameter and no output schema, the description covers query construction but omits expected return behavior and default query semantics. Since there are no annotations or schema-level descriptions, this leaves some ambiguity, though basic usage is inferable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema only defines 'query' as an object with additionalProperties allowed and no property documentation. The description compensates meaningfully by listing common accepted keys and example values, giving concrete guidance that the schema alone does not provide. It is not exhaustive, but it is sufficient for typical searches.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Search test plans') with a clear resource and filtering scope, making it easy to understand what the tool does. It does not explicitly differentiate itself from sibling tools, but the resource is unambiguous and distinct from test case or test run getters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Search' and the example query keys imply the tool is used to find or filter test plans, but no explicit when-to-use guidance or alternative tool comparisons are provided. There are no exclusions or when-not-to-use conditions, so usage is mostly implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_test_runsA

Search test runs. query common keys: {"plan": }, {"build__version__product": }, {"summary__icontains": "trip"}, {"stop_date__isnull": true} (still open).

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral burden. 'Search test runs' clearly indicates a non-destructive read, and the '(still open)' note clarifies the stop_date filter semantics. It does not disclose return format, pagination, or whether an empty query returns all runs, so behavior is only partially transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence states the operation, then immediately gives compact code examples. Every part earns its place, and no redundant wording is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter search tool with no output schema and no annotations, the common-key examples provide the core invocation knowledge an agent needs. Missing details include whether query is optional, what an empty search returns, and response shape or pagination; these gaps prevent a higher score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the schema only defines query as an unconstrained object, so the description compensates well by listing common JSON keys and example value types. It gives enough concrete syntax to construct a valid filter object, though it does not enumerate all possible keys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific verb 'Search' and the resource 'test runs,' making the tool's function immediately clear among siblings like kiwi_get_test_executions or kiwi_get_run_cases. It stops short of explicitly contrasting it with those siblings, so it doesn't quite earn a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides concrete query patterns (plan, product, summary icontains, still-open runs), which conveys how to use the tool and implies it should be used for searching test runs. However, there is no explicit guidance about when to prefer this over sibling tools such as kiwi_get_test_executions or kiwi_get_run_cases, and no exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_usersA

List Kiwi users. Filter e.g. {"username": "sqa1"} or {"email__icontains": "acme.com"}. Requires the 'auth.view_user' permission on your Kiwi account; without it Kiwi returns error -32098 ("Authentication failed").

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden and does well by naming the required permission and the exact error/code returned when it is missing. It could also mention read-only semantics or return shape, but the permission/error detail is substantive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. It front-loads the purpose, then adds filter examples and a permission warning, each sentence earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-optional-parameter read tool, the description covers purpose, filter syntax, permission requirements, and failure behavior. The main gap is the absence of explicit return-value or pagination details, which is more noticeable because no output schema or annotations are provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers the query parameter generically with 100% coverage, giving a baseline of 3. The description adds value by providing user-specific filter examples ('username', 'email__icontains') that are not present in the schema's generic product-oriented example.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'List Kiwi users', a clear verb and resource, and the filter examples confirm it returns user records. The 'users' resource is unambiguous and no sibling tool handles users, so it is well differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The tool's intended use is immediately apparent: list Kiwi users with optional filtering. It provides practical filter patterns and a permission prerequisite, though it does not explicitly compare against sibling getter tools or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_get_versionsA

List product versions. Typical filter: {"product": }.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoKiwi ORM filter dict, e.g. {"name": "My Product"} or {"product_id": 3}. Omit or pass {} to list everything.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. 'List' implies a read-only retrieval operation, and the typical filter hints at scoping behavior. However, it does not disclose response shape, pagination behavior, or any other runtime traits, though for a simple list tool this is a moderate gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler. The primary action is front-loaded, and the typical filter guidance is provided immediately afterward, making the description easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read operation with one optional query parameter, the description plus schema is mostly sufficient. However, there is no output schema and no mention of return values, ordering, or limits, so an agent has to infer what a successful response looks like.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents the query parameter thoroughly, including examples and the omit-to-list-everything behavior, so schema coverage is 100%. The description adds a typical filter example, but it is largely redundant and its key 'product' is slightly inconsistent with the schema's 'product_id' example, reducing its additional value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'List product versions.' This clearly distinguishes the tool from siblings like kiwi_get_products and kiwi_get_builds, and there is no vagueness about what entity is being retrieved.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Typical filter: {"product": <product_id>}' provides clear contextual guidance for a common use case. It does not explicitly exclude alternatives or state when-not-to-use, but the unique resource makes the intended usage fairly obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_remove_case_from_planB

Unlink a test case from a test plan.

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYes
plan_idYes

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of explaining behavior. 'Unlink' indicates a state change but does not disclose whether the operation is idempotent, what happens if the case is not currently linked, whether the test case or plan is modified beyond the relationship, or what the expected response is.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundant wording. It front-loads the core action clearly, though it is so brief that some useful behavioral detail is omitted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-integer mutation with no nested objects or output schema, the description is minimally viable: an agent can infer what to call and what parameters to provide. However, it leaves behavioral details like error cases and side effects unstated, which is a noticeable gap given there are no annotations to fill it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides zero description coverage for both parameters. The description helps only by implying that case_id refers to the test case and plan_id refers to the test plan, but it adds no further meaning about relationship requirements, value ranges, or behavioral effects of the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific action, 'unlink', and identifies the exact object and container: a test case and a test plan. This makes the core purpose unambiguous and distinguishes it from related operations like adding a case to a plan or unlinking from a run, though it does not explicitly name sibling alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: use this when a test case should no longer be associated with a test plan. There is no explicit guidance on when not to use it, what prerequisites exist, or how it differs from similar operations like removing a case from a run.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_remove_case_from_runB

Remove a test case (and its executions) from a run.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
case_idYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description bears the full burden of behavioral disclosure. It does disclose a meaningful side effect — that executions are also removed — which exceeds a bare 'removes from run' statement. However, it does not state whether the removal is permanent, whether executions are irreversibly destroyed, or how the tool behaves if the case is not in the run.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single efficient sentence with zero filler. The core action is front-loaded, and the side-effect clarification is placed in a compact parenthetical that earns its space. Every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-integer-parameter tool with no output schema, the description covers the essential action and its cascade effect. However, because annotations are absent, the safety profile of this destructive operation is left unclear — an agent cannot tell whether removal is reversible or whether it permanently deletes execution data. A brief caveat about permanence would make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for undocumented parameters, but it adds no meaning beyond what the parameter names already imply. The phrase 'from a run' weakly maps run_id to a run and case_id to a test case, but there is no detail on types, relationships, or expected value semantics. For a 0%-coverage schema, this is insufficient compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (remove), a resource (test case), and a location (from a run), which fully captures the operation. It also distinguishes itself from the similarly named sibling kiwi_remove_case_from_plan by specifying 'run' rather than 'plan'. The parenthetical '(and its executions)' adds precision about scope, removing any ambiguity about what removal entails.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as kiwi_remove_case_from_plan or kiwi_delete_test_case. An agent must infer from the name alone that this targets run membership rather than plan membership or permanent case deletion. The description gives no explicit context, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_rpcA

Call ANY Kiwi JSON-RPC method directly. method e.g. "TestCase.filter", "TestPlan.tree", "Bug.report". params is the POSITIONAL argument array Kiwi expects (usually a single dict for *.filter, or [id, {patch}] for *.update). Unguarded — you are responsible for correct method + params. Full method list: https://kiwitcms.readthedocs.io/en/latest/modules/tcms.rpc.api.html

ParametersJSON Schema
NameRequiredDescriptionDefault
methodYesNamespace.method, e.g. TestRun.filter
paramsNoPositional params array; defaults to []

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral disclosure burden. It explicitly says 'Unguarded' and that the caller is responsible for correct method and params, signaling that this tool can perform arbitrary, potentially destructive RPC calls. It also explains the positional param convention and links to the full API list, though it does not cover auth, rate limits, or error/return behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: it states the purpose first, then param semantics, then the safety warning, and ends with the reference URL. Every sentence contributes meaningful guidance with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a generic passthrough tool, the description is largely complete: method syntax, param shape, examples, risk warning, and canonical documentation link are all present. It could be slightly stronger by explicitly advising agents to prefer specialized sibling tools for covered methods or by noting that return values vary by method, but the documentation URL mitigates these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters with 100% coverage. The description adds valuable meaning beyond the schema: concrete method examples and the positional-array convention, including typical shapes for *.filter and *.update calls. This helps an agent construct params correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action: 'Call ANY Kiwi JSON-RPC method directly.' It includes concrete method examples and contrasts with the specialized sibling tools by emphasizing direct, unguarded access to the full RPC API. An agent can immediately understand this is a generic passthrough/fallback tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when this tool is appropriate: any Kiwi JSON-RPC method, especially those not covered by dedicated wrappers. It warns the caller to be responsible and points to the full method documentation. It does not explicitly state 'prefer a sibling tool when one exists,' but the generic-purpose framing makes the intended use clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_executions_bulkA

Record results for many cases in one run (mirrors TestRail add_results_for_cases). For each item, finds the execution for {run_id, case_id}, sets its status, and optionally adds a comment. status may be a status id (number) or name (string).

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
resultsYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full disclosure burden. It transparently explains that the tool mutates executions by finding them via run_id and case_id, setting status, and optionally adding a comment. It also clarifies the accepted status forms. It does not discuss not-found or partial-failure behavior, but the core side effects are well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact: two sentences, front-loaded with the main action, and free of filler. The TestRail mirror reference is brief and useful for users familiar with that API.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter mutating tool with no annotations and no output schema, the description covers the essentials: inputs, selection logic, mutation behavior, and optional comment. It stops short of describing what happens when an execution is not found or whether the write is atomic, but these are reasonable omissions given the clarity of the described bulk-update flow.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does so by explaining the relationship between run_id and case_id, specifying that status can be an id or name, and noting that comment is optional. This adds meaningful semantics beyond the bare schema types, though it does not fully document edge cases or expected result values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Record results for many cases in one run') and elaborates the exact behavior: find the execution for run_id/case_id, set its status, and optionally add a comment. This clearly distinguishes it from sibling kiwi_update_test_execution, which updates a single execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'many cases in one run' clearly signals when this bulk tool is appropriate, and the existence of kiwi_update_test_execution implies the single-case alternative. It does not explicitly name an alternative or provide a when-not-to-use condition, but the usage context is strong and unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_test_caseA

Update one test case. values is a partial field dict (e.g. {"summary": "...", "text": "...", "case_status_id": 2}).

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes
case_idYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral disclosure burden. It adds meaningful context by explaining that `values` is a partial field dict, implying partial updates rather than full replacement. However, it omits details such as required permissions, error behavior, and whether the operation is idempotent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One concise sentence with a leading action, a clear parameter explanation, and an illustrative example. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the essential invocation: which case to update and how to provide partial field values. However, with no output schema and no annotations, key contextual information like return value, error handling, and valid field names is missing, so it is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It clearly explains `values` as a partial field dict with concrete examples, which adds real meaning beyond the bare schema. `case_id` is self-explanatory as an integer identifier.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Update') and resource ('one test case'), making the tool's purpose clear. The singular phrasing distinguishes it from the sibling kiwi_update_test_cases without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'one test case' clearly signals this is for single-case updates, implying the plural sibling is for bulk updates. It does not explicitly name alternatives or state when not to use it, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_test_casesA

Apply the same values patch to many cases (mirrors TestRail update_cases). Runs one TestCase.update per id and reports per-id success/failure.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes
case_idsYes

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly reveals that the operation is non-atomic (one TestCase.update per id) and that results report per-id success/failure, which is meaningful behavioral information beyond the tool name alone. It does not detail output format or failure continuation, but the key non-obvious behavior is covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the core operation appears first, followed by the TestRail reference and the per-id execution detail. Every sentence contributes useful information without repeating schema or annotation content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately explains what the tool does and its batch behavior, but for a tool with no annotations, no output schema, and an open `values` object, it lacks the exact return shape, valid value keys, and what happens when individual updates fail. This is sufficient to select the tool but not fully sufficient to invoke it with complete confidence.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does add meaning by describing `values` as a shared patch applied to many `case_ids`, which is helpful. However, `values` is an unconstrained object with additionalProperties true, and the description does not enumerate or link to the valid fields, leaving a significant semantic gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states a specific action ('Apply the same values patch to many cases') and resource (test cases), and clearly distinguishes itself from kiwi_update_test_case by emphasizing bulk operation over many ids. The detail 'Runs one TestCase.update per id' further clarifies the exact operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly conveys the intended usage: updating multiple cases with an identical `values` patch. It does not explicitly name alternatives or state exclusions such as 'use kiwi_update_test_case for a single case', but the bulk-vs-singular context is strong enough to guide tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_test_executionA

Update one execution. values e.g. {"status_id": 2} (see kiwi_get_execution_statuses), or {"tested_by_id": 5, "stop_date": "2026-09-08 16:30:00"}.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes
execution_idYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description is the only source of behavioral information. It accurately signals mutation and shows concrete values with a date format and a status reference, but it does not disclose whether the update is partial or replacing, what happens on invalid keys, or what the API returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One information-dense sentence with a front-loaded action, followed by two short JSON examples and a cross-reference. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The example values and date format make basic calls feasible, but the values object is unconstrained in the schema and described only with 'e.g.', leaving the full set of updatable fields unknown. No return behavior is mentioned and there is no output schema, so an agent may not know what to expect after the call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It does so for the values object with realistic examples (status_id, tested_by_id, stop_date) and a specific date format, plus a reference for valid statuses. execution_id is left to its self-explanatory name and integer type, which is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with the specific action 'Update one execution', clearly identifying the resource and singular scope. It distinguishes itself from the bulk sibling kiwi_update_executions_bulk by explicitly saying 'one'. The example values further clarify the operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The singular scope implies this is for single-execution updates rather than bulk updates, and the pointer to kiwi_get_execution_statuses indicates when to consult a sibling for valid status IDs. It does not explicitly name the bulk alternative or list exclusions, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_test_planB

Update a test plan with a partial field dict.

ParametersJSON Schema
NameRequiredDescriptionDefault
valuesYes
plan_idYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the transparency burden. It discloses that the update is partial—only fields in the provided dict are changed—and that it is a mutation. However, it does not state permissions, whether the plan must exist, error behavior, or what response is returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with the action and resource; no filler or repetition. It earns its place by adding the 'partial field dict' semantics. Appropriate for the tool's small surface area.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no annotations or output schema, and the values schema is intentionally unconstrained, so the description should provide more operational context. Missing valid field names, example usage, or update semantics leave an agent guessing what to pass in values. As a mutation endpoint, it also omits response and error behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. 'Partial field dict' tells the agent that values is a mapping of field names to update, which adds meaning to the open-values schema, but it does not enumerate valid test-plan fields or clarify plan_id beyond its name. The schema's types are left unexpanded.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the action ('Update') and the resource ('a test plan') clearly, and the 'partial field dict' qualifier distinguishes it from full create/replace operations. It is not merely restating the tool name; it adds the partial-update semantic. The 'test plan' resource scope also differentiates it from sibling update tools for cases, runs, and executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to prefer this over create_test_plan, delete_test_plan, add_case_to_plan, or other mutation tools. The only usage cue is the verb 'Update', which is implied rather than explicit. There are no prerequisites, exclusions, or alternative-selection conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

kiwi_update_test_runA

Update a test run with a partial field dict (e.g. {"stop_date": "2026-09-08 17:00:00"}).

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYes
valuesYes

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It does reveal that the update is partial, which is useful, and gives a date format example, but it does not mention permissions, validation, side effects, or whether changes are reversible. This is too thin for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One focused sentence with a concrete example and no wasted words. The key information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is minimal for a tool whose main parameter is an unrestricted object. It lacks a list of updatable fields, any output contract, and behavioral context such as validation or error conditions. An agent would likely need to guess or consult another source before calling this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds some meaning by explaining that 'values' is a partial field dict and provides one example key, but it does not enumerate supported fields, clarify run_id semantics, or explain what fields are valid in the arbitrary values object.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Update'), a clear resource ('a test run'), and the mechanism ('partial field dict'). This distinguishes it from sibling update tools like kiwi_update_test_case and kiwi_update_test_execution by resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies this tool is for modifying an existing test run, not for creating, getting, or deleting one. It does not explicitly name alternatives or exclusions, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 49 tool updatesv1.0.0
    • First observedkiwi_add_case_to_plan
    • First observedkiwi_add_case_to_run
    • First observedkiwi_add_execution_comment
    • First observedkiwi_add_execution_link
    • First observedkiwi_add_plan_tag
    • First observedkiwi_add_run_attachment
    • First observedkiwi_add_run_tag
    • First observedkiwi_add_test_case_attachment
    • First observedkiwi_add_test_case_comment
    • First observedkiwi_add_test_case_component
    • First observedkiwi_add_test_case_tag
    • First observedkiwi_create_test_case
    • First observedkiwi_create_test_plan
    • First observedkiwi_create_test_run
    • First observedkiwi_delete_entity
    • First observedkiwi_delete_test_case
    • First observedkiwi_delete_test_run
    • First observedkiwi_export_test_cases_for_rag
    • First observedkiwi_get_builds
    • First observedkiwi_get_case_statuses
    • First observedkiwi_get_categories
    • First observedkiwi_get_classifications
    • First observedkiwi_get_components
    • First observedkiwi_get_execution_comments
    • First observedkiwi_get_execution_history
    • First observedkiwi_get_execution_links
    • First observedkiwi_get_execution_statuses
    • First observedkiwi_get_plan_types
    • First observedkiwi_get_priorities
    • First observedkiwi_get_products
    • First observedkiwi_get_run_cases
    • First observedkiwi_get_tags
    • First observedkiwi_get_test_case
    • First observedkiwi_get_test_case_history
    • First observedkiwi_get_test_cases
    • First observedkiwi_get_test_executions
    • First observedkiwi_get_test_plans
    • First observedkiwi_get_test_runs
    • First observedkiwi_get_users
    • First observedkiwi_get_versions
    • First observedkiwi_remove_case_from_plan
    • First observedkiwi_remove_case_from_run
    • First observedkiwi_rpc
    • First observedkiwi_update_executions_bulk
    • First observedkiwi_update_test_case
    • First observedkiwi_update_test_cases
    • First observedkiwi_update_test_execution
    • First observedkiwi_update_test_plan
    • First observedkiwi_update_test_run

TDQS

B3.4/5.0

Scored across 49 tools

Disambiguation4/5

Most tools map to a distinct resource and action, and the descriptions make the case/plan/run/execution domains fairly clear. The main ambiguities are kiwi_delete_entity and kiwi_rpc, which intentionally overlap with specific tools, plus kiwi_get_run_cases vs kiwi_get_test_executions for retrieving run-level case data.

Naming Consistency4/5

The vast majority follow a consistent kiwi_ + verb_noun pattern: get_, create_, update_, delete_, add_, remove_. Minor deviations include the generic kiwi_delete_entity, the raw kiwi_rpc, and the slightly inconsistent kiwi_update_executions_bulk versus kiwi_update_test_execution.

Tool Count2/5

49 tools is well into the over-scoped range and creates a heavy tool-selection burden for agents. Even though Kiwi TCMS is a broad test-management domain, the count is inflated by many reference-list tools and singular/bulk/comment/link variants that could be consolidated.

Completeness4/5

Core workflows are well covered: test cases, plans, runs, executions, comments, attachments, links, history, and bulk updates. Gaps exist around removing comments/tags/links and creating reference entities like builds, versions, or products through dedicated wrappers, but kiwi_rpc and kiwi_delete_entity provide fallbacks for those edge cases.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

  • An MCP server that provides access to Testiny projects, test cases and test runs

  • Free public MCP for AI agents — 193 tools, 44 workflows. No API key.

  • Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.

  • Your org's AI agents, tasks, runs, search, and brain files as MCP tools and resources.

Related MCP Servers

  • A
    license
    A
    quality
    F
    maintenance
    Enables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.
    42
    743 npm
    44
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    An MCP server that exposes the QMetry Test Management for Jira Cloud REST API as tools for MCP-compatible clients. It enables users to manage test cases, test cycles, test executions, test plans, folders, and automation rules through natural language interactions.
    30
    26 npm
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    An MCP server that exposes Kiwi TCMS as a set of AI-callable tools, enabling assistants to create and manage test plans, test cases, test runs, and executions directly from a conversation.
    1
    -