TestRail MCP Server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v2.5.0
Disambiguation4/5
Most tools have distinct purposes, but get_results and get_tests could be confused (results vs tests). Mutate/query tools are differentiated by action parameter, which helps. Overall, clear boundaries.
Naming Consistency3/5
Mixed conventions: 'add_' for creation, 'mutate_' for create/update, 'query_' for retrieval. Plural vs singular (update_case vs update_cases). Some verbs are uncommon (mutate, query). Not fully consistent.
Tool Count3/5
24 tools is slightly above the ideal range (3-15). It feels a bit heavy but covers a wide domain. Each tool seems necessary, but some could be merged (e.g., mutate_* with action parameter).
Completeness2/5
Missing delete operations for cases, runs, sections, suites. No test plan management. While CRUD for core entities is present, lack of deletion causes dead ends. Utilities like get_labels are helpful.
Average 4/5 across 24 of 24 tools scored. Lowest: 2.9/5.
See the Tool Scores section below for per-tool breakdowns.
- 6 of 6 community issues answered or closed in the last 6 months
- 18 commits in the last 12 weeks
- Last stable release on August 25, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate the tool is a write operation (readOnlyHint=false) and not destructive. The description adds no additional behavioral context, such as whether results are appended or replaced, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that communicates the core function without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple add operation, the description combined with the full schema coverage and annotations is adequate. It lacks explicit guidance on usage context, but the essentials are covered. A more detailed description could mention that results are appended to an existing run.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover all parameters (100%), providing detailed information for run_id and each field in the results array. The description itself adds no extra parameter semantics beyond what the schema already includes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (add) and the resource (test results to a test run). It is specific but does not explicitly distinguish from the sibling tool 'add_results_for_cases'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'add_results_for_cases' or 'get_results'. The description lacks context about prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a non-read-only, non-destructive mutation. The description adds valuable behavioral context: directories are automatically zipped and there is a 256MB upload limit. These details go beyond the annotation hints and help the agent anticipate side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences, front-loaded with the primary purpose and followed by two relevant operational details. Every sentence earns its place, with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core action and significant behavioral constraints (zip, size), but it lacks return-value expectations or prerequisites like requiring the entity to exist. With no output schema, the description carries the burden of explaining outcomes, and it is silent on what the caller should expect on success.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all three parameters with descriptions (100% coverage), so the baseline is 3. The description adds context for file_path (directory zipping) but does not introduce new meaning for entity_type or entity_id beyond the schema's existing descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Add an attachment to a test case or test run in TestRail.' It specifies the verb and resource, making the core function obvious. However, it does not differentiate from the sibling tool add_attachment_to_run, which likely serves a similar purpose for runs, creating potential ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to choose this tool over alternatives like add_attachment_to_run. The description mentions behavioral details (directory zipping, size limit) but does not state exclusions, prerequisites, or scenarios where another tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description only restates the scope without adding extra behavioral context like error handling or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that directly states the tool's purpose without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read-only tool, the description is sufficient when combined with annotations and schema. It lacks explicit output format details, but that is not critical given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the single parameter test_id is fully described in the schema. The description adds no additional parameter meaning, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'results' scoped to 'a specific test', which is unambiguous. It does not explicitly name alternative tools, but the resource is unique among siblings, so it distinguishes implicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternative guidance is given, but the purpose implies usage when you need results for a known test_id. This is minimally adequate but lacks clear context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe read operation. The description adds context about how template IDs determine available fields, which is useful, but it does not disclose additional behavioral traits like pagination or return format. Given the annotations, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the purpose, and each sentence adds value. It is appropriately sized with no wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with one parameter and strong annotations, the description is complete enough. It states the purpose and the significance of the output. However, since there is no output schema, it could be slightly more explicit about returning a list of templates, but the verb 'Get' implies this adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for the single parameter project_id, including a description that cross-references get_projects. The tool description adds no additional parameter semantics. Since the schema does the heavy lifting, the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and names the resource ('test case templates') scoped to a project. It also explains the significance of template IDs, making the purpose clear. However, it does not explicitly distinguish from siblings like get_case_fields, so it lacks differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to obtain template IDs before creating/updating test cases, but it does not explicitly state when to use this tool versus alternatives. The schema description for project_id provides a helpful cross-reference to get_projects, but no explicit when/when-not guidance is present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, idempotent, and non-destructive. The description adds the dual-mode behavior (single vs. many) driven by payload.action, which is beyond the annotations and helps the agent understand how the tool's output varies. No contradictory behavior is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the essential purpose and operation selection without any filler. It earns its place efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a moderately complex oneOf schema and no output schema. The description explains the operation modes but does not describe the return format, error behavior, or any limitations like pagination. While annotations cover safety, the description leaves out details that would be helpful for a complete understanding of expected responses.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already well-documented. The description reiterates the action parameter's role ('set payload.action to one or many') but does not add new meaning to suite_id or project_id beyond what the schema provides. This aligns with the baseline for fully described schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'retrieve' and the resource 'test suite(s)', and specifies the two modes: single or all for a project. This distinguishes it from sibling tools like get_cases or query_run by targeting suites specifically, especially with query_suite vs mutate_suite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives operational instructions ('Set payload.action to one or many') but does not provide guidance on when to use this tool versus alternatives. It lacks explicit when-to-use or when-not-to-use context, such as 'use mutate_suite for modifications.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, covering the safety profile. The description adds no behavioral context beyond the optional filter, which is parameter-related. This is acceptable given the annotations but not enhanced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant wording. Every word contributes to the core meaning: the action, the resource, the scope, and the optional filter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with good annotations and full schema coverage, the description adequately scopes the operation and the optional filter. However, with no output schema, it does not describe return values or pagination, but this is not critical for such a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both run_id and status_id fully described in the schema, including a reference to get_statuses for valid IDs. The tool description adds no additional parameter meaning, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get' with a clear resource 'tests' and scoping 'for a test run', distinguishing it from siblings like get_cases or get_results. The optional status filter further clarifies the exact functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing tests for a test run, but provides no explicit guidance on when to prefer this tool over alternatives or any exclusion criteria. The context is clear but no alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the operational duality (one vs many) and the requirement to set payload.action, but does not reveal additional behaviors such as file overwrite handling, error responses, or rate limits. With annotation coverage, the description's additional value is modest.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise at two sentences, with the primary function first and the operation selector second. Every word earns its place; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a query/download tool with no output schema, the description adequately conveys the two call patterns and the filesystem effect. However, it does not specify the return format for the 'many' action (e.g., an array of attachment metadata) beyond the schema's property descriptions. This is a minor gap given the schema richness, but the description alone is slightly incomplete for the listing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% parameter description coverage, detailing action, attachment_id, output_file, entity_type, and entity_id. The description reinforces the action parameter usage ('Set payload.action to 'one' or 'many'') but adds no new information beyond the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's function: downloading a single attachment or listing all attachments for a test case/test run. The verbs 'download' and 'list' plus the resource are specific, and the second sentence explains the action discriminator, distinguishing it from sibling add_attachment tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage contexts via the two action modes but provides no explicit guidance on when to choose this over the alternative sibling tools like add_attachment or get_cases. It does not state exclusions or prerequisites, so usage guidance is only partially inferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only indicate false for readOnly, idempotent, and destructive, which are not very informative. The description adds the behavioral trait of partial updates, but doesn't disclose return values, side effects, or error handling beyond the schema. It's adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose, and every sentence adds value. It's concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential update behavior and partial-update semantics, but it doesn't mention what the tool returns or any side-effect beyond the update. Since there is no output schema, the description could do more to explain the response, but the key operational details are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds meaningful semantics by explaining that only specified fields are changed (partial update). This goes beyond the schema's description of the fields parameter, which focuses on the flat-map structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a test case in TestRail' with a specific verb and resource. It doesn't explicitly distinguish from the sibling tool update_cases, but the singular 'a test case' and the tool name imply single-case updates, making the purpose clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage context: it requires knowing valid field names from get_case_fields and supports partial updates, which tells the agent how to use it. It doesn't explicitly mention when not to use it (e.g., for bulk updates), but the prerequisite and partial-update behavior are useful guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a non-readonly, non-idempotent, non-destructive operation. The description adds the fact that field names must be valid and come from get_case_fields, but it does not disclose other behavioral traits such as return values, error conditions, or side effects. With annotations already covering the safety profile, the description provides only marginal additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, both sentences earning their place. It states the core purpose first, then adds the critical prerequisite about field names. There is no filler or redundancy, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (three parameters, one nested object, no output schema), the description provides essential workflow context by pointing to get_case_fields. It does not explain return values or error behavior, but the rich schema descriptions and annotations cover most needs. The description is sufficient for an agent to invoke the tool correctly in a typical workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and each parameter (title, section_id, fields) has a thorough description, including the flat map nature of fields and the need to use system_name values. The main description adds no significant parameter semantics beyond what the schema already provides. Baseline 3 is appropriate due to full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description immediately states the tool's function: 'Create a new test case in TestRail.' This is a specific verb+resource that clearly distinguishes it from sibling tools like update_case or get_case. The addition of the TestRail platform name adds context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains a key prerequisite: knowing valid field names returned by get_case_fields. This gives the agent a clear actionable step before calling add_case. It does not explicitly mention when not to use the tool (e.g., for updates), but the name and purpose make that distinction obvious. The guidance is valuable and directly references a sibling tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnlyHint=false, idempotentHint=false, and destructiveHint=false, so the description does not need to repeat those. The description adds minimal behavioral context beyond confirming it's a write operation ('Add'), but does not disclose side effects like partial failures or validation behavior. It stays consistent with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that states the core functionality and key differentiator. No filler or redundant wording, earning a top score for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with full schema coverage and clear annotations, the description is sufficiently complete. It identifies the action, target, and unique usage caveat. While it omits details about the response format, the absence of an output schema and the presence of thorough parameter docs make this adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (run_id, case_id, status_id, comment, defects) fully described including where to source IDs. The description itself adds no additional parameter-level meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Add'), resource ('test results to a test run'), and a distinguishing mechanism ('using case IDs instead of test IDs'). This effectively differentiates it from sibling tools like add_results, which likely uses test IDs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'using case IDs instead of test IDs' provides clear context for when to use this tool versus alternatives, implying that add_results is for test IDs. However, it does not explicitly name the alternative tool, stopping short of a full when/when-not specification.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds the useful detail that the response includes custom fields, which is not in the annotations or schema. This extra context about the return payload justifies a score above baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that front-loads the purpose and the key detail about custom fields. There is no wasted wording or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and strong annotations, the description is mostly complete. It explains what is retrieved (including custom fields), though it does not exhaustively list all return fields. The absence of an output schema means a bit more detail could help, but the core guidance is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema fully documents the single parameter (case_id) with a description and example format. The tool description adds no additional information about the parameter, so the baseline score of 3 applies given the 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action (get) and resource (test case), and distinguishes itself from sibling tools like get_cases (plural) and get_case_fields by noting it returns the case's custom fields. This clearly identifies what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving a single test case's details, but provides no explicit guidance on when to use this over alternatives like get_tests or get_cases. The context is inferred from the name and sibling list rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and destructiveHint, so the description doesn't need to repeat safety. It adds context about the return content (IDs and names for add_case), but doesn't discuss edge cases like pagination or multi-suite behavior (which is covered in the schema). Meets the baseline given annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action, no filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no output schema, the description covers what it returns and how it's used. It lacks detail on the response format (e.g., JSON shape) but that's not expected without an output schema. The presence of optional parameters is documented in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, so the baseline is 3. The description doesn't add anything about parameters beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') with a clear resource ('sections for a project') and explicitly states the return type (IDs and names) plus its downstream use with add_case, distinguishing it from sibling tools like get_tests or get_cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies usage by stating that the returned IDs/names can be used with add_case, giving a clear scenario. However, it doesn't name any alternatives or explicitly state when not to use it, so it falls short of the highest bar.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal a non-read-only, non-idempotent mutation. The description adds that it creates or updates a test run, but it does not disclose side effects such as whether updating replaces all fields or whether creating without case_ids includes all cases. No contradiction but limited extra context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose, no repetition or fluff. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a complex oneOf schema, and the description orients the agent appropriately without restating schema details. It could mention required fields per action, but the schema already covers that, so overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema coverage is 100% with detailed descriptions for each parameter including action consts. The description's instruction 'Set payload.action to create or update' adds little beyond the schema's own descriptions, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verbs 'Create' and 'update' and identifies the exact resource 'test run in TestRail'. It clearly distinguishes this tool from sibling tools like mutate_suite by naming the target resource and operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells the agent to set payload.action to 'create' or 'update', providing clear context for when to use each mode. However, it does not explicitly name alternatives or exclusion criteria, so it falls short of the highest bar.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=false, destructiveHint=false. The description adds no further behavioral details beyond restating the action mechanism, such as error cases, partial update behavior, or permission requirements. It does not contradict the annotations, but it contributes minimal additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that immediately names the resource and scope. Every word earns its place; there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the dual-action complexity (oneOf two payload shapes), the schema and description together provide sufficient guidance for an agent to select and invoke the tool correctly. The absence of an output schema means return-value documentation is unnecessary. Minor gap: prerequisite context like suite_id requirements is embedded in the schema but not reiterated in the description, though that is acceptable given schema richness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% description coverage, with each property (name, action, project_id, suite_id, parent_id, section_id, description) already documented. The description's instruction to set payload.action duplicates schema constraints and adds no new semantic meaning. Baseline 3 is appropriate because the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a new section or update an existing section in TestRail.' It specifies the resource (sections) and the action (create/update), which distinguishes it from sibling tools focused on suites, runs, or queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the operation selection mechanism via payload.action, implying that this tool handles both creating and updating sections. However, it does not explicitly contrast with alternatives like get_sections or other mutation tools, so there is no strong 'when not to use' guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a mutation (readOnlyHint=false), and the description adds the dual create/update mode controlled by 'payload.action'. It doesn't disclose additional behavioral details like whether updates are partial, permissions required, or side effects. Given the annotations cover some safety profile, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that clearly states the purpose and the key usage instruction. Every word earns its place; it is concise without being under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (one parameter with two possible shapes), the description covers the essential context: what the tool does and how to select create vs. update. The schema enriches the details. However, it does not mention potential return values or side effects, though the annotations provide some behavioral context. Slightly more detail could be added, but it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all parameter descriptions are already present in the schema. The description only restates the 'action' values ('create' or 'update') which are already defined as consts in the schema. It adds no new semantic meaning beyond what the schema provides, matching the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Create a new test suite or update an existing one in TestRail.' This clearly states the tool's function and distinguishes it from sibling tools like mutate_section or mutate_run. The mention of 'create' or 'update' operations further pins down the exact behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use the tool: when you need to create or update a test suite. It specifies the operation via the 'action' field, giving concrete usage context. However, it does not explicitly mention alternatives or when not to use this tool, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds no extra behavioral context beyond what the schema provides (the action parameter), so it meets the baseline but offers no additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and contains no wasted words. It efficiently conveys both the operation and the mechanism.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (oneOf schema) and strong annotations, the description is sufficient. It lacks mention of return format or pagination, but these are not critical for a read-only query tool and are not required by the absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with detailed descriptions for the payload and action values. The description merely restates the action logic ('Set payload.action to one or many'), adding no new semantics beyond what the schema already defines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves 'a single project or all projects in TestRail,' using the specific verb 'retrieve' and resource 'projects.' It distinguishes from sibling tools by the resource (projects vs tests, cases, suites, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool by stating what it retrieves, and the sibling context clearly shows this is the project-query tool. However, it does not explicitly mention alternatives or exclusions, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint. The description adds the dual-mode behavior ('one' vs 'many'), which is useful context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, front-loaded sentences that efficiently convey the tool's purpose and key usage pattern.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description adequately explains the tool's retrieval purpose and action modes. While no output schema is provided, the annotations cover safety, and filtering details are in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented. The description clarifies the action parameter's role but doesn't add new info beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves test runs (single or all) for a project, with a specific verb and resource. It distinguishes from sibling tools like mutate_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like get_cases or get_results. The description implies read-only use but doesn't differentiate within sibling retrieval tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, covering the safety profile. The description adds the domain-specific detail about Multi-select case field type ID 12, but does not disclose output format or behavior for invalid or unknown refs. This is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no filler. Every phrase adds meaningful context: numeric reference IDs, Multi-select field, type ID 12, textual values, and project scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple three-parameter tool with fully documented schema and safety annotations, the description gives sufficient context for selection and invocation. The absence of an output schema leaves the exact return shape unspecified, but this is a minor gap given the tool's simplicity and clear purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all three parameters with descriptions, so the description does not need to repeat parameter details. It adds overall context, such as the transformation from reference IDs to textual values, but no parameter-specific meaning beyond the schema. Baseline 3 is appropriate given the high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific operation: resolving numeric reference IDs to textual values for a Multi-select case field. It identifies the field type (ID 12) and the project scope. This clearly distinguishes it from sibling tools like get_case_fields or query_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies the intended use case: converting opaque numeric references into readable values for a Multi-select field in a project. It does not explicitly name alternatives or state when not to use it, but the context is clear enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already declaring readOnlyHint=true, idempotentHint=true, and destructiveHint=false, the description adds behavioral value by specifying that the operation returns 'all' configuration groups and configurations, implying no filtering. It does not mention pagination or return format, but the safety profile is fully covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the action and resource, with no unnecessary words. Every word contributes to the meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with strong annotations, the description is adequately complete: resource, scope, and all-encompassing behavior are stated. No output schema exists, but the description makes the primary return concept clear without needing to list result fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents project_id with a description ('The ID of the project'), covering 100% of parameters. The tool description adds no additional parameter semantics beyond restating the project scope.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') with a clear resource ('configuration groups and configurations') and scope ('for a project in TestRail'). It clearly distinguishes from sibling tools like get_tests or get_users by naming the exact resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: this is the tool to retrieve configuration groups and configurations for a given project. It does not name alternatives or state when not to use it, but the resource name and scope are unambiguous relative to the listed siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is known. The description adds value by clarifying the return content ('label IDs and titles') and the terminology ('sometimes called tags'), which is helpful beyond the structured annotations. No contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no wasted words. The key verb and resource are front-loaded: 'Get all available test case labels.' The second sentence adds useful return and use-case context. It is appropriately sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with one parameter and no output schema, the description adequately conveys what the tool does and what it returns. It mentions the return items ('label IDs and titles') and their purpose. It does not explicitly state that the output is a list, but 'all available' implies a collection, which is sufficient for this low-complexity tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter (project_id) with a clear description that fully aligns with the tool description. Since schema description coverage is 100%, the description adds little additional meaning to the parameter. Baseline 3 is appropriate; the schema already documents the parameter well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get all available test case labels (sometimes called tags) for a project.' It uses a specific verb ('Get') and a specific resource ('test case labels'), and the parenthetical note ('sometimes called tags') prevents ambiguity. This distinctly differentiates it from sibling tools like get_tests or get_users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when the output can be used: 'Returns label IDs and titles that can be used when creating or updating test cases.' This gives clear context for invocation. It does not explicitly mention when not to use it or name alternative tools, but for a simple read operation the use case is well implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate only that the operation is not read-only, not idempotent, and not destructive. The description adds the dependency on get_case_fields and the need for valid field names, but it does not disclose whether updates are partial/merge or full replacement, or how errors are handled beyond the schema's note. Some behavioral context is added, but significant gaps remain.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, prerequisite, and efficiency comparison. Every sentence earns its place with no redundancy, and the most important information (bulk update) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a bulk mutation tool with no output schema and sparse annotations, the description covers purpose, usage alternative, and prerequisite. It lacks details about return value or partial failure behavior, but the rich schema and sibling context compensate somewhat. A more complete description might mention whether the update merges or replaces existing fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with detailed descriptions for both parameters: case_ids is a simple array, and fields is fully explained with format, example, and error conditions. The description's mention of 'same field values' adds little beyond the schema, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a bulk update of multiple test cases with the same field values, using a specific verb ('bulk update') and resource ('test cases'). It distinguishes itself from the sibling update_case tool by noting it is more efficient than calling update_case multiple times, and it mentions the prerequisite of get_case_fields for valid field names.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides usage guidance by comparing to update_case, stating 'More efficient than calling update_case multiple times,' which implies the appropriate context for using this tool. It also clarifies that get_case_fields must be called first to obtain valid field names, serving as a clear prerequisite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds behavioral context by explaining that omitting project_id returns all fields across all projects, making the scope transparent. It also discloses the return format ('types and options for dropdown fields'), going beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, parameter guidance, and return info. Front-loaded with the main action, no redundant phrasing, and every sentence contributes value. Structurally excellent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional parameter, high-quality annotations, and no output schema, the description covers the necessary context: what it returns, how to scope it, and the behavior when omitted. There are no significant gaps for the complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides a detailed description of project_id, covering both the normal use case and the fallback behavior. The tool description mostly reiterates this information without adding new semantic details about parameter formatting, constraints, or usage beyond the schema coverage, which is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get the field schema for test cases for a specific project.' It specifies the resource (field schema), the scope (test cases, project), and the return content (fields with types and options). This distinguishes it from sibling tools like get_cases or get_templates which manage test data rather than schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'You should normally provide project_id to get fields applicable to your project' and notes that omitting it is 'rarely what you want.' While it doesn't name alternative tools, it clearly instructs when to use the parameter and when not, which is strong usage direction for a simple tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, covering safety. The description adds behavioral context not captured by annotations: client-side filtering occurs after fetching all cases, API-side filtering is more efficient, and output_file can save responses for large datasets. It also discloses that returns include IDs, titles, and requested fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action, and every clause adds value. It efficiently conveys scope, filtering options, and return fields without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 7 parameters, no output schema, and nested objects, the description covers what is returned, how to filter efficiently, and references companion tools for finding IDs. Together with the schema, it gives the agent sufficient context to select and invoke the tool correctly for typical use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description provides a useful high-level summary of filtering options (section, API params, where) but does not add syntax details beyond the schema. Given the schema already explains each parameter thoroughly, this is adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get all test cases for a project' with a specific verb and resource, and distinguishes from sibling get_case (singular) by implying a bulk-list operation. It also lists key filtering capabilities, making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use client-side 'where' vs API-side 'filter' (efficiency) and references get_case_fields, get_sections, and get_suites for prerequisites. However, it does not explicitly contrast with get_case or query_project for single-case or cross-project queries, so it stops short of full exclusionary guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds that it returns priority IDs and names but does not disclose other behavioral details such as authentication or rate limits. This meets the baseline for annotation-backed read tools.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. The first sentence states the core purpose with examples, and the second adds the return format and usage context. Ideal size for a simple lookup tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a low-complexity tool with no parameters and no output schema. The description fully explains what it returns (priority IDs and names) and why those are useful (for creating/updating test cases). No additional context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description appropriately focuses on the return value rather than parameter semantics, and there are no parameters to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and resource ('all available test case priorities'), with examples and a clear statement that it returns priority IDs and names. It is distinct from sibling tools, none of which mention priorities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly states the returned IDs/names can be used when creating or updating test cases, providing contextual usage. However, it does not explicitly exclude alternative tools or state when not to use it, so it lacks explicit when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, idempotent, and non-destructive. The description adds behavioral context by revealing that it returns both IDs and names, and that these are interoperable with other tools. This goes beyond the annotations and helps the agent understand the tool's place in workflows.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys all necessary information: what it gets, example values, and the return format. Every clause earns its place; there is no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple no-parameter, read-only listing tool, the description is complete. It explains the output (status IDs and names) and how it can be used elsewhere. No output schema exists, so the description's mention of return values is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter details, and there is no schema coverage gap. It adds no extra parameter semantics, but none are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb ('Get') and resource ('all available test statuses'), and provides concrete examples (Passed, Failed, Blocked). It also indicates the output type (status IDs and names), which distinguishes it from sibling tools like get_tests or get_cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when this tool is useful by noting that the returned IDs and names can be used with add_result and get_tests. This gives clear context for usage, though it does not explicitly mention when not to use it or compare to alternatives. Given that no sibling tool directly overlaps with statuses, this is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description discloses important behavior: global fetch may be forbidden for non-admin accounts, and it falls back to merging users across projects. This adds meaningful context about permissions and fallback, which annotations don't convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the core purpose, the second explains the fallback scenario. Every word earns its place with no redundancy or padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with only two optional parameters and rich annotations, the description fully covers purpose, scope, and fallback edge cases. No output schema exists, but the tool's behavior and expected results are clear from the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full descriptions for both parameters (project_id and fallback_all_projects) with 100% coverage. The description adds context about merging behavior, but it doesn't add new parameter-level details beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets active users from TestRail, with specific scope (globally or per-project). It distinguishes from siblings, which all deal with tests, cases, sections, or runs rather than users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to retrieve active users globally or per-project) and describes the fallback behavior for non-admin accounts. It doesn't explicitly name alternatives, but no sibling tool serves the same purpose, so this is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a mutating operation (readOnlyHint=false). The description adds crucial behavioral context beyond annotations: the tool is deprecated, and there is a maximum upload size limit. It does not detail all potential side effects, but the annotations and description together offer a solid understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, perfectly front-loaded with the core purpose, followed by deprecation and size limit. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool with no output schema, the description fully covers what it does, when not to use it (deprecation with alternative), and a critical constraint (size). It is complete for effective tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full descriptions for both parameters (run_id, file_path), so baseline is 3. The description adds the 256MB upload size constraint, which directly informs the file_path parameter and adds practical meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add' and the resource 'attachment to a test run' within TestRail. It also distinguishes itself from the sibling 'add_attachment' tool by explicitly marking itself as deprecated and recommending the alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Deprecated: Prefer add_attachment tool', giving the agent direct guidance to use a different tool instead. It also provides the maximum upload size (256MB), which is a practical constraint for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![testrail-mcp-server MCP server](https://glama.ai/mcp/servers/uarlouski/testrail-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/uarlouski/testrail-mcp-server)

Score Badge

Copy to your README.md:

[![testrail-mcp-server MCP server](https://glama.ai/mcp/servers/uarlouski/testrail-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/uarlouski/testrail-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/uarlouski/testrail-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server