Skip to main content
Glama
TheQtCompanyRnD

squish-mcp

Official

Squish MCP Server

IMPORTANT

This repository is outdated and no longer maintained. Anew version of Squish MCP is available in the Qt Customer Portal.

The new version equips AI agents with the capability of directly interacting with your application under test (AUT), which enables them to generate sensible test cases for complex applications such as Qt Creator, and reduces the amount of tokens required for generating tests. See the release announcement for more details.


Squish MCP v0.1.0 (outdated)

Squish Model Context Protocol (MCP) server enables AI agents to run and create Squish (documentation) test scripts and test suites and analyze the results.

Demo video of Squish MCP in action

Requirements

Tested on Windows 11, macOS, and Ubuntu 22.04.

Known Limitations

  • Supported AUTs. Only Qt Widgets and QML-based applications are currently tested.

  • Real names should be enabled. Without enabling real names, the performance gains provided by Squish MCP are severely reduced. To enable real name generation, modify the snapshot filter file at <path-to-squish>/etc/qt_snapshot_filter.xml and change <realname exclude="yes"/> to <realname exclude="no"/>.

  • Avoid spaces in paths. VS Code has issues with spaces in file paths, which can cause otherwise correct configurations to fail. See vscode#214931.

Related MCP server: cypress-runner-mcp

Installation

  1. Clone the repository:

    git clone <repository-url>
    cd squish-mcp
  2. Install Python dependencies:

    uv sync
  3. Configure the MCP server in VS Code. The exact configuration entry will vary depending on your agent. For GitHub Copilot in VS Code, it may look like this (.vscode/mcp.json in your working directory):

    {
        "servers": {
            "SquishMCP": {
                "command": "uv",
                "args": ["run", "--directory", "<path-to-squish-mcp>", "squish_mcp"],
                "env": {
                    "SQUISH_PREFIX": "<path-to-squish>",
                }
            }
        }
    }

    MCP configuration documentation for VS Code (GitHub Copilot)

While this project is primarily tested with GitHub Copilot in VS Code, MCP is an open standard and other agents should work as well. Some alternatives:

By default the server is started as a subprocess by the VS Code extension (stdio transport). Alternatively, you can start the server manually over HTTP for more control over the underlying process.

Start the server in a terminal:

SQUISH_PREFIX=<path-to-squish> uv run --directory <path-to-squish-mcp> squish_mcp --transport http --host localhost --port 8000

Then configure the MCP client to connect to it:

{
    "servers": {
        "Squish-MCP": {
            "url": "http://localhost:8000/mcp",
            "type": "http"
        }
    }
}

Usage

Once your MCP server is configured and recognized by your agent, you are ready to start prompting. Most of the MCP tools operate in the context of a single test suite and accept the test_suite_path argument (a concrete suite_* directory path).

Example Usage

The examples below use the addressbook application bundled with Squish, found at <path-to-squish>/examples/qt/addressbook/addressbook.

Execute a Test Case / Test Suite

You can ask the agent to run either a full test suite or an individual test case. The agent will automatically determine paths and execute the appropriate Squish commands.

[!tip] Example prompt Run all Squish tests for the addressbook

The agent will locate the relevant tests and execute all it can relate to the addressbook. It may ask for clarifications if needed.

Generate Test Cases

You can ask the agent to generate Squish test cases for your application under test (AUT).

[!tip] Example prompt Create a new test suite for the addressbook application <path-to-squish>/examples/qt/addressbook/addressbook. Add a test to the suite that adds a contact in the addressbook and verifies its presence in the table.

The agent will create a test case and determine how to address individual objects (e.g. the add-button) in the application. It may reach out to the user, run intermediate test cases to programmatically scan the application's objects, or use existing test cases as reference.

If you already have Squish test cases available, you can reference them in the prompt to match the style and improve the quality of the generated test case.

Generate a BDD Test Case

Squish has its own implementation and structure for running BDD tests. Squish MCP is aware of this structure and can produce both feature files and step function implementations.

[!tip] Example prompt (assumes a generated test case from the prior steps) Convert the test case tst_add_contact into a BDD test

Squish MCP will create both test.py and test.feature files with proper BDD structure, including step definitions and feature file format.

The MCP server offers a tool to generate an object map from a given object snapshot, which needs to be prepared beforehand. It serves as a helper for the agent when generating test cases to produce correct object references, but can also be used directly.

Configuration

SQUISH-RULES.yaml

For adding general "rules of thumb" for the LLM to follow, you can customize project-specific patterns and conventions:

memories:
  requested_patterns:
    - pattern: "user asks for a screenshot verification"
      context: "Use the verify_image() function from global scripts"
    - pattern: "BDD step function with variable input"
      context: "Use |any| notation for variable parameters"
    - pattern: "Any time you, the LLM/ the AI agent, aren't sure of what to do in a test script"
      context: "Add a 'TODO: *' comment where you explain what needs to be done in a missing area."

You can use SQUISH-RULES.yaml.example for reference and create your own SQUISH-RULES.yaml in the src/squish_mcp/server/tools/analysis directory.

Environment variables

  • SQUISH_GLOBAL_SCRIPTS: Directory storing scripts available for all Squish test cases, see Squish documentation for more information.

High-Level Architecture Overview

License

See the LICENSE file for details.

Available Tools

19 tools
analyze_bdd_contextB

Analyze how BDD is used in a specific test suite.

Args: test_suite_path: Path to a suite_* directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of BDD test structure found
bdd_suiteNoBDD suite metadata with step directories and files
step_filesYesFlattened step files from the suite
feature_filesYesFeature files with content and used steps
has_bdd_testsYesWhether any BDD tests were found in the suite
relationshipsYesMappings between feature steps and step definitions
step_definitionsYesFlattened step definitions from the suite

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must bear full burden. It does not disclose whether the tool modifies state, requires permissions, or has side effects. Output schema exists but return behavior is unspecified.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with front-loaded action. However, omission of behavioral details makes it less effective than it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Designed for a specialized analysis task, yet lacks description of output or return schema. An output schema exists but is not referenced, so the agent remains uninformed about what is returned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%. The description adds clarification that 'test_suite_path' should be a path to a 'suite_*' directory, improving over the plain string type. However, it lacks details on path format or validation rules.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Analyze' and resource 'how BDD is used in a specific test suite', specifying it applies to a suite directory. It distinguishes from siblings like 'analyze_bdd_documentation' which likely targets documentation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like 'analyze_bdd_documentation' or other analysis tools. Lacks context on prerequisites or best use scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_bdd_documentationA

Parse the local BDD-specific Squish documentation to find relevant information.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlYesSource documentation URL
messageYesSummary of BDD documentation loaded
hook_patternsYesAvailable BDD hook patterns
file_structureYesExpected BDD file structure and organization
content_previewYesPreview of the documentation content
placeholder_syntaxYesSupported BDD placeholder syntax patterns
context_object_featuresYesFeatures exposed on BDD context object
implementation_examplesYesImplementation examples from docs
step_definition_patternsYesStep patterns represented as [type, pattern] pairs

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose all behavioral traits. It only says 'parse...to find relevant information' without explaining the operation's nature (e.g., search vs. full dump), output format, or any side effects. This is insufficient for a tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short sentence (13 words) and is efficiently front-loaded with the verb 'Parse'. However, it could be slightly more structured or informative without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters and an output schema exists (signalled by context), the description adequately specifies the tool's domain (BDD documentation). It could mention that it returns parsed content, but the output schema likely covers that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters (100% description coverage by default). With 0 parameters, baseline is 4 as per guidelines. The description adds no param info, but none is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (Parse) and the specific resource (local BDD-specific Squish documentation). It distinguishes from sibling tools like analyze_squish_api_documentation by focusing on BDD-specific content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus sibling tools such as analyze_squish_api_documentation or analyze_squish_rules. No when-to-use or when-not-to-use information is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_existing_patternsC

Analyze test suite to find common patterns that should be used for generating new test cases.

Args: test_suite_path: Path to a suite_* directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
api_usageYesSquish API call frequencies, e.g. {'waitForObject': 12}
common_importsYesMost frequently used imports across test scripts
object_ref_filesYesFiles containing object reference definitions
object_referencesYesObject references found across test scripts
global_scripts_directoriesYesPaths to global script directories
object_ref_primary_locationYesPrimary storage location for object references (e.g. 'suite_names', 'global_pom')
global_scripts_available_functionsYesFunctions available from global script directories

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It only states the purpose and argument, omitting side effects, permissions, or output characteristics. The tool implies mutation? No, it's analysis, but nothing about idempotency or impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with two sentences, front-loading the purpose. Every sentence adds value, though it could briefly mention the output to improve completeness without harming conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema (unknown content), so return values may be covered there. The description is adequate for a single-parameter analysis tool but lacks hints about the output structure (e.g., a list of patterns) or behavioral details, leaving some gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'test_suite_path' has a schema description coverage of 0%, but the description adds 'Path to a `suite_*` directory.', which clarifies the expected format beyond a generic string. This provides useful semantic context, though no further constraints are given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Analyze' and the resource 'test suite' to find 'common patterns' for generating new test cases. It distinguishes from sibling tools like 'analyze_bdd_context' by focusing on pattern extraction, though the differentiation is implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs alternatives, such as other analyze tools or pattern-related functions. The description does not mention prerequisites or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_global_scriptsC

Analyze global scripts usage patterns that should be used for generating new test cases.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
filesYesScript files with path, functions, classes, imports, and content preview
messageYesSummary of global scripts found
directoriesYesPaths to global script directories

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries full burden. It only says 'analyze', implying a read-only operation, but does not disclose safety, side effects, or dependencies. The description is too brief to provide transparency beyond the basic action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, no unnecessary words. However, it is so concise that it sacrifices clarity, not earning a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no parameters and an output schema, so the output format is handled. But the description does not explain the input context (e.g., where global scripts come from) or how the analysis results should be used. It is adequate but leaves gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, and schema coverage is 100% (empty schema). The description does not add meaning beyond the schema, but for a parameterless tool, the baseline is 4. However, it could clarify what is being analyzed (e.g., currently loaded scripts), which is missing, so 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Analyze global scripts usage patterns' which specifies the action and resource, but it is vague about what 'global scripts' refers to and what the analysis entails. It distinguishes from sibling tools like 'analyze_bdd_context' but lacks specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like 'analyze_existing_patterns' or 'generate_test_template'. It does not mention prerequisites or context for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_object_map_structureB

Analyze the current object map structure to understand existing patterns.

Args: test_suite_path: Path to a suite_* directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryYesHigh-level summary with counts and strategy
object_filesYesObject definition files with their contents and patterns
existing_objectsYesAll existing object definitions keyed by object name
page_organizationYesPage organization strategy and associated directory or suite
reference_patternsYesCommon containers, naming, and property patterns

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must bear the full burden of behavioral disclosure. It neither states nor implies whether the tool is read-only, has side effects, or requires specific permissions. The name suggests a read operation, but this is not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, and contains no superfluous information. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of annotations and the presence of an output schema (not shown), the description is minimally adequate. It does not explain prerequisites, what 'current' means, or how the output is structured, leaving gaps for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'test_suite_path' is described as 'Path to a `suite_*` directory,' which adds meaningful context beyond the schema's type-only definition. This helps an agent understand what value to provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool analyzes the current object map structure to understand existing patterns, using a specific verb and resource. However, it does not differentiate from sibling tools like analyze_object_references or analyze_bdd_context, which may have overlapping purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like analyze_object_references or analyze_existing_patterns. There are no exclusions or contextual hints to guide decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_object_referencesB

Analyze object-reference patterns that should be used for generating new test cases. This includes where and how object maps are stored and how the objects are then accessed.

Args: test_suite_path: Path to a suite_* directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
filesYesObject file locations with type, path, object counts, and previews
messageYesSummary of object reference files found

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must convey behavioral traits. It states the analysis scope but omits side effects, resource usage, permissions, or output behavior (output schema exists but is not described). An agent cannot know if this is a safe read operation or if it modifies state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief, front-loading the purpose in the first sentence, followed by the parameter explanation. The 'Args:' section is arguably redundant with the schema but still clear and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists, return values need not be detailed. However, the description lacks prerequisites, error conditions, or expected directory structure beyond the naming convention. For a tool with one parameter and a clear purpose, it is minimally adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, yet the description adds meaning by describing 'test_suite_path' as a path to a `suite_*` directory. This clarifies the expected format beyond the bare schema, which only states type 'string'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it 'Analyze object-reference patterns' for generating new test cases, which is a specific verb and resource. It distinguishes from siblings like analyze_object_map_structure by focusing on patterns for test case generation, though it doesn't explicitly differentiate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for generating test cases but provides no explicit when-to-use or when-not-to-use guidance. Alternatives such as analyze_existing_patterns are not mentioned, leaving the agent to infer context from the sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_squish_api_documentationB

Parse the local Squish documentation to find relevant information like the available functions and code snippets.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of API documentation loaded
sectionsYesDocumentation section headings
functionsYesAvailable Squish API function names
local_pathYesPath to local API documentation file
code_examplesYesCode examples from the documentation
content_previewYesPreview of the documentation content

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility for behavioral disclosure. It indicates a read operation (parsing), but does not clarify if any external effects occur, or if there are access requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with action and resource. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no parameters and an output schema exists, so the description adequately conveys the main functionality. However, it lacks usage guidance and does not mention the output format or any limitations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, so schema coverage is effectively 100%. The description doesn't add parameter info because none exist. Baseline for 0 params is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool parses local Squish documentation to find functions and code snippets. It specifies a unique resource and action, though it doesn't fully distinguish from siblings like analyze_bdd_documentation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus other analysis tools. It does not mention alternative tools or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_squish_rulesB

Analyze project-specific Squish rules (explicitly defined patterns).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
contextYesProject context entries from rules file
messageYesSummary of project rules loaded
memoriesYesLearned patterns and remembered project context

TDQS

B3.3/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not mention any behavioral aspects such as side effects, return behavior, or requirements. For a tool that performs analysis, this is a critical gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no unnecessary words, perfectly concise for a zero-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While the tool has zero parameters and an output schema exists, the description lacks context about what Squish rules are and what the analysis entails, which may confuse an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has no parameters, and schema coverage is 100% by definition. The description adds no parameter information, but none is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool analyzes 'project-specific Squish rules' with the qualifier 'explicitly defined patterns', which clearly identifies the resource and distinguishes it from sibling tools like analyze_existing_patterns or analyze_bdd_context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus similar analysis tools. With 18 sibling tools, explicit usage context is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_caseA

Create a new Squish test case within an existing test suite.

Args: suite_path: Absolute path to the test suite directory (must start with 'suite_') test_case_name: Name of the test case (will be prefixed with 'tst_' if not already) test_content: Python code content for the test.py file (optional) is_bdd: Whether to create a BDD test with proper QtCare Squish BDD structure (default: False) test_description: Description for BDD feature file (optional, recommended for BDD tests)

ParametersJSON Schema
NameRequiredDescriptionDefault
is_bddNo
suite_pathYes
test_contentNo
test_case_nameYes
test_descriptionNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
is_bddYesWhether this is a BDD test case with .feature file
messageYesSummary of what was created
feature_pathNoPath to the .feature file (BDD tests only)
test_py_pathYesAbsolute path to the created test.py file
files_createdYesAll file paths that were created
test_case_pathYesAbsolute path to the created test case directory
suite_conf_pathNoPath to suite.conf that was updated

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It reveals naming conventions (suite_path must start with 'suite_', test_case_name prefixed with 'tst_'), optional behavior (test_content default empty, is_bdd flag toggles BDD structure), and specific QtCare Squish BDD structure. It does not mention side effects like file creation, permissions, or error handling, but the core behavior is well-covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and structured. The first sentence states the main purpose, followed by a clear Args block that is easy to scan. No unnecessary words; every sentence earns its place. It is front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 5 parameters (2 required) and no annotations, the description covers all parameters and their behaviors, including naming conventions and optionality. It does not describe the output (though output schema exists) or error conditions, but for a creation tool, it provides sufficient context to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does excellently. Each parameter is explained with constraints and defaults: suite_path absolute and must start with 'suite_', test_case_name prefixed, test_content optional Python code, is_bdd default false, test_description for BDD. This adds significant meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the action ('Create a new Squish test case') and the resource ('within an existing test suite'). It distinguishes from sibling tools like 'create_test_suite' (creates suite, not case) and 'generate_test_template' (generates template, not directly creates). The verb 'create' is specific and the target is well-defined.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context by requiring an existing test suite suite_path, but does not explicitly state when to use this tool versus alternatives like 'create_test_suite' or 'generate_bdd_template'. It provides clear prerequisites (suite_path must exist, naming conventions) but lacks explicit 'when to use' and 'when not to use' guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_suiteA

Create a new Squish test suite with the standard directory structure.

Creates suite.conf and shared/names.py inside the given directory. The suite directory must not already exist.

Args: suite_path: Absolute path to the test suite directory to create (must start with 'suite_') wrapper: Name of the wrapper to use (default 'Qt'). Written as WRAPPERS in suite.conf.

ParametersJSON Schema
NameRequiredDescriptionDefault
wrapperNo
suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of what was created
names_pathYesAbsolute path to the created names.py file
suite_pathYesAbsolute path to the created test suite directory
files_createdYesRelative paths of all created files
suite_conf_pathYesAbsolute path to the created suite.conf file

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses creation of specific files and path constraints, but does not discuss error handling, overwrite behavior, or whether additional files may be created.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is reasonably concise with front-loaded purpose, but uses bullet points unnecessarily; every sentence adds value, though could be tighter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having an output schema (stated in context but not provided), the description does not mention return value or success/failure indicators. For a creation tool, output description is important for agent understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description adds meaningful context: suite_path must be absolute and start with 'suite_', and wrapper has a default and is written as 'WRAPPERS' in suite.conf.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it creates a new Squish test suite with a standard directory structure, specifying the files created (suite.conf, shared/names.py). This distinguishes it from siblings like create_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context on prerequisites (directory must not exist) and default wrapper, but does not explicitly state when to use this tool versus alternatives like create_test_case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_object_referencesC

Extract object references from test content to understand what objects are being used.

Args: test_content: The test code to analyze

ParametersJSON Schema
NameRequiredDescriptionDefault
test_contentYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
names_objectsYesObjects referenced via names.py (e.g. names.someButton)
direct_objectsYesObjects referenced with inline property dicts
primary_patternYesDominant reference style: 'names', 'global_scripts', 'direct', or 'unknown'
unknown_objectsYesObject references that could not be classified
total_referencesYesTotal count of all object references found
global_script_objectsYesObjects referenced via global script modules

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must disclose behavioral traits. It only states the function without mentioning side effects, permissions, or return behavior. Read-only nature is implied but not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Fairly concise but includes an unnecessary 'Args:' section that adds little. The core purpose is stated in one sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having an output schema (not shown), the description does not explain what 'object references' are or what the output contains. For a one-parameter tool, more detail on expected content would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

One parameter with 0% schema coverage. The description adds 'The test code to analyze', providing some semantic meaning beyond the type string. However, it could be more specific about format requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts object references from test content, with a specific verb and resource. It differentiates from sibling 'analyze_object_references' by specifying 'test content' as input.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool over alternatives. The description lacks context about prerequisites or when this tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_bdd_templateB

Generate a BDD test template that can be used as a starting point for creating new BDD-style test cases.

Args: test_case_name: Name of the test case test_description: Description for the BDD feature (recommended)

ParametersJSON Schema
NameRequiredDescriptionDefault
test_case_nameYes
test_descriptionNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of what was generated
feature_templateYesContent for test.feature with Gherkin Background and Scenario sections
test_py_templateYesContent for test.py with BDD boilerplate (source, setupHooks, etc.)
step_definitions_templateYesContent for step definitions Python file

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears the full burden of behavioral disclosure. It states that a template is generated but does not specify whether this creates a file, returns the template, or modifies any state. There is no mention of side effects, authentication needs, or output format. The tool likely is read-only, but this is not confirmed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise—two lines plus a simple arg list. Every word serves a purpose, with no fluff or repetition. It is well-structured and front-loads the purpose statement, making it easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 2 parameters, no annotations, and an output schema (not visible), the description is adequate but not thorough. It covers purpose and basic param semantics but lacks usage context, behavioral details, and output explanation. For a simple template generator, this may be minimally viable but has clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It provides brief but clear meanings for both parameters: 'Name of the test case' and 'Description for the BDD feature (recommended).' The inclusion of '(recommended)' for test_description adds nuance. However, it lacks details like expected format, length limits, or relationships between parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool generates a BDD test template as a starting point for BDD-style test cases. It uses a specific verb ('Generate') and resource ('BDD test template'), distinguishing it from sibling tools like 'generate_test_template' (generic) and 'create_test_case' (actual creation). However, it could more explicitly differentiate usage scenarios.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions the tool 'can be used as a starting point for creating new BDD-style test cases,' but provides no guidance on when to use it vs. alternatives like 'generate_test_template' or 'create_test_case.' There are no when-not-to use instructions, explicit context, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_page_objects_from_snapshotA

Generate page object references from an XML object snapshot file.

This tool:

  1. Calls parse_object_snapshot.py to filter XML elements and generate basic definitions

  2. Analyzes object reference patterns to understand existing style and locations

  3. Determines output format (simple dicts vs classes/functions) based on existing patterns

  4. Writes the generated Python code to a local temporary file and returns its path

Args: xml_file_path: Absolute path to the XML object snapshot file page_name: Name of the page/component these objects belong to test_suite_path: Path to a suite_* directory. output_directory: Existing directory where the temporary output file should be created

Note: To generate the XML snapshot file, a testcase script has to call saveObjectSnapshot(some_object, "snapshot_name.xml"), where some_object is the root object of the component to capture. In case of an empty testcase, a variation of the following can be used: import object all_top_level_objects = object.topLevelObjects() top_level_object = all_top_level_objects[0] # change to a suitable index in case of multi-window apps. saveObjectSnapshot(top_level_object, "snapshot_name.xml") The snapshot will be placed next to the test script that is being executed, so the xml_file_path should be constructed accordingly.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_nameYes
xml_file_pathYes
test_suite_pathYes
output_directoryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of what was generated
summaryYesHuman-readable summary including format and temporary output path
xml_fileYesSource XML snapshot file path
page_nameYesName of the page/component these objects belong to
objects_foundYesNumber of object definitions generated
object_contextNoCurrent object reference context, if available
temp_file_pathYesAbsolute path to a local temporary file containing the generated object definitions
output_strategyYesStrategy used for output format and target location
pattern_analysisYesAnalysis of existing object reference patterns in the project

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries full burden. It discloses that the tool calls an external script, analyzes patterns, determines output format, writes to a temporary file, and returns its path. This is transparent for a code generation tool, though it does not mention potential side effects (e.g., file overwrite) beyond creating a temp file.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections (steps, args, note) but is somewhat lengthy due to a detailed note about generating the XML snapshot. The first sentence fronts the purpose, but the note could be condensed or referenced elsewhere.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 required params, output schema present), the description covers purpose, input, processing, and output. It explains a key prerequisite (generating the XML snapshot). It does not explain return values beyond the file path, but an output schema exists. Missing usage guidance slightly reduces completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description compensates by explaining all four parameters in a structured 'Args' block. Each parameter gets a purpose and expected type (e.g., 'Absolute path to the XML object snapshot file'). This adds meaning beyond the bare JSON schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool generates page object references from an XML snapshot file, with a specific verb ('Generate') and resource ('page object references'). The steps and argument list further clarify its function, distinguishing it from sibling tools like 'analyze_object_references' or 'extract_object_references' by focusing on code generation from a snapshot.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

While the description explains prerequisites (how to generate the XML snapshot), it provides no explicit guidance on when to use this tool versus alternatives. Sibling tools such as 'analyze_object_references' or 'extract_object_references' serve different purposes, but no comparison or selection criteria are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_test_templateA

Generate a test template based on existing patterns and project conventions.

Args: test_suite_path: Path to a suite_* directory. test_case_name: Name of the test case test_description: Optional description of what the test should do

ParametersJSON Schema
NameRequiredDescriptionDefault
test_case_nameYes
test_suite_pathYes
test_descriptionNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesSummary of what was generated
templateYesGenerated Python test script content, ready to write to test.py
object_patternYesObject reference pattern used in the template (e.g. 'suite_names', 'global_simple'), or null if unknown

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions 'based on existing patterns and project conventions' but does not disclose behavioral traits such as side effects, permissions, or what happens if patterns are missing. The output schema exists but is not described, leaving return behavior unclear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with a single sentence followed by an Args list. It is front-loaded with the purpose and contains no unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 3 parameters, no annotations, and an output schema present, the description covers purpose and param semantics moderately. However, it lacks behavioral transparency and usage guidelines, leaving some gaps in completeness for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description adds meaning for two of three parameters: test_suite_path includes a naming pattern 'suite_*', and test_description adds 'what the test should do'. test_case_name is merely restated. This compensates partially but not fully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'generate' and resource 'test template', adding context 'based on existing patterns and project conventions'. This distinguishes it from siblings like 'create_test_case' or 'generate_bdd_template'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives like 'generate_bdd_template' or 'create_test_case'. The usage is implied through the domain, but no exclusions or comparative guidance are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_global_script_dirsA

Get the global script directories configured in the active Squish test suite. This uses the squishrunner --config getGlobalScriptDirs command.

Returns: List of global script directory paths.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description provides transparency by mentioning the underlying squishrunner command and stating the return type (list of paths). It lacks explicit safety statements but is sufficient for a read-only getter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences plus a return line. All information is front-loaded and every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero parameters and an output schema exists, the description fully covers the tool's purpose, method, and return value. No gaps are evident.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so schema coverage is 100%. The description adds context by explaining what the tool returns and how it works, meeting the baseline for zero parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'global script directories' within the context of 'the active Squish test suite'. It is specific and distinguishes from the sibling 'set_global_script_dirs' tool by nature of being a getter.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus the sibling 'set_global_script_dirs' tool. The usage is implied but not stated, leaving the agent to infer context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_suite_configurationB

Read the suite configuration from a suite.conf file.

Args: suite_path: Absolute path to the test suite directory

ParametersJSON Schema
NameRequiredDescriptionDefault
suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
contentYesRaw content of the suite.conf file
messageYesSummary of the suite configuration

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden, but it only states 'Read' without detailing side effects, permissions, error handling, or output format. It does not disclose what happens if the file is missing or the path is invalid. For a read operation, basic transparency is achieved, but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences front-loaded with the core purpose and a single parameter description. Every sentence serves a purpose, and there is no extraneous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (not shown), the description does not need to detail return values, but it lacks information about error cases, file expectations, or permissions. For a tool with no annotations, more context would improve completeness, but it is acceptable for a simple read operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema coverage is 0%, so the description must compensate. It explains suite_path as 'Absolute path to the test suite directory', which adds meaning beyond the schema's raw 'string'. However, it is brief and could include format constraints or examples. It provides adequate but minimal additional context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool reads a suite configuration from a specific file (suite.conf). It specifies the resource and action. However, it could be more precise about what 'suite configuration' encompasses, but it effectively distinguishes from sibling tools that are about analysis or creation rather than reading configuration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidelines are provided about when to use this tool versus alternatives. There is no mention of prerequisites, when not to use, or how it relates to other tools like analyze_bdd_context or create_test_suite. The description is purely functional.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_testA

Run a Squish test suite or test case.

You need to figure out the test suite and test case absolute paths on your own. You may be asked to either run a full test suite or a single test case within a test suite.

This is a wrapper around the squishrunner command line tool.

IMPORTANT PATH HANDLING AND SQUISH TEST NAMING CONVENTIONS:

  • All paths are absolute paths, but you should infer missing ones from context.

  • Test suites are named like suite_<test_suite_name> - these are folders under the root path

  • Test cases are named like tst_<test_name> - these are folders under the test suite path

  • Test files are under the test case folders and are named like test.py

Args: test_suite_path: Absolute path to the test suite which you should figure out on your own Example: "/Users/yourname/projects/addressbook/suite_py" context: Dictionary variables of options to pass to Squish Format: {"VAR_NAME": "value"} test_case_name: Name of the test case to run. Example: "tst_general"

Returns: Test run results with squishrunner output.

ParametersJSON Schema
NameRequiredDescriptionDefault
contextYes
test_case_nameNo
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
stderrYesStandard error output from squishrunner
stdoutYesStandard output from squishrunner

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that it is a wrapper around squishrunner, requires absolute paths, and returns test results, but does not discuss side effects, error handling, or performance characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections for purpose, path handling, args, and returns. It is slightly verbose but every sentence adds necessary context for a complex tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and the presence of an output schema, the description covers the essential aspects: purpose, parameter usage, and return value. Minor gaps include missing details on prerequisites and error scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description fully compensates by explaining each parameter (test_suite_path, context, test_case_name) with examples and format details, adding significant value beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs a Squish test suite or test case, with a specific verb and resource. It distinguishes from sibling tools (analyze_, create_, etc.) by focusing on execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that paths must be inferred and provides naming conventions, implying when to use the tool. However, it does not explicitly exclude alternative tools or specify when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_global_script_dirsA

Set the global script directories for Squish test execution.

Args: directories: List of directory paths to set as global script directories

ParametersJSON Schema
NameRequiredDescriptionDefault
directoriesYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Indicates it sets (mutates) directories, which is clear, but does not specify whether it replaces or appends, persistence, or required permissions. With no annotations, more detail is needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Short and to the point, but the Args line is redundant with the schema. Could be more concise by omitting the Args section.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple mutation with one parameter and no output schema, but misses details on how directories are set (overwrite vs append) and impact on execution.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Repeats schema information ('List of directory paths') without adding semantic meaning like accepted formats or constraints beyond the type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it sets global script directories for Squish test execution, distinguishing from sibling 'get_global_script_dirs' which retrieves.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use versus alternatives. The name implies configuration setting, but lacks context like when to set vs get or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

suggest_code_improvementsB

Analyze test content and suggest improvements based on project patterns.

Args: test_content: The test code to analyze test_suite_path: Path to a suite_* directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_contentYes
test_suite_pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
totalYesTotal number of suggestions found
suggestionsYesOrdered list of improvement suggestions

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. It states the tool analyzes and suggests improvements but does not disclose side effects, required permissions, or whether the tool modifies state. This is insufficient for a tool that likely operates on code files.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences with an Args list. Every word serves a purpose, and the information is front-loaded. No fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists, the description does not need to explain return values. However, it lacks details on how 'project patterns' are used, the format of suggestions, and typical use cases. It provides the basics but is incomplete for a tool with many siblings and no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain parameters. It adds meaning by specifying 'test_suite_path' expects a path to a 'suite_*' directory. However, 'test_content' is merely restated as 'The test code to analyze', adding little beyond the parameter name. Partial compensation for missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool analyzes test content and suggests improvements based on project patterns. It uses a specific verb ('analyze' and 'suggest') and resource ('test content'). While it distinguishes from siblings like 'analyze_existing_patterns', the exact nature of improvements is not elaborated, leaving minor ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or typical scenarios. The sibling set includes many analysis tools, but no differentiation is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A3.5/5.0
Disambiguation4/5

Most tools have distinct purposes, but there is some overlap among analysis tools (e.g., analyze_bdd_context vs analyze_bdd_documentation, analyze_existing_patterns vs analyze_global_scripts). Descriptions are detailed enough to differentiate, but an agent might occasionally misinterpret which analysis tool to use.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case (e.g., analyze_bdd_context, create_test_case, generate_bdd_template). There are no deviations or mixed conventions, making the naming predictable.

Tool Count5/5

With 19 tools covering analysis, creation, running, and configuration, the set is well-scoped for a Squish testing server. Each tool serves a clear role without being excessive or insufficient.

Completeness4/5

The tool surface covers core workflows: creating suites/test cases, analyzing patterns, generating templates, running tests, and managing global scripts. Minor gaps exist, such as missing update/delete operations for test cases or suites, but agents can work around them.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Provides AI assistants with seamless access to SmartBear's suite of testing and monitoring tools, enabling querying testing data, analyzing performance metrics, and managing test automation through natural language.
    15,125
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/TheQtCompanyRnD/squish-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server