Skip to main content
Glama
Sign In
SmartBear MCP server

SmartBear MCP server

Official
by SmartBear
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Hybrid

Contract Testing: Review Pact Tests

contract-testing_review_pact_tests
Read-onlyIdempotent

Review and refine Pact contract tests with PactFlow AI using source code, OpenAPI specs, error messages, and optional instructions to improve test accuracy and resolve failures.

Instructions

Review Pact tests using PactFlow AI. You can provide the following inputs: (1) Pact tests to be reviewed along with metadata

Toolset: Review Pact Tests

Parameters:

  • pactTests (object) required: Primary pact tests that needs to be refined.

  • code (array): Collection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls

  • userInstructions (string): Optional free-form instructions that provide additional context or specify areas of focus during the refinement process of the Pact test.

  • errorMessages (array): Optional error output from failed contract test runs. These can be used to better understand the context or failures observed and guide the recommendations toward resolving specific issues.

  • openapi (any)

Input Schema

NameRequiredDescriptionDefault
codeNoCollection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls
openapiNoIf provided, the OpenAPI document which describes the API being tested and is accompanied by a matcher which will be used to identify the interactions in the OpenAPI document which are relevant to the Pact refinement process.
pactTestsYesPrimary pact tests that needs to be refined.
errorMessagesNoOptional error output from failed contract test runs. These can be used to better understand the context or failures observed and guide the recommendations toward resolving specific issues.
userInstructionsNoOptional free-form instructions that provide additional context or specify areas of focus during the refinement process of the Pact test.

Output Schema

NameRequiredDescriptionDefault
promptNo
instructionsNo
recommendationsNo
requiresPromptExecutionNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.43.0
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "additionalProperties": {},
+  "properties": {
+    "instructions": {
+      "type": "string"
+    },
+    "prompt": {
+      "type": "string"
+    },
+    "recommendations": {
+      "items": {
+        "additionalProperties": {},
+        "properties": {
+          "confidence": {
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "diff": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "recommendation": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "recommendation"
+        ],
+        "type": "object"
+      },
+      "type": "array"
+    },
+    "requiresPromptExecution": {
+      "const": true,
+      "type": "boolean"
+    }
+  },
+  "type": "object"
+}
  2. Addedv0.40.0
  3. Removedv0.34.0
  4. First observedv0.27.1

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description remains high-level, stating the inputs it accepts but not its runtime behavior (e.g., LLM latency, context window limits, specific dependency requirements). However, the strong annotations (readOnlyHint: true, destructiveHint: false) significantly reduce the need for behavioral disclosure in the description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and immediately front-loads the main purpose and the required inputs. The listing of inputs provides a clear structure, though the initial phrasing is slightly wordy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of analyzing tests alongside multiple code files and OpenAPI specs, the description provides adequate guidance on what data is expected for effective review. It covers the necessary scope without attempting to detail internal processing logic.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds valuable context beyond the input schema for all major parameters. For example, it clarifies that the 'code' array should include client code and data models, and that 'errorMessages' should be used to guide recommendations toward resolving specific issues.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The title and description clearly state the tool's function: reviewing Pact tests using PactFlow AI. The verb 'Review' is specific, and it clearly defines the resource being acted upon, distinguishing it from management or generation tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

While the description clearly states what the tool does, it offers minimal guidance on when to use it compared to alternatives like 'contract-testing_generate_pact_tests' or simply running existing tests. The implied usage is clear but not explicitly articulated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools