Contract Testing: Matrix

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that the tool is read-only, idempotent, and non-destructive. The description adds useful behavioral detail beyond that: it describes exactly what the returned matrix exposes, the relationship between versions, and that failures come with detailed cause information. No contradiction with annotations exists. It omits exact response formatting, but the annotations and descriptive return semantics reduce the impact of that gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening paragraph is strong and front-loaded, and the bulleted use cases make the intended scenarios easy to scan. However, the Parameters section duplicates what the input schema already documents, and the use-case list is somewhat verbose and overlapping. The structure is clear but not every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with strong annotations and no output schema, the description provides enough context to understand what will be returned and why an agent would call it: version relationships, pact files, verification outcomes, and failure details. It covers parameter selection, use cases, and system context (Pact Broker or Pactflow). It is not exhaustive about the exact response shape, but it is complete enough for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, so the description should add value beyond the schema. It does add a useful rule of thumb for latestby (cvp for one-selector can-i-deploy queries, cvpv for two-selector queries), but it largely repeats the limit and latestby descriptions already present in the schema. The required q parameter is only marked as required, leaving its selector semantics to the schema entries; this is a meaningful gap because q is the core query parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the operation ('Retrieve the comprehensive contract verification matrix') and the resource domain: consumer/provider versions, pact files, and verification results. It is specific about what the matrix contains and that it highlights failures. It does not explicitly distinguish itself from the sibling contract-testing_can_i_deploy, even though it references can-i-deploy in its use cases, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section provides explicit contexts for using the tool, such as identifying passed/failed combinations, diagnosing failures, visualizing compatibility, and advanced selector queries. This gives an agent clear situational guidance. However, there is no explicit 'when not to use' or comparison to the closely related can_i_deploy sibling, so the guidance is not complete enough for a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.