DevPilot MCP

Summarize test failures

summarize_test_failures

Read-onlyIdempotent

Summarize a previous test run's failures: failing tests, error messages, top project stack frames, likely fix source files, and failure groups by shared error, largest first.

Instructions

Summarize the failures of a previous run_tests call: each failing test with its message, top project stack frames, and likely_source_files (non-test files from the stack — where the fix probably goes), plus groups of failures sharing the same normalized error, largest first.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "run_id": "<run_id>"
    },
    "name": "summarize_test_failures"
  }
}

Input Schema

Name	Required	Description	Default
`run_id`	Yes	The run_id returned by run_tests.

Output Schema

Name	Required	Description	Default
`groups`	Yes
`run_id`	Yes
`failures`	Yes
`truncated`	Yes
`workspace`	Yes
`total_failures`	Yes

Schema Changelog

Changes observed during successful MCP inspections.

First observedv1.0.0Sep 29, 2026

TDQS

A4.2/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true and destructiveHint=false (and closed-world), so safety is covered. The description adds interpretive context absent from the schema, notably that likely_source_files are non-test files from the stack 'where the fix probably goes' and that failures are grouped by normalized error, largest first — ordering behavior the structured fields do not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence, front-loaded with the verb and the source of the input, followed by the contents. It is long but every clause carries new information; the parenthetical is slightly heavy but earns its place by explaining field meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema present, the description need not enumerate return values in detail, and annotations cover the safety profile. Combined, an agent has everything needed to select and call this tool on a valid run_id.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter at 100% schema description coverage, with the pattern and origin (run_id returned by run_tests) documented in the schema itself. The description adds only the implicit linkage that the run_id comes from a prior run_tests call, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (summarize) and resource (failures of a previous run_tests call), and it enumerates the returned artifacts (failing test, message, top project stack frames, likely_source_files, error groups). It is clearly distinguishable from get_issue, analyze_issue, search_codebase and run_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It anchors usage to a prerequisite — this operates on the output of a previous run_tests call — so an agent knows it must run tests first. There is no explicit when-not-use or named alternative, which keeps it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server