Skip to main content
Glama

Test summary

test_summary
Read-onlyIdempotent

Counts a project's tests by state, tag, or annotation from test files without running them, and groups counts by file, tag, or project to show where skipped tests concentrate.

Instructions

Counts a project's tests by state (active, skip, fixme, todo, expectedToFail, notLoaded) and by tag and annotation, from the test files, without running them. Use it first, for questions such as how many tests are skipped, or which files or tags have the most skipped tests; use find_tests to list the tests themselves. states, tags and grep count only the tests they match. groupBy "file", "tag" or "project" adds the counts of each, sorted by name and paged with limit and cursor; a test with no tag is in no tag group, and one whose projects the scan cannot tell in no project group. configs lists the config files read and their projects. Example: {"states": ["skip", "fixme"], "groupBy": "file"}.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "test_summary"
  }
}

Input Schema

NameRequiredDescriptionDefault
grepNoKeeps tests whose fullName (describe titles and title joined with " > ") matches this regular expression, ignoring case.
tagsNoKeeps tests with one of these tags, written as in the titles, such as "@smoke".
limitNoThe most items to return.
cursorNonextCursor from the previous answer, for the next page. Keep the other arguments the same.
statesNoKeeps tests in one of these states.
groupByNoAdds the counts of each file, tag or project.
relatedNoChanged files, relative to the project root as `git diff --name-only` writes them, such as ["pages/order-ticket.ts"]. Keeps the tests a change to one of them can affect: those whose dependencies include it, and those that reach code the scan can't follow (hasUnfollowedDependencies).
patternsNoGlobs of the test files, relative to `directory`, such as ["tests/**/*.spec.ts"]. Default: the ones the server was started with, or without them the files the configs in `directory` and below it run.
directoryNoA folder inside the project to scan, relative to its root, such as "packages/web". Default: the root.
frameworkNoThe test runner. Default: the one the server was started with, otherwise the one the project in `directory` uses.
expandCasesNoLists each case of a .each or .for table and each pass of a loop as its own test, with the name the runner gives it and caseIndex, as the runner registers them. Ids differ from those without it, so pass the same value to get_tests. Default: false, one test per declaration with caseCount.

Output Schema

NameRequiredDescriptionDefault
hintYes
totalYes
groupsYes
configsYesThe config files the scanned test files use, with their projects; [] when none was found.
summaryYesCounts over the matching tests; fileCount is every file scanned.
frameworkYes
nextCursorYes
scanErrorsYesProblems of level error in the scan, which may leave tests out of the answer.
frameworkVersionYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.9.0
    • addedInput schema / additionalProperties
      Added value: +false
  2. Changed7 schema fields changedv0.7.0
    • addedInput schema / properties / expandCases
      Added value: +{
      +  "description": "Lists each case of a .each or .for table and each pass of a loop as its own test, with the name the runner gives it and caseIndex, as the runner registers them. Ids differ from those without it, so pass the same value to get_tests. Default: false, one test per declaration with caseCount.",
      +  "type": "boolean"
      +}
    • changedInput schema / properties / framework / description
      Previous value: -"The test runner. Default: the one the server was started with."New value: +"The test runner. Default: the one the server was started with, otherwise the one the project in `directory` uses."
    • changedInput schema / properties / framework / enum
      Previous value: -[
      -  "playwright",
      -  "vitest"
      -]New value: +[
      +  "playwright",
      +  "vitest",
      +  "jest",
      +  "mocha",
      +  "jasmine",
      +  "cypress",
      +  "webdriverio",
      +  "node",
      +  "bun"
      +]
    • changedOutput schema / definitions / ConfigRecord / description
      Previous value: -"A Playwright or Vitest config that scanned files use: the nearest one in a file's folder or a folder above it, up to the root, as the runner started in that folder would. Read as code, never run."New value: +"A runner's config that scanned files use. For Playwright and Vitest, the nearest one in a file's folder or a folder above it, up to the root, as the runner started in that folder would; for Jest, the one at the root and the projects it names; for the other runners, the one config they read for the whole run. Read as code, never run."
    • changedOutput schema / definitions / ConfigRecord / properties / projects / description
      Previous value: -"The projects the config runs tests in: Playwright's `projects`, or one for a config without them; Vitest's, with each browser instance as one. Null when the scan can't work them out."New value: +"The projects the config runs tests in: Playwright's `projects`, or one for a config without them; Vitest's, with each browser instance as one; Cypress's testing types, `e2e` and `component`; one for the other runners. Null when the scan can't work them out."
    • changedOutput schema / definitions / Framework / anyOf
      Previous value: -[
      -  {
      -    "enum": [
      -      "playwright",
      -      "vitest"
      -    ],
      -    "type": "string"
      -  },
      -  {
      -    "type": "string"
      -  }
      -]New value: +[
      +  {
      +    "enum": [
      +      "playwright",
      +      "vitest",
      +      "jest",
      +      "mocha",
      +      "jasmine",
      +      "cypress",
      +      "webdriverio",
      +      "node",
      +      "bun"
      +    ],
      +    "type": "string"
      +  },
      +  {
      +    "type": "string"
      +  }
      +]
    • changedOutput schema / definitions / ProjectRecord / properties / testDir / description
      Previous value: -"The folder the project's test files are found in, from the project root: Playwright's `testDir`, Vitest's `dir`."New value: +"The folder the project's test files are found in, from the project root: Playwright's `testDir`, Vitest's `dir`, Jest's `rootDir`, Jasmine's `spec_dir`, Bun's `[test] root`; for the other runners, the config's folder."
  3. Changed8 schema fields changedv0.4.1
    • changedInput schema / properties / groupBy / description
      Previous value: -"Adds the counts of each file or each tag."New value: +"Adds the counts of each file, tag or project."
    • changedInput schema / properties / groupBy / enum
      Previous value: -[
      -  "file",
      -  "tag"
      -]New value: +[
      +  "file",
      +  "tag",
      +  "project"
      +]
    • changedInput schema / properties / patterns / description
      Previous value: -"Globs of the test files, relative to `directory`, such as [\"tests/**/*.spec.ts\"]. Default: the ones the server was started with."New value: +"Globs of the test files, relative to `directory`, such as [\"tests/**/*.spec.ts\"]. Default: the ones the server was started with, or without them the files the configs in `directory` and below it run."
    • addedInput schema / properties / related
      Added value: +{
      +  "description": "Changed files, relative to the project root as `git diff --name-only` writes them, such as [\"pages/order-ticket.ts\"]. Keeps the tests a change to one of them can affect: those whose dependencies include it, and those that reach code the scan can't follow (hasUnfollowedDependencies).",
      +  "items": {
      +    "minLength": 1,
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
    • addedOutput schema / definitions / ConfigRecord
      Added value: +{
      +  "description": "A Playwright or Vitest config that scanned files use: the nearest one in a file's folder or a folder above it, up to the root, as the runner started in that folder would. Read as code, never run.",
      +  "properties": {
      +    "extends": {
      +      "description": "Files the config takes settings from through imports, such as a shared base config, in the order read.",
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "projects": {
      +      "anyOf": [
      +        {
      +          "items": {
      +            "$ref": "#/definitions/ProjectRecord"
      +          },
      +          "type": "array"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "The projects the config runs tests in: Playwright's `projects`, or one for a config without them; Vitest's, with each browser instance as one. Null when the scan can't work them out."
      +    },
      +    "relativeFilePath": {
      +      "description": "Path from the project root, with `/` separators.",
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "relativeFilePath",
      +    "extends",
      +    "projects"
      +  ],
      +  "type": "object"
      +}
    • addedOutput schema / definitions / ProjectRecord
      Added value: +{
      +  "properties": {
      +    "dependencies": {
      +      "description": "Playwright's projects that run before this one; `[]` for Vitest.",
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "name": {
      +      "description": "As the runner names it: `''` for a project without a name.",
      +      "type": "string"
      +    },
      +    "testDir": {
      +      "description": "The folder the project's test files are found in, from the project root: Playwright's `testDir`, Vitest's `dir`.",
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "name",
      +    "testDir",
      +    "dependencies"
      +  ],
      +  "type": "object"
      +}
    • addedOutput schema / properties / configs
      Added value: +{
      +  "description": "The config files the scanned test files use, with their projects; [] when none was found.",
      +  "items": {
      +    "$ref": "#/definitions/ConfigRecord"
      +  },
      +  "type": "array"
      +}
    • changedOutput schema / required
      Previous value: -[
      -  "framework",
      -  "frameworkVersion",
      -  "summary",
      -  "groups",
      -  "total",
      -  "nextCursor",
      -  "scanErrors",
      -  "hint"
      -]New value: +[
      +  "framework",
      +  "frameworkVersion",
      +  "summary",
      +  "configs",
      +  "groups",
      +  "total",
      +  "nextCursor",
      +  "scanErrors",
      +  "hint"
      +]
  4. Changed2 schema fields changedv0.2.0
    • addedOutput schema / definitions / Summary / properties / benchmarkCount
      Added value: +{
      +  "minimum": 0,
      +  "type": "integer"
      +}
    • changedOutput schema / definitions / Summary / required
      Previous value: -[
      -  "fileCount",
      -  "suiteCount",
      -  "testCount",
      -  "activeCount",
      -  "skipCount",
      -  "fixmeCount",
      -  "todoCount",
      -  "expectedToFailCount",
      -  "notLoadedCount",
      -  "onlyCount",
      -  "conditionalCount",
      -  "parallelCount",
      -  "serialCount",
      -  "parameterizedCount",
      -  "inLoopCount",
      -  "inConditionCount",
      -  "inFunctionCount",
      -  "typeTestCount",
      -  "diagnosticCount",
      -  "tagCounts",
      -  "annotationCounts"
      -]New value: +[
      +  "fileCount",
      +  "suiteCount",
      +  "testCount",
      +  "activeCount",
      +  "skipCount",
      +  "fixmeCount",
      +  "todoCount",
      +  "expectedToFailCount",
      +  "notLoadedCount",
      +  "onlyCount",
      +  "conditionalCount",
      +  "parallelCount",
      +  "serialCount",
      +  "parameterizedCount",
      +  "inLoopCount",
      +  "inConditionCount",
      +  "inFunctionCount",
      +  "typeTestCount",
      +  "benchmarkCount",
      +  "diagnosticCount",
      +  "tagCounts",
      +  "annotationCounts"
      +]
  5. First observedv0.1.0

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so safety is covered. The description adds genuinely new behavior: the scan never runs the tests, filters only count the tests they match, groupBy adds sorted per-group counts paged by limit/cursor, and tests without a resolvable tag or project fall into no group. It does not cover whether the scan can be slow or cached, which would have earned a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the find_tests alternative are front-loaded, and the trailing example clarifies filter+groupBy composition. The middle sentence about groupBy and null groups is dense but each clause carries semantics; a small amount of tightening (the 'configs' aside) would make it a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter read-only counting tool with an output schema, the description covers the behavioral essentials — no execution, filter scoping, grouping and paging semantics, fallback defaults deferred to schema — so an agent has everything needed to call it correctly. Return-value shape is rightly left to the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description goes beyond it by explaining what groupBy produces (per-file/tag/project counts sorted by name, paged via limit and cursor) and the edge case for ungroupable tests, plus a concrete invocation example. It leaves the remaining params (related, patterns, directory, framework, expandCases) entirely to the schema, which is acceptable given full coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — counts a project's tests by state, tag and annotation, from test files without running them — and explicitly differentiates from find_tests, which lists the tests themselves. An agent can distinguish it from get_tests, find_tests and scan_diagnostics on purpose alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit ordering rule ('Use it first'), concrete example question types ('how many tests are skipped', 'which files or tags have the most skipped tests'), and a named alternative with its selecting condition ('use find_tests to list the tests themselves'). This is exactly the when/when-not routing an agent needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.