Skip to main content
Glama

Generate Newman report

generate_newman_report

Runs a Postman collection and writes JSON, JUnit, or HTML Extra reports to an output directory for auditing, test results, and CI/CD pipelines.

Instructions

Runs a collection and generates a report for auditing or CI/CD (JSON, JUnit, HTML Extra).

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "collectionSource": "<collectionSource>",
      "reportFormats": "<reportFormats>",
      "outputDir": "<outputDir>"
    },
    "name": "generate_newman_report"
  }
}

Input Schema

NameRequiredDescriptionDefault
insecureNoIgnore certificate errors
outputDirYesOutput directory
reportFormatsYesReport formats
environmentPathNoEnvironment path
collectionSourceYesCollection (path, https URL, or Postman UID)
iterationDataPathNoIteration data path

Output Schema

NameRequiredDescriptionDefault
statsYes
abortedYes
reportsYes
successYes
timingsYes
failuresYes
truncatedYes
consoleLogsYes
exportedFilesYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, openWorldHint=true and destructiveHint=false, so the safety profile is covered. The description adds that requests are actually executed against a collection source and that artifacts are produced in three formats, but says nothing about files written to outputDir, overwrite behavior, or whether network calls hit live endpoints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the action front-loaded and the format list trailing as parenthetical detail. Nothing is wasted, though there is no structural separation of purpose versus usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and 100% schema coverage documents every parameter, so the description needn't carry return-value or argument detail. The only real gap is the missing disambiguation from sibling execution tools, which is minor given how complete the structured data is.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all six parameters including required collectionSource, reportFormats and outputDir are already documented in the schema. The description's format list (JSON, JUnit, HTML Extra) merely restates the reportFormats enum values, adding no syntax or constraints beyond the structured data.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb+resource pair ('runs a collection and generates a report') and enumerates the report formats produced. It is clear what the tool does, but it never explicitly distinguishes itself from the sibling run_newman_collection, leaving the agent to infer that reporting is the differentiator.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a use context ('for auditing or CI/CD'), which is more than nothing, but offers no when-not guidance and does not name the closest alternative (run_newman_collection or analyze_newman_failure). The agent must infer selection from context alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.