Server Quality Checklist

Tool Scores

• run_requestA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint false and destructiveHint false, so the agent knows this can perform writes and is not destructive. The description adds little beyond 'execute' and 'structured result,' and does not mention safety gates like allowWrites or approval flows. It does not contradict annotations, but it also does not provide additional behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no redundancy. The first sentence front-loads the core action and scope, and the second sentence adds a practical use case. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 9-parameter tool with nested objects and no output schema, the description is minimal. It does not explain the approval workflow (allowWrites, allowProduction, approvalNote) or describe the structure of the 'structured result.' However, the schema provides detailed descriptions for many parameters, partially compensating for the description's brevity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%, providing a solid baseline. The description adds some meaning by clarifying that the request can be 'optionally scoped to a folder,' which helps interpret folderId and folderName. However, it does not elaborate on other parameters such as environment, allowProduction, or timeout, which are already documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Execute a single named request') with an optional scope ('optionally scoped to a folder') and a distinct use case ('re-running one failing test case'). This distinguishes it from sibling tools like run_folder, which would run multiple requests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for use ('Useful for re-running one failing test case') but does not explicitly mention when to use run_folder instead. The 'single named request' wording implies the contrast with run_folder, but no alternative tool is named or exclusion is stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_foldersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds value by noting the output fields (name, id, path, request count) but does not disclose additional behavioral traits such as pagination or error behavior. Given the strong annotation coverage, a score of 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core function and output fields, followed by a single usage context. No unnecessary words or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one well-schema'd parameter and no output schema, the description covers the essential context: what it lists, what fields are returned, and when to use it. It could mention potential limitations (e.g., no filtering) but is complete for the agent's selection and invocation needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because the single 'collection' parameter is thoroughly described in the schema as a Postman collection v2.1 JSON object. The description adds no additional parameter-level meaning, aligning with the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists folders in a Postman collection and specifies the returned fields (name, id, path, request count). This distinct verb+resource combination differentiates it from sibling tools like run_folder and preview_requests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use to confirm the folder created for a Jira ticket before running it.' This tells the agent when to invoke the tool relative to running a folder. It does not explicitly mention when not to use it or list alternatives, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_requestsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, idempotent, and non-destructive. The description adds value by disclosing the return content (redacted URLs, headers, body mode/preview, method counts, write-request count, warnings) and the fact that it does not execute HTTP calls, which reinforces the safety profile without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first defines function and outputs, the second gives usage guidance. Every sentence earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 params, nested objects, no output schema), the description provides a solid summary of return values and the key use case. It doesn't detail all return fields but enough for an agent to form a mental model. The absence of an output schema is partially compensated by listing the return categories.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter description coverage, so the baseline is 3. The description adds minimal extra meaning beyond the schema, mainly reinforcing 'folder or single request' and mentioning environment for auth, but the schema already provides thorough per-parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resolves variables for a folder or single request without executing HTTP calls, using a specific verb and resource. It also distinguishes itself from sibling tools like run_folder/run_request by explicitly positioning itself as a pre-execution preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this before run_folder/run_request to confirm targets, auth scope, and write-method safety.' This directly provides usage context relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_folderA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavior beyond annotations: it resoles variables, runs pre-request scripts for token auth, fires requests, and evaluates pm.test scripts. Annotations are sparse (readOnlyHint false, destructiveHint false), and the description enriches the safety profile without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence front-loads the purpose and behavior, the second explains targeting. No fluff or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core workflow (execution, scripts, return of structured results) and leverages rich schema annotations. It does not detail return structure (no output schema) or safety workflow, but the parameter schema covers those gaps adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds useful semantic context for folderName (typically a Jira ticket key) and folderId (for ambiguous names), going beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Execute') and resource ('collection folder'), and clearly distinguishes from siblings by noting it runs every request and evaluates pm.test scripts. This is a clear, non-tautological purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for running a full folder and provides targeting guidance (folderName typically a Jira key, or folderId). It does not explicitly state exclusions or alternatives, but the context is unambiguous for choosing between this and run_request.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.