Server Quality Checklist

Tool Scores

• get_operationB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for disclosing behavior. It indicates a read-only operation ("Get") and public access ("public explorer operation"), but does not state potential error behaviors, whether the operation is actually invoked, or any side effects. This is minimal disclosure beyond the tool's name and purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the verb and resource. Every word adds value, and there is no redundancy or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one parameter, output schema present), the description covers the basic purpose but lacks contextual guidance such as how to obtain operation_id or when to prefer this over siblings. The output schema covers return values, so that is not a gap, but the overall context is only moderately complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides zero description for the operation_id parameter (0% coverage), and the description does not elaborate on what operation_id should contain or how to obtain it. It only indirectly suggests it identifies an operation, leaving the agent without enough semantic detail to correctly supply the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving parameters and response schemas for a specific operation. This distinguishes it from sibling tools like list_operations (listing all) and invoke_operation (executing), as it focuses on metadata retrieval for a single operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when you need schema details for a single operation) but does not explicitly mention alternatives or exclusions. The sibling tool names provide context, but the description itself lacks direct guidance on choosing this tool over list_operations or invoke_operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoke_operationA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a key behavioral requirement: writes require explicit confirm=true. However, with no annotations, it does not explain other behavioral aspects like idempotency, side effects, or error responses, leaving the agent partially informed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with essential information front-loaded. Every word earns its place, and the critical safety note about writes requiring confirmation is clearly stated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a minimal viable explanation for a generic invoke tool, especially since an output schema exists. However, it lacks guidance on how parameters like body, query, and path_params relate to the operation_id, and does not reference how to discover operation details via sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for all 5 parameters, but it only adds meaning to confirm by clarifying it is required for writes. Other parameters (operation_id, body, query, path_params) receive no explanatory context beyond their raw schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Call' with a resource 'documented explorer operation', clearly indicating this tool executes an operation. This distinguishes it from sibling tools list_operations and get_operation, which are about discovery and retrieval, not execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for executing operations, contrasting with list/get siblings, but does not explicitly state when to use this tool versus alternatives. The confirm=true instruction gives guidance for write operations but no exclusions or alternative scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_operationsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It notes the spec is 'public' and 'checked-in,' which suggests the operation is read-only and requires no special access. However, it does not explicitly state that it makes no modifications or describe the output format, though the output schema covers the latter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the verb and object immediately. There is no redundant language or unnecessary detail, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, an output schema exists (covering return values), and the purpose is straightforward, the description is sufficient for an agent to understand what the tool does. The 'public' and 'checked-in' qualifiers add useful context without overexplaining.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters, so there is nothing for the description to explain beyond what the empty input schema already shows. The description accurately implies a parameterless invocation, and the baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all operations in the checked-in public BlazeMeter v4 explorer spec' uses the specific verb 'List' with a clear resource ('operations') and scope ('checked-in public BlazeMeter v4 explorer spec'). This distinguishes it from sibling tools get_operation and invoke_operation, which imply single-item access and execution respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool is for enumerating all operations, implying it should be used to discover available operation names before calling get_operation or invoke_operation. It does not explicitly name alternatives or state when not to use it, so it falls short of the highest score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.