Server Quality Checklist

Tool Scores

• execute_operationA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description bears the full burden of behavioral disclosure. It only states that the tool returns the API response body or an error string, but it does not disclose potential side effects, required permissions, idempotency, or whether the operation could be destructive. As a generic operation executor, this ambiguity is significant. The return type is mentioned, but broader behavioral traits are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the main action ('Execute one API operation by name'). It consists of two sentences and every clause contributes necessary guidance—how to get the operation name, what parameters to provide, and what the tool returns. No superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a generic executor with an output schema (though not detailed in the description), the description covers the essential workflow: source the operation name, supply parameters, and receive a response or error. It references discover_operations and the operation schema, which provides context for dynamically resolving details. It does not explain error handling formats or authentication, but for a wrapper tool this is a reasonable level of completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions are completely absent (0% coverage), so the description must compensate. It mentions path_params, query_params, and body_data by name and says they are provided 'as required by the schema,' but it does not explain their formats or how they map to the operation's requirements. The term 'optionally' for body_data is useful, but overall the description adds minimal semantic value beyond what the parameter names already imply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool executes a single API operation by name, which is a specific verb+resource combination. It distinguishes itself from the sibling tool discover_operations by explicitly referencing operation names obtained from that tool, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by telling the agent to use operation_name from discover_operations and to supply path_params, query_params, and optionally body_data as required by the schema. It does not explicitly mention when not to use this tool, but the reference to discover_operations implies a prerequisite and differentiates the two tools. No explicit alternatives or exclusions are given, so this is not a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_operationsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden. It discloses the internal mechanism (RAG over the OpenAPI schema), states what is returned (operation entries with specific fields), and explicitly notes that no execution happens. While it doesn't mention potential permissions or errors, it gives sufficient transparency for a read-only discovery operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: purpose, output details, and usage guidance. It is front-loaded with the key action and avoids unnecessary fluff, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 params) and has an output schema, so the description doesn't need to detail return values. It explains the tool's role in the overall workflow, what it returns, and the next step. The only gap is the undocumented 'k' parameter, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The description clarifies 'query' as a natural-language query, but makes no mention of the 'k' parameter. Given 'k' has a default of 5 and is likely the number of results, the description should explain this but does not, leaving a significant semantic gap for the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific action—'Discover which API operations match a natural-language query'—and clearly identifies the resource (API operations). It distinguishes itself from the sibling tool 'execute_operation' by explicitly stating 'No execution happens here' and framing this as a discovery step for finding tool IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent exactly when to use this tool: 'Use this to find tool IDs, then call execute_operation with the chosen operation_name and params.' It also clarifies the boundary by stating no execution occurs, which helps the agent avoid using it for execution purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.