Server Quality Checklist

Tool Scores

• get_toolsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that the tool returns 'dynamic tools that user has access to' and 'all flows with name and id that user has access to (if any),' which adds some behavioral context about access control and conditional returns. However, it lacks details on rate limits, error handling, or what 'dynamic tools' entails, leaving gaps for a tool that retrieves system metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with four sentences that are front-loaded with the core purpose. Each sentence adds value: the first states the purpose, the second clarifies access and dynamics, the third explains usage in a workflow, and the fourth notes the inclusion of flows. There's minimal redundancy, though it could be slightly more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of retrieving system tools and flows, no annotations, and no output schema, the description provides basic purpose and usage but lacks details on return format, pagination, or error cases. It's adequate for a read operation but incomplete for guiding an agent on handling the response or understanding limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('question' and 'tool') with descriptions. The description doesn't add any meaning beyond what the schema provides regarding parameters; it focuses on the tool's purpose and usage instead. This meets the baseline of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'retrieve a list of available tools from the Veyrax API' and mentions it also returns flows. It specifies the verb ('retrieve') and resource ('list of available tools'), but doesn't explicitly differentiate from sibling tools like 'get_flow' which presumably retrieves a specific flow rather than a list of tools and flows.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating 'You can use this tool to get the list of tools... and then use tool_call tool to call the tool with the provided parameters,' suggesting a workflow. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_flow' or provide clear exclusions. The guidance is present but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tool_callB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool executes methods with parameters but fails to describe critical traits: error handling, authentication requirements, rate limits, or what happens if parameters are invalid. For a meta-tool that invokes other tools, this lack of behavioral context is a significant gap, though it doesn't contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that front-load the core purpose. Each sentence adds value: the first defines the tool's function, and the second specifies required inputs. There's no wasted text, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (meta-invocation with 4 parameters, no output schema, and no annotations), the description is minimally adequate. It covers the basic purpose and usage context but lacks details on behavior, error cases, or return values. Without annotations or output schema, more completeness is needed for safe operation, but it's not entirely inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing clear documentation for all 4 parameters. The description adds minimal value beyond the schema, only reiterating that parameters are 'required' and referencing 'get-tools tool response.' It doesn't explain parameter interactions or provide examples, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'execute a specific method of another tool with the provided parameters based on get-tools tool response.' It specifies the verb ('execute') and resource ('method of another tool'), and distinguishes it from siblings by referencing 'get-tools tool response.' However, it doesn't explicitly name the sibling tools or contrast their purposes directly, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: 'based on get-tools tool response' implies this tool should be used after get_tools to invoke methods discovered there. It distinguishes from get_tools (which lists tools) and get_flow (unclear but likely different). However, it lacks explicit when-not-to-use guidance or alternative scenarios, such as direct tool calls without get_tools, keeping it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flowA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool as a retrieval operation ('retrieve'), which implies it's likely read-only, and mentions what information is returned ('description, steps and input schema'). However, it doesn't address important behavioral aspects like error conditions (what happens with invalid flowId), authentication requirements, rate limits, or whether the operation is idempotent. The description adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with three sentences that each serve a purpose: stating the tool's function, explaining what a workflow is, and providing usage prerequisites. It's front-loaded with the core purpose. However, the second sentence explaining workflows could be considered slightly verbose for a tool description, and there's some redundancy in mentioning 'get-tools method' when 'get_tools' is a sibling tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 1 parameter with 100% schema coverage but no annotations and no output schema, the description is moderately complete. It explains the purpose and provides usage context, but lacks important behavioral details about error handling, authentication, and what the return structure looks like. For a retrieval tool with no output schema, the description should ideally provide more information about the return format beyond 'description, steps and input schema'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'flowId' well-documented in the schema. The description doesn't add any meaningful parameter information beyond what's already in the schema. It mentions 'flowId' in the context of where to obtain it, but this is usage guidance rather than parameter semantics. The baseline of 3 is appropriate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'retrieve a specific workflow by its ID' with the resource being 'workflow' and the verb being 'retrieve'. It provides additional context about what a workflow is ('sequence of steps...'), which helps understanding. However, it doesn't explicitly differentiate from sibling tools like 'get_tools' or 'tool_call' beyond mentioning 'get-tools method' as a source for flowId.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'once you have a flowId' and mentions two sources for obtaining flowId ('user directly OR using get-tools method'). This gives practical guidance on prerequisites. However, it doesn't explicitly state when NOT to use this tool or provide direct alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.