Server Quality Checklist

Tool Scores

• get_toolsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool returns dynamic tools and flows based on user access, which adds some context about permissions and scope. However, it doesn't cover critical aspects like whether this is a read-only operation, potential rate limits, error conditions, or response format details, leaving significant gaps for a tool that fetches system metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise at four sentences, but it could be more front-loaded. The first sentence states the core purpose, but subsequent sentences mix usage guidance and additional return details without a clear hierarchical structure. Some redundancy exists (e.g., mentioning tool list retrieval multiple times).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (fetching dynamic system metadata with access control), no annotations, and no output schema, the description is incomplete. It lacks details on return format (e.g., structure of tool/flow lists), error handling, authentication needs, or how the parameters influence results, making it inadequate for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the two required parameters (question and tool). The description doesn't add any parameter-specific information beyond what's in the schema, such as explaining how these parameters affect the search or results. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'retrieve a list of available tools from the Veyrax API' and mentions it also returns flows. It specifies the verb ('retrieve') and resource ('list of available tools'), but doesn't explicitly differentiate from sibling tools like get_flow or tool_call, which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some implied usage guidance: it suggests using this tool to get tool information before calling tool_call, and mentions it returns dynamic tools and flows. However, it lacks explicit when-to-use rules, alternatives, or exclusions compared to siblings like get_flow, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flowB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden of behavioral disclosure. It states this is a retrieval operation but doesn't mention whether it requires authentication, has rate limits, what happens with invalid IDs, or what the return format looks like. The description adds some context about what a workflow contains, but lacks critical behavioral details for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences. The first sentence clearly states the purpose, the second provides helpful context about workflows, and the third gives usage guidance. Each sentence earns its place, though the second sentence could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter retrieval tool with no output schema and no annotations, the description provides adequate basic information but lacks details about return values, error conditions, and behavioral constraints. It covers the core purpose and usage but doesn't fully compensate for the missing structured information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'flowId' parameter. The description adds minimal value beyond the schema by mentioning that flow IDs can come from 'user directly OR using get-tools method', but doesn't provide format examples or validation rules. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'retrieve a specific workflow by its ID' with additional context about what a workflow is. It distinguishes from sibling 'get_tools' by focusing on retrieving a single workflow rather than listing tools. However, it doesn't explicitly differentiate from 'tool_call' which might execute workflows.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'once you have a flowId' and mentions 'get-tools method' as an alternative way to obtain flow IDs. It doesn't explicitly state when NOT to use it or provide detailed alternatives, but the guidance is sufficient for basic usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tool_callB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that parameters are 'based on get-tools tool response,' which hints at dependency, but doesn't disclose critical behavioral traits like error handling, authentication needs, rate limits, or what happens if invalid tool/method names are provided. For a meta-tool that executes other tools, this lack of transparency is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded with the main purpose. Every sentence earns its place by explaining the tool's function and prerequisite. However, it could be slightly more concise by combining ideas, and the structure is simple but effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a meta-execution tool with no annotations and no output schema, the description is moderately complete. It covers the purpose and prerequisite but lacks details on behavioral aspects like error handling or return values. For a tool that dynamically invokes other tools, more context on execution flow and results would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds minimal value beyond the schema by mentioning that parameters are 'required for that method' and 'based on get-tools tool response,' but doesn't provide additional syntax, format, or examples. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'execute a specific method of another tool with the provided parameters based on get-tools tool response.' It specifies the verb ('execute'), resource ('method of another tool'), and prerequisite ('based on get-tools tool response'). However, it doesn't explicitly differentiate from sibling tools like 'get_flow' or 'get_tools' beyond mentioning the latter as a prerequisite.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: after using 'get_tools' to discover available tools and methods. It implies usage by stating 'based on get-tools tool response,' which guides the agent to first call 'get_tools.' However, it doesn't explicitly state when NOT to use it or name alternatives, such as directly calling other tools if their methods are already known.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.