Server Quality Checklist

Tool Scores

• get_providerA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It does not mention whether the tool is read-only, requires authentication, or has side effects. The description focuses on the information returned but omits operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys multiple aspects efficiently. It is front-loaded with 'Everything about one provider,' but the density may slightly reduce readability. Still, it is concise and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description lists the key information returned (route, pricing, scorecards), which provides adequate context. However, it does not specify the exact format or structure of the response. For a simple tool with one parameter, this is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the schema already describes the single parameter (slug) fully. The description adds context about the route pattern but does not significantly enhance understanding beyond the schema. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides comprehensive information about a single provider, including route, pricing, and scorecards. It distinguishes itself from sibling tools like 'get_scorecard' and 'recommend_tools' by focusing on a specific provider's full details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'when to choose or avoid it,' implying usage context, but does not explicitly state when to use this tool versus alternatives like 'get_scorecard' or 'about_auxiliar.' No explicit when-not or sibling differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scorecardA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It mentions returning a leaderboard with metrics and dates, but does not disclose whether the tool is read-only, has rate limits, requires authentication, or any other side effects. The description lacks transparency about the tool's behavior beyond its output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first defines the output, the second gives usage guidance. Every word adds value with no repetition or fluff. It is front-loaded with the key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter with 100% schema coverage, no output schema, and no annotations, the description adequately explains the return data (leaderboard, metrics, dates). It could mention if there are any limits (e.g., number of providers) but is otherwise complete for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema coverage is 100% with detailed enum descriptions for the 'job' parameter. The description adds context about the output (leaderboard, metrics, dates) but does not add meaning to the parameter itself beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies exactly what the tool does: provides a full benchmark leaderboard for one job/verb, listing providers with metrics and dates. It clearly distinguishes from sibling tools like get_provider and recommend_tools by focusing on comparison rather than selection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the use case: 'Use to compare all options rather than be handed a pick.' This tells the agent when to invoke this tool (when comparing providers) and implies not to use it when a single recommendation is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_toolsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the tool's behavior well: it ranks providers based on measured benchmarks, returns executable routes, numbers, and caveats. There is no mention of side effects, but as a recommendation tool, destructive actions are unlikely. The description is transparent about inputs and outputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each serving a distinct purpose: the first states what the tool does and how it works, the second states what it returns and when to use it. No superfluous information, front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and no output schema, the description covers the essential purpose, usage, and return types. It could be more specific about the output structure, but 'executable routes, numbers, and honest caveats' gives adequate guidance. The absence of output schema does not leave critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and each parameter has a clear description in the schema. The tool description adds little beyond the schema for parameter meanings, but it provides overall context. Baseline 3 is appropriate as the description does not significantly augment parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Recommend') and resource ('web-access providers'), explicitly states the ranking criteria (measured benchmarks), and distinguishes from siblings by focusing on provider comparison for a given job.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes 'Use when deciding which search/scraping/crawling/extraction/browser API to use', providing clear context. However, it does not mention when not to use it or explicitly contrast with sibling tools, though the sibling names imply different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• about_auxiliarA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It accurately describes the tool as an informational resource with no side effects or behavioral traits beyond providing explanations. There is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two sentences to convey the essence of the tool. It front-loads the purpose and ends with a direct instruction, wasting no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema), the description provides sufficient context about what the tool offers. It could perhaps mention the output format, but is otherwise complete for an informational tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are automatically satisfied. The description adds context about the gateway's capabilities, but this is not parameter-specific. Baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool provides information about what auxiliar.ai is and how to use it as an API gateway. It distinguishes itself from sibling tools (get_provider, get_scorecard, recommend_tools) by being the introductory tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this to learn how to get a key and call the gateway,' which provides clear context for when to use it. It does not mention alternatives or exclusions but implies this is for initial orientation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.