Server Quality Checklist

Tool Scores

• match_tools_for_queryB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose meaningful behavior: semantic evaluation, selection of only necessary tools, and pruning to prevent token/context overload. Missing are operational details such as output shape, whether external models are invoked, or how max_tools interacts with pruning, but the core behavior is stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence that front-loads the mechanism ('RAG over Tool Capabilities') and then states purpose and benefit. It contains no filler, though the phrase 'RAG over Tool Capabilities' is slightly jargon-heavy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The overall intent is clear, and an output schema exists to cover return values. However, with no annotations and two underdocumented parameters, the description leaves the meaning and effect of max_tools ambiguous and gives no guidance on expected query formatting or limits. It is minimally viable but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description mentions neither 'query' nor 'max_tools.' The purpose implies that 'query' is the input request, but 'max_tools' semantics are entirely unexplained—an agent cannot tell whether it is a hard cap, a target count, or something else. The description provides no compensation for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Evaluates') and a clear object ('semantic capability fit' of tools), and it states the outcome: pick only necessary tools and prune irrelevant ones. This clearly differentiates the tool from sibling content-search tools because it is a meta-selection/routing tool rather than a search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: when deciding which tools are relevant for a query and avoiding token bloat. However, it does not explicitly state when not to use it, nor does it reference alternatives among the sibling tools, so an agent must infer the routing role rather than being told.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_repo_implementationsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It adds meaningful non-obvious behavior: stripping license boilerplate and formatting clean implementation snippets. However, it omits important context such as what 'verified' means, how results are ranked, or any rate or scope limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with two distinct sentences that each add value: one for purpose and one for output processing. There is no fluff, repetition, or extraneous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema covers return-value structure, but the description still needs to explain parameter semantics, verification criteria, and when this tool is preferred over siblings. Since the schema provides no parameter descriptions and annotations are absent, the current description leaves essential usage context missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage across all four parameters, and the description does not compensate by explaining topic, top_k, language, or min_stars. An agent gets no guidance on how to set these parameters or what defaults mean.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a concrete resource ('verified open-source GitHub repositories'), and a clear intent ('code implementations and architectural patterns'). This distinguishes it from the sibling tools, which target academic papers and general web search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for finding code implementations in GitHub repositories, but it never explicitly states when to use this tool versus alternatives, nor does it mention exclusions or when not to use it. The context signals hint at alternative tools, but the description itself provides no direct routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fallback_web_searchB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It adds useful facts beyond the schema: 'zero-cost' and the anti-poisoning filter applied to snippets. It does not, however, explain limitations, failure behavior, or what the filtered result set looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences deliver the core purpose, scope, cost trait, and filtering behavior with no filler. The most decision-relevant information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter search tool with an output schema, the description is largely sufficient for an agent to invoke it correctly. The main gap is the underspecified fallback trigger, but the operational context is otherwise clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain either parameter. While 'query' and 'max_results' are fairly self-explanatory from names and schema defaults, the description adds no meaning beyond what the input schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a DuckDuckGo search and specifies topical scope ('recent news, product launches, and general topics'). It is distinguishable from academic/repository-focused siblings by the explicit 'general topics' scope, though it does not name a specific sibling for contrast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'fallback' implies this tool is used when primary search options are unavailable or unsuitable, and the topic list gives useful context. However, it never explicitly states when to use this tool versus alternatives or when not to use it, leaving the fallback trigger to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unified_research_contextA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses several meaningful behaviors: it prunes tools to prevent API waste, sanitizes snippets against prompt injection, reranks on CPU, and returns bounded context. This is strong transparency, though it stops short of covering all operational details such as required auth or external API calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with a clear pipeline label, followed by a numbered list that makes each behavior easy to scan. Every line adds some information, though the phrasing is somewhat jargon-heavy and could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so the return type does not need to be explained in detail. The description covers the main invocation-relevant behaviors: tool selection, pruning, sanitization, reranking, and bounded output. It is incomplete only in that it does not specify what a valid `query` looks like or how this tool relates to sibling matching tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It indirectly clarifies `top_k` by saying it selects the 'top 1-2 optimal tools' and clarifies `query` by mentioning 'semantic capability match.' However, neither parameter is explicitly defined, and the relationship between `top_k` and the stated 'top 1-2' boundary is left ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes a specific multi-source research context pipeline and states the outcome: 'returns bounded, dense grounded context.' The numbered behaviors make the tool's role clear, though it does not explicitly contrast itself with the sibling match_tools_for_query, which appears closely related.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as search_academic_papers, fetch_paper_deep_context, or match_tools_for_query. The description implies it is useful for multi-source context gathering with tool selection, but it does not state conditions, exclusions, or when a sibling would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_academic_papersA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it does disclose the core behavioral trait: results are 'sanitized' abstracts with publication years, citation counts, and authors. It does not detail ranking, rate limits, or how 'sanitized' is defined, but for a search tool the return behavior and non-destructive nature are sufficiently conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the tool's purpose and output format. There is no filler, repetition, or irrelevant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate at a high level given an output schema exists: it names sources and return fields, and parameter names hint at usage. However, it lacks parameter semantics, usage boundaries, and guidance on how to choose among siblings, leaving the agent to infer key call details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the five parameters. Parameter names like query, top_k, and min_citations are somewhat self-explanatory, but the description adds no meaning about valid ranges, filter behavior, or how year_start and year_end interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Search'), a clear resource ('peer-reviewed papers and preprints'), and specific data sources ('arXiv and Semantic Scholar') while listing return fields. This distinguishes it from siblings like search_repo_implementations and fallback_web_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided about when to prefer this tool over siblings such as fetch_paper_deep_context, search_repo_implementations, or fallback_web_search. The intended use is only implied by the tool name and description, so an agent gets no exclusion criteria or alternative routing hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_paper_deep_contextA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It goes beyond the schema by disclosing that responses are sanitized, LaTeX equations are normalized, and prompt injection payloads are neutralized. It does not cover every edge behavior, but the most important security-related traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The primary function is front-loaded, followed by concise behavioral details about normalization and security. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple enough that the description covers the main purpose, and an output schema exists so return values need not be explained. However, max_tokens is undocumented in both schema and description, and there is no guidance about when to choose this tool over unified_research_context. This leaves minor but real gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It tells the agent that the tool targets an arXiv ID, but it does not explain arxiv_id format expectations or the meaning of max_tokens and its default behavior. Without this, an agent cannot infer parameter semantics from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Fetch') and a precise resource: structured, sanitized abstract and detailed technical context for a specific arXiv ID. This clearly distinguishes it from the sibling search tools, which are about discovering papers rather than retrieving deep context for a known ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a specific arXiv ID' gives clear contextual guidance: use this when you already have an ID and need deep, sanitized technical context. It does not explicitly name alternatives or exclusions, but the intended trigger condition is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.