Server Quality Checklist

Tool Scores

• quick_searchA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explains what the tool does and what it returns, but it does not describe how 'auto' selects a search engine, potential failures, rate limits, or other behavioral nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured, front-loading the purpose and using clear Args/Returns sections. Every sentence contributes useful information without unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool, the description includes all invocation-relevant details: required query, optional parameters with defaults, engine choices, and return format. It is only missing explicit usage context relative to the sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by documenting all three parameters: query, max_results with default and max, and search_engine with its enum options. This is exactly the information an agent needs beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('fast web search') and resource (the web), and specifies the return format (structured snippets with source URLs). It is distinguishable from the sibling research-report tools, though it does not explicitly name or contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'fast' implies it is for quick searches rather than deep research, but there is no explicit guidance about when to choose this tool over siblings like deep_research. No exclusions or alternative-selection criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_research_reportsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden and it does disclose the action (list) and outcome shape (JSON with filename, query, creation date). It does not mention ordering, empty-directory behavior, or error conditions, leaving minor ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two front-loaded sentences; the purpose appears in the first sentence and the return format is cleanly separated. No filler or repeated schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless listing tool, the description is nearly complete: it names the source, scope, and return fields. With no output schema, explaining the return fields is essential and done well; it only lacks an explicit pointer to sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are trivially complete; per calibration, baseline 4. The description reinforces that no arguments are needed by focusing entirely on the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List'), resource ('previously generated research reports'), and location ('reports directory'). Differentiates from siblings by explicitly scoping to saved reports rather than generating or reading one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied through 'previously generated' — it is the inventory step before read_research_report and separate from deep_research/quick_search. However, it does not explicitly state when to use it or name alternatives, so the agent must infer placement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_research_reportA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the core behavior (read from disk) and return value (full text), but does not explicitly state that it is read-only with no side effects, nor describe error behavior for missing files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, front-loaded sections (summary, Args, Returns) with no redundant content. The example filename earns its place by illustrating the expected format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For one parameter and no output schema, the description adequately explains input and return. The only notable gap is the missing connection to list_research_reports for valid filenames, which is a usage-guidance issue rather than a completeness issue.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates by providing an Args section that explains the purpose of 'filename' and includes a concrete date-format example. It could add where the filename comes from, but the meaning is clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Retrieve and read'), a resource ('saved research report'), and the storage location ('from disk'), making it clearly distinct from siblings like list_research_reports (which lists) and deep_research (which creates).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clear context – it is for saved reports – but does not explicitly say when not to use it or point to list_research_reports for discovering available filenames. The alternatives are inferable from sibling names but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deep_researchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on full behavioral disclosure. It transparently explains the autonomous multi-agent process, including coordination of Web Researcher, Research Analyst, and Technical Writer, and states that it gathers live web data and produces a cited Markdown report. It does not mention runtime expectations, potential costs, or side effects, but the core behavior is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and efficient: a clear one-sentence summary, a brief explanation of the agent workflow, a compact Args list with inline value explanations, and a Returns line. Every sentence adds value and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, no output schema, and no annotations, the description covers the essential information: purpose, workflow, all parameters, and the return value as a Markdown report. It could be more complete with explicit guidance on when to select this tool over quick_search and what resource or time implications exist, but nothing critical is missing for invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must carry the parameter documentation burden. It succeeds by explaining the query as the research topic, defining 'depth' values ('standard' vs 'deep'), listing valid search_engine options with a preferred default, and providing concrete examples for optional model and provider values. Every parameter receives meaningful semantic context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Run an autonomous multi-agent deep research investigation on a topic.' It also differentiates itself from sibling tools like quick_search by emphasizing 'deep research' and a 'publication-grade Markdown report with source citations,' making the outcome and scope distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for thorough, cited research through phrases like 'deep research investigation' and 'exhaustive' depth, but it never explicitly explains when to use this tool over quick_search or when not to use it. No alternatives or exclusions are mentioned, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.