Server Quality Checklist

Tool Scores

• list_available_modelsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose behavioral aspects such as side effects, permissions, or whether it is read-only, relying solely on the verb 'list' to imply non-mutating behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose without superfluous details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description sufficiently conveys the operation and its scope, meeting all necessary contextual requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters to explain; the description is complete and needs no additional parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and the resource (Qwen/Ollama models) within a specific context (local Ollama server), distinguishing it from sibling tools that focus on page analysis and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not specify when to use this tool versus alternatives, lacking explicit guidance on conditions or comparisons with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_site_and_analyzeA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency load. It successfully discloses the core behaviors: uses external search engine, parsing through trafilatura and Qwen, handles layout changes, reports progress per match/chunk, and has long latency per match. It does not mention possible rate limits, no-match behavior, failure modes, or network auth, but covers most relevant behavioral traits for an AI agent considering this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured into a high-level workflow, mechanism, performance note, and Args section. Every sentence contributes useful information such as time estimates, layout-resilience, or parameter semantics. It is concise without sacrificing essential guidance, and the 'Args:' list makes scanability easy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-step search-and-analysis tool, the description covers when to use it, how it works, what each parameter means, time cost, progress reporting, and the fact that output follows analyze-style extraction. Because an output schema exists, avoiding return-value detail is appropriate. The description provides enough for an agent to decide if this tool is appropriate and set expectations about runtime.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description is responsible for parameter meanings. It provides meaningful explanations for all four params: `site` with an example domain, `phrase` as an exact phrase, `question` as what info to extract, and `max_results` as maximum matching pages with a default of 5. This fully bridges the schema gap, though some examples or constraints are minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific, action-oriented phrase: 'Search a specific site for pages/listings containing `phrase`, then run `analyze_page`-style extraction on each match to answer `question`.' This clearly defines the tool's responsibility and differentiates it from the sibling tool `analyze_page`, which focuses on a single page. The scope (site-level search plus extraction) is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use it ('Works on any site without site-specific scraping code'), how it works (DuckDuckGo `site:` search, trafilatura + Qwen), and yields a practical performance caveat: 'Each match takes roughly 30s-3min to fetch and analyze.' It does not explicitly name an alternative for single-page analysis, though it references `analyze_page`-style extraction, so it provides clear context but no explicit 'when not to use' statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_pageA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description takes on full burden. It thoroughly discloses behavior: uses local Qwen model, splits long pages into chunks (map-reduce) to avoid skipping content, reports progress (page fetch retries, per-chunk analysis), and warns about long execution time (30s-3min). This is highly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with each sentence adding value. It front-loads the core function, then explains special behavior (chunking and progress) and a clear Args section. No redundant phrases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 params, output schema exists, no annotations), the description covers all essential aspects: purpose, parameters, behavior (chunking, progress), and time expectations. The presence of an output schema reduces need for return value details. It is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description provides clear meanings for both parameters: 'url' is the page to fetch/analyze, and 'question' is what to extract or answer. This compensates fully for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear action: fetching a web page and using a Qwen model to extract/answer a question about its content. It distinguishes itself from siblings by focusing on a single URL analysis, whereas search_site_and_analyze implies searching first, and list_available_models is about models.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states when to use: to fetch a specific page and answer a question about its content. It implicitly contrasts with search_site_and_analyze by mentioning 'a web page' (single URL) vs. site search. However, it does not explicitly mention alternatives or when not to use it, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.