Server Quality Checklist

Tool Scores

• web_readA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It mentions 'clean content' and 'up to 20 URLs', but fails to disclose credit costs (only hinted in parameter descriptions), rate limits, authentication requirements, side effects, or error handling. This is a significant gap for a web scraping tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences, with no wasted words. It front-loads the core purpose and then adds a key output characteristic and a constraint. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no output schema, no annotations), the description is adequate but leaves gaps. It does not explain credit costs, the difference between basic/advanced extraction beyond schema, or return value format. Important behavioral aspects are missing, making it incomplete for an agent to use optimally.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds high-level context about the tool's output, but does not significantly enhance the parameter meanings beyond what the schema already provides. The schema descriptions for extractDepth and includeImages are detailed, so the description adds little extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Extract clean content from web pages.' It specifies the verb 'extract' and the resource 'content from web pages', and distinguishes itself from siblings like web_search by focusing on extracting content from given URLs rather than searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for extracting content from specific URLs, but it does not explicitly state when to use this tool versus alternatives like web_search. It lacks explicit 'when-not-to-use' guidance or mention of prerequisites, though the context of sibling tools provides some implicit differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• credit_balanceA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose behavioral traits such as whether the operation is read-only, requires authentication, or what side effects exist. It only states the purpose, leaving the agent uninformed about important safety or permission aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is front-loaded with the action and resource, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is minimal. For a simple read operation, it might be sufficient, but it lacks details about the return format or any usage constraints, making it only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters (schema description coverage 100%), so the description adds meaning beyond the schema by naming the resource ('credit balance and usage'). Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The tool is named 'credit_balance' and the description states 'Check your provider API credit balance and usage.' This specifies a clear verb ('Check') and resource ('provider API credit balance and usage'), distinguishing it from siblings like web_search or github_get_repo_structure which handle different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use or not use this tool, nor does it mention alternatives. However, given the distinct purpose (credit balance) and sibling tools (web and GitHub operations), the usage context is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the provider is server-side (Tavily or SearXNG) and mentions filtering options, but does not cover rate limits, credit costs, or result limitations beyond schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences contain all essential information with no unnecessary words. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no output schema), the description is adequate but could provide more context on result structure, error handling, or combination of parameters. It doesn't explain the return format beyond basic fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description lists parameter groups (domain, topic, time range, search depth) but adds little detail beyond the schema's own parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search the web' and specifies return fields (titles, URLs, snippets). It distinguishes from siblings like web_read (reading a specific page) and GitHub tools, making it unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for web searches but does not explicitly state when to use over alternatives (e.g., web_read). No guidance on when not to use or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• github_read_fileA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses file size limit (512 KB) and binary rejection, plus tokenless operation. No annotation provided, so description bears full burden; it covers key constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, limitations, token info. No wasted words, front-loaded with key action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input, behavior, and return value clearly. No output schema needed; 'returns decoded text' is sufficient. Addresses constraints adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions (100% coverage). Description adds no extra param details; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Read', resource 'file in a GitHub repository', and mentions 'Returns the decoded text'. Distinct from sibling tools like github_get_repo_structure (directory) and github_search_repo.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on token usage and anonymous rate limits. Implicitly clear when to use (to read file content). No explicit exclusions or alternatives, but purpose is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• github_get_repo_structureA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses non-recursive behavior, anonymous rate limits, and token usage for higher limits, which are key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with primary action, no redundant words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description indirectly implies output (list of items). For a simple directory listing, this is mostly complete; could mention return format but not necessary given low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions. Description adds value beyond schema by explaining defaults for ref (default branch) and path (root), and guidance on forward slashes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'List' and resource 'directory structure', and clarifies 'single level, non-recursive', distinguishing it from recursive alternatives and sibling tools like github_read_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use ('understanding project layout') and mentions token requirements, but does not explicitly exclude scenarios or name sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• github_search_repoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that code search requires indexing, that anonymous rate limits apply, and that 'all' fans out in parallel. It does not detail side effects or error handling, but the disclosed behaviors are sufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, concise and front-loaded: the first sentence states the primary purpose. Every sentence adds essential information without redundancy, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, 2 required, no output schema, and multiple search types, the description covers usage, constraints, and behavior adequately. It could note return format, but completeness is high for the complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. The description adds value by explaining that 'all' fans out in parallel, that state is ignored for code, and default maxResults values. This enhances understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a GitHub repository across code, issues, and pull requests in one call, specifying the underlying API endpoints. It differentiates from sibling tools like github_get_repo_structure (structure) and github_read_file (reading files), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on authentication (anonymous vs. token) and a prerequisite for code search (repo must be indexed). It implies usage for searching across repo content but does not explicitly state when not to use it or compare with alternatives, missing a small bit of guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.