Server Quality Checklist

Tool Scores

• github_searchA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for disclosing behavioral traits. It fails to mention any limitations, rate limits, pagination behavior, authentication requirements, or the nature of the return value. While the read-only nature of a search is implied, the description does not explicitly state it or provide any cautionary context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero fluff. The description is front-loaded with the core verb and resource, then adds a concise usage preference. Every word earns its place, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no output schema, and no annotations, the description is too sparse. It does not explain how to construct a query, what the return format looks like, or how pagination works. The high-level purpose is clear, but operational details are entirely missing, leaving the agent unable to use the tool correctly beyond a simple repository search.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for explaining parameters. It explains the 'type' parameter by listing the searchable entities (repositories, code, issues, users), but it does not clarify the meaning of 'query', 'page', 'stars', or 'language', leaving the agent without guidance on how to use these parameters effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Search') and resource ('GitHub'), and enumerates the specific searchable entities (repositories, code, issues, users). It also provides a distinctive use case ('open source repos, code patterns, and real-world usage examples'), which differentiates it from sibling tools like web_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Prefer for open source repos, code patterns, and real-world usage examples' explicitly indicates when to use this tool. It gives clear context but does not mention when not to use it or explicitly name alternatives, though the sibling web_search is implied as the alternative for non-GitHub searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the type of data returned ('diagnostics, provider health, budget state'), implying a read-only operation, but it does not disclose side effects, response format, or any potential caveats such as staleness or access restrictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the verb and resources. Every word is informative, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no input parameters and no output schema, the description covers the essential context: it names the categories of data returned. It is slightly light on what 'server diagnostics' entails or how budget state might be represented, but for a simple status tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no ambiguity to resolve. The input schema is empty and description coverage is 100% (vacuously). The baseline for zero parameters is 4, and the description appropriately adds no parameter information since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves server diagnostics, provider health, and budget state. The verb 'Get' plus the specific resources make the purpose unambiguous and easily distinguishable from sibling tools like web_search and github_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but gives no guidance on when to use it or how it compares to alternatives. It does not mention prerequisites, exclusions, or recommend this tool over others for specific scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must convey behavioral traits. It conveys that the tool returns current/realtime information and implies a read-only operation, but it does not disclose output format, pagination, rate limits, or any side effects. The reference to 'server instructions' offloads detail rather than providing it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every word earns its place. The imperative instruction adds practical value without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter schema and no output schema, the description covers the tool's purpose, topical scope, and a clear usage rule. It could be more complete by stating return value expectations, but the combination of the schema and the pointer to server instructions makes it adequate for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the query parameter description is very detailed (keyword style, quotes, site: operator, date handling). The tool description adds general guidance on when to search but no additional parameter-level detail, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's action ('Search the web') and its scope ('real-time data, current events, documentation, and up-to-date factual information'). It distinguishes itself from the sibling 'github_search' by targeting general web content rather than code repositories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a strong usage directive: always search first for technical specifics rather than relying on training data. It also lists the types of information this tool is meant for, though it does not explicitly mention when to prefer alternative tools like 'github_search'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_search_usageA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains the purpose and timing but does not disclose any side effects, error conditions, or what happens if called prematurely. For a simple reporting tool, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the purpose and a clear usage instruction. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param tool with no output schema, the description provides the essential 'what' and 'when.' It could mention what the response looks like or that this is the final step, but the given context is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the params are fully documented there. The description adds no extra meaning beyond what the schema already states, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Report which search results were actually cited') and clearly distinguishes this from sibling tools like web_search and status. It also gives a precise context ('in the agent's final answer').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this after you have formulated your response using web_search results,' providing a clear timing condition. It doesn't explicitly name alternatives, but the relationship to web_search is evident and the instruction is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.