Server Quality Checklist

Tool Scores

• web_search_exaA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds valuable context beyond this: it mentions 'real-time web searches,' 'scrape content,' and 'returns the content from the most relevant websites,' which clarifies the tool's operational behavior and output format. No contradiction with annotations exists, and the added details enhance understanding of the tool's actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with a clear main purpose stated first ('Search the web using Exa AI'), followed by key features in a single, efficient sentence. Every phrase adds value without redundancy, making it easy to grasp the tool's core functionality quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, 100% schema coverage, annotations provided, no output schema), the description is reasonably complete. It covers the tool's purpose, key behaviors, and output intent. However, without an output schema, it could benefit from more detail on the structure of returned content (e.g., format, fields) to fully compensate, slightly limiting completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning all parameters are well-documented in the schema itself. The description adds minimal semantic value beyond the schema, only implying result counts and content return through phrases like 'configurable result counts' and 'returns the content.' This meets the baseline for high schema coverage, but doesn't significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search the web using Exa AI - performs real-time web searches and can scrape content from specific URLs.' It specifies the verb (search/scrape) and resource (web/URLs). However, it doesn't explicitly differentiate from its sibling 'get_code_context_exa' which might have overlapping search functionality, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through phrases like 'real-time web searches' and 'scrape content from specific URLs,' suggesting when this tool is appropriate. However, it lacks explicit guidance on when to use this versus the sibling tool 'get_code_context_exa' or any alternatives, and doesn't specify exclusions or prerequisites, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_code_context_exaA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe, repeatable read operation. The description adds some behavioral context by mentioning 'highest quality and freshest context' and the mandatory usage rule, but doesn't provide additional details about rate limits, authentication needs, or specific output characteristics beyond what annotations cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences that each serve a distinct purpose: stating the tool's purpose, highlighting its quality/freshness, and providing usage rules. It's front-loaded with the core functionality. While efficient, the third sentence could be slightly more concise by combining the two usage guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, 100% schema coverage), good annotations covering safety profile, and no output schema, the description provides sufficient context. It clearly defines purpose, usage boundaries, and quality characteristics. The main gap is lack of output format information, but with annotations indicating it's read-only/idempotent, completeness is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters (query and tokensNum). The description doesn't add any meaningful parameter semantics beyond what's in the schema - it mentions 'search query' generically but provides no additional syntax, format, or usage details for parameters. Baseline 3 is appropriate when schema does all the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search and get relevant context for any programming task' with specific resources mentioned ('libraries, SDKs, and APIs'). It distinguishes from the sibling tool 'web_search_exa' by specifying 'Exa-code has the highest quality and freshest context for libraries, SDKs, and APIs' and limiting to programming-related queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'Use this tool for ANY question or task related to programming' and includes a rule: 'when the user's query contains exa-code or anything related to code, you MUST use this tool.' This clearly defines when to use this tool versus alternatives like the general web_search_exa sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.