Server Quality Checklist

Tool Scores

• searchC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description does not disclose any behavioral traits such as rate limits, result format, or safety. The description adds minimal value beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise at one sentence, but could be more structured. No additional information is provided that would help an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and simple tool, description lacks details on return values, pagination, or any constraints. Incomplete for effective selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds no additional meaning beyond the schema's 'Search query' and 'Max results'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description 'Search the web' states a verb and resource, but is too vague. It doesn't differentiate from sibling tools like 'crawl' or 'scrape'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Missing context for when search is appropriate compared to crawl or scrape.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_crawl_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only says 'check progress' without disclosing whether the operation is idempotent, safe (read-only), or what the output format is (e.g., status, percentage, errors). This is insufficient for an agent to understand behavioral implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and front-loaded. However, it could be slightly more informative without becoming verbose. It earns its place but leaves room for improvement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, the description should explain the return value or result of checking progress. It does not mention what fields or statuses are returned, making it incomplete for an agent to interpret the response. The tool's simplicity (1 param) does not excuse this omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter 'id' described as 'Crawl job ID'. The description adds no additional meaning beyond the schema. Baseline score of 3 is appropriate as the schema already documents the parameter adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Check crawl job progress' clearly states the verb and resource, indicating the tool monitors an ongoing crawl job. It distinguishes from sibling tools like 'crawl' (initiates), 'scrape' (one-time), and 'search' (queries). However, it could be more specific about what 'progress' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For example, it does not say to use after starting a crawl with the 'crawl' tool or to avoid for completed jobs. The description lacks context for appropriate invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states the basic function without disclosing behavioral traits such as JavaScript execution (implied by waitFor parameter), rate limits, or error handling. The description adds minimal transparency beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence: 'Extract markdown from a URL'. It is front-loaded and contains no redundant information, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, and no annotations, the description is insufficient. It does not explain the effects of optional parameters (formats, onlyMainContent, waitFor) or typical use cases, leaving the agent without enough context to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add meaning beyond the parameter names and schema descriptions; it repeats the same level of detail without clarifying valid formats or the purpose of waitFor.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Extract markdown from a URL' clearly states a specific action and resource. It is not a tautology and conveys the core function. However, it does not differentiate from sibling tools like 'crawl', which may extract content from multiple URLs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., 'crawl' for multi-page extraction, 'search' for querying). It lacks any when-to-use or when-not-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawlB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description does not disclose behavioral traits such as rate limits, robots.txt adherence, page count limits, or depth behavior. Essentially opaque aside from basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise single sentence, but lacks any structure like bullet points or front-loading of key information. Could be improved without increasing length significantly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description does not explain return format or content structure, nor does it clarify boundary behaviors (e.g., what happens when maxDepth or limit is omitted). Incomplete for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with brief descriptions (url: 'Start URL', maxDepth: 'Max depth', limit: 'Max pages'). Description adds no additional meaning beyond schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Crawl', resource 'a website', and outcome 'page content'. It effectively distinguishes from siblings like 'scrape' and 'search' which imply different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use crawl vs alternatives like scrape or search. Missing context about appropriate use cases or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tool_enableB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It discloses activation and listing behavior but omits important details like session scope, side effects, permissions required, or what 'activate' entails (e.g., success response, error states). Minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action, no wasted words. Every sentence provides value: action and usage variant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and a single parameter, the description should clarify what happens upon activation (e.g., confirmation, tool enabled for the session). It only implies listing behavior but does not describe activation outcome or return value, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the description 'Tool name (omit to list available tools)'. The tool description adds 'Use without arguments to list available tools', which reinforces but does not add significant new meaning beyond the schema. Baseline 3 appropriate due to high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool activates an advanced tool for the session, using 'Activate' as the verb and specifying 'advanced tool'. It differentiates from the sibling 'tool_disable' by implication, though not explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage guidance: 'Use without arguments to list available tools'. However, lacks guidance on when to use this tool versus the sibling 'tool_disable' or other alternatives, and no explicit 'when not to use'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tool_disableA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description discloses deactivation for session and listing behavior, but does not cover permissions, reversibility, or side effects of deactivation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, each adding value, front-loaded with main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequately covers two modes and references sibling for more depth; missing details on confirmation or immediate effect, but minor given tool simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter; description adds minimal extra meaning ('omit to list active tools') which is already in schema description. Baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb (deactivate), resource (advanced tool), and scope (session). Distinguishes from sibling tool_enable by referencing it for full list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes two usage modes (list active tools or deactivate) and references tool_enable for alternatives, but lacks explicit 'when not to use' for other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.