Server Quality Checklist

Tool Scores

• search_by_domainB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds only that the search matches subdomains, but omits important behavior such as the meaning of 'recent' (e.g., time window), result ordering, pagination, or its read-only nature. This is insufficient for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, consisting of two short sentences with no redundant information. Every word contributes to the purpose, and the subdomain-matching detail adds behavioral value without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a relatively simple search tool with an output schema, but the description lacks usage guidelines and parameter semantics. It covers the core purpose and a key behavioral trait (subdomain matching), yet leaves gaps that could confuse an agent trying to invoke it correctly, especially given the 0% schema description coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter documentation. It does not explain 'domain', 'days', or 'size' beyond a vague hint that scans are 'recent'. The defaults are present in the schema but their meaning is left to inference, which is inadequate at this coverage level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Find' and clearly identifies the resource as 'recent scans of a domain', with an explicit note that it matches the domain and its subdomains. This distinguishes it from sibling tools like search_by_ip, search_by_asn, and search_by_hash, which target different resource types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to prefer this tool over alternatives such as search_scans, search_by_ip, or search_by_asn. The description implies domain-based search but does not state prerequisites, exclusions, or a comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_asnA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only says 'Find recent scans' without disclosing that this is a read-only operation, the meaning of 'recent' (tied to the days parameter), result size limits, or any other behavioral traits. This is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action and target, and includes an example. There is no wordiness or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three parameters and an output schema, the description provides the core purpose but omits context about default time range, result pagination, and how it relates to sibling search tools. It is adequate but minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The example 'AS15169' gives a format hint for the asn parameter, but there is no explanation of the days or size parameters. With schema description coverage at 0%, the description fails to compensate for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds recent scans within an autonomous system, using an example ASN ('AS15169'). This specifies a unique verb+resource+scope, distinguishing it from sibling search tools like search_by_domain, search_by_ip, and search_by_hash.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context that this tool is for ASN-based search, making it obvious when to use it. However, it does not explicitly exclude alternatives or mention when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_and_waitA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. It discloses the submit-poll-summarize behavior and API key requirement, but it does not mention side effects like scan creation or quota consumption, nor does it explain blocking/timeout behavior beyond the default timeout parameter. This is moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the core action and outcome. Every sentence adds value: the action, the usage guidance, and the authentication requirement. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema means return values don't need explanation. However, with no annotations and zero parameter description coverage, the description misses important context about optional parameters and side effects. It is adequate for a default workflow but incomplete for full parameter understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only mentions 'URL' and gives no explanation of tags, country, visibility, or timeout_seconds. The defaults in the schema provide some implicit meaning, but the description adds little beyond restating the primary input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Submit a URL and wait for the finished, summarised result.' It clearly distinguishes the tool from siblings by noting that it handles the full submit, poll, and summarise cycle in one call, unlike separate scan_url or get_scan_result steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says this is the tool to reach for in most workflows and describes the combined cycle, which gives clear context for when to use it. It does not explicitly state when not to use it or mention alternatives, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_ipA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It communicates that results are 'recent scans' and that only pages served from the IP are included, which is useful. But it does not mention potential side effects, rate limits, or that it is a read-only operation, leaving some gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first sentence front-loading the core function and the second sentence adding practical value. Every word earns its place; there is no redundancy or fluff, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, and the description covers its primary function and use case. Since an output schema exists, return-value details are not required. However, parameters like 'days' and 'size' are left entirely to the schema, and the lack of annotations means the description could provide more operational context, such as default behavior or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only alludes to the 'ip' parameter via the tool name and description, and gives no explicit meaning for 'days' (lookback period) or 'size' (result limit). The word 'recent' hints at days, but the guidance is too vague to be helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Find recent scans of pages served from an IP address.' It uses a specific verb and resource, and the IP focus distinguishes it from sibling tools like search_by_domain or search_by_asn. The additional use case about spotting co-hosted content reinforces its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'Good for spotting what else is hosted alongside something suspicious.' This implies when to use the tool (when investigating a suspicious IP). However, it does not explicitly mention alternatives or exclusions, so it stops short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_screenshot_urlA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key behavioral trait: the tool returns a URL instead of binary image data, and explains why. This adds meaningful context. It does not cover error handling or authentication, but for a simple getter tool, the transparency is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, and every sentence adds value. The second sentence justifies the return type without any fluff, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description adequately states the purpose and return behavior, and the presence of an output schema likely covers return structure. However, it lacks any parameter description and does not mention prerequisites, potential errors, or whether the URL is temporary, making it only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter, 'uuid', with 0% description coverage, and the tool description does not explain what the uuid represents beyond the generic phrase 'for a scan'. It adds no details about format, required context, or how to obtain the uuid, leaving the agent to infer from the parameter name and tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets a screenshot URL for a scan, with the verb 'Get' and specific resource 'screenshot URL'. It further distinguishes itself by explaining it returns a link rather than image bytes, setting it apart from sibling tools like get_scan_result or get_page_dom.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by stating that screenshots are large and 'meant for a human to open', implying this tool is appropriate when a human will view the image. However, it does not explicitly mention alternative tools or exclusion scenarios, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_hashA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral transparency burden. It makes clear the operation is a read-only search ('Find scans'), and adds useful context about pivoting from a hash to pages. However, it does not explicitly state that this is a read-only operation or mention any potential limitations (e.g., rate limits, whether results are limited by the 'size' parameter).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the core function, and adds a concise, valuable use case in the second sentence. Every word contributes meaning; there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple hash-based search with an output schema, so the description does not need to explain return values. It covers the purpose, the input, and a recommended scenario well. Minor gap: it does not mention the optional 'size' parameter or any search constraints (e.g., time range), but those are partially addressed by the schema and output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It explains the 'sha256' parameter clearly as a 'SHA-256 response hash' of a resource. However, it does not mention the 'size' parameter at all, even though its meaning might be inferred from its name and default value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds scans that loaded a resource with a specific SHA-256 response hash, using the verb 'Find' plus a specific resource type. It distinguishes itself from sibling search tools (search_by_domain, search_by_ip, search_by_asn) by focusing on a hash pivot, and adds a strategic use case ('map a campaign').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when you have a known-bad file hash and want to find all pages serving it, often the fastest way to map a campaign. It does not explicitly mention alternative tools or when not to use it, but the context is strong enough to guide appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_page_domA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behaviors: it requires an API key, truncates to max_chars, and notes that full DOMs regularly exceed 1 MB. This gives the agent important operational context beyond the schema, though it does not cover edge cases like missing DOM snapshots.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, front-loaded with the main purpose, and every sentence adds value (use cases, API key, truncation/scale). No unnecessary filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and presence of an output schema, the description provides sufficient context: what it does, when to use it, and critical behavioral caveats. It misses a direct note about the uuid semantics, but overall it is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions max_chars and explains truncation, but does not clarify the meaning of the required 'uuid' parameter, which is essential for invocation. Since schema description coverage is 0%, the description must compensate more, and it only partially does so for max_chars, not uuid.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch the captured DOM snapshot for a scan,' identifying the specific resource and action. It distinguishes this tool from siblings like get_scan_result and get_screenshot_url by focusing on the DOM snapshot, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context with 'Useful for inspecting injected scripts, hidden form fields, or obfuscated payloads,' but does not explicitly mention when not to use this tool or name alternatives. This is clear context without exclusions, fitting a score of 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_indicatorA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool aggregates recent scans, computes specific statistics (malicious count, verdict scores, recurring tags/brands, hosting spread), and warns to read the caveats field before acting. This gives meaningful insight into the tool's behavior beyond a simple 'assess' action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly written with no filler. It front-loads the purpose, then adds useful detail about the computed stats and concludes with an important caveat warning – every sentence contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex, and the description covers its core analysis, accepted indicator types, and the caveats field. However, it omits explanation of the 'days' parameter, which is a minor gap given the output schema exists and the rest is well described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It clarifies that the indicator parameter accepts domains, IPs, URLs, or SHA-256 hashes, but does not mention the 'days' parameter or its default of 180. Since the description partially compensates but leaves 'days' undocumented, a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Build a reputation picture') and clearly scopes the resource to domains, IPs, URLs, or hashes. It explicitly distinguishes itself from passthrough tools by stating it provides aggregated analysis, setting it apart from scan lookup siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an aggregated reputation assessment is needed, contrasting with raw passthrough. However, it does not explicitly name alternative tools or state when not to use it, so context is clear but exclusions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_scansA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavioral traits. It adds useful context by stating 'Works without an API key, at lower rate limits,' which informs about authentication and rate constraints. However, it does not elaborate on other behaviors such as pagination, sorting, or any limits on the 'size' parameter. The rate-limit mention adds significant value, but not enough for a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence definition, followed by a curated list of useful fields, then three concrete examples. Every part contributes to understanding how to invoke the tool. It is appropriately sized for the complexity and avoids redundant or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are already specified. The description covers the main functionality, query syntax, useful fields, and even rate-limit behavior. It is complete enough for an AI agent to construct valid queries, though it could have briefly mentioned that it's the general-purpose alternative to the specialized search tools. The examples and field list make it largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description compensates by providing a detailed field list and query examples that clarify the syntax and semantics of the 'query' parameter. While the 'size' parameter is not described, it has a default and is self-explanatory. The query examples offer meaning well beyond the bare schema type, leading to a score of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search historical urlscan.io scans with an ElasticSearch query string.' This specifies the verb, resource, and method, and distinguishes it from sibling tools like 'search_by_domain' or 'search_by_ip' by emphasizing the flexible ElasticSearch query capability. The inclusion of useful fields and examples reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on how to use the tool through examples and a list of searchable fields, making it apparent that this is for ad-hoc, query-based searching rather than specific lookups like by domain or IP. It does not explicitly state when not to use it or name alternatives, but the sibling names and the tool's generic nature imply appropriate usage. There is no exclusion guidance, so a 4 is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_countriesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior (listing country codes) but doesn't add details about authentication, rate limits, or other potential side effects. The phishing rationale is more about usage guidance than behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the core function, and the second offers a relevant rationale. It is well-structured and front-loaded with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema present, the description is sufficient. It explains what the tool does and why it is useful, allowing an agent to correctly select and invoke it without further clarification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline of 4 applies. No parameter explanations are needed since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'List country codes available as scanner exit locations' with a specific verb and resource. It clearly distinguishes itself from sibling scanning/search tools by focusing on available countries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by explaining the geo-targeted phishing scenario and how re-scanning from the right country helps. While it doesn't explicitly name alternatives or exclusions, the intended use case is effectively communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_urlA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does a good job: it discloses the asynchronous behavior (immediate UUID return, 10-30 second completion), the API key requirement, and the visibility values. It lacks details on permission side-effects or potential failures, but overall it is notably transparent for a submission tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the action, and every sentence contributes value: purpose, behavior, alternative tool, and key constraints. No filler or redundancies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the existence of an output schema, return values don't need explaining. The description covers the core purpose, async behavior, alternative, and the most critical parameters. However, it leaves several parameters unexplained and does not mention how the scan result will be retrieved, such as via get_scan_result, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 0%, so the description must compensate, but it only explains two of six parameters: visibility (enum) and country (ISO code). The url, tags, referer, and user_agent parameters are not described, leaving the agent to infer their meaning from names and purpose. This partial compensation meets a minimum viable level but not more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the primary action: 'Submit a URL to urlscan.io for scanning.' It further specifies the immediate return with a UUID and distinguishes itself from the sibling tool scan_and_wait, making the purpose unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly directs users to use scan_and_wait instead when they want the finished result in a single call, directly addressing when to use this tool vs. an alternative. Also provides temporal context (10-30 second scan duration) that helps select this async option.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quotasA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states this is a read-only quota display and requires an API key, but does not explicitly confirm it consumes no quota or describe edge cases like missing/invalid keys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences. The first states the core purpose, the second adds practical usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple no-param tool with an output schema, the description covers purpose, timing, and prerequisite clearly. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters and schema coverage is 100%, so there is no param semantic burden. The baseline for zero params is 4, and the description needs no further parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it shows remaining urlscan.io API quota for the configured key. This is a specific verb+resource combo that distinguishes it from the many scanning and search sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Worth checking before a batch of scans', giving clear context for when to use it. It also notes the API key prerequisite, but does not mention when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scan_resultA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the need for an API key, the default summary response, and the important behavior that full=True can return a multi-megabyte document that may dominate the context window. It does not cover error handling or rate limits, but the disclosed details are valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every sentence adds value. The warning about size is concise and actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with an output schema, the description is sufficient. It covers the purpose, auth requirement, default behavior, and a critical caveat about large responses. The output schema handles return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage, so the description must add meaning. It clarifies that uuid identifies the scan and explains that full controls whether a complete document is returned instead of a summary, including the consequence of setting it to true. This goes beyond the bare parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch the result of a completed scan by UUID', which is a specific verb+resource combination. It distinguishes this from sibling tools like search_scans (which searches) and scan_url (which initiates scans).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'completed scan' implies the tool should be used after a scan has finished, and the warning about setting full=True for the complete document provides practical guidance. However, it does not explicitly name alternative tools or state when not to use this tool, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_capabilitiesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds valuable behavioral context by disclosing that submission and quota lookup need an API key while search and result retrieval do not, and it suggests using the tool for troubleshooting refusals. It stops short of explicitly stating read-only behavior, but the nature of a 'report' implies it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences), front-loaded with the core purpose, and every sentence adds meaningful value. The second sentence provides essential usage context without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no parameters, the description fully covers the tool's purpose, usage conditions, and operational context. With an output schema present and low complexity, the description is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so per the rubric the baseline is 4. The description does not need to add parameter-level detail, and the schema already covers the empty parameter set fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Report' and identifies the resource as 'operations available with the current configuration.' This clearly states what the tool does and distinguishes it from sibling tools like scan_url or get_quotas, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when to call this tool ('Call this first if something is unexpectedly refused') and provides concrete context about API key requirements for different operation types. This gives clear guidance on usage and when to prefer it over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.