Server Quality Checklist

Tool Scores

• bevigil_get_app_packagesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate readOnly, idempotent, and non-destructive, so the description does not need to repeat those. It adds a minor transparency detail by stating 'Returns package IDs and app names where available,' which hints at potential missing app names. However, it does not elaborate on rate limits, errors, or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, using only three sentences to convey the purpose, use case, and expected return. There is no redundant or extraneous information, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately states what is returned: 'package IDs and app names where available.' It covers the core functionality and use case. However, it does not mention potential limitations, error conditions, or how pagination parameters affect results, which could be considered a slight gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for all three parameters, and the tool description does not add additional meaning beyond that. Since schema coverage is 100%, the baseline is 3. The description's generic mention of returning packages does not enrich the parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Find Android application packages that reference a given hostname or domain.' It specifies the verb (find), the resource (application packages), and the context (hostname/domain). This distinguishes it from sibling tools like get_hosts or get_urls, which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: 'Use this for reverse lookups — given a server/domain, discover which mobile apps communicate with it.' This implies when to use the tool and differentiates it from alternatives, though it does not explicitly mention when not to use it or compare directly to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_get_hostsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, so the safety profile is covered. The description adds behavioral context by stating it 'Returns a list of unique hostnames/domains extracted from the app,' which clarifies the output nature. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three succinct sentences: the first states the purpose, the second gives the use case, and the third covers requirements and output. No wasted words, and critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, complete parameter schema, and rich annotations, the description is fairly complete. It covers purpose, use case, requirements, and return type. It does not mention pagination behavior explicitly, but the schema handles that, so the description is adequate for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all three parameters (package_id, limit, offset), so the schema already explains them thoroughly. The description adds minimal parameter-related value beyond reinforcing that package_id is required; it does not elaborate on limit/offset semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get hostnames and domains') and the resource ('Android application's code'). It explicitly differentiates from sibling tools like bevigil_get_subdomains and bevigil_get_urls by focusing on hostnames/domains and their use case (discovering backend servers, APIs, third-party services).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear use context: 'Use this to discover backend servers, APIs, and third-party services that a mobile app communicates with.' It also specifies a prerequisite ('Requires an Android package ID'). However, it does not explicitly mention alternatives or when not to use it, though the sibling tool names imply differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_get_s3_bucketsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety profile is covered. The description adds that finding a bucket name does NOT confirm vulnerability, which is critical context for interpreting results. This goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences: purpose, value, and caveat. It is front-loaded with the primary action and resource, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the tool's purpose and adds a security caveat. It does not describe return format or pagination behavior, but the schema covers parameters and annotations cover safety. For a read-only search tool, this is adequate but could mention pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description does not add any parameter-specific meaning beyond what the schema provides, such as package_id semantics or pagination details. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds Amazon S3 bucket names referenced in an Android application's code, and differentiates from siblings by focusing on S3 buckets specifically. It also notes the useful security context of discovering cloud storage assets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovering cloud storage assets and includes a caution about not confirming vulnerability, which guides interpretation. However, it does not explicitly state when to use this over siblings, though the purpose is clear enough given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_get_subdomainsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false. The description adds meaningful behavioral context beyond these: results are drawn from BeVigil's mobile app index rather than DNS, are unique subdomains, and reflect only what appears in that indexed dataset. This helps set expectations about coverage and source without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, and every sentence adds value: what the tool does, why it is useful, what it returns, and a caveat about data provenance. There is no filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple read-only lookup with three well-documented parameters and informative annotations, the description is complete enough for an agent to select and invoke it correctly. It covers input domain, output type, data source, and practical use cases, and no output schema is needed to understand the list-of-subdomains return shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with domain, limit, and offset all already documented. The tool description adds no parameter-specific semantics beyond the schema; it only restates the domain concept and output shape. Since the schema carries the full parameter burden, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Discover') with a clear resource ('subdomains associated with a domain') and identifies the unique data source ('mobile applications indexed by BeVigil'). It also states the output ('list of unique subdomains'), which clearly differentiates it from sibling tools focused on hosts, URLs, S3 buckets, app packages, and wordlists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by naming attack surface mapping and asset discovery as intended use cases. It also includes an important scoping note that results come from mobile app analysis, not traditional DNS enumeration, which helps users judge relevance. It does not explicitly name sibling alternatives or exclusions, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_get_urlsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool read-only, idempotent, and non-destructive, so the description only needs to add behavioral context beyond safety. It does so by noting results 'may include internal/staging endpoints that are not publicly indexed by search engines,' which is a meaningful data-quality caveat. It also states the return shape as a list of full URLs, adding value without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four short sentences, each adding distinct value: what the tool retrieves, where the data comes from, the use case, the return type, and a caveat. It is front-loaded with the core purpose and contains no filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple paginated list tool with full schema coverage and strong annotations, the description is complete. It identifies the input domain, explains the result is a list of full URLs, and adds an important caveat about internal/staging endpoints. No output schema exists, but the description sufficiently covers return semantics without needing to repeat schema pagination fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for domain, limit, and offset already embedded in the input schema. The tool description does not add parameter-level details, so it provides no additional semantic value beyond the schema. A baseline of 3 is appropriate given the schema carries the full burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pairing ('Get URLs associated with a domain') and clearly distinguishes this tool from siblings focused on hosts, subdomains, S3 buckets, app packages, and wordlists. It also adds provenance ('as discovered from mobile application analysis') and clarifies the output is full URLs, making the purpose immediately unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use case: 'Useful for finding API endpoints, web paths, and resources referenced by apps.' This is explicit context for when to use the tool, though it does not name alternative tools or state when not to use it. That places it just below the top tier because exclusions and sibling comparisons are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_get_wordlistA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds that it extracts data from the app's code and returns a list of path/endpoint strings, which clarifies the output structure. This goes beyond annotations without repeating them. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (two sentences) and front-loaded with the primary purpose. Every phrase adds value, such as 'generated from the app's code' and 'returns a list of path/endpoint strings'—no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with three parameters fully described in the schema and comprehensive annotations, the description is sufficient. It covers what the tool does, what it returns, and typical use cases. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds no parameter-specific insights. All three parameters (package_id, limit, offset) are already well-described in the schema. The description focuses on output and use cases, which is acceptable given the schema's completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'get a wordlist generated from an Android application's code' and explicitly lists contents (URL paths, API endpoints, file paths, parameters). This distinguishes it from siblings like get_urls or get_hosts by focusing on a consolidated wordlist for fuzzing and API mapping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is 'useful for fuzzing, directory brute-forcing, and understanding the app's API surface,' which provides clear use cases. It does not explicitly say when not to use it or directly compare to alternative tools, but the intent is evident from the stated purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bevigil_investigate_appA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false不言而喻，说明是安全只读操作。Description adds the crucial caveat that findings are observed data and do not indicate vulnerabilities without further analysis, which is valuable context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is two sentences plus a note, all front-loaded with purpose and usage distinction. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (combines multiple lookups) but has only one parameter and strong annotations. The description mentions the combined nature, the alternative, and the 'observed data' caveat. Without an output schema, it gives enough context for the agent to understand its scope and limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description covers the single parameter fully (100% coverage). The tool description repeats the schema's definition but adds no new semantics. Baseline 3 is appropriate since the schema already provides complete parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a comprehensive investigation of an Android app, combining multiple lookups (hosts, S3 buckets, parameters, wordlist) into a single report. It explicitly contrasts with querying individual asset types, distinguishing it from sibling tools like bevigil_get_hosts and bevigil_get_subdomains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use this tool: 'Use this when you need a broad overview of an app's exposed infrastructure rather than querying individual asset types.' This directly addresses alternatives and sets up a clear decision rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.