Server Quality Checklist

Tool Scores

• get_web_urlB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It only mentions the return value ('web URL from Cyberbro API') but does not disclose read-only nature, authentication needs, or side effects. Behavioral traits are inadequately addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences and clearly separated args/returns. It is front-loaded with the core purpose. Minor improvement possible by structuring the return statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is mostly adequate but lacks guidance on when to use vs siblings and does not explain if the URL requires authentication. A more complete description would mention usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds only a restatement of the parameter purpose ('Analysis ID to get the web URL for'). No format, constraints, or examples are provided, leaving the agent with minimal additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a web GUI URL for a given analysis ID, using a specific verb and resource. It distinguishes from siblings like get_analysis_results (raw data) and is_analysis_complete (status check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like get_analysis_results or analyze_observable. The description merely states the function without context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• is_analysis_completeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description lacks behavioral details such as whether the operation is read-only, error handling (e.g., if analysis_id is invalid), or rate limits. It only states that it returns completion status without specifying the format or possible values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with the main purpose stated upfront. However, it could include more details without becoming overly long. The structure is clear but minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is somewhat complete. It explains the action and return type ('completion status'), but lacks specifics on the possible status values and error scenarios. Given the simplicity, it is adequate but could be improved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description only repeats the parameter name and purpose ('analysis_id: Analysis ID to check') without adding any new semantic information beyond what the schema's title already provides. It does not explain constraints, allowed values, or provide examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check if the analysis is complete') and the resource ('for the given analysis_id'). The tool name itself is specific, and it is distinct from sibling tools like get_analysis_results which retrieve full results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., get_analysis_results). It does not specify that it should be used for polling or as a prerequisite before retrieving results.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_analysis_resultsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full disclosure burden. It describes a read operation ('retrieve') but does not disclose error behavior (e.g., if analysis_id is invalid), idempotency, or any side effects. The description adds minimal value beyond the tool name and schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and uses a clear docstring format with Args and Returns sections. Every sentence serves a purpose, but the Returns section is vague ('analysis results from Cyberbro API'). Still, it is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description should cover the return format, error cases, and lifecycle expectations. It only states 'the analysis results' generically. Combined with no annotations, the agent lacks context on when results are available or how to interpret them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It restates the parameter as 'analysis_id: Analysis ID to retrieve results for', which adds little meaning beyond the schema's 'title'. No constraints, formats, or examples are provided, leaving the parameter under-documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve the results of a previous analysis by analysis_id', specifying the verb, resource, and key parameter. This distinguishes it from siblings like analyze_observable (starts analysis) and is_analysis_complete (checks status), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after an analysis is started, but does not explicitly state when to use it versus alternatives such as is_analysis_complete or get_web_url. No conditions, prerequisites, or when-not-to-use information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_observableB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. It lacks details on error handling, rate limits, response time, or side effects. Simply states it triggers analysis and returns response.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Short and direct, with front-loaded purpose. Uses Args/Returns structure but still efficient. No redundant sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Missing key context: no output schema, no explanation of asynchronous behavior, no details on response format. With siblings like get_analysis_results, description should clarify that this returns initial response, not final results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning to parameters by explaining 'text' as observable(s) and 'engines' as list of engine names. Notes multiple observables separated by spaces, but does not specify engine source or format constraints beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it triggers analysis for observables (IP, domain, URL, hash, chrome extension id) using Cyberbro. It distinguishes from siblings like get_analysis_results and get_engines by indicating it initiates analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for initiating analysis, but no explicit when-to-use or alternatives compared to siblings. Mentions support for multiple observables, offering some guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_enginesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It only states that the tool lists engines, lacking details on side effects, caching, or any constraints. The return info is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short but contains redundancy (the returns line repeats the purpose). It could be more concise by merging the last sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description adequately covers the tool's purpose and usage order. It is sufficient for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has no parameters (0 params), so description compensation is unnecessary. Baseline for zero parameters is 4, and no additional param info is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists available Cyberbro engines and positions it as the first tool to call. It distinguishes from siblings like analyze_observable, but does not explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage hint (call this first), but does not specify when not to use it or offer alternatives. Usage context is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.