Server Quality Checklist

Tool Scores

• get_unpacked_filesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (extracts and classifies memory artifacts), how it handles file storage (with fallback directory logic), and what it returns (structured dictionary with paths and metadata). It provides useful context about snapshot types and classification criteria, though it doesn't mention error conditions or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, details, parameters, returns) and front-loaded key information. Most sentences earn their place by providing necessary context, though some formatting (like the bulleted snapshot types) could be more concise. The overall length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations, no output schema), the description provides comprehensive coverage. It explains what the tool does, how it behaves, what parameters mean, what it returns, and includes important behavioral details like fallback directory logic. The main gap is the lack of explicit error handling or performance information, but overall it's quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must fully compensate. It provides clear semantic explanations for all three parameters: webid ('submission ID of the analysis'), run ('index of the sandbox run'), and save_path ('optional base directory to store unpacked files'). It also explains default values and fallback behavior for save_path, adding significant value beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve and classify') and resource ('in-memory unpacked binaries from a sandbox analysis'). It distinguishes from sibling tools like get_dropped_files by focusing specifically on memory-resident unpacked executables rather than files written to disk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through phrases like 'from a sandbox analysis' and 'during the dynamic execution of the submitted sample,' suggesting this tool should be used after analysis execution. However, it doesn't explicitly state when to use this tool versus alternatives like get_memory_dumps or get_dropped_files, nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_domain_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behaviors: it extracts domains gathered by the sandbox engine, returns empty arrays when no domains were gathered, explains the filtering logic for malicious elements and indicators, and describes the return structure. However, it doesn't mention rate limits, authentication requirements, or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (description, args, returns, notes) but somewhat verbose in the parameter explanations. The 'Returns' section could be more concise, and some sentences repeat information. Overall efficient but with minor redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description provides excellent completeness: explains purpose, parameters, return structure with nested objects, filtering behavior, edge cases (empty arrays), and includes helpful notes about detection subsystems and impact levels. Covers all essential aspects for a domain retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantics for all 4 parameters: explains webid as submission ID, run as sandbox run index, and gives comprehensive explanations of the two boolean filters including their default values and the practical implications of True/False settings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves domains from a completed analysis with their detection indicators. It specifies the resource (domains in analysis), the verb (retrieve/extract), and distinguishes from siblings by focusing specifically on domain information rather than files, IPs, URLs, or other analysis components.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (completed analysis with sandbox engine data) but doesn't explicitly state when to use this tool versus alternatives like get_ip_info or get_url_info. It mentions filtering capabilities but doesn't provide guidance on tool selection among the sibling analysis tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dropped_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it specifies the return format in detail (dictionary structure), notes that empty arrays are returned if no dropped files were gathered, explains the filtering logic for malicious elements and indicators, and describes the impact levels of indicators. However, it doesn't mention potential errors, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns, Notes) and front-loaded purpose statement. While comprehensive, some sentences could be more concise (e.g., the repeated explanations of 'malicious' filtering). Overall, most content earns its place by adding necessary context beyond structured fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, 0% schema coverage, no annotations, and no output schema, the description provides exceptional completeness: it fully documents all parameters, explains the return structure in detail, includes important notes about empty returns, and gives operational context. No significant gaps remain given the complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description fully compensates by providing comprehensive parameter semantics: it explains all 4 parameters (webid, run, only_malicious_elements, only_malicious_indicators) with clear definitions, default values, and operational implications. The detailed explanations of the boolean parameters' effects on filtering are particularly valuable beyond basic schema information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Retrieve metadata for files dropped in a completed analysis') and distinguishes it from sibling tools like 'get_dropped_files' by specifying it returns metadata and detection indicators rather than the files themselves. It explicitly mentions the resource ('dropped files') and context ('for a specific sandbox run of an analysis').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it's for 'a completed analysis' and 'a specific sandbox run,' but doesn't explicitly state when to use this tool versus alternatives like 'get_dropped_files' or 'get_analysis_info.' It provides some operational guidance (e.g., 'Use 0 for the first run') but lacks explicit comparisons or exclusions for sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ai_summariesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and effectively discloses key behavioral traits: it explains caching behavior ('Reports are cached in memory'), clarifies the purpose of the run parameter for distinguishing environments, and describes the return structure. It does not cover aspects like error handling or rate limits, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns, notes), front-loading the core purpose. It is appropriately sized, though the notes section could be slightly more concise, but every sentence adds valuable information without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (2 parameters, no annotations, no output schema), the description is largely complete: it covers purpose, parameters, return values, and behavioral notes. It could improve by mentioning error cases or authentication needs, but it provides sufficient context for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate, and it does so comprehensively. It explains both parameters: webid as 'The submission ID of the analysis (unique identifier)' and run as 'The index of the analysis run to retrieve the report for', including default values and usage examples, adding full meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve the AI summaries for a specific analysis run') and resource ('analysis run'), distinguishing it from siblings like get_analysis_info or get_list_of_recent_analyses by focusing on AI summaries rather than general analysis data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for a specific analysis run'), including optional parameter usage and default behavior. However, it does not explicitly state when not to use it or name alternatives among siblings, such as get_analysis_info for broader metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dropped_filesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behaviors: it downloads and extracts files 'as-is without renaming or classification', details output path logic with fallback rules, and describes the return structure. It doesn't mention permissions, rate limits, or error handling, but covers core operational traits adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by behavioral details, output logic, and parameter/return explanations in a logical flow. Every sentence adds value—no fluff or repetition—making it efficiently structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description is largely complete: it covers purpose, behavior, parameter meanings, and return format. It lacks details on error cases or authentication needs, but given the context, it provides sufficient guidance for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context for all three parameters: webid as 'Joe Sandbox analysis ID', run as 'Run index (default: 0)', and save_path as 'Optional base path to save dumps'. This clarifies purpose beyond schema titles, though it could detail format constraints (e.g., webid structure).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('download all dropped files'), the resource ('from a Joe Sandbox analysis'), and distinguishes it from siblings like get_dropped_info (which likely provides metadata) and get_unpacked_files (which handles different file types). The verb+resource combination is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'for further inspection' and distinguishes from metadata tools like get_dropped_info through its focus on file extraction. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings, missing full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_analysis_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a read-only retrieval operation (implied by 'Retrieve'), handles invalid/expired IDs with error returns, and explains the structure and utility of the 'runs' array. It lacks details on rate limits or authentication needs, but covers the core operational behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement, usage guidance, parameter explanation, and detailed return value documentation. It is appropriately sized for the tool's complexity, though the extensive list of return fields could be slightly condensed without losing clarity. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter, no output schema, no annotations), the description is highly complete. It covers purpose, usage, parameter meaning, return structure with examples, and edge cases (invalid IDs). For a retrieval tool, this provides all necessary context for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must fully compensate. It clearly explains the single parameter 'webid' as 'The submission ID (also called webid) returned when the sample was uploaded,' adding essential context beyond the schema's basic type definition. This is comprehensive for the one parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve metadata and status') and resource ('previously submitted analysis by its submission ID'), distinguishing it from siblings like 'submit_analysis_job' (which creates analyses) and 'get_list_of_recent_analyses' (which lists multiple analyses). The purpose is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Use this tool to check whether an analysis is finished...'), providing clear context for its application. However, it does not mention when not to use it or name specific alternatives among the sibling tools (e.g., 'search_analysis' might overlap in some scenarios), which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ip_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a read-only retrieval tool (implied by 'Retrieve', 'extracts'), mentions that it returns data from a 'completed analysis', specifies that an empty array is returned if no IPs were gathered, and details the structure and semantics of the return data including severity classifications. However, it lacks information on error conditions, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns, notes) and front-loaded key information. While comprehensive, some sentences could be more concise (e.g., the explanation of 'only_malicious_indicators' uses multiple sentences where one might suffice). Overall, it efficiently conveys necessary information without significant waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no annotations, no output schema), the description provides excellent completeness. It covers the purpose, all parameter semantics, detailed return structure with nested object explanations, and important behavioral notes. The description fully compensates for the lack of structured metadata, making the tool's functionality clear to an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage (titles only provide parameter names), the description fully compensates by providing detailed semantic explanations for all four parameters. It clearly explains the purpose of 'webid' as the submission ID, 'run' as the sandbox run index, and provides nuanced explanations for the two boolean filters including their default behaviors and the implications of setting them to True or False.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve', 'extracts') and resource ('IP addresses in a completed analysis', 'IP addresses gathered by the sandbox engine'), distinguishing it from sibling tools like get_domain_info or get_url_info by focusing exclusively on IP addresses. It precisely defines what the tool does without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for retrieving IP addresses from a completed sandbox analysis. It mentions optional filtering parameters for controlling output based on severity, but does not explicitly state when to use this tool versus alternatives like get_analysis_info or search_analysis, nor does it provide exclusion criteria or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_list_of_recent_analysesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well: it discloses the default behavior (returns latest 20 analyses), return format (list of dictionaries with specific fields), and purpose (summary for follow-up actions). It doesn't mention rate limits, authentication needs, or pagination behavior, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured and appropriately sized: front-loaded with core purpose, followed by behavioral details, parameter explanation, and return format. Every sentence adds value—no redundancy or fluff. The bulleted list of return fields is efficient for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and 1 parameter with 0% schema coverage, the description is quite complete: it covers purpose, behavior, parameter semantics, and return structure. However, it lacks details on error handling, authentication, or rate limits, which could be relevant given the sibling tools suggest a security analysis context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage (only title 'Limit'), so the description must compensate fully. It clearly explains the 'limit' parameter's purpose ('override to retrieve more or fewer entries'), default value (20), and effect ('number of most recent analyses to return'), adding significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('List') and resource ('recent analyses submitted by the user'), distinguishing it from siblings like 'get_analysis_info' (detailed single analysis) or 'search_analysis' (filtered search). It explicitly mentions 'sandbox analyses' and 'current account' context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('summary of the most recent sandbox analyses') and implies usage for follow-up actions like downloading artifacts. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, though the distinction is reasonably inferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_memory_dumpsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it describes the extraction process ('extracts all contents into a local directory'), file handling ('extracted as-is without renaming or classification'), and output path logic with fallback behavior. It doesn't mention permissions, rate limits, or error handling, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with purpose first, then behavioral details, output logic, and parameter explanations. Every sentence adds value with zero waste, and it's appropriately sized for a tool with 3 parameters and complex behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description provides substantial context including purpose, behavior, parameter semantics, and return value structure. It lacks details about authentication requirements, error conditions, or rate limits, but covers most operational aspects well given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all 3 parameters: webid ('Joe Sandbox analysis ID'), run ('Run index with default 0'), and save_path ('Optional base path to save dumps'). It adds meaningful context beyond basic types, including default values and optional status.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('download and extract memory dumps') and resource ('from a Joe Sandbox analysis'), distinguishing it from siblings like get_analysis_info or get_dropped_files which handle different data types. It precisely defines what the tool does with memory snapshots.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for retrieving memory dumps from Joe Sandbox analyses) but doesn't explicitly mention when not to use it or name specific alternatives among the sibling tools. The context is well-defined but lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pcap_fileA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: the tool downloads and saves a file locally, specifies the naming convention (`{webid}-{run}.pcap`), explains fallback directory logic if save_path is invalid, and outlines the return structure. It covers mutation (file creation) and error handling, though it doesn't mention permissions, rate limits, or file size considerations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core purpose, followed by elaboration on the PCAP content, file handling details, and clear parameter/return sections. Every sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, file download operation) and lack of annotations or output schema, the description does an excellent job covering purpose, behavior, parameters, and returns. It explains the local file save process and fallback logic. A slight gap exists in not detailing potential errors (e.g., invalid webid) or performance aspects, but it's largely complete for practical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It provides detailed semantics for all three parameters: webid as the required submission ID, run as the optional sandbox run index with default value, and save_path as the optional custom directory with fallback behavior. This adds substantial meaning beyond the bare schema, clarifying usage and consequences.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve', 'downloads') and resource ('network traffic capture (PCAP) file from a sandbox analysis'), distinguishing it from sibling tools like get_analysis_info or get_memory_dumps which retrieve different analysis artifacts. It explicitly mentions the content of the PCAP file (DNS requests, HTTP traffic, TCP/UDP communications), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to retrieve the full packet capture from a sandbox analysis run. It implicitly suggests alternatives by mentioning the PCAP contains specific traffic types, but does not explicitly name when not to use it or which sibling tools might be better for other data (e.g., get_domain_info for domain-specific insights). The guidance is sufficient but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_url_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing behavioral traits. It explains what data gets returned, truncation behavior for long URLs, empty array returns when no URLs exist, and the filtering logic based on severity assessment. The only minor gap is lack of explicit mention about authentication requirements or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns, Notes) and front-loads the core purpose. While comprehensive, some sentences could be more concise (e.g., the explanation of 'only_malicious_indicators' uses multiple sentences that could be streamlined). Overall, most content earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, no output schema), the description provides complete context. It explains the purpose, parameters, return structure with detailed field descriptions, and behavioral notes. The return value documentation effectively substitutes for a missing output schema by detailing the dictionary structure and all nested fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic explanations for all 4 parameters. It clarifies that 'webid' is a required submission ID, 'run' is an index from the runs array, and both boolean parameters control filtering based on malicious classification with clear explanations of what 'True' and 'False' values mean.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve', 'extracts') and resources ('urls in a completed analysis', 'associated detection indicators'). It distinguishes from siblings by focusing specifically on URL extraction from sandbox analysis, unlike tools like get_domain_info or get_ip_info that handle different data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool - specifically for retrieving URL information from completed sandbox analyses. However, it doesn't explicitly state when NOT to use it or name alternative tools for related but different purposes (e.g., using get_analysis_info for general analysis metadata).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_analysis_jobA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively explains key behaviors: the tool can block/wait for completion (wait_for_analysis_end), has a timeout mechanism, requires exactly one input type, and includes important configuration details like internet_access requirements. It also describes the return structure, which is crucial given the lack of output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and bullet points, making it easy to parse. While comprehensive, it's appropriately sized for a complex tool with 21 parameters. Some sentences could be slightly more concise, but overall it's efficient and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 21 parameters, no annotations, and no output schema, the description is remarkably complete. It covers the purpose, usage constraints, behavioral characteristics, parameter semantics, and return structure. The detailed explanation of the return dictionary is particularly valuable given the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage for 21 parameters, the description compensates excellently. It organizes parameters into logical groups (wait/timeout, file upload options, sandbox configuration), explains the 'exactly one' constraint for input types, provides default values, and clarifies the purpose of key parameters like internet_access and archive_password. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Submit a file, URL, website, or command line for sandbox analysis using Joe Sandbox.' It specifies the exact action (submit for analysis) and the resources involved (four input types), distinguishing it from sibling tools that retrieve analysis results rather than initiate them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: for submitting items to Joe Sandbox. It includes specific guidance like 'Only one input type must be provided' and 'For URL and website analysis, make sure internet_access=True.' However, it doesn't explicitly mention when NOT to use it or name alternatives among siblings, though the sibling names suggest this is the primary submission tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_signature_infoA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by explaining what the tool returns (a dictionary with specific structure), the nature of the data (malware-related signatures with high/low impact indicators), and the filtering behavior. It doesn't mention rate limits, authentication needs, or error conditions, but provides substantial behavioral context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns) and front-loaded information. Every sentence earns its place by providing essential context about what the tool does, how to use parameters, and what to expect in return, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of malware analysis tools and the absence of both annotations and output schema, the description provides complete context. It explains the tool's purpose, parameter usage, return structure with detailed field descriptions, and the significance of high/low impact indicators, making it fully self-contained for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description provides excellent parameter semantics in the 'Args' section, explaining all three parameters with clear purpose, defaults, and usage guidance. It adds significant value beyond the bare schema, fully compensating for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieve', 'extracts') and resources ('high-impact signature detections', 'sandbox analysis report'). It distinguishes itself from siblings like get_analysis_info or get_process_info by focusing specifically on detection signatures rather than general analysis data or other report components.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (to extract detection signatures from analysis reports) and mentions optional filtering parameters. However, it doesn't explicitly state when NOT to use it or name specific alternative tools for different types of analysis data, though the sibling list suggests alternatives like get_analysis_info for general information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_analysisA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and delivers excellent behavioral transparency. It discloses critical operational details: case-insensitive searches, AND logic for multiple parameters, time zone considerations (CET/CEST), exclusive date handling, and explicit warnings about unsupported parameters ('q' parameter not supported).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Notes, Examples) and every sentence adds value. It could be slightly more front-loaded by moving the 'must provide at least one parameter' requirement earlier, but overall it's efficiently organized with no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 15-parameter search tool with no annotations and no output schema, the description provides excellent coverage of input behavior, constraints, and examples. The main gap is lack of information about return values (format, pagination, etc.), but given the tool's primary focus is search filtering rather than output structure, this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage and 15 parameters, the description compensates fully by explaining each parameter's matching behavior (exact vs substring match), providing format requirements (ISO 8601 for dates), enumerating detection values, and giving concrete examples. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search the JoeSandbox Cloud for malware analyses using structured search parameters.' It specifies the exact resource ('malware analyses'), the platform ('JoeSandbox Cloud'), and the method ('structured search parameters'), distinguishing it from sibling tools like 'get_list_of_recent_analyses' which likely returns unfiltered recent analyses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context with the note 'You must provide at least one of the supported parameters' and explains AND logic for multiple parameters. However, it doesn't explicitly state when to use this tool versus alternatives like 'get_list_of_recent_analyses' or other sibling tools, though the structured search focus is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_process_infoA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so comprehensively. It describes the tool's behavior (traversing execution trees, returning structured hierarchies), discloses that results can be large and deeply nested, provides formatting suggestions for readability, and explains error handling (returns error dictionary on failure).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, node details, formatting suggestions, args, returns) and appropriately sized. While comprehensive, some formatting examples could be slightly condensed, but every sentence adds value and the information is front-loaded with the core purpose first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (process tree extraction with hierarchical data), no annotations, and no output schema, the description provides complete context. It explains what the tool does, detailed node structure, parameter meanings, return format, error handling, and even provides formatting recommendations for the complex nested output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description fully compensates by providing detailed parameter semantics. It explains that 'webid' is the submission ID of the analysis and 'run' is the index of the sandbox run to inspect, including that run defaults to 0 and comes from the 'runs' array in analysis info. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts and returns the full process tree from a Joe Sandbox report, specifying it traverses the execution tree and returns a structured process hierarchy. It distinguishes from siblings by focusing specifically on process tree extraction rather than summaries, analysis info, or other report components.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for extracting process trees from analysis runs) and mentions the result can be large and deeply nested. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools for different data needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.