Server Quality Checklist

Tool Scores

• analyze_dhcp_packetsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It transparently warns about the critical limitation regarding web-uploaded files and supports that with a detailed list of unsupported formats. However, the statement 'Then write data to user's system' is vague and unexplained, leaving uncertainty about a potentially impactful side effect (e.g., file system writes) without further detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headings and each section serves a purpose, which aids readability. However, the 'Then write data to user's system' sentence appears out of place and confusing, and the format lists could be slightly tightened. Overall, it is appropriately sized for the information conveyed, with minor unnecessary elements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description covers the essential elements: the parameter's meaning, supported input formats, critical limitations, and the return type ('structured dictionary'). The only notable gap is the ambiguous 'write data' phrase, which if clarified would make the description fully complete. The output schema is present, so detailed return-value documentation is not required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only a bare string type with no description (0% schema coverage). The description fully compensates by explaining that pcap_file accepts an HTTP URL or absolute local file path, enumerating supported and unsupported input formats, and clarifying what the parameter represents. This gives the agent all necessary semantic information to populate the parameter correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Analyze DHCP packets from a PCAP file and return comprehensive analysis results,' which clearly identifies the tool's action and resource, and distinguishes it from sibling analyzers via the DHCP focus. However, the appended 'Then write data to user's system' introduces ambiguity about a secondary purpose or side effect, slightly undermining the clarity of the core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit input format guidance, listing supported formats (URL, absolute local path) and unsupported ones (web-uploaded files, base64, relative paths). This gives clear context on when the tool can be used, but it does not reference sibling tools or explicitly state when to prefer DHCP analysis over DNS/ICMP/capinfos, leaving the choice of tool partly implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_icmp_packetsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavioral traits. It goes beyond trivial by explaining the file upload limitation (web upload unsupported), specifying supported input formats, and clarifying the return type (structured dictionary). It does not mention side effects, but as an analysis tool, the main non-obvious behavior (file accessibility) is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (warning, supported formats, unsupported, args, returns) and front-loads the main purpose. While the Args section somewhat repeats the format details, the overall length is justified and each sentence adds value. The use of emojis and bullet points improves readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only one parameter and an output schema (not shown but noted), the description provides sufficient context: the purpose, input format restrictions, and return type. It does not detail the internal analysis contents, but the output schema is expected to cover that. The file access limitation is a critical contextual detail that is addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only declares pcap_file as a string with no description (coverage 0%). The description compensates by defining exactly what the parameter expects: an HTTP URL or absolute local file path, with examples. This gives the agent the necessary information to format the argument correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Analyze ICMP packets'), the resource ('from a PCAP file'), and the outcome ('return comprehensive analysis results'). It distinguishes itself from sibling tools like analyze_dns_packets and analyze_dhcp_packets by explicitly focusing on ICMP traffic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear setup guidance about file access (URL or local path) and unsupported upload methods, but it does not explicitly state when to use this tool over its siblings or when not to use it. Usage is implied through the name and protocol focus, but no alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_capinfosA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly explains the input requirement (path/URL, not content) and the return structure ('structured dictionary containing PCAP metadata'). It does not mention error handling or side effects, but for a read-only metadata tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an important note, args, and returns sections. Each sentence adds value: the main purpose, the input format warning, examples, and the output details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the presence of an output schema, the description is complete. It covers input format, expected output, and even example paths. There are no gaps that would prevent correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain the parameter. It does: 'pcap_file: Path to local PCAP file or HTTP URL to remote PCAP file (NOT file content - must be a path or URL)' and even gives example formats. This fully compensates for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Return metadata from a PCAP file, similar to Wireshark's capinfos utility.' This is a specific verb and resource, clearly distinguishing it from sibling tools like analyze_dns_packets which analyze packet types rather than file metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: 'This tool expects a FILE PATH or URL, not file content' and 'File uploads are NOT supported - save the file locally first.' It does not explicitly mention alternatives, but the purpose is clear enough to infer when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_dns_packetsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so thoroughly. It warns about the file upload limitation, enumerates supported and unsupported input formats, and specifies the parameter type. It also discloses the return type ('structured dictionary'). This goes beyond a typical description, giving the agent essential behavioral knowledge.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: intro, warning, supported formats, unsupported formats, args, returns. Every sentence adds necessary information; the warning is important and the bullet lists are scannable. It is longer than average but each part earns its place, and the key limitation is front-loaded with the emoji alert.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter, no annotations, and an existing output schema, the description covers all necessary context: what it does, how to pass the file, what limitations exist, and what to expect in return. It explains the return value even though an output schema exists, and provides enough setup guidance to use the tool correctly. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only shows 'pcap_file' as a generic string with no description. The tool description compensates fully by defining it as an HTTP URL or absolute local file path, and by listing supported and unsupported formats. This provides crucial semantic meaning that the schema lacks entirely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Analyze DNS packets from a PCAP file'. This clearly distinguishes it from sibling tools like DHCP and ICMP analyzers. It also states the output type ('comprehensive analysis results'), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use context ('Analyze DNS packets from a PCAP file') and clear when-not-to-use guidance via the unsupported inputs (web-uploaded files, base64, relative paths). However, it does not explicitly name alternative tools for other packet types, so it lacks the full 'alternatives' component for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.