Server Quality Checklist

Tool Scores

• analyze_tcp_conversationB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It usefully states that findings are evidenced by frame numbers, but it does not explicitly confirm the operation is read-only, describe failure behavior, or mention any prerequisites beyond the two parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense, front-loaded line with no filler. Every phrase contributes substantive information about what the tool analyzes and how results are presented.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers key outputs and the single-conversation scope, but with no output schema and no annotations it leaves gaps: how to source stream_id, whether the file must already be loaded, and what error conditions exist. It is adequate for selection but not fully sufficient for invocation without relying on external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented file_path and stream_id parameters. It only hints that a single TCP conversation is involved, offering no detail on what stream_id represents, how to obtain it, or accepted file path formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's focus: RTT, retransmissions, zero-window events, and duplicate ACKs for a single TCP conversation. Though it lacks an explicit verb, the scope is specific and distinguishable from siblings like list_tcp_conversations or detect_security_anomalies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to choose this tool over alternatives, nor any mention of prerequisites such as first listing available TCP conversations or obtaining a stream_id. The intended use is only implied by the tool name and contents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_json_rpc_exchangeB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does explain the core transformation and outputs, but it omits important details such as whether the operation is read-only, what the output format is, and how failures like invalid stream ids are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single information-dense sentence with no filler. Each clause adds a distinct behavior, and the main action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without annotations, output schema, or parameter documentation, the description leaves critical operating context unclear: how to obtain stream_id, what the reassembled output looks like, and what edge cases apply. It communicates the core function but not enough for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have no schema descriptions and 0% coverage. The description never mentions file_path or stream_id, nor does it clarify how the stream_id identifies a specific TCP conversation. The agent must rely entirely on parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific operation: reassemble a TCP conversation payload, classify JSON-RPC messages, and pair requests with responses by id. This clearly distinguishes the tool from generic siblings like analyze_tcp_conversation or get_capture_summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to use this tool versus alternatives such as analyze_tcp_conversation. The intended context is only implied by the phrase 'a TCP conversation's payload,' leaving the agent to infer the right selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capture_summaryB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It does disclose the kind of output an agent can expect, which is useful, but it does not state side effects, error behavior, file type constraints, or whether the operation is purely read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with a colon-separated list of contents. Every word earns its place and there is no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter summary tool with no output schema, the description lists the key return components sufficiently. It lacks explicit path-format detail and usage context, but the core calling information is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. Referencing a PCAP file gives some meaning to file_path, but it does not explicitly describe the expected path format or association between the parameter and the file to analyze.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a clear resource (PCAP file) and a specific deliverable (duration, packet count, protocols, top talkers). It does not explicitly contrast itself with sibling tools, but the summary focus is distinct from TCP conversation, security anomaly, and JSON-RPC tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use or when-not-to-use guidance is provided. There is no mention of alternatives or conditions that would route the agent to siblings like list_tcp_conversations or detect_security_anomalies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_security_anomaliesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral transparency. It discloses the nature of the operation (scanning a PCAP for specific anomaly types) and the output ordering (ranked by severity). However, it does not mention side effects, whether the input file is modified, access requirements, performance implications, or error conditions. It adds some behavioral context but not a complete picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence with no fluff. It front-loads the action and resource, then enumerates the detection categories and ranking behavior. Every word contributes to understanding what the tool does, making it an efficiently structured description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple signature (one required parameter) and an output schema, which covers return-value details. The description adequately explains the tool's purpose and scope. However, it lacks explicit guidance on when to use the tool versus siblings and does not disclose whether the operation is read-only or has side effects. For a single-param scanner, the description is mostly complete but misses a few contextual details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema documentation coverage is 0%, so the description must compensate. The phrase 'Scan a PCAP' directly informs the agent that the file_path parameter should be a path to a PCAP capture file. This adds meaningful semantics beyond the bare string schema. Though it does not explicitly bind file_path to the PCAP or mention format constraints, the single parameter's purpose is clearly established.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Scan a PCAP'), the resource (PCAP file), and the specific detection targets (port scans, ARP spoofing, high-entropy DNS queries, cleartext credentials). The verb is specific and the resource is concrete. It distinguishes this tool from siblings like get_capture_summary or list_tcp_conversations by focusing on security anomaly detection rather than general capture metadata or conversation listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need to detect security anomalies in a PCAP, use this tool. It does not explicitly state when not to use it or mention alternative tools. The behavioral distinction from siblings is inferable from the description, but no direct guidance such as 'use for threat hunting' or 'not for general traffic analysis' is provided, so guidance is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tcp_conversationsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. 'Enumerate ... in a PCAP' implies a read-only listing operation and the metrics phrase explains what will be computed, but the description does not disclose ordering, side effects, file-handling constraints, or scaling limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence containing the verb, object, input context, and emitted data with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter listing tool with an output schema available, the description is largely complete: it specifies the input, the granularity of the listing, and the metrics returned. It only lacks explicit guidance about relationship to sibling tools and operational caveats.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the file_path parameter. It does so by placing the operation 'in a PCAP', clarifying that file_path should point to a capture file. For a single self-named parameter, this is adequate, though path formats or constraints are not detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Enumerate'), a precise resource ('TCP conversations (4-tuple) in a PCAP'), and clarifies the output (packet/byte counts and timing). This clearly separates it from siblings like analyze_tcp_conversation, which implies a deeper per-conversation analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: this tool is for producing an overview of all TCP conversations in a capture. However, it does not explicitly state when to prefer this over analyze_tcp_conversation or otherwise name alternatives and exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.