Server Quality Checklist

Tool Scores

• enrich_log_lineB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions enrichment with AbuseIPDB data but lacks details on rate limits, authentication requirements, error handling, or what the output looks like (e.g., structured data vs. raw text). This is inadequate for a tool that likely involves external API calls and data processing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part of the sentence contributes directly to explaining what the tool does, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving external API data enrichment) and lack of annotations and output schema, the description is insufficient. It doesn't cover behavioral aspects like rate limits or output format, leaving critical gaps for an AI agent to use it effectively in context with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond implying that 'log_line' contains IP addresses and 'threshold'/'max_age_days' relate to AbuseIPDB filtering. This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Extract and enrich'), the target resource ('IP addresses from a log line'), and the data source ('with AbuseIPDB data'). It distinguishes itself from sibling tools like 'check_ip' or 'bulk_check' by focusing on log line processing rather than direct IP checking or bulk operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_ip' for single IPs or 'bulk_check' for multiple IPs. It doesn't mention prerequisites, such as needing AbuseIPDB access, or exclusions, like whether it handles IPv6 addresses or specific log formats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_blacklistB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Retrieve' implies a read operation, it doesn't specify whether this is a real-time query or cached data, what format the results come in, whether there are rate limits, or any authentication requirements. The description is too minimal for a tool that presumably accesses external threat intelligence data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a simple retrieval tool and front-loads the essential information about what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that retrieves threat intelligence data with no annotations and no output schema, the description is insufficient. It doesn't explain what format the blacklist returns (e.g., list of IPs with metadata), whether results are paginated, or any behavioral characteristics. The agent would be left guessing about important operational aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage with clear documentation of both parameters. The description adds no additional parameter information beyond what's already in the schema. Since schema coverage is high, the baseline score of 3 is appropriate - the description doesn't add value but doesn't need to compensate for schema gaps either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve') and the specific resource ('AbuseIPDB blacklist of malicious IP addresses'), making the purpose immediately understandable. It distinguishes this tool from its siblings by focusing on retrieving a blacklist rather than checking individual IPs or performing bulk operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings like 'check_ip' or 'bulk_check'. It doesn't mention any prerequisites, alternatives, or contextual factors that would help an agent decide between this retrieval operation and other available tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bulk_checkB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the service (AbuseIPDB) and batch capability, but doesn't disclose rate limits, authentication requirements, what constitutes a 'check' (e.g., reputation scoring, blacklist lookup), error handling, or response format. For a tool with 3 parameters and no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Check multiple IP addresses in batch against AbuseIPDB'). Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., confidence scores, report details), error conditions, or behavioral constraints like rate limits. For a batch operation with external service integration, more context is needed for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific semantics beyond what's in the schema (e.g., it doesn't explain how threshold interacts with AbuseIPDB's scoring system). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('check multiple IP addresses in batch') and the target resource ('against AbuseIPDB'), distinguishing it from sibling tools like 'check_ip' (likely single IP) and 'check_block' (likely IP block). It explicitly mentions the batch capability which differentiates it from single-item alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for batch checking IPs against AbuseIPDB, but doesn't explicitly state when to use this versus alternatives like 'check_ip' or 'check_block'. It provides context about the service (AbuseIPDB) but lacks explicit guidance on use cases, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_blockB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the service (AbuseIPDB) but does not disclose behavioral traits such as rate limits, authentication requirements, error handling, or what the output looks like (e.g., reputation score, report details). This is a significant gap for a tool with external dependencies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of reputation checking with an external service, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral aspects (e.g., rate limits, auth), output format, and error handling, which are crucial for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters ('network' as CIDR and 'max_age_days' with constraints). The description adds no additional parameter semantics beyond what the schema provides, such as examples of valid CIDR formats or implications of the age filter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('check the reputation') and resource ('CIDR block') using a named service ('AbuseIPDB'). It distinguishes this tool from sibling tools like 'check_ip' (likely for individual IPs) and 'bulk_check' (likely for multiple checks).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reputation checking of CIDR blocks, but does not explicitly state when to use this tool versus alternatives like 'check_ip' for single IPs or 'bulk_check' for multiple blocks. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_ipA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the service (AbuseIPDB) but does not disclose behavioral traits such as rate limits, authentication needs, error handling, or what the output looks like. For a tool with no annotations, this leaves significant gaps in understanding its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary details. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It lacks information on behavioral aspects like rate limits, authentication, and output format. For a tool with 4 parameters and no structured output, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description does not add any meaning beyond what the schema provides, such as explaining parameter interactions or usage examples. Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check the reputation') and resource ('a single IP address'), specifying the service provider ('using AbuseIPDB'). It distinguishes from sibling tools like 'bulk_check' (multiple IPs) and 'check_block' (IP block) by emphasizing 'single IP address'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context for checking IP reputation via AbuseIPDB, but does not explicitly state when to use this tool versus alternatives like 'bulk_check' or 'check_block'. It provides clear purpose but lacks explicit exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.