Server Quality Checklist

Tool Scores

• ipinfo_generate_map_urlA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context: the structure of the returned MapResult (url, mapped_ip_count, skipped_ips, etc.), how skipped_ips are capped at 100, the meaning of truncated, and error handling with ToolError. This goes beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two paragraphs) and front-loaded with the main purpose. Each sentence adds value, explaining the return value and error behavior. It could be slightly more structured, but it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter, rich annotations, output schema present), the description is complete. It explains the return structure, handling of filtered IPs, and error reporting. No missing information is needed for proper use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for the single parameter (ips), with a clear description and examples. The tool description does not add additional meaning beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Build an interactive ipinfo.io map for a set of IPs.' This is a specific verb+resource that distinguishes it from sibling tools like ipinfo_lookup_ips and ipinfo_summarize_ips.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for mapping IPs but provides no explicit guidance on when to use this tool versus alternatives, nor does it mention when not to use it. No examples of use cases or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ipinfo_summarize_ipsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond readOnly/ idempotent annotations, it details shared processing path with sibling, filtering/failed IP handling, and error format. Beneficial context that annotations alone don't provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise paragraphs with front-loaded purpose. Every sentence adds value—no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 params and an output schema, description covers purpose, behavior, error handling, and relationships thoroughly. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions and examples. Description adds no extra per-param details, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it aggregates IP lookups into counts and percentages. Distinguishes from sibling ipinfo_lookup_ips by emphasizing summary output and large log-analysis use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says this is preferred for large log-analysis when underlying records are not needed, and clarifies percentage basis. Could be improved by stating when not to use it, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ipinfo_check_residential_proxyA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, openWorldHint. The description adds details on return structure (ResidentialProxyDetails with fields), and error conditions (auth_insufficient_scope vs auth_invalid). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: purpose, return fields, use cases, sibling distinction, auth requirements. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one parameter, output schema exists, rich annotations), the description is complete. It covers purpose, usage context, behavioral nuances, and authentication, leaving no gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'ip' parameter (includes description, format, examples). The description does not add parameter-specific details beyond the schema, but baseline 3 is appropriate for this high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Classify whether an IP is a known residential-proxy exit node.' This is a specific verb-resource combination and distinguishes it from the sibling 'ipinfo_lookup_ips' which provides general privacy flags.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use ('fraud, bot, and ad-fraud detection') and when-not-to-use ('distinct from the general privacy flags on ipinfo_lookup_ips'). Also mentions alternative tool for VPN/Tor/hosting detection. Authentication requirements and error distinctions are clearly specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ipinfo_lookup_ipsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses rich behavioral details: dedup/filtering of invalid IPs, output order, omission of failed IPs, error conditions (auth_insufficient_scope, api_error, timeout), default detail level, cap at 1000 IPs, and performance characteristics. Annotations only indicate readOnlyHint, idempotentHint, and openWorldHint, so the description adds substantial value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-structured, starting with the core purpose followed by bullet-like points on behavior, errors, limits, and alternatives. It is front-loaded with essential information. While it is long, every sentence adds value, and the complexity warrants the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (batch lookup, error handling, token tiers, output structure, limits, related tools), the description is comprehensive. It covers input handling, output format, error scenarios, performance, and alternatives. An output schema exists, but the description complements it well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions and examples for both parameters. The description adds context: the 'detail' parameter's impact on response size and block omission, and that 'ips' are filtered. While schema alone is adequate, the description clarifies behavior beyond schema, justifying a score above baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Geolocate one or more IPs and return ISP/ASN details.' It uses a specific verb and resource, and distinguishes from siblings by naming alternatives for different use cases (e.g., ipinfo_summarize_ips for larger batches, ipinfo_check_residential_proxy for proxy detection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use this tool (for batch IP lookup with per-IP details) and when not (for >1000 IPs, use ipinfo_summarize_ips or ipinfo_generate_map_url; for residential proxy, use ipinfo_check_residential_proxy). It also covers token tier issues and fallback strategies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ipinfo_lookup_my_ipA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already include readOnlyHint, idempotentHint, openWorldHint. Description adds critical context: on stdio transports the result is the server's outbound IP, not the end user's, and errors raise ToolError with JSON envelope, which annotations don't cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each providing essential information: purpose, transport caveat, and alternative usage. No redundant or superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-argument tool with output schema and comprehensive annotations, the description covers edge cases (stdio transport) and error format, making it fully sufficient for proper invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters; schema coverage is 100% (empty schema). Description confirms 'No arguments.' Baseline for 0 parameters is 4, as no additional meaning is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool geolocates the calling client's own IP, with no arguments. It distinguishes from sibling 'ipinfo_lookup_ips' for when a specific IP is available.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states no arguments needed, warns about stdio transport behavior (server's outbound IP vs end user's IP), and directs to alternative tool if caller has a specific IP. Also mentions error handling via ToolError with JSON envelope.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.