Server Quality Checklist

Tool Scores

• search_itemsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool 'supports basic query, optional mediatype/collection filters, fields, sort, rows, and page,' which hints at search capabilities but lacks details on permissions, rate limits, pagination behavior, or response format. For a search tool with 7 parameters and no annotation coverage, this leaves significant gaps in understanding how the tool behaves beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the main purpose ('Search Internet Archive items using Advanced Search') and follows with key features. It avoids redundancy and wastes no words, making it appropriately sized for the tool's complexity. A point is deducted as it could be slightly more structured, but overall it's concise and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters, no annotations, and an output schema exists (which reduces the need to explain return values), the description is minimally adequate. It covers the basic purpose and hints at parameters, but lacks details on usage guidelines, behavioral traits, and parameter semantics. For a search tool with moderate complexity, this leaves room for improvement, scoring at the minimum viable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It lists parameters like 'basic query, optional mediatype/collection filters, fields, sort, rows, and page,' which adds some meaning beyond the schema's property names, but doesn't explain what each parameter does, acceptable values, or how they interact. With 7 parameters and low coverage, this partial compensation is insufficient, scoring below the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Internet Archive items using Advanced Search, specifying the target resource and verb. It distinguishes from siblings like get_archived_page and get_snapshots by focusing on search functionality rather than retrieving specific pages or snapshots. However, it doesn't explicitly contrast with siblings, keeping it at 4 instead of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'supports basic query, optional mediatype/collection filters, fields, sort, rows, and page,' which implies usage for searching with various filters, but provides no explicit guidance on when to use this tool versus alternatives like get_archived_page or get_snapshots. There's no mention of prerequisites, exclusions, or comparative contexts, resulting in minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_archived_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It hints at a retrieval operation ('Retrieve content') but lacks critical details: authentication requirements, rate limits, error handling, or what 'id_ mode' entails. The mention of timestamp format is useful but insufficient for a tool interacting with an external service like Wayback Machine.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the main purpose. Both sentences are relevant: the first states the action and key parameter detail, the second adds a conditional parameter behavior. There's no wasted text, though the ambiguous 'id_ mode' could be clarified for better efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 0% schema coverage, no annotations, and an output schema (which reduces need to describe returns), the description is moderately complete. It covers timestamp format and a parameter condition but misses explanations for 'url' and 'id_ mode', and lacks behavioral context like errors or limits. This is adequate but has clear gaps for a tool with external dependencies.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter documentation. The description adds some semantics: it explains the timestamp format (YYYYMMDDHHMMSS) and clarifies that 'original' parameter triggers 'id_ mode'. However, it doesn't explain the 'url' parameter or what 'id_ mode' means, leaving gaps for 3 parameters. This partial compensation justifies a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve content') and resource ('archived webpage from the Wayback Machine'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from its siblings (get_snapshots, search_items), which would require a 5. The mention of 'using YYYYMMDDHHMMSS timestamp' adds specificity but not sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_snapshots or search_items. It mentions a conditional ('If original=true, request id_ mode') but this is a parameter usage note, not a contextual guideline for tool selection. Without any when-to-use or when-not-to-use information, the score is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_snapshotsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool retrieves a 'list of available snapshots', implying a read-only operation, but doesn't cover important aspects like rate limits, authentication needs, pagination behavior, error handling, or what 'available' means in practice. The description adds minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and follows with essential parameter details. There's no wasted text, and it's appropriately sized for a tool with multiple parameters. However, it could be slightly more structured by separating usage guidance from parameter semantics.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no annotations, but with an output schema), the description is minimally adequate. It covers the basic action and some parameter details, but lacks usage guidelines, behavioral context, and full parameter explanations. The presence of an output schema reduces the need to describe return values, but overall completeness is limited.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds some semantic value by explaining date formats ('YYYYMMDD') and match_type options, which aren't covered in the schema (0% description coverage). However, it doesn't explain the purpose of 'from_', 'to', or 'limit' parameters, or how they interact with the URL and match_type. With 5 parameters and low schema coverage, the description partially compensates but leaves significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('list of available Wayback Machine snapshots for a URL'), making the purpose specific and understandable. However, it doesn't explicitly distinguish this tool from sibling tools like 'get_archived_page' or 'search_items', which likely have related but different functions in the Wayback Machine context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_archived_page' or 'search_items'. It mentions date formats and match_type values, but these are parameter details rather than usage context. There's no indication of prerequisites, constraints, or typical scenarios for selecting this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.