Server Quality Checklist

Tool Scores

• get_snapshotA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint), the description adds significant behavioral context: the response always includes 'days_from_requested', the URL can resolve to a neighboring form, and it does not return the page's contents. These are not in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively long but front-loaded with the main purpose. Every sentence adds value, covering purpose, parameter usage, key caveats, and non-behavior, making it appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters, an output schema, and strong annotations, the description is thorough: it explains the main purpose, parameter semantics, important behavioral nuances, and explicitly states what the tool does not do. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already has 100% parameter coverage, and the description adds extra meaning by explaining how to use 'at' (omit for most recent) and that the URL may resolve to a neighboring form. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds the Wayback Machine capture of a web page closest to a given date, using a specific verb and resource. It also differentiates from sibling tools by noting it links to the capture but does not return page contents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance on when to use it (closest capture to a date) and explains the 'at' parameter's behavior, but does not explicitly name alternative tools for different needs, such as searching inside content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_itemA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent/non-destructive annotations, the description reveals default behavior (basic section), the potentially large size of files list, the always-reported file_count and total_bytes, and the trade-off of full_metadata. This gives the agent a clear picture of output size and how to control it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences deliver all key information with no padding. The first sentence states the core purpose, the second explains default behavior, the third details the optional sections and filtering, and the fourth reveals an always-returned metric. Each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, output schema presence, and strong annotations, the description covers everything an agent needs: how to get the item, what sections are available, size implications, and default metrics. No critical gaps remain for selecting and invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 80% schema coverage, the baseline is 3, but the description adds meaningful semantics: it explains what 'basic' covers, that 'files' lists derivatives and can be filtered by format, and that full_metadata is rarely needed. It does not mention max_description_chars, but the schema already provides bounds; overall the description enriches the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Read one Internet Archive item by its identifier,' which clearly states a specific verb (read) and resource (item). It distinguishes from sibling search/list tools by targeting a single known item rather than searching, and it references the upstream search tools that produce the identifier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that identifiers come from search_items or search_inside, implicitly guiding when to use this tool (after searching). It also provides clear guidance on section selection, advising against full_metadata as 'large and rarely needed' and recommending format filtering for files, which helps choose the right parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_snapshotsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds extensive behavior beyond the read-only annotations: deduplication of repeated captures, interleaving of multiple site addresses, the difference between capture date and change date, and the row-based index count. It also discloses the performance characteristic ('tens of seconds') and pagination mechanics. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although lengthy, every sentence carries critical caveats. It is front-loaded with a clear one-sentence purpose followed by well-organized behavioral details. The density of information is high, and the structure supports easy scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity—pagination, deduplication, multi-address handling, and interpretation of capture dates—the description covers all essential caveats. An output schema exists, so return fields are not needed. It also sets performance expectations, making the description complete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for all three parameters, so the baseline is 3. The description adds extra meaning about cursor usage ('pass the next_cursor from the previous answer') and clarifies the pagination model ('rows rather than positions, so there is no page number and no arithmetic to do'), which goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List Wayback Machine captures of a web page, oldest first, with the dates they were taken.' It explicitly differentiates from the sibling get_snapshot: 'Answers how long a page has been archived and how often, which get_snapshot cannot.' This makes the purpose unmistakable and distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance by contrasting with get_snapshot and advises on the tool's slowness: 'This route is slow, tens of seconds on a heavily archived address.' It also explains how to paginate with the cursor, so the agent knows exactly when to use this tool and how to proceed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_insideA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint, the description discloses key behaviors: accent/case/punctuation folding, quoted phrase preservation, unquoted tokenization, the meaning of 'total' (document count, not occurrence count), lack of page numbers, and the inside_container nuance with matched_file. This is substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place. It opens with the core purpose, then logically covers query syntax, matching behavior, pagination, result semantics, and edge cases, ending with alternatives. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema already documents return values, the description covers essential edge cases (accent folding, quoted vs unquoted, container documents, page limitations) and provides enough context for an agent to invoke the tool correctly. It feels complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with thorough parameter descriptions. The description adds extra semantics for the query parameter (quoting, accent folding) and mentions paging behavior. It does not deeply explain limit or excerpt parameters, but schema already covers them, so the added value is meaningful without being fully compensatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches full text inside scanned documents, explicitly distinguishing it from metadata search. It names the resource (digitised books, newspapers, documents) and the specific capability (optical recognition text), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs siblings: 'Use search_items or search_books instead when looking for a work by its title, author or subject.' It also explains quoting, accent folding, and paging behavior, giving clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_itemsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/openWorld/idempotent annotations, the description discloses important quirks: matching covers titles/creators/descriptions, not scan contents; the year fields reflect a depositor-typed date with placeholder/fragment/era issues; and sorting on 'oldest'/'newest' uses declared dates. This is substantive behavioral context that the annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than the ideal but remains front-loaded with the core purpose in the first sentence. Every sentence earns its place, covering matching semantics, date quirks, and alternatives. It is verbose in places (e.g., the date fragment sentence) but stays focused and logically organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and strong annotations, the description only needed to explain behavior, and it does so thoroughly. It covers matching, date-side-effects, media type guidance, sort semantics, and how to proceed to get_item. No critical context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 71%, but the description adds meaning beyond the schema: it explains that year_from/year_to operate on the declared date with no era (so 1712 BCE matches 1700–1750), clarifies sort behavior, and explains the rationale for media_type. Although page and limit are not elaborated, they are self-evident. This compensates well for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Search the Internet Archive catalogue: films, books, recordings, images, software and datasets.' It clearly states the scope and distinguishes from siblings by naming search_inside and get_item. This fully clarifies what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs when to use alternatives: 'use search_inside for a phrase within a book' and mentions that get_item takes the identifier. It also strongly recommends setting media_type. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_booksA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint, openWorldHint, and idempotentHint. The description adds meaningful caveats beyond annotations: free-text matching reads titles and authors together, first_published_year is derived from edition records and can be inaccurate, and archive_identifiers lists up to 3 scans. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but front-loaded with the main purpose and every sentence carries non-redundant value, such as the warning to read authors before treating a result and the explanation of scan vs. edition. For a 13-parameter tool, this density is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich input schema, output schema, and annotations, the description fully covers the two search modes, sorting rationale, data-quality caveats, scan/archive handling, and sibling tool connections. It provides sufficient context for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema coverage is high (85%), the description adds important semantics: it distinguishes query from criteria, explains that criteria combine, clarifies subject can carry prizes/lists, and describes how sort options like newest/oldest rank on a potentially unreliable field. This goes well beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Find a book on Open Library, the Internet Archive's catalogue of works.' It clearly differentiates from siblings by explaining that search_inside finds a phrase within a work and that this tool identifies a work, while comparing it to the item catalogue.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use 'query' vs. 'criteria' ('Pass query when you know what you are looking for... Pass the criteria instead when you do not'), explains when to use sort by rating/readers, and names alternates: use search_inside for phrases and pass archive_identifiers to get_item. This gives clear decision guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.