Server Quality Checklist

Tool Scores

• get_from_archiveB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns extracted article text, which is a useful behavioral detail. However, it omits other contextual traits such as rate limits, failure modes, or what happens when a source is unavailable, so it only partially meets the transparency bar.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded, with the main action in the first sentence and a compact list of allowed sources. It avoids fluff, though the formatting of the list with line breaks could be cleaner.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, parameter constraints, and return value, which is acceptable for a simple 2-parameter tool with an output schema. However, it lacks usage context relative to sibling tools and any error/edge-case behavior, making it only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds the list of valid values for 'source', which is essential given the schema has no enums. It also implies 'url' is the original URL to fetch. However, it provides no further context about URL format or what each source actually represents, so the added meaning is limited.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetch'), resource ('an archived version of a URL'), and constraint ('from a specific source'). It distinguishes from siblings like 'search_archives' by implying a direct fetch rather than a search, though it does not explicitly compare to alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives like 'search_archives' or 'remove_paywall'. The description lists allowed sources but does not explain the selection context or any exclusions, leaving the choice of tool ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_domainA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses a meaningful behavioral consequence: the has_paywall flag determines whether archive search is attempted first or skipped. Still, it does not mention outcomes like duplicate-domain handling, idempotency, or permissions, leaving notable gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: one for purpose, two for parameter behavior. No filler or repetition; front-loaded and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose and the key behavioral switch, and an output schema exists to document returns. It could be more complete regarding duplicate/format expectations, but it is adequate for this relatively simple CRUD tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It explains has_paywall semantics in detail and implies domain through the tool name/purpose, but notes is left undefined. This compensates for the two required parameters but not the optional one.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Register a domain in the paywall knowledge base,' using a specific verb and resource. This clearly distinguishes it from sibling tools like remove_paywall and search_archives. No ambiguity in intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear conditional guidance: set has_paywall=true for paywalled sites (archives tried first) and false for non-paywalled sites (archive search skipped). However, it doesn't explicitly state when to choose add_domain over sibling tools like remove_paywall, so it lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• domain_infoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly implies a read-only lookup ('Look up stored knowledge') and details the information returned. Doesn't cover missing-domain behavior, but the operation's nature is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences front-load the purpose, list the return values, and point to an alternative. Every sentence earns its place with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool with an output schema, the description sufficiently covers functionality and returns. The cross-reference to add_domain adds context. Minor edge cases are not essential here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implies the 'domain' parameter is the target domain, but gives no format or example (e.g., bare domain vs full URL), leaving ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description starts with a specific verb+resource: 'Look up stored knowledge about a paywall domain.' It enumerates the returned data (status, notes, success rates, best archive order), which clearly distinguishes it from sibling tools like search_archives or add_domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: it's for looking up stored metadata about a domain. Explicitly names add_domain as the alternative for registering new domains. Doesn't exclude search_archives or get_from_archive, but the scope is self-evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_paywallA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It transparently lists the sequence of attempts (Googlebot fetch, 12ft.io, archive services) and the prioritization logic, but it doesn't disclose potential failure modes, rate limits, or that some sources may be unreliable. It describes what it does but not edge-case behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet rich: first sentence states purpose, middle details the process in an ordered list-like manner, last states output. No redundancy, every sentence adds information, and the most important info (purpose) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (multiple fallbacks, parallel requests), and the description covers the methodology well while the output schema handles return details. Missing a note on failure behavior or expected latency, but given the output schema and clear process, it's nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It does: the first sentence specifies 'article URL', giving the single `url` parameter clear semantic meaning. It doesn't dictate format, but for a single-string parameter this is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Remove') and resource ('paywall') and immediately distinguishes itself from sibling tools like search_archives or get_from_archive by stating the goal is paywall removal. It further clarifies it works through internet archives, making the tool's scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool: when a paywalled article URL needs full content extraction. It details the multi-step fallback strategy, but does not explicitly compare with sibling tools or state exclusions (e.g., when not to use), so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_archivesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output (list of snapshot URLs), enumerates six archive sources, and explicitly states what the tool does not do (extract content). This gives a clear behavioral picture for a simple search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the main action, then output description and a pointer to an alternative. Every sentence earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter tool and an existing output schema, the description covers the tool's purpose, return type, and boundaries. The list of archive sources adds useful context beyond what structured fields would provide.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description implies the URL parameter is the target of the search ('Search all archive sources for snapshots of a URL'). However, it does not add any format or constraint details (e.g., encoding, absolute URL), leaving some ambiguity for a single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'all archive sources for snapshots of a URL'. It lists the specific archive sources and explicitly differentiates from sibling tools like remove_paywall by stating it does not extract content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit alternative: 'use remove_paywall for full article retrieval'. It implies the tool is for discovering snapshots, not fetching content. However, it does not differentiate from other siblings like get_from_archive, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.