Server Quality Checklist

Tool Scores

• check_authorizationA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses return type (registry record or 'unknown') and important caveats: absence is not judgment, perceptual matches need human confirmation. No annotations exist, so description carries burden; covers key behavioral aspects but omits auth or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff: first defines purpose, second covers query options and return value with caveats. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers main purpose and return type, but missing explanation of maxDistance parameter and details about the registry record structure. With no output schema, more detail on return value would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Explains sha256 and pdq parameters meaningfully (exact bytes vs. perceptual fingerprint from capture_evidence), but fails to describe maxDistance parameter at all. With 0% schema coverage, description fills some gaps but leaves one parameter unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states the tool checks content against a registered work in the Official AI registry, using clear verb 'Check' and specific resource. Differentiates from siblings like capture_evidence and draft_notice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on query methods (sha256 or pdq) and mentions pdq comes from capture_evidence, but does not explicitly state when to use this tool over siblings or when not to use it. No exclusion criteria or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• capture_evidenceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It discloses that the tool creates an independent archive, timestamp, hash, and fingerprint, implying a non-destructive snapshot. However, it does not mention authentication needs, rate limits, or what happens if the URL is unreachable—gaps that a richer description or annotations would fill.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding distinct value: what evidence is captured, return type, and optimal usage order. No unnecessary words or repetition. Front-loaded with the key action and components.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description covers purpose, evidence types, return value, and usage order. It lacks error or auth context but is otherwise complete. A manifest structure description would improve completeness but is not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description explains the overall tool behavior but adds little to individual parameter semantics beyond what the schema already provides (e.g., URL of infringing post, additional media URLs). The connection between mediaUrls and the PDQ fingerprint is implied but not explicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool captures evidence of a URL hosting infringing content, listing specific actions (archive snapshot, timestamp, SHA-256, PDQ fingerprint) and output (typed evidence manifest). It distinguishes itself from siblings (check_authorization, draft_notice, get_playbook) which handle other workflow steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Run this FIRST, before the content is taken down or changed,' providing clear temporal guidance. It does not explicitly exclude scenarios or name alternatives, but the sibling tools imply a workflow where this is the initial step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• draft_noticeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behaviors: deterministic computation of statutory fields, production of a draft (not a final filing), inclusion of filing instructions, and need for human review. It does not cover potential errors or rate limits, but the core safety profile is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three focused sentences, each valuable: purpose, nature of output, and prerequisites. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately complex tool with nested objects and no output schema, the description covers purpose, output nature, and required prior steps. It lacks details on error handling or return structure, but the core completeness is good.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides high-level context that statutory fields are computed deterministically, but does not detail individual parameters beyond naming required ones. The schema provides descriptions for 67% of parameters, so the description adds marginal value beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool drafts a takedown notice from evidence, with specific verb 'draft' and resource 'takedown notice'. It distinguishes from siblings by referencing required prior tools (capture_evidence, get_playbook).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies prerequisites ('Use capture_evidence first', 'get_playbook to pick the right reportPathId') and clarifies the output is a draft requiring human review. It does not explicitly list when not to use, but the context implies it's only for drafting, not filing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_playbookA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that facts have citations with retrieval dates and unverifiable facts are marked UNVERIFIED, ensuring data quality transparency. No mention of rate limits or auth, but acceptable for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the action and list key deliverables. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description fully explains what is returned (playbook contents with citations and verification status) and lists all allowed inputs. Complete for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description of platform parameter lists enum values and notes 'Omit to fetch all platforms', adding context beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Get the takedown playbook for a platform' and lists the contents (legal bases, URLs, etc.) and valid platforms (meta, youtube, etc.). Clearly distinguishes from siblings like capture_evidence or draft_notice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when needing the playbook for a specific platform, but no explicit when-not or comparison to siblings. Could be improved by stating not to use for actual takedown submission.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.