Server Quality Checklist

Tool Scores

• find_tagsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavior. Describes live autocomplete and example output, but lacks details on limits, empty results, or number of suggestions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: two sentences plus arg list. Front-loaded with purpose and example. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has output schema, so return values are covered. Description provides purpose, usage timing, and param details. Missing minor behavioral notes but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, description explains both parameters with examples and allowed values for 'kind', adding meaning beyond schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it resolves fuzzy wording to canonical tag names, with an example. Differentiates from sibling tools like search_works.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use before search_works when unsure of exact spelling', providing clear context. Does not include when-not-to-use, but sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool is a read operation (get) and describes return content, but does not mention auth requirements, rate limits, or potential side effects. The indication of read-only behavior is implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, two sentences with no filler. The first sentence states purpose and outputs, the second explains the parameter. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple interface (one parameter) and presence of an output schema, the description covers the key aspects: what the tool does and what it returns. It is sufficiently complete for a 'get by ID' operation, though error handling or edge cases are not discussed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description's second sentence explains the 'work_id' parameter in detail: it is a numeric AO3 work ID from search results or URLs, providing essential usage context beyond the schema's type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the full metadata card for a single work, listing specific content (tags, stats, summary, series info). It effectively distinguishes from siblings like 'read_works' (multiple works) and 'get_work_text' (text only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when needing comprehensive metadata for one work) and contrasts with sibling names that suggest different scopes. However, it lacks explicit guidance on when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_worksA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that a separate model does the reading, never returns raw text, handles batch splitting, retries on safety blocks, and explains refusal behavior. It could be slightly more concise but adds substantial context beyond the tool's name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and usage but runs long with repeated points about not receiving raw text. Every section earns its place, but some redundancy could be trimmed without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (batch reading, safety filters, comparison reports) and the presence of an output schema, the description covers all necessary context: depth, cap limits, refusal handling, and batching behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no description of parameters in schema), but the description explains that work_ids are numeric AO3 IDs and query is a free-form question. This adds meaning beyond the bare schema, though a structured breakdown would improve clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool reads full fics via a separate AI and produces structured reports. It distinguishes itself from siblings like get_work, get_work_text, and search_works by emphasizing that it is the only approved way to read fic content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides when-to-use and when-not-to-use guidance, stating it is the ONLY approved way to read a fic and must be used before recommending or judging. It also explains alternatives (reading alone for long fics) and retry strategies for refusals.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_work_textA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the trade-offs: bypasses mini reader, returns raw text, can be huge (150k+ words), and why it's an escape hatch. Also mentions the max_words parameter to mitigate context impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is thorough and well-structured, front-loading the critical warning and recommendation. While slightly verbose, every part adds value, though a shorter alternative could be possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers usage, parameters, and behavioral impact adequately. With an output schema present, the description need not detail return values. Minor gap: doesn't specify if text includes HTML or formatting.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description explains work_id as 'numeric AO3 work id' and max_words as 'cap the text to first N words, 0 = whole fic', with a suggestion to sample. This adds necessary context missing from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns the raw full text of a fic, and explicitly distinguishes it from read_works by contrasting structured reports vs raw text. The purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (exact wording needed, user asks to quote/close-read) and when-not-to-use (prefer read_works in almost every case), with reasoning about context burden and token waste.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_worksA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses that search is cheap and reading is delegated, that query is full-text and narrows hard, that filters multiply and can cut to zero, and that results include numeric ids and k/h ratio. Also explains wildcard behavior and recovery steps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear headings (RECOMMENDATION WORKFLOW, SEARCH STRATEGY, etc.) and bullet points. However, the description is quite long; while every part is valuable, it could be condensed slightly without losing meaning. Still, the structure aids readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (14 parameters, no annotations, and an output schema exists that covers return values), the description is fully complete. It covers search behavior, error recovery, output interpretation, and links to sibling tools. No gaps remain for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds extensive meaning for every parameter. For example, query lists operator syntax, warns against stuffing synonyms; fandom explains exact-match pitfalls and wildcard use; tags lists popular canonical tags. Each parameter is given detailed guidance, far beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Starts with 'Search AO3 for works' — a specific verb+resource. Distinguishes from siblings read_works and find_tags, and explains the recommendation workflow where this tool is for initial search and shortlisting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (for broad search to over-fetch) and when-not-to (to read fics — delegated to read_works). Gives comprehensive strategy: use wildcards, recover from 0 results by broadening queries, and contrast with find_tags. Also advises on recommended pages for recommendation hunts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.