Server Quality Checklist

Tool Scores

• search_contentA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses one behavioral trait—natural language term extraction—but omits critical details like result format, ordering, pagination, or authentication. The verb 'search' implies read-only, but that is not explicitly stated, and there is no mention of what the response will look like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two efficient sentences with no redundancy. The first sentence clearly states the tool's purpose and scope, while the second adds a distinctive behavioral detail. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description provides adequate purpose and one behavioral note, but it fails to describe the return value or any usage constraints. An agent knows what to search for but not what to expect back, which is a meaningful gap for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, including a detailed example for 'query'. The tool description adds no additional parameter meaning beyond the schema. The natural-language extraction behavior is already explained in the query parameter's own description, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Search' and identifies the resource as 'JW.Org content' with explicit content types (articles, videos, publications, audio, scriptures). This clearly distinguishes it from sibling get_article and get_scripture, which target specific items rather than broad search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for natural-language searches across a broad content set, but it does not explicitly say when to prefer this over get_article or get_scripture, nor provide exclusions. The phrase 'Extracts meaningful search terms from natural language queries' gives some context for its query style, but no direct alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the operation is a retrieval and gives expected output content (paragraphs and scripture references), but it does not mention potential errors, rate limits, or any side effects. It is adequate but lacks deeper behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, front-loading the action and stating the key output. Every word earns its place, making it concise and well structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers the essential context: what it retrieves and what it returns. It could mention edge cases like invalid URLs or non-article links, but for the given complexity it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'url' parameter, which is already clearly described as 'The article URL from wol.jw.org'. The description adds no additional parameter semantics beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Retrieve') and clearly identifies the resource ('full article content from a JW.Org URL'). It also explicitly states what is returned (text with paragraphs and scripture references), which differentiates it from sibling tools like search_content and get_scripture.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you have a specific article URL) and what it returns, but it does not explicitly state when NOT to use it or mention alternative tools. There is no direct comparison to siblings, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scriptureA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that the tool 'returns the scripture text and reference,' but does not mention error behavior (e.g., invalid reference), handling of the optional translation parameter, or any other side effects. This is minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately states the main action, and includes useful examples without any filler. Every word contributes to understanding the tool's purpose and behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters and no output schema, the description covers the essential purpose, input examples, and return value. It is not missing critical information, though it could optionally mention what happens for invalid references or how the translation parameter affects output. This is adequate for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for both parameters (100% coverage), and the description adds valuable context by giving examples of the reference format. It also clarifies that the return value includes both text and reference, which helps understand the translation parameter's role. This exceeds the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Get') and resource ('scripture text'), and provides concrete examples of valid references. This clearly distinguishes it from sibling tools like search_content, which would be used for searching rather than direct reference lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you have a specific scripture reference), but does not explicitly mention when to use this tool versus alternatives, or any exclusions. It would be improved by noting that search_content is for searching/finding references, while this tool is for retrieving text by a known reference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cache_statsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden of behavioral disclosure. 'Get' implies a read-only operation, but the description does not mention response format, real-time accuracy, or any side effects. It discloses the basic action but lacks deeper context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. Every phrase adds value, specifying exactly what statistics are included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is complete. It names the two key metrics, which is sufficient for an agent to understand what the tool returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (vacuously). The description adds no parameter details, but the baseline for 0 params is 4, and no additional semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('cache statistics') while specifying key metrics (hit rate, entry count). It is distinct from sibling tools like search_content, get_article, and get_scripture, which are content-focused.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case (retrieving cache stats) but does not provide explicit guidance on when to use vs. alternatives. However, given the simplicity and distinct resource, the context is sufficient without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.