Server Quality Checklist

Tool Scores

• search_contentA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions natural language term extraction, but this is already present in the schema's parameter description. It does not disclose any safety profile (e.g., read-only), output structure, or other behavioral traits beyond what the schema already provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the purpose and immediately describing the key behavioral addition (natural language extraction). Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for basic selection and usage, but given there is no output schema and no annotations, it lacks details about return format or pagination. It also does not clarify how it interacts with sibling tools, though the context signals partially compensate. This is a minimum viable description with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with detailed descriptions, so the baseline is 3. The description adds no additional parameter semantics beyond what the schema already documents (e.g., filter enum, limit, language).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Search' with a clear resource 'JW.Org content' and enumerates content types (articles, videos, publications, audio, scriptures), which distinguishes it from sibling tools like get_article and get_scripture. It also introduces the natural language extraction behavior, making its scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the broad search tool for JW.Org content, while siblings get_article and get_scripture are for specific retrieval, but it does not explicitly state 'use this when...' or mention alternatives. The context is clear enough for an agent to infer, but it lacks explicit when-to-use versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scriptureA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must convey behavior. It states returns 'scripture text and reference' and provides valid reference formats. It does not mention the translation default or error handling, but for a simple read tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: two sentences that front-load the action and immediately give examples. No redundant or unnecessary wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a simple two-parameter schema and no output schema, the description is adequate. However, it lacks guidance on when to use this tool over search_content, and it doesn't explain translation parameter defaults or behavior, making it a minimum-viable description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds an example for the reference parameter but adds no extra context for translation codes, relying on the schema's 'Bible translation code' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get scripture text by reference' with concrete examples ('John 3:16'). It distinguishes from siblings like search_content and get_article by targeting specific scripture references.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use when you have a direct scripture reference. However, no explicit when-not or alternative guidance is provided, and sibling tools are not mentioned, leaving the agent to infer context from names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cache_statsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It clearly indicates a read operation ('get') and specifies the output fields, but does not mention side effects, permissions, rate limits, or any non-obvious behaviors. The minimal description is honest but lacks explicit clarification of safety/authorization.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It directly states what the tool does and what it returns, earning full marks.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is sufficiently complete. It tells the agent exactly what to expect: cache statistics with hit rate and entry count. No further elaboration is required for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to add beyond the schema. The baseline for 0 parameters is 4, and the description appropriately says nothing about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('get') with a clear resource ('cache statistics') and enumerates the returned metrics (hit rate, entry count). This unambiguously distinguishes it from sibling tools like search_content and get_article, which are content-focused.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention any use case such as performance monitoring, nor does it exclude any specific context. With sibling tools being content retrieval, there is no explicit differentiation beyond the resource type.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It describes the return format (article text with paragraphs and scripture references) but does not mention potential errors, rate limits, or explicitly confirm read-only behavior. Adequate for a simple retrieval tool but not rich in context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy, front-loaded with the key action. Every word contributes to understanding the tool's purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), and the description covers purpose and return content. However, it lacks guidance on when to use this tool versus siblings, which is a minor gap given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the 'url' parameter is well described in the schema as 'The article URL from wol.jw.org'. The description adds minimal extra meaning beyond confirming the source domain, and the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve full article content from a JW.Org URL' with a specific verb and resource. It distinguishes itself from sibling tools like search_content and get_scripture by focusing on full article content from a URL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a JW.Org URL is available and full article content is needed. It provides clear context but does not explicitly mention alternatives or when not to use this tool, such as when searching for content via search_content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.