Server Quality Checklist

Tool Scores

• list_booksB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of disclosing behavioral traits. It does not mention that this is a read-only operation, whether it requires authentication, what the response format looks like, or if it could be destructive. The description is too brief to convey behavioral context beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that is front-loaded with the key action and resource. It wastes no words, but could arguably be slightly more informative without losing conciseness, hence a 4 rather than a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema, a mandatory parameter, and no annotations, the description should provide more context about what the response contains (e.g., book names, chapters count) and whether the output is sorted or ordered. The current description is insufficient for an agent to understand the complete behavior or handle edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 100% and the single parameter 'translation' has a clear description including an example value and a cross-reference to 'list_bibles' for available IDs. The tool description adds no additional semantic value beyond what the schema already provides, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and the clear resource 'books in a specific Bible translation'. It unambiguously states what the tool does and distinguishes it from siblings like 'list_bibles' (which lists translations) and 'get_passage' (which retrieves content).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the tool is for enumerating books within a translation, but does not explicitly state when to use it versus alternatives like 'get_chapter' or 'list_bibles'. It provides no guidance on prerequisites (e.g., user must have a valid translation ID) or exclusions (e.g., not for retrieving chapters).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_passageA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It states the core behavior (retrieving a verse or range) but does not disclose what happens on invalid input (e.g., nonexistent verse, invalid translation) or describe the output format. It is not misleading but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 17 words, front-loading the key information ('Get a single Bible verse or a range of verses'). Every word is meaningful, and there is no redundancy or extraneous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters (4 required), no output schema, and no annotations, the description adequately covers the retrieval action. However, it could be more complete by specifying the return format (e.g., the verse text) to fully guide the agent. The lack of output schema shifts the burden to the description, which partially meets it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds minimal value beyond the schema by mentioning 'verse number(s)', hinting at the endVerse parameter for ranges. However, it does not explain the difference between verse and endVerse or the nature of the translation ID, which are already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'single Bible verse or a range of verses', specifying the method using book, chapter, and verse numbers. This distinguishes it from sibling tools like list_bibles, list_books, and get_chapter, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool instead of alternatives like get_chapter for whole chapters, or prerequisites like listing books or translations. The description implies usage by describing the action, but does not offer conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_chapterA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosure. It clearly states that the tool returns an entire chapter including verses and section titles, which is a read operation. It does not mention potential limitations, performance, or authentication requirements, but for a simple data retrieval tool, this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately conveys the tool's purpose. Every word is necessary, and there is no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with three parameters and no output schema, the description provides adequate context about what the tool returns. It could be slightly improved by explicitly contrasting with get_passage to clarify usage boundaries, but overall it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all three parameters having descriptions in the input schema. The tool description does not add any additional meaning or format details beyond what the schema already provides, so it meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'entire chapter of the Bible', with additional detail about including all verses and section titles. This distinguishes it from sibling tools like list_bibles (listing translations) and get_passage (likely a subset of verses).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool is for retrieving whole chapters, but it does not explicitly state when to use it versus siblings like get_passage (which might cover a range of verses). No guidance on prerequisites or alternatives is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_biblesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears full responsibility for behavioral disclosure. It correctly identifies the operation as a read/list action and specifies the data source (the Bibles directory). However, it omits details such as whether the list is sorted, what happens if the directory is empty or missing, or if any authorization is required. For a simple parameterless tool this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that immediately conveys the tool's action ('List...'), the resource ('Bible translations'), and the context ('installed in the Bibles directory'). Every word earns its place; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and is part of a small family of tools, the description provides enough high-level context to understand its role. It could mention the nature of the output (e.g., 'returns a list of translation names') to be fully self-contained, but for a simple list operation it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (vacuously). The description does not need to add parameter-level meaning. With no parameters, a baseline score of 4 is appropriate, and the description adds no additional param information, which is fine.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the specific resource 'available Bible translations', with the scope 'installed in the Bibles directory'. This effectively distinguishes it from sibling tools like list_books (which lists books within a translation) and get_passage/get_chapter (which retrieve content). The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives. While it is implied that one would call list_bibles first to see available translations before using list_books or get_passage, no direct usage context or exclusions are provided. The guidance is minimal, relying on the user to infer the workflow from the tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.