Server Quality Checklist

Tool Scores

• read_shelf_fileB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, indicating a safe read operation, which the description aligns with by using 'Read'. The description adds minimal context beyond this, not detailing aspects like rate limits, authentication needs, or file format handling. With annotations covering the safety profile, a baseline score is appropriate, as the description adds little extra behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, clearly front-loaded with the core action. It's appropriately sized for the tool's simplicity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has annotations (readOnlyHint) and an output schema (which handles return values), the description's minimalism is somewhat acceptable. However, for a tool with 2 parameters and 0% schema coverage, it lacks details on parameter meanings and usage context, making it incomplete for full agent understanding without relying heavily on structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description mentions 'shelf' and 'file' but doesn't explain what 'shelf_id' or 'path' mean, their formats, or examples. It adds minimal semantic value beyond the schema, resulting in a baseline score due to incomplete compensation for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Read' and the resource 'a single file from a shelf', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_shelf_tree' or 'search_shelf', which might also involve reading shelf data, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'get_shelf_tree' for browsing or 'search_shelf' for finding files. It lacks any context on prerequisites, exclusions, or comparisons, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_shelfB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation 'readOnlyHint: true' already indicates this is a safe read operation. The description adds minimal behavioral context by implying text-based searching across files, but doesn't disclose details like search scope, performance, or output format. With annotations covering safety, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to grasp immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, read-only), annotations provide safety info, and an output schema exists (so return values are documented elsewhere), the description is reasonably complete. However, it could benefit from more detail on parameter usage or search behavior to fully compensate for low schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The description mentions 'text' and 'files in a shelf', which loosely relates to 'query' and 'shelf_id', but doesn't explain the purpose of parameters like 'mode', 'case_sensitive', or 'max_matches'. It adds some meaning but doesn't fully compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search for text') and target resource ('across files in a shelf'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_shelf_tree' or 'read_shelf_file', which might also involve shelf content access, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_shelf_tree' or 'read_shelf_file', nor does it mention any prerequisites or exclusions. It's a basic statement of function without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_shelf_treeA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, indicating a safe read operation. The description adds value by specifying it retrieves 'full file tree and file contents', which clarifies scope beyond just metadata. However, it doesn't disclose behavioral traits like rate limits, authentication needs, or response format details, relying on annotations for basic safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Get the full file tree and file contents'). There is no wasted wording, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (retrieving tree and contents), annotations cover read-only safety, and an output schema exists (so return values are documented elsewhere), the description is reasonably complete. It specifies scope but could improve by mentioning sibling differentiation or behavioral details like pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema only documents 'shelf_id' as a required string. The description compensates by implying 'shelf_id' identifies the shelf to retrieve, adding semantic context. With only one parameter, this is sufficient for clarity, though it doesn't detail format constraints beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('full file tree and file contents for a shelf'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'read_shelf_file' (which might get individual files) or 'search_shelf' (which might search within shelves), leaving some ambiguity about uniqueness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'read_shelf_file' or 'search_shelf'. It lacks context on prerequisites (e.g., needing a valid shelf_id) or exclusions, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_shelvesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the agent knows this is a safe read operation. The description adds that it lists shelves 'available to the authenticated user,' which provides useful context about access control. However, it doesn't describe pagination behavior, rate limits, or other traits beyond what annotations cover, resulting in a baseline score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List shelves') and adds necessary context ('available to the authenticated user'). There is zero wasted text, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (list operation), annotations covering safety (readOnlyHint), and an output schema (which handles return values), the description is reasonably complete. It specifies the resource and user scope, though it could improve by hinting at pagination or sibling tool distinctions. Overall, it provides adequate context for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema documents parameters (page, limit) without descriptions. The tool description doesn't mention parameters at all, failing to compensate for the coverage gap. With 2 parameters and no output schema details in the description, it meets the minimum viable baseline but adds no value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('shelves'), with the scope 'available to the authenticated user' providing useful context. It distinguishes from siblings like 'get_shelf_tree' (hierarchical view) and 'search_shelf' (filtered search) by implying a straightforward listing without filtering or structuring. However, it doesn't explicitly name these distinctions, keeping it at a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving available shelves, but lacks explicit guidance on when to use this tool versus alternatives like 'search_shelf' (for filtering) or 'get_shelf_tree' (for hierarchical data). No when-not-to-use or prerequisite information is provided, making it only adequate with implied context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.