Server Quality Checklist

Tool Scores

• get_main_pageB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the conditional behavior based on the 'zim' parameter, which is useful. However, it lacks details on permissions, rate limits, error handling, or what 'loaded ZIM' means in practice. For a tool with no annotations, this leaves significant gaps in understanding its operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just two sentences—with zero wasted words. It front-loads the core purpose and efficiently explains the parameter's effect. Every sentence earns its place by adding critical information, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is somewhat complete but has gaps. It covers the basic purpose and parameter semantics adequately. However, without annotations or output schema, it should ideally mention more about behavioral aspects like return format or error cases. It's minimally viable but could be more comprehensive for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter 'zim': it explains that if no zim name is given, it returns the main page of every loaded ZIM. This clarifies the parameter's optional nature and its impact on behavior. Since schema description coverage is 0%, the description compensates well by providing semantic understanding beyond the bare schema, though it could specify what a 'zim name' format entails.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch') and resource ('main/home page of a ZIM'), making the purpose understandable. It distinguishes from siblings like 'get_article' by focusing on the main page rather than specific articles, though it doesn't explicitly contrast them. The purpose is specific but could be more precise about what a 'main page' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it by stating the conditional behavior: use with a zim name to fetch a specific ZIM's main page, or without to get all loaded ZIMs' main pages. However, it doesn't provide explicit guidance on when to choose this tool over alternatives like 'list_libraries' or 'search', nor does it mention any prerequisites or exclusions. The usage context is implied but not fully articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the scope ('across all loaded ZIMs') and describes the return format ('Each hit includes the source ZIM, path, title and a short snippet'), but doesn't address important behavioral aspects like whether this is a read-only operation, performance characteristics, error conditions, or pagination behavior. For a search tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise. It begins with the core purpose statement, then provides a well-organized parameter section with clear explanations for each parameter. Every sentence earns its place, and the information is front-loaded with the most important details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search across multiple ZIMs), no annotations, no output schema, and 3 parameters, the description provides adequate but incomplete coverage. It explains parameters well and describes the return format, but lacks information about authentication requirements, rate limits, error handling, or the relationship between 'limit per ZIM' and total results across multiple ZIMs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the lack of parameter documentation in the schema. The description successfully adds meaning for all three parameters: explaining 'query' as 'search terms', 'limit' as 'max results per ZIM' with default value, and 'kind' as an 'optional filter' with enumerated values. This provides clear semantic understanding beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'Full-text search across all loaded ZIMs' which is a specific verb+resource combination. It distinguishes itself from sibling tools like 'get_article' (retrieves specific content), 'get_main_page' (gets homepage), and 'list_libraries' (lists available libraries). However, it doesn't explicitly contrast with these siblings in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the phrase 'across all loaded ZIMs' and the parameter documentation, but doesn't explicitly state when to use this tool versus alternatives. No guidance is provided about when not to use it or what specific scenarios warrant choosing 'search' over sibling tools like 'get_article' for direct content retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the article is fetched 'as plain text ready for reading', which adds context about the output format. However, it lacks details on error handling (e.g., what happens if the path is invalid), performance (e.g., speed or size limits), or side effects, leaving gaps in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by a structured 'Args:' section that efficiently explains parameters without redundancy. Every sentence adds value, and there is no wasted text, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is partially complete. It covers the purpose and parameters well but lacks information on return values (since no output schema exists), error conditions, or dependencies on sibling tools. This leaves some gaps for an AI agent to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It provides clear semantics for both parameters: 'path' is explained as 'the ZIM entry path (as returned in search results, e.g., ``A/Aspirin``)', and 'zim' is described as 'optional filename or path of the ZIM to read from' with behavior when omitted. This adds meaningful context beyond the bare schema, though it could include examples or constraints for 'zim'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Fetch a single article by path') and resource ('article'), distinguishing it from sibling tools like 'list_libraries' (which lists libraries) and 'search' (which searches across content). It also specifies the output format ('as plain text ready for reading'), which further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'as returned in search results' for the path parameter, suggesting it should be used after a search. However, it does not explicitly state when to use this tool versus alternatives like 'get_main_page' or provide exclusions, leaving some ambiguity about the ideal context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_librariesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior by specifying the return format ('list of archives with their kind, metadata, and aggregate capabilities') and implying it's a read-only inventory operation (no destructive actions mentioned). However, it lacks details on potential errors, rate limits, or authentication requirements, which are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by details on returns and usage guidance. Every sentence earns its place: the first explains what it does, the second details the output, and the third provides critical context on when to use it. There is no wasted text, and it's structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple inventory with 0 parameters) and lack of annotations/output schema, the description is largely complete. It covers purpose, output details, and usage context. However, it doesn't specify the exact structure of the returned list or error handling, which are minor omissions for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description appropriately focuses on output semantics, explaining what information is returned (e.g., archive kinds, metadata, capabilities like 'plan_route'). This adds value beyond the empty input schema, justifying a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Inventory') and resource ('ZIM archives this server can read'), distinguishing it from sibling tools like get_article, get_main_page, and search by focusing on system inventory rather than content retrieval. It explicitly mentions what is returned ('list of archives with their kind, metadata, and aggregate capabilities'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Call this first so you know what sources and tools are available.' This indicates it should be used as an initial step to discover available resources before invoking other tools, clearly differentiating it from siblings that perform content operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.