Server Quality Checklist

Tool Scores

• random_pageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It reveals the uniform randomness and the expected gibberish content, providing a clear expectation of output quality. However, it does not elaborate on potential edge cases or the exact nature of 'content', but this is sufficient for a zero-parameter tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and every word adds value. The playful tone is appropriate and does not compromise clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with no output schema, the description fully covers what the tool does, how it behaves (uniform randomness), and what to expect (gibberish). It is complete and leaves no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema already reflects this with 100% coverage. No parameter explanations are needed, and the description appropriately focuses on the tool's behavior rather than inputs. The baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Jump to a uniformly random address'), the resource ('Library of Babel'), and the result ('return its content'). It distinguishes itself from sibling tools like get_page and search_text by emphasizing randomness, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for random sampling ('Jump to a uniformly random address') but does not explicitly state when to use this tool versus alternatives or when not to use it. The context of siblings suggests it is for random discovery, but no explicit guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behavioral traits: deterministic results ('Every address always returns the same content') and that most addresses contain gibberish. It does not mention error handling or output format, but for a simple read operation these are less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no wasted words: purpose, determinism, and coordinate/usage guidance. Each sentence earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides a solid amount of context: purpose, coordinate ranges, deterministic behavior, and tool-selection guidance. It leaves out explicit failure semantics and return value details, but it is largely complete for deterministic page retrieval.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 20% (hexagon), so the description compensates by listing all five coordinates with their ranges and explaining hexagon as a base-36 string id. This adds meaningful context beyond the raw schema, though it stops short of mentioning default values for optional parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Retrieve the exact text') and resource ('a page at a given address in the Library of Babel'), making the tool's function immediately clear. It also distinguishes itself from siblings like search_text by emphasizing direct address-based retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance: 'Almost every address is gibberish — meaningful pages are found via search_text, not by browsing.' This names an alternative tool and states when not to use get_page, while also implying it should be used for direct, known-address retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_pageA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral constraints: exact 3200-character input and alphabet limitations (a-z, comma, space, period), making it an exact-match operation. However, it does not describe error handling or return format for non-matching text, which would be useful for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. The first sentence states the core function, and the second provides inverse relationship and alternative. Every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter lookup tool with no output schema, the description is complete: it gives purpose, input constraints, related tools, and an alternative for partial text. The return value (address) is clearly stated, and there are no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the 'text' parameter with a basic description, but the tool description adds critical semantic constraints: exact length and allowed character set. This goes beyond the schema's 'full page content' and significantly aids correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact function: given a 3200-character page, return its address. It uses specific verbs (return, lives at) and clearly distinguishes itself from siblings by describing the inverse of get_page and pointing to search_text for partial text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: use for complete exact pages, and explicitly instructs 'For partial text, use search_text instead.' It also implies the relationship with get_page as the inverse operation, giving a clear when-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_textA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the behavioral burden. It discloses that no actual scanning occurs, text is lowercased and restricted to a 29-symbol alphabet, unsupported characters are dropped/converted, and padding/offset affect the resulting address. This goes beyond a simple read/write hint and provides operational detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, front-loads the purpose, and every sentence adds value. It is somewhat dense as a single paragraph but remains appropriate in length and clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description explains the return value (an address), how to validate it (get_page returns your text), and the behavioral implications of parameters. For a tool with three parameters and no output schema, this is thoroughly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema coverage is 100%, the description adds meaningful semantics: the text length limit, the alphabet restriction, character handling, and that offset/padding are part of the page and thus change the address. This enriches understanding beyond the schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Find the address of a page containing the given text.' It also distinguishes itself from siblings by explaining that it constructs and inverts the address mapping rather than scanning, making its unique functionality clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: you want an address for a page containing given text, and get_page on that address returns the text. It implies the relationship with sibling tools but does not explicitly list alternatives or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.