Server Quality Checklist

Tool Scores

• wiki_statsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns page count, method count, and categories, which gives some insight into output, but does not address side effects, authentication, or error behavior. For a simple read-only stats tool this is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the purpose and provides concrete examples. It has no unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a statistics tool with no parameters and no output schema, the description gives a clear idea of the return content (page count, method count, categories). It leaves minor ambiguity about whether categories are counts or lists, but overall it is sufficiently complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty schema, so there is nothing to document. The description does not need to add parameter details, and the baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves statistics about the wiki knowledge base, with specific examples (page count, method count, categories). This distinguishes it from siblings like wiki_read and wiki_search, which focus on content retrieval or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to choose this tool over alternatives. It does not mention any exclusions, prerequisites, or comparison to sibling tools, leaving the usage context inferred but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wiki_listA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly states what the tool returns (title, method_count, DLC versions) and what it does not return (full content). It also notes the optional filtering behavior. This is substantial transparency for a read-only list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the main action, and includes only essential details about the return value. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description is complete enough: it explains what is returned and what is omitted. It does not mention pagination or limits, but that is not critical for a straightforward list endpoint. The metadata fields are enumerated, which adds clarity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the only parameter 'category' with a description and enum values. The tool description adds 'optionally filtered' and lists the enum values, but this mostly repeats schema info. With 100% schema coverage, the description does not need to add much, and any addition is marginal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('wiki pages'), and clearly distinguishes from siblings by stating it returns metadata for all pages, not full content. This differentiates it from wiki_read (reading a specific page) and wiki_search (searching).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need a list of wiki pages with metadata, optionally filtered by category. However, it does not explicitly mention alternatives or when not to use this tool. The sibling tools exist but no exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wiki_readA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description explains the return content (methods, descriptions, DLC badges, wikilinks), giving the agent an idea of what to expect. However, it does not disclose error behavior for missing pages or any potential side effects, though the tool is clearly a read operation. The description carries most of the burden but leaves some edge cases uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states the primary action and resource, second lists the return contents and gives a usage hint. No redundant or filler text. Information is front-loaded and every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a single parameter and no output schema, but the description explains the return value (full page with methods, descriptions, DLC badges, wikilinks) and provides usage context. It omits details about failure modes but for a simple read tool, the description is sufficiently complete for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with examples, so the schema already documents the 'page' parameter well. The description reinforces that the tool reads 'by name', but adds no additional semantic detail beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Read' with a clear resource ('Isaac API wiki page by name'), stating exactly what the tool does. It distinguishes from sibling tools by emphasizing full page retrieval, whereas wiki_search, wiki_list, and wiki_stats serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance to 'Use wiki_search first to find the right page name', which clarifies the intended workflow. It does not explicitly exclude scenarios or mention alternatives, but the sequencing hint is strong actionable context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wiki_searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns 'complete page content (not fragments)' and explains the benefit for the LLM. This adds meaningful behavioral context beyond just saying 'search', though it doesn't mention any side effects or rate limits, which are likely irrelevant for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main function, followed by a concrete use case. Every phrase adds value and there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, but the description explains what is returned: 'complete page content (not fragments)' with wikilinks and class context. It also covers the search scope. For a simple search tool with three well-documented parameters, this is sufficient and complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description doesn't add additional meaning about the parameters; it only mentions the search behavior and return type, not parameter details like query syntax or category filtering.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb and resource: 'Full-text search across all Isaac API wiki pages.' It clearly states the tool's function and differentiates it from siblings like wiki_read and wiki_list by focusing on search across all pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this to find relevant API classes, enums, or tutorials.' It implies when to use search versus reading or listing pages, but doesn't explicitly say when not to use it or name alternatives, so it's not a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.