Server Quality Checklist

Tool Scores

• create_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Create' implies a write operation, it doesn't address permissions needed, whether the page becomes immediately visible, error conditions, or what happens on duplicate titles. This is inadequate for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after creation, potential side effects, or error handling. Given the complexity of creating a wiki page, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds no additional parameter information beyond what's in the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('new wiki page'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'update_page', but it's not vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'update_page' or 'read_page'. There's no mention of prerequisites, context, or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_categoriesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers minimal information. It implies a read-only operation ('Get') but doesn't specify aspects like authentication needs, rate limits, error conditions, or what happens if the page doesn't exist. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of a single, clear sentence that directly states the tool's purpose without any wasted words. Every part of the sentence earns its place by conveying essential information, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for effective tool use. It doesn't explain what the tool returns (e.g., a list of category names, IDs, or full objects), error handling, or dependencies. For a tool with no structured output or behavioral hints, the description should provide more context to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'title' parameter fully documented in the schema itself. The description adds no additional meaning beyond what the schema provides (e.g., it doesn't clarify format expectations or edge cases for the title). Thus, it meets the baseline score of 3, as the schema handles the heavy lifting for parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('categories a page belongs to'), making it immediately understandable. It distinguishes itself from siblings like 'get_page_history' or 'read_page' by focusing on page categorization rather than content or history. However, it doesn't explicitly differentiate from all siblings (e.g., 'search_pages' might also involve categories), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing page), exclusions, or comparisons to siblings like 'search_pages' which might also retrieve category-related data. Without such context, the agent must infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_page_historyC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action but does not describe key traits such as whether this is a read-only operation, if it requires authentication, how revisions are ordered (e.g., chronological), or what the output format looks like. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action and resource, making it easy to parse quickly. Every part of the sentence earns its place by conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of retrieving revision history (which may involve multiple revisions, ordering, and metadata), the description is incomplete. It lacks details on output structure, error handling, or behavioral traits like pagination or rate limits. With no annotations and no output schema, the description does not provide enough context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('title' and 'limit' with a default). The description does not add any meaning beyond the schema, such as explaining what constitutes a 'revision' or how the limit affects performance. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('revision history of a page'), making the purpose understandable. However, it does not distinguish this tool from potential siblings like 'read_page' or 'search_pages' in terms of scope or output type, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it does not specify if this is for auditing changes, comparing versions, or recovering content, nor does it mention prerequisites like needing the page title. Without such context, the agent lacks direction on appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only states the basic action ('search for pages') without covering key behavioral traits such as whether this is a read-only operation, how results are sorted or ranked, if there are rate limits, authentication requirements, or what the return format looks like. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, with every part earning its place by conveying the core functionality. There is zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search operation, lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the search returns (e.g., page titles, snippets, IDs), how results are structured, or any behavioral aspects like pagination or error handling. The description alone is insufficient for an agent to fully understand how to use this tool effectively in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for both parameters ('query' and 'limit'), including defaults and constraints. The description adds no additional parameter semantics beyond what the schema provides, such as explaining search syntax or result types. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Search for pages in the wiki using keywords', which includes a specific verb ('search'), resource ('pages in the wiki'), and method ('using keywords'). However, it doesn't explicitly distinguish this search tool from potential sibling alternatives like 'read_page' or 'get_categories', which could also involve finding content. The description is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer 'search_pages' over 'read_page' for finding specific content, or how it relates to 'get_categories' for browsing. There are no exclusions, prerequisites, or context provided, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Update' implies a mutation operation, but the description doesn't specify permissions required, whether changes are reversible, rate limits, or what happens if the page doesn't exist. This leaves significant behavioral gaps for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the essential information (update + wiki page) and doesn't include unnecessary elaboration or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns, error conditions, or behavioral constraints. Given the complexity of updating a wiki page (which could involve permissions, versioning, etc.), more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any parameter-specific information beyond what the schema already provides (title, content, summary). This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update') and resource ('an existing wiki page'), making the purpose immediately understandable. It distinguishes from siblings like 'create_page' (new vs existing) and 'read_page' (read vs update), though it doesn't explicitly mention these distinctions in the text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_page' for new pages or 'read_page' for viewing. It doesn't mention prerequisites (e.g., page must exist) or contextual considerations, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does ('fetch raw wikitext') but doesn't describe any behavioral traits like error handling (e.g., what happens if the page doesn't exist), performance characteristics, authentication requirements, or rate limits. For a read operation with zero annotation coverage, this leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that immediately communicates the core functionality without any wasted words. It's front-loaded with the essential information and there's no unnecessary elaboration. Every word earns its place in this compact description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, read-only operation) and 100% schema coverage, the description is adequate but incomplete. It lacks output information (no output schema exists), doesn't mention error cases or behavioral constraints, and provides minimal context about when to use it. For a basic read tool, it meets minimum requirements but doesn't provide the completeness needed for optimal agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'title' clearly documented in the schema. The description doesn't add any parameter-specific information beyond what the schema provides (it doesn't explain formatting, constraints, or examples for the title parameter). With high schema coverage, the baseline score of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetch') and resource ('raw wikitext content of a page'), making the purpose immediately understandable. It distinguishes from siblings like 'get_page_history' (historical data) and 'search_pages' (searching), though it doesn't explicitly mention these alternatives. The verb+resource combination is specific but could be more precise about what distinguishes it from similar read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when raw wikitext is needed, but provides no explicit guidance on when to use this tool versus alternatives like 'get_categories' or 'search_pages'. There's no mention of prerequisites, error conditions, or typical use cases. The context is clear from the description alone, but lacks the explicit when/when-not guidance that would elevate the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.