Server Quality Checklist

Tool Scores

• write_wikiA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It mentions auto-generating frontmatter but fails to reveal important behaviors such as overwrite semantics, what happens if a page already exists, or whether the index is rebuilt. This is a significant gap for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately communicates the tool's purpose. It is appropriately sized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple create operation with a relatively small parameter set and an output schema. The description covers the essential purpose and a key behavioral detail (frontmatter generation). While it could mention overwrite behavior, the low complexity and rich schema make this generally sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter (title, body, tags, page_type) is already documented. The description does not add any parameter-specific meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '新建一个 Wiki 页（自动生成 frontmatter）' clearly states the action (create) and resource (Wiki page), with a specific detail about frontmatter generation. It distinguishes from siblings like query and read_wiki which are read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied by the tool name and the nature of the write operation, but there is no explicit guidance about when to use this vs alternatives like read_wiki or ingest. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ingestA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the main behaviors (extract concepts, build links, mark contradictions) and implies the side effect of creating a Wiki page, but it does not mention overwrite behavior, reversibility, or other side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core behavior efficiently. It avoids fluff and includes a parenthetical that adds key detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema likely covers return values, and the one-parameter schema is simple. However, the description does not explain integration with sibling tools, such as whether ingest already updates the wiki index or whether rebuild_index must be called separately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for the single parameter raw_path, which already includes a description and example. The tool description adds marginal value by linking 'raw material' to the parameter, but does not provide substantive additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compiles a raw material into a Wiki page, with specific processing actions: extracting concepts, creating links, and marking contradictions. This differentiates it from sibling tools like write_wiki, read_wiki, and rebuild_index.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for raw materials that need to be converted into Wiki pages, but it does not explicitly state when to use this tool versus alternatives like write_wiki. No prerequisites or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the hybrid retrieval method and the return of Wiki snippets, but does not explicitly state whether the operation is read-only or what 'link expansion' entails in terms of side effects. More detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence in Chinese that front-loads the action and result. Every word adds value, with no filler or repetition, making it both compact and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description provides sufficient context: it explains what the tool does (hybrid retrieval + link expansion) and what it returns (Wiki snippets). It could be more complete by elaborating on 'link expansion' and potential prerequisites, but the current level is adequate for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'question' parameter, so the baseline is 3. The tool description adds minimal extra meaning about the parameter beyond indicating it is the input for the retrieval process. It does not clarify format, length, or any constraints beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs hybrid retrieval and link expansion, returning relevant Wiki snippets. This is a specific action (retrieval) targeting a specific resource (Wiki snippets), and the inclusion of 'hybrid' and 'link expansion' differentiates it from sibling tools like read_wiki or write_wiki.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for searching Wiki content based on a user's question, but it does not explicitly state when to use it instead of alternatives like read_wiki or ingest. No exclusions or alternative references are provided, leaving usage guidance to be inferred from the name and purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_wikiA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns full text including frontmatter, which is useful behavioral detail, but does not mention error handling, output format, or side effects. Reading inherently implies non-destructive behavior, but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, directly states the action and scope, and contains no redundant information. It is both concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with an output schema, the description sufficiently conveys the scope of the operation. It includes the important detail about frontmatter, and the output schema presumably covers return structure. Slight lack of usage guidance and error behavior, but these are not critical for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a clear description of the 'page' parameter, including automatic underscore conversion. The tool description adds no additional meaning beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a full Wiki page including frontmatter, with a specific verb and resource. It distinguishes itself from siblings like write_wiki, query, rebuild_index, and ingest.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose of reading a page is implied when full content is needed, but no explicit guidance is given about when to use this tool versus query for searching or write_wiki for editing. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rebuild_indexA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It mentions 'full rebuild', indicating a destructive/recreating action, and specifies the source from existing pages. However, it does not disclose side effects such as search unavailability during rebuild, required permissions, or whether the operation is reversible/concurrent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, compact sentence that front-loads the verb and resource, with no wasted words. It captures the essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a zero-parameter tool with an output schema, the description adequately covers what the tool does and its source. It lacks explicit guidance on when to run it relative to other tools, but the description is sufficient for a simple invocation action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description does not need to explain parameter meanings. The baseline of 4 applies, and the description adds no extraneous param-related noise.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: to fully rebuild both vector and BM25 indexes from existing Wiki pages. The verb 'rebuild' is specific, and it names the resource ('index') and source ('existing Wiki pages'), distinguishing it from siblings like 'ingest' or 'query'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use after content changes ('from existing Wiki pages') but does not explicitly say when to use it versus alternatives like 'ingest' or whether it is needed after every write. No exclusions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.