Server Quality Checklist

Tool Scores

• search_wikiC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It does not mention return format, whether it searches full text, whether it synthesizes answers, limitations, or any side effects. This is a significant gap for a tool that 'answers questions'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (one sentence), which is concise, but the brevity sacrifices clarity. It does not provide enough information to be adequately helpful, so it scores mid-range.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a single parameter, no annotations, and no output schema, so the description must compensate. It only says it answers questions from a local wiki, giving no context about result format, performance, or typical use cases. This is minimally viable but lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage and only one parameter 'query'. The description does not add any semantic detail about the query beyond the vague implication that it is a question, failing to explain expected format, language, or scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('answer a question') tied to a resource ('the local wiki vault'), and the name 'search_wiki' supports this. It distinguishes from sibling tools 'add_document' and 'lint_wiki' by implying a retrieval/QA function, though 'answer' is somewhat ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives, no mention of prerequisites or exclusions, and does not reference sibling tools. It merely states what the tool does.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_documentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetch and compile' but does not disclose side effects such as creating/modifying files in the vault, network requests, auth requirements, or failure modes. It is vague about what 'compile' entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler. Every word contributes to conveying the tool's core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with one parameter, but it still lacks important context: no mention of side effects, when to use it, or what happens on success/failure. The description is too minimal for an agent to safely invoke it without further assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, but the description does indicate that the 'url' parameter is the target URL to fetch and compile. However, it adds no detail about URL format, validity, or edge cases. It provides essential context but not much beyond the parameter property name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Fetch and compile a URL into the wiki vault' uses a specific verb ('fetch and compile') and names the resource (URL into the wiki vault), clearly distinguishing it from sibling tools like search_wiki and lint_wiki. It clearly states what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, exclusions, or situations where search_wiki or lint_wiki would be more appropriate. Usage context is only implied by the verb and name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_wikiA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden. It accurately describes the audit's focus, but does not disclose whether the tool is read-only, returns only a report, or if any side effects occur. 'Audit' hints at non-modifying behavior, but explicit statements about output or side effects would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence that front-loads the verb and resource. Every word adds value, specifying the exact types of audit findings without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless lint/audit tool, the description is quite complete. It communicates the tool's purpose and scope. While it does not describe the output format, the absence of an output schema and the simplicity of the tool make this acceptable; the description still gives a clear picture of behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema coverage is effectively 100%. The description does not need to explain parameter behavior. The tool's purpose as an audit with no inputs is clear, matching the baseline for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Audit') and resource ('the wiki vault'), and narrows the scope with the types of findings ('contradictions, orphans, and missing cross-references'). This distinctly differentiates it from siblings add_document and search_wiki, which are separate operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: when you need to audit the wiki for quality issues. While it does not explicitly name alternatives or exclusions, the context is clear and the sibling tools (add_document, search_wiki) are obviously different, providing indirect when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.