Server Quality Checklist

Tool Scores

• scan_vaultC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full disclosure burden. It only hints that the analysis runs in the background, but fails to mention whether it is read-only, if it modifies the vault, performance implications, or what the return value looks like. This is minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. However, it is under-specified, which undermines the value of its brevity. It is concise but not sufficiently informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 2 enums, no output schema, and no annotations, the description is far too thin. It does not explain how mode/focus affect results, what outputs are provided, or how it fits with sibling tools. The description is inadequate for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters have basic descriptions (e.g., 'Analysis mode', 'Analysis focus area'). The tool description adds no extra meaning to the parameters, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('perform') and resource ('Obsidian vault'), but 'comprehensive background analysis' is vague and doesn't specify what the scan actually inspects (health, content, relationships). It also doesn't differentiate from sibling tools like get_insights, which may similarly analyze vault data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool instead of alternatives, nor which mode/focus to select for different scenarios. The description does not mention any exclusions, prerequisites, or comparisons with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of explaining side effects, permissions, and scope. It only says 'create', implying output generation, but does not disclose whether it writes files, modifies vault data, or returns a report directly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no filler. It front-loads the action and output type, though it may sacrifice substance for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description lacks vital context about what the report contains, how the analysis is performed, and what the function returns. It is too minimal for a tool with four parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for all parameters (100% coverage), so the baseline is 3. The description adds no extra meaning to the parameters beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('create') and resource ('analysis report'), and mentions format flexibility. It does not explicitly differentiate from sibling tools like get_insights, but the name and action make its role as a report generator evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus scan_vault, get_insights, or track_changes. It does not state prerequisites, use cases, or conditions that indicate this tool is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_insightsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden. It implies a read operation ('retrieve') but does not disclose whether insights are precomputed or generated on-the-fly, what the response format is, or any side effects. This leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the action ('Retrieve actionable insights') and adds the source context. There is no wasted wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema and annotations, the description is too sparse. It does not explain what the insights represent, how the filters interact, or what a returned result looks like. The tool's role among its siblings is not clarified, so the description is incomplete for an agent with no other context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for all four parameters, including enums for category, priority, and timeframe. The description adds no parameter-specific information, but the schema itself is fully self-explanatory, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'retrieve' and identifies the resource as 'actionable insights from vault analysis', clearly stating the tool's function. It does not explicitly mention the insight categories (orphans, connections, gaps, productivity) or distinguish it from generate_report, but the purpose is generally clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like scan_vault, track_changes, or generate_report. There are no exclusions, prerequisites, or contextual clues beyond the vague 'from vault analysis'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• track_changesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior, but it only says 'monitor' with no details on side effects, return format, or performance implications. It does not explain whether this is a read-only operation, what data it returns, or how changes are tracked, which is a significant gap for a monitoring tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler words or redundant information. It is front-loaded and appropriately sized for a one-line description, though the brevity comes at the cost of detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters (one required, one with enums) and no output schema or annotations. The description fails to explain what the tool returns, how the optional parameters affect results, or any practical usage context. This is completely inadequate for a tool that could involve querying historical changes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter already has a description. The tool description adds no additional meaning about how these parameters affect behavior, so the baseline score applies without extra credit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('monitor') and resource ('vault evolution and changes over time'), which conveys the core function. It is distinguishable from sibling tools like scan_vault and generate_report, though it could be more explicit about what kind of output is produced.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. Sibling tools exist, but the description does not mention any use cases, exclusions, or selection criteria, leaving the agent to infer based solely on the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.