Server Quality Checklist

Tool Scores

• workflowC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavior. It lists actions but does not explain side effects, required permissions, or what happens on transition/reset. The referral to an external document for usage guidance leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively short and front-loaded with the main purpose. However, it packs multiple concepts (actions, flow, external reference) into one sentence, which may reduce clarity. It could be restructured for better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, no annotations, and 0% schema coverage, the description is insufficient. It fails to explain return values, edge cases, or how to correctly use the transition parameter. The tool is too complex for such sparse documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. It mentions actions but does not clarify the 'transition' parameter beyond its type. No detail on expected values or behavior, leaving the agent without needed context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool manages agent workflow state for a vault, listing specific actions (status, transition, history, reset). It provides a typical flow and references external guidance, making the purpose discernible and distinct from sibling tools like edit or view.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a typical flow (search → open_note → save → done) and suggests reading vault://overview for usage. However, it does not explicitly state when to use this tool versus siblings like system or vault, nor does it mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• viewB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It conveys vault scope and action-specific behaviors (e.g., heading-aware retrieval, hybrid search), but does not cover all actions' details, rate limits, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose, but the action list is dense and could be more concise. It is not overly long, but some redundancy exists (e.g., 'search' vs 'semantic_search' both mention retrieval).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and low schema coverage (25%), the description fails to explain return values, edge cases, or behavioral details for many parameters. The tool is multi-action and moderately complex, so more completeness is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25%, yet the description does not describe any parameters beyond the actions. Parameters like maxChunks, heading, headingDepth, directory, and items remain unexplained, requiring the agent to infer from schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Read and search markdown notes' which clearly identifies the verb and resource. The list of actions further clarifies capabilities, but sibling tools (edit, system, vault, workflow) are not explicitly contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides brief explanations for each action (e.g., 'search uses TF-IDF'), giving some guidance on when to use each. However, it does not explicitly state when to prefer this tool over siblings or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vaultB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions 'whole-file writes' for create/update/delete, providing some behavioral context. Since no annotations are provided, more detail on auth, rate limits, or side effects would be needed for a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. The first sentence states the purpose, the second enumerates actions with brief comments. It is concise but could be better structured with bullet points.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, multiple actions) and lack of output schema/annotations, the description does not cover return values, errors, or permissions. It references an external overview doc, but is not self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to the action parameter by specifying each action's purpose (e.g., 'list (browse notes)', 'create_from_template (scaffold from template)'). This adds value beyond the schema descriptions for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it manages vault notes and lists all actions. It distinguishes the vault as a general markdown notes vault, but does not explicitly differentiate from sibling tools like 'view' or 'edit'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It only hints at reading a separate overview document for search strategy and conventions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• systemA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It reveals some behavioral traits (e.g., reindex is 'async rebuild', save_overview persists host-written overview) but does not disclose safety, permissions, or consequences of misuse. Some behaviors are hinted but not comprehensively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose and lists actions concisely. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple actions, some writing), the description lacks details on return values or side effects. No output schema exists, so it would benefit from mentioning what each action returns. It covers the main actions adequately but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 3 of 4 parameters with descriptions (75% coverage). The tool description adds meaning by explaining the action enum values (e.g., 'status' linked to indexing/backlinks health), which the schema does not describe. This compensates for the missing action description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'System administration for this vault' and enumerates specific actions with brief explanations (e.g., 'status (indexing/backlinks/workflow health)'). This distinguishes it well from sibling tools like edit, vault, view, and workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for system-level tasks via the listed actions but provides no explicit guidance on when to use this tool versus alternatives, nor does it mention when not to use it. Usage is implied but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• editA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses safety intent, batch sequential execution stops on first error, dryRun returns preview without saving. Does not cover reversibility or permissions, but key behaviors are transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single dense paragraph, front-loaded purpose, every sentence adds value. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters, batch mode, and no output schema, description provides a good overview but lacks details on parameter formats and interpretation. References vault://overview for strategy, but standalone completeness is moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is low (42%). Description adds context for overall tool functionality and mentions key patterns (heading/block ID, line/string replace, frontmatter_set), but many parameters like startLine, endLine, searchText, replaceAll lack explanation. Partially compensates but could do more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it edits notes in a general markdown notes vault, with specific operations (AST, line/string replacement, frontmatter_set, batch, dryRun). Differentiates from sibling tools (system, vault, view, workflow) which are not editing-focused.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes supported edit modes (by heading, block ID, line/string, frontmatter) and features like batch limits (max 50) and dryRun. Implicitly limits usage to this vault scope. Could explicitly state when not to use or mention alternatives, but provides operational guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.