Server Quality Checklist

Tool Scores

• systemC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It lists actions but doesn't describe what each does (e.g., 'info' might return system status, 'commands' could list available commands, 'fetch_web' likely retrieves web content). It omits critical details like permissions needed, rate limits, side effects, or response format, leaving significant gaps for a tool with multiple operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief but under-specified—it's a single phrase listing actions without elaboration. While concise, it lacks structure and front-loading of key information. Every word earns its place, but the content is insufficient, making it more of a placeholder than a helpful description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (multiple actions under one tool), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns or how actions differ, leaving the agent to guess. For a tool with 2 parameters and varied operations, this minimal description is inadequate to ensure correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema documents both parameters: 'action' with enum values and 'url' for fetch_web. The description adds no meaning beyond this—it doesn't explain what each action entails or when 'url' is required. Baseline 3 is appropriate since the schema does the heavy lifting, but the description fails to compensate with additional context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'System operations - info, commands, fetch_web' lists three operations but doesn't specify what each does or what 'system operations' means. It's vague and doesn't clearly distinguish this tool from siblings like 'edit', 'vault', 'view', or 'workflow'. The description essentially restates the tool name 'system' with appended action names, making it tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't explain the context for 'info', 'commands', or 'fetch_web', nor does it mention prerequisites or exclusions. Without any usage instructions, an agent must infer from the action names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• editC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'auto-buffers content' for the window action and 'supports fuzzy matching' in the schema, but doesn't explain what these mean operationally. Critical behavioral aspects like whether edits are destructive, require specific permissions, have side effects, or how errors are handled are completely undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single phrase listing five actions. While efficient, it's arguably too terse given the tool's complexity. The structure is front-loaded with the action names, but lacks any explanatory context. Every word earns its place, but more context would be helpful for such a multi-action tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 11-parameter tool with no annotations and no output schema, the description is inadequate. It doesn't explain what gets edited (files? buffers? notes?), how different actions relate to each other, what the expected outcomes are, or any error conditions. The agent must rely entirely on parameter schemas without understanding the broader editing context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly with descriptions and enums. The description adds minimal value by listing the five action names, but doesn't explain their differences or relationships to parameters. The baseline of 3 is appropriate when the schema does the heavy lifting, though the description could have better explained how actions map to parameter requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lists five specific actions (window, append, patch, at_line, from_buffer) which gives some specificity about what the tool does, but it doesn't clearly state the overall purpose or what resource is being edited. The phrase 'Smart editing operations' is vague and doesn't distinguish this from potential sibling tools like 'view' or 'workflow'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when to choose 'edit' over other sibling tools like 'view' or 'workflow', nor any context about which action to select for different editing scenarios. The agent must infer usage entirely from parameter names and enums.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workflowC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions providing 'guidance and suggestions,' it doesn't describe what form these take, whether they're actionable recommendations, informational tips, or something else. There's no information about permissions needed, rate limits, side effects, or what constitutes 'current context' that triggers the suggestions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise at just 6 words. It's front-loaded with the core purpose ('Workflow guidance and suggestions') and adds qualifying context ('based on current context'). There's no wasted language, though the brevity contributes to the vagueness noted in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what kind of output to expect (structured data, natural language, actionable steps), what domains it covers, or how the 'current context' is determined. The combination of vague purpose, missing behavioral details, and unspecified output format leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no additional meaning about parameters beyond what's in the schema. It doesn't explain how 'type' relates to 'workflow guidance' or what values might be appropriate for 'type' beyond the schema's basic type declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Workflow guidance and suggestions based on current context' states a general purpose (providing guidance/suggestions) but lacks specificity about what resources or domains it operates on. It doesn't distinguish itself from sibling tools like 'edit', 'system', 'vault', or 'view' - all of which could potentially provide guidance in different contexts. The description is vague about what exactly 'workflow' refers to.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance about when to use this tool versus the sibling tools. There's no mention of prerequisites, appropriate contexts, or alternatives. The phrase 'based on current context' implies some situational awareness but doesn't specify what constitutes appropriate context or when other tools might be better choices.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vaultC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it lists operations, it provides no information about permissions required, whether operations are destructive, rate limits, error conditions, or what happens during conflicts. For a tool with 11 parameters including 'delete' and 'update' actions, this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single phrase listing the core operations. Every word earns its place, and it's front-loaded with the essential information. There's no wasted verbiage or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 11 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns, how different actions behave, error handling, or performance characteristics. The description provides only a high-level capability list without the context needed for effective tool selection and use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. According to the scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'File and folder operations' followed by a list of specific actions (list, read, create, update, delete, search). This provides a clear verb+resource combination. However, it doesn't differentiate this tool from potential sibling tools like 'view' or 'edit', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools named 'edit', 'system', 'view', and 'workflow', there's no indication of which scenarios call for 'vault' versus those other tools. The description simply lists capabilities without context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• viewC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'viewing and navigation', which implies read-only operations, but doesn't clarify if this tool requires specific permissions, has side effects (e.g., opening files in Obsidian), or handles errors. For a tool with multiple parameters and actions, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose ('Content viewing and navigation'), followed by a list of actions. It wastes no words, though it could be slightly more structured by grouping related actions or adding brief explanations. Overall, it's efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, multiple actions) and lack of annotations or output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., file content, window details), how actions differ in behavior, or prerequisites (e.g., 'path' required for 'file' action). For a multi-action tool, this leaves too much undefined for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all parameters thoroughly (e.g., 'action' with enum values, 'lineNumber', 'path', etc.). The description adds minimal value by listing action names but doesn't provide additional context like how parameters interact (e.g., 'path' is needed for 'file' action) or usage examples. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Content viewing and navigation' and lists specific actions (file, window, active, open_in_obsidian), which provides a good overview of what it does. However, it doesn't explicitly differentiate this viewing/navigation tool from sibling tools like 'edit' (which likely modifies content) or 'vault' (which might manage files), leaving some ambiguity about when to choose this tool over alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'edit' or 'vault'. It lists actions but doesn't explain the context for choosing 'view' over other tools, such as for read-only operations versus modifications. This lack of explicit usage context leaves the agent to infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.