Server Quality Checklist

Tool Scores

• delete-toolsetC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions deletion but fails to detail critical aspects like permissions required, whether deletion is reversible, confirmation requirements beyond the 'confirm' parameter, or error handling. This leaves significant gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that states the tool's purpose without unnecessary words. It is front-loaded and efficient, making it easy to understand at a glance, with no wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature and lack of annotations or output schema, the description is insufficient. It does not cover behavioral traits like safety warnings, return values, or error conditions, leaving the agent with incomplete information for proper invocation in a complex environment.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('name' and 'confirm'). The description does not add any semantic details beyond what the schema provides, such as explaining the 'confirm' parameter's role in safety or naming conventions. Baseline 3 is appropriate as the schema handles the parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and the resource ('a saved toolset configuration'), making the purpose immediately understandable. It distinguishes from siblings like 'list-saved-toolsets' or 'build-toolset' by specifying deletion. However, it doesn't explicitly differentiate from potentially similar destructive operations among siblings, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as 'unequip-toolset' or other siblings. It lacks context about prerequisites, dependencies, or scenarios where deletion is appropriate, offering only a basic statement of function without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• equip-toolsetC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'filter available tools' which hints at read-only behavior, but doesn't clarify if this is a mutation (e.g., changing system state), what permissions are needed, or what happens on success/failure. For a tool with zero annotation coverage, this is insufficient behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without redundancy. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (toolset management), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'equip' entails operationally, how filtering works, or what the agent should expect after invocation. This leaves significant gaps for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'name' documented as 'Name of the toolset to equip'. The description adds no additional parameter semantics beyond this, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('equip') and the resource ('saved toolset configuration'), specifying that it 'filters available tools'. This provides a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get-active-toolset' or 'build-toolset', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a saved toolset first), exclusions, or comparisons to siblings like 'unequip-toolset' or 'get-active-toolset'. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unequip-toolsetB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the tool unequips and shows available tools, but doesn't clarify if this is a read-only operation, if it requires specific permissions, what happens to the unequipped toolset (e.g., saved or lost), or any rate limits. This is inadequate for a mutation-like tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the primary action ('unequip') and includes the secondary effect. There is no wasted verbiage, and it directly communicates the tool's function without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and the tool's complexity (involving state change and listing), the description is incomplete. It lacks details on behavioral traits (e.g., permissions, side effects), return values, or error conditions. This leaves significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description adds no parameter-specific information, which is appropriate here. Baseline is 4 for zero parameters, as no compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('unequip') and resource ('currently equipped toolset'), and mentions the secondary effect ('show all available tools'). It distinguishes from siblings like 'equip-toolset' and 'get-active-toolset' by focusing on removal rather than addition or retrieval. However, it doesn't explicitly contrast with all siblings (e.g., 'delete-toolset' also removes but permanently).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies usage when switching from an equipped toolset to viewing available tools, but doesn't specify prerequisites (e.g., must have an active toolset) or compare to siblings like 'delete-toolset' for permanent removal. This leaves gaps in agent decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-active-toolsetB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves information, implying a read-only operation, but doesn't specify if it requires authentication, has rate limits, returns structured data, or handles errors. The description adds minimal context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get detailed information about the currently equipped toolset') and adds a clarifying detail ('including availability status'). There is zero waste, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters, 100% schema coverage, and an output schema exists (so return values are documented elsewhere), the description is minimally adequate. However, as a read operation with no annotations, it lacks behavioral details like authentication needs or error handling, leaving some gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. Baseline is 4 for zero parameters, as it avoids unnecessary complexity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get detailed information') and resource ('currently equipped toolset'), specifying what information is retrieved ('including availability status'). It distinguishes from siblings like 'list-available-tools' or 'list-saved-toolsets' by focusing on the currently equipped toolset, though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an equipped toolset first), exclusions, or compare it to siblings like 'list-available-tools' for broader listings. Usage is implied by the purpose but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-saved-toolsetsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool lists configurations with 'detailed information including server configurations and tool counts', which adds useful context about return content. However, it doesn't mention behavioral traits like pagination, rate limits, or authentication needs, leaving gaps for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all saved toolset configurations') and adds specific details ('with detailed information including server configurations and tool counts'). Every word earns its place with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple read operation with no parameters) and the presence of an output schema (which handles return values), the description is reasonably complete. It specifies the resource and details included in the output, though it could benefit from more behavioral context like usage scenarios or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the lack of inputs. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any. Baseline is 4 for zero parameters, as it avoids unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'all saved toolset configurations', specifying the scope. It distinguishes from siblings like 'get-active-toolset' by listing all saved ones, but doesn't explicitly differentiate from 'list-available-tools' which might list different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get-active-toolset' for the currently active toolset or 'list-available-tools' for available tools. The description implies usage for retrieving saved configurations but lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add-tool-annotationA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains the tool's purpose and provides an example, but doesn't disclose important behavioral traits like whether annotations are persistent across sessions, if they override existing ones, what permissions are required, or how they integrate with the toolset system. The example helps but leaves operational details unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly front-loaded with the core purpose in the first sentence, followed by supporting details about what annotations provide, and concludes with a practical example. Every sentence earns its place with no redundant information, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters with nested objects), 100% schema coverage, and presence of an output schema, the description is reasonably complete. It explains the purpose and provides an example, though it could benefit from mentioning how these annotations affect tool usage in practice or referencing the output schema's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds value by providing a concrete example that illustrates how both parameters (toolRef and notes) work together in practice, showing the structure of annotations with specific field examples. This enhances understanding beyond the schema's technical definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Add contextual annotations'), target resource ('to a tool in the current toolset'), and purpose ('to guide LLM usage'). It distinguishes this from sibling tools like 'list-available-tools' or 'build-toolset' by focusing on annotation management rather than tool discovery or set construction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('to guide LLM usage') and includes an example annotation structure. However, it doesn't explicitly state when NOT to use it or mention alternatives like modifying existing annotations versus adding new ones, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build-toolsetA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It clearly indicates this is a creation/save operation ('Build and save'), mentions the auto-equip behavior, and provides an example. However, it doesn't disclose important behavioral aspects like whether duplicate toolset names are allowed, what happens on failure, or if there are rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement, helpful analogy, essential requirements, and a concrete example. Every sentence adds value without redundancy, and the example demonstrates proper usage while reinforcing key concepts.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (creation operation with 4 parameters) and the presence of an output schema, the description provides good coverage of the core functionality. It explains the purpose, requirements, and includes a helpful example. However, with no annotations, it could better address behavioral aspects like error conditions or constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds meaningful context beyond the schema by explaining the purpose of tool selection ('pick the exact tools you need'), providing a concrete example with specific tool names, and emphasizing the requirement to specify tools with either namespacedName or refId.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Build and save a custom toolset') and resource ('by selecting specific tools'), distinguishing it from siblings like delete-toolset or equip-toolset. The workshop analogy reinforces the core purpose of assembling tools for specific tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for a specific task or workflow') and mentions using list-available-tools to see options, but doesn't explicitly state when NOT to use it or contrast it with alternatives like add-tool-annotation for modifying existing toolsets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-available-toolsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover key behavioral traits (read-only, non-destructive, idempotent, closed-world), so the description adds value by explaining the return structure ('structured data showing tools grouped by server') and referencing methods ('namespacedName' or 'refId'). It does not disclose additional aspects like rate limits or auth needs, but does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. Each sentence adds value: the first defines the action, the second explains the return data and referencing methods, and the third provides a usage example, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters), rich annotations, and presence of an output schema, the description is complete. It covers purpose, return structure, and usage example, leaving output details to the schema, making it fully adequate for the agent's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema description coverage, the baseline is high. The description compensates by explaining that no parameters are needed ('Call with no parameters'), which clarifies usage beyond the empty schema, though it does not add semantic details about non-existent parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Discover all tools available from connected MCP servers') and resource ('tools'), distinguishing it from sibling tools like 'list-saved-toolsets' which deals with saved toolsets rather than available tools. It provides concrete examples of how tools can be referenced, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool ('to see all tools organized by server with detailed metadata'), including an example call with no parameters. However, it does not explicitly state when not to use it or name alternatives among siblings, such as 'get-active-toolset' for currently equipped tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.