Server Quality Checklist

Tool Scores

• skill_costsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key trait: the described costs are for always-loaded descriptions, not optional invocations. It also implies read-only behavior. However, it doesn't detail output structure or whether the list is sorted descending (though 'top' implies it). Reasonable but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the main purpose and followed by actionable insight. No wasted words; every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple query tool with no output schema, the description sufficiently explains what it returns (ranked skills with token costs). It does not explicitly state whether token counts are included or the sort order, but 'top of this list' implies descending order. Most users would find it complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (limit, filter) having descriptions in the schema. The tool description adds no parameter-specific information, so it relies on the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool ranks skills by the token cost of their always-loaded descriptions, with a specific verb ('ranked') and resource ('skills'). This distinguishes it from sibling tools like duplicate_skills (which finds duplicates) and context_budget (which measures overall budget).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'The top of this list is where trimming pays off most,' which gives clear guidance on when to use the tool (to identify costly skills for trimming). While it doesn't name alternatives or exclusions, the use case is strong and implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• context_budgetA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains the tool is a read-only style overview but does not disclose return format, pagination, or any side effects. This is adequate for a simple query tool but not highly detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the key answer and a direct usage directive. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description covers the key aspects: what it computes, by which breakdown, and when to use it. It is slightly thin on what precisely the numbers mean, but sufficient for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100%, so there is nothing to explain. The description adds meaning by clarifying the output's breakdown by source, matching the 0-param baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides the headline context budget consumption before conversation starts, broken down by source. This is specific and distinct from sibling tools that drill into details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises 'Use this first', giving clear ordering guidance relative to sibling tools. It does not state when not to use it, but the directive is strong and practical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duplicate_skillsA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure. It explains the comparison mechanism (content hash), the outcome categories (identical vs. differing copies), and the recommended actions (safe to remove vs. flagged for review). This gives the agent a complete understanding of the tool's behavior and side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the definition of what the tool handles, followed by crucial behavioral details. Every word earns its place; no filler or redundancy exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description adequately covers the core behavior and implications. It explains the duplication source, cost impact, comparison method, and recommended actions. A brief mention of the return format (e.g., a list) would make it fully complete, but the description already provides sufficient context for correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty, so there are no parameter semantics to explain. The baseline for no parameters is 4, and the description correctly focuses on behavior rather than input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool's scope: 'Skills whose name appears more than once' identifies the resource and intent. It distinguishes from sibling tools by focusing specifically on duplicate skills, not budget, costs, memory, or servers. The lack of an explicit verb (e.g., 'list' or 'report') prevents a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to identify duplicate skills that incur full description cost. It explains the safety implications of removal ('identical copies are safe to remove') and when review is needed, giving the agent actionable guidance. It does not explicitly compare against alternatives, but sibling tools are topically distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_serversA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral transparency. It explicitly discloses that only environment variable NAMES are read and values are never read, providing a non-obvious privacy/safety guarantee. It also clarifies that the output includes which config declares each server, which is useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence states the core purpose, and the second adds a critical detail about environment variable values. There is no fluff, and the information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: no parameters and no output schema. The description adequately conveys what the tool returns (servers and their declaring configs) and includes a safety note. It does not explicitly describe the return format (e.g., list vs table), but for an inventory tool this is enough. Slightly more detail about the source of the config data would have made it a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so no parameter documentation is needed. Per the rubric, 0 parameters earns a baseline 4. There is no missing parameter information to penalize.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as an inventory of every MCP server configured across agent clients on the machine, plus which config declares each server. This specific verb-like framing ('Every MCP server configured...') distinguishes it from sibling tools like context_budget or skill_costs, and matches the title 'MCP server inventory'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: 'useful for spotting servers you forgot you installed.' This gives the agent context for when to use the tool, though it does not explicitly mention when not to use it or name alternative tools. That matches a 4: clear context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_filesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool never returns file contents, which is a significant behavioral guarantee. It explains the scope (all memory files loaded every session) and that it only reports sizes and import graph, adding context beyond safe operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences. The first sets the context (what files and the hidden cost), the second clearly states what the tool reports and what it excludes. Every clause adds value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description covers the essential behavior: the subject files, the load mechanism, the output (sizes and import graph), and the explicit exclusion of contents. It is sufficiently complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters; it appropriately focuses on output and scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports sizes and import graph of memory files, with a specific verb ('Reports') and resource (CLAUDE.md/AGENTS.md and imports). It distinguishes itself from sibling tools by focusing on memory file loading costs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: to understand the hidden cost of memory files (including @import pull-ins). It explicitly states what it does not do (never file contents), which provides a clear boundary, though it does not explicitly name alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.