Server Quality Checklist

Tool Scores

• get_moduleA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description alone must convey behavioral traits. It indicates a read operation but does not mention error handling, permissions, or that it is read-only. The description is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the main purpose, and has no unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with two well-documented parameters, the description covers the essential functionality. The absence of an output schema is acceptable as the return type (detailed module content) is implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema describes parameters completely. The description adds minimal value by listing section types already in the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed content from a specific module, with an optional section filter. It distinguishes itself from siblings like list_modules (which lists modules) and get_source_code (which retrieves code).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide guidance on when to use this tool versus alternatives. It lacks context for choosing between get_module and other sibling tools like query_architecture or trace_concern.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must supply behavioral context. Describes return data but omits details like sorting, pagination, or empty result handling. Adequate for a simple listing tool, but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff. First sentence states purpose and output; second provides usage guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description reasonably hints at return structure (IDs, titles, concerns). Covers filtering by category. Lacks details on sorting or limits, but sufficient for typical discovery use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'category' with enum values. Schema already fully describes it (100% coverage). Description adds minor clarification (M01-M22, SUPP-*), but not substantial beyond schema. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states action 'List all available architecture modules' with specific return fields (IDs, titles, key concerns). Differentiates from siblings like get_module (single) and search_patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to discover what knowledge is available before querying specific modules,' providing clear context. Could be improved by mentioning when not to use (e.g., for specific details use get_module), but the intent is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trace_concernA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It states the tool 'Shows how one concept is handled at different layers', giving a behavioral hint but lacks details on output format, side effects, or whether the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences, no filler—with the purpose front-loaded and immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is nearly sufficient, though mentioning the return type or that it is read-only would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'concern', and the description does not add further semantics beyond what the schema already provides, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('trace') and resource ('architectural concern across all modules') and provides concrete examples ('prompt cache', 'error recovery') that distinguish it from sibling tools which focus on modules, source code, or patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool (to see how a concept spans layers), but it does not explicitly mention when not to use it or direct to alternatives, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_source_codeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses key behaviors: 'Supports line-range extraction' and 'Returns at most 500 lines per call.' This is good for a read tool, though it doesn't mention any potential limits on file paths (e.g., only within src/) or error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences (20 words total) convey the purpose, usage context, and a key constraint. No wasted words; front-loaded with the main verb and resource. Perfectly concise for this simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with 3 parameters and no output schema, the description is largely complete. It covers what the tool does, when to use it, and a key behavioral constraint. It could mention the return format (plain text?), but that is not essential given the simplicity. Slight gap: no mention of error handling or whether file path must be relative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema already describes all three parameters with clear descriptions. The description adds value only by summarizing ('Supports line-range extraction'), which is useful but not essential beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'read' and explicitly states the resource ('actual Claude Code source code files'). It distinguishes from sibling tools like 'get_module' (module abstraction) and 'list_modules' by indicating this is for raw implementation details referenced in module analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this to see implementation details referenced in module analysis.' This implies when to use it relative to siblings. However, it does not explicitly state when not to use it or mention alternatives, so slight room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_architectureA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool searches and returns results at requested depth, which covers core behavior. However, it does not disclose potential limitations (e.g., no results, rate limits) or explicitly confirm read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two sentences covering purpose and usage, plus an example-based parameter guide. No superfluous words, and front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a search tool given no output schema: it explains what is returned (module analysis at depth) and how to use parameters. However, it could briefly mention the return format (e.g., text summaries) to fully inform the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, baseline is 3, but the description adds significant value: for 'query' it provides examples; for 'depth' it explains each enum value with detail; for 'modules' it clarifies optionality and format. This greatly aids correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the architecture knowledge base and returns module analysis at a requested depth. It distinguishes from siblings by specifying 'features, patterns, or subsystems', which are not covered by sibling tools like search_patterns or trace_concern.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'Use this to understand how specific features, patterns, or subsystems are implemented.' This tells when to use the tool but lacks explicit exclusions or alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_patternsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses pagination, character and section caps, and hints to fetch full content via get_module. Since no annotations are present, this adequately informs about expected behavior without missing critical details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. First sentence states core function and return type; second explains pagination and module usage. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers key behavioral aspects (pagination, caps, get_module link) given no output schema and 5 parameters. Missing explicit return format or error handling, but overall sufficient for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds value beyond the schema by explaining defaults (800 chars, 12 sections), the purpose of module_id for focusing, and the get_module fallback. Schema coverage is 100%, but description enriches understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'search' and specific resource 'reusable design patterns and principles'. Distinguishes from sibling get_module by explicitly suggesting its use for full content retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use (searching patterns) and how to focus (module_id). Mentions get_module as alternative for full text, but no explicit 'when not to use' or list of all alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.