Server Quality Checklist

Tool Scores

• get-documentationA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context about search flexibility (title, ID, partial matches) and batch processing capabilities (single or array inputs). However, it doesn't cover important aspects like response format, error handling, or performance considerations (e.g., rate limits), leaving gaps for a retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It front-loads the core purpose, then details search methods, and ends with usage guidance. Each sentence adds value, though the search methods section could be slightly more concise. No wasted text, but minor room for tightening.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete for a retrieval tool. It covers purpose, search methods, and usage workflow adequately. However, it lacks details on return values (format, structure), error cases, or limitations, which are important for a tool with flexible input options and no structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'section' parameter thoroughly. The description adds marginal value by reiterating support for 'single string or array of strings' and mentioning search methods (title, ID, partial matches), but doesn't provide additional syntax, format details, or examples beyond what the schema specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieves full documentation content for Macroforge sections.' It specifies the verb ('retrieves'), resource ('documentation content'), and scope ('Macroforge sections'). However, it doesn't explicitly differentiate from sibling tools like 'list-sections' or 'get-macro-info' beyond mentioning 'list-sections' in usage context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'After calling list-sections, analyze the use_cases and fetch ALL relevant sections at once.' It suggests a workflow with 'list-sections' as a precursor and recommends batching fetches. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings like 'get-macro-info'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-macro-infoA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes what the tool returns (information about macros, decorators, options) and the two usage modes, but it lacks details on potential side effects, error handling, rate limits, or authentication needs. For a read-only tool, this is adequate but minimal, as it doesn't fully compensate for the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise, with no wasted words. It starts with the core purpose, lists what information is returned, and provides clear usage instructions in two bullet points. Every sentence adds value, and it is front-loaded with essential information, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple read operation with one optional parameter) and the absence of annotations and output schema, the description is moderately complete. It explains what the tool does and how to use it, but it doesn't detail the return format, error cases, or how it differs from sibling tools. For a tool without output schema, more information on return values would be beneficial, but it's adequate for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'name' parameter documented as 'Optional: specific macro or decorator name to look up.' The description adds value by explaining the semantics: using no parameters returns a full manifest, while using the name parameter retrieves specific info. However, it doesn't provide additional details beyond what the schema already covers, such as format examples or constraints, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get documentation for Macroforge macros and decorators.' It specifies the resource (macros and decorators) and verb (get documentation), making the intent unambiguous. However, it does not explicitly differentiate from sibling tools like 'get-documentation' or 'list-sections,' which might have overlapping purposes, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: 'Use without parameters to get the full manifest... Use with a name parameter to get info for a specific macro or decorator.' This explains when to use each mode, but it does not mention when to choose this tool over alternatives like 'get-documentation' or 'list-sections,' nor does it specify any prerequisites or exclusions, so it's not fully comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• expand-codeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool returns (expanded code, diagnostics, help text) and its purpose for debugging, but lacks details on error handling, performance implications, or any side effects. It does not contradict annotations, but could benefit from more behavioral context like rate limits or memory usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose in the first sentence. Each subsequent section ('Shows:', 'Useful for:') adds value without redundancy, and the bullet points are concise. There is no wasted text, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (transforming code with diagnostics) and no output schema, the description does a good job explaining the return values (expanded code, diagnostics, help text). However, it could be more complete by mentioning potential limitations or edge cases, such as handling invalid input or the scope of macro expansion. With no annotations, it adequately covers the tool's purpose and output but leaves some behavioral gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('code' and 'filename'). The description does not add any additional semantic details beyond what the schema provides, such as explaining the format of 'code' or when 'filename' is necessary. Baseline score of 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Expands Macroforge macros in TypeScript code') and resource ('TypeScript code'), distinguishing it from siblings like 'get-documentation' or 'get-macro-info' by focusing on transformation rather than information retrieval. It explicitly mentions the output ('returns the transformed result') and the context of @derive decorators.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for: Seeing what code the macros generate, Understanding how @derive decorators transform your classes, Debugging macro expansion issues'), which helps differentiate it from siblings. However, it does not explicitly state when not to use it or name specific alternatives, such as when to prefer 'macroforge-autofixer' for fixing issues instead of just expanding.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• macroforge-autofixerA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it returns structured JSON diagnostics with specific fields (level, message, location, help, notes, summary) and details what it detects (e.g., invalid macro names, malformed decorators, syntax errors). However, it doesn't mention potential side effects like rate limits, authentication needs, or performance characteristics, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, starting with the core purpose and key output details. Most sentences add value, such as listing diagnostic fields and detection capabilities. However, the list of detections is somewhat verbose and could be more concise without losing clarity, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (validation with diagnostics) and no output schema, the description is mostly complete: it explains the purpose, usage guidelines, behavioral output (JSON structure), and detection scope. It lacks details on error handling or edge cases, but for a tool with no annotations and 100% schema coverage, it provides sufficient context for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('code' and 'filename'). The description does not add any specific meaning or usage details beyond what the schema provides, such as examples or constraints on the 'code' parameter. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Validates TypeScript code with @derive decorators using Macroforge's native validation.' It specifies the verb (validates), resource (TypeScript code with @derive decorators), and method (Macroforge's native validation), distinguishing it from sibling tools like 'expand-code' or 'get-documentation' which serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: 'This tool MUST be used before sending Macroforge code to the user' and 'If require_another_tool_call_after_fixing is true, fix the issues and validate again.' It also implies when to use it (for validation before sharing code) and suggests an alternative workflow (re-validation after fixing), though it doesn't name specific sibling alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-sectionsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return format (title, use_cases, path, category) and workflow behavior, but lacks details on potential limitations like pagination, error handling, or performance characteristics. It doesn't contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (description, returns, workflow, example), front-loaded with the core purpose, and every sentence adds value without redundancy. It efficiently communicates necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, no annotations, and no output schema, the description provides good context by detailing the return structure and workflow. However, it could be more complete by mentioning any prerequisites (e.g., authentication) or limitations, though the lack of complexity keeps it adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description appropriately adds no parameter information, as none are needed, and focuses on output semantics instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Lists' and the resource 'all Macroforge documentation sections', making the purpose specific and unambiguous. It distinguishes this tool from siblings like 'get-documentation' by focusing on listing sections rather than retrieving content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The WORKFLOW section provides explicit guidance: 'Call list-sections FIRST for any Macroforge-related task', followed by steps for analysis and calling 'get-documentation'. It clearly states when to use this tool versus alternatives, with a named sibling tool mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.