Server Quality Checklist

Tool Scores

• axom_mcp_transformA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable context beyond annotations by detailing supported formats, transformation rules, template support, and chain operations. Annotations already indicate it's read-only, non-destructive, and idempotent, but the description clarifies what 'transformation' entails (e.g., field mapping, filtering, sorting) and mentions advanced features like Jinja2 templates and chaining, which enhance behavioral understanding without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, using bullet points and clear sections ('Supported Formats', 'Transformation Rules', 'Template Support', 'Chain Support') to organize information efficiently. Each sentence adds value without redundancy, making it easy to scan and understand the tool's capabilities quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, multiple formats and rules) and rich annotations, the description is largely complete. It covers key aspects like formats, rules, templates, and chaining. However, without an output schema, it does not describe return values or error handling, which could be helpful for a transformation tool. The annotations provide safety context, but output details are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents all 6 parameters thoroughly. The description adds some semantic context by listing formats and rules that correspond to parameters like 'input_format', 'output_format', and 'rules', but does not provide additional syntax, examples, or details beyond what the schema specifies. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Transform data between formats and structures') and lists the supported formats and transformation rules. It distinguishes itself from sibling tools like 'analyze', 'discover', 'exec', and 'memory' by focusing specifically on data transformation rather than analysis, discovery, execution, or memory operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the listing of supported formats and transformation rules, suggesting this tool should be used for data format conversion and restructuring tasks. However, it does not explicitly state when to use this tool versus alternatives like 'axom_mcp_analyze' for data analysis or provide specific exclusions or prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• axom_mcp_discoverA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=false. The description adds valuable behavioral context beyond annotations: it explains what 'chain parameter' does ('act on discovered resources'), describes filtering capabilities (glob patterns, type filters), and mentions 'comprehensive discovery' for the 'all' domain. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Discovery Domains, Filter Options, Chain Support) and uses bullet points for readability. It's appropriately sized for a multi-domain discovery tool, though the 'Filter Options' section lists options not directly mapped to schema parameters (e.g., 'importance' isn't a schema parameter), creating minor redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, multiple domains) and rich annotations (4 hints), the description provides good contextual completeness. It explains domains, filtering, and chaining behavior. The main gap is lack of output schema, so return values aren't described, but the discovery focus makes this less critical than for mutation tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters. The description adds some semantic context: it explains what 'domain' values mean (files=list/search, tools=list MCP tools, etc.), mentions filter options (pattern, type, memory_type, importance), and clarifies 'chain' purpose. However, it doesn't provide significant additional parameter details beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover available resources, structures, and capabilities' with specific domains enumerated (files, tools, memory, capabilities, all). It distinguishes from siblings like 'analyze', 'exec', 'memory', and 'transform' by focusing on discovery rather than analysis, execution, or transformation operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool by listing discovery domains and filter options. It doesn't explicitly state when NOT to use it or name specific alternatives among siblings, but the domain-based structure implies usage scenarios (e.g., use for 'files' discovery vs. 'memory' operations with axom_mcp_memory).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• axom_mcp_execA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations. Annotations indicate destructiveHint=true and readOnlyHint=false, but the description elaborates with security details (allowed directories, input size limits), chain-reaction functionality with variable substitution, and specific conditions for write/shell operations. This provides valuable operational context that annotations alone don't cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Operations, Chain Reactions, Example, Security) and front-loads the core purpose. Most sentences add value, though the example could be slightly more concise. Overall, it efficiently communicates complex functionality without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and complex functionality (multiple operations, chaining, security constraints), the description provides substantial context. It covers operation types, chaining mechanics, security restrictions, and usage conditions. The main gap is lack of information about return values or error handling, but given the annotations and detailed operational guidance, it's reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds some value by explaining the 'chain' parameter's purpose and providing an example, and it clarifies that 'target' serves as either a file path or command depending on operation. However, it doesn't significantly enhance understanding of parameters beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs and resources: 'Execute file operations and shell commands with chain-reaction support.' It explicitly lists the three operation types (read, write, shell) and distinguishes this tool from its siblings by emphasizing its execution capabilities, unlike analyze, discover, memory, or transform tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for file operations and shell commands with chain support. It mentions that write and shell operations are disabled when AXOM_READ_ONLY=true, offering some exclusion guidance. However, it doesn't explicitly compare to sibling tools or provide detailed alternatives for specific scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• axom_mcp_memoryA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond what annotations provide. While annotations indicate destructiveHint=true (for delete operations), the description clarifies the full range of actions including non-destructive ones (read, list, search). It also provides memory type definitions, naming conventions, content formats, and default expiration policies - all useful behavioral information not captured in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Memory Types, Naming Convention, Content Format, Actions) and front-loads the core purpose. While comprehensive, some information could be more concise - the memory type descriptions are somewhat verbose. Overall, it's efficiently organized with minimal wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 9 parameters, no output schema, and rich annotations, the description provides substantial context about memory types, naming conventions, content formats, and action behaviors. However, it doesn't describe return values or error conditions, which would be helpful given the absence of an output schema. The tool's complexity warrants this level of detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents all 9 parameters thoroughly. The description adds some semantic context about memory types and naming conventions, but doesn't provide additional parameter-specific guidance beyond what's already in the schema descriptions. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose with specific verbs ('Store, retrieve, search, and manage persistent memories') and resource ('in the Axom database'). It clearly distinguishes this memory management tool from its siblings (analyze, discover, exec, transform) which appear to have different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use different actions (write, read, list, search, delete) and memory types, but doesn't explicitly state when NOT to use this tool versus its sibling tools. It offers good internal guidance but lacks cross-tool comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• axom_mcp_analyzeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only, non-destructive, and idempotent behavior, which the description does not contradict. The description adds valuable context beyond annotations by detailing analysis types, focus areas, depth levels, and chain support, providing insight into how the tool behaves and what users can expect from its output. It doesn't mention rate limits or specific auth needs, but with annotations covering safety, this is sufficient for a high score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, using bullet points and clear headings to organize information efficiently. Every sentence and section earns its place by adding specific value, such as defining analysis types and depth levels, without unnecessary repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity with multiple parameters and no output schema, the description does a good job of explaining what the tool does and how to configure it. It covers analysis types, focus areas, depth levels, and chain support, which helps users understand the tool's capabilities. However, without an output schema, it could benefit from more details on return values or result formats, slightly limiting completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context by explaining the meaning of analysis types, focus areas, and depth levels, which complements the schema's enum descriptions. However, it doesn't provide additional syntax or format details beyond what the schema offers, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Analyze code and data with configurable depth and scope,' which is a specific verb+resource combination. It distinguishes itself from sibling tools like 'discover,' 'exec,' 'memory,' and 'transform' by focusing exclusively on analysis rather than discovery, execution, memory operations, or transformations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage through detailed breakdowns of analysis types, focus areas, and depth levels, helping users understand when to apply different configurations. However, it lacks explicit guidance on when to use this tool versus its siblings, such as distinguishing analysis from transformation or execution tasks, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.