Server Quality Checklist

Tool Scores

• soul_getB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds value by specifying 'detailed information', which hints at richer output beyond basic data, but doesn't disclose behavioral traits like rate limits, authentication needs, or error handling. With annotations covering safety, this is adequate but not rich in additional context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It's front-loaded and every word earns its place, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 required parameters, no output schema), the description is minimally complete. It states what the tool does but lacks context on output format, error cases, or integration with siblings. With annotations covering read-only behavior, it's adequate but could be more informative for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for 'owner' and 'name' parameters. The description doesn't add any semantic details beyond what the schema provides, such as examples or constraints. Baseline 3 is appropriate since the schema fully documents parameters, and no extra value is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'detailed information about a specific persona', making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'soul_search' or 'soul_scan', which prevents a perfect score, but the specificity of 'detailed information about a specific persona' is sufficient for clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'soul_search' or 'soul_scan'. It mentions 'specific persona', which implies it's for retrieving details of a known persona, but lacks explicit when-to-use or when-not-to-use instructions, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• soul_installA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false, implying a write operation, which aligns with the description's 'Download' and 'generate' actions. The description adds context about downloading from ClawSouls and generating a file, but does not disclose behavioral traits like authentication needs, rate limits, or side effects beyond what annotations provide. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and outcome without unnecessary words. Every part of the sentence ('Download a persona from ClawSouls and generate a CLAUDE.md file') contributes directly to understanding the tool's purpose, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a write operation with 3 parameters) and the absence of an output schema, the description is minimally adequate. It covers the basic action but lacks details on the generated file's format, location, or any error conditions. With annotations providing some safety context, it meets a baseline level of completeness but leaves gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for owner, name, and output_dir parameters. The description does not add meaning beyond the schema, such as explaining what a 'Soul owner' or 'Soul name' represents in ClawSouls context, but the high schema coverage justifies the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Download a persona from ClawSouls') and the outcome ('generate a CLAUDE.md file'), using precise verbs and identifying the resource. It distinguishes itself from sibling tools like soul_get, soul_search, and soul_scan by focusing on installation and file generation rather than retrieval, searching, or scanning operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or comparisons to sibling tools such as soul_get (which might retrieve persona details) or soul_search (which might find personas), leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• soul_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds context about searchable fields (keyword, category, tag), which is useful beyond annotations. However, it lacks details on behavioral traits like pagination, rate limits, or result format, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and scope without unnecessary words. Every part of the sentence contributes directly to understanding the tool's purpose, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with three parameters), lack of output schema, and annotations covering safety, the description is adequate but incomplete. It explains what the tool does but omits details on output format, error handling, or interaction with siblings, which could aid the agent in proper invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented in the schema. The description mentions 'keyword, category, or tag', aligning with the 'query' and 'category' parameters, but adds no additional semantic context beyond what the schema provides. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search'), the resource ('AI agent personas on ClawSouls'), and the methods ('by keyword, category, or tag'), making the purpose specific and actionable. It distinguishes from siblings like 'memory_search' by specifying the resource type (personas vs. memories).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching personas, but does not explicitly state when to use this tool versus alternatives like 'soul_get' (which might retrieve a specific persona) or 'soul_scan' (which might scan broadly). No exclusions or prerequisites are mentioned, leaving usage context somewhat open-ended.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• soul_rollback_checkA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds useful context about what the tool does (comparison-based drift detection) and what it returns (drift severity and recommended actions), which goes beyond the annotations. However, it doesn't disclose additional behavioral traits like rate limits, error conditions, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are front-loaded with the core purpose and efficiently convey the action, inputs, and outputs without any wasted words. Every sentence earns its place by adding value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (comparison-based analysis with two object parameters), the description is complete enough for understanding its purpose and usage. It explains what the tool does and what it returns, though without an output schema, more detail on return values (e.g., structure of severity and actions) could be helpful. Annotations cover safety, and schema covers parameters well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (current_files and original_files) well-described in the schema as objects mapping filenames to content. The description doesn't add any parameter-specific semantics beyond what the schema provides, such as format details or constraints, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Detect persona drift by comparing'), the resource ('current Soul Spec files against their original committed versions'), and the output ('Returns drift severity and recommended actions'). It distinguishes this tool from siblings like soul_scan or soul_search by focusing on comparison-based drift detection rather than scanning or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to detect drift in Soul Spec files, but it doesn't explicitly state when to use this tool versus alternatives like soul_scan or provide exclusions. The context is clear (comparing current vs. original files), but no explicit guidance on when-not-to-use or named alternatives is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable context beyond this by specifying what information is returned (files, sizes, dates, git status) and that it shows 'current status'—implying real-time data without modification. No contradictions with annotations exist, and the description enhances understanding of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('show current status') and lists all key details without redundancy. Every word contributes to understanding the tool's function, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, read-only operation) and lack of output schema, the description adequately covers what the tool does and what information it returns. It could be more complete by mentioning output format or error handling, but for a status-checking tool with good annotations, it provides sufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'memory_dir' fully documented in the schema. The description does not add any parameter-specific information beyond what the schema provides, such as default behavior or path examples. However, with high schema coverage and only one parameter, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('show current status') and resource ('agent memory files'), with explicit details about what information is included ('list files, sizes, last modified dates, and git status'). It distinguishes itself from siblings like memory_detail, memory_search, and memory_sync by focusing on status overview rather than detailed inspection, searching, or synchronization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking memory file status, but does not explicitly state when to use this tool versus alternatives like memory_detail (for detailed file info) or memory_search (for finding specific content). No exclusions or prerequisites are mentioned, leaving usage context somewhat open to interpretation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_syncA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false, aligning with the description's 'sync' actions that involve writes (e.g., push, init). The description adds context by explaining what each action does (e.g., 'push: commit & push local changes'), which goes beyond annotations. However, it lacks details on error handling, rate limits, or authentication needs for the Git repo, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by a concise list of actions. Every sentence earns its place by clarifying functionality without redundancy, making it efficient and easy to parse for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, 4 actions) and rich schema coverage (100%), the description is mostly complete. It explains the tool's role and actions but lacks output details (no output schema) and does not cover edge cases like merge conflicts or network failures. With annotations providing some safety context, it's adequate but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter well-documented in the schema (e.g., action enum descriptions). The description adds minimal value beyond the schema by listing the actions but does not provide additional semantics like format examples or constraints. Baseline 3 is appropriate given the comprehensive schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Sync agent memory files with a remote Git repository for multi-agent Swarm Memory.' It specifies the verb ('sync'), resource ('agent memory files'), and context ('multi-agent Swarm Memory'), distinguishing it from sibling tools like memory_search or memory_status that likely query rather than synchronize.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by listing the four actions (init, push, pull, status) and their purposes, helping an agent understand when to use each. However, it does not explicitly state when to choose this tool over alternatives (e.g., vs. manual Git commands or other sync tools) or mention any prerequisites, such as needing Git installed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• soul_scanA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, indicating a safe read operation. The description adds valuable behavioral context beyond annotations: it specifies the analysis scope (53 safety patterns), output format (grade A+ to F), and that it provides actionable recommendations. This enhances transparency about what the tool does and returns, though it doesn't mention rate limits or authentication details beyond the optional API key.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded and concise with two sentences that efficiently convey purpose, scope, and output. Every sentence adds value: the first defines the action and target, the second details the analysis and results. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (safety analysis with multiple patterns), rich annotations (readOnlyHint), and high schema coverage, the description is mostly complete. It explains the analysis process and output format well. However, without an output schema, it could benefit from more detail on the return structure (e.g., JSON format of recommendations), but the mention of 'actionable recommendations' partially compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal parameter semantics beyond the schema—it implies the files parameter is for Soul Spec files and mentions the API key is for premium rules, but this is redundant with schema descriptions. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Run', 'Analyzes') and resources ('SoulScan safety verification on Soul Spec files'). It distinguishes from sibling tools by focusing on safety analysis rather than memory operations or other soul-related functions like installation or rollback.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (analyzing persona files for safety) but doesn't explicitly state when to use this tool versus alternatives. It mentions 'premium rules' with an optional API key, suggesting a use case for enhanced analysis, but lacks clear guidance on when to choose this over other soul-related tools like soul_search or soul_get.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_detailA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, which the description aligns with by using 'Fetch' (a read operation). The description adds context about using it after search results, but does not disclose additional behavioral traits like error handling, rate limits, or output format. With annotations covering safety, this is adequate but not rich in extra details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the purpose, and the second provides usage guidelines. It is front-loaded with the core function and efficiently conveys essential information without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 4 parameters), annotations cover safety (readOnlyHint), and schema coverage is 100%, the description is mostly complete. However, with no output schema, it does not explain return values or format, leaving a minor gap in full context for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description does not add meaning beyond the schema, as it does not explain parameter interactions or usage nuances. Baseline 3 is appropriate when the schema handles parameter documentation effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and the resource 'full content of a specific memory section,' distinguishing it from sibling tools like memory_search (which searches) and memory_status (which checks status). It specifies retrieving detailed content after search results, making the purpose specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use after memory_search to get details for high-scoring results,' providing clear when-to-use guidance and naming the alternative tool (memory_search) for context. This helps the agent understand the workflow and avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_searchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds valuable context beyond this: it specifies the ranking method ('TF-IDF ranking'), output format ('compact index'), token usage implications ('uses more tokens' for enhanced mode), and file scope. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with core functionality and efficiently structured in three sentences with zero waste. Each sentence adds distinct value: the first defines the tool's purpose, the second guides usage with alternatives, and the third clarifies scope. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, rich annotations (readOnlyHint), and full schema coverage, the description is mostly complete. It covers purpose, usage, and behavioral context well. However, without an output schema, it could benefit from more detail on return values (e.g., format of 'compact index'), slightly limiting completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds minimal semantic value beyond the schema, such as hinting at the enhanced parameter's token impact, but does not provide significant additional meaning. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search agent memory using TF-IDF ranking'), the resource ('MEMORY.md + memory/*.md files'), and the output format ('compact index of matching sections with relevance scores'). It explicitly distinguishes from sibling 'memory_detail' by stating 'Use memory_detail to fetch full content of interesting results,' showing clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('Search agent memory') versus alternatives ('Use memory_detail to fetch full content of interesting results'). It also implies context by specifying the scope ('Searches MEMORY.md + memory/*.md files'), giving clear direction for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.