Server Quality Checklist

Tool Scores

• cwtools-configA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true and openWorldHint=true, indicating safe read operations with open-world assumptions. The description adds valuable context beyond annotations: it specifies the source (CWTools for Stellaris), the file types (.cwt files), version options (stable/latest), and directory structure (config/ and subdirectories like common/). This enhances understanding of the tool's behavior without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences. The first sentence states the core purpose, and the second elaborates on versions and file retrieval options. Every sentence adds value without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema) and rich annotations (readOnlyHint, openWorldHint), the description is fairly complete. It covers what the tool does, source details, and retrieval options. However, it could improve by mentioning potential outputs (e.g., file contents or listings) or error cases, but annotations help mitigate gaps. Sibling tools provide some context, but the description stands adequately on its own.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for all parameters (file, listOnly, version). The description adds some semantic context by mentioning '特定のファイルまたは全ファイル' (specific files or all files) and 'config/直下およびサブディレクトリ' (config root and subdirectories), which aligns with the 'file' parameter. However, it doesn't provide additional details beyond what the schema already covers, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'CWToolsのStellarisコンフィグルール(.cwt files)を取得します' (retrieves CWTools Stellaris config rules). It specifies the resource (.cwt files) and the scope (from stable or development versions, specific files or all files). However, it doesn't explicitly differentiate from sibling tools like 'stellaris-search' or 'stellaris-validate', which might have overlapping domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning retrieval from '最新の安定版または開発版' (latest stable or development versions) and files in 'config/直下およびサブディレクトリ' (config root and subdirectories). However, it lacks explicit guidance on when to use this tool versus alternatives like 'stellaris-search' (which might search within configs) or 'stellaris-validate' (which might validate configs), and does not specify exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stellaris-searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and openWorldHint=true, indicating safe read operations with open-ended data. The description adds valuable behavioral context beyond annotations by specifying that it 'returns matches as code snippets' and 'efficiently searches large files,' which helps the agent understand output format and performance characteristics without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two concise sentences that efficiently convey the tool's purpose and key capabilities ('search documentation' and 'return code snippets efficiently'). Every sentence adds value without redundancy or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema), the description is mostly complete. It covers the purpose, resource, and behavioral traits, but lacks details on output format beyond 'code snippets' (e.g., structure or examples). With annotations providing safety hints, it's adequate but could be slightly enhanced for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters (keyword, docType, contextLines). The description does not add any parameter-specific semantics beyond what the schema provides, such as explaining parameter interactions or usage nuances, meeting the baseline score of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('search documentation from the OldEnt/stellaris-triggers-modifiers-effects-list repository') and resource ('Stellaris latest version documentation'), distinguishing it from sibling tools like cwtools-config, stellaris-validate, and stellaris-version by focusing on keyword search rather than configuration, validation, or version checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('search Stellaris documentation for keywords'), but does not explicitly state when not to use it or name specific alternatives among the sibling tools. It implies usage for efficient searching of large files, which is helpful but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stellaris-validateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, indicating a safe, read-only operation with open-world semantics. The description adds valuable context about checking 'validity in the latest version of Stellaris' and 'modding compatibility,' which clarifies the tool's behavioral scope beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of two sentences that efficiently convey the tool's purpose and usage context without any wasted words. Every sentence adds clear value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema), the description is mostly complete. It explains what the tool does and its context, but lacks details on return values or error handling. With annotations covering safety and semantics, and schema covering parameters, the description is adequate but could be slightly enhanced.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters (docType and element). The description does not add significant meaning beyond the schema, as it only mentions the parameter types without additional syntax or format details. Baseline 3 is appropriate given the comprehensive schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check'/'validate') and resources ('triggers, effects, modifiers, scopes, localizations'), and distinguishes it from siblings by mentioning 'modding compatibility verification' which suggests a validation focus different from search or configuration tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage ('useful for modding compatibility verification'), but does not explicitly state when not to use it or name alternatives among sibling tools. It implies usage during modding but lacks exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stellaris-versionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true and openWorldHint=true, indicating this is a safe read operation with no side effects. The description adds valuable context beyond annotations by specifying what information is retrieved (build ID, last update time, version numbers) and its purpose for modding. It does not contradict annotations, as '取得します' (get) aligns with read-only behavior. However, it lacks details on rate limits or authentication needs, though these may not be critical for this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the core purpose, and the second elaborates on the specific information provided. Every sentence adds value by clarifying scope and utility, with no wasted words or redundancy. It is front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, read-only annotations, no output schema), the description is complete enough for an agent to understand its function and use case. It covers what the tool does, what information it returns, and the context (modding). However, without an output schema, it could benefit from more detail on the return format, though the annotations and lack of parameters mitigate this gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, focusing instead on the tool's output and purpose. This meets the baseline for tools with no parameters, as it avoids unnecessary repetition of schema information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'ステラリスの現在のバージョン情報を取得します' (Get current version information for Stellaris). It specifies the verb '取得します' (get/retrieve) and resource 'バージョン情報' (version information), and distinguishes from siblings by mentioning modding-specific details like build ID, last update time, and version numbers, which none of the sibling tools (cwtools-config, stellaris-search, stellaris-validate) appear to cover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating it provides 'modding に必要な情報' (information necessary for modding), suggesting this tool is for modding-related tasks. However, it does not explicitly state when to use this tool versus alternatives like stellaris-search or stellaris-validate, nor does it provide exclusions or prerequisites. The guidance is present but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.