Server Quality Checklist

Tool Scores

• validate_mod_configsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of behavioral disclosure. It does not state whether validation is read-only or if it modifies anything, whether it reports errors via return values or side effects, or any side effects like writing logs. The description lists checks but does not disclose the output format or how failures are surfaced. For a tool named 'validate' with no annotation safety profile, this gap is significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and resource, then enumerates the specific checks. Every phrase is informative with no fluff. It is concise and structured effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's validation action and no output schema, the description should clarify what constitutes a successful validation, what the response format is (e.g., a list of issues), and whether it is a read-only operation. The description lists issue categories but does not describe the return value or side effects. This is insufficient for an agent to know what to expect when invoking it. Sibling tools have varied purposes, so more contextual detail on the validation outcome would help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% since the single parameter configDir is described as 'Path to the config/ directory'. The description adds a bit of context by implying the directory contains mod config files, but does not clarify if it should be a relative/absolute path or any naming conventions. Since the schema already documents the parameter clearly, the description adds minimal extra value, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Validate' and the resource 'mod config files in a directory', and lists specific checks (empty values, invalid numbers, parse errors, oversized strings). It distinguishes from siblings by specifying a directory-level validation action, whereas read_mod_config and list_mod_configs operate individually or list, and compare/find-orphaned have different purposes. The purpose is clear but not fully explicit about the output or success/failure criteria, so not a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for validating config files, but does not state when to use it over alternatives. Siblings like read_mod_config or compare_mod_configs might be preferable for specific tasks, but no exclusions or alternative conditions are given. The context of validating for 'common issues' provides some guidance, but it lacks explicit when-not or comparison to other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_orphaned_configsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states 'find config files' which implies a read-only operation, but it never explicitly says it is read-only, whether it returns file paths or names, whether it recurses into subdirectories, or any edge cases (e.g., configs with no mod JAR mapping). The absence of output schema and side-effect disclosure leaves the agent underinformed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, no fluff. The first sentence states the action and condition; the second gives usage context. Perfectly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the purpose is clear and the two params are self-explanatory, the lack of output details and explicit read-only confirmation makes it incomplete for an agent. For a find operation, the return type (file paths, list, count) and any potential side effects (none expected) should be stated, especially given there are no annotations to carry that burden.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with clear descriptions ('Path to the mods/ directory,' 'Path to the config/ directory'). The description adds no extra semantics beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds config files whose mod JAR is missing, distinguishing it from siblings like list_mod_configs (lists all) and validate_mod_configs (validates). The specific verb 'find' plus the condition 'JAR no longer present' leaves no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear trigger context: 'Useful after removing mods.' This indicates when to call it, though it does not mention when *not* to use it or alternatives like read_mod_config to inspect a single config.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mod_configsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden — and it does add value by stating the returned fields (file path, mod ID, format, size), which tells a caller what to expect. Still, the description doesn't disclose anything about recursion depth, error behavior on bad directories, sorting, or whether this is a pure read (obvious for 'list' but not stated). Adding return-field semantics is useful, but the missing behavioral detail and absence of any mention of file count limits or permission requirements keeps it at the baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, minimal words, no filler or repetition. The core action is front-loaded ('List all mod config files...'), and the return contract follows immediately in the second sentence. Every word carries meaning without verbosity or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward read-only list tool with one well-documented parameter, the description covers both the input (config directory) and output (path, mod ID, format, size), which is the complete contract a caller needs. It misses only minor niceties like error behavior for invalid paths or what 'format' specifically refers to, but these are edge concerns that don't impede correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (single configDir parameter is described), meeting the baseline of 3. The description reinforces the parameter's purpose by defining the scope ('in a config directory'), but adds no additional semantic details about the parameter beyond that. The parameter is simple and adequately documented — no penalty, but the description doesn't exceed the schema's coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb+resource combination: 'List all mod config files in a config directory (recursively)' — the 'all' + 'recursively' specifies scope beyond a simple list, and the returned fields (path, mod ID, format, size) clarify what is returned. It stands apart from siblings like find_orphaned_configs (different task) and read_mod_config (singular vs. collection), though it doesn't explicitly name the sibling it is not.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: an agent can infer this is the tool to call when it needs an inventory of config files before reading/validating them. However, there is no explicit statement of when NOT to use it or which sibling to choose instead, so the guidance is implicit in the purpose rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_mod_configA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the tool reads and parses and returns a parsed config tree, implying a read-only operation, but it does not disclose error behavior on missing or malformed files, nor the exact structure of the returned tree.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences deliver the core action, format constraints, and return type without redundancy. The most important information is front-loaded, and every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple one-parameter tool, but there is no output schema or annotations to fill in the gaps. The phrase 'parsed config tree' is vague about the actual return shape, and error conditions are not covered, leaving some ambiguity for an agent deciding how to handle failures.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single configPath parameter, so the baseline is 3. The description adds meaningful format-to-loader mapping (.cfg for Forge, .toml for Fabric, .json for any) that goes beyond the schema's generic path description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('Read and parse a mod config file') and specifies supported formats (.cfg for Forge, .toml for Fabric, .json for any). This clearly distinguishes it from sibling tools like list_mod_configs or validate_mod_configs, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by naming when each format applies, but it does not explicitly state when to choose this tool over alternatives. It does not mention exclusions or refer to sibling tools such as get_mod_default_config or compare_mod_configs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_mod_configsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description states the tool reports added, removed, changed, and unchanged files with key-level diff info, which gives a good sense of the output. However, it does not disclose potential side effects (e.g., whether it modifies anything), performance implications, or error conditions. Since it's a read-only comparison tool, the lack of explicit 'read-only' statement is a minor gap, but the description does convey the core behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it states the primary action ('Compare two config directories') and the key output ('Reports added, removed, changed, and unchanged config files with key-level diff info') in two sentences. Every word earns its place, and the example in parentheses adds useful context without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (two parameters, no output schema, no annotations), the description is fairly complete. It explains the purpose, the output categories, and the diff granularity. The only missing piece is explicit guidance on error handling or edge cases (e.g., what happens if directories don't exist), but for a comparison tool, the description is sufficient for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (oldConfigDir and newConfigDir) with clear descriptions. The tool description adds the context that these are directories to compare, but does not add additional semantics beyond what the schema provides. The baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: comparing two config directories and reporting added, removed, changed, and unchanged files with key-level diff info. It uses a specific verb ('Compare') and resource ('config directories'), and the example 'before/after a modpack update' clarifies the use case. It distinguishes itself from siblings like read_mod_config (which reads a single config) and list_mod_configs (which lists configs) by focusing on comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: comparing two config directories, e.g., before/after a modpack update. It implies the use case but does not explicitly state when not to use it or mention alternatives. However, the sibling tools are distinct enough that an agent can infer the appropriate choice. The description could be improved by explicitly noting that this is for directory-level comparison, not single-file comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mod_default_configA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose side effects or permissions. It doesn't mention read-only nature or error behavior for unsupported mods, though the phrase 'known' hints at potential failure. Lacks explicit disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using only two sentences to convey purpose and supported mods. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  As a simple getter tool, the description is adequately complete. It doesn't specify the return format, but for a default config retrieval, the structure is likely standard. Without an output schema, slight ambiguity remains, but it's acceptable for this type of tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter modId is fully described with examples in the schema. The description also specifies supported mods, adding context beyond the schema. Parameter semantics are clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves the default config for a specific mod, with a list of supported mods. This distinguishes it from sibling tools like read_mod_config, which presumably handles current configs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a default/baseline config is needed, contrasting with reading the current config. It doesn't explicitly state when not to use it, but the distinction is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.