Server Quality Checklist

Tool Scores

• get_game_logsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that a log file must exist and what the tool shows (errors, warnings, print output), but it does not mention behavioral traits such as whether the operation is read-only, what the return format is, or potential side effects (e.g., if the tool could block or incur costs). This is insufficient for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no waste. The core purpose is front-loaded, followed by prerequisites and capabilities. The first sentence contains a parenthetical requirement that is helpful but slightly verbose. Overall, it is efficient and structured for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tail-and-filter tool with two optional parameters and no output schema, the description covers the basics: what it does, prerequisites, and filter option. However, it omits any description of the return value (the log content format) and does not clarify how this tool differs from the sibling 'game_logs', which may be critical for correct selection. Given the tool's relative simplicity, a 3 reflects the gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('lines' as integer count, 'filter' as regex). The description adds a brief mention of optional regex filtering (e.g., 'error|warning|Lua') which mirrors the schema, but provides no additional semantic meaning beyond what the schema offers. Thus a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Tail the newest Desynced log file'. It identifies the resource (Desynced log file) and purpose (show Lua errors, warnings, print output). However, it does not explicitly distinguish itself from the sibling tool 'game_logs' or other log-related tools, so it lacks the explicit differentiation that would warrant a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful prerequisites (game must run with -log; -moddev recommended) but gives no guidance on when to use this tool versus the sibling 'game_logs' or others. It does not state when not to use it or mention any alternative conditions. The agent is left to infer appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_game_fileB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose that reading is a non-destructive operation, what happens if the file is missing, whether path is case-sensitive, or what the return format is. The mention of line range is the only behavioral hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise—two sentences—with the core purpose front-loaded and an example serving as a concrete anchor. Every word earns its place; no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and three parameters, the description is incomplete. It fails to describe the return format (full file content? line array?), the semantics of line ranges (inclusive boundaries), error conditions (missing file, out-of-range lines), and any limits (file size). An agent would need to infer or probe these details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain parameter meanings. It mentions 'relative path' and 'optional line range' but does not clarify that from_line defaults to 1, what to_line does (inclusive/exclusive), or how they interact. The example path helps relative_path but not the line parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Read') and resource ('file from the workspace'), specifies allowed packages ('main' or any mod), and provides a concrete example path. This clearly differentiates it from siblings like search_game_source (searching vs. reading a known file).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by describing read scope and optional line range, but it does not explicitly advise when to use this tool versus alternatives such as search_game_source or when not to use it. It gives context but no explicit routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_lua_apiA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It does disclose the return shape (matching lines plus trailing context of signatures and descriptions), which is useful. However, it does not explicitly state that this is a read-only, non-destructive lookup, nor does it disclose search behavior such as case sensitivity, matching semantics, or the effect of the context_lines and max_results parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with the core purpose front-loaded and the scope (modules/metatables) enumerated efficiently. The sentence about return format is necessary context. It is focused with no filler, though it could be slightly more compact given the long module list.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool, the description covers what is searched (modules and metatables) and what is returned (signatures + descriptions). The three parameters are documented through names and defaults in the schema. The main gap is the absence of contrast with search_game_source and lack of search-behavior detail, which prevents a 5 but the tool remains usable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%; only 'query' carries an example in the schema, while max_results and context_lines have no prose. The description adds nothing about parameters. This gap is partially mitigated by the fact that max_results and context_lines are self-explanatory from their names and defaults, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (search) and resource (official Desynced Lua API reference), enumerates the exact modules and metatables covered, and describes the return format (matching lines with signatures + descriptions). It is clear and specific, though it does not explicitly differentiate itself from the closely named sibling search_game_source, which searches game source rather than API docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear this is for looking up API reference documentation, which implies usage context. However, it gives no explicit when-to-use or when-not-to-use guidance, and it does not mention the alternative sibling search_game_source or state the condition that would select one over the other. The distinction between API docs and game source is left for the agent to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• game_logsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the buffer accumulates while the server is connected and survives crashes up to the moment of death, which are useful behavioral traits. It does not explicitly state the operation is read-only or mention side effects, but the nature of the tool implies no mutation. Given the lack of annotations, this is a solid disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, tightly written, with the core purpose front-loaded. The secondary sentence adds meaningful behavioral context about buffer accumulation and crash survival. Every part earns its place with no unnecessary wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a log-retrieval tool with no output schema and no annotations, the description covers the essential aspects: what it returns, how it behaves, and the connection requirement. It does not address potential differences from the similarly named sibling get_game_logs, which is a minor gap, but overall the agent has enough to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: both 'lines' and 'filter' are well-documented with types, defaults, ranges, and examples. The tool description adds no additional parameter context beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb and resource: 'Return the recent output captured live from the running game' with specific content types (print, errors, warnings, BOOT lines). However, it does not differentiate from the sibling tool 'get_game_logs', which appears to serve a similar purpose, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: the agent can infer this is the tool to retrieve live game logs. But there is no explicit guidance on when to choose this over alternatives like get_game_logs, nor any mention of exclusions or prerequisites (e.g., must have debugger link). It provides context about buffer behavior but stops short of routing the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• game_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It does add value by explaining the connection mechanism (Lua debugger) and the graceful failure behavior when the game isn't running. However, it does not explicitly state whether the operation is read-only or if any side effects occur, and it doesn't describe error behavior beyond the graceful failure. This provides some context but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero fluff. The first sentence front-loads the main purpose and the exact fields reported, while the second sentence adds the failure condition. This is appropriately sized and well-structured, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema. The description covers what it does, what it returns, and the graceful failure mode. However, it does not mention whether the tool has side effects or if it's safe for read-only use, which might be expected given the absence of annotations. For its simplicity, it is fairly complete, but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty, so there are no parameters to document. The description doesn't need to add parameter semantics, and the baseline for 0 parameters is 4. Since there is nothing to clarify, this score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (connect to Lua debugger and report) and a resource (live game status) with concrete output fields (tick, map loaded, player faction, entity counts). This is clearly distinct from sibling tools like game_logs or game_eval, which focus on logs or evaluation rather than current state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It only mentions the failure condition when the game isn't running with -moddev, which is a precondition, not a usage recommendation. There's no mention of selecting this over other tools for specific scenarios, leaving the agent without direction on tool choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_game_sourceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose side effects and limitations. It states it returns matches with context, implying a read-only search, but does not explicitly say it is non-destructive or mention any caveats (e.g., regex performance, indexing behavior, or that it searches the full workspace). The description is adequate for safety but does not go beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core function (regex search over source) and then the primary use case. Every word adds value, with no fluff or redundant phrases. It is efficiently structured for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with four parameters and no output schema, the description covers the essential context: what is searched, what is returned, and when to use it. It does not explain parameter details or explicitly differentiate from search_lua_api, but the core usage is clear. The gaps are minor given the tool's simplicity and the schema's partial coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (query and path_filter have descriptions; max_results and context_lines have defaults but no descriptions). The description does not explain these parameters, their defaults, or how they interact (e.g., what 'context' refers to). It only hints at context via 'with context' but does not clarify the context_lines parameter. The description fails to compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact verb ('Regex search'), the resource ('Lua/JSON source of the base game and all mods'), and the output ('file:line matches with context'). It is specific enough to distinguish from sibling tools like search_lua_api (API search) and read_game_file (single file read). No ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly gives a use case: 'find how the base game defines/uses something before overriding it in a mod.' This tells the agent when to use it, but it does not name alternatives directly or explain when not to use it (e.g., for API docs vs source). The guidance is clear but lacks explicit exclusions or sibling routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• game_evalA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses timeout behavior, the multiplayer desync side effect, and the debugger context. It doesn't specify return serialization or error behavior, but the critical operational caveats are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each purposeful: the core action, the usage pattern, and the behavioral caveats. Front-loads the main purpose and avoids redundancy; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an eval tool with no output schema, it explains how to obtain values and flags side effects. It doesn't detail error handling or result formatting, but given the tool's open-ended nature, the essential usage and risks are covered. Slightly more on timeout_ms would push it to 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description effectively explains that chunk should contain Lua code and demonstrates the 'return' pattern, which maps to the schema's example. It touches on timeout indirectly via the 'can time out' note, but timeout_ms itself is not explicitly described. Since schema coverage is only 50%, this partial compensation is acceptable but not complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: executing a Lua chunk in the game's VM via the debugger and returning the result. It gives a concrete pattern ('return <expr>') and distinguishes from sibling tools that search or read files, making the intent unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It instructs to use 'return <expr>' for output, warns that the VM only responds during Lua execution (timeout in menu/paused), and flags multiplayer desync—implicitly saying when not to use it. However, it does not explicitly contrast with alternatives like search_lua_api, so a small gap remains.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• game_reloadA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It states the effect ('reloads all mod Lua from disk') and the context ('in the running game'), which conveys that the tool mutates the running game's state. However, it does not mention potential side effects (e.g., resetting mod state, losing unsaved changes) or prerequisites beyond a running game, which would be helpful for a state-changing tool. The description is functional but not fully transparent about consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no fluff. It front-loads the core action ('Trigger Debug.Reload()'), then explains the effect and the recommended usage. Every sentence contributes necessary information, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and no output schema, the description covers the essential context: what it does, when to use it, and the operational context (running game). It implicitly distinguishes itself from sibling tools by its purpose. Minor gaps like failure behavior or idempotency are not critical for this action, so the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100% (vacuously, as there is nothing to cover). The description does not need to add parameter meaning, and none is missing. The baseline for zero parameters is 4, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Trigger Debug.Reload()' and the resource: 'reloads all mod Lua from disk'. It also provides a precise analogy to pressing F7 in-game, which makes the purpose unambiguous. Among sibling tools (game_status, game_logs, list_mods, etc.), this is the only one that performs a reload, so it is easily distinguished.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use context: 'Use after deploying mod changes to the game's mods directory.' This is a clear trigger condition. However, it does not mention when not to use it or name alternative tools, though the unique action makes that less critical. The guidance is sufficient for an agent to decide when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It states exactly what data is extracted (id/name/version from def.json) and from where (game mods and workspace mods). It does not explicitly mention that it is a read-only operation, but that is strongly implied and no side effects are suggested. It adequately discloses scope and output content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the action ('List mods in the game's mods directory') and then adds necessary detail (def.json fields, workspace comparison, purpose). No unnecessary words; every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool with no output schema, the description is remarkably complete. It explains what is listed, the sources (game mods directory and workspace), the extracted fields (id/name/version), and the practical use case (spotting version drift). An agent can call this tool correctly with full confidence in its behavior and expected result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (empty properties). Per the guidelines, the baseline for 0 parameters is 4. The description adds context about what data is listed, which complements the empty schema adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List', the resource 'mods' (in the game's mods directory and workspace), and specifics like 'id/name/version from each def.json'. It explicitly distinguishes its scope from sibling tools (game status, logs, LUA API, etc.) by focusing on mods and version drift.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose statement 'to spot version drift between both' implies a clear use case. It does not explicitly say 'use this when you need to compare installed mods vs workspace mods', but the context and sibling tooling make that evident. It lacks explicit when-not-to-use guidance, but the single purpose is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.