Server Quality Checklist

Tool Scores

• compile_tolkA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must convey behavioral traits. It discloses the return values (Fift code, BoC base64, code hash) and automatic stdlib import resolution, which is useful. However, it omits details about error behavior, potential side effects, or limitations, leaving moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the main verb and purpose. Every sentence provides essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema, no annotations), the description covers the input format, entrypoint requirement, stdlib handling, and return values comprehensively. It lacks explicit mention of error handling or constraints, but the overall context is well-provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are documented in the schema (100% coverage), providing a baseline of 3. The description adds meaningful context by explaining the sources map format and explicitly requiring the entrypoint file to be included in sources, which goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Compiles Tolk smart contract source code using @ton/tolk-js', which clearly identifies the verb and resource. It naturally distinguishes from sibling tools like check_tolk_syntax and get_compiler_version by focusing on the compilation action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for compilation and provides some input expectations (sources map, entrypoint inclusion), but it does not explicitly state when to use this tool versus alternatives or mention any exclusions. Sibling tools are not referenced in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_tolk_syntaxA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden and discloses the return contract: either an OK with a code hash or a detailed error message. It also states what the tool does not do (return full compiled output), adding transparency, though it does not explicitly mention side-effect-freeness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of three compact sentences, each serving a clear purpose: what it does, what it returns, and when to use it. There is no redundant or irrelevant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-documented parameters and no output schema, the description sufficiently covers purpose, behavior, and use case. The sibling tools provide additional context, and nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% description coverage for both parameters (sources object and required entrypointFileName), meeting the baseline. The tool description itself adds no parameter-specific details, which is acceptable given the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Validates') with the resource ('Tolk smart contract source code') and explicitly contrasts with full compilation, distinguishing itself from sibling compile_tolk. It clearly states the tool's primary function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states that it validates without returning full compiled output and emphasizes a faster feedback loop, which clearly implies use during iterative development when full compilation is not needed. It does not explicitly name alternatives, but the context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_compiler_versionA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return value (version string) and that it can verify compiler availability, but it does not detail error behavior (e.g., what happens if the compiler is unavailable) or explicitly state the absence of side effects. This is adequate for a zero-parameter read-only getter, but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, front-loaded with the primary action, and every sentence earns its place. It avoids redundancy and clearly conveys the tool's purpose and usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple version-check tool with zero parameters and no output schema, the description is complete: it states what it returns, when to use it, and its relation to compilation. It provides all necessary context without over-explaining.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so the description does not need to elaborate on parameter meanings. The baseline for zero-parameter tools is 4, and the description adequately explains the tool's purpose without needing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Returns the version string of the Tolk compiler (WASM build from @ton/tolk-js).' This is a specific verb+resource statement that distinguishes it from the sibling tools (compile_tolk, check_tolk_syntax) by focusing solely on version retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to verify the compiler is available and check its version before compiling.' This tells the agent when to invoke it (before compiling), though it does not explicitly name alternatives or exclusion scenarios. Still, the guidance is clear enough for a simple version-check tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.