Server Quality Checklist

Tool Scores

• gerbil_evalC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool evaluates an expression and returns a result, but lacks critical details such as execution environment constraints, error handling, security implications, or performance characteristics. This is insufficient for a tool that executes code, leaving significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the core functionality, and the second provides usage guidance with a concrete example. Every sentence adds value, and it's front-loaded with the main purpose. A slight deduction because the example could be more efficiently integrated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a code evaluation tool with no annotations and no output schema, the description is incomplete. It doesn't cover important aspects like return value format, error responses, execution limits, or safety considerations. For a tool that executes arbitrary code, this lack of context is a significant shortfall.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters (expression and imports). The description adds minimal value beyond the schema: it reiterates the purpose of imports ('to make module bindings available') and provides an example, but doesn't explain parameter interactions or edge cases. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Evaluate a Gerbil Scheme expression using gxi and return the result.' It specifies the verb ('evaluate'), resource ('Gerbil Scheme expression'), and method ('using gxi'), but doesn't explicitly differentiate it from sibling tools like gerbil_check_syntax or gerbil_expand_macro, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance on when to use this tool. It mentions the imports parameter to make module bindings available and gives an example, but doesn't explain when to choose this tool over alternatives like gerbil_check_syntax for syntax validation or gerbil_expand_macro for macro expansion. No explicit when-not-to-use or prerequisite information is included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gerbil_module_exportsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It describes what the tool does (lists exports) but doesn't mention behavioral traits like whether it requires specific permissions, how it handles errors, if there are rate limits, or what the return format looks like. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second provides a concrete example that reinforces usage. Every sentence earns its place with no wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single parameter, no output schema, no annotations), the description is adequate but incomplete. It covers purpose and provides an example, but lacks details on behavioral traits and output format, which are important for a tool with no annotations. This meets minimum viability but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single parameter 'module_path' with examples. The description adds value by providing an additional example (':std/text/json' returns read-json, write-json) that illustrates the output semantics, going beyond the schema's input-focused documentation. With only one parameter, this earns a high score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('exported symbols from a Gerbil module'), and distinguishes it from siblings by focusing on module exports rather than syntax checking, evaluation, or other operations. The example reinforces the specific functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the example (e.g., for module ':std/text/json'), but doesn't explicitly state when to use this tool versus alternatives like 'gerbil_list_std_modules' or 'gerbil_apropos'. It provides context but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gerbil_expand_macroA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return behavior ('Returns the core-expanded form using syntax->datum') and gives an example output format, but lacks details on error handling, performance, or side effects. It adequately describes the core function without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a brief explanation of the return and a helpful example. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides sufficient context for a read-only tool with two parameters. It explains what the tool does and the return format, though it could benefit from more behavioral details like error cases. It is mostly complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description does not add meaning beyond the schema, as it does not explain parameter interactions or provide additional context for 'imports' beyond what the schema states. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Show the macro expansion') and resource ('of a Gerbil Scheme expression'), distinguishing it from siblings like gerbil_eval (evaluation) or gerbil_check_syntax (syntax checking). It provides a concrete example to illustrate the transformation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for macro expansion scenarios with the example, but does not explicitly state when to use this tool versus alternatives like gerbil_eval for execution or gerbil_check_syntax for validation. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gerbil_aproposA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's behavior (substring matching across all namespaces using ##apropos) and gives an example, but lacks details on permissions, rate limits, error handling, or output format. It adequately describes the core operation but misses advanced behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey purpose and usage. The example is relevant and adds clarity without unnecessary details, making every sentence earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is complete enough for basic use. However, it lacks information on output format, error cases, or integration with sibling tools, which could be helpful for an AI agent in a broader context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single parameter 'pattern' as a string for substring matching. The description adds minimal value by mentioning 'pattern string' and giving an example, but does not provide additional syntax or format details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('search for', 'find') and resources ('Gerbil/Gambit symbols'), and distinguishes it from siblings by specifying it searches across all namespaces using Gambit ##apropos. The example further clarifies the functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (searching for symbols matching a pattern), but does not explicitly state when not to use it or name alternatives among the sibling tools (e.g., gerbil_module_exports for specific modules). The example helps illustrate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gerbil_check_syntaxA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's behavior ('check if syntactically valid,' 'uses the Gerbil expander to verify expansion') but lacks details on error handling, performance characteristics, or output format. It adequately covers the core operation but misses additional context like rate limits or response structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded and concise with two sentences that directly state the tool's purpose and method. Every sentence adds value: the first defines the action and scope, and the second explains the underlying mechanism. There is no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (validation without execution), no annotations, and no output schema, the description is minimally complete. It covers what the tool does and how, but lacks details on return values, error cases, or integration with sibling tools. It's adequate for basic understanding but could be more comprehensive for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (code and imports). The description adds no specific parameter semantics beyond what the schema provides, such as examples or constraints. Baseline 3 is appropriate as the schema handles parameter documentation effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check if Gerbil Scheme code is syntactically valid') and resource ('Gerbil Scheme code'), distinguishing it from siblings like gerbil_eval (which evaluates) and gerbil_expand_macro (which expands macros). It explicitly notes the tool 'uses the Gerbil expander to verify the code can be expanded,' providing technical specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('check if Gerbil Scheme code is syntactically valid without evaluating it'), implying it's for validation rather than execution. However, it doesn't explicitly state when not to use it or name alternatives among siblings, though the purpose differentiation suggests gerbil_eval as an alternative for evaluation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gerbil_list_std_modulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the action ('scanning the installation') and output format ('Returns module paths like :std/text/json, :std/iter, etc.'), which adds useful context beyond the input schema. However, it lacks details on potential side effects, error handling, or performance implications (e.g., scanning might be resource-intensive), leaving some behavioral aspects unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by optional filtering and return details in subsequent sentences. Each sentence earns its place by adding necessary information without redundancy, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is mostly complete. It covers the purpose, usage with filtering, and output format. However, without an output schema, it could benefit from more detail on the return structure (e.g., list format, error cases), slightly reducing completeness for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'prefix' fully documented in the schema. The description adds minimal value by reiterating the filtering purpose and providing examples ('e.g. "std/text", "std/net", "gerbil"'), but does not explain semantics beyond what the schema already states, such as how the prefix matching works (e.g., case sensitivity). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List available Gerbil standard library modules by scanning the installation') and resource ('Gerbil standard library modules'), distinguishing it from siblings like gerbil_apropos (search) or gerbil_check_syntax (validation). It explicitly mentions what it returns ('module paths like :std/text/json, :std/iter, etc.'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by mentioning the optional filtering capability ('Optionally filter by prefix'), which helps differentiate it from tools like gerbil_module_exports (which might list exports of a specific module). However, it does not explicitly state when not to use this tool or name specific alternatives among siblings, such as using gerbil_apropos for searching modules by content rather than listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.