Server Quality Checklist

Tool Scores

• connectB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Connect') but does not disclose behavioral traits such as whether this is a one-time or persistent connection, error handling, authentication needs, or what happens upon successful connection. The example adds minimal context but leaves key operational details unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, with the first sentence stating the purpose clearly and the second providing a practical example. Every sentence earns its place by reinforcing understanding without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a connection tool with no annotations and no output schema, the description is incomplete. It lacks information about what the tool returns upon success or failure, connection persistence, or error conditions, which are critical for an agent to use it effectively in a workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters ('host' and 'port') well-documented in the schema. The description adds an example that illustrates parameter usage but does not provide additional semantic meaning beyond what the schema already specifies, such as format constraints or default values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Connect to') and target resource ('an nREPL server'), with an example that reinforces the purpose. It distinguishes itself from sibling tools like 'eval_form' and 'get_ns_vars' by focusing on establishing a connection rather than evaluating code or retrieving namespace variables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to establish a connection to an nREPL server, but it does not provide explicit guidance on when to use this tool versus alternatives or any prerequisites. The example suggests typical usage scenarios but lacks context about timing or dependencies.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ns_varsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the return format (a map with keys as var names and values containing metadata and current values), which is helpful, but doesn't cover aspects like permissions needed, rate limits, error conditions, or whether it's a read-only operation (though 'Get' implies reading).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The example and return format details are useful but could be slightly more streamlined. Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (1 parameter, no output schema, no annotations), the description is somewhat complete but has gaps. It explains the return structure well, but lacks information on usage context, error handling, and behavioral traits like safety or performance considerations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'ns' as 'Namespace to inspect'. The description adds an example usage with {:ns "main"} but doesn't provide additional semantic context beyond what the schema states, such as namespace format or scope limitations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'all public vars (functions, values) in a namespace with their metadata and current values', which is specific and comprehensive. It distinguishes from sibling tools (connect, eval_form) by focusing on namespace inspection rather than connection or evaluation operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It includes an example usage but doesn't mention prerequisites, constraints, or compare it to sibling tools like eval_form, which might also interact with namespace variables.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eval_formA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that namespace changes 'persist for subsequent evaluations,' which is useful behavioral context. However, it lacks details on error handling, side effects, or performance implications, leaving gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with a clear purpose statement followed by specific examples. Each sentence earns its place by illustrating use cases without unnecessary elaboration, making it efficient and easy to understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (evaluating code with potential side effects), no annotations, and no output schema, the description is moderately complete. It covers purpose and usage examples but lacks details on return values, error formats, or security considerations, which are important for a code evaluation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('code' and 'ns'). The description adds value by providing examples that illustrate parameter usage, such as showing code snippets and namespace context, but does not add new semantic details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Evaluate Clojure code in a specific namespace or the current one.' It specifies the verb ('evaluate'), resource ('Clojure code'), and scope ('specific namespace or the current one'), distinguishing it from sibling tools like 'connect' and 'get_ns_vars' which likely serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage through examples, such as defining functions, reloading code, and evaluating in a namespace. However, it does not explicitly state when to use this tool versus alternatives like 'get_ns_vars' or any exclusions, leaving some ambiguity in tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.