Server Quality Checklist

Tool Scores

• install_packageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Install') but doesn't describe what happens during installation (e.g., dependencies resolved, package added to environment), potential side effects (e.g., system changes, conflicts), or error conditions. For a mutation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized for a simple tool and front-loaded with the core action. Every part of the sentence earns its place by specifying what, how, and the tool used.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with no annotations and no output schema), the description is incomplete. It doesn't explain what the tool returns (e.g., success/failure, installation details), behavioral traits, or usage context. For a package installation tool that modifies the environment, more information is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'package' documented in the schema as 'Package name to install (e.g., 'pandas')'. The description adds no additional parameter information beyond what the schema provides. According to the rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Install') and the resource ('a Python package'), and specifies the tool used ('using uv'). It distinguishes from sibling tools like 'execute_python' and 'list_variables' by focusing on package installation rather than code execution or variable listing. However, it doesn't explicitly differentiate from hypothetical similar tools (e.g., 'install_package_with_pip'), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., Python environment setup), when not to use it (e.g., for system packages), or compare to other installation methods. The agent must infer usage from the purpose alone, which is insufficient for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_variablesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this is a read-only operation, what format the output takes (e.g., list of names vs. values), or any constraints like session dependencies. More context is needed for a mutation-aware agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and scope, making it easy to parse quickly. Every part of the sentence contributes directly to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on output format or behavioral context, which could be important for an agent. It meets basic needs but leaves gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description appropriately doesn't add param info, aligning with the schema. A baseline of 4 is given as it handles the zero-param case correctly without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('list') and resource ('variables'), specifying scope ('in the current session'). It's specific enough to understand the action, though it doesn't explicitly differentiate from sibling tools like execute_python or install_package, which perform different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context on prerequisites, such as whether a session must be active, or comparisons to other tools for variable management. It's a basic statement without usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_pythonA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: execution of Python code, output return, and variable persistence. However, it lacks details on safety (e.g., sandboxing, timeout), error handling, or resource limits, which are important for a code execution tool. The description does not contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of two concise sentences that directly convey the core functionality and a key behavioral trait. Every sentence earns its place by providing essential information without redundancy or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a code execution tool, no annotations, and no output schema, the description is incomplete. It covers the basic purpose and variable persistence but lacks details on output format, error responses, or execution environment. This leaves gaps for an AI agent to understand the full behavior and use cases effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('code' and 'reset') with descriptions. The description adds no additional parameter semantics beyond what the schema provides, such as examples or constraints. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Execute Python code') and the resource ('Python code'), with the additional detail that it 'return[s] the output'. It distinguishes from sibling tools by focusing on execution rather than package installation (install_package) or variable listing (list_variables).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by stating that 'Variables persist between executions', which implicitly suggests when to use this tool (for sequential code execution with shared state) versus alternatives like resetting the session. However, it does not explicitly name alternatives or state when-not to use it, such as for package management tasks handled by install_package.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.