Server Quality Checklist

Tool Scores

• execute_codeA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It usefully states that variables persist between calls and that all output (stdout, results, errors) is collected and returned as a single string when execution completes, which is valuable behavioral context. It does not mention potential side effects of running arbitrary code or timeout/interruption behavior, but the disclosed persistence and output capture are meaningful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose, two sentences of behavioral context, and short Args/Returns sections. Every line contributes information and the format is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema, the description covers the action, the parameter, persistence, and return format. The only notable gaps are that it does not explicitly state the requirement to connect to a kernel first or mention that interrupt_kernel can stop a long-running execution, though 'connected IPython kernel' strongly implies the prerequisite.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the property name and type with 0% description coverage, so the description must explain the parameter. The Args section says 'code: Python code to execute,' which clarifies that the string is a code snippet rather than a file path or command. It adds little beyond restating the tool's purpose and gives no examples or constraints, so it only partially compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states that the tool executes Python code on the connected IPython kernel, giving a specific verb and resource. It is immediately clear this is the tool for running code, and the sibling tools (connect, status, interrupt) are distinct actions. However, it does not explicitly compare itself to those siblings, so it stops short of a full 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage only when a kernel is already connected ('connected IPython kernel'), but it gives no explicit guidance on when to use execute_code as opposed to kernel_status or interrupt_kernel. There is no stated prerequisite action such as 'call connect_to_kernel first', and no discussion of alternatives. This is sufficient to understand the basic purpose but leaves usage decisions mostly to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connect_to_kernelA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It discloses that the tool connects to an existing kernel, uses a connection file or environment variable fallback, and returns a status message. It does not mention side effects, idempotency, failure behavior, or whether an already-connected state matters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with a clear first sentence followed by Args and Returns sections. Every line adds useful information with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an output schema, the description covers the essential purpose, parameter semantics, fallback behavior, and return value. It omits edge cases like error handling or what happens if the connection file is invalid, but the basic invocation context is fully supplied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates well for the single parameter by explaining that connection_file is the path to a kernel connection JSON file, is optional, and falls back to the IPYTHON_MCP_CONNECTION environment variable when not provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Connect to an existing IPython kernel using its connection file.' This clearly differentiates it from sibling tools like execute_code, kernel_status, and interrupt_kernel, which serve obviously different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to establish a connection to an existing kernel, and the fallback to the IPYTHON_MCP_CONNECTION environment variable provides some context. However, it does not explicitly state when to prefer this over siblings, such as before executing code, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kernel_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the disclosure burden. It clearly labels the operation as a check and states the return type (status message). It does not mention side effects, but 'check' reasonably implies a read-only operation and the output schema covers further return details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences: the first states the action and target, the second states the return value. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, output-schema-backed status check, the description is nearly complete. It could add an explicit note about being non-mutating or about prerequisites, but given the low complexity, the current definition is sufficient for reliable selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so there is nothing for the description to add. The baseline of 4 applies because no parameter semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') with a clear resource ('kernel connection status'), and that purpose is plainly distinct from the sibling tools, which connect, execute code, or interrupt rather than report status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to call this versus the sibling tools or whether a kernel must exist first. The agent can infer usage from the name, but the description itself offers no explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• interrupt_kernelA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the core behavior (sending SIGINT to interrupt execution) and the return value. It does not cover edge cases like no active kernel or loss of execution state, but for a zero-parameter signal tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences: mechanism, use case, and return value. It is front-loaded with the most important information and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with an output schema, the description covers the action, purpose, and response. It does not explicitly state that a kernel must already be connected, but the operation and related sibling tools make this reasonably clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. No parameter documentation is needed, and the description correctly avoids describing nonexistent arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: interrupt the current kernel execution by sending SIGINT. This distinguishes it from siblings like execute_code, connect_to_kernel, and kernel_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit use cases: stopping long-running code or infinite loops. It does not explicitly mention alternatives or when not to use it, but the context is sufficient for an agent to choose this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.