Server Quality Checklist

Tool Scores

• port_conflictA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint false, and idempotentHint true. The description adds that it returns the blocking server and free ports, which is useful context beyond the annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first declares read-only behavior and functionality, the second provides usage guidance. No extraneous words; each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description fully covers purpose, trigger condition, and return value. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter. The description repeats the schema's description ('The contended TCP port, e.g. 3000.') without adding new meaning. Baseline 3 is appropriate since the schema already provides sufficient detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool diagnoses an EADDRINUSE situation by returning the blocking dev server and 5 free alternative ports. Its verb 'diagnose' and resource 'port conflict' are specific, and it distinguishes itself from sibling tools (e.g., kill_server, find_zombies) that serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using this tool when a 'listen EADDRINUSE' error occurs, specifying the desired outcome (culprit and free ports). While it doesn't list alternatives or when not to use, the context is clear enough for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• port_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds value by explaining the return format (same fields as `list_dev_servers`) and handling edge cases (nothing listening or not a recognised dev process). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with the key verb 'Read-only' front-loaded. Every sentence adds necessary information: purpose, return format details, and usage scenario. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of annotations (read-only, idempotent, non-destructive) and an output schema (implied), the description is complete. It covers behavior for all inputs (port in range) and edge cases (free port, unrecognised process). The tool's scope is well-defined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a well-described parameter (port). The description adds 'TCP port number, 1–65535, e.g. 3000,' which mirrors the schema. Since the schema already does the heavy lifting, the description does not add significant new semantics beyond clarifying the example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inspects a single TCP port and reports the dev server holding it. It distinguishes from sibling tools like `list_dev_servers` (which lists all servers) and implies a specific function ('what's on :3000?').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Useful for answering "what's on :3000?" before starting or killing something,' which provides clear context for when to use. However, it does not explicitly mention when not to use or list alternatives, though sibling tool names provide some guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_dev_serversA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations: it is read-only, uses lsof/netstat based on OS, notes Windows limitations on cwd/framework fields, and details the full output schema. Annotations confirm read-only, and no contradictions exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no fluff. It starts with the purpose, lists output fields, and ends with platform details and return structure. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter tool, the description is exhaustive. It covers platform differences, limitations, output format, and all fields. The output schema exists and is described. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters; the description correctly states 'takes no arguments'. With 0 parameters, baseline is 4, and the description adds no extra param information but confirms the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists local development servers listening on TCP ports, with a specific verb 'lists' and resource 'dev servers'. It includes a long list of frameworks and distinguishes itself from siblings like find_zombies or kill_server by focusing on active listening servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when one wants to enumerate dev servers, but does not explicitly state when to use it over alternatives like port_info or port_conflict. No exclusion criteria or alternative tool names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_zombiesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description complements the annotations (readOnlyHint, destructiveHint, idempotentHint) by explicitly stating 'Read-only' and 'This tool never kills anything'. It details the exact heuristic logic and default filtering behavior, adding substantial context beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise, with three sentences packed with essential information. It front-loads core purpose, then covers behavior, parameter, safety, and output format without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single boolean parameter, clear heuristic), the description fully covers matching criteria, default exclusion behavior, the parameter's effect, relationship to sibling tools, and the return format. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the single parameter (include_excluded) is 100%, and the schema already includes a thorough description. The tool description restates the parameter's effect but does not add significant new semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool flags dev servers that appear abandoned, and precisely defines the three matching criteria (uptime > 6h, CPU < 1%, memory > 100MB). It distinguishes itself from sibling tools by explicitly stating it never kills anything and that results should be passed to kill_server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool is read-only and should be used to identify candidates, while kill_server is for termination. It also describes the default exclusion of noise processes and the include_excluded parameter, giving clear guidance on when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kill_serverA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Rich behavioral details beyond annotations: dry-run behavior, escalation from SIGTERM to SIGKILL with timeout, force flag effect, safety guardrails (refuses low PIDs and non-dev-server processes). Annotations provide destructiveHint, but description adds deep context without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense, front-loaded sentences with zero redundancy. Every clause adds essential information (alternatives, safety, escalation, constraints).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of output schema (not shown) and annotations, the description covers behavior, constraints, safety mechanisms, and parameter relationships completely. No gaps for an AI agent to misunderstand.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes all 4 parameters fully (100% coverage). The description adds value by contextualizing the mutex between pid/port, explaining the safe default behavior, and clarifying the effect of confirm vs force beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses active verbs ('terminates', 'reports', 'sends') and clearly specifies the resource ('local dev server') and scope ('by pid or port'). It distinguishes from siblings like list_dev_servers and find_zombies by focusing solely on termination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Provide exactly one of pid or port' and explains the dry-run default and confirm requirement. While it doesn't contrast with siblings (not needed as they serve different purposes), it gives clear context for when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.