Server Quality Checklist

Tool Scores

• open_portC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavior. It only states 'open a serial port connection,' but does not mention that the tool establishes a persistent state requiring a corresponding 'close' call, nor does it reference permissions or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, making it concise. However, it lacks substance and does not front-load critical information beyond the obvious, potentially failing to earn its place given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the presence of an output schema (not shown), the description omits crucial context such as the nature of the opened connection, lifecycle requirements, and its relationship to sibling tools. This leaves gaps for an AI agent to understand proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all 6 parameters with descriptions, achieving 100% coverage. The description adds no additional parameter context beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('open') and the resource ('serial port connection'), providing a basic purpose. However, it does not differentiate this tool from siblings like 'list_ports_tool' or 'close_port', which could cause confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, such as the port being available, or when it is appropriate to open a connection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_portC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It does not mention blocking behavior, timeout handling, what happens if no data is available, or whether the read is destructive. Critical details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) and front-loaded, but it sacrifices important contextual information. It earns its place minimally but is not overly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters and an output schema, the description is incomplete. It lacks behavioral details like blocking, partial reads, and error conditions. Even with output schema, more context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with adequate parameter descriptions. The tool description adds marginal value beyond the schema, only reiterating the read action. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('read data') and the resource ('an open serial port'), and it distinguishes from sibling tools like write_port, open_port, etc. However, it could be more specific about reading bytes vs strings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs alternatives, such as prerequisites (port must be opened first) or contraindications. It mentions the port must be open but does not elaborate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_portB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It only states the basic action, omitting side effects like resource cleanup, error handling, or state changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, efficient and free of redundancy. However, it could include more detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with full parameter schema and an output schema, the description is minimally adequate but lacks behavioral context and usage guidelines, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema; both describe the port parameter simply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('close') and the resource ('open serial port connection'), effectively distinguishing it from sibling tools like open_port, read_port, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool, such as prerequisites (port must be open) or when not to use it. It lacks differentiation in usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_portB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must convey behavioral traits. It states the action but omits important details such as error handling (e.g., if the port is not open), blocking behavior, write confirmation, or side effects. The description is too sparse to fully inform the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that is front-loaded with the action. However, it is overly concise and misses opportunities to include useful context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to explain return values. However, it lacks usage guidelines and behavioral transparency, making it just barely adequate for a simple write operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and both parameters are described adequately in the schema. The description adds no extra meaning beyond what the schema provides; it merely reiterates the concepts, earning a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Write data to an open serial port' uses a specific verb ('Write') and clearly identifies the resource (data to an open serial port). It distinguishes itself from sibling tools like open_port, close_port, and read_port, which cover other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., the port must be opened first) or when not to use it, leaving the agent to infer usage context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_port_configB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose all behavioral traits. It only states 'Get' (implying read-only) but does not mention prerequisites (e.g., port must be open), error conditions, or that configuration is static after opening.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resource, no unnecessary words. Efficiently communicates the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool complexity is low (one parameter, output schema exists). However, the description omits the implicit dependency on open_port (the port must already be open), which a new agent might not infer from sibling names alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (port parameter described as 'The name of the port to query.'). The description adds no additional meaning beyond what the schema already provides, scoring baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('configuration of an open serial port'), clearly distinguishing it from siblings like read_port (reads data) or list_ports_tool (lists available ports).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For example, it doesn't mention that the port must be open first (using open_port) or that this tool should be used to check settings before reading/writing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ports_toolA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description only states it returns a list. It does not disclose whether it is safe/read-only, any side effects, or prerequisites like permissions. The description carries the full burden but provides minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, the first of which clearly states the purpose. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema (presumably detailed), the description is adequate but could mention that this is a read-only operation or list common use cases. With 5 sibling tools, some relational context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters with 100% coverage, so the description need not add parameter details. It adds that the tool returns a list with details, which is sufficient for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'list all available serial ports', which is specific verb+resource. It distinguishes from sibling tools like open_port or write_port, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For example, it could mention that this should be used before opening a port to get available options, but it does not.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.