Server Quality Checklist

Tool Scores

• remove_hostC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits but fails to do so. It does not mention whether removal is permanent, if it triggers any side effects, or if special permissions are required. 'Remove' is vague; the tool could delete the domain entirely or just deactivate monitoring.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose without any extraneous words. It is maximally concise while still being understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter), the description provides basic completeness. However, the presence of an output schema suggests the tool returns data, yet the description does not hint at what is returned. Additionally, the description does not address the context of sibling tools that may perform related operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no description for the 'domain' parameter, and the tool description does not clarify its format (e.g., must it be a fully qualified domain name? with or without protocol?). With 0% schema description coverage, the tool should compensate but does not, leaving the agent without essential formatting information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('remove') and the resource ('domain from the monitored host list'). It differentiates from the sibling tool 'add_monitored_host' which performs the opposite action. However, the phrasing 'monitored host list' could be more precise, e.g., 'list of monitored hosts'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or any conditions for use. It only describes what the tool does, leaving the agent to infer usage context from the name and sibling tools. There are no exclusions or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_monitored_hostB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions an immediate check but does not disclose permissions, overwrite behavior, rate limits, or error conditions. Partial coverage of behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no filler. Every word contributes to understanding the tool's core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Low complexity with 2 parameters and an output schema present. Description covers the main action but lacks parameter details and does not explain what 'immediately check' means or what the response contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description does not explain the domain format, port default, or any constraints beyond the schema's type and required fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('add') and resource ('domain to monitored host list'), plus an immediate side effect ('check its certificate'). It distinguishes from siblings like check_certificate (no add) and remove_host (remove).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like check_certificate or scan_all. No explicit when-not or condition specifications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_expiry_reportA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only hints at read-only behavior ('stored certificate data'). Lacks disclosure of side effects, rate limits, or authentication requirements. Return format is not described despite having an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with clear subject and qualifier. No extraneous information; efficiently conveys core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Low complexity (zero parameters, output schema exists). Description covers purpose adequately, but lacks guidance on output structure or use cases. However, output schema mitigates need for return format details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter documentation is needed. Baseline of 4 applies as description adds no value but also doesn't need to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool returns stored certificate data categorized into critical, warning, and ok. Verb 'Return' and resource 'stored certificate data' are specific, distinguishing it from sibling tools like scan_all or check_certificate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like check_certificate or scan_all. The description implies a read-only report but doesn't specify context or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_certificateA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool does not update stored state, but lacks details on error handling, rate limits, or authentication. It is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and key behavioral trait. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input (2 params) and the presence of an output schema, the description adequately covers the main purpose and side-effect behavior. It could mention error conditions or use cases, but overall it is complete enough for a straightforward fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it adds no additional meaning to 'domain' or 'port' beyond what is in the schema. The parameter types and defaults are clear from the schema, but no semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'fetch' and the resource 'live SSL/TLS certificate for a domain', and adds a key behavioral distinction ('without updating stored state') that differentiates it from siblings that may store data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a read-only check, but does not explicitly specify when to use this tool versus alternatives like get_expiry_report or scan_all. No exclusions or contextual guidance are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_allA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavior. It discloses parallelism and result updating, but does not mention failure handling, authentication needs, or side effects. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 11 words, front-loaded with key action. No wasted words, perfectly sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters and an output schema present, the description covers purpose, parallelism, and update behavior. For a simple tool, this is sufficiently complete. Could mention that it iterates over all hosts, but implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100% trivially. Description adds no param info, which is acceptable since none needed. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool checks certificates for all monitored hosts in parallel and updates stored results. The verb 'Check' and resource 'certificates for all monitored hosts' are specific. It distinguishes from sibling 'check_certificate' which likely targets a single host.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clear context: use when you want to check certificates for all monitored hosts simultaneously. No explicit when-not or alternatives, but contrast with sibling names implies single-host alternative. Adequate but could be more explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.